2024-09-16T07:11:17.885 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T07:11:17.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T07:11:17.999 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852 branch: wip-athakkar-testing-2024-09-15-1936 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7906852' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-athakkar-testing-2024-09-15-1936 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 4ef5f0587d1cad82c4295c20951c303dc342ac74 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4ef5f0587d1cad82c4295c20951c303dc342ac74 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-athakkar-testing-2024-09-15-1936 sha1: 4ef5f0587d1cad82c4295c20951c303dc342ac74 owner: scheduled_khiremat@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6756 sha1: 4ef5f0587d1cad82c4295c20951c303dc342ac74 sleep_before_teardown: 0 subset: 3/4 suite: fs:volumes suite_branch: wip-athakkar-testing-2024-09-15-1936 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_4ef5f0587d1cad82c4295c20951c303dc342ac74/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 4ef5f0587d1cad82c4295c20951c303dc342ac74 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMNv1jjasIxNL8MxftPAQlXdIxvvAPHQ+UGxPSWhs1QW2lgcDzLSFS58pD4FvJ4tgcw8vUkbiIX9lg2C+dCgMiU= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0L/CaUqEMFkrNLzDAkVgiUhck6icOxm1c+d+l9PWmYh/kV45yz03a/WyIFfeirS4sEtO9ddK+zreai5IDFXPU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_07:02:54 tube: smithi user: khiremat verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T07:11:17.999 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_4ef5f0587d1cad82c4295c20951c303dc342ac74/qa; will attempt to use it 2024-09-16T07:11:18.000 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_4ef5f0587d1cad82c4295c20951c303dc342ac74/qa/tasks 2024-09-16T07:11:18.000 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T07:11:18.007 INFO:teuthology.task.internal:Checking packages... 2024-09-16T07:11:18.031 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '4ef5f0587d1cad82c4295c20951c303dc342ac74' 2024-09-16T07:11:18.031 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T07:11:18.031 INFO:teuthology.packaging:ref: None 2024-09-16T07:11:18.032 INFO:teuthology.packaging:tag: None 2024-09-16T07:11:18.032 INFO:teuthology.packaging:branch: wip-athakkar-testing-2024-09-15-1936 2024-09-16T07:11:18.032 INFO:teuthology.packaging:sha1: 4ef5f0587d1cad82c4295c20951c303dc342ac74 2024-09-16T07:11:18.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-athakkar-testing-2024-09-15-1936 2024-09-16T07:11:18.175 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4938.g4ef5f058 2024-09-16T07:11:18.177 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T07:11:18.183 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T07:11:18.183 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T07:11:18.226 INFO:teuthology.task.internal:Saving configuration 2024-09-16T07:11:18.247 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T07:11:18.260 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T07:11:18.283 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 07:06:29.312296', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMNv1jjasIxNL8MxftPAQlXdIxvvAPHQ+UGxPSWhs1QW2lgcDzLSFS58pD4FvJ4tgcw8vUkbiIX9lg2C+dCgMiU='} 2024-09-16T07:11:18.298 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 07:06:29.314286', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0L/CaUqEMFkrNLzDAkVgiUhck6icOxm1c+d+l9PWmYh/kV45yz03a/WyIFfeirS4sEtO9ddK+zreai5IDFXPU='} 2024-09-16T07:11:18.298 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T07:11:18.306 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-16T07:11:18.306 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-16T07:11:18.306 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T07:11:18.395 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f46f36e3ac0>, signals=[15]) 2024-09-16T07:11:18.395 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T07:11:18.402 INFO:teuthology.task.internal:Opening connections... 2024-09-16T07:11:18.402 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:11:18.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:11:18.483 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:11:18.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:11:18.562 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T07:11:18.569 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-09-16T07:11:18.587 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-09-16T07:11:18.587 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-09-16T07:11:18.645 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T07:11:18.646 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-09-16T07:11:18.646 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T07:11:18.646 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T07:11:18.646 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T07:11:18.647 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-09-16T07:11:18.671 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-09-16T07:11:18.690 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-09-16T07:11:18.691 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-09-16T07:11:18.748 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-09-16T07:11:18.748 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-09-16T07:11:18.748 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-09-16T07:11:18.748 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-09-16T07:11:18.748 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-09-16T07:11:18.748 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-09-16T07:11:18.748 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T07:11:18.748 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-09-16T07:11:18.749 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-09-16T07:11:18.749 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T07:11:18.749 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-09-16T07:11:18.749 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T07:11:18.749 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T07:11:18.749 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T07:11:18.749 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-09-16T07:11:18.771 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T07:11:18.780 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T07:11:18.808 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T07:11:18.809 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-09-16T07:11:18.811 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-09-16T07:11:18.828 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T07:11:18.842 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T07:11:18.843 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-09-16T07:11:18.873 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-09-16T07:11:18.933 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T07:11:18.983 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T07:11:19.003 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-16T07:11:19.003 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-16T07:11:19.003 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-16T07:11:19.003 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T07:11:19.004 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T07:11:19.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:11:19.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-16T07:11:19.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:11:19.027 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-16T07:11:19.192 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T07:11:19.192 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-16T07:11:19.192 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-09-16T07:11:19.200 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T07:11:19.201 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-16T07:11:19.201 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-16T07:11:19.210 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-503.el9.x86_64 2024-09-16T07:11:19.210 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T07:11:19.210 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-16T07:11:19.211 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum upgrade -y linux-firmware 2024-09-16T07:11:19.219 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-503.el9.x86_64 2024-09-16T07:11:19.219 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T07:11:19.220 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-16T07:11:19.220 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum upgrade -y linux-firmware 2024-09-16T07:11:21.623 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-16T07:11:22.109 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.2 MB 00:01 2024-09-16T07:11:23.416 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-16T07:11:29.125 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-16T07:11:30.567 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-16T07:11:31.530 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-16T07:11:36.919 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T07:11:37.915 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 20 MB 00:14 2024-09-16T07:11:39.021 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-16T07:11:39.022 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2024-09-16T07:11:39.022 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-16T07:11:39.114 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-16T07:11:39.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T07:11:39.256 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-16T07:11:39.256 DEBUG:teuthology.orchestra.run.smithi177:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-16T07:11:39.259 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi177.front.sepia.ceph.com to complete... 2024-09-16T07:11:42.774 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.5 MB 00:01 2024-09-16T07:11:44.458 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-16T07:11:45.404 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-16T07:11:45.563 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16 07:11:45 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-16T07:11:45.563 INFO:teuthology.orchestra.run.smithi177.stderr:FINISHED --2024-09-16 07:11:45-- 2024-09-16T07:11:45.563 INFO:teuthology.orchestra.run.smithi177.stderr:Total wall clock time: 6.3s 2024-09-16T07:11:45.563 INFO:teuthology.orchestra.run.smithi177.stderr:Downloaded: 1 files, 343M in 6.0s (57.2 MB/s) 2024-09-16T07:11:45.567 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-16T07:11:45.567 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-16T07:11:47.680 INFO:teuthology.orchestra.run.smithi177.stdout:Verifying... ######################################## 2024-09-16T07:11:47.829 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing... ######################################## 2024-09-16T07:11:47.936 INFO:teuthology.orchestra.run.smithi177.stdout:Updating / installing... 2024-09-16T07:11:50.768 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-09-16T07:11:52.864 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-16T07:11:52.865 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-16T07:11:52.865 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-16T07:11:52.988 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-16T07:11:52.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T07:11:53.139 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-16T07:11:53.140 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-16T07:11:53.142 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi071.front.sepia.ceph.com to complete... 2024-09-16T07:11:55.981 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-16T07:11:59.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16 07:11:59 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-16T07:11:59.476 INFO:teuthology.orchestra.run.smithi071.stderr:FINISHED --2024-09-16 07:11:59-- 2024-09-16T07:11:59.476 INFO:teuthology.orchestra.run.smithi071.stderr:Total wall clock time: 6.3s 2024-09-16T07:11:59.476 INFO:teuthology.orchestra.run.smithi071.stderr:Downloaded: 1 files, 343M in 6.0s (57.0 MB/s) 2024-09-16T07:11:59.479 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-16T07:11:59.480 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-16T07:12:01.584 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying... ######################################## 2024-09-16T07:12:01.772 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing... ######################################## 2024-09-16T07:12:01.895 INFO:teuthology.orchestra.run.smithi071.stdout:Updating / installing... 2024-09-16T07:12:09.857 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-16T07:12:35.958 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-16T07:12:35.959 DEBUG:teuthology.orchestra.run.smithi177:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-16T07:12:36.054 INFO:teuthology.orchestra.run.smithi177.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-16T07:12:36.055 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-16T07:12:36.056 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-16T07:12:36.057 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-16T07:12:36.058 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-16T07:12:36.059 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-16T07:12:36.060 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-16T07:12:36.061 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-16T07:12:36.062 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-16T07:12:36.063 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-16T07:12:36.064 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-16T07:12:36.065 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-16T07:12:36.066 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-16T07:12:36.067 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-16T07:12:36.068 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-16T07:12:36.069 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-16T07:12:36.070 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-16T07:12:36.070 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-16T07:12:36.070 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-16T07:12:36.070 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-16T07:12:36.070 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-16T07:12:36.070 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-16T07:12:36.070 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-16T07:12:36.070 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-16T07:12:36.071 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-16T07:12:36.071 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-16T07:12:36.071 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-16T07:12:36.071 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-16T07:12:36.071 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-16T07:12:36.071 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-16T07:12:36.072 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-16T07:12:36.072 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-16T07:12:36.072 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-16T07:12:36.072 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-16T07:12:36.072 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-16T07:12:36.072 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-16T07:12:36.073 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-16T07:12:36.073 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-16T07:12:36.073 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-16T07:12:36.073 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-16T07:12:36.073 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-16T07:12:36.073 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-16T07:12:36.073 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-16T07:12:36.074 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-16T07:12:36.074 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-16T07:12:36.074 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-16T07:12:36.074 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-16T07:12:36.074 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-16T07:12:36.074 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-16T07:12:36.074 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-16T07:12:36.074 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-16T07:12:36.075 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-16T07:12:36.075 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-16T07:12:36.075 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-16T07:12:36.075 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-16T07:12:36.075 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-16T07:12:36.075 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-16T07:12:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-16T07:12:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-16T07:12:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-16T07:12:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-16T07:12:36.076 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-16T07:12:36.077 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-16T07:12:36.077 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-16T07:12:36.077 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-16T07:12:36.077 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-16T07:12:36.077 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-16T07:12:36.077 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-16T07:12:36.078 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-16T07:12:36.079 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-16T07:12:36.080 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-16T07:12:36.081 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-16T07:12:36.082 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-16T07:12:36.083 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-16T07:12:36.084 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-16T07:12:36.085 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-16T07:12:36.086 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-16T07:12:36.087 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-16T07:12:36.088 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-16T07:12:36.089 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-16T07:12:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-16T07:12:36.091 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-16T07:12:36.092 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-16T07:12:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-16T07:12:36.094 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-16T07:12:36.095 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-16T07:12:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-16T07:12:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-16T07:12:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-16T07:12:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-16T07:12:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-16T07:12:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-16T07:12:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-16T07:12:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-16T07:12:36.097 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-16T07:12:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-16T07:12:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-16T07:12:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-16T07:12:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-16T07:12:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-16T07:12:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-16T07:12:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-16T07:12:36.098 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-16T07:12:36.099 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-16T07:12:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-16T07:12:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-16T07:12:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-16T07:12:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-16T07:12:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-16T07:12:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-16T07:12:36.100 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-16T07:12:36.101 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-16T07:12:36.101 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-16T07:12:36.101 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-16T07:12:36.101 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-16T07:12:36.101 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-16T07:12:36.101 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-16T07:12:36.101 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-16T07:12:36.101 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-16T07:12:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-16T07:12:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-16T07:12:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-16T07:12:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-16T07:12:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-16T07:12:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-16T07:12:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-16T07:12:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-16T07:12:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-16T07:12:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-16T07:12:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-16T07:12:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-16T07:12:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-16T07:12:36.104 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-16T07:12:36.105 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-16T07:12:36.105 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-16T07:12:36.105 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-16T07:12:36.105 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-16T07:12:36.105 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-16T07:12:36.105 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-16T07:12:36.105 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-16T07:12:36.106 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-16T07:12:36.107 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-16T07:12:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-16T07:12:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-16T07:12:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-16T07:12:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-16T07:12:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-16T07:12:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-16T07:12:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-16T07:12:36.108 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-16T07:12:36.109 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-16T07:12:36.109 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-16T07:12:36.109 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-16T07:12:36.109 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-16T07:12:36.109 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-16T07:12:36.109 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-16T07:12:36.109 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-16T07:12:36.109 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-16T07:12:36.110 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-16T07:12:36.111 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-16T07:12:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-16T07:12:36.113 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-16T07:12:36.114 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-16T07:12:36.115 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-16T07:12:36.116 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-16T07:12:36.117 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-16T07:12:36.118 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-16T07:12:36.118 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-16T07:12:36.118 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-16T07:12:36.118 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-16T07:12:36.118 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-16T07:12:36.118 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-16T07:12:36.118 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-16T07:12:36.118 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-16T07:12:36.119 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-16T07:12:36.120 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-16T07:12:36.121 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-16T07:12:36.122 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-16T07:12:36.123 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-16T07:12:36.124 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-16T07:12:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-16T07:12:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-16T07:12:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-16T07:12:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-16T07:12:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-16T07:12:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-16T07:12:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-16T07:12:36.125 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-16T07:12:36.126 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-16T07:12:36.127 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-16T07:12:36.128 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-16T07:12:36.129 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-16T07:12:36.130 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-16T07:12:36.131 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-16T07:12:36.132 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-16T07:12:36.133 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-16T07:12:36.134 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-16T07:12:36.135 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-16T07:12:36.136 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-16T07:12:36.137 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-16T07:12:36.138 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-16T07:12:36.139 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-16T07:12:36.140 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-16T07:12:36.141 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-16T07:12:36.142 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-16T07:12:36.143 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-16T07:12:36.144 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-16T07:12:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-16T07:12:36.146 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-16T07:12:36.147 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-16T07:12:36.148 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-16T07:12:36.149 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-16T07:12:36.150 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-16T07:12:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-16T07:12:36.152 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-16T07:12:36.152 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-16T07:12:36.152 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-16T07:12:36.152 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-16T07:12:36.152 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-16T07:12:36.152 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-16T07:12:36.152 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-16T07:12:36.152 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-16T07:12:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-16T07:12:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-16T07:12:36.155 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-16T07:12:36.156 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-16T07:12:36.157 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-16T07:12:36.158 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-16T07:12:36.159 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-16T07:12:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-16T07:12:36.161 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-16T07:12:36.162 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-16T07:12:36.162 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-16T07:12:36.162 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-16T07:12:36.162 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-16T07:12:36.162 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-16T07:12:36.162 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-16T07:12:36.162 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-16T07:12:36.162 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-16T07:12:36.163 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-16T07:12:36.163 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-16T07:12:36.163 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-16T07:12:36.163 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-16T07:12:36.163 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-16T07:12:36.163 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-16T07:12:36.163 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-16T07:12:36.164 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-16T07:12:36.164 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-16T07:12:36.164 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-16T07:12:36.164 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-16T07:12:36.164 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-16T07:12:36.164 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-16T07:12:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-16T07:12:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-16T07:12:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-16T07:12:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-16T07:12:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-16T07:12:36.165 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-16T07:12:36.166 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-16T07:12:36.166 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-16T07:12:36.166 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-16T07:12:36.166 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-16T07:12:36.166 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-16T07:12:36.166 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-16T07:12:36.166 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-16T07:12:36.166 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-16T07:12:36.167 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-16T07:12:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-16T07:12:36.169 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-16T07:12:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-16T07:12:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-16T07:12:36.172 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-16T07:12:36.172 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-16T07:12:36.172 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-16T07:12:36.172 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-16T07:12:36.172 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-16T07:12:36.172 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-16T07:12:36.172 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-16T07:12:36.172 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-16T07:12:36.173 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-16T07:12:36.174 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-16T07:12:36.175 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-16T07:12:36.176 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-16T07:12:36.177 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-16T07:12:36.178 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-16T07:12:36.179 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-16T07:12:36.180 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-16T07:12:36.181 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-16T07:12:36.182 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-16T07:12:36.183 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-16T07:12:36.184 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-16T07:12:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-16T07:12:36.186 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-16T07:12:36.187 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-16T07:12:36.188 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-16T07:12:36.189 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-16T07:12:36.190 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-16T07:12:36.190 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-16T07:12:36.190 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-16T07:12:36.190 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-16T07:12:36.190 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-16T07:12:36.190 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-16T07:12:36.190 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-16T07:12:36.191 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-16T07:12:36.191 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-16T07:12:36.191 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-16T07:12:36.191 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-16T07:12:36.191 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-16T07:12:36.191 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-16T07:12:36.192 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-16T07:12:36.192 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-16T07:12:36.192 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-16T07:12:36.192 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-16T07:12:36.192 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-16T07:12:36.192 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-16T07:12:36.193 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-16T07:12:36.193 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-16T07:12:36.193 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-16T07:12:36.193 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-16T07:12:36.193 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-16T07:12:36.193 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-16T07:12:36.194 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-16T07:12:36.194 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-16T07:12:36.194 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-16T07:12:36.194 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-16T07:12:36.194 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-16T07:12:36.194 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-16T07:12:36.194 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-16T07:12:36.195 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-16T07:12:36.195 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-16T07:12:36.195 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-16T07:12:36.195 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-16T07:12:36.195 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-16T07:12:36.195 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-16T07:12:36.196 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-16T07:12:36.196 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-16T07:12:36.196 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-16T07:12:36.196 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-16T07:12:36.196 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-16T07:12:36.196 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-16T07:12:36.197 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-16T07:12:36.197 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-16T07:12:36.197 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-16T07:12:36.197 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-16T07:12:36.197 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-16T07:12:36.198 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-16T07:12:36.198 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-16T07:12:36.198 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-16T07:12:36.198 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-16T07:12:36.198 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-16T07:12:36.198 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-16T07:12:36.199 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-16T07:12:36.199 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-16T07:12:36.199 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-16T07:12:36.199 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-16T07:12:36.199 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-16T07:12:36.199 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-16T07:12:36.200 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-16T07:12:36.200 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-16T07:12:36.200 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-16T07:12:36.200 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-16T07:12:36.200 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-16T07:12:36.201 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-16T07:12:36.201 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-16T07:12:36.201 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-16T07:12:36.201 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-16T07:12:36.201 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-16T07:12:36.201 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-16T07:12:36.202 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-16T07:12:36.202 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-16T07:12:36.202 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-16T07:12:36.202 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-16T07:12:36.202 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-16T07:12:36.202 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-16T07:12:36.203 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-16T07:12:36.203 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-16T07:12:36.203 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-16T07:12:36.203 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-16T07:12:36.203 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-16T07:12:36.203 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-16T07:12:36.204 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-16T07:12:36.204 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-16T07:12:36.204 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-16T07:12:36.204 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-16T07:12:36.204 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-16T07:12:36.204 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-16T07:12:36.205 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-16T07:12:36.206 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-16T07:12:36.207 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-16T07:12:36.208 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-16T07:12:36.209 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-16T07:12:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-16T07:12:36.211 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-16T07:12:36.212 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-16T07:12:36.213 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-16T07:12:36.214 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-16T07:12:36.215 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-16T07:12:36.216 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-16T07:12:36.217 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-16T07:12:36.218 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-16T07:12:36.219 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-16T07:12:36.220 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-16T07:12:36.221 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-16T07:12:36.222 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-16T07:12:36.223 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-16T07:12:36.224 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-16T07:12:36.225 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-16T07:12:36.226 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-16T07:12:36.227 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-16T07:12:36.228 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-16T07:12:36.229 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-16T07:12:36.230 INFO:teuthology.orchestra.run.smithi177.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T07:12:36.235 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:36.235 DEBUG:teuthology.orchestra.run.smithi177:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-16T07:12:36.258 INFO:teuthology.orchestra.run.smithi177.stdout:postinstall scriptlet (using /bin/sh): 2024-09-16T07:12:36.258 INFO:teuthology.orchestra.run.smithi177.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout:fi 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout:for file in vmlinuz System.map config; do 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: fi 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout:done 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T07:12:36.259 INFO:teuthology.orchestra.run.smithi177.stdout:fi 2024-09-16T07:12:36.260 INFO:teuthology.orchestra.run.smithi177.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-16T07:12:36.260 INFO:teuthology.orchestra.run.smithi177.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:36.260 INFO:teuthology.orchestra.run.smithi177.stdout:fi 2024-09-16T07:12:36.260 INFO:teuthology.orchestra.run.smithi177.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-16T07:12:36.260 INFO:teuthology.orchestra.run.smithi177.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T07:12:36.260 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:36.260 INFO:teuthology.orchestra.run.smithi177.stdout:fi 2024-09-16T07:12:36.261 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -qi grub2-tools 2024-09-16T07:12:36.334 INFO:teuthology.orchestra.run.smithi177.stdout:Name : grub2-tools 2024-09-16T07:12:36.334 INFO:teuthology.orchestra.run.smithi177.stdout:Epoch : 1 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Version : 2.06 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Release : 82.el9 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: x86_64 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Group : Unspecified 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Size : 8274670 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:License : GPLv3+ 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T07:12:36.335 INFO:teuthology.orchestra.run.smithi177.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T07:12:36.336 INFO:teuthology.orchestra.run.smithi177.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T07:12:36.336 INFO:teuthology.orchestra.run.smithi177.stdout:Packager : builder@centos.org 2024-09-16T07:12:36.336 INFO:teuthology.orchestra.run.smithi177.stdout:Vendor : CentOS 2024-09-16T07:12:36.336 INFO:teuthology.orchestra.run.smithi177.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T07:12:36.336 INFO:teuthology.orchestra.run.smithi177.stdout:Summary : Support tools for GRUB. 2024-09-16T07:12:36.336 INFO:teuthology.orchestra.run.smithi177.stdout:Description : 2024-09-16T07:12:36.336 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:12:36.336 INFO:teuthology.orchestra.run.smithi177.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T07:12:36.337 INFO:teuthology.orchestra.run.smithi177.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T07:12:36.337 INFO:teuthology.orchestra.run.smithi177.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T07:12:36.337 INFO:teuthology.orchestra.run.smithi177.stdout:hardware devices. 2024-09-16T07:12:36.337 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:12:36.337 INFO:teuthology.orchestra.run.smithi177.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T07:12:36.338 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T07:12:36.338 INFO:teuthology.task.kernel:Updating grub on smithi177 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:36.338 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T07:12:36.933 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2024-09-16T07:12:38.352 INFO:teuthology.orchestra.run.smithi177.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T07:12:38.374 INFO:teuthology.orchestra.run.smithi177.stderr:done 2024-09-16T07:12:38.376 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-16T07:12:38.392 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.fqhoHaR1Jw 2024-09-16T07:12:38.392 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fqhoHaR1Jw 2024-09-16T07:12:38.470 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.fqhoHaR1Jw 2024-09-16T07:12:38.613 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.fqhoHaR1Jw is 6KB 2024-09-16T07:12:38.662 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.fqhoHaR1Jw 2024-09-16T07:12:38.677 DEBUG:teuthology.orchestra.run.smithi177:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T07:12:38.744 INFO:teuthology.orchestra.run.smithi177.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T07:12:38.745 INFO:teuthology.orchestra.run.smithi177.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T07:12:38.745 INFO:teuthology.orchestra.run.smithi177.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T07:12:38.745 INFO:teuthology.orchestra.run.smithi177.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-16T07:12:38.746 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:38.901 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2024-09-16T07:12:49.958 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-16T07:12:49.959 DEBUG:teuthology.orchestra.run.smithi071:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-16T07:12:50.044 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:50.044 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:50.044 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:50.044 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:50.044 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-16T07:12:50.044 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-16T07:12:50.044 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-16T07:12:50.044 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-16T07:12:50.045 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-16T07:12:50.046 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-16T07:12:50.047 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-16T07:12:50.048 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-16T07:12:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-16T07:12:50.050 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-16T07:12:50.051 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-16T07:12:50.052 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-16T07:12:50.053 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-16T07:12:50.054 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-16T07:12:50.055 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-16T07:12:50.056 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-16T07:12:50.057 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-16T07:12:50.058 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-16T07:12:50.059 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-16T07:12:50.060 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-16T07:12:50.061 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-16T07:12:50.062 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-16T07:12:50.063 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-16T07:12:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-16T07:12:50.065 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-16T07:12:50.066 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-16T07:12:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-16T07:12:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-16T07:12:50.069 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-16T07:12:50.070 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-16T07:12:50.071 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-16T07:12:50.072 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-16T07:12:50.073 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-16T07:12:50.074 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-16T07:12:50.075 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-16T07:12:50.076 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-16T07:12:50.077 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-16T07:12:50.078 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-16T07:12:50.079 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-16T07:12:50.080 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-16T07:12:50.081 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-16T07:12:50.082 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-16T07:12:50.083 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-16T07:12:50.084 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-16T07:12:50.085 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-16T07:12:50.086 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-16T07:12:50.087 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-16T07:12:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-16T07:12:50.089 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-16T07:12:50.090 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-16T07:12:50.091 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-16T07:12:50.092 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-16T07:12:50.093 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-16T07:12:50.094 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-16T07:12:50.095 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-16T07:12:50.096 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-16T07:12:50.097 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-16T07:12:50.098 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-16T07:12:50.099 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-16T07:12:50.100 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-16T07:12:50.100 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-16T07:12:50.100 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-16T07:12:50.100 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-16T07:12:50.101 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-16T07:12:50.102 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-16T07:12:50.103 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-16T07:12:50.104 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-16T07:12:50.105 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-16T07:12:50.106 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-16T07:12:50.107 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-16T07:12:50.108 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-16T07:12:50.109 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-16T07:12:50.110 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-16T07:12:50.111 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-16T07:12:50.111 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-16T07:12:50.111 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-16T07:12:50.111 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-16T07:12:50.111 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-16T07:12:50.111 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-16T07:12:50.111 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-16T07:12:50.111 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-16T07:12:50.112 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-16T07:12:50.113 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-16T07:12:50.114 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-16T07:12:50.115 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-16T07:12:50.116 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-16T07:12:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-16T07:12:50.118 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-16T07:12:50.119 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-16T07:12:50.120 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-16T07:12:50.121 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-16T07:12:50.122 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-16T07:12:50.123 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-16T07:12:50.124 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-16T07:12:50.125 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-16T07:12:50.126 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-16T07:12:50.127 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-16T07:12:50.128 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-16T07:12:50.129 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-16T07:12:50.130 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-16T07:12:50.131 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-16T07:12:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-16T07:12:50.133 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-16T07:12:50.134 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-16T07:12:50.135 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-16T07:12:50.136 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-16T07:12:50.137 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-16T07:12:50.138 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-16T07:12:50.139 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-16T07:12:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-16T07:12:50.141 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-16T07:12:50.142 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-16T07:12:50.143 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-16T07:12:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-16T07:12:50.145 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-16T07:12:50.146 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-16T07:12:50.147 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-16T07:12:50.148 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-16T07:12:50.149 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-16T07:12:50.150 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-16T07:12:50.151 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-16T07:12:50.152 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-16T07:12:50.153 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-16T07:12:50.154 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-16T07:12:50.155 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-16T07:12:50.156 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-16T07:12:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-16T07:12:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-16T07:12:50.159 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-16T07:12:50.160 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-16T07:12:50.161 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-16T07:12:50.162 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-16T07:12:50.163 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-16T07:12:50.164 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-16T07:12:50.165 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-16T07:12:50.165 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-16T07:12:50.165 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-16T07:12:50.165 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-16T07:12:50.165 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-16T07:12:50.165 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-16T07:12:50.165 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-16T07:12:50.169 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-16T07:12:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-16T07:12:50.171 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-16T07:12:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-16T07:12:50.173 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-16T07:12:50.174 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-16T07:12:50.175 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-16T07:12:50.176 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-16T07:12:50.177 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-16T07:12:50.178 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-16T07:12:50.179 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-16T07:12:50.180 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-16T07:12:50.181 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-16T07:12:50.182 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-16T07:12:50.183 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-16T07:12:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-16T07:12:50.185 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-16T07:12:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-16T07:12:50.187 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-16T07:12:50.188 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-16T07:12:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-16T07:12:50.190 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-16T07:12:50.191 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-16T07:12:50.192 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-16T07:12:50.193 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-16T07:12:50.194 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-16T07:12:50.195 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-16T07:12:50.196 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-16T07:12:50.197 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T07:12:50.202 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:50.202 DEBUG:teuthology.orchestra.run.smithi071:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-16T07:12:50.225 INFO:teuthology.orchestra.run.smithi071.stdout:postinstall scriptlet (using /bin/sh): 2024-09-16T07:12:50.225 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T07:12:50.225 INFO:teuthology.orchestra.run.smithi071.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout:for file in vmlinuz System.map config; do 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout: fi 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout:done 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-09-16T07:12:50.226 INFO:teuthology.orchestra.run.smithi071.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-16T07:12:50.227 INFO:teuthology.orchestra.run.smithi071.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:50.227 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-09-16T07:12:50.227 INFO:teuthology.orchestra.run.smithi071.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-16T07:12:50.227 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T07:12:50.227 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:50.227 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-09-16T07:12:50.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 82.el9 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8274670 2024-09-16T07:12:50.308 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T07:12:50.309 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T07:12:50.310 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-09-16T07:12:50.310 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:12:50.310 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T07:12:50.310 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T07:12:50.311 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:50.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T07:12:50.862 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-09-16T07:12:52.197 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T07:12:52.243 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-09-16T07:12:52.245 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-16T07:12:52.261 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.JbFjNsFo01 2024-09-16T07:12:52.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JbFjNsFo01 2024-09-16T07:12:52.328 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.JbFjNsFo01 2024-09-16T07:12:52.469 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.JbFjNsFo01 is 6KB 2024-09-16T07:12:52.519 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.JbFjNsFo01 2024-09-16T07:12:52.534 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T07:12:52.597 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T07:12:52.597 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T07:12:52.597 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T07:12:52.597 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-16T07:12:52.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:12:52.757 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-09-16T07:13:08.918 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-16T07:13:08.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-16T07:13:08.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:13:22.765 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T07:13:22.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-16T07:13:22.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:13:27.357 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-09-16T07:13:36.365 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-16T07:13:36.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:13:39.421 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-09-16T07:13:41.181 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-16T07:13:50.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-16T07:13:50.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:13:51.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-16T07:13:51.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:14:23.468 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-09-16T07:14:24.050 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-16T07:14:24.051 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-16T07:14:24.051 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-09-16T07:14:24.085 INFO:teuthology.orchestra.run.smithi177.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:14:24.085 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:14:24.085 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T07:14:24.086 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T07:14:25.087 DEBUG:teuthology.parallel:result is None 2024-09-16T07:14:50.192 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T07:15:02.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-16T07:15:02.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:15:02.553 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-16T07:15:03.040 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-16T07:15:03.040 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-16T07:15:03.041 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-16T07:15:03.075 INFO:teuthology.orchestra.run.smithi071.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:15:03.076 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T07:15:03.076 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T07:15:03.076 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T07:15:04.079 DEBUG:teuthology.parallel:result is None 2024-09-16T07:15:04.080 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T07:15:04.087 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T07:15:04.087 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T07:15:04.091 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T07:15:04.133 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T07:15:04.139 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T07:15:04.180 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T07:15:04.180 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T07:15:04.184 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T07:15:04.239 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T07:15:04.245 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T07:15:04.246 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T07:15:04.268 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T07:15:04.369 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T07:15:04.397 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T07:15:04.401 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T07:15:04.431 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T07:15:04.434 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T07:15:04.440 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T07:15:04.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T07:15:04.445 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T07:15:04.500 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T07:15:04.509 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T07:15:04.509 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T07:15:04.513 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T07:15:04.547 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T07:15:04.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T07:15:04.725 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T07:15:04.785 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T07:15:04.858 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:15:04.858 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T07:15:04.912 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T07:15:04.977 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T07:15:05.051 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T07:15:05.115 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T07:15:05.185 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:15:05.185 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T07:15:05.240 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-09-16T07:15:05.244 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-09-16T07:15:05.322 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T07:15:05.373 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T07:15:05.842 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T07:15:05.850 INFO:teuthology.task.internal:Starting timer... 2024-09-16T07:15:05.850 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T07:15:05.892 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T07:15:05.941 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-09-16T07:15:06.027 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-09-16T07:15:06.031 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-09-16T07:15:06.112 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-09-16T07:15:06.115 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T07:15:06.115 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-09-16T07:15:06.155 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-09-16T07:15:06.155 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-09-16T07:15:06.193 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-09-16T07:15:06.194 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2024-09-16T07:15:06.194 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T07:15:06.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:15:06.253 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T07:15:06.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:15:06.310 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T07:15:06.310 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-09-16T07:15:06.377 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-09-16T07:15:06.442 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T07:15:06.452 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T07:15:06.463 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T07:15:06.464 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T07:19:49.309 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2024-09-16T07:19:49.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-16T07:19:49.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:19:49.425 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-16T07:19:49.532 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-16T07:19:49.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-16T07:19:49.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T07:19:49.619 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-09-16T07:19:49.732 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-16T07:19:49.732 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T07:19:49.744 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T07:19:49.744 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T07:19:49.745 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T07:19:49.749 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T07:19:49.749 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T07:19:49.805 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T07:19:49.833 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T07:19:49.834 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T07:19:49.863 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T07:19:49.885 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-09-16T07:19:49.908 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-09-16T07:19:49.914 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-09-16T07:19:49.937 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T07:19:49.938 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-09-16T07:19:49.967 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T07:19:49.967 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T07:19:49.997 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T07:19:50.061 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-16T07:19:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T07:19:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-16T07:19:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:19:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:19:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:19:50.068 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:19:50.099 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-09-16T07:19:50.105 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T07:19:50.105 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-09-16T07:19:50.105 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:19:50.105 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:19:50.105 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:19:50.105 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:19:50.106 INFO:teuthology.run_tasks:Running task install... 2024-09-16T07:19:50.115 DEBUG:teuthology.task.install:project ceph 2024-09-16T07:19:50.115 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74'}} 2024-09-16T07:19:50.115 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74'} 2024-09-16T07:19:50.115 INFO:teuthology.task.install:Using flavor: default 2024-09-16T07:19:50.123 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T07:19:50.124 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T07:19:50.124 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74', 'tag': None, 'wait_for_package': False} 2024-09-16T07:19:50.125 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ef5f0587d1cad82c4295c20951c303dc342ac74 2024-09-16T07:19:50.127 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74', 'tag': None, 'wait_for_package': False} 2024-09-16T07:19:50.127 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ef5f0587d1cad82c4295c20951c303dc342ac74 2024-09-16T07:19:50.274 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-15-1936/4ef5f0587d1cad82c4295c20951c303dc342ac74/centos/9/flavors/default/ 2024-09-16T07:19:50.274 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4938.g4ef5f058 2024-09-16T07:19:50.277 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-15-1936/4ef5f0587d1cad82c4295c20951c303dc342ac74/centos/9/flavors/default/ 2024-09-16T07:19:50.278 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4938.g4ef5f058 2024-09-16T07:19:50.387 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-15-1936/4ef5f0587d1cad82c4295c20951c303dc342ac74/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-15-1936/4ef5f0587d1cad82c4295c20951c303dc342ac74/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-15-1936/4ef5f0587d1cad82c4295c20951c303dc342ac74/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T07:19:50.387 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:19:50.387 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T07:19:50.392 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-15-1936/4ef5f0587d1cad82c4295c20951c303dc342ac74/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-15-1936/4ef5f0587d1cad82c4295c20951c303dc342ac74/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-15-1936/4ef5f0587d1cad82c4295c20951c303dc342ac74/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T07:19:50.393 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:19:50.393 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T07:19:50.444 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T07:19:50.444 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ef5f0587d1cad82c4295c20951c303dc342ac74/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T07:19:50.450 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T07:19:50.450 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ef5f0587d1cad82c4295c20951c303dc342ac74/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T07:19:50.540 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T07:19:50.547 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T07:19:50.619 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T07:19:50.625 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T07:19:50.673 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-09-16T07:19:50.676 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-09-16T07:19:50.680 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-09-16T07:19:50.683 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-09-16T07:19:50.957 INFO:teuthology.orchestra.run.smithi177.stdout:45 files removed 2024-09-16T07:19:50.959 INFO:teuthology.orchestra.run.smithi071.stdout:45 files removed 2024-09-16T07:19:50.989 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T07:19:50.995 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T07:19:51.683 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 284 kB/s | 84 kB 00:00 2024-09-16T07:19:51.691 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 275 kB/s | 84 kB 00:00 2024-09-16T07:19:51.992 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-16T07:19:52.000 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-16T07:19:52.276 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T07:19:52.764 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 7.4 kB/s | 1.9 kB 00:00 2024-09-16T07:19:52.834 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-16T07:19:52.851 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-16T07:19:54.752 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-16T07:19:56.710 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 20 MB 00:02 2024-09-16T07:20:00.286 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-16T07:20:01.252 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-16T07:20:01.745 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 115 kB/s | 19 kB 00:00 2024-09-16T07:20:02.643 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-16T07:20:02.828 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-16T07:20:03.657 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-16T07:20:08.890 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-16T07:20:09.929 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-16T07:20:11.369 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T07:20:11.370 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T07:20:11.381 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T07:20:11.382 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T07:20:11.383 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T07:20:11.383 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T07:20:11.385 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T07:20:11.387 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T07:20:11.388 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T07:20:11.454 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-16T07:20:11.475 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-09-16T07:20:11.475 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-09-16T07:20:11.475 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-09-16T07:20:11.475 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-16T07:20:11.475 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T07:20:11.475 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 6.6 k 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 5.5 M 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 900 k 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 145 k 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 1.5 M 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 154 k 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 14 M 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 7.4 M 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 49 k 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 11 M 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 46 M 2024-09-16T07:20:11.476 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 285 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 784 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 225 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 27 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 18 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 32 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 755 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 127 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T07:20:11.477 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 161 k 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 320 k 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 305 k 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 100 k 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 85 k 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 3.2 M 2024-09-16T07:20:11.478 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 170 k 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 3.5 M 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 3.2 M 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T07:20:11.479 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 22 M 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 34 k 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 2.4 M 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 291 k 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 4.8 M 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 17 M 2024-09-16T07:20:11.480 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 16 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 25 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 161 k 2024-09-16T07:20:11.481 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 508 k 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 5.5 M 2024-09-16T07:20:11.482 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T07:20:11.483 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 45 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 148 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T07:20:11.484 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T07:20:11.485 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T07:20:11.486 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T07:20:11.487 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T07:20:11.488 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout:Installing weak dependencies: 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T07:20:11.489 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:20:11.490 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-16T07:20:11.490 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-09-16T07:20:11.490 INFO:teuthology.orchestra.run.smithi177.stdout:Install 149 Packages 2024-09-16T07:20:11.490 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 8 Packages 2024-09-16T07:20:11.490 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:20:11.490 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 235 M 2024-09-16T07:20:11.490 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-16T07:20:12.208 INFO:teuthology.orchestra.run.smithi177.stdout:(1/157): ceph-19.3.0-4938.g4ef5f058.el9.x86_64. 53 kB/s | 6.6 kB 00:00 2024-09-16T07:20:12.383 INFO:teuthology.orchestra.run.smithi177.stdout:(2/157): ceph-fuse-19.3.0-4938.g4ef5f058.el9.x8 5.0 MB/s | 900 kB 00:00 2024-09-16T07:20:12.400 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T07:20:12.401 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T07:20:12.413 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T07:20:12.414 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T07:20:12.415 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T07:20:12.416 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T07:20:12.417 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T07:20:12.420 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T07:20:12.421 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T07:20:12.475 INFO:teuthology.orchestra.run.smithi177.stdout:(3/157): ceph-immutable-object-cache-19.3.0-493 1.5 MB/s | 145 kB 00:00 2024-09-16T07:20:12.487 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 6.6 k 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 5.5 M 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 900 k 2024-09-16T07:20:12.512 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 145 k 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 1.5 M 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 154 k 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 14 M 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 7.4 M 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 49 k 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 11 M 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 46 M 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 285 k 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 784 k 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 225 k 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 27 k 2024-09-16T07:20:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 18 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 32 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 755 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 127 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T07:20:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 161 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 320 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 305 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 100 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 85 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 3.2 M 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 170 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T07:20:12.515 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 3.5 M 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 3.2 M 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 22 M 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 34 k 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 2.4 M 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 291 k 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 4.8 M 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 17 M 2024-09-16T07:20:12.516 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4938.g4ef5f058.el9 ceph-noarch 16 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 25 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 161 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T07:20:12.517 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 508 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 5.5 M 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T07:20:12.518 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T07:20:12.519 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 45 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.3.0-4938.g4ef5f058.el9 ceph 148 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T07:20:12.520 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T07:20:12.521 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T07:20:12.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T07:20:12.523 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T07:20:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-16T07:20:12.525 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-16T07:20:12.526 INFO:teuthology.orchestra.run.smithi071.stdout:Install 149 Packages 2024-09-16T07:20:12.526 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 8 Packages 2024-09-16T07:20:12.526 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:20:12.526 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 235 M 2024-09-16T07:20:12.526 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-16T07:20:12.550 INFO:teuthology.orchestra.run.smithi177.stdout:(4/157): ceph-base-19.3.0-4938.g4ef5f058.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-16T07:20:12.684 INFO:teuthology.orchestra.run.smithi177.stdout:(5/157): ceph-mds-19.3.0-4938.g4ef5f058.el9.x86 11 MB/s | 2.4 MB 00:00 2024-09-16T07:20:12.726 INFO:teuthology.orchestra.run.smithi177.stdout:(6/157): ceph-mgr-19.3.0-4938.g4ef5f058.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-09-16T07:20:13.089 INFO:teuthology.orchestra.run.smithi071.stdout:(1/157): ceph-19.3.0-4938.g4ef5f058.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-09-16T07:20:13.093 INFO:teuthology.orchestra.run.smithi177.stdout:(7/157): ceph-mon-19.3.0-4938.g4ef5f058.el9.x86 12 MB/s | 4.8 MB 00:00 2024-09-16T07:20:13.456 INFO:teuthology.orchestra.run.smithi071.stdout:(2/157): ceph-fuse-19.3.0-4938.g4ef5f058.el9.x8 2.4 MB/s | 900 kB 00:00 2024-09-16T07:20:13.460 INFO:teuthology.orchestra.run.smithi177.stdout:(8/157): ceph-common-19.3.0-4938.g4ef5f058.el9. 16 MB/s | 22 MB 00:01 2024-09-16T07:20:13.510 INFO:teuthology.orchestra.run.smithi177.stdout:(9/157): ceph-selinux-19.3.0-4938.g4ef5f058.el9 504 kB/s | 25 kB 00:00 2024-09-16T07:20:13.539 INFO:teuthology.orchestra.run.smithi071.stdout:(3/157): ceph-immutable-object-cache-19.3.0-493 1.7 MB/s | 145 kB 00:00 2024-09-16T07:20:13.760 INFO:teuthology.orchestra.run.smithi177.stdout:(10/157): ceph-osd-19.3.0-4938.g4ef5f058.el9.x8 17 MB/s | 17 MB 00:01 2024-09-16T07:20:13.927 INFO:teuthology.orchestra.run.smithi177.stdout:(11/157): ceph-radosgw-19.3.0-4938.g4ef5f058.el 13 MB/s | 11 MB 00:00 2024-09-16T07:20:13.939 INFO:teuthology.orchestra.run.smithi071.stdout:(4/157): ceph-mds-19.3.0-4938.g4ef5f058.el9.x86 6.0 MB/s | 2.4 MB 00:00 2024-09-16T07:20:13.952 INFO:teuthology.orchestra.run.smithi177.stdout:(12/157): cephfs-mirror-19.3.0-4938.g4ef5f058.e 1.1 MB/s | 225 kB 00:00 2024-09-16T07:20:13.978 INFO:teuthology.orchestra.run.smithi177.stdout:(13/157): cephfs-shell-19.3.0-4938.g4ef5f058.el 541 kB/s | 27 kB 00:00 2024-09-16T07:20:14.019 INFO:teuthology.orchestra.run.smithi177.stdout:(14/157): libcephfs-devel-19.3.0-4938.g4ef5f058 474 kB/s | 32 kB 00:00 2024-09-16T07:20:14.053 INFO:teuthology.orchestra.run.smithi177.stdout:(15/157): libcephfs2-19.3.0-4938.g4ef5f058.el9. 9.9 MB/s | 755 kB 00:00 2024-09-16T07:20:14.056 INFO:teuthology.orchestra.run.smithi071.stdout:(5/157): ceph-base-19.3.0-4938.g4ef5f058.el9.x8 4.6 MB/s | 5.5 MB 00:01 2024-09-16T07:20:14.078 INFO:teuthology.orchestra.run.smithi177.stdout:(16/157): libcephsqlite-19.3.0-4938.g4ef5f058.e 2.7 MB/s | 161 kB 00:00 2024-09-16T07:20:14.095 INFO:teuthology.orchestra.run.smithi177.stdout:(17/157): librados-devel-19.3.0-4938.g4ef5f058. 3.0 MB/s | 127 kB 00:00 2024-09-16T07:20:14.137 INFO:teuthology.orchestra.run.smithi177.stdout:(18/157): libradosstriper1-19.3.0-4938.g4ef5f05 8.5 MB/s | 508 kB 00:00 2024-09-16T07:20:14.190 INFO:teuthology.orchestra.run.smithi071.stdout:(6/157): ceph-mgr-19.3.0-4938.g4ef5f058.el9.x86 6.1 MB/s | 1.5 MB 00:00 2024-09-16T07:20:14.238 INFO:teuthology.orchestra.run.smithi177.stdout:(19/157): python3-ceph-argparse-19.3.0-4938.g4e 452 kB/s | 45 kB 00:00 2024-09-16T07:20:14.330 INFO:teuthology.orchestra.run.smithi177.stdout:(20/157): python3-ceph-common-19.3.0-4938.g4ef5 1.6 MB/s | 148 kB 00:00 2024-09-16T07:20:14.413 INFO:teuthology.orchestra.run.smithi177.stdout:(21/157): python3-cephfs-19.3.0-4938.g4ef5f058. 1.9 MB/s | 161 kB 00:00 2024-09-16T07:20:14.505 INFO:teuthology.orchestra.run.smithi177.stdout:(22/157): librgw2-19.3.0-4938.g4ef5f058.el9.x86 13 MB/s | 5.5 MB 00:00 2024-09-16T07:20:14.530 INFO:teuthology.orchestra.run.smithi177.stdout:(23/157): python3-rados-19.3.0-4938.g4ef5f058.e 2.7 MB/s | 320 kB 00:00 2024-09-16T07:20:14.581 INFO:teuthology.orchestra.run.smithi177.stdout:(24/157): python3-rbd-19.3.0-4938.g4ef5f058.el9 4.0 MB/s | 305 kB 00:00 2024-09-16T07:20:14.629 INFO:teuthology.orchestra.run.smithi071.stdout:(7/157): ceph-mon-19.3.0-4938.g4ef5f058.el9.x86 8.4 MB/s | 4.8 MB 00:00 2024-09-16T07:20:14.631 INFO:teuthology.orchestra.run.smithi177.stdout:(25/157): python3-rgw-19.3.0-4938.g4ef5f058.el9 997 kB/s | 100 kB 00:00 2024-09-16T07:20:14.656 INFO:teuthology.orchestra.run.smithi177.stdout:(26/157): rbd-fuse-19.3.0-4938.g4ef5f058.el9.x8 1.1 MB/s | 85 kB 00:00 2024-09-16T07:20:14.748 INFO:teuthology.orchestra.run.smithi177.stdout:(27/157): rbd-nbd-19.3.0-4938.g4ef5f058.el9.x86 1.8 MB/s | 170 kB 00:00 2024-09-16T07:20:14.840 INFO:teuthology.orchestra.run.smithi177.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4938.g 373 kB/s | 34 kB 00:00 2024-09-16T07:20:14.957 INFO:teuthology.orchestra.run.smithi177.stdout:(29/157): rbd-mirror-19.3.0-4938.g4ef5f058.el9. 9.8 MB/s | 3.2 MB 00:00 2024-09-16T07:20:15.015 INFO:teuthology.orchestra.run.smithi177.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4938.g4ef5f05 886 kB/s | 154 kB 00:00 2024-09-16T07:20:15.557 INFO:teuthology.orchestra.run.smithi071.stdout:(8/157): ceph-common-19.3.0-4938.g4ef5f058.el9. 8.1 MB/s | 22 MB 00:02 2024-09-16T07:20:15.657 INFO:teuthology.orchestra.run.smithi071.stdout:(9/157): ceph-selinux-19.3.0-4938.g4ef5f058.el9 252 kB/s | 25 kB 00:00 2024-09-16T07:20:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:(10/157): ceph-radosgw-19.3.0-4938.g4ef5f058.el 8.9 MB/s | 11 MB 00:01 2024-09-16T07:20:15.882 INFO:teuthology.orchestra.run.smithi177.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 8.5 MB/s | 7.4 MB 00:00 2024-09-16T07:20:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:(11/157): cephfs-mirror-19.3.0-4938.g4ef5f058.e 3.8 MB/s | 225 kB 00:00 2024-09-16T07:20:15.974 INFO:teuthology.orchestra.run.smithi177.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4938.g4e 3.1 MB/s | 291 kB 00:00 2024-09-16T07:20:16.000 INFO:teuthology.orchestra.run.smithi071.stdout:(12/157): cephfs-shell-19.3.0-4938.g4ef5f058.el 325 kB/s | 27 kB 00:00 2024-09-16T07:20:16.041 INFO:teuthology.orchestra.run.smithi177.stdout:(33/157): ceph-mgr-rook-19.3.0-4938.g4ef5f058.e 741 kB/s | 49 kB 00:00 2024-09-16T07:20:16.075 INFO:teuthology.orchestra.run.smithi071.stdout:(13/157): libcephfs-devel-19.3.0-4938.g4ef5f058 424 kB/s | 32 kB 00:00 2024-09-16T07:20:16.183 INFO:teuthology.orchestra.run.smithi071.stdout:(14/157): libcephfs2-19.3.0-4938.g4ef5f058.el9. 6.8 MB/s | 755 kB 00:00 2024-09-16T07:20:16.258 INFO:teuthology.orchestra.run.smithi177.stdout:(34/157): ceph-mgr-dashboard-19.3.0-4938.g4ef5f 11 MB/s | 14 MB 00:01 2024-09-16T07:20:16.283 INFO:teuthology.orchestra.run.smithi177.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4938.g4 68 kB/s | 16 kB 00:00 2024-09-16T07:20:16.317 INFO:teuthology.orchestra.run.smithi177.stdout:(36/157): ceph-volume-19.3.0-4938.g4ef5f058.el9 4.7 MB/s | 285 kB 00:00 2024-09-16T07:20:16.359 INFO:teuthology.orchestra.run.smithi071.stdout:(15/157): ceph-osd-19.3.0-4938.g4ef5f058.el9.x8 7.9 MB/s | 17 MB 00:02 2024-09-16T07:20:16.360 INFO:teuthology.orchestra.run.smithi177.stdout:(37/157): cephfs-top-19.3.0-4938.g4ef5f058.el9. 442 kB/s | 18 kB 00:00 2024-09-16T07:20:16.384 INFO:teuthology.orchestra.run.smithi071.stdout:(16/157): libcephsqlite-19.3.0-4938.g4ef5f058.e 806 kB/s | 161 kB 00:00 2024-09-16T07:20:16.393 INFO:teuthology.orchestra.run.smithi177.stdout:(38/157): cephadm-19.3.0-4938.g4ef5f058.el9.noa 7.1 MB/s | 784 kB 00:00 2024-09-16T07:20:16.426 INFO:teuthology.orchestra.run.smithi071.stdout:(17/157): librados-devel-19.3.0-4938.g4ef5f058. 1.9 MB/s | 127 kB 00:00 2024-09-16T07:20:16.459 INFO:teuthology.orchestra.run.smithi071.stdout:(18/157): libradosstriper1-19.3.0-4938.g4ef5f05 6.6 MB/s | 508 kB 00:00 2024-09-16T07:20:16.518 INFO:teuthology.orchestra.run.smithi177.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 504 kB/s | 80 kB 00:00 2024-09-16T07:20:16.518 INFO:teuthology.orchestra.run.smithi071.stdout:(19/157): python3-ceph-argparse-19.3.0-4938.g4e 773 kB/s | 45 kB 00:00 2024-09-16T07:20:16.552 INFO:teuthology.orchestra.run.smithi177.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.9 MB/s | 801 kB 00:00 2024-09-16T07:20:16.577 INFO:teuthology.orchestra.run.smithi071.stdout:(20/157): python3-ceph-common-19.3.0-4938.g4ef5 2.5 MB/s | 148 kB 00:00 2024-09-16T07:20:16.660 INFO:teuthology.orchestra.run.smithi071.stdout:(21/157): python3-cephfs-19.3.0-4938.g4ef5f058. 1.9 MB/s | 161 kB 00:00 2024-09-16T07:20:16.736 INFO:teuthology.orchestra.run.smithi071.stdout:(22/157): python3-rados-19.3.0-4938.g4ef5f058.e 4.2 MB/s | 320 kB 00:00 2024-09-16T07:20:16.802 INFO:teuthology.orchestra.run.smithi071.stdout:(23/157): python3-rbd-19.3.0-4938.g4ef5f058.el9 4.5 MB/s | 305 kB 00:00 2024-09-16T07:20:16.886 INFO:teuthology.orchestra.run.smithi071.stdout:(24/157): librgw2-19.3.0-4938.g4ef5f058.el9.x86 12 MB/s | 5.5 MB 00:00 2024-09-16T07:20:16.911 INFO:teuthology.orchestra.run.smithi071.stdout:(25/157): python3-rgw-19.3.0-4938.g4ef5f058.el9 920 kB/s | 100 kB 00:00 2024-09-16T07:20:16.937 INFO:teuthology.orchestra.run.smithi071.stdout:(26/157): rbd-fuse-19.3.0-4938.g4ef5f058.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-16T07:20:16.977 INFO:teuthology.orchestra.run.smithi177.stdout:(41/157): ceph-test-19.3.0-4938.g4ef5f058.el9.x 13 MB/s | 46 MB 00:03 2024-09-16T07:20:16.987 INFO:teuthology.orchestra.run.smithi071.stdout:(27/157): rbd-nbd-19.3.0-4938.g4ef5f058.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-16T07:20:17.002 INFO:teuthology.orchestra.run.smithi177.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 395 kB/s | 191 kB 00:00 2024-09-16T07:20:17.027 INFO:teuthology.orchestra.run.smithi177.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 532 kB/s | 253 kB 00:00 2024-09-16T07:20:17.063 INFO:teuthology.orchestra.run.smithi071.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4938.g 456 kB/s | 34 kB 00:00 2024-09-16T07:20:17.078 INFO:teuthology.orchestra.run.smithi177.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-16T07:20:17.103 INFO:teuthology.orchestra.run.smithi177.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-16T07:20:17.155 INFO:teuthology.orchestra.run.smithi071.stdout:(29/157): rbd-mirror-19.3.0-4938.g4ef5f058.el9. 13 MB/s | 3.2 MB 00:00 2024-09-16T07:20:17.161 INFO:teuthology.orchestra.run.smithi177.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-16T07:20:17.180 INFO:teuthology.orchestra.run.smithi071.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4938.g4ef5f05 1.3 MB/s | 154 kB 00:00 2024-09-16T07:20:17.187 INFO:teuthology.orchestra.run.smithi177.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-16T07:20:17.220 INFO:teuthology.orchestra.run.smithi177.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-09-16T07:20:17.395 INFO:teuthology.orchestra.run.smithi177.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 612 kB/s | 107 kB 00:00 2024-09-16T07:20:17.488 INFO:teuthology.orchestra.run.smithi177.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 236 kB/s | 22 kB 00:00 2024-09-16T07:20:17.521 INFO:teuthology.orchestra.run.smithi177.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-09-16T07:20:17.555 INFO:teuthology.orchestra.run.smithi177.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-09-16T07:20:17.581 INFO:teuthology.orchestra.run.smithi177.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-16T07:20:17.605 INFO:teuthology.orchestra.run.smithi177.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-16T07:20:17.639 INFO:teuthology.orchestra.run.smithi177.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-09-16T07:20:17.665 INFO:teuthology.orchestra.run.smithi177.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-16T07:20:17.698 INFO:teuthology.orchestra.run.smithi177.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-09-16T07:20:17.723 INFO:teuthology.orchestra.run.smithi177.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-16T07:20:17.740 INFO:teuthology.orchestra.run.smithi177.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-16T07:20:17.747 INFO:teuthology.orchestra.run.smithi071.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-16T07:20:17.847 INFO:teuthology.orchestra.run.smithi071.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4938.g4e 2.9 MB/s | 291 kB 00:00 2024-09-16T07:20:17.859 INFO:teuthology.orchestra.run.smithi177.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2024-09-16T07:20:17.891 INFO:teuthology.orchestra.run.smithi177.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-16T07:20:17.897 INFO:teuthology.orchestra.run.smithi071.stdout:(33/157): ceph-mgr-rook-19.3.0-4938.g4ef5f058.e 984 kB/s | 49 kB 00:00 2024-09-16T07:20:17.908 INFO:teuthology.orchestra.run.smithi177.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 510 kB/s | 24 kB 00:00 2024-09-16T07:20:17.933 INFO:teuthology.orchestra.run.smithi177.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-16T07:20:17.991 INFO:teuthology.orchestra.run.smithi177.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-16T07:20:18.042 INFO:teuthology.orchestra.run.smithi177.stdout:(65/157): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-16T07:20:18.075 INFO:teuthology.orchestra.run.smithi177.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-16T07:20:18.081 INFO:teuthology.orchestra.run.smithi071.stdout:(34/157): ceph-mgr-dashboard-19.3.0-4938.g4ef5f 15 MB/s | 14 MB 00:00 2024-09-16T07:20:18.106 INFO:teuthology.orchestra.run.smithi071.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4938.g4 79 kB/s | 16 kB 00:00 2024-09-16T07:20:18.131 INFO:teuthology.orchestra.run.smithi071.stdout:(36/157): ceph-volume-19.3.0-4938.g4ef5f058.el9 5.6 MB/s | 285 kB 00:00 2024-09-16T07:20:18.542 INFO:teuthology.orchestra.run.smithi177.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:00 2024-09-16T07:20:18.713 INFO:teuthology.orchestra.run.smithi071.stdout:(37/157): ceph-test-19.3.0-4938.g4ef5f058.el9.x 15 MB/s | 46 MB 00:03 2024-09-16T07:20:18.740 INFO:teuthology.orchestra.run.smithi071.stdout:(38/157): cephfs-top-19.3.0-4938.g4ef5f058.el9. 30 kB/s | 18 kB 00:00 2024-09-16T07:20:18.774 INFO:teuthology.orchestra.run.smithi071.stdout:(39/157): cephadm-19.3.0-4938.g4ef5f058.el9.noa 1.1 MB/s | 784 kB 00:00 2024-09-16T07:20:19.326 INFO:teuthology.orchestra.run.smithi177.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-16T07:20:19.357 INFO:teuthology.orchestra.run.smithi071.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 124 kB/s | 80 kB 00:00 2024-09-16T07:20:19.583 INFO:teuthology.orchestra.run.smithi071.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 236 kB/s | 191 kB 00:00 2024-09-16T07:20:19.716 INFO:teuthology.orchestra.run.smithi071.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 821 kB/s | 801 kB 00:00 2024-09-16T07:20:19.801 INFO:teuthology.orchestra.run.smithi177.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 1.9 MB/s | 910 kB 00:00 2024-09-16T07:20:19.918 INFO:teuthology.orchestra.run.smithi177.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-16T07:20:20.102 INFO:teuthology.orchestra.run.smithi177.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 2.4 MB/s | 4.9 MB 00:02 2024-09-16T07:20:20.117 INFO:teuthology.orchestra.run.smithi071.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 334 kB/s | 253 kB 00:00 2024-09-16T07:20:20.302 INFO:teuthology.orchestra.run.smithi177.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-09-16T07:20:20.376 INFO:teuthology.orchestra.run.smithi071.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 162 kB/s | 106 kB 00:00 2024-09-16T07:20:20.385 INFO:teuthology.orchestra.run.smithi177.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-16T07:20:20.534 INFO:teuthology.orchestra.run.smithi071.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-16T07:20:20.677 INFO:teuthology.orchestra.run.smithi177.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-09-16T07:20:20.736 INFO:teuthology.orchestra.run.smithi177.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-16T07:20:20.878 INFO:teuthology.orchestra.run.smithi177.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-16T07:20:20.917 INFO:teuthology.orchestra.run.smithi071.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 169 kB/s | 135 kB 00:00 2024-09-16T07:20:21.045 INFO:teuthology.orchestra.run.smithi177.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-16T07:20:21.092 INFO:teuthology.orchestra.run.smithi071.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 177 kB/s | 125 kB 00:00 2024-09-16T07:20:21.095 INFO:teuthology.orchestra.run.smithi177.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-09-16T07:20:21.145 INFO:teuthology.orchestra.run.smithi177.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 385 kB/s | 19 kB 00:00 2024-09-16T07:20:21.271 INFO:teuthology.orchestra.run.smithi177.stdout:(80/157): postgresql-server-13.16-1.el9.x86_64. 4.3 MB/s | 5.8 MB 00:01 2024-09-16T07:20:21.301 INFO:teuthology.orchestra.run.smithi071.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 280 kB/s | 215 kB 00:00 2024-09-16T07:20:21.363 INFO:teuthology.orchestra.run.smithi177.stdout:(81/157): python3-babel-2.9.1-2.el9.noarch.rpm 6.1 MB/s | 6.0 MB 00:00 2024-09-16T07:20:21.388 INFO:teuthology.orchestra.run.smithi177.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 712 kB/s | 172 kB 00:00 2024-09-16T07:20:21.438 INFO:teuthology.orchestra.run.smithi177.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 209 kB/s | 35 kB 00:00 2024-09-16T07:20:21.503 INFO:teuthology.orchestra.run.smithi177.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.8 MB/s | 250 kB 00:00 2024-09-16T07:20:21.572 INFO:teuthology.orchestra.run.smithi177.stdout:(85/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-09-16T07:20:21.597 INFO:teuthology.orchestra.run.smithi177.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 823 kB/s | 77 kB 00:00 2024-09-16T07:20:21.601 INFO:teuthology.orchestra.run.smithi071.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-09-16T07:20:21.622 INFO:teuthology.orchestra.run.smithi177.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-16T07:20:21.656 INFO:teuthology.orchestra.run.smithi177.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-16T07:20:21.681 INFO:teuthology.orchestra.run.smithi177.stdout:(89/157): python3-oauthlib-3.1.1-5.el9.noarch.r 915 kB/s | 222 kB 00:00 2024-09-16T07:20:21.706 INFO:teuthology.orchestra.run.smithi177.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-16T07:20:21.718 INFO:teuthology.orchestra.run.smithi071.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-16T07:20:21.732 INFO:teuthology.orchestra.run.smithi177.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-16T07:20:21.757 INFO:teuthology.orchestra.run.smithi177.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-09-16T07:20:21.768 INFO:teuthology.orchestra.run.smithi071.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 229 kB/s | 107 kB 00:00 2024-09-16T07:20:21.782 INFO:teuthology.orchestra.run.smithi177.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 713 kB/s | 54 kB 00:00 2024-09-16T07:20:21.827 INFO:teuthology.orchestra.run.smithi071.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-16T07:20:21.841 INFO:teuthology.orchestra.run.smithi177.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-09-16T07:20:21.852 INFO:teuthology.orchestra.run.smithi071.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-16T07:20:21.874 INFO:teuthology.orchestra.run.smithi177.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 480 kB/s | 44 kB 00:00 2024-09-16T07:20:21.878 INFO:teuthology.orchestra.run.smithi071.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 300 kB/s | 33 kB 00:00 2024-09-16T07:20:21.925 INFO:teuthology.orchestra.run.smithi177.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 724 kB/s | 36 kB 00:00 2024-09-16T07:20:21.975 INFO:teuthology.orchestra.run.smithi177.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-09-16T07:20:21.978 INFO:teuthology.orchestra.run.smithi071.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-09-16T07:20:22.000 INFO:teuthology.orchestra.run.smithi177.stdout:(98/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-16T07:20:22.003 INFO:teuthology.orchestra.run.smithi071.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-09-16T07:20:22.042 INFO:teuthology.orchestra.run.smithi177.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.5 MB/s | 304 kB 00:00 2024-09-16T07:20:22.067 INFO:teuthology.orchestra.run.smithi177.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-09-16T07:20:22.087 INFO:teuthology.orchestra.run.smithi071.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-16T07:20:22.101 INFO:teuthology.orchestra.run.smithi177.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-16T07:20:22.154 INFO:teuthology.orchestra.run.smithi071.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.2 MB/s | 3.0 MB 00:00 2024-09-16T07:20:22.196 INFO:teuthology.orchestra.run.smithi071.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-16T07:20:22.221 INFO:teuthology.orchestra.run.smithi071.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 737 kB/s | 160 kB 00:00 2024-09-16T07:20:22.280 INFO:teuthology.orchestra.run.smithi071.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-16T07:20:22.301 INFO:teuthology.orchestra.run.smithi177.stdout:(102/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-16T07:20:22.327 INFO:teuthology.orchestra.run.smithi177.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 246 kB/s | 64 kB 00:00 2024-09-16T07:20:22.346 INFO:teuthology.orchestra.run.smithi071.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-16T07:20:22.352 INFO:teuthology.orchestra.run.smithi177.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 179 kB/s | 45 kB 00:00 2024-09-16T07:20:22.372 INFO:teuthology.orchestra.run.smithi071.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 163 kB/s | 24 kB 00:00 2024-09-16T07:20:22.397 INFO:teuthology.orchestra.run.smithi071.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 404 kB/s | 47 kB 00:00 2024-09-16T07:20:22.402 INFO:teuthology.orchestra.run.smithi177.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T07:20:22.436 INFO:teuthology.orchestra.run.smithi177.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-16T07:20:22.461 INFO:teuthology.orchestra.run.smithi177.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-16T07:20:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-16T07:20:22.498 INFO:teuthology.orchestra.run.smithi071.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-16T07:20:22.523 INFO:teuthology.orchestra.run.smithi071.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 307 kB/s | 38 kB 00:00 2024-09-16T07:20:22.570 INFO:teuthology.orchestra.run.smithi177.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-16T07:20:22.611 INFO:teuthology.orchestra.run.smithi177.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.2 MB/s | 26 kB 00:00 2024-09-16T07:20:22.637 INFO:teuthology.orchestra.run.smithi177.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T07:20:22.665 INFO:teuthology.orchestra.run.smithi071.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-16T07:20:22.685 INFO:teuthology.orchestra.run.smithi177.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-16T07:20:22.729 INFO:teuthology.orchestra.run.smithi177.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-16T07:20:22.763 INFO:teuthology.orchestra.run.smithi177.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 25 MB/s | 839 kB 00:00 2024-09-16T07:20:22.807 INFO:teuthology.orchestra.run.smithi071.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-16T07:20:22.813 INFO:teuthology.orchestra.run.smithi177.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-16T07:20:22.838 INFO:teuthology.orchestra.run.smithi177.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T07:20:22.864 INFO:teuthology.orchestra.run.smithi177.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T07:20:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:(70/157): openblas-openmp-0.3.26-2.el9.x86_64.r 12 MB/s | 4.9 MB 00:00 2024-09-16T07:20:23.059 INFO:teuthology.orchestra.run.smithi071.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-16T07:20:23.060 INFO:teuthology.orchestra.run.smithi177.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-16T07:20:23.060 INFO:teuthology.orchestra.run.smithi177.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-16T07:20:23.060 INFO:teuthology.orchestra.run.smithi177.stdout:(119/157): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-09-16T07:20:23.060 INFO:teuthology.orchestra.run.smithi177.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-09-16T07:20:23.060 INFO:teuthology.orchestra.run.smithi177.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 6.8 MB/s | 220 kB 00:00 2024-09-16T07:20:23.060 INFO:teuthology.orchestra.run.smithi177.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-16T07:20:23.083 INFO:teuthology.orchestra.run.smithi071.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-09-16T07:20:23.107 INFO:teuthology.orchestra.run.smithi177.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-16T07:20:23.133 INFO:teuthology.orchestra.run.smithi177.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 425 kB/s | 11 kB 00:00 2024-09-16T07:20:23.145 INFO:teuthology.orchestra.run.smithi071.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-09-16T07:20:23.158 INFO:teuthology.orchestra.run.smithi177.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-16T07:20:23.183 INFO:teuthology.orchestra.run.smithi177.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-16T07:20:23.208 INFO:teuthology.orchestra.run.smithi177.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 786 kB/s | 19 kB 00:00 2024-09-16T07:20:23.233 INFO:teuthology.orchestra.run.smithi071.stdout:(74/157): postgresql-server-13.16-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-09-16T07:20:23.233 INFO:teuthology.orchestra.run.smithi177.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 798 kB/s | 20 kB 00:00 2024-09-16T07:20:23.258 INFO:teuthology.orchestra.run.smithi071.stdout:(75/157): python3-attrs-20.3.0-7.el9.noarch.rpm 501 kB/s | 87 kB 00:00 2024-09-16T07:20:23.284 INFO:teuthology.orchestra.run.smithi177.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-09-16T07:20:23.309 INFO:teuthology.orchestra.run.smithi177.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T07:20:23.341 INFO:teuthology.orchestra.run.smithi071.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-09-16T07:20:23.343 INFO:teuthology.orchestra.run.smithi177.stdout:(131/157): protobuf-compiler-3.14.0-13.el9.x86_ 850 kB/s | 863 kB 00:01 2024-09-16T07:20:23.368 INFO:teuthology.orchestra.run.smithi177.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-16T07:20:23.375 INFO:teuthology.orchestra.run.smithi071.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-16T07:20:23.418 INFO:teuthology.orchestra.run.smithi177.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-16T07:20:23.443 INFO:teuthology.orchestra.run.smithi177.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.1 MB/s | 272 kB 00:00 2024-09-16T07:20:23.459 INFO:teuthology.orchestra.run.smithi071.stdout:(78/157): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-09-16T07:20:23.484 INFO:teuthology.orchestra.run.smithi071.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-16T07:20:23.485 INFO:teuthology.orchestra.run.smithi177.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 252 kB/s | 16 kB 00:00 2024-09-16T07:20:23.509 INFO:teuthology.orchestra.run.smithi071.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-09-16T07:20:23.511 INFO:teuthology.orchestra.run.smithi177.stdout:(136/157): python3-natsort-7.1.1-5.el9.noarch.r 344 kB/s | 58 kB 00:00 2024-09-16T07:20:23.536 INFO:teuthology.orchestra.run.smithi177.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-16T07:20:23.562 INFO:teuthology.orchestra.run.smithi177.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 765 kB/s | 90 kB 00:00 2024-09-16T07:20:23.576 INFO:teuthology.orchestra.run.smithi071.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-09-16T07:20:23.578 INFO:teuthology.orchestra.run.smithi177.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-09-16T07:20:23.595 INFO:teuthology.orchestra.run.smithi177.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-16T07:20:23.610 INFO:teuthology.orchestra.run.smithi071.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-16T07:20:23.612 INFO:teuthology.orchestra.run.smithi177.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-16T07:20:23.635 INFO:teuthology.orchestra.run.smithi071.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 276 kB/s | 35 kB 00:00 2024-09-16T07:20:23.637 INFO:teuthology.orchestra.run.smithi177.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-16T07:20:23.663 INFO:teuthology.orchestra.run.smithi177.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-16T07:20:23.688 INFO:teuthology.orchestra.run.smithi177.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-16T07:20:23.713 INFO:teuthology.orchestra.run.smithi177.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-16T07:20:23.727 INFO:teuthology.orchestra.run.smithi071.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-16T07:20:23.739 INFO:teuthology.orchestra.run.smithi177.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-16T07:20:23.752 INFO:teuthology.orchestra.run.smithi071.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-16T07:20:23.764 INFO:teuthology.orchestra.run.smithi177.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T07:20:23.789 INFO:teuthology.orchestra.run.smithi177.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-16T07:20:23.836 INFO:teuthology.orchestra.run.smithi071.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-09-16T07:20:23.839 INFO:teuthology.orchestra.run.smithi177.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T07:20:23.861 INFO:teuthology.orchestra.run.smithi071.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 576 kB/s | 77 kB 00:00 2024-09-16T07:20:23.886 INFO:teuthology.orchestra.run.smithi071.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 315 kB/s | 42 kB 00:00 2024-09-16T07:20:23.965 INFO:teuthology.orchestra.run.smithi177.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 19 MB/s | 2.3 MB 00:00 2024-09-16T07:20:23.978 INFO:teuthology.orchestra.run.smithi071.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-09-16T07:20:24.003 INFO:teuthology.orchestra.run.smithi071.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-16T07:20:24.046 INFO:teuthology.orchestra.run.smithi071.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-09-16T07:20:24.073 INFO:teuthology.orchestra.run.smithi177.stdout:(151/157): librados2-19.3.0-4938.g4ef5f058.el9. 11 MB/s | 3.5 MB 00:00 2024-09-16T07:20:24.088 INFO:teuthology.orchestra.run.smithi071.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-16T07:20:24.113 INFO:teuthology.orchestra.run.smithi071.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 491 kB/s | 54 kB 00:00 2024-09-16T07:20:24.123 INFO:teuthology.orchestra.run.smithi177.stdout:(152/157): NetworkManager-libnm-1.49.90-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-09-16T07:20:24.166 INFO:teuthology.orchestra.run.smithi177.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 445 kB/s | 41 kB 00:00 2024-09-16T07:20:24.197 INFO:teuthology.orchestra.run.smithi071.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 382 kB/s | 42 kB 00:00 2024-09-16T07:20:24.225 INFO:teuthology.orchestra.run.smithi071.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-09-16T07:20:24.249 INFO:teuthology.orchestra.run.smithi177.stdout:(154/157): librbd1-19.3.0-4938.g4ef5f058.el9.x8 7.1 MB/s | 3.2 MB 00:00 2024-09-16T07:20:24.299 INFO:teuthology.orchestra.run.smithi177.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 1.4 MB/s | 249 kB 00:00 2024-09-16T07:20:24.314 INFO:teuthology.orchestra.run.smithi071.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-16T07:20:24.341 INFO:teuthology.orchestra.run.smithi177.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 7.7 MB/s | 1.4 MB 00:00 2024-09-16T07:20:24.370 INFO:teuthology.orchestra.run.smithi071.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 243 kB/s | 36 kB 00:00 2024-09-16T07:20:24.391 INFO:teuthology.orchestra.run.smithi177.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-16T07:20:24.397 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-16T07:20:24.397 INFO:teuthology.orchestra.run.smithi177.stdout:Total 18 MB/s | 235 MB 00:12 2024-09-16T07:20:24.431 INFO:teuthology.orchestra.run.smithi071.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-09-16T07:20:24.481 INFO:teuthology.orchestra.run.smithi071.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-16T07:20:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-16T07:20:24.598 INFO:teuthology.orchestra.run.smithi071.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 821 kB/s | 96 kB 00:00 2024-09-16T07:20:24.799 INFO:teuthology.orchestra.run.smithi071.stdout:(102/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-16T07:20:24.824 INFO:teuthology.orchestra.run.smithi071.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 199 kB/s | 45 kB 00:00 2024-09-16T07:20:24.849 INFO:teuthology.orchestra.run.smithi071.stdout:(104/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 206 kB/s | 64 kB 00:00 2024-09-16T07:20:24.908 INFO:teuthology.orchestra.run.smithi071.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-16T07:20:24.941 INFO:teuthology.orchestra.run.smithi071.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-16T07:20:24.966 INFO:teuthology.orchestra.run.smithi071.stdout:(107/157): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-09-16T07:20:24.992 INFO:teuthology.orchestra.run.smithi071.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-09-16T07:20:25.025 INFO:teuthology.orchestra.run.smithi071.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-16T07:20:25.051 INFO:teuthology.orchestra.run.smithi071.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-16T07:20:25.076 INFO:teuthology.orchestra.run.smithi071.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T07:20:25.109 INFO:teuthology.orchestra.run.smithi071.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-16T07:20:25.152 INFO:teuthology.orchestra.run.smithi071.stdout:(113/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-09-16T07:20:25.210 INFO:teuthology.orchestra.run.smithi071.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-16T07:20:25.227 INFO:teuthology.orchestra.run.smithi071.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-16T07:20:25.244 INFO:teuthology.orchestra.run.smithi071.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-16T07:20:25.261 INFO:teuthology.orchestra.run.smithi071.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 856 kB/s | 14 kB 00:00 2024-09-16T07:20:25.294 INFO:teuthology.orchestra.run.smithi071.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-16T07:20:25.320 INFO:teuthology.orchestra.run.smithi071.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-09-16T07:20:25.353 INFO:teuthology.orchestra.run.smithi071.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-16T07:20:25.379 INFO:teuthology.orchestra.run.smithi071.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-16T07:20:25.404 INFO:teuthology.orchestra.run.smithi071.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T07:20:25.438 INFO:teuthology.orchestra.run.smithi071.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-16T07:20:25.463 INFO:teuthology.orchestra.run.smithi071.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-16T07:20:25.488 INFO:teuthology.orchestra.run.smithi071.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-16T07:20:25.513 INFO:teuthology.orchestra.run.smithi071.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-09-16T07:20:25.539 INFO:teuthology.orchestra.run.smithi071.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-16T07:20:25.594 INFO:teuthology.orchestra.run.smithi071.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 356 kB/s | 20 kB 00:00 2024-09-16T07:20:25.656 INFO:teuthology.orchestra.run.smithi071.stdout:(129/157): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-09-16T07:20:25.681 INFO:teuthology.orchestra.run.smithi071.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T07:20:25.740 INFO:teuthology.orchestra.run.smithi071.stdout:(131/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-09-16T07:20:25.782 INFO:teuthology.orchestra.run.smithi071.stdout:(132/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.1 MB/s | 2.0 MB 00:00 2024-09-16T07:20:25.807 INFO:teuthology.orchestra.run.smithi071.stdout:(133/157): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-09-16T07:20:25.832 INFO:teuthology.orchestra.run.smithi071.stdout:(134/157): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-09-16T07:20:25.857 INFO:teuthology.orchestra.run.smithi071.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 330 kB/s | 16 kB 00:00 2024-09-16T07:20:25.891 INFO:teuthology.orchestra.run.smithi071.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 848 kB/s | 28 kB 00:00 2024-09-16T07:20:25.916 INFO:teuthology.orchestra.run.smithi071.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-16T07:20:25.942 INFO:teuthology.orchestra.run.smithi071.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-16T07:20:25.967 INFO:teuthology.orchestra.run.smithi071.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T07:20:26.017 INFO:teuthology.orchestra.run.smithi071.stdout:(140/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-16T07:20:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-09-16T07:20:26.068 INFO:teuthology.orchestra.run.smithi071.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-16T07:20:26.093 INFO:teuthology.orchestra.run.smithi071.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 984 kB/s | 74 kB 00:00 2024-09-16T07:20:26.118 INFO:teuthology.orchestra.run.smithi071.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-16T07:20:26.143 INFO:teuthology.orchestra.run.smithi071.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 807 kB/s | 20 kB 00:00 2024-09-16T07:20:26.169 INFO:teuthology.orchestra.run.smithi071.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-16T07:20:26.211 INFO:teuthology.orchestra.run.smithi071.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-16T07:20:26.236 INFO:teuthology.orchestra.run.smithi071.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-16T07:20:26.345 INFO:teuthology.orchestra.run.smithi071.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-09-16T07:20:26.456 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-16T07:20:26.503 INFO:teuthology.orchestra.run.smithi071.stdout:(150/157): librbd1-19.3.0-4938.g4ef5f058.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-16T07:20:26.562 INFO:teuthology.orchestra.run.smithi071.stdout:(151/157): librados2-19.3.0-4938.g4ef5f058.el9. 9.9 MB/s | 3.5 MB 00:00 2024-09-16T07:20:26.639 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-16T07:20:26.639 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-16T07:20:27.121 INFO:teuthology.orchestra.run.smithi071.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 73 kB/s | 41 kB 00:00 2024-09-16T07:20:27.480 INFO:teuthology.orchestra.run.smithi071.stdout:(153/157): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:00 2024-09-16T07:20:27.638 INFO:teuthology.orchestra.run.smithi071.stdout:(154/157): NetworkManager-1.49.90-1.el9.x86_64. 1.8 MB/s | 2.3 MB 00:01 2024-09-16T07:20:27.888 INFO:teuthology.orchestra.run.smithi071.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 324 kB/s | 249 kB 00:00 2024-09-16T07:20:28.513 INFO:teuthology.orchestra.run.smithi071.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 1.3 MB/s | 1.4 MB 00:01 2024-09-16T07:20:29.080 INFO:teuthology.orchestra.run.smithi071.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 1.7 MB/s | 2.4 MB 00:01 2024-09-16T07:20:29.087 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-16T07:20:29.088 INFO:teuthology.orchestra.run.smithi071.stdout:Total 14 MB/s | 235 MB 00:16 2024-09-16T07:20:29.091 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-16T07:20:29.092 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-16T07:20:31.146 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-16T07:20:31.326 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-16T07:20:31.327 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-16T07:20:33.779 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-16T07:20:33.779 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-16T07:20:36.095 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-16T07:20:36.280 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-16T07:20:36.331 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-16T07:20:36.396 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-16T07:20:36.440 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-16T07:20:36.472 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-16T07:20:36.533 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-16T07:20:36.918 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-16T07:20:37.055 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:19.3.0-4938.g4ef5f058.el9.x86_64 8/165 2024-09-16T07:20:37.207 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:19.3.0-4938.g4ef5f058.el9.x86_64 8/165 2024-09-16T07:20:37.239 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:19.3.0-4938.g4ef5f058.el9.x86_64 9/165 2024-09-16T07:20:37.352 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:19.3.0-4938.g4ef5f058.el9.x86_64 9/165 2024-09-16T07:20:37.387 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:19.3.0-4938.g4ef5f058.el9.x86_64 10/165 2024-09-16T07:20:37.429 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-16T07:20:37.645 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-16T07:20:37.701 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-16T07:20:37.728 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:19.3.0-4938.g4ef5f058.el9.x86_64 14/165 2024-09-16T07:20:37.910 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4938.g4ef5f058.el9.x86_64 14/165 2024-09-16T07:20:37.937 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:19.3.0-4938.g4ef5f058.el9.x86 15/165 2024-09-16T07:20:38.013 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4938.g4ef5f058.el9.x86 15/165 2024-09-16T07:20:38.038 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T07:20:38.079 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T07:20:38.331 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T07:20:38.519 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T07:20:38.702 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-16T07:20:38.813 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T07:20:38.929 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T07:20:39.197 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T07:20:39.335 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-16T07:20:39.368 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-16T07:20:39.423 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-16T07:20:39.845 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-16T07:20:39.907 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-16T07:20:39.954 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-16T07:20:40.026 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-16T07:20:40.073 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-16T07:20:40.105 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-16T07:20:40.194 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-16T07:20:40.257 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-16T07:20:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-16T07:20:40.355 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-16T07:20:40.476 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-16T07:20:40.521 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:19.3.0-4938.g4ef5f058.el9. 34/165 2024-09-16T07:20:40.564 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:19.3.0-4938.g4ef5f058.el 35/165 2024-09-16T07:20:40.593 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:19.3.0-4938.g4ef5f058.el9.x86_6 36/165 2024-09-16T07:20:40.674 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-16T07:20:40.729 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-16T07:20:40.784 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-16T07:20:40.819 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-16T07:20:40.984 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-16T07:20:40.986 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-16T07:20:41.039 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-16T07:20:41.047 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-16T07:20:41.108 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-16T07:20:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-16T07:20:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-16T07:20:41.258 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-16T07:20:41.633 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-16T07:20:41.680 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-16T07:20:41.776 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.3.0-4938.g4ef5f058.el9.x86_64 8/165 2024-09-16T07:20:41.865 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-16T07:20:41.913 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-16T07:20:41.930 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.3.0-4938.g4ef5f058.el9.x86_64 8/165 2024-09-16T07:20:41.961 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.3.0-4938.g4ef5f058.el9.x86_64 9/165 2024-09-16T07:20:41.961 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-16T07:20:42.000 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:19.3.0-4938.g4ef5f058.el9.x86_6 46/165 2024-09-16T07:20:42.054 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-16T07:20:42.073 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.3.0-4938.g4ef5f058.el9.x86_64 9/165 2024-09-16T07:20:42.108 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.3.0-4938.g4ef5f058.el9.x86_64 10/165 2024-09-16T07:20:42.148 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-16T07:20:42.257 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-16T07:20:42.311 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-16T07:20:42.377 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-16T07:20:42.417 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-16T07:20:42.439 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-16T07:20:42.466 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.3.0-4938.g4ef5f058.el9.x86_64 14/165 2024-09-16T07:20:42.515 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-16T07:20:42.596 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4938.g4ef5f058.el9.x86_64 14/165 2024-09-16T07:20:42.597 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-16T07:20:42.633 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.3.0-4938.g4ef5f058.el9.x86 15/165 2024-09-16T07:20:42.646 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-16T07:20:42.686 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-16T07:20:42.716 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4938.g4ef5f058.el9.x86 15/165 2024-09-16T07:20:42.743 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T07:20:42.744 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-16T07:20:42.782 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T07:20:42.785 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-16T07:20:42.831 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-16T07:20:42.902 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-16T07:20:43.045 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T07:20:43.076 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-16T07:20:43.130 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-16T07:20:43.174 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-16T07:20:43.216 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T07:20:43.291 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-16T07:20:43.331 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-16T07:20:43.373 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-16T07:20:43.411 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-16T07:20:43.427 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-16T07:20:43.436 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-16T07:20:43.548 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-16T07:20:43.555 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T07:20:43.651 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T07:20:43.703 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-16T07:20:43.744 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-16T07:20:43.786 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-16T07:20:43.828 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-16T07:20:43.872 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-16T07:20:43.911 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-16T07:20:43.924 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T07:20:43.948 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-16T07:20:43.983 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-16T07:20:44.016 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-16T07:20:44.060 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-16T07:20:44.070 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-16T07:20:44.106 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-16T07:20:44.118 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-16T07:20:44.151 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-16T07:20:44.217 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-16T07:20:44.271 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-16T07:20:44.275 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-16T07:20:44.313 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-16T07:20:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-16T07:20:44.376 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-16T07:20:44.433 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-16T07:20:44.438 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-16T07:20:44.477 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-16T07:20:44.511 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-16T07:20:44.533 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-16T07:20:44.599 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-16T07:20:44.672 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-16T07:20:44.706 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-16T07:20:44.752 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-16T07:20:44.846 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-16T07:20:44.893 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.3.0-4938.g4ef5f058.el9. 34/165 2024-09-16T07:20:44.936 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.3.0-4938.g4ef5f058.el 35/165 2024-09-16T07:20:44.965 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.3.0-4938.g4ef5f058.el9.x86_6 36/165 2024-09-16T07:20:45.080 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-16T07:20:45.134 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-16T07:20:45.229 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-16T07:20:45.389 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-16T07:20:45.418 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-16T07:20:45.473 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-16T07:20:45.477 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-16T07:20:45.511 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-16T07:20:45.627 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-16T07:20:45.710 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-16T07:20:45.772 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-16T07:20:45.869 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-16T07:20:45.914 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-16T07:20:46.114 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-16T07:20:46.147 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-16T07:20:46.185 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-16T07:20:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-16T07:20:46.279 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-16T07:20:46.486 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-16T07:20:46.538 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-16T07:20:46.577 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.3.0-4938.g4ef5f058.el9.x86_6 46/165 2024-09-16T07:20:46.654 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-16T07:20:46.870 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-16T07:20:46.945 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-16T07:20:47.049 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-16T07:20:47.094 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-16T07:20:47.123 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-16T07:20:47.215 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-16T07:20:47.232 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-16T07:20:47.263 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-16T07:20:47.300 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-16T07:20:47.358 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-16T07:20:47.399 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-16T07:20:47.477 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-16T07:20:47.527 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-16T07:20:47.700 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-16T07:20:47.760 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-16T07:20:47.803 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-16T07:20:47.954 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-16T07:20:47.994 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-16T07:20:48.037 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-16T07:20:48.066 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-16T07:20:48.107 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-16T07:20:48.222 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-16T07:20:48.383 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-16T07:20:48.431 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-16T07:20:48.468 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-16T07:20:48.512 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-16T07:20:48.568 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-16T07:20:48.607 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-16T07:20:48.661 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-16T07:20:48.696 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-16T07:20:48.729 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-16T07:20:48.772 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-16T07:20:48.786 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-16T07:20:48.805 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-16T07:20:48.882 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-16T07:20:48.907 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-16T07:20:48.909 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-16T07:20:48.934 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-16T07:20:48.971 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-16T07:20:49.009 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-16T07:20:49.155 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-16T07:20:49.213 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-16T07:20:49.290 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-16T07:20:49.329 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:19.3.0-4938.g4ef5f058.el9.x86_64 102/165 2024-09-16T07:20:49.441 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:19.3.0-4938.g4ef5f058.el9.x86_64 102/165 2024-09-16T07:20:49.477 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 103/165 2024-09-16T07:20:49.525 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-16T07:20:50.052 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-16T07:20:50.090 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:19.3.0-4938.g4ef5f058.el9.x86_64 106/165 2024-09-16T07:20:50.105 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-16T07:20:50.156 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-16T07:20:50.181 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:19.3.0-4938.g4ef5f058.el9.x86_64 106/165 2024-09-16T07:20:50.198 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-16T07:20:50.208 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 107/165 2024-09-16T07:20:50.348 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-16T07:20:50.423 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-16T07:20:50.476 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-16T07:20:50.572 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-16T07:20:50.609 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-16T07:20:50.866 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-16T07:20:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-16T07:20:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-16T07:20:51.822 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-16T07:20:51.968 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-16T07:20:52.751 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 108/165 2024-09-16T07:20:52.813 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 108/165 2024-09-16T07:20:53.465 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 108/165 2024-09-16T07:20:53.500 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:19.3.0-4938.g4ef5f058.el9.x86_64 109/165 2024-09-16T07:20:53.523 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-16T07:20:53.628 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-16T07:20:53.658 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:19.3.0-4938.g4ef5f058.el9.x86_64 109/165 2024-09-16T07:20:53.659 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T07:20:53.659 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T07:20:53.659 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:20:53.664 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-16T07:20:53.697 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-16T07:20:53.701 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:19.3.0-4938.g4ef5f058.el9.x86_64 110/165 2024-09-16T07:20:54.050 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-16T07:20:54.100 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.3.0-4938.g4ef5f058.el9.x86_64 102/165 2024-09-16T07:20:54.213 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.3.0-4938.g4ef5f058.el9.x86_64 102/165 2024-09-16T07:20:54.277 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 103/165 2024-09-16T07:20:54.329 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-16T07:20:54.883 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-16T07:20:54.919 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.3.0-4938.g4ef5f058.el9.x86_64 106/165 2024-09-16T07:20:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.3.0-4938.g4ef5f058.el9.x86_64 106/165 2024-09-16T07:20:55.024 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 107/165 2024-09-16T07:20:57.605 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 108/165 2024-09-16T07:20:57.642 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 108/165 2024-09-16T07:20:58.300 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 108/165 2024-09-16T07:20:58.345 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.3.0-4938.g4ef5f058.el9.x86_64 109/165 2024-09-16T07:20:58.484 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.3.0-4938.g4ef5f058.el9.x86_64 109/165 2024-09-16T07:20:58.484 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T07:20:58.484 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T07:20:58.484 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:20:58.530 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.3.0-4938.g4ef5f058.el9.x86_64 110/165 2024-09-16T07:21:06.818 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4938.g4ef5f058.el9.x86_64 110/165 2024-09-16T07:21:06.818 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-09-16T07:21:06.818 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-09-16T07:21:06.818 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-09-16T07:21:06.818 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-09-16T07:21:06.818 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-09-16T07:21:06.819 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-09-16T07:21:06.819 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-09-16T07:21:06.819 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:07.032 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:19.3.0-4938.g4ef5f058.el9.noarch 111/165 2024-09-16T07:21:07.068 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:19.3.0-4938.g4ef5f058.el9.noarch 111/165 2024-09-16T07:21:07.068 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:07.068 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T07:21:07.068 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:08.675 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:19.3.0-4938.g4ef5f058.el9.x86_64 112/165 2024-09-16T07:21:08.732 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:19.3.0-4938.g4ef5f058.el9.x86_64 112/165 2024-09-16T07:21:08.732 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:08.732 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T07:21:08.732 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T07:21:08.733 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T07:21:08.733 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:08.984 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:19.3.0-4938.g4ef5f058.el9.x86_64 113/165 2024-09-16T07:21:09.034 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:19.3.0-4938.g4ef5f058.el9.x86_64 113/165 2024-09-16T07:21:09.034 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:09.034 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T07:21:09.034 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T07:21:09.034 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T07:21:09.035 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:09.518 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:19.3.0-4938.g4ef5f058.el9.x86_64 114/165 2024-09-16T07:21:09.568 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:19.3.0-4938.g4ef5f058.el9.x86_64 114/165 2024-09-16T07:21:09.568 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:09.569 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T07:21:09.569 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T07:21:09.569 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T07:21:09.569 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:09.701 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-16T07:21:09.761 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-16T07:21:09.826 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-16T07:21:10.051 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-16T07:21:10.114 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-16T07:21:10.199 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-16T07:21:10.307 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-16T07:21:10.465 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-16T07:21:10.513 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-16T07:21:10.553 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-16T07:21:10.579 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-16T07:21:11.110 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4938.g4ef5f058.el9.x86_64 110/165 2024-09-16T07:21:11.110 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-09-16T07:21:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-09-16T07:21:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-09-16T07:21:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-09-16T07:21:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-09-16T07:21:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-09-16T07:21:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-09-16T07:21:11.111 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:11.137 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 126/165 2024-09-16T07:21:11.212 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 126/165 2024-09-16T07:21:11.278 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4938.g4ef5f058.e 127/165 2024-09-16T07:21:11.323 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.3.0-4938.g4ef5f058.el9.noarch 111/165 2024-09-16T07:21:11.359 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.3.0-4938.g4ef5f058.el9.noarch 111/165 2024-09-16T07:21:11.359 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:11.359 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T07:21:11.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:11.360 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4938.g4ef5f058. 128/165 2024-09-16T07:21:11.392 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4938.g4ef5f058.el9.noa 129/165 2024-09-16T07:21:12.967 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.3.0-4938.g4ef5f058.el9.x86_64 112/165 2024-09-16T07:21:13.026 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.3.0-4938.g4ef5f058.el9.x86_64 112/165 2024-09-16T07:21:13.026 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:13.026 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T07:21:13.026 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T07:21:13.026 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T07:21:13.026 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:13.301 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.3.0-4938.g4ef5f058.el9.x86_64 113/165 2024-09-16T07:21:13.352 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.3.0-4938.g4ef5f058.el9.x86_64 113/165 2024-09-16T07:21:13.353 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:13.353 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T07:21:13.353 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T07:21:13.353 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T07:21:13.353 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:13.852 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.3.0-4938.g4ef5f058.el9.x86_64 114/165 2024-09-16T07:21:13.902 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.3.0-4938.g4ef5f058.el9.x86_64 114/165 2024-09-16T07:21:13.903 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:13.903 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T07:21:13.903 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T07:21:13.903 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T07:21:13.903 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:14.026 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-16T07:21:14.112 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-16T07:21:14.177 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-16T07:21:14.413 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-16T07:21:14.490 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-16T07:21:14.584 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-16T07:21:14.701 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-16T07:21:14.869 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-16T07:21:14.921 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-16T07:21:14.970 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-16T07:21:15.001 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4938.g4ef5f058.el9.noa 129/165 2024-09-16T07:21:15.003 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-16T07:21:15.065 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4938.g4ef5f058.el9.n 130/165 2024-09-16T07:21:15.453 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 126/165 2024-09-16T07:21:15.486 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 126/165 2024-09-16T07:21:15.528 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4938.g4ef5f058.e 127/165 2024-09-16T07:21:15.610 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4938.g4ef5f058. 128/165 2024-09-16T07:21:15.650 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4938.g4ef5f058.el9.noa 129/165 2024-09-16T07:21:16.145 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4938.g4ef5f058.el9.n 130/165 2024-09-16T07:21:16.172 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4938.g4ef 131/165 2024-09-16T07:21:16.324 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4938.g4ef 131/165 2024-09-16T07:21:16.514 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4938.g4ef5f058.el 132/165 2024-09-16T07:21:16.550 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:19.3.0-4938.g4ef5f058.el9.x86_64 133/165 2024-09-16T07:21:16.600 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4938.g4ef5f058.el9.x86_64 133/165 2024-09-16T07:21:16.600 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:16.600 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T07:21:16.600 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T07:21:16.601 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T07:21:16.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:16.659 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:19.3.0-4938.g4ef5f058.el9.noarch 134/165 2024-09-16T07:21:16.685 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4938.g4ef5f058.el9.noarch 134/165 2024-09-16T07:21:16.792 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:19.3.0-4938.g4ef5f058.el9.x86_64 135/165 2024-09-16T07:21:16.874 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 136/165 2024-09-16T07:21:17.900 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-16T07:21:17.945 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 138/165 2024-09-16T07:21:17.995 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 138/165 2024-09-16T07:21:17.995 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:17.996 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T07:21:17.996 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T07:21:17.996 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T07:21:17.996 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:18.045 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4938.g4ef5f 139/165 2024-09-16T07:21:18.094 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4938.g4ef5f 139/165 2024-09-16T07:21:18.095 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:18.095 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T07:21:18.095 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:18.154 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 140/165 2024-09-16T07:21:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 140/165 2024-09-16T07:21:18.205 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:18.205 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T07:21:18.205 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T07:21:18.205 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T07:21:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:18.555 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 141/165 2024-09-16T07:21:18.605 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 141/165 2024-09-16T07:21:18.606 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:18.606 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T07:21:18.606 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T07:21:18.606 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T07:21:18.606 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:19.304 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4938.g4ef5f058.el9.noa 129/165 2024-09-16T07:21:19.364 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4938.g4ef5f058.el9.n 130/165 2024-09-16T07:21:20.443 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4938.g4ef5f058.el9.n 130/165 2024-09-16T07:21:20.472 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4938.g4ef 131/165 2024-09-16T07:21:20.625 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4938.g4ef 131/165 2024-09-16T07:21:20.813 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4938.g4ef5f058.el 132/165 2024-09-16T07:21:20.841 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.3.0-4938.g4ef5f058.el9.x86_64 133/165 2024-09-16T07:21:20.893 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4938.g4ef5f058.el9.x86_64 133/165 2024-09-16T07:21:20.893 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:20.893 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T07:21:20.893 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T07:21:20.893 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T07:21:20.893 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:20.950 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.3.0-4938.g4ef5f058.el9.noarch 134/165 2024-09-16T07:21:20.976 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4938.g4ef5f058.el9.noarch 134/165 2024-09-16T07:21:21.082 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.3.0-4938.g4ef5f058.el9.x86_64 135/165 2024-09-16T07:21:21.149 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 136/165 2024-09-16T07:21:22.180 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-16T07:21:22.219 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 138/165 2024-09-16T07:21:22.271 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 138/165 2024-09-16T07:21:22.271 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:22.271 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T07:21:22.271 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T07:21:22.272 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T07:21:22.272 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:22.328 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4938.g4ef5f 139/165 2024-09-16T07:21:22.378 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4938.g4ef5f 139/165 2024-09-16T07:21:22.378 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:22.378 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T07:21:22.378 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:22.428 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 140/165 2024-09-16T07:21:22.480 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 140/165 2024-09-16T07:21:22.480 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:22.480 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T07:21:22.480 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T07:21:22.480 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T07:21:22.480 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:22.829 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 141/165 2024-09-16T07:21:22.880 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 141/165 2024-09-16T07:21:22.881 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T07:21:22.881 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T07:21:22.881 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T07:21:22.881 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T07:21:22.881 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:23.259 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:19.3.0-4938.g4ef5f058.el9.x86_64 142/165 2024-09-16T07:21:23.300 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 143/165 2024-09-16T07:21:23.344 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 144/165 2024-09-16T07:21:23.373 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-16T07:21:23.408 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-16T07:21:23.662 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-shell-2:19.3.0-4938.g4ef5f058.el9.x86_64 147/165 2024-09-16T07:21:23.722 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-16T07:21:23.752 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:19.3.0-4938.g4ef5f058.el9.x86_ 149/165 2024-09-16T07:21:23.792 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-16T07:21:23.864 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-16T07:21:24.637 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T07:21:24.692 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T07:21:24.734 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T07:21:25.988 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:19.3.0-4938.g4ef5f058.el9.noarch 153/165 2024-09-16T07:21:26.089 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-16T07:21:26.150 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-16T07:21:26.266 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-16T07:21:26.300 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-16T07:21:26.325 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-16T07:21:26.361 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-16T07:21:26.361 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T07:21:26.469 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T07:21:26.469 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T07:21:26.578 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T07:21:26.602 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-16T07:21:26.619 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T07:21:26.619 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T07:21:26.855 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T07:21:26.855 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T07:21:26.955 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T07:21:27.029 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T07:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T07:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-16T07:21:27.241 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:27.558 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.3.0-4938.g4ef5f058.el9.x86_64 142/165 2024-09-16T07:21:27.599 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 143/165 2024-09-16T07:21:27.644 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 144/165 2024-09-16T07:21:27.672 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-16T07:21:27.738 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-16T07:21:27.980 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-shell-2:19.3.0-4938.g4ef5f058.el9.x86_64 147/165 2024-09-16T07:21:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-16T07:21:28.067 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.3.0-4938.g4ef5f058.el9.x86_ 149/165 2024-09-16T07:21:28.099 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-16T07:21:28.168 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-16T07:21:28.970 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T07:21:29.015 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T07:21:29.057 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T07:21:30.329 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:19.3.0-4938.g4ef5f058.el9.noarch 153/165 2024-09-16T07:21:30.445 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-16T07:21:30.508 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-16T07:21:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-16T07:21:30.681 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-16T07:21:30.706 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-16T07:21:30.742 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-16T07:21:30.743 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T07:21:30.850 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T07:21:30.851 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T07:21:31.001 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T07:21:31.034 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-16T07:21:31.053 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T07:21:31.053 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T07:21:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T07:21:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T07:21:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T07:21:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T07:21:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T07:21:31.714 INFO:teuthology.orchestra.run.smithi071.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-16T07:21:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:33.349 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:19.3.0-4938.g4ef5f058.el9.x86_64 1/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:19.3.0-4938.g4ef5f058.el9.x86_64 2/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 3/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 4/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4938.g4ef5f 5/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:19.3.0-4938.g4ef5f058.el9.x86_64 6/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:19.3.0-4938.g4ef5f058.el9.x86_64 7/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:19.3.0-4938.g4ef5f058.el9.x86_64 8/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:19.3.0-4938.g4ef5f058.el9.x86_64 9/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 10/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:19.3.0-4938.g4ef5f058.el9.x86_64 11/165 2024-09-16T07:21:33.350 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:19.3.0-4938.g4ef5f058.el9.x86_64 12/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 13/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-shell-2:19.3.0-4938.g4ef5f058.el9.x86_64 14/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:19.3.0-4938.g4ef5f058.el9.x86_ 15/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:19.3.0-4938.g4ef5f058.el9.x86_64 16/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:19.3.0-4938.g4ef5f058.el9.x86_64 17/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:19.3.0-4938.g4ef5f058.el9.x86_6 18/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:19.3.0-4938.g4ef5f058.el9.x86 19/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:19.3.0-4938.g4ef5f058.el9.x86_64 20/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4938.g4ef5f058.el 21/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:19.3.0-4938.g4ef5f058.el9. 22/165 2024-09-16T07:21:33.351 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:19.3.0-4938.g4ef5f058.el9.x86_6 23/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:19.3.0-4938.g4ef5f058.el9.x86_64 24/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 25/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 26/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 27/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 28/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 29/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4938.g4ef5f058. 30/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4938.g4ef5f058.el9.noa 31/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4938.g4ef5f058.el9.n 32/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4938.g4ef 33/165 2024-09-16T07:21:33.352 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4938.g4ef5f058.el 34/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4938.g4ef5f058.el9.noarch 35/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4938.g4ef5f058.e 36/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:19.3.0-4938.g4ef5f058.el9.noarch 37/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 38/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:19.3.0-4938.g4ef5f058.el9.noarch 39/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-16T07:21:33.353 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-16T07:21:33.354 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-16T07:21:33.355 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-16T07:21:33.356 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-16T07:21:33.357 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-16T07:21:33.358 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-16T07:21:33.359 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-16T07:21:33.360 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-16T07:21:33.361 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:19.3.0-4938.g4ef5f058.el9.x86_64 150/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:19.3.0-4938.g4ef5f058.el9.x86_64 152/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-16T07:21:33.362 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-16T07:21:33.363 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-16T07:21:35.707 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-16T07:21:35.708 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.709 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.710 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T07:21:35.711 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T07:21:35.712 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T07:21:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.714 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T07:21:35.715 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T07:21:35.716 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T07:21:35.717 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T07:21:35.718 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T07:21:35.719 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T07:21:35.720 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T07:21:35.720 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T07:21:35.720 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T07:21:35.720 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T07:21:35.720 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:35.720 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-16T07:21:36.347 DEBUG:teuthology.parallel:result is None 2024-09-16T07:21:37.998 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.3.0-4938.g4ef5f058.el9.x86_64 1/165 2024-09-16T07:21:37.998 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.3.0-4938.g4ef5f058.el9.x86_64 2/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 3/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 4/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4938.g4ef5f 5/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.3.0-4938.g4ef5f058.el9.x86_64 6/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.3.0-4938.g4ef5f058.el9.x86_64 7/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.3.0-4938.g4ef5f058.el9.x86_64 8/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.3.0-4938.g4ef5f058.el9.x86_64 9/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 10/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.3.0-4938.g4ef5f058.el9.x86_64 11/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.3.0-4938.g4ef5f058.el9.x86_64 12/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 13/165 2024-09-16T07:21:37.999 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-shell-2:19.3.0-4938.g4ef5f058.el9.x86_64 14/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.3.0-4938.g4ef5f058.el9.x86_ 15/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.3.0-4938.g4ef5f058.el9.x86_64 16/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.3.0-4938.g4ef5f058.el9.x86_64 17/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.3.0-4938.g4ef5f058.el9.x86_6 18/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.3.0-4938.g4ef5f058.el9.x86 19/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.3.0-4938.g4ef5f058.el9.x86_64 20/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4938.g4ef5f058.el 21/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.3.0-4938.g4ef5f058.el9. 22/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.3.0-4938.g4ef5f058.el9.x86_6 23/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.3.0-4938.g4ef5f058.el9.x86_64 24/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 25/165 2024-09-16T07:21:38.000 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 26/165 2024-09-16T07:21:38.001 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 27/165 2024-09-16T07:21:38.001 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 28/165 2024-09-16T07:21:38.001 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 29/165 2024-09-16T07:21:38.012 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4938.g4ef5f058. 30/165 2024-09-16T07:21:38.012 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4938.g4ef5f058.el9.noa 31/165 2024-09-16T07:21:38.012 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4938.g4ef5f058.el9.n 32/165 2024-09-16T07:21:38.012 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4938.g4ef 33/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4938.g4ef5f058.el 34/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4938.g4ef5f058.el9.noarch 35/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4938.g4ef5f058.e 36/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.3.0-4938.g4ef5f058.el9.noarch 37/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 38/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:19.3.0-4938.g4ef5f058.el9.noarch 39/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-16T07:21:38.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-16T07:21:38.014 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-16T07:21:38.015 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-16T07:21:38.016 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-16T07:21:38.017 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-16T07:21:38.018 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-16T07:21:38.019 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-16T07:21:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.3.0-4938.g4ef5f058.el9.x86_64 150/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.3.0-4938.g4ef5f058.el9.x86_64 152/165 2024-09-16T07:21:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-16T07:21:38.022 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-16T07:21:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T07:21:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-09-16T07:21:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T07:21:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T07:21:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T07:21:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:19.3.0-4938.g4ef5f058.el9.noarch 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T07:21:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T07:21:40.260 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T07:21:40.260 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T07:21:40.260 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T07:21:40.260 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T07:21:40.260 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T07:21:40.260 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T07:21:40.261 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T07:21:40.262 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T07:21:40.263 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T07:21:40.264 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T07:21:40.264 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T07:21:40.264 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T07:21:40.264 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T07:21:40.264 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T07:21:40.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T07:21:40.264 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T07:21:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T07:21:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T07:21:40.267 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T07:21:40.268 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T07:21:40.269 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T07:21:40.270 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T07:21:40.270 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T07:21:40.270 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T07:21:40.270 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T07:21:40.270 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T07:21:40.270 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T07:21:40.270 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T07:21:40.270 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.3.0-4938.g4ef5f058.el9.x86_64 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T07:21:40.271 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T07:21:40.272 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T07:21:40.272 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T07:21:40.272 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T07:21:40.272 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T07:21:40.272 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T07:21:40.272 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:40.273 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-16T07:21:40.916 DEBUG:teuthology.parallel:result is None 2024-09-16T07:21:40.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ef5f0587d1cad82c4295c20951c303dc342ac74 2024-09-16T07:21:41.060 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T07:21:41.110 INFO:teuthology.orchestra.run.smithi071.stdout:19.3.0-4938.g4ef5f058.el9 2024-09-16T07:21:41.111 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4938.g4ef5f058.el9 2024-09-16T07:21:41.111 INFO:teuthology.task.install:The correct ceph version 19.3.0-4938.g4ef5f058 is installed. 2024-09-16T07:21:41.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4ef5f0587d1cad82c4295c20951c303dc342ac74 2024-09-16T07:21:41.249 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T07:21:41.299 INFO:teuthology.orchestra.run.smithi177.stdout:19.3.0-4938.g4ef5f058.el9 2024-09-16T07:21:41.300 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4938.g4ef5f058.el9 2024-09-16T07:21:41.300 INFO:teuthology.task.install:The correct ceph version 19.3.0-4938.g4ef5f058 is installed. 2024-09-16T07:21:41.302 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T07:21:41.302 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:41.302 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T07:21:41.361 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:21:41.362 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T07:21:41.422 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T07:21:41.423 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:41.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T07:21:41.481 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T07:21:41.578 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:21:41.579 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T07:21:41.637 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T07:21:41.732 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T07:21:41.732 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:41.732 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T07:21:41.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T07:21:41.888 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:21:41.888 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T07:21:41.943 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T07:21:42.038 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T07:21:42.038 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:42.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T07:21:42.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T07:21:42.192 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:21:42.192 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T07:21:42.247 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T07:21:42.342 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T07:21:42.424 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T07:21:42.424 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /var/log/ceph 2024-09-16T07:21:42.429 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2024-09-16T07:21:42.483 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T07:21:42.484 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T07:21:42.487 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T07:21:42.541 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T07:21:42.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T07:21:42.545 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T07:21:42.610 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-16T07:21:42.610 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-16T07:21:42.610 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-16T07:21:42.610 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:42.611 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T07:21:42.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T07:21:42.817 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-16T07:21:42.874 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:21:42.875 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T07:21:42.910 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T07:21:43.077 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-16T07:21:43.136 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T07:21:43.136 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T07:21:43.137 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852', 'branch': 'wip-athakkar-testing-2024-09-15-1936', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7906852', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-athakkar-testing-2024-09-15-1936'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-athakkar-testing-2024-09-15-1936', 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74'}}, 'owner': 'scheduled_khiremat@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6756, 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74', 'sleep_before_teardown': 0, 'subset': '3/4', 'suite': 'fs:volumes', 'suite_branch': 'wip-athakkar-testing-2024-09-15-1936', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_4ef5f0587d1cad82c4295c20951c303dc342ac74/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi071.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMNv1jjasIxNL8MxftPAQlXdIxvvAPHQ+UGxPSWhs1QW2lgcDzLSFS58pD4FvJ4tgcw8vUkbiIX9lg2C+dCgMiU=', 'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0L/CaUqEMFkrNLzDAkVgiUhck6icOxm1c+d+l9PWmYh/kV45yz03a/WyIFfeirS4sEtO9ddK+zreai5IDFXPU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4ef5f0587d1cad82c4295c20951c303dc342ac74', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_07:02:54', 'tube': 'smithi', 'user': 'khiremat', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T07:21:43.137 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T07:21:43.142 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T07:21:43.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T07:21:43.221 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T07:21:43.277 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:43.277 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T07:21:43.312 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T07:21:43.312 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-09-16T07:21:43.389 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T07:21:43.390 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T07:21:43.390 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-09-16T07:21:43.390 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T07:21:43.390 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T07:21:43.390 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-16 07:21:32.093547532 +0000 2024-09-16T07:21:43.390 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-16 07:19:22.997279351 +0000 2024-09-16T07:21:43.390 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-16 07:19:22.997279351 +0000 2024-09-16T07:21:43.390 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-16 07:19:22.997279351 +0000 2024-09-16T07:21:43.390 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T07:21:43.484 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-16T07:21:43.484 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-16T07:21:43.484 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000357299 s, 1.4 MB/s 2024-09-16T07:21:43.487 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T07:21:43.523 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 657 Links: 1 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-16 07:21:32.094547514 +0000 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-16 07:19:23.461271309 +0000 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-16 07:19:23.461271309 +0000 2024-09-16T07:21:43.599 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-16 07:19:23.461271309 +0000 2024-09-16T07:21:43.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T07:21:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-16T07:21:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-16T07:21:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000255604 s, 2.0 MB/s 2024-09-16T07:21:43.695 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T07:21:43.731 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 678 Links: 1 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-16 07:21:32.095547497 +0000 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-16 07:19:23.942262972 +0000 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-16 07:19:23.942262972 +0000 2024-09-16T07:21:43.808 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-16 07:19:23.942262972 +0000 2024-09-16T07:21:43.809 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T07:21:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-16T07:21:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-16T07:21:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000263642 s, 1.9 MB/s 2024-09-16T07:21:43.905 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T07:21:43.941 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-09-16T07:21:44.016 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T07:21:44.016 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T07:21:44.017 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 695 Links: 1 2024-09-16T07:21:44.017 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T07:21:44.017 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T07:21:44.017 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-16 07:21:32.095547497 +0000 2024-09-16T07:21:44.017 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-16 07:19:24.402254999 +0000 2024-09-16T07:21:44.017 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-16 07:19:24.402254999 +0000 2024-09-16T07:21:44.017 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-16 07:19:24.402254999 +0000 2024-09-16T07:21:44.017 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T07:21:44.108 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-16T07:21:44.108 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-16T07:21:44.108 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000274347 s, 1.9 MB/s 2024-09-16T07:21:44.111 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T07:21:44.147 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T07:21:44.147 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:21:44.147 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T07:21:44.185 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T07:21:44.185 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-09-16T07:21:44.261 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T07:21:44.262 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T07:21:44.262 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-16T07:21:44.262 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T07:21:44.262 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T07:21:44.262 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-16 07:21:27.584991163 +0000 2024-09-16T07:21:44.262 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-16 07:19:22.388109782 +0000 2024-09-16T07:21:44.262 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-16 07:19:22.388109782 +0000 2024-09-16T07:21:44.262 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-16 07:19:22.388109782 +0000 2024-09-16T07:21:44.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T07:21:44.354 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-16T07:21:44.354 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-16T07:21:44.355 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000520053 s, 985 kB/s 2024-09-16T07:21:44.357 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T07:21:44.393 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-09-16T07:21:44.468 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T07:21:44.468 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T07:21:44.468 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-16T07:21:44.468 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T07:21:44.469 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T07:21:44.469 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-16 07:21:27.585991136 +0000 2024-09-16T07:21:44.469 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-16 07:19:22.856096738 +0000 2024-09-16T07:21:44.469 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-16 07:19:22.856096738 +0000 2024-09-16T07:21:44.469 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-16 07:19:22.856096738 +0000 2024-09-16T07:21:44.469 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T07:21:44.561 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-16T07:21:44.561 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-16T07:21:44.561 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000290559 s, 1.8 MB/s 2024-09-16T07:21:44.563 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T07:21:44.600 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-16 07:21:27.586991108 +0000 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-16 07:19:23.307084168 +0000 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-16 07:19:23.307084168 +0000 2024-09-16T07:21:44.677 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-16 07:19:23.307084168 +0000 2024-09-16T07:21:44.678 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T07:21:44.770 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-16T07:21:44.770 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-16T07:21:44.771 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000315369 s, 1.6 MB/s 2024-09-16T07:21:44.773 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T07:21:44.809 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-09-16T07:21:44.884 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T07:21:44.885 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T07:21:44.885 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-16T07:21:44.885 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T07:21:44.885 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T07:21:44.885 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-16 07:21:27.586991108 +0000 2024-09-16T07:21:44.885 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-16 07:19:23.749071849 +0000 2024-09-16T07:21:44.885 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-16 07:19:23.749071849 +0000 2024-09-16T07:21:44.885 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-16 07:19:23.749071849 +0000 2024-09-16T07:21:44.886 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T07:21:44.976 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-16T07:21:44.977 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-16T07:21:44.977 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.00023783 s, 2.2 MB/s 2024-09-16T07:21:44.979 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T07:21:45.016 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T07:21:45.016 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T07:21:45.016 INFO:tasks.ceph:Generating config... 2024-09-16T07:21:45.017 INFO:tasks.ceph:[client] pid file = 2024-09-16T07:21:45.017 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-16T07:21:45.017 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-16T07:21:45.017 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-16T07:21:45.018 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-16T07:21:45.018 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T07:21:45.018 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T07:21:45.018 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T07:21:45.018 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T07:21:45.018 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T07:21:45.018 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-16T07:21:45.018 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T07:21:45.018 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T07:21:45.018 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-16T07:21:45.018 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T07:21:45.018 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T07:21:45.018 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T07:21:45.018 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T07:21:45.019 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T07:21:45.019 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T07:21:45.019 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T07:21:45.019 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T07:21:45.019 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T07:21:45.019 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T07:21:45.019 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T07:21:45.019 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T07:21:45.019 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T07:21:45.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T07:21:45.108 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T07:21:45.113 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T07:21:45.199 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T07:21:45.253 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.71'), ('mon.b', '172.21.15.177'), ('mon.c', '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]')] 2024-09-16T07:21:45.254 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.71,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-16T07:21:45.254 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:45.255 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T07:21:45.289 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.71 --add b 172.21.15.177 --addv c '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:21:45.407 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:21:45.407 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: generated fsid 5821e025-6a94-4c05-a5f1-d6094eac5c68 2024-09-16T07:21:45.408 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = quincy 2024-09-16T07:21:45.408 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 0 2024-09-16T07:21:45.408 INFO:teuthology.orchestra.run.smithi071.stdout:fsid 5821e025-6a94-4c05-a5f1-d6094eac5c68 2024-09-16T07:21:45.408 INFO:teuthology.orchestra.run.smithi071.stdout:last_changed 2024-09-16T07:21:45.408638+0000 2024-09-16T07:21:45.408 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-09-16T07:21:45.408638+0000 2024-09-16T07:21:45.409 INFO:teuthology.orchestra.run.smithi071.stdout:min_mon_release 17 (quincy) 2024-09-16T07:21:45.409 INFO:teuthology.orchestra.run.smithi071.stdout:election_strategy: 1 2024-09-16T07:21:45.409 INFO:teuthology.orchestra.run.smithi071.stdout:0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-16T07:21:45.412 INFO:teuthology.orchestra.run.smithi071.stdout:1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-09-16T07:21:45.413 INFO:teuthology.orchestra.run.smithi071.stdout:2: [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] mon.c 2024-09-16T07:21:45.413 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T07:21:45.414 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T07:21:45.448 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5821e025-6a94-4c05-a5f1-d6094eac5c68... 2024-09-16T07:21:45.449 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T07:21:45.493 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T07:21:45.586 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: chdir = "" 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: auth supported = cephx 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: filestore xattr use omap = true 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: mon clock drift allowed = 1.000 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: osd crush chooseleaf type = 0 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: auth debug = true 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.587 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on old message = true 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on bug = true 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: mon pg warn max object skew = 0 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default size = 2 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow primary affinity = true 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow pg remap = true 2024-09-16T07:21:45.588 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on legacy crush tunables = false 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on crush straw calc version zero = false 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on no sortbitwise = false 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on osd down out interval zero = false 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on too few osds = false 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: mon_allow_pool_size_one = true 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: osd default data pool replay window = 5 2024-09-16T07:21:45.589 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: mon allow pool delete = true 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: mon cluster log file level = debug 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: debug asserts on shutdown = true 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: mon health detail to clog = false 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: mon host = "172.21.15.71,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on skipped message = False 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = 5821e025-6a94-4c05-a5f1-d6094eac5c68 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout:[osd] 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: osd journal size = 100 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.590 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub load threshold = 5.0 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub max interval = 600 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd mclock profile = high_recovery_ops 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd recover clone overlap = true 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd recovery max chunk = 1048576 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug shutdown = true 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug op order = true 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug verify stray on activate = true 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug trim objects = true 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.591 INFO:teuthology.orchestra.run.smithi071.stdout: osd open classes on start = true 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug pg log writeout = true 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: osd deep scrub update digest min age = 30 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: osd map max advance = 10 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: journal zero on create = true 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: filestore ondisk finisher threads = 3 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: filestore apply finisher threads = 3 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: bdev debug aio = true 2024-09-16T07:21:45.592 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug misdirected ops = true 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: bdev async discard = True 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: bdev enable discard = True 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore allocator = bitmap 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore block size = 96636764160 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore fsck on mount = True 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluefs = 1/20 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluestore = 1/20 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: debug osd = 20 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: debug rocksdb = 4/10 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd full ratio = 0.9 2024-09-16T07:21:45.593 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd nearfull ratio = 0.8 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: osd failsafe full ratio = 0.95 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: osd objectstore = bluestore 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr] 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: debug mgr = 20 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: mgr/telemetry/nag = false 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: client mount timeout = 30 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: debug client = 20 2024-09-16T07:21:45.594 INFO:teuthology.orchestra.run.smithi071.stdout: debug finisher = 20 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout:[mon] 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: debug paxos = 20 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: mon data avail warn = 5 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: mon mgr mkfs grace = 240 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd reporter subtree level = osd 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd prime pg temp = true 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-09-16T07:21:45.595 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: # 1m isn't quite enough 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: mon_down_mkfs_grace = 2m 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_filestore_osds = false 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: mon down mkfs grace = 300 2024-09-16T07:21:45.596 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout:[client] 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: rgw cache enabled = true 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable ops log = true 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable usage log = true 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = "" 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.a] 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.b] 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.c] 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.a] 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.c] 2024-09-16T07:21:45.597 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.b] 2024-09-16T07:21:45.598 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.d] 2024-09-16T07:21:45.598 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-09-16T07:21:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-09-16T07:21:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T07:21:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-09-16T07:21:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-09-16T07:21:45.598 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T07:21:45.599 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T07:21:45.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.71,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on skipped message = False 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 5821e025-6a94-4c05-a5f1-d6094eac5c68 2024-09-16T07:21:45.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-09-16T07:21:45.602 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug trim objects = true 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-09-16T07:21:45.603 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = bitmap 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2024-09-16T07:21:45.604 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-09-16T07:21:45.605 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 30 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: debug finisher = 20 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-09-16T07:21:45.606 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-09-16T07:21:45.607 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: mon down mkfs grace = 300 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-09-16T07:21:45.608 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-09-16T07:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2024-09-16T07:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2024-09-16T07:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2024-09-16T07:21:45.609 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2024-09-16T07:21:45.616 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T07:21:45.617 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T07:21:45.717 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T07:21:45.717 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:45.717 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T07:21:45.753 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:45.753 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T07:21:45.829 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:21:45.830 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:45.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T07:21:45.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T07:21:45.947 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:21:45.947 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:21:46.022 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:21:46.022 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:21:46.022 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T07:21:46.022 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T07:21:46.101 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:21:46.101 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:21:46.177 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T07:21:46.177 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T07:21:46.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T07:21:46.280 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T07:21:46.286 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T07:21:46.397 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T07:21:46.404 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T07:21:46.404 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T07:21:46.507 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T07:21:46.512 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-16T07:21:46.567 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T07:21:46.669 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T07:21:46.674 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-16T07:21:46.729 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T07:21:46.833 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T07:21:46.838 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-16T07:21:46.893 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-16T07:21:46.997 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-16T07:21:47.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-16T07:21:47.057 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T07:21:47.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T07:21:47.140 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T07:21:47.168 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-16T07:21:47.249 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-16T07:21:47.277 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T07:21:47.358 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T07:21:47.386 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-16T07:21:47.466 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-16T07:21:47.493 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T07:21:47.494 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T07:21:47.494 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T07:21:47.550 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T07:21:47.550 INFO:tasks.ceph:role: osd.0 2024-09-16T07:21:47.550 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:21:47.550 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T07:21:47.612 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T07:21:47.612 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T07:21:47.612 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T07:21:47.612 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T07:21:47.612 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T07:21:47.612 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-16T07:21:47.613 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T07:21:47.613 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T07:21:47.613 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T07:21:47.613 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T07:21:48.015 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-16T07:21:48.019 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-16T07:21:48.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T07:21:48.153 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T07:21:48.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T07:21:48.271 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T07:21:48.271 INFO:tasks.ceph:role: osd.1 2024-09-16T07:21:48.271 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:21:48.271 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T07:21:48.333 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T07:21:48.333 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T07:21:48.333 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T07:21:48.333 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T07:21:48.333 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T07:21:48.334 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-16T07:21:48.334 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T07:21:48.334 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T07:21:48.334 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T07:21:48.334 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T07:21:48.691 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-16T07:21:48.695 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-16T07:21:48.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T07:21:48.779 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T07:21:48.841 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T07:21:48.896 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T07:21:48.896 INFO:tasks.ceph:role: osd.2 2024-09-16T07:21:48.896 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:21:48.896 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T07:21:48.958 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T07:21:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T07:21:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T07:21:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T07:21:48.958 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T07:21:48.958 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-16T07:21:48.959 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T07:21:48.959 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T07:21:48.959 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T07:21:48.959 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T07:21:49.340 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-16T07:21:49.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-16T07:21:49.344 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T07:21:49.446 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T07:21:49.507 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T07:21:49.562 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T07:21:49.562 INFO:tasks.ceph:role: osd.3 2024-09-16T07:21:49.562 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:21:49.562 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T07:21:49.624 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T07:21:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T07:21:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T07:21:49.624 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T07:21:49.625 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T07:21:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-09-16T07:21:49.625 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T07:21:49.625 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T07:21:49.625 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T07:21:49.625 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T07:21:50.026 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-09-16T07:21:50.031 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-09-16T07:21:50.032 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T07:21:50.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T07:21:50.287 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:21:50.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:50.386+0000 7f18abc15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T07:21:50.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:50.387+0000 7f18abc15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T07:21:50.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:50.387+0000 7f18abc15640 -1 bdev(0x5633e60a5800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-16T07:21:50.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:50.388+0000 7f18abc15640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T07:21:53.431 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T07:21:53.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:21:53.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:53.578+0000 7ff6b61dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T07:21:53.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:53.579+0000 7ff6b61dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T07:21:53.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:53.579+0000 7ff6b61dd640 -1 bdev(0x55ad21c2d800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-16T07:21:53.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:53.579+0000 7ff6b61dd640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T07:21:56.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T07:21:56.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:21:56.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:56.798+0000 7f2b971cc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T07:21:56.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:56.798+0000 7f2b971cc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T07:21:56.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:56.798+0000 7f2b971cc640 -1 bdev(0x559fb8e79800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-16T07:21:56.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:56.798+0000 7f2b971cc640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T07:21:59.827 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T07:21:59.884 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:21:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:59.976+0000 7f6a9c958640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T07:21:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:59.976+0000 7f6a9c958640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T07:21:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:59.976+0000 7f6a9c958640 -1 bdev(0x5568cf2ad800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-16T07:21:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:21:59.977+0000 7f6a9c958640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T07:22:02.999 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T07:22:03.055 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T07:22:03.117 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T07:22:03.117 INFO:tasks.ceph:role: osd.4 2024-09-16T07:22:03.117 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:22:03.117 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T07:22:03.182 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T07:22:03.584 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-09-16T07:22:03.589 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-16T07:22:03.589 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T07:22:03.731 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T07:22:03.797 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T07:22:03.853 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T07:22:03.853 INFO:tasks.ceph:role: osd.5 2024-09-16T07:22:03.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:22:03.853 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T07:22:03.919 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T07:22:03.919 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T07:22:03.920 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T07:22:03.920 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T07:22:03.920 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T07:22:03.920 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-16T07:22:03.920 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T07:22:03.920 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T07:22:03.920 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T07:22:03.920 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T07:22:04.274 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-09-16T07:22:04.278 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-16T07:22:04.279 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T07:22:04.364 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T07:22:04.427 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T07:22:04.494 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T07:22:04.494 INFO:tasks.ceph:role: osd.6 2024-09-16T07:22:04.494 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:22:04.494 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T07:22:04.560 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T07:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T07:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T07:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T07:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T07:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-16T07:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T07:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T07:22:04.561 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T07:22:04.562 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T07:22:04.942 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-09-16T07:22:04.947 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-16T07:22:04.947 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T07:22:05.049 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T07:22:05.112 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T07:22:05.167 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T07:22:05.167 INFO:tasks.ceph:role: osd.7 2024-09-16T07:22:05.167 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:22:05.167 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T07:22:05.229 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T07:22:05.229 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T07:22:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T07:22:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T07:22:05.230 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T07:22:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-16T07:22:05.230 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T07:22:05.230 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T07:22:05.230 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T07:22:05.230 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T07:22:05.642 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-09-16T07:22:05.646 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-16T07:22:05.646 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T07:22:05.789 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T07:22:05.852 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:22:05.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:05.956+0000 7fae3ec12640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T07:22:05.957 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:05.958+0000 7fae3ec12640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T07:22:05.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:05.958+0000 7fae3ec12640 -1 bdev(0x55a802ac1800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-16T07:22:05.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:05.958+0000 7fae3ec12640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T07:22:08.999 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T07:22:09.056 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:22:09.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:09.147+0000 7fab9e8d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T07:22:09.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:09.148+0000 7fab9e8d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T07:22:09.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:09.148+0000 7fab9e8d6640 -1 bdev(0x556dd3e25800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-16T07:22:09.148 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:09.148+0000 7fab9e8d6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T07:22:12.161 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T07:22:12.221 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:22:12.313 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:12.313+0000 7f637a54f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T07:22:12.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:12.314+0000 7f637a54f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T07:22:12.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:12.314+0000 7f637a54f640 -1 bdev(0x55a519b55800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-16T07:22:12.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:12.314+0000 7f637a54f640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T07:22:13.139 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:22:13.144 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:22:15.573 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T07:22:15.628 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:22:15.720 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:15.720+0000 7fddd461c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T07:22:15.721 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:15.721+0000 7fddd461c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T07:22:15.721 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:15.721+0000 7fddd461c640 -1 bdev(0x55ae9cafb800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-16T07:22:15.721 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:15.721+0000 7fddd461c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T07:22:18.730 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T07:22:18.787 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T07:22:18.787 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:18.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T07:22:18.849 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:18.849 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-16T07:22:18.948 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:18.948 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-16T07:22:19.044 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:19.045 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T07:22:19.140 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:19.140 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T07:22:19.235 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:19.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T07:22:19.330 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:19.330 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T07:22:19.425 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:19.425 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T07:22:19.482 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:19.483 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-16T07:22:19.579 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:19.579 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-16T07:22:19.674 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:19.674 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T07:22:19.769 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:19.769 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T07:22:19.864 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:19.864 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T07:22:19.959 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:19.960 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T07:22:20.055 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:20.055 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T07:22:20.093 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:20.093 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-16T07:22:20.170 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:20.170 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T07:22:20.204 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:20.205 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-16T07:22:20.279 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T07:22:20.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T07:22:20.283 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T07:22:20.333 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.y] 2024-09-16T07:22:20.333 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA== 2024-09-16T07:22:20.333 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.a] 2024-09-16T07:22:20.333 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg== 2024-09-16T07:22:20.333 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.c] 2024-09-16T07:22:20.333 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw== 2024-09-16T07:22:20.333 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.0] 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w== 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.1] 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ== 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.2] 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ== 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.3] 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ== 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.x] 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q== 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.b] 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag== 2024-09-16T07:22:20.334 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.d] 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA== 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.4] 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw== 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.5] 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg== 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.6] 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw== 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.7] 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg== 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:22:20.335 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:22:20.336 INFO:teuthology.orchestra.run.smithi071.stdout:[client.2] 2024-09-16T07:22:20.336 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg== 2024-09-16T07:22:20.336 INFO:teuthology.orchestra.run.smithi071.stdout:[client.1] 2024-09-16T07:22:20.336 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg== 2024-09-16T07:22:20.336 INFO:teuthology.orchestra.run.smithi071.stdout:[client.3] 2024-09-16T07:22:20.336 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA== 2024-09-16T07:22:20.371 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2024-09-16T07:22:20.371 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA== 2024-09-16T07:22:20.371 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2024-09-16T07:22:20.371 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg== 2024-09-16T07:22:20.371 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2024-09-16T07:22:20.371 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw== 2024-09-16T07:22:20.371 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w== 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ== 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ== 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ== 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q== 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag== 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2024-09-16T07:22:20.372 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA== 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.4] 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw== 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.5] 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg== 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.6] 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw== 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.7] 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg== 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout:[client.0] 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout:[client.2] 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg== 2024-09-16T07:22:20.373 INFO:teuthology.orchestra.run.smithi177.stdout:[client.1] 2024-09-16T07:22:20.374 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg== 2024-09-16T07:22:20.374 INFO:teuthology.orchestra.run.smithi177.stdout:[client.3] 2024-09-16T07:22:20.374 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA== 2024-09-16T07:22:20.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T07:22:20.381 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T07:22:20.508 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T07:22:20.512 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T07:22:20.608 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T07:22:20.611 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T07:22:20.708 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:20.712 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:20.804 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:20.808 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:20.902 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:20.914 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.008 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.012 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T07:22:21.110 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T07:22:21.203 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T07:22:21.208 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T07:22:21.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T07:22:21.308 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T07:22:21.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.410 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.504 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.508 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.601 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.605 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.699 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.703 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T07:22:21.796 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T07:22:21.800 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T07:22:21.897 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T07:22:21.901 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T07:22:21.998 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T07:22:22.002 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T07:22:22.099 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T07:22:22.103 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T07:22:22.199 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T07:22:22.199 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T07:22:22.254 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T07:22:22.362 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T07:22:22.417 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T07:22:22.474 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T07:22:22.583 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T07:22:22.638 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T07:22:22.693 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T07:22:22.802 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T07:22:22.858 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:22:22.862 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T07:22:22.896 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T07:22:22.896 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T07:22:22.896 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T07:22:22.938 INFO:tasks.ceph.mon.a:Started 2024-09-16T07:22:22.938 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T07:22:22.938 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T07:22:22.942 INFO:tasks.ceph.mon.b:Started 2024-09-16T07:22:22.942 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T07:22:22.942 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T07:22:22.946 INFO:tasks.ceph.mon.c:Started 2024-09-16T07:22:22.946 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T07:22:22.946 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T07:22:22.946 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T07:22:22.950 INFO:tasks.ceph.mgr.y:Started 2024-09-16T07:22:22.950 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T07:22:22.950 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T07:22:22.954 INFO:tasks.ceph.mgr.x:Started 2024-09-16T07:22:22.955 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-16T07:22:22.955 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-16T07:22:22.958 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-16T07:22:22.959 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-16T07:22:23.005 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-16T07:22:23.005 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-16T07:22:23.051 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-16T07:22:23.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-16T07:22:23.097 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-16T07:22:23.097 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-16T07:22:23.143 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-16T07:22:23.143 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-16T07:22:23.190 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-16T07:22:23.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-16T07:22:23.236 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-16T07:22:23.236 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-16T07:22:23.285 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-16T07:22:23.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-16T07:22:23.333 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-16T07:22:23.334 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-16T07:22:23.368 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-16T07:22:23.368 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-16T07:22:23.418 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-16T07:22:23.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-16T07:22:23.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.426+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500415e2e0 con 0x7f50041f9b40 2024-09-16T07:22:23.430 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-16T07:22:23.431 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 -- 172.21.15.71:0/3347320594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d4161460 msgr2=0x7f11d40b89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 --2- 172.21.15.71:0/3347320594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d4161460 0x7f11d40b89a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00c680 tx=0x7f11cc00cb40 comp rx=0 tx=0).stop 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 -- 172.21.15.71:0/3347320594 shutdown_connections 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 --2- 172.21.15.71:0/3347320594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d4161460 0x7f11d40b89a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 --2- 172.21.15.71:0/3347320594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11d40b5480 0x7f11d40b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 --2- 172.21.15.71:0/3347320594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11d40b4f40 0x7f11d415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 -- 172.21.15.71:0/3347320594 >> 172.21.15.71:0/3347320594 conn(0x7f11d409d0b0 msgr2=0x7f11d41ea990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 -- 172.21.15.71:0/3347320594 shutdown_connections 2024-09-16T07:22:23.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f11d9e00640 1 -- 172.21.15.71:0/3347320594 wait complete. 2024-09-16T07:22:23.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.426+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50040b3e70 con 0x7f50041f9b40 2024-09-16T07:22:23.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.427+0000 7f5000c00640 1 -- 172.21.15.71:0/3079632785 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ff400cc10 con 0x7f50041f9b40 2024-09-16T07:22:23.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.428+0000 7f5000c00640 1 -- 172.21.15.71:0/3079632785 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4ff4003930 con 0x7f50041f9b40 2024-09-16T07:22:23.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.450+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50040b4070 con 0x7f50041f9b40 2024-09-16T07:22:23.456 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-16T07:22:23.457 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-16T07:22:23.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.457+0000 7f5000c00640 1 -- 172.21.15.71:0/3079632785 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f50040b4070 con 0x7f50041f9b40 2024-09-16T07:22:23.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.468+0000 7fb9a9e00640 1 -- 172.21.15.71:0/3527137666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fb978005100 con 0x7fb9a40af170 2024-09-16T07:22:23.476 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-16T07:22:23.476 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-16T07:22:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.476+0000 7f6177e00640 1 -- 172.21.15.71:0/1666559062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f615c004800 msgr2=0x7f615c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.476+0000 7f6177e00640 1 --2- 172.21.15.71:0/1666559062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f615c004800 0x7f615c004c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61780b2530 tx=0x7f615c005830 comp rx=0 tx=0).stop 2024-09-16T07:22:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.477+0000 7f6177400640 1 -- 172.21.15.71:0/1666559062 reap_dead start 2024-09-16T07:22:23.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.479+0000 7f5000c00640 1 -- 172.21.15.71:0/3079632785 <== mon.1 v2:172.21.15.177:3300/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f4ff40047a0 con 0x7f50041f9b40 2024-09-16T07:22:23.489 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-16T07:22:23.489 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-16T07:22:23.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.485+0000 7fb99b400640 1 -- 172.21.15.71:0/3527137666 <== mon.0 v2:172.21.15.71:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7fb990004900 con 0x7fb9a40af170 2024-09-16T07:22:23.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.485+0000 7fb99b400640 1 -- 172.21.15.71:0/3527137666 <== mon.0 v2:172.21.15.71:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fb990004c70 con 0x7fb9a40af170 2024-09-16T07:22:23.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.477+0000 7f6177e00640 1 -- 172.21.15.71:0/1666559062 shutdown_connections 2024-09-16T07:22:23.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.478+0000 7f6177e00640 1 -- 172.21.15.71:0/1666559062 >> 172.21.15.71:0/1666559062 conn(0x7f61780cf9f0 msgr2=0x7f61780cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.479+0000 7f6177e00640 1 -- 172.21.15.71:0/1666559062 shutdown_connections 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.479+0000 7f6177e00640 1 -- 172.21.15.71:0/1666559062 wait complete. 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.481+0000 7f6177e00640 1 Processor -- start 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.483+0000 7f6177e00640 1 -- start start 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.485+0000 7f6177e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f615c004800 0x7f61780c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.485+0000 7f6177e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61781e2360 0x7f61781e27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.485+0000 7f6177e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780c9070 0x7f61780c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.485+0000 7f6177e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f61780cad60 con 0x7f61781e2360 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.485+0000 7f6177e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61780cabe0 con 0x7f615c004800 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.485+0000 7f6177e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f61780b3970 con 0x7f61780c9070 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.486+0000 7f6176a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f615c004800 0x7f61780c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.486+0000 7f6176a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f615c004800 0x7f61780c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:57464/0 (socket says 172.21.15.71:57464) 2024-09-16T07:22:23.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.486+0000 7f6176a00640 1 -- 172.21.15.71:0/523893744 learned_addr learned my addr 172.21.15.71:0/523893744 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.487+0000 7f6176000640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61781e2360 0x7f61781e27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.487+0000 7f6176a00640 1 -- 172.21.15.71:0/523893744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780c9070 msgr2=0x7f61780c94f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.487+0000 7f6176a00640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780c9070 0x7f61780c94f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.487+0000 7f6176a00640 1 -- 172.21.15.71:0/523893744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61781e2360 msgr2=0x7f61781e27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.489+0000 7f6176a00640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61781e2360 0x7f61781e27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.489+0000 7f6176a00640 1 -- 172.21.15.71:0/523893744 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61781f2570 con 0x7f615c004800 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.490+0000 7f6176000640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61781e2360 0x7f61781e27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.491+0000 7fb998c00640 1 -- 172.21.15.71:0/3527137666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a40af170 msgr2=0x7fb9a41492e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.491+0000 7fb998c00640 1 --2- 172.21.15.71:0/3527137666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a40af170 0x7fb9a41492e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb99000c890 tx=0x7fb99000cd50 comp rx=0 tx=0).stop 2024-09-16T07:22:23.500 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-16T07:22:23.501 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-16T07:22:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.492+0000 7fb998c00640 1 -- 172.21.15.71:0/3527137666 shutdown_connections 2024-09-16T07:22:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.492+0000 7fb998c00640 1 --2- 172.21.15.71:0/3527137666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9a40b2b40 0x7fb9a41507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.492+0000 7fb998c00640 1 --2- 172.21.15.71:0/3527137666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a40af170 0x7fb9a41492e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.492+0000 7fb998c00640 1 --2- 172.21.15.71:0/3527137666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9a406fba0 0x7fb9a4148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.492+0000 7fb998c00640 1 -- 172.21.15.71:0/3527137666 >> 172.21.15.71:0/3527137666 conn(0x7fb9a40b1840 msgr2=0x7fb9a40af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.492+0000 7fb998c00640 1 -- 172.21.15.71:0/3527137666 shutdown_connections 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.492+0000 7fb998c00640 1 -- 172.21.15.71:0/3527137666 wait complete. 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.492+0000 7f6176a00640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f615c004800 0x7f61780c6110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f617000ebf0 tx=0x7f617000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.493+0000 7f616ea00640 1 -- 172.21.15.71:0/523893744 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170010630 con 0x7f615c004800 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.493+0000 7f616ea00640 1 -- 172.21.15.71:0/523893744 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f61700108a0 con 0x7f615c004800 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.499+0000 7f616ea00640 1 -- 172.21.15.71:0/523893744 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170010cb0 con 0x7f615c004800 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.499+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61781e2e90 con 0x7f615c004800 2024-09-16T07:22:23.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.499+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61781e3e30 con 0x7f615c004800 2024-09-16T07:22:23.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.505+0000 7f616ea00640 1 -- 172.21.15.71:0/523893744 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f617001f020 con 0x7f615c004800 2024-09-16T07:22:23.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.505+0000 7f616ea00640 1 -- 172.21.15.71:0/523893744 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f61700059b0 con 0x7f615c004800 2024-09-16T07:22:23.517 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-16T07:22:23.517 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-16T07:22:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.509+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61780cbc20 con 0x7f615c004800 2024-09-16T07:22:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.517+0000 7f616ea00640 1 -- 172.21.15.71:0/523893744 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6170026070 con 0x7f615c004800 2024-09-16T07:22:23.548 DEBUG:tasks.ceph:set 18 configs 2024-09-16T07:22:23.548 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi071') 2024-09-16T07:22:23.548 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi071') 2024-09-16T07:22:23.548 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi071') 2024-09-16T07:22:23.548 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi071') 2024-09-16T07:22:23.549 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi071') 2024-09-16T07:22:23.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.555+0000 7fb9ca800640 1 -- 172.21.15.71:0/622263683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c40b2b40 msgr2=0x7fb9c40cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.556+0000 7fb9ca800640 1 --2- 172.21.15.71:0/622263683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c40b2b40 0x7fb9c40cdeb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005fd0 tx=0x7fb9bc02a820 comp rx=0 tx=0).stop 2024-09-16T07:22:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.557+0000 7fb9ca800640 1 -- 172.21.15.71:0/622263683 shutdown_connections 2024-09-16T07:22:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.557+0000 7fb9ca800640 1 --2- 172.21.15.71:0/622263683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c40b2b40 0x7fb9c40cdeb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.557+0000 7fb9ca800640 1 --2- 172.21.15.71:0/622263683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c40af170 0x7fb9c41622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.557+0000 7fb9ca800640 1 -- 172.21.15.71:0/622263683 >> 172.21.15.71:0/622263683 conn(0x7fb9c40b1840 msgr2=0x7fb9c40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.557+0000 7fb9ca800640 1 -- 172.21.15.71:0/622263683 shutdown_connections 2024-09-16T07:22:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.557+0000 7fb9ca800640 1 -- 172.21.15.71:0/622263683 wait complete. 2024-09-16T07:22:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.560+0000 7fb9ca800640 1 Processor -- start 2024-09-16T07:22:23.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.563+0000 7fb9ca800640 1 -- start start 2024-09-16T07:22:23.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.570+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c406fba0 0x7fb9c4145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.570+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c40af170 0x7fb9c415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.570+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c40b2b40 0x7fb9c415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.570+0000 7fb9ca800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4070100 con 0x7fb9c406fba0 2024-09-16T07:22:23.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.570+0000 7fb9ca800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c406ff80 con 0x7fb9c40af170 2024-09-16T07:22:23.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.570+0000 7fb9ca800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c4070280 con 0x7fb9c40b2b40 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.571+0000 7fb9c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c406fba0 0x7fb9c4145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.571+0000 7fb9c2a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c40af170 0x7fb9c415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.571+0000 7fb9c2a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c40af170 0x7fb9c415cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:57474/0 (socket says 172.21.15.71:57474) 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.571+0000 7fb9c2a00640 1 -- 172.21.15.71:0/2723647786 learned_addr learned my addr 172.21.15.71:0/2723647786 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.571+0000 7fb9c3e00640 1 --2- 172.21.15.71:0/2723647786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c40b2b40 0x7fb9c415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.572+0000 7fb9c3e00640 1 -- 172.21.15.71:0/2723647786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c40af170 msgr2=0x7fb9c415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.572+0000 7fb9c3e00640 1 --2- 172.21.15.71:0/2723647786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c40af170 0x7fb9c415cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.572+0000 7fb9c3e00640 1 -- 172.21.15.71:0/2723647786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c406fba0 msgr2=0x7fb9c4145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.572+0000 7fb9c3e00640 1 --2- 172.21.15.71:0/2723647786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c406fba0 0x7fb9c4145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.575+0000 7fb9c3e00640 1 -- 172.21.15.71:0/2723647786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c41457b0 con 0x7fb9c40b2b40 2024-09-16T07:22:23.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.577+0000 7fb9c3e00640 1 --2- 172.21.15.71:0/2723647786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c40b2b40 0x7fb9c415af60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009000 tx=0x7fb9bc033c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.577+0000 7fb9bb400640 1 -- 172.21.15.71:0/2723647786 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc041020 con 0x7fb9c40b2b40 2024-09-16T07:22:23.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.580+0000 7fb9bb400640 1 -- 172.21.15.71:0/2723647786 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fb9bc033df0 con 0x7fb9c40b2b40 2024-09-16T07:22:23.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.580+0000 7fb9bb400640 1 -- 172.21.15.71:0/2723647786 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc037040 con 0x7fb9c40b2b40 2024-09-16T07:22:23.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.590+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c4158910 con 0x7fb9c40b2b40 2024-09-16T07:22:23.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.590+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c4158ed0 con 0x7fb9c40b2b40 2024-09-16T07:22:23.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.591+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c40cdeb0 con 0x7fb9c40b2b40 2024-09-16T07:22:23.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.594+0000 7fb9bb400640 1 -- 172.21.15.71:0/2723647786 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb9bc044020 con 0x7fb9c40b2b40 2024-09-16T07:22:23.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.594+0000 7fb9bb400640 1 -- 172.21.15.71:0/2723647786 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb9bc03d540 con 0x7fb9c40b2b40 2024-09-16T07:22:23.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.596+0000 7fb9bb400640 1 -- 172.21.15.71:0/2723647786 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb9bc04b070 con 0x7fb9c40b2b40 2024-09-16T07:22:23.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.620+0000 7f31e0a00640 1 -- 172.21.15.71:0/2102215358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31dc0b4ae0 msgr2=0x7f31dc15b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.620+0000 7f31e0a00640 1 --2- 172.21.15.71:0/2102215358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31dc0b4ae0 0x7f31dc15b070 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f31c80083e0 tx=0x7f31c802a0c0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.620+0000 7f31e0a00640 1 -- 172.21.15.71:0/2102215358 shutdown_connections 2024-09-16T07:22:23.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.620+0000 7f31e0a00640 1 --2- 172.21.15.71:0/2102215358 >> v2:172.21.15.71:3300/0 conn(0x7f31dc06f8d0 0x7f31dc06fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.620+0000 7f31e0a00640 1 --2- 172.21.15.71:0/2102215358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31dc0b4ae0 0x7f31dc15b070 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.620+0000 7f31e0a00640 1 -- 172.21.15.71:0/2102215358 >> 172.21.15.71:0/2102215358 conn(0x7f31dc0b1840 msgr2=0x7f31dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.620+0000 7f31e0a00640 1 -- 172.21.15.71:0/2102215358 shutdown_connections 2024-09-16T07:22:23.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.621+0000 7f31e0a00640 1 -- 172.21.15.71:0/2102215358 wait complete. 2024-09-16T07:22:23.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.624+0000 7f31e0a00640 1 Processor -- start 2024-09-16T07:22:23.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.625+0000 7f31e0a00640 1 -- start start 2024-09-16T07:22:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.626+0000 7f31e0a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31dc06f8d0 0x7f31dc1f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.626+0000 7f31e0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31dc0b4700 0x7f31dc153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.626+0000 7f31e0a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31dc0b4ae0 0x7f31dc1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.626+0000 7f31e0a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc0b24b0 con 0x7f31dc0b4700 2024-09-16T07:22:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.626+0000 7f31e0a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc0b2330 con 0x7f31dc06f8d0 2024-09-16T07:22:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.626+0000 7f31e0a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f31dc0cdad0 con 0x7f31dc0b4ae0 2024-09-16T07:22:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.631+0000 7f31d9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31dc0b4700 0x7f31dc153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.631+0000 7f31d9e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31dc06f8d0 0x7f31dc1f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.631+0000 7f31d9e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31dc06f8d0 0x7f31dc1f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:57480/0 (socket says 172.21.15.71:57480) 2024-09-16T07:22:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.631+0000 7f31d9e00640 1 -- 172.21.15.71:0/1322205399 learned_addr learned my addr 172.21.15.71:0/1322205399 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.632+0000 7f31d9400640 1 -- 172.21.15.71:0/1322205399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31dc0b4ae0 msgr2=0x7f31dc1568f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:22:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.632+0000 7f31d9400640 1 --2- 172.21.15.71:0/1322205399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31dc0b4ae0 0x7f31dc1568f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.632+0000 7f31d9400640 1 -- 172.21.15.71:0/1322205399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31dc06f8d0 msgr2=0x7f31dc1f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.632+0000 7f31d9400640 1 --2- 172.21.15.71:0/1322205399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31dc06f8d0 0x7f31dc1f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.632+0000 7f31d9400640 1 -- 172.21.15.71:0/1322205399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc153d60 con 0x7f31dc0b4700 2024-09-16T07:22:23.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.637+0000 7f31d9400640 1 --2- 172.21.15.71:0/1322205399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31dc0b4700 0x7f31dc153820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f31c8030000 tx=0x7f31c80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.637+0000 7f31d2000640 1 -- 172.21.15.71:0/1322205399 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8041020 con 0x7f31dc0b4700 2024-09-16T07:22:23.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.638+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc1f43f0 con 0x7f31dc0b4700 2024-09-16T07:22:23.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.640+0000 7f31d2000640 1 -- 172.21.15.71:0/1322205399 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f31c8003ca0 con 0x7f31dc0b4700 2024-09-16T07:22:23.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.640+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc1f49b0 con 0x7f31dc0b4700 2024-09-16T07:22:23.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.640+0000 7f31d2000640 1 -- 172.21.15.71:0/1322205399 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8005240 con 0x7f31dc0b4700 2024-09-16T07:22:23.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.640+0000 7f31d2000640 1 -- 172.21.15.71:0/1322205399 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31c8044020 con 0x7f31dc0b4700 2024-09-16T07:22:23.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.649+0000 7f31d2000640 1 -- 172.21.15.71:0/1322205399 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f31c800a0d0 con 0x7f31dc0b4700 2024-09-16T07:22:23.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.649+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc1f4bf0 con 0x7f31dc0b4700 2024-09-16T07:22:23.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.659+0000 7f31d2000640 1 -- 172.21.15.71:0/1322205399 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f31dc1f4bf0 con 0x7f31dc0b4700 2024-09-16T07:22:23.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.690+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f500406f060 con 0x7f50041f9b40 2024-09-16T07:22:23.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.710+0000 7f616ea00640 1 -- 172.21.15.71:0/523893744 <== mon.1 v2:172.21.15.177:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6170010e50 con 0x7f615c004800 2024-09-16T07:22:23.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.710+0000 7f5000c00640 1 -- 172.21.15.71:0/3079632785 <== mon.1 v2:172.21.15.177:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ff4003b30 con 0x7f50041f9b40 2024-09-16T07:22:23.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.712+0000 7f31d2000640 1 -- 172.21.15.71:0/1322205399 <== mon.0 v2:172.21.15.71:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31c802aba0 con 0x7f31dc0b4700 2024-09-16T07:22:23.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.715+0000 7f5000c00640 1 -- 172.21.15.71:0/3079632785 <== mon.1 v2:172.21.15.177:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f4ff4026080 con 0x7f50041f9b40 2024-09-16T07:22:23.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.721+0000 7fb9bb400640 1 -- 172.21.15.71:0/2723647786 <== mon.2 v2:172.21.15.177:3301/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9bc0333c0 con 0x7fb9c40b2b40 2024-09-16T07:22:23.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.726+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50041f9b40 msgr2=0x7f5004154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.726+0000 7f500b800640 1 --2- 172.21.15.71:0/3079632785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50041f9b40 0x7f5004154f10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5004069420 tx=0x7f4ff400eba0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.726+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 shutdown_connections 2024-09-16T07:22:23.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.726+0000 7f500b800640 1 --2- 172.21.15.71:0/3079632785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50041f9b40 0x7f5004154f10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.726+0000 7f500b800640 1 --2- 172.21.15.71:0/3079632785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f500415afa0 0x7f500415b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.726+0000 7f500b800640 1 --2- 172.21.15.71:0/3079632785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50040aee40 0x7f500415d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.726+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 >> 172.21.15.71:0/3079632785 conn(0x7f50040cfa20 msgr2=0x7f50040afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.745+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 shutdown_connections 2024-09-16T07:22:23.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.752+0000 7f500b800640 1 -- 172.21.15.71:0/3079632785 wait complete. 2024-09-16T07:22:23.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.802+0000 7fccd9800640 1 -- 172.21.15.71:0/2368226696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fccd406f2f0 msgr2=0x7fccd415c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.802+0000 7fccd9800640 1 --2- 172.21.15.71:0/2368226696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fccd406f2f0 0x7fccd415c3f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fccc0005fd0 tx=0x7fccc002a580 comp rx=0 tx=0).stop 2024-09-16T07:22:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.802+0000 7fccd9800640 1 -- 172.21.15.71:0/2368226696 shutdown_connections 2024-09-16T07:22:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.802+0000 7fccd9800640 1 --2- 172.21.15.71:0/2368226696 >> v2:172.21.15.71:3300/0 conn(0x7fccd40cddb0 0x7fccd40ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.802+0000 7fccd9800640 1 --2- 172.21.15.71:0/2368226696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fccd406f2f0 0x7fccd415c3f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.802+0000 7fccd9800640 1 -- 172.21.15.71:0/2368226696 >> 172.21.15.71:0/2368226696 conn(0x7fccd40b1840 msgr2=0x7fccd40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.802+0000 7fccd9800640 1 -- 172.21.15.71:0/2368226696 shutdown_connections 2024-09-16T07:22:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.802+0000 7fccd9800640 1 -- 172.21.15.71:0/2368226696 wait complete. 2024-09-16T07:22:23.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.804+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fb9c40cd660 con 0x7fb9c40b2b40 2024-09-16T07:22:23.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.805+0000 7fccd9800640 1 Processor -- start 2024-09-16T07:22:23.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.806+0000 7fccd9800640 1 -- start start 2024-09-16T07:22:23.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.807+0000 7fccd9800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fccd406f2f0 0x7fccd41fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.807+0000 7fccd9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd406f6d0 0x7fccd41fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.807+0000 7fccd9800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fccd40cddb0 0x7fccd4206de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.816+0000 7fccd2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd406f6d0 0x7fccd41fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.816+0000 7fccd9800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fccd40ce8e0 con 0x7fccd406f6d0 2024-09-16T07:22:23.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.816+0000 7fccd9800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fccd40ce760 con 0x7fccd406f2f0 2024-09-16T07:22:23.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.816+0000 7fccd9800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fccd40cea60 con 0x7fccd40cddb0 2024-09-16T07:22:23.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.817+0000 7fccd2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd406f6d0 0x7fccd41fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44806/0 (socket says 172.21.15.71:44806) 2024-09-16T07:22:23.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.817+0000 7fccd2000640 1 -- 172.21.15.71:0/2947897517 learned_addr learned my addr 172.21.15.71:0/2947897517 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.826+0000 7fccd2000640 1 -- 172.21.15.71:0/2947897517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fccd40cddb0 msgr2=0x7fccd4206de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.826+0000 7fccd2000640 1 --2- 172.21.15.71:0/2947897517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fccd40cddb0 0x7fccd4206de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.826+0000 7fccd2000640 1 -- 172.21.15.71:0/2947897517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fccd406f2f0 msgr2=0x7fccd41fd3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:22:23.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.826+0000 7fccd2000640 1 --2- 172.21.15.71:0/2947897517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fccd406f2f0 0x7fccd41fd3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.826+0000 7fccd2000640 1 -- 172.21.15.71:0/2947897517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd40aa330 con 0x7fccd406f6d0 2024-09-16T07:22:23.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.829+0000 7fccd2000640 1 --2- 172.21.15.71:0/2947897517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd406f6d0 0x7fccd41fd8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00c870 tx=0x7fcccc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.829 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi071') 2024-09-16T07:22:23.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.831+0000 7fb9bb400640 1 -- 172.21.15.71:0/2723647786 <== mon.2 v2:172.21.15.177:3301/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v7)=0 v7) ==== 117+0+0 (secure 0 0 0) 0x7fb9bc0042a0 con 0x7fb9c40b2b40 2024-09-16T07:22:23.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.832+0000 7fcccaa00640 1 -- 172.21.15.71:0/2947897517 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc003b70 con 0x7fccd406f6d0 2024-09-16T07:22:23.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.834+0000 7fcccaa00640 1 -- 172.21.15.71:0/2947897517 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcccc003d10 con 0x7fccd406f6d0 2024-09-16T07:22:23.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.834+0000 7fcccaa00640 1 -- 172.21.15.71:0/2947897517 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc003b70 con 0x7fccd406f6d0 2024-09-16T07:22:23.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.835+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c40b2b40 msgr2=0x7fb9c415af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.835+0000 7fb9ca800640 1 --2- 172.21.15.71:0/2723647786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c40b2b40 0x7fb9c415af60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009000 tx=0x7fb9bc033c80 comp rx=0 tx=0).stop 2024-09-16T07:22:23.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.836+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 shutdown_connections 2024-09-16T07:22:23.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.836+0000 7fb9ca800640 1 --2- 172.21.15.71:0/2723647786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c40b2b40 0x7fb9c415af60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.836+0000 7fb9ca800640 1 --2- 172.21.15.71:0/2723647786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c40af170 0x7fb9c415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.836+0000 7fb9ca800640 1 --2- 172.21.15.71:0/2723647786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c406fba0 0x7fb9c4145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.836+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 >> 172.21.15.71:0/2723647786 conn(0x7fb9c40b1840 msgr2=0x7fb9c40af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.838+0000 7fd70b400640 1 -- 172.21.15.71:0/674453221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec008390 msgr2=0x7fd6ec008c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.838+0000 7fd70b400640 1 --2- 172.21.15.71:0/674453221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec008390 0x7fd6ec008c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000bcf0 tx=0x7fd6ec009c40 comp rx=0 tx=0).stop 2024-09-16T07:22:23.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.838+0000 7fd708a00640 1 -- 172.21.15.71:0/674453221 reap_dead start 2024-09-16T07:22:23.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.838+0000 7fd70b400640 1 -- 172.21.15.71:0/674453221 shutdown_connections 2024-09-16T07:22:23.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.838+0000 7fd70b400640 1 -- 172.21.15.71:0/674453221 >> 172.21.15.71:0/674453221 conn(0x7fd7040cfdd0 msgr2=0x7fd7040d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.839+0000 7fd70b400640 1 -- 172.21.15.71:0/674453221 shutdown_connections 2024-09-16T07:22:23.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.839+0000 7fd70b400640 1 -- 172.21.15.71:0/674453221 wait complete. 2024-09-16T07:22:23.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.841+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd40aa610 con 0x7fccd406f6d0 2024-09-16T07:22:23.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.843+0000 7fcccaa00640 1 -- 172.21.15.71:0/2947897517 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcccc00a7f0 con 0x7fccd406f6d0 2024-09-16T07:22:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.844+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 shutdown_connections 2024-09-16T07:22:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.844+0000 7fb9ca800640 1 -- 172.21.15.71:0/2723647786 wait complete. 2024-09-16T07:22:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.844+0000 7fd70b400640 1 Processor -- start 2024-09-16T07:22:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.844+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd40aabd0 con 0x7fccd406f6d0 2024-09-16T07:22:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.844+0000 7fd70b400640 1 -- start start 2024-09-16T07:22:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.848+0000 7fcccaa00640 1 -- 172.21.15.71:0/2947897517 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcccc003970 con 0x7fccd406f6d0 2024-09-16T07:22:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.847+0000 7fd70b400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6ec008390 0x7fd7041f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.847+0000 7fd70b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7040b56a0 0x7fd70415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.847+0000 7fd70b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7040b3c50 0x7fd704158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.847+0000 7fd70b400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd7040ce6d0 con 0x7fd7040b3c50 2024-09-16T07:22:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.847+0000 7fd70b400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7040d0b00 con 0x7fd7040b56a0 2024-09-16T07:22:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.847+0000 7fd70b400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd7040ce850 con 0x7fd6ec008390 2024-09-16T07:22:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.849+0000 7fd708a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7040b3c50 0x7fd704158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.849+0000 7fd708a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7040b3c50 0x7fd704158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44834/0 (socket says 172.21.15.71:44834) 2024-09-16T07:22:23.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.849+0000 7fd708a00640 1 -- 172.21.15.71:0/3407065654 learned_addr learned my addr 172.21.15.71:0/3407065654 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.850+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f61780cd210 con 0x7f615c004800 2024-09-16T07:22:23.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.853+0000 7fd708a00640 1 -- 172.21.15.71:0/3407065654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6ec008390 msgr2=0x7fd7041f02e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:22:23.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.853+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd40cef10 con 0x7fccd406f6d0 2024-09-16T07:22:23.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.853+0000 7fd708a00640 1 --2- 172.21.15.71:0/3407065654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6ec008390 0x7fd7041f02e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.853+0000 7fd708a00640 1 -- 172.21.15.71:0/3407065654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7040b56a0 msgr2=0x7fd70415b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.853+0000 7fd708a00640 1 --2- 172.21.15.71:0/3407065654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7040b56a0 0x7fd70415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.854+0000 7fd708a00640 1 -- 172.21.15.71:0/3407065654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7041f0820 con 0x7fd7040b3c50 2024-09-16T07:22:23.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.855+0000 7fd708a00640 1 --2- 172.21.15.71:0/3407065654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7040b3c50 0x7fd704158150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7040cc0d0 tx=0x7fd6fc00de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.862+0000 7fd700c00640 1 -- 172.21.15.71:0/3407065654 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc01c070 con 0x7fd7040b3c50 2024-09-16T07:22:23.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.862+0000 7fd700c00640 1 -- 172.21.15.71:0/3407065654 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6fc00bcc0 con 0x7fd7040b3c50 2024-09-16T07:22:23.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.862+0000 7fd700c00640 1 -- 172.21.15.71:0/3407065654 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc01c070 con 0x7fd7040b3c50 2024-09-16T07:22:23.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.864+0000 7fd70b400640 1 -- 172.21.15.71:0/3407065654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7041f0ac0 con 0x7fd7040b3c50 2024-09-16T07:22:23.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.864+0000 7fcccaa00640 1 -- 172.21.15.71:0/2947897517 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcccc021020 con 0x7fccd406f6d0 2024-09-16T07:22:23.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.865+0000 7fd70b400640 1 -- 172.21.15.71:0/3407065654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7041f1080 con 0x7fd7040b3c50 2024-09-16T07:22:23.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.865+0000 7f616ea00640 1 -- 172.21.15.71:0/523893744 <== mon.1 v2:172.21.15.177:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v8)=0 v8) ==== 108+0+0 (secure 0 0 0) 0x7f6170026070 con 0x7f615c004800 2024-09-16T07:22:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.867+0000 7fd700c00640 1 -- 172.21.15.71:0/3407065654 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6fc017430 con 0x7fd7040b3c50 2024-09-16T07:22:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.867+0000 7fd700c00640 1 -- 172.21.15.71:0/3407065654 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6fc020700 con 0x7fd7040b3c50 2024-09-16T07:22:23.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7fcd7a000640 1 -- 172.21.15.71:0/484277991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd74165280 msgr2=0x7fcd7416b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7fcd7a000640 1 --2- 172.21.15.71:0/484277991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd74165280 0x7fcd7416b250 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009830 tx=0x7fcd6002a5d0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f615c004800 msgr2=0x7f61780c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7f6177e00640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f615c004800 0x7f61780c6110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f617000ebf0 tx=0x7f617000c6a0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 shutdown_connections 2024-09-16T07:22:23.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7f6177e00640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780c9070 0x7f61780c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7f6177e00640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61781e2360 0x7f61781e27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.872+0000 7f6177e00640 1 --2- 172.21.15.71:0/523893744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f615c004800 0x7f61780c6110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.872+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 >> 172.21.15.71:0/523893744 conn(0x7f61780cf9f0 msgr2=0x7f6178074030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7fcd7a000640 1 -- 172.21.15.71:0/484277991 shutdown_connections 2024-09-16T07:22:23.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7fcd7a000640 1 --2- 172.21.15.71:0/484277991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd74165280 0x7fcd7416b250 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.871+0000 7fcd7a000640 1 -- 172.21.15.71:0/484277991 >> 172.21.15.71:0/484277991 conn(0x7fcd740b1840 msgr2=0x7fcd740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.872+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 shutdown_connections 2024-09-16T07:22:23.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.872+0000 7f6177e00640 1 -- 172.21.15.71:0/523893744 wait complete. 2024-09-16T07:22:23.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.875+0000 7fd6f6000640 1 -- 172.21.15.71:0/3407065654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d8005290 con 0x7fd7040b3c50 2024-09-16T07:22:23.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.880+0000 7fd700c00640 1 -- 172.21.15.71:0/3407065654 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd6fc025070 con 0x7fd7040b3c50 2024-09-16T07:22:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.887+0000 7fcd7a000640 1 -- 172.21.15.71:0/484277991 shutdown_connections 2024-09-16T07:22:23.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.887+0000 7fcd7a000640 1 -- 172.21.15.71:0/484277991 wait complete. 2024-09-16T07:22:23.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.890+0000 7fcd7a000640 1 Processor -- start 2024-09-16T07:22:23.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.896+0000 7fcd7a000640 1 -- start start 2024-09-16T07:22:23.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.905+0000 7fcd7a000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd7406fb90 0x7fcd74204c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.905+0000 7fcd7a000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd740b3aa0 0x7fcd742051a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.905+0000 7fcd73400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd7406fb90 0x7fcd74204c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.905+0000 7fcd73400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd7406fb90 0x7fcd74204c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:57510/0 (socket says 172.21.15.71:57510) 2024-09-16T07:22:23.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.910+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd740b3aa0 0x7fcd742051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.910+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd740b3aa0 0x7fcd742051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:37350/0 (socket says 172.21.15.71:37350) 2024-09-16T07:22:23.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.910+0000 7fcd73400640 1 -- 172.21.15.71:0/187772902 learned_addr learned my addr 172.21.15.71:0/187772902 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.913+0000 7fcd7a000640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd74165280 0x7fcd74162370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.913+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7408b790 con 0x7fcd74165280 2024-09-16T07:22:23.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.913+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd740cefd0 con 0x7fcd7406fb90 2024-09-16T07:22:23.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.913+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7408b910 con 0x7fcd740b3aa0 2024-09-16T07:22:23.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.913+0000 7fcd73e00640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd74165280 0x7fcd74162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.914+0000 7fcd73e00640 1 -- 172.21.15.71:0/187772902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd740b3aa0 msgr2=0x7fcd742051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.914+0000 7fcd73e00640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd740b3aa0 0x7fcd742051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.914+0000 7fcd73e00640 1 -- 172.21.15.71:0/187772902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd7406fb90 msgr2=0x7fcd74204c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.914+0000 7fcd73e00640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd7406fb90 0x7fcd74204c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.914+0000 7fcd73e00640 1 -- 172.21.15.71:0/187772902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd741637f0 con 0x7fcd74165280 2024-09-16T07:22:23.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.915+0000 7fcd73e00640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd74165280 0x7fcd74162370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00ca70 tx=0x7fcd6c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.915+0000 7fcd6b400640 1 -- 172.21.15.71:0/187772902 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c027020 con 0x7fcd74165280 2024-09-16T07:22:23.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.915+0000 7fcd6b400640 1 -- 172.21.15.71:0/187772902 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd6c00aae0 con 0x7fcd74165280 2024-09-16T07:22:23.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.915+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd74163230 con 0x7fcd74165280 2024-09-16T07:22:23.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.915+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd74202840 con 0x7fcd74165280 2024-09-16T07:22:23.924 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi071') 2024-09-16T07:22:23.924 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi071') 2024-09-16T07:22:23.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.916+0000 7fcd6b400640 1 -- 172.21.15.71:0/187772902 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c023430 con 0x7fcd74165280 2024-09-16T07:22:23.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.917+0000 7fcd6b400640 1 -- 172.21.15.71:0/187772902 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd6c02a020 con 0x7fcd74165280 2024-09-16T07:22:23.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.917+0000 7fcd6b400640 1 -- 172.21.15.71:0/187772902 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd6c02cb10 con 0x7fcd74165280 2024-09-16T07:22:23.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.926+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4c005290 con 0x7fcd74165280 2024-09-16T07:22:23.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.929+0000 7fdaf9a00640 1 -- 172.21.15.71:0/2509633554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40cb840 msgr2=0x7fdaf40cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.929+0000 7fdaf9a00640 1 --2- 172.21.15.71:0/2509633554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40cb840 0x7fdaf40cbc20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40cd4b0 tx=0x7fdae002a580 comp rx=0 tx=0).stop 2024-09-16T07:22:23.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.929+0000 7fdaf9a00640 1 -- 172.21.15.71:0/2509633554 shutdown_connections 2024-09-16T07:22:23.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.929+0000 7fdaf9a00640 1 --2- 172.21.15.71:0/2509633554 >> v2:172.21.15.71:3300/0 conn(0x7fdaf406e5a0 0x7fdaf406e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.929+0000 7fdaf9a00640 1 --2- 172.21.15.71:0/2509633554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40cb840 0x7fdaf40cbc20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.929+0000 7fdaf9a00640 1 -- 172.21.15.71:0/2509633554 >> 172.21.15.71:0/2509633554 conn(0x7fdaf40cf9f0 msgr2=0x7fdaf40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.929+0000 7fdaf9a00640 1 -- 172.21.15.71:0/2509633554 shutdown_connections 2024-09-16T07:22:23.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.929+0000 7fdaf9a00640 1 -- 172.21.15.71:0/2509633554 wait complete. 2024-09-16T07:22:23.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.935+0000 7fcd6b400640 1 -- 172.21.15.71:0/187772902 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcd6c031020 con 0x7fcd74165280 2024-09-16T07:22:23.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.933+0000 7fdaf9a00640 1 Processor -- start 2024-09-16T07:22:23.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.935+0000 7fdaf9a00640 1 -- start start 2024-09-16T07:22:23.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.937+0000 7fdaf9a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdaf406e5a0 0x7fdaf4151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.937+0000 7fdaf9a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40ca040 0x7fdaf4152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.937+0000 7fdaf9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf40cb840 0x7fdaf4150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.937+0000 7fdaf9a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf40b3970 con 0x7fdaf40cb840 2024-09-16T07:22:23.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.937+0000 7fdaf9a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf40cadd0 con 0x7fdaf40ca040 2024-09-16T07:22:23.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.937+0000 7fdaf9a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf40b3af0 con 0x7fdaf406e5a0 2024-09-16T07:22:23.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.939+0000 7fdaf9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf40cb840 0x7fdaf4150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.939+0000 7fdaf9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf40cb840 0x7fdaf4150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44864/0 (socket says 172.21.15.71:44864) 2024-09-16T07:22:23.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.939+0000 7fdaf9000640 1 -- 172.21.15.71:0/1052171307 learned_addr learned my addr 172.21.15.71:0/1052171307 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.939+0000 7fdaf3400640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40ca040 0x7fdaf4152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.941+0000 7fc417200640 1 -- 172.21.15.71:0/1209649813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc418070650 msgr2=0x7fc4180ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.941+0000 7fc417200640 1 --2- 172.21.15.71:0/1209649813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc418070650 0x7fc4180ce1e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc41806e610 tx=0x7fc40802a580 comp rx=0 tx=0).stop 2024-09-16T07:22:23.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.940+0000 7fdaf3400640 1 -- 172.21.15.71:0/1052171307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdaf406e5a0 msgr2=0x7fdaf4151d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:22:23.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.940+0000 7fdaf3400640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdaf406e5a0 0x7fdaf4151d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.940+0000 7fdaf3400640 1 -- 172.21.15.71:0/1052171307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf40cb840 msgr2=0x7fdaf4150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.940+0000 7fdaf3400640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf40cb840 0x7fdaf4150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.940+0000 7fdaf3400640 1 -- 172.21.15.71:0/1052171307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf41522b0 con 0x7fdaf40ca040 2024-09-16T07:22:23.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.941+0000 7fdaf9000640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf40cb840 0x7fdaf4150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:23.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.942+0000 7fdaf3400640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40ca040 0x7fdaf4152690 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b3f0 tx=0x7fdae400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.942+0000 7fc417200640 1 -- 172.21.15.71:0/1209649813 shutdown_connections 2024-09-16T07:22:23.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.942+0000 7fc417200640 1 --2- 172.21.15.71:0/1209649813 >> v2:172.21.15.71:3300/0 conn(0x7fc41806e940 0x7fc41806edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.942+0000 7fdaebe00640 1 -- 172.21.15.71:0/1052171307 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4015020 con 0x7fdaf40ca040 2024-09-16T07:22:23.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.942+0000 7fc417200640 1 --2- 172.21.15.71:0/1209649813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc418070650 0x7fc4180ce1e0 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc41806e610 tx=0x7fc40802a580 comp rx=0 tx=0).stop 2024-09-16T07:22:23.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.942+0000 7fdaebe00640 1 -- 172.21.15.71:0/1052171307 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdae4011920 con 0x7fdaf40ca040 2024-09-16T07:22:23.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.942+0000 7fdaebe00640 1 -- 172.21.15.71:0/1052171307 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4011d70 con 0x7fdaf40ca040 2024-09-16T07:22:23.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.942+0000 7fc417200640 1 -- 172.21.15.71:0/1209649813 >> 172.21.15.71:0/1209649813 conn(0x7fc4180cf9f0 msgr2=0x7fc4180cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.944+0000 7fc417200640 1 -- 172.21.15.71:0/1209649813 shutdown_connections 2024-09-16T07:22:23.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.944+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf40cf040 con 0x7fdaf40ca040 2024-09-16T07:22:23.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.944+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf41579c0 con 0x7fdaf40ca040 2024-09-16T07:22:23.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.944+0000 7fdaebe00640 1 -- 172.21.15.71:0/1052171307 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdae4018020 con 0x7fdaf40ca040 2024-09-16T07:22:23.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.946+0000 7fdaebe00640 1 -- 172.21.15.71:0/1052171307 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdae4019ab0 con 0x7fdaf40ca040 2024-09-16T07:22:23.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.944+0000 7fc417200640 1 -- 172.21.15.71:0/1209649813 wait complete. 2024-09-16T07:22:23.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.946+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf4072f90 con 0x7fdaf40ca040 2024-09-16T07:22:23.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.950+0000 7fc417200640 1 Processor -- start 2024-09-16T07:22:23.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.951+0000 7fc417200640 1 -- start start 2024-09-16T07:22:23.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.952+0000 7fdaebe00640 1 -- 172.21.15.71:0/1052171307 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fdaf4072f90 con 0x7fdaf40ca040 2024-09-16T07:22:23.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.955+0000 7fc417200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc41806e940 0x7fc41814e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.956+0000 7fc417200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc41815dc00 0x7fc41814ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.956+0000 7fc417200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc41814c360 0x7fc41814cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc417200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc41806f680 con 0x7fc41815dc00 2024-09-16T07:22:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc417200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc41806f500 con 0x7fc41806e940 2024-09-16T07:22:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc417200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc41806f800 con 0x7fc41814c360 2024-09-16T07:22:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc416800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc41814c360 0x7fc41814cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc416800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc41814c360 0x7fc41814cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:37362/0 (socket says 172.21.15.71:37362) 2024-09-16T07:22:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc416800640 1 -- 172.21.15.71:0/2922611140 learned_addr learned my addr 172.21.15.71:0/2922611140 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc415e00640 1 --2- 172.21.15.71:0/2922611140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc41806e940 0x7fc41814e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc416800640 1 -- 172.21.15.71:0/2922611140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc41806e940 msgr2=0x7fc41814e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc416800640 1 --2- 172.21.15.71:0/2922611140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc41806e940 0x7fc41814e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc416800640 1 -- 172.21.15.71:0/2922611140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc41815dc00 msgr2=0x7fc41814ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc416800640 1 --2- 172.21.15.71:0/2922611140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc41815dc00 0x7fc41814ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.957+0000 7fc416800640 1 -- 172.21.15.71:0/2922611140 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41814f450 con 0x7fc41814c360 2024-09-16T07:22:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.958+0000 7fc416800640 1 --2- 172.21.15.71:0/2922611140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc41814c360 0x7fc41814cf70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc404009920 tx=0x7fc404009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.958+0000 7fc40e000640 1 -- 172.21.15.71:0/2922611140 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40401d070 con 0x7fc41814c360 2024-09-16T07:22:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.958+0000 7fc40e000640 1 -- 172.21.15.71:0/2922611140 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc404010a90 con 0x7fc41814c360 2024-09-16T07:22:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.958+0000 7fc40e000640 1 -- 172.21.15.71:0/2922611140 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404018440 con 0x7fc41814c360 2024-09-16T07:22:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.959+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41814f730 con 0x7fc41814c360 2024-09-16T07:22:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.959+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41814a240 con 0x7fc41814c360 2024-09-16T07:22:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.967+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f31dc06fd40 con 0x7f31dc0b4700 2024-09-16T07:22:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.968+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41814a440 con 0x7fc41814c360 2024-09-16T07:22:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.968+0000 7fc40e000640 1 -- 172.21.15.71:0/2922611140 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc404020070 con 0x7fc41814c360 2024-09-16T07:22:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.968+0000 7fc40e000640 1 -- 172.21.15.71:0/2922611140 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc404022bb0 con 0x7fc41814c360 2024-09-16T07:22:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.967+0000 7f355bc00640 1 -- 172.21.15.71:0/3993623833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3554161590 msgr2=0x7f35540b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.967+0000 7f355bc00640 1 --2- 172.21.15.71:0/3993623833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3554161590 0x7f35540b6e00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3540009830 tx=0x7f354002a560 comp rx=0 tx=0).stop 2024-09-16T07:22:23.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.971+0000 7f355bc00640 1 -- 172.21.15.71:0/3993623833 shutdown_connections 2024-09-16T07:22:23.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.971+0000 7f355bc00640 1 --2- 172.21.15.71:0/3993623833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35540b5630 0x7f35540cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.971+0000 7f355bc00640 1 --2- 172.21.15.71:0/3993623833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3554161590 0x7f35540b6e00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.973+0000 7fc40e000640 1 -- 172.21.15.71:0/2922611140 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc41814a440 con 0x7fc41814c360 2024-09-16T07:22:23.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.974+0000 7f31d2000640 1 -- 172.21.15.71:0/1322205399 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f31dc1f4bf0 con 0x7f31dc0b4700 2024-09-16T07:22:23.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.971+0000 7f355bc00640 1 -- 172.21.15.71:0/3993623833 >> 172.21.15.71:0/3993623833 conn(0x7f35540d3fb0 msgr2=0x7f35540d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.975+0000 7f355bc00640 1 -- 172.21.15.71:0/3993623833 shutdown_connections 2024-09-16T07:22:23.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.975+0000 7f355bc00640 1 -- 172.21.15.71:0/3993623833 wait complete. 2024-09-16T07:22:23.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.978+0000 7f355bc00640 1 Processor -- start 2024-09-16T07:22:23.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.978+0000 7f355bc00640 1 -- start start 2024-09-16T07:22:23.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.980+0000 7f355bc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35540b5630 0x7f355409e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f355bc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35540b7340 0x7f35540b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f355bc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3554161590 0x7f35540b9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:23.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31dc0b4700 msgr2=0x7f31dc153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f31e0a00640 1 --2- 172.21.15.71:0/1322205399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31dc0b4700 0x7f31dc153820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f31c8030000 tx=0x7f31c80038a0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 shutdown_connections 2024-09-16T07:22:23.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f31e0a00640 1 --2- 172.21.15.71:0/1322205399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31dc0b4ae0 0x7f31dc1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f31e0a00640 1 --2- 172.21.15.71:0/1322205399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31dc0b4700 0x7f31dc153820 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f31e0a00640 1 --2- 172.21.15.71:0/1322205399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31dc06f8d0 0x7f31dc1f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 >> 172.21.15.71:0/1322205399 conn(0x7f31dc0b1840 msgr2=0x7f31dc0ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 shutdown_connections 2024-09-16T07:22:23.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f355bc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f35540cfe80 con 0x7f35540b7340 2024-09-16T07:22:23.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.982+0000 7f31e0a00640 1 -- 172.21.15.71:0/1322205399 wait complete. 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f355bc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f35540cfd00 con 0x7f35540b5630 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.981+0000 7f355bc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3554162430 con 0x7f3554161590 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.982+0000 7f3559c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3554161590 0x7f35540b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.982+0000 7f3559c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3554161590 0x7f35540b9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:37396/0 (socket says 172.21.15.71:37396) 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.982+0000 7f3559c00640 1 -- 172.21.15.71:0/31967439 learned_addr learned my addr 172.21.15.71:0/31967439 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.982+0000 7f3553e00640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35540b7340 0x7f35540b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.982+0000 7f3559200640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35540b5630 0x7f355409e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.983+0000 7f3553e00640 1 -- 172.21.15.71:0/31967439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3554161590 msgr2=0x7f35540b9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.983+0000 7f3553e00640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3554161590 0x7f35540b9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.983+0000 7f3553e00640 1 -- 172.21.15.71:0/31967439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35540b5630 msgr2=0x7f355409e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.983+0000 7f3553e00640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35540b5630 0x7f355409e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.983+0000 7f3553e00640 1 -- 172.21.15.71:0/31967439 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540a08e0 con 0x7f35540b7340 2024-09-16T07:22:23.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.984+0000 7fcd38a00640 1 -- 172.21.15.71:0/2285230639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd340cf430 msgr2=0x7fcd340ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.984+0000 7fcd31400640 1 -- 172.21.15.71:0/2285230639 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0305f0 con 0x7fcd340cf430 2024-09-16T07:22:23.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.984+0000 7fcd38a00640 1 --2- 172.21.15.71:0/2285230639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd340cf430 0x7fcd340ae960 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009be0 tx=0x7fcd2c02b9a0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.985+0000 7f3553e00640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35540b7340 0x7f35540b8f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f354800e9f0 tx=0x7f354800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:23.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.986+0000 7f3550c00640 1 -- 172.21.15.71:0/31967439 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354800ccc0 con 0x7f35540b7340 2024-09-16T07:22:23.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.986+0000 7f3550c00640 1 -- 172.21.15.71:0/31967439 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f354800ce60 con 0x7f35540b7340 2024-09-16T07:22:23.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.986+0000 7f3550c00640 1 -- 172.21.15.71:0/31967439 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548004280 con 0x7f35540b7340 2024-09-16T07:22:23.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.986+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35542053b0 con 0x7f35540b7340 2024-09-16T07:22:23.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.987+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3554205970 con 0x7f35540b7340 2024-09-16T07:22:23.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.988+0000 7fcd38a00640 1 -- 172.21.15.71:0/2285230639 shutdown_connections 2024-09-16T07:22:23.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.988+0000 7fcd38a00640 1 --2- 172.21.15.71:0/2285230639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd340cf430 0x7fcd340ae960 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.988+0000 7fcd38a00640 1 -- 172.21.15.71:0/2285230639 >> 172.21.15.71:0/2285230639 conn(0x7fcd340cf9f0 msgr2=0x7fcd340cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.989+0000 7f3550c00640 1 -- 172.21.15.71:0/31967439 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3548004b70 con 0x7f35540b7340 2024-09-16T07:22:23.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.989+0000 7f3550c00640 1 -- 172.21.15.71:0/31967439 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3548003c00 con 0x7f35540b7340 2024-09-16T07:22:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.991+0000 7fcd38a00640 1 -- 172.21.15.71:0/2285230639 shutdown_connections 2024-09-16T07:22:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.994+0000 7fe1bc200640 1 -- 172.21.15.71:0/2552692014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b40cddb0 msgr2=0x7fe1b40ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.994+0000 7fe1bc200640 1 --2- 172.21.15.71:0/2552692014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b40cddb0 0x7fe1b40ce200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac005fd0 tx=0x7fe1ac02a840 comp rx=0 tx=0).stop 2024-09-16T07:22:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.990+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3554205b70 con 0x7f35540b7340 2024-09-16T07:22:23.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.995+0000 7f3550c00640 1 -- 172.21.15.71:0/31967439 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3548004420 con 0x7f35540b7340 2024-09-16T07:22:23.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.996+0000 7fe1bc200640 1 -- 172.21.15.71:0/2552692014 shutdown_connections 2024-09-16T07:22:23.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.996+0000 7fe1bc200640 1 --2- 172.21.15.71:0/2552692014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b40cddb0 0x7fe1b40ce200 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.996+0000 7fe1bc200640 1 --2- 172.21.15.71:0/2552692014 >> v2:172.21.15.71:3300/0 conn(0x7fe1b415c3f0 0x7fe1b40733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.996+0000 7fe1bc200640 1 -- 172.21.15.71:0/2552692014 >> 172.21.15.71:0/2552692014 conn(0x7fe1b40b1840 msgr2=0x7fe1b40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.996+0000 7fe1bc200640 1 -- 172.21.15.71:0/2552692014 shutdown_connections 2024-09-16T07:22:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.997+0000 7fe1bc200640 1 -- 172.21.15.71:0/2552692014 wait complete. 2024-09-16T07:22:23.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:23.998+0000 7fcd38a00640 1 -- 172.21.15.71:0/2285230639 wait complete. 2024-09-16T07:22:23.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.000+0000 7fe1bc200640 1 Processor -- start 2024-09-16T07:22:23.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.000+0000 7fe1bc200640 1 -- start start 2024-09-16T07:22:24.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.002+0000 7fe1bc200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b406f2f0 0x7fe1b41441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.003+0000 7fe1bc200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1b40cddb0 0x7fe1b415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.003+0000 7fe1b9800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b406f2f0 0x7fe1b41441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.003+0000 7fe1b9800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b406f2f0 0x7fe1b41441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:37412/0 (socket says 172.21.15.71:37412) 2024-09-16T07:22:24.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.003+0000 7fe1b9800640 1 -- 172.21.15.71:0/2940073569 learned_addr learned my addr 172.21.15.71:0/2940073569 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:24.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.004+0000 7fe1b8e00640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1b40cddb0 0x7fe1b415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.004+0000 7fe1bc200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1b415c3f0 0x7fe1b4144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.005+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b40cead0 con 0x7fe1b40cddb0 2024-09-16T07:22:24.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.005+0000 7fe1ba200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1b415c3f0 0x7fe1b4144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.005+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b40ce950 con 0x7fe1b415c3f0 2024-09-16T07:22:24.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.005+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b40cec50 con 0x7fe1b406f2f0 2024-09-16T07:22:24.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.005+0000 7fc243e00640 1 -- 172.21.15.71:0/3679291362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc228003a00 msgr2=0x7fc228003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.006+0000 7fe1ba200640 1 -- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b406f2f0 msgr2=0x7fe1b41441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.006+0000 7fe1ba200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b406f2f0 0x7fe1b41441f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.006+0000 7fe1ba200640 1 -- 172.21.15.71:0/2940073569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1b40cddb0 msgr2=0x7fe1b415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.007+0000 7fc243e00640 1 --2- 172.21.15.71:0/3679291362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc228003a00 0x7fc228003e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc234009880 tx=0x7fc228005830 comp rx=0 tx=0).stop 2024-09-16T07:22:24.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.007+0000 7fc243400640 1 -- 172.21.15.71:0/3679291362 reap_dead start 2024-09-16T07:22:24.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.007+0000 7fe1ba200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1b40cddb0 0x7fe1b415b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.007+0000 7fe1ba200640 1 -- 172.21.15.71:0/2940073569 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b4144730 con 0x7fe1b415c3f0 2024-09-16T07:22:24.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.007+0000 7fe1ba200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1b415c3f0 0x7fe1b4144a10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009000 tx=0x7fe1ac031600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:24.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.007+0000 7fe1b8e00640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1b40cddb0 0x7fe1b415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:24.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.008+0000 7fe1b1600640 1 -- 172.21.15.71:0/2940073569 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac041070 con 0x7fe1b415c3f0 2024-09-16T07:22:24.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.008+0000 7fe1b1600640 1 -- 172.21.15.71:0/2940073569 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1ac031b10 con 0x7fe1b415c3f0 2024-09-16T07:22:24.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.008+0000 7fe1b1600640 1 -- 172.21.15.71:0/2940073569 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac037040 con 0x7fe1b415c3f0 2024-09-16T07:22:24.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.008+0000 7fc243e00640 1 -- 172.21.15.71:0/3679291362 shutdown_connections 2024-09-16T07:22:24.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.009+0000 7fc243e00640 1 -- 172.21.15.71:0/3679291362 >> 172.21.15.71:0/3679291362 conn(0x7fc2440d3e20 msgr2=0x7fc2440d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.009+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b415a020 con 0x7fe1b415c3f0 2024-09-16T07:22:24.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.010+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b415a5e0 con 0x7fe1b415c3f0 2024-09-16T07:22:24.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.013+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b40704c0 con 0x7fe1b415c3f0 2024-09-16T07:22:24.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.015+0000 7fcd38a00640 1 Processor -- start 2024-09-16T07:22:24.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.015+0000 7fcd38a00640 1 -- start start 2024-09-16T07:22:24.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.016+0000 7fc243e00640 1 -- 172.21.15.71:0/3679291362 shutdown_connections 2024-09-16T07:22:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.017+0000 7f2344a00640 1 -- 172.21.15.71:0/1528578218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23400b5270 msgr2=0x7f23400d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.017+0000 7f2344a00640 1 --2- 172.21.15.71:0/1528578218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23400b5270 0x7f23400d26f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f232c009830 tx=0x7f232c02a410 comp rx=0 tx=0).stop 2024-09-16T07:22:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.017+0000 7f2344a00640 1 -- 172.21.15.71:0/1528578218 shutdown_connections 2024-09-16T07:22:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.017+0000 7f2344a00640 1 --2- 172.21.15.71:0/1528578218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23400b5270 0x7f23400d26f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.017+0000 7f2344a00640 1 --2- 172.21.15.71:0/1528578218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2340161460 0x7f23400d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.017+0000 7f2344a00640 1 -- 172.21.15.71:0/1528578218 >> 172.21.15.71:0/1528578218 conn(0x7f234009d0b0 msgr2=0x7f23400d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.018+0000 7f2344a00640 1 -- 172.21.15.71:0/1528578218 shutdown_connections 2024-09-16T07:22:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.018+0000 7fcd38a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd340ce0c0 0x7fcd341f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.019 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.019+0000 7fcd32800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd340ce0c0 0x7fcd341f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fe1b1600640 1 -- 172.21.15.71:0/2940073569 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1ac03a070 con 0x7fe1b415c3f0 2024-09-16T07:22:24.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fe1b1600640 1 -- 172.21.15.71:0/2940073569 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1ac004140 con 0x7fe1b415c3f0 2024-09-16T07:22:24.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fe1b1600640 1 -- 172.21.15.71:0/2940073569 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe1b40704c0 con 0x7fe1b415c3f0 2024-09-16T07:22:24.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.019+0000 7fc243e00640 1 -- 172.21.15.71:0/3679291362 wait complete. 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.019+0000 7fcd32800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd340ce0c0 0x7fcd341f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44902/0 (socket says 172.21.15.71:44902) 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.019+0000 7fcd32800640 1 -- 172.21.15.71:0/4256998521 learned_addr learned my addr 172.21.15.71:0/4256998521 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.019+0000 7fcd38a00640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd340cf430 0x7fcd342134c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fcd38a00640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd3416f0f0 0x7fcd341f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd340cf1b0 con 0x7fcd340ce0c0 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd340cf030 con 0x7fcd340cf430 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fcd31e00640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd340cf430 0x7fcd342134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3416f580 con 0x7fcd3416f0f0 2024-09-16T07:22:24.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.020+0000 7fcd33200640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd3416f0f0 0x7fcd341f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.022+0000 7fc243e00640 1 Processor -- start 2024-09-16T07:22:24.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.022+0000 7fc243e00640 1 -- start start 2024-09-16T07:22:24.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.018+0000 7f2344a00640 1 -- 172.21.15.71:0/1528578218 wait complete. 2024-09-16T07:22:24.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.021+0000 7f2344a00640 1 Processor -- start 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.023+0000 7f2344a00640 1 -- start start 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f2344a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23400b4d40 0x7f23401f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f2344a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23400b5270 0x7f23400caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f2344a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f234015e180 0x7f23400cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f2344a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f23400b3ac0 con 0x7f234015e180 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f2344a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23400d1220 con 0x7f23400b5270 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f2344a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f23400b3c40 con 0x7f23400b4d40 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f233de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23400b5270 0x7f23400caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f233de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23400b5270 0x7f23400caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:57598/0 (socket says 172.21.15.71:57598) 2024-09-16T07:22:24.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.025+0000 7f233de00640 1 -- 172.21.15.71:0/2806207271 learned_addr learned my addr 172.21.15.71:0/2806207271 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:24.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.026+0000 7f233e800640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23400b4d40 0x7f23401f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.027+0000 7f233de00640 1 -- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23400b4d40 msgr2=0x7f23401f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.027+0000 7f233de00640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23400b4d40 0x7f23401f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.027+0000 7f233de00640 1 -- 172.21.15.71:0/2806207271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f234015e180 msgr2=0x7f23400cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.027+0000 7f233f200640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f234015e180 0x7f23400cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fcd32800640 1 -- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd3416f0f0 msgr2=0x7fcd341f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fcd32800640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd3416f0f0 0x7fcd341f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fcd32800640 1 -- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd340cf430 msgr2=0x7fcd342134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fc243e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc244156020 0x7fc2441580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7f233de00640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f234015e180 0x7f23400cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7f233de00640 1 -- 172.21.15.71:0/2806207271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2340201670 con 0x7f23400b5270 2024-09-16T07:22:24.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fc243e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2441f95d0 0x7fc244158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fc243e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2441fbed0 0x7fc2441fc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fc242a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc244156020 0x7fc2441580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fc242a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc244156020 0x7fc2441580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:37446/0 (socket says 172.21.15.71:37446) 2024-09-16T07:22:24.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.029+0000 7fc242a00640 1 -- 172.21.15.71:0/3268903882 learned_addr learned my addr 172.21.15.71:0/3268903882 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:24.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.029+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441621c0 con 0x7fc2441f95d0 2024-09-16T07:22:24.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.029+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc244162040 con 0x7fc2441fbed0 2024-09-16T07:22:24.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.029+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc244162340 con 0x7fc244156020 2024-09-16T07:22:24.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.029+0000 7fc243400640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2441fbed0 0x7fc2441fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fcd32800640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd340cf430 0x7fcd342134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.028+0000 7fcd32800640 1 -- 172.21.15.71:0/4256998521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd341f8290 con 0x7fcd340ce0c0 2024-09-16T07:22:24.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.029+0000 7fcd31e00640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd340cf430 0x7fcd342134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:24.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.029+0000 7fc242000640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2441f95d0 0x7fc244158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.029+0000 7f233e800640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23400b4d40 0x7f23401f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:24.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.030+0000 7f233f200640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f234015e180 0x7f23400cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:22:24.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.030+0000 7f233de00640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23400b5270 0x7f23400caec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f233000c460 tx=0x7f233000c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:24.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.030+0000 7fcd32800640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd340ce0c0 0x7fcd341f7b90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c006010 tx=0x7fcd2c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:24.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.031+0000 7f2336a00640 1 -- 172.21.15.71:0/2806207271 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330017070 con 0x7f23400b5270 2024-09-16T07:22:24.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.031+0000 7f2336a00640 1 -- 172.21.15.71:0/2806207271 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23300035f0 con 0x7f23400b5270 2024-09-16T07:22:24.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.031+0000 7f2336a00640 1 -- 172.21.15.71:0/2806207271 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330003a40 con 0x7f23400b5270 2024-09-16T07:22:24.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.032+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23400ccc30 con 0x7f23400b5270 2024-09-16T07:22:24.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.033+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2340201bf0 con 0x7f23400b5270 2024-09-16T07:22:24.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.034+0000 7f2336a00640 1 -- 172.21.15.71:0/2806207271 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f233001a070 con 0x7f23400b5270 2024-09-16T07:22:24.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.034+0000 7f2336a00640 1 -- 172.21.15.71:0/2806207271 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2330012740 con 0x7f23400b5270 2024-09-16T07:22:24.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.034+0000 7fcd2aa00640 1 -- 172.21.15.71:0/4256998521 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c046070 con 0x7fcd340ce0c0 2024-09-16T07:22:24.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.034+0000 7fcd2aa00640 1 -- 172.21.15.71:0/4256998521 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd2c004250 con 0x7fcd340ce0c0 2024-09-16T07:22:24.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.034+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd341f8570 con 0x7fcd340ce0c0 2024-09-16T07:22:24.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.034+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3416a5e0 con 0x7fcd340ce0c0 2024-09-16T07:22:24.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.030+0000 7fc242a00640 1 -- 172.21.15.71:0/3268903882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2441fbed0 msgr2=0x7fc2441fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.030+0000 7fc242a00640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2441fbed0 0x7fc2441fc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.030+0000 7fc242a00640 1 -- 172.21.15.71:0/3268903882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2441f95d0 msgr2=0x7fc244158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.035+0000 7fc242a00640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2441f95d0 0x7fc244158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.034+0000 7fcd2aa00640 1 -- 172.21.15.71:0/4256998521 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0046a0 con 0x7fcd340ce0c0 2024-09-16T07:22:24.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.035+0000 7fcd2aa00640 1 -- 172.21.15.71:0/4256998521 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd2c030730 con 0x7fcd340ce0c0 2024-09-16T07:22:24.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.035+0000 7fcd2aa00640 1 -- 172.21.15.71:0/4256998521 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd2c031660 con 0x7fcd340ce0c0 2024-09-16T07:22:24.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.035+0000 7fc242a00640 1 -- 172.21.15.71:0/3268903882 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2441fcbe0 con 0x7fc244156020 2024-09-16T07:22:24.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.035+0000 7fc242000640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2441f95d0 0x7fc244158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:24.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.035+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23400b4180 con 0x7f23400b5270 2024-09-16T07:22:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.036+0000 7fc242a00640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc244156020 0x7fc2441580e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc23402a820 tx=0x7fc234032bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.036+0000 7fc23aa00640 1 -- 172.21.15.71:0/3268903882 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234037040 con 0x7fc244156020 2024-09-16T07:22:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.036+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0c005290 con 0x7fcd340ce0c0 2024-09-16T07:22:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.036+0000 7fc23aa00640 1 -- 172.21.15.71:0/3268903882 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc23403d470 con 0x7fc244156020 2024-09-16T07:22:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.036+0000 7fc23aa00640 1 -- 172.21.15.71:0/3268903882 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23403d880 con 0x7fc244156020 2024-09-16T07:22:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.038+0000 7f2336a00640 1 -- 172.21.15.71:0/2806207271 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2330003be0 con 0x7f23400b5270 2024-09-16T07:22:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.040+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2441f8af0 con 0x7fc244156020 2024-09-16T07:22:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.040+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2441fc8c0 con 0x7fc244156020 2024-09-16T07:22:24.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.044+0000 7fcd2aa00640 1 -- 172.21.15.71:0/4256998521 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcd2c034020 con 0x7fcd340ce0c0 2024-09-16T07:22:24.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.044+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24415c610 con 0x7fc244156020 2024-09-16T07:22:24.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.044+0000 7fc23aa00640 1 -- 172.21.15.71:0/3268903882 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc234046450 con 0x7fc244156020 2024-09-16T07:22:24.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.045+0000 7fc23aa00640 1 -- 172.21.15.71:0/3268903882 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc23403ca50 con 0x7fc244156020 2024-09-16T07:22:24.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.049+0000 7fc23aa00640 1 -- 172.21.15.71:0/3268903882 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc23404b060 con 0x7fc244156020 2024-09-16T07:22:24.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.058+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fccd415c1d0 con 0x7fccd406f6d0 2024-09-16T07:22:24.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.059+0000 7fd6f6000640 1 -- 172.21.15.71:0/3407065654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fd6d8005580 con 0x7fd7040b3c50 2024-09-16T07:22:24.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.063+0000 7fcccaa00640 1 -- 172.21.15.71:0/2947897517 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7fcccc00ae20 con 0x7fccd406f6d0 2024-09-16T07:22:24.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.072+0000 7fd700c00640 1 -- 172.21.15.71:0/3407065654 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7fd6fc006070 con 0x7fd7040b3c50 2024-09-16T07:22:24.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd406f6d0 msgr2=0x7fccd41fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fccd9800640 1 --2- 172.21.15.71:0/2947897517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd406f6d0 0x7fccd41fd8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00c870 tx=0x7fcccc00cd30 comp rx=0 tx=0).stop 2024-09-16T07:22:24.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fd6f6000640 1 -- 172.21.15.71:0/3407065654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7040b3c50 msgr2=0x7fd704158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fd6f6000640 1 --2- 172.21.15.71:0/3407065654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7040b3c50 0x7fd704158150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7040cc0d0 tx=0x7fd6fc00de30 comp rx=0 tx=0).stop 2024-09-16T07:22:24.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 shutdown_connections 2024-09-16T07:22:24.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fccd9800640 1 --2- 172.21.15.71:0/2947897517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fccd40cddb0 0x7fccd4206de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fccd9800640 1 --2- 172.21.15.71:0/2947897517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd406f6d0 0x7fccd41fd8f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fccd9800640 1 --2- 172.21.15.71:0/2947897517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fccd406f2f0 0x7fccd41fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 >> 172.21.15.71:0/2947897517 conn(0x7fccd40b1840 msgr2=0x7fccd4155900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fd6f6000640 1 -- 172.21.15.71:0/3407065654 shutdown_connections 2024-09-16T07:22:24.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fd6f6000640 1 --2- 172.21.15.71:0/3407065654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7040b3c50 0x7fd704158150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fd6f6000640 1 --2- 172.21.15.71:0/3407065654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7040b56a0 0x7fd70415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.074+0000 7fd6f6000640 1 --2- 172.21.15.71:0/3407065654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6ec008390 0x7fd7041f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.075+0000 7fd6f6000640 1 -- 172.21.15.71:0/3407065654 >> 172.21.15.71:0/3407065654 conn(0x7fd7040cfdd0 msgr2=0x7fd7040b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.075+0000 7fd6f6000640 1 -- 172.21.15.71:0/3407065654 shutdown_connections 2024-09-16T07:22:24.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.075+0000 7fd6f6000640 1 -- 172.21.15.71:0/3407065654 wait complete. 2024-09-16T07:22:24.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.076+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 shutdown_connections 2024-09-16T07:22:24.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.076+0000 7fccd9800640 1 -- 172.21.15.71:0/2947897517 wait complete. 2024-09-16T07:22:24.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.118+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fdaf40cf560 con 0x7fdaf40ca040 2024-09-16T07:22:24.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.128+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fcd4c005580 con 0x7fcd74165280 2024-09-16T07:22:24.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.129+0000 7fdaebe00640 1 -- 172.21.15.71:0/1052171307 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v12)=0 v12) ==== 119+0+0 (secure 0 0 0) 0x7fdaf40cf560 con 0x7fdaf40ca040 2024-09-16T07:22:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.134+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40ca040 msgr2=0x7fdaf4152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.134+0000 7fdaf9a00640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40ca040 0x7fdaf4152690 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b3f0 tx=0x7fdae400b8b0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.135 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.136+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 shutdown_connections 2024-09-16T07:22:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.136+0000 7fdaf9a00640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf40cb840 0x7fdaf4150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.136+0000 7fdaf9a00640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdaf40ca040 0x7fdaf4152690 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.136+0000 7fdaf9a00640 1 --2- 172.21.15.71:0/1052171307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdaf406e5a0 0x7fdaf4151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.136+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 >> 172.21.15.71:0/1052171307 conn(0x7fdaf40cf9f0 msgr2=0x7fdaf40cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.137+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 shutdown_connections 2024-09-16T07:22:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.137+0000 7fdaf9a00640 1 -- 172.21.15.71:0/1052171307 wait complete. 2024-09-16T07:22:24.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.140+0000 7fcd6b400640 1 -- 172.21.15.71:0/187772902 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7fcd6c02c400 con 0x7fcd74165280 2024-09-16T07:22:24.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.143+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd74165280 msgr2=0x7fcd74162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.143+0000 7fcd7a000640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd74165280 0x7fcd74162370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00ca70 tx=0x7fcd6c00cf30 comp rx=0 tx=0).stop 2024-09-16T07:22:24.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.144+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fc41806edb0 con 0x7fc41814c360 2024-09-16T07:22:24.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.144+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 shutdown_connections 2024-09-16T07:22:24.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.144+0000 7fcd7a000640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd74165280 0x7fcd74162370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.144+0000 7fcd7a000640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd740b3aa0 0x7fcd742051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.144+0000 7fcd7a000640 1 --2- 172.21.15.71:0/187772902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd7406fb90 0x7fcd74204c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.144+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 >> 172.21.15.71:0/187772902 conn(0x7fcd740b1840 msgr2=0x7fcd740d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.145+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 shutdown_connections 2024-09-16T07:22:24.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.145+0000 7fcd7a000640 1 -- 172.21.15.71:0/187772902 wait complete. 2024-09-16T07:22:24.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.156+0000 7fc40e000640 1 -- 172.21.15.71:0/2922611140 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7fc41814a440 con 0x7fc41814c360 2024-09-16T07:22:24.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.159+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc41814c360 msgr2=0x7fc41814cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.159+0000 7fc417200640 1 --2- 172.21.15.71:0/2922611140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc41814c360 0x7fc41814cf70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc404009920 tx=0x7fc404009de0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.159+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 shutdown_connections 2024-09-16T07:22:24.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.159+0000 7fc417200640 1 --2- 172.21.15.71:0/2922611140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc41814c360 0x7fc41814cf70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.159+0000 7fc417200640 1 --2- 172.21.15.71:0/2922611140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc41815dc00 0x7fc41814ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.159+0000 7fc417200640 1 --2- 172.21.15.71:0/2922611140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc41806e940 0x7fc41814e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.159+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 >> 172.21.15.71:0/2922611140 conn(0x7fc4180cf9f0 msgr2=0x7fc418156320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.159+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 shutdown_connections 2024-09-16T07:22:24.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.160+0000 7fc417200640 1 -- 172.21.15.71:0/2922611140 wait complete. 2024-09-16T07:22:24.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.170+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f3554162860 con 0x7f35540b7340 2024-09-16T07:22:24.176 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.176 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.176+0000 7f3550c00640 1 -- 172.21.15.71:0/31967439 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f3548005000 con 0x7f35540b7340 2024-09-16T07:22:24.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.179+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35540b7340 msgr2=0x7f35540b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.179+0000 7f355bc00640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35540b7340 0x7f35540b8f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f354800e9f0 tx=0x7f354800eeb0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.179+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 shutdown_connections 2024-09-16T07:22:24.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.179+0000 7f355bc00640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3554161590 0x7f35540b9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.179+0000 7f355bc00640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35540b7340 0x7f35540b8f00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.179+0000 7f355bc00640 1 --2- 172.21.15.71:0/31967439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35540b5630 0x7f355409e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.179+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 >> 172.21.15.71:0/31967439 conn(0x7f35540d3fb0 msgr2=0x7f35540b8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.180+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 shutdown_connections 2024-09-16T07:22:24.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.180+0000 7f355bc00640 1 -- 172.21.15.71:0/31967439 wait complete. 2024-09-16T07:22:24.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.187+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fcd0c005580 con 0x7fcd340ce0c0 2024-09-16T07:22:24.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.192+0000 7fcd2aa00640 1 -- 172.21.15.71:0/4256998521 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7fcd2c0318b0 con 0x7fcd340ce0c0 2024-09-16T07:22:24.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.193+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd340ce0c0 msgr2=0x7fcd341f7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.193+0000 7fcd38a00640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd340ce0c0 0x7fcd341f7b90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c006010 tx=0x7fcd2c003f10 comp rx=0 tx=0).stop 2024-09-16T07:22:24.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.193+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 shutdown_connections 2024-09-16T07:22:24.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.193+0000 7fcd38a00640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd3416f0f0 0x7fcd341f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.193+0000 7fcd38a00640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd340cf430 0x7fcd342134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.193+0000 7fcd38a00640 1 --2- 172.21.15.71:0/4256998521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd340ce0c0 0x7fcd341f7b90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.193+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 >> 172.21.15.71:0/4256998521 conn(0x7fcd340cf9f0 msgr2=0x7fcd340af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.194+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 shutdown_connections 2024-09-16T07:22:24.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.194+0000 7fcd38a00640 1 -- 172.21.15.71:0/4256998521 wait complete. 2024-09-16T07:22:24.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.195+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fe1b40ce200 con 0x7fe1b415c3f0 2024-09-16T07:22:24.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.200+0000 7fe1b1600640 1 -- 172.21.15.71:0/2940073569 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v17)=0 v17) ==== 120+0+0 (secure 0 0 0) 0x7fe1b4144730 con 0x7fe1b415c3f0 2024-09-16T07:22:24.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1b415c3f0 msgr2=0x7fe1b4144a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1b415c3f0 0x7fe1b4144a10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009000 tx=0x7fe1ac031600 comp rx=0 tx=0).stop 2024-09-16T07:22:24.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 shutdown_connections 2024-09-16T07:22:24.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1b415c3f0 0x7fe1b4144a10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1b40cddb0 0x7fe1b415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 --2- 172.21.15.71:0/2940073569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1b406f2f0 0x7fe1b41441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 >> 172.21.15.71:0/2940073569 conn(0x7fe1b40b1840 msgr2=0x7fe1b41527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 shutdown_connections 2024-09-16T07:22:24.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.202+0000 7fe1bc200640 1 -- 172.21.15.71:0/2940073569 wait complete. 2024-09-16T07:22:24.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.212+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f2340160bc0 con 0x7f23400b5270 2024-09-16T07:22:24.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.217+0000 7f2336a00640 1 -- 172.21.15.71:0/2806207271 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f2330003dc0 con 0x7f23400b5270 2024-09-16T07:22:24.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23400b5270 msgr2=0x7f23400caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23400b5270 0x7f23400caec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f233000c460 tx=0x7f233000c920 comp rx=0 tx=0).stop 2024-09-16T07:22:24.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 shutdown_connections 2024-09-16T07:22:24.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f234015e180 0x7f23400cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23400b5270 0x7f23400caec0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 --2- 172.21.15.71:0/2806207271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23400b4d40 0x7f23401f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 >> 172.21.15.71:0/2806207271 conn(0x7f234009d0b0 msgr2=0x7f23400b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 shutdown_connections 2024-09-16T07:22:24.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.220+0000 7f2344a00640 1 -- 172.21.15.71:0/2806207271 wait complete. 2024-09-16T07:22:24.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.225+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fc2441f88b0 con 0x7fc244156020 2024-09-16T07:22:24.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.232+0000 7fc23aa00640 1 -- 172.21.15.71:0/3268903882 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v19)=0 v19) ==== 125+0+0 (secure 0 0 0) 0x7fc23403ddb0 con 0x7fc244156020 2024-09-16T07:22:24.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc244156020 msgr2=0x7fc2441580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc244156020 0x7fc2441580e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc23402a820 tx=0x7fc234032bf0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 shutdown_connections 2024-09-16T07:22:24.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2441fbed0 0x7fc2441fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2441f95d0 0x7fc244158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 --2- 172.21.15.71:0/3268903882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc244156020 0x7fc2441580e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 >> 172.21.15.71:0/3268903882 conn(0x7fc2440d3e20 msgr2=0x7fc24406d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 shutdown_connections 2024-09-16T07:22:24.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.233+0000 7fc243e00640 1 -- 172.21.15.71:0/3268903882 wait complete. 2024-09-16T07:22:24.235 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.235 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.235 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.236 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.249 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi071') 2024-09-16T07:22:24.249 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T07:22:24.249 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T07:22:24.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.445+0000 7f11aea00640 1 -- 172.21.15.71:0/1270577718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a806efa0 msgr2=0x7f11a8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.445+0000 7f11aea00640 1 --2- 172.21.15.71:0/1270577718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a806efa0 0x7f11a8073440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005fd0 tx=0x7f11a002a8b0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.445+0000 7f11aea00640 1 -- 172.21.15.71:0/1270577718 shutdown_connections 2024-09-16T07:22:24.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.445+0000 7f11aea00640 1 --2- 172.21.15.71:0/1270577718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a806efa0 0x7f11a8073440 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.445+0000 7f11aea00640 1 -- 172.21.15.71:0/1270577718 >> 172.21.15.71:0/1270577718 conn(0x7f11a80b1840 msgr2=0x7f11a80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.445+0000 7f11aea00640 1 -- 172.21.15.71:0/1270577718 shutdown_connections 2024-09-16T07:22:24.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.445+0000 7f11aea00640 1 -- 172.21.15.71:0/1270577718 wait complete. 2024-09-16T07:22:24.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.448+0000 7f11aea00640 1 Processor -- start 2024-09-16T07:22:24.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.448+0000 7f11aea00640 1 -- start start 2024-09-16T07:22:24.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.449+0000 7f11aea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a806efa0 0x7f11a815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.449+0000 7f11aea00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a80cccd0 0x7f11a8141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.449+0000 7f11aea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8145750 0x7f11a8141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.449+0000 7f11aea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80ce6a0 con 0x7f11a8145750 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.449+0000 7f11aea00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80ce520 con 0x7f11a806efa0 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.449+0000 7f11aea00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11a80ce820 con 0x7f11a80cccd0 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8145750 0x7f11a8141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8145750 0x7f11a8141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44942/0 (socket says 172.21.15.71:44942) 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 -- 172.21.15.71:0/3161116182 learned_addr learned my addr 172.21.15.71:0/3161116182 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f119f400640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a80cccd0 0x7f11a8141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11a7e00640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a806efa0 0x7f11a815e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 -- 172.21.15.71:0/3161116182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a80cccd0 msgr2=0x7f11a8141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a80cccd0 0x7f11a8141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 -- 172.21.15.71:0/3161116182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a806efa0 msgr2=0x7f11a815e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a806efa0 0x7f11a815e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 -- 172.21.15.71:0/3161116182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80ad4f0 con 0x7f11a8145750 2024-09-16T07:22:24.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.450+0000 7f11aca00640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8145750 0x7f11a8141ea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009cd0 tx=0x7f11a00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.451+0000 7f11a4c00640 1 -- 172.21.15.71:0/3161116182 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0040070 con 0x7f11a8145750 2024-09-16T07:22:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.451+0000 7f11a4c00640 1 -- 172.21.15.71:0/3161116182 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11a0004330 con 0x7f11a8145750 2024-09-16T07:22:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.451+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a80ad7d0 con 0x7f11a8145750 2024-09-16T07:22:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.451+0000 7f11a4c00640 1 -- 172.21.15.71:0/3161116182 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0004700 con 0x7f11a8145750 2024-09-16T07:22:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.451+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a80add90 con 0x7f11a8145750 2024-09-16T07:22:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.451+0000 7f11a4c00640 1 -- 172.21.15.71:0/3161116182 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11a0039040 con 0x7f11a8145750 2024-09-16T07:22:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.451+0000 7f11a4c00640 1 -- 172.21.15.71:0/3161116182 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11a0045d20 con 0x7f11a8145750 2024-09-16T07:22:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.452+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a80d0480 con 0x7f11a8145750 2024-09-16T07:22:24.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.454+0000 7f11a4c00640 1 -- 172.21.15.71:0/3161116182 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f11a80d0480 con 0x7f11a8145750 2024-09-16T07:22:24.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.522+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f11a8144e70 con 0x7f11a8145750 2024-09-16T07:22:24.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.525+0000 7f11a4c00640 1 -- 172.21.15.71:0/3161116182 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f11a0045460 con 0x7f11a8145750 2024-09-16T07:22:24.525 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-09-16T07:22:24.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.526+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8145750 msgr2=0x7f11a8141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:24.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.526+0000 7f11aea00640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8145750 0x7f11a8141ea0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009cd0 tx=0x7f11a00039f0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.526+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 shutdown_connections 2024-09-16T07:22:24.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.526+0000 7f11aea00640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11a8145750 0x7f11a8141ea0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.526+0000 7f11aea00640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a80cccd0 0x7f11a8141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.526+0000 7f11aea00640 1 --2- 172.21.15.71:0/3161116182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a806efa0 0x7f11a815e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:24.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.526+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 >> 172.21.15.71:0/3161116182 conn(0x7f11a80b1840 msgr2=0x7f11a80cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:24.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.527+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 shutdown_connections 2024-09-16T07:22:24.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:24.527+0000 7f11aea00640 1 -- 172.21.15.71:0/3161116182 wait complete. 2024-09-16T07:22:24.541 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T07:22:24.541 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T07:22:24.541 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:24.541 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T07:22:24.600 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:24.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T07:22:24.698 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:24.698 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T07:22:24.796 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:22:24.796 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T07:22:24.891 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:24.891 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T07:22:24.948 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:24.948 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T07:22:25.048 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:25.048 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T07:22:25.145 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:22:25.145 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T07:22:25.242 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new f9b13ba3-d953-4d09-90fc-c07a6d703d34 0 2024-09-16T07:22:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.437+0000 7fe395c00640 1 -- 172.21.15.177:0/2881579262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 msgr2=0x7fe3900b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.437+0000 7fe395c00640 1 --2- 172.21.15.177:0/2881579262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 0x7fe3900b3ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe388005fd0 tx=0x7fe38802a6e0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.437 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.437+0000 7fe395c00640 1 -- 172.21.15.177:0/2881579262 shutdown_connections 2024-09-16T07:22:25.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.437+0000 7fe395c00640 1 --2- 172.21.15.177:0/2881579262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 0x7fe3900b3ac0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.437+0000 7fe395c00640 1 -- 172.21.15.177:0/2881579262 >> 172.21.15.177:0/2881579262 conn(0x7fe39009d0b0 msgr2=0x7fe3900d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:25.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.437+0000 7fe395c00640 1 -- 172.21.15.177:0/2881579262 shutdown_connections 2024-09-16T07:22:25.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.437+0000 7fe395c00640 1 -- 172.21.15.177:0/2881579262 wait complete. 2024-09-16T07:22:25.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.438+0000 7fe395c00640 1 Processor -- start 2024-09-16T07:22:25.438 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.439+0000 7fe395c00640 1 -- start start 2024-09-16T07:22:25.441 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.439+0000 7fe395c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3900cf830 0x7fe390150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.441 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.439+0000 7fe395c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 0x7fe390153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.441 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.439+0000 7fe395c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe390161460 0x7fe3901537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.441 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.439+0000 7fe395c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe3900b50c0 con 0x7fe3900d2a00 2024-09-16T07:22:25.441 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.439+0000 7fe395c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe3900b4f40 con 0x7fe390161460 2024-09-16T07:22:25.441 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.439+0000 7fe395c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe3900b4000 con 0x7fe3900cf830 2024-09-16T07:22:25.441 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe38fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3900cf830 0x7fe390150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe395200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe390161460 0x7fe3901537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe38fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3900cf830 0x7fe390150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46714/0 (socket says 172.21.15.177:46714) 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe38fe00640 1 -- 172.21.15.177:0/869298830 learned_addr learned my addr 172.21.15.177:0/869298830 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe38f400640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 0x7fe390153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe395200640 1 -- 172.21.15.177:0/869298830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3900cf830 msgr2=0x7fe390150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe395200640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3900cf830 0x7fe390150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe395200640 1 -- 172.21.15.177:0/869298830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 msgr2=0x7fe390153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe395200640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 0x7fe390153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe395200640 1 -- 172.21.15.177:0/869298830 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3900cd2e0 con 0x7fe390161460 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.440+0000 7fe38f400640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 0x7fe390153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.441+0000 7fe395200640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe390161460 0x7fe3901537c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe388009000 tx=0x7fe388032d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.441+0000 7fe387e00640 1 -- 172.21.15.177:0/869298830 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388041020 con 0x7fe390161460 2024-09-16T07:22:25.442 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.441+0000 7fe387e00640 1 -- 172.21.15.177:0/869298830 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe388004350 con 0x7fe390161460 2024-09-16T07:22:25.443 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.441+0000 7fe387e00640 1 -- 172.21.15.177:0/869298830 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388036040 con 0x7fe390161460 2024-09-16T07:22:25.443 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.441+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3900cd5c0 con 0x7fe390161460 2024-09-16T07:22:25.443 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.441+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe390151600 con 0x7fe390161460 2024-09-16T07:22:25.443 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.441+0000 7fe387e00640 1 -- 172.21.15.177:0/869298830 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe388044020 con 0x7fe390161460 2024-09-16T07:22:25.443 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.441+0000 7fe387e00640 1 -- 172.21.15.177:0/869298830 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fe38803d420 con 0x7fe390161460 2024-09-16T07:22:25.443 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.442+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3900b9ce0 con 0x7fe390161460 2024-09-16T07:22:25.444 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.444+0000 7fe387e00640 1 -- 172.21.15.177:0/869298830 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe38804b030 con 0x7fe390161460 2024-09-16T07:22:25.517 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.518+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f9b13ba3-d953-4d09-90fc-c07a6d703d34", "id": 0} v 0) -- 0x7fe3900b4590 con 0x7fe390161460 2024-09-16T07:22:25.523 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.524+0000 7fe387e00640 1 -- 172.21.15.177:0/869298830 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f9b13ba3-d953-4d09-90fc-c07a6d703d34", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fe388004840 con 0x7fe390161460 2024-09-16T07:22:25.524 INFO:teuthology.orchestra.run.smithi177.stdout:0 2024-09-16T07:22:25.525 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe390161460 msgr2=0x7fe3901537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.525 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe390161460 0x7fe3901537c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe388009000 tx=0x7fe388032d10 comp rx=0 tx=0).stop 2024-09-16T07:22:25.525 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 shutdown_connections 2024-09-16T07:22:25.525 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe390161460 0x7fe3901537c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.525 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3900d2a00 0x7fe390153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 --2- 172.21.15.177:0/869298830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3900cf830 0x7fe390150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 >> 172.21.15.177:0/869298830 conn(0x7fe39009d0b0 msgr2=0x7fe3900d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:25.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 shutdown_connections 2024-09-16T07:22:25.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.525+0000 7fe395c00640 1 -- 172.21.15.177:0/869298830 wait complete. 2024-09-16T07:22:25.541 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 9885ba32-5e7a-4af5-b083-eeadd8847f47 1 2024-09-16T07:22:25.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.694+0000 7f0077800640 1 -- 172.21.15.177:0/1511619233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00700cccd0 msgr2=0x7f0070073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.694+0000 7f0077800640 1 --2- 172.21.15.177:0/1511619233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00700cccd0 0x7f0070073440 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f00700af780 tx=0x7f005c02a4a0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.695+0000 7f0077800640 1 -- 172.21.15.177:0/1511619233 shutdown_connections 2024-09-16T07:22:25.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.695+0000 7f0077800640 1 --2- 172.21.15.177:0/1511619233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00700cccd0 0x7f0070073440 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.695 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.695+0000 7f0077800640 1 --2- 172.21.15.177:0/1511619233 >> v2:172.21.15.71:3300/0 conn(0x7f007006efa0 0x7f00700cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.696 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.695+0000 7f0077800640 1 -- 172.21.15.177:0/1511619233 >> 172.21.15.177:0/1511619233 conn(0x7f00700b1840 msgr2=0x7f00700b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:25.696 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.695+0000 7f0077800640 1 -- 172.21.15.177:0/1511619233 shutdown_connections 2024-09-16T07:22:25.696 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.695+0000 7f0077800640 1 -- 172.21.15.177:0/1511619233 wait complete. 2024-09-16T07:22:25.698 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.696+0000 7f0077800640 1 Processor -- start 2024-09-16T07:22:25.698 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.696+0000 7f0077800640 1 -- start start 2024-09-16T07:22:25.698 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.697+0000 7f0077800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007006efa0 0x7f0070154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.698 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.697+0000 7f0077800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00700cccd0 0x7f0070157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.698 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.697+0000 7f0077800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0070145750 0x7f0070154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.698 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.697+0000 7f0077800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00700ce480 con 0x7f0070145750 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.697+0000 7f0077800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00700ce300 con 0x7f007006efa0 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.697+0000 7f0077800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f00700ce600 con 0x7f00700cccd0 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f0074e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007006efa0 0x7f0070154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f0074e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007006efa0 0x7f0070154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:51760/0 (socket says 172.21.15.177:51760) 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f006fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00700cccd0 0x7f0070157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f0074e00640 1 -- 172.21.15.177:0/1262524360 learned_addr learned my addr 172.21.15.177:0/1262524360 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f0075800640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0070145750 0x7f0070154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f006fe00640 1 -- 172.21.15.177:0/1262524360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007006efa0 msgr2=0x7f0070154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f006fe00640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007006efa0 0x7f0070154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f006fe00640 1 -- 172.21.15.177:0/1262524360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0070145750 msgr2=0x7f0070154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.699 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f006fe00640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0070145750 0x7f0070154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.700 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f006fe00640 1 -- 172.21.15.177:0/1262524360 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007013e3b0 con 0x7f00700cccd0 2024-09-16T07:22:25.700 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f0074e00640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007006efa0 0x7f0070154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:25.700 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.698+0000 7f006fe00640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00700cccd0 0x7f0070157cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f005c031000 tx=0x7f005c032c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:25.701 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.699+0000 7f006cc00640 1 -- 172.21.15.177:0/1262524360 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c040070 con 0x7f00700cccd0 2024-09-16T07:22:25.701 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.699+0000 7f006cc00640 1 -- 172.21.15.177:0/1262524360 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f005c032e70 con 0x7f00700cccd0 2024-09-16T07:22:25.701 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.699+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007013e690 con 0x7f00700cccd0 2024-09-16T07:22:25.701 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.699+0000 7f006cc00640 1 -- 172.21.15.177:0/1262524360 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0055d0 con 0x7f00700cccd0 2024-09-16T07:22:25.701 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.699+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f007013ec50 con 0x7f00700cccd0 2024-09-16T07:22:25.701 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.699+0000 7f006cc00640 1 -- 172.21.15.177:0/1262524360 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f005c039040 con 0x7f00700cccd0 2024-09-16T07:22:25.701 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.699+0000 7f006cc00640 1 -- 172.21.15.177:0/1262524360 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f005c004e20 con 0x7f00700cccd0 2024-09-16T07:22:25.701 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.700+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00700cdbb0 con 0x7f00700cccd0 2024-09-16T07:22:25.702 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.702+0000 7f006cc00640 1 -- 172.21.15.177:0/1262524360 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f005c003d70 con 0x7f00700cccd0 2024-09-16T07:22:25.776 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.776+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9885ba32-5e7a-4af5-b083-eeadd8847f47", "id": 1} v 0) -- 0x7f007013e0e0 con 0x7f00700cccd0 2024-09-16T07:22:25.780 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.780+0000 7f006cc00640 1 -- 172.21.15.177:0/1262524360 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9885ba32-5e7a-4af5-b083-eeadd8847f47", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f005c0047a0 con 0x7f00700cccd0 2024-09-16T07:22:25.780 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-09-16T07:22:25.781 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.781+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00700cccd0 msgr2=0x7f0070157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.781+0000 7f0077800640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00700cccd0 0x7f0070157cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f005c031000 tx=0x7f005c032c80 comp rx=0 tx=0).stop 2024-09-16T07:22:25.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.781+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 shutdown_connections 2024-09-16T07:22:25.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.781+0000 7f0077800640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0070145750 0x7f0070154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.781+0000 7f0077800640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00700cccd0 0x7f0070157cd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.781+0000 7f0077800640 1 --2- 172.21.15.177:0/1262524360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007006efa0 0x7f0070154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.781+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 >> 172.21.15.177:0/1262524360 conn(0x7f00700b1840 msgr2=0x7f00700a9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:25.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.781+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 shutdown_connections 2024-09-16T07:22:25.782 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.782+0000 7f0077800640 1 -- 172.21.15.177:0/1262524360 wait complete. 2024-09-16T07:22:25.796 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 6bd108a5-715d-4894-b6c9-ffb9c4b8aaae 2 2024-09-16T07:22:25.952 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.952+0000 7f68ea400640 1 -- 172.21.15.177:0/3024886633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 msgr2=0x7f68e40b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.952 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.952+0000 7f68ea400640 1 --2- 172.21.15.177:0/3024886633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 0x7f68e40b33b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f68d0005fd0 tx=0x7f68d002a5f0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.952 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.952+0000 7f68ea400640 1 -- 172.21.15.177:0/3024886633 shutdown_connections 2024-09-16T07:22:25.953 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.952+0000 7f68ea400640 1 --2- 172.21.15.177:0/3024886633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 0x7f68e40b33b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.953 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.952+0000 7f68ea400640 1 -- 172.21.15.177:0/3024886633 >> 172.21.15.177:0/3024886633 conn(0x7f68e409d0b0 msgr2=0x7f68e40d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:25.953 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.952+0000 7f68ea400640 1 -- 172.21.15.177:0/3024886633 shutdown_connections 2024-09-16T07:22:25.953 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.952+0000 7f68ea400640 1 -- 172.21.15.177:0/3024886633 wait complete. 2024-09-16T07:22:25.953 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.954+0000 7f68ea400640 1 Processor -- start 2024-09-16T07:22:25.953 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.954+0000 7f68ea400640 1 -- start start 2024-09-16T07:22:25.955 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68ea400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 0x7f68e40b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.955 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68ea400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68e40d2a00 0x7f68e415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.955 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68ea400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68e4161460 0x7f68e406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:25.955 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68ea400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68e40b50c0 con 0x7f68e40d2a00 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68ea400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f68e40b4f40 con 0x7f68e40cf830 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68ea400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f68e40b4000 con 0x7f68e4161460 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68e9000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 0x7f68e40b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68e9000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 0x7f68e40b97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:51780/0 (socket says 172.21.15.177:51780) 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68e9000640 1 -- 172.21.15.177:0/1330915202 learned_addr learned my addr 172.21.15.177:0/1330915202 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68e3e00640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68e40d2a00 0x7f68e415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68e9a00640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68e4161460 0x7f68e406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68e9000640 1 -- 172.21.15.177:0/1330915202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68e4161460 msgr2=0x7f68e406e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.955+0000 7f68e9000640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68e4161460 0x7f68e406e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.956 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68e9000640 1 -- 172.21.15.177:0/1330915202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68e40d2a00 msgr2=0x7f68e415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:25.957 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68e9000640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68e40d2a00 0x7f68e415b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:25.957 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68e9000640 1 -- 172.21.15.177:0/1330915202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e41f7ca0 con 0x7f68e40cf830 2024-09-16T07:22:25.957 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68e9000640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 0x7f68e40b97e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f68d800c3a0 tx=0x7f68d800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:25.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68e0c00640 1 -- 172.21.15.177:0/1330915202 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8017070 con 0x7f68e40cf830 2024-09-16T07:22:25.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68e0c00640 1 -- 172.21.15.177:0/1330915202 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68d80053c0 con 0x7f68e40cf830 2024-09-16T07:22:25.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e415bf10 con 0x7f68e40cf830 2024-09-16T07:22:25.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68e0c00640 1 -- 172.21.15.177:0/1330915202 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8003850 con 0x7f68e40cf830 2024-09-16T07:22:25.958 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e41f8240 con 0x7f68e40cf830 2024-09-16T07:22:25.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.956+0000 7f68e0c00640 1 -- 172.21.15.177:0/1330915202 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f68d801a070 con 0x7f68e40cf830 2024-09-16T07:22:25.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.957+0000 7f68e0c00640 1 -- 172.21.15.177:0/1330915202 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f68d8011c70 con 0x7f68e40cf830 2024-09-16T07:22:25.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.957+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e40b4500 con 0x7f68e40cf830 2024-09-16T07:22:25.959 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:25.959+0000 7f68e0c00640 1 -- 172.21.15.177:0/1330915202 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f68d8021070 con 0x7f68e40cf830 2024-09-16T07:22:26.033 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.033+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6bd108a5-715d-4894-b6c9-ffb9c4b8aaae", "id": 2} v 0) -- 0x7f68e40d3e90 con 0x7f68e40cf830 2024-09-16T07:22:26.037 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.038+0000 7f68e0c00640 1 -- 172.21.15.177:0/1330915202 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6bd108a5-715d-4894-b6c9-ffb9c4b8aaae", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f68d8011590 con 0x7f68e40cf830 2024-09-16T07:22:26.037 INFO:teuthology.orchestra.run.smithi177.stdout:2 2024-09-16T07:22:26.039 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 msgr2=0x7f68e40b97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.039 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 0x7f68e40b97e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f68d800c3a0 tx=0x7f68d800c860 comp rx=0 tx=0).stop 2024-09-16T07:22:26.039 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 shutdown_connections 2024-09-16T07:22:26.039 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68e4161460 0x7f68e406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.039 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68e40d2a00 0x7f68e415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.039 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 --2- 172.21.15.177:0/1330915202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68e40cf830 0x7f68e40b97e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.039 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 >> 172.21.15.177:0/1330915202 conn(0x7f68e409d0b0 msgr2=0x7f68e40b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:26.040 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 shutdown_connections 2024-09-16T07:22:26.040 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.039+0000 7f68ea400640 1 -- 172.21.15.177:0/1330915202 wait complete. 2024-09-16T07:22:26.053 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 557e4e8f-f744-46b6-9d7e-81ca39b7f2fb 3 2024-09-16T07:22:26.209 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.208+0000 7f43a9000640 1 -- 172.21.15.177:0/993538511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43a40cf830 msgr2=0x7f43a40b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.209 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.208+0000 7f43a9000640 1 --2- 172.21.15.177:0/993538511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43a40cf830 0x7f43a40b33b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f438c009830 tx=0x7f438c02a530 comp rx=0 tx=0).stop 2024-09-16T07:22:26.209 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.209+0000 7f43a9000640 1 -- 172.21.15.177:0/993538511 shutdown_connections 2024-09-16T07:22:26.209 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.209+0000 7f43a9000640 1 --2- 172.21.15.177:0/993538511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43a40cf830 0x7f43a40b33b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.209 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.209+0000 7f43a9000640 1 -- 172.21.15.177:0/993538511 >> 172.21.15.177:0/993538511 conn(0x7f43a409d0b0 msgr2=0x7f43a40d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:26.209 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.209+0000 7f43a9000640 1 -- 172.21.15.177:0/993538511 shutdown_connections 2024-09-16T07:22:26.209 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.209+0000 7f43a9000640 1 -- 172.21.15.177:0/993538511 wait complete. 2024-09-16T07:22:26.210 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.210+0000 7f43a9000640 1 Processor -- start 2024-09-16T07:22:26.210 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.210+0000 7f43a9000640 1 -- start start 2024-09-16T07:22:26.211 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.211+0000 7f43a9000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a40cf830 0x7f43a41f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.211 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.211+0000 7f43a9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43a40d2a00 0x7f43a40c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.211 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.211+0000 7f43a9000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a4161460 0x7f43a406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.211 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.211+0000 7f43a9000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43a40b50c0 con 0x7f43a40d2a00 2024-09-16T07:22:26.212 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.211+0000 7f43a9000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f43a40b4f40 con 0x7f43a4161460 2024-09-16T07:22:26.212 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.211+0000 7f43a9000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43a40b4000 con 0x7f43a40cf830 2024-09-16T07:22:26.212 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a40cf830 0x7f43a41f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.212 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a4161460 0x7f43a406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.212 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f439be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43a40d2a00 0x7f43a40c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.212 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a40cf830 0x7f43a41f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46752/0 (socket says 172.21.15.177:46752) 2024-09-16T07:22:26.212 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3400640 1 -- 172.21.15.177:0/2089855860 learned_addr learned my addr 172.21.15.177:0/2089855860 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-16T07:22:26.213 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3e00640 1 -- 172.21.15.177:0/2089855860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a40cf830 msgr2=0x7f43a41f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.213 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3e00640 1 --2- 172.21.15.177:0/2089855860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a40cf830 0x7f43a41f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3e00640 1 -- 172.21.15.177:0/2089855860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43a40d2a00 msgr2=0x7f43a40c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3e00640 1 --2- 172.21.15.177:0/2089855860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43a40d2a00 0x7f43a40c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3e00640 1 -- 172.21.15.177:0/2089855860 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a41f4750 con 0x7f43a4161460 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3400640 1 --2- 172.21.15.177:0/2089855860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a40cf830 0x7f43a41f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.212+0000 7f43a3e00640 1 --2- 172.21.15.177:0/2089855860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a4161460 0x7f43a406da90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c730 tx=0x7f439c00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.213+0000 7f439b400640 1 -- 172.21.15.177:0/2089855860 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c003b70 con 0x7f43a4161460 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.213+0000 7f439b400640 1 -- 172.21.15.177:0/2089855860 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f439c003d10 con 0x7f43a4161460 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.213+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a41f4a30 con 0x7f43a4161460 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.213+0000 7f439b400640 1 -- 172.21.15.177:0/2089855860 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c00aab0 con 0x7f43a4161460 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.213+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a41f4450 con 0x7f43a4161460 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.213+0000 7f439b400640 1 -- 172.21.15.177:0/2089855860 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f439c00ac50 con 0x7f43a4161460 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.213+0000 7f439b400640 1 -- 172.21.15.177:0/2089855860 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f439c005560 con 0x7f43a4161460 2024-09-16T07:22:26.214 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.214+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a40b4500 con 0x7f43a4161460 2024-09-16T07:22:26.215 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.216+0000 7f439b400640 1 -- 172.21.15.177:0/2089855860 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f439c0050d0 con 0x7f43a4161460 2024-09-16T07:22:26.289 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.290+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "557e4e8f-f744-46b6-9d7e-81ca39b7f2fb", "id": 3} v 0) -- 0x7f43a406d790 con 0x7f43a4161460 2024-09-16T07:22:26.294 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.294+0000 7f439b400640 1 -- 172.21.15.177:0/2089855860 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "557e4e8f-f744-46b6-9d7e-81ca39b7f2fb", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f439c005310 con 0x7f43a4161460 2024-09-16T07:22:26.294 INFO:teuthology.orchestra.run.smithi177.stdout:3 2024-09-16T07:22:26.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.295+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a4161460 msgr2=0x7f43a406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.295+0000 7f43a9000640 1 --2- 172.21.15.177:0/2089855860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a4161460 0x7f43a406da90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c730 tx=0x7f439c00cbf0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.295+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 shutdown_connections 2024-09-16T07:22:26.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.295+0000 7f43a9000640 1 --2- 172.21.15.177:0/2089855860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a4161460 0x7f43a406da90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.295+0000 7f43a9000640 1 --2- 172.21.15.177:0/2089855860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43a40d2a00 0x7f43a40c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.295+0000 7f43a9000640 1 --2- 172.21.15.177:0/2089855860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a40cf830 0x7f43a41f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.295+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 >> 172.21.15.177:0/2089855860 conn(0x7f43a409d0b0 msgr2=0x7f43a4160690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:26.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.296+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 shutdown_connections 2024-09-16T07:22:26.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.296+0000 7f43a9000640 1 -- 172.21.15.177:0/2089855860 wait complete. 2024-09-16T07:22:26.310 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 91f8990b-1328-48a3-aa70-6be7326adfc6 4 2024-09-16T07:22:26.463 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.463+0000 7f12ee800640 1 -- 172.21.15.177:0/4044184375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e80b43c0 msgr2=0x7f12e80d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.463 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.463+0000 7f12ee800640 1 --2- 172.21.15.177:0/4044184375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e80b43c0 0x7f12e80d2830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f12e0005fd0 tx=0x7f12e002aa10 comp rx=0 tx=0).stop 2024-09-16T07:22:26.464 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.463+0000 7f12ee800640 1 -- 172.21.15.177:0/4044184375 shutdown_connections 2024-09-16T07:22:26.464 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.463+0000 7f12ee800640 1 --2- 172.21.15.177:0/4044184375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e80b43c0 0x7f12e80d2830 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.464 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.463+0000 7f12ee800640 1 --2- 172.21.15.177:0/4044184375 >> v2:172.21.15.71:3300/0 conn(0x7f12e8161460 0x7f12e80d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.464 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.463+0000 7f12ee800640 1 -- 172.21.15.177:0/4044184375 >> 172.21.15.177:0/4044184375 conn(0x7f12e809d0b0 msgr2=0x7f12e80d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:26.464 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.463+0000 7f12ee800640 1 -- 172.21.15.177:0/4044184375 shutdown_connections 2024-09-16T07:22:26.464 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.463+0000 7f12ee800640 1 -- 172.21.15.177:0/4044184375 wait complete. 2024-09-16T07:22:26.464 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.465+0000 7f12ee800640 1 Processor -- start 2024-09-16T07:22:26.465 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.465+0000 7f12ee800640 1 -- start start 2024-09-16T07:22:26.467 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ee800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12e80b43c0 0x7f12e815b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.467 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ee800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12e80b5480 0x7f12e80b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.467 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ee800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e8161460 0x7f12e80b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ee800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80cef30 con 0x7f12e80b5480 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ee800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80b5210 con 0x7f12e80b43c0 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ee800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f12e80cf0b0 con 0x7f12e8161460 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ede00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e8161460 0x7f12e80b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ed400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12e80b43c0 0x7f12e815b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ed400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12e80b43c0 0x7f12e815b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:51798/0 (socket says 172.21.15.177:51798) 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12ed400640 1 -- 172.21.15.177:0/904473112 learned_addr learned my addr 172.21.15.177:0/904473112 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.466+0000 7f12eca00640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12e80b5480 0x7f12e80b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ede00640 1 -- 172.21.15.177:0/904473112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12e80b43c0 msgr2=0x7f12e815b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ede00640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12e80b43c0 0x7f12e815b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.468 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ede00640 1 -- 172.21.15.177:0/904473112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12e80b5480 msgr2=0x7f12e80b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ede00640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12e80b5480 0x7f12e80b94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ede00640 1 -- 172.21.15.177:0/904473112 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80ccba0 con 0x7f12e8161460 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ed400640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12e80b43c0 0x7f12e815b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ede00640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e8161460 0x7f12e80b7650 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12e0032000 tx=0x7f12e0003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12e5600640 1 -- 172.21.15.177:0/904473112 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0047070 con 0x7f12e8161460 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12e5600640 1 -- 172.21.15.177:0/904473112 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12e002f670 con 0x7f12e8161460 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e80cce80 con 0x7f12e8161460 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12e5600640 1 -- 172.21.15.177:0/904473112 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0033050 con 0x7f12e8161460 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.467+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e80cd440 con 0x7f12e8161460 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.468+0000 7f12e5600640 1 -- 172.21.15.177:0/904473112 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f12e004a070 con 0x7f12e8161460 2024-09-16T07:22:26.469 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.468+0000 7f12e5600640 1 -- 172.21.15.177:0/904473112 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f12e002fd40 con 0x7f12e8161460 2024-09-16T07:22:26.470 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.469+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e80cf560 con 0x7f12e8161460 2024-09-16T07:22:26.472 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.472+0000 7f12e5600640 1 -- 172.21.15.177:0/904473112 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f12e002f840 con 0x7f12e8161460 2024-09-16T07:22:26.544 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.544+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "91f8990b-1328-48a3-aa70-6be7326adfc6", "id": 4} v 0) -- 0x7f12e80cdf80 con 0x7f12e8161460 2024-09-16T07:22:26.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.548+0000 7f12e5600640 1 -- 172.21.15.177:0/904473112 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91f8990b-1328-48a3-aa70-6be7326adfc6", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f12e0004b40 con 0x7f12e8161460 2024-09-16T07:22:26.549 INFO:teuthology.orchestra.run.smithi177.stdout:4 2024-09-16T07:22:26.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.549+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e8161460 msgr2=0x7f12e80b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.549 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.549+0000 7f12ee800640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e8161460 0x7f12e80b7650 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12e0032000 tx=0x7f12e0003f10 comp rx=0 tx=0).stop 2024-09-16T07:22:26.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.550+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 shutdown_connections 2024-09-16T07:22:26.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.550+0000 7f12ee800640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12e8161460 0x7f12e80b7650 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.550+0000 7f12ee800640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12e80b5480 0x7f12e80b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.550+0000 7f12ee800640 1 --2- 172.21.15.177:0/904473112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12e80b43c0 0x7f12e815b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.550+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 >> 172.21.15.177:0/904473112 conn(0x7f12e809d0b0 msgr2=0x7f12e80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:26.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.550+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 shutdown_connections 2024-09-16T07:22:26.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.550+0000 7f12ee800640 1 -- 172.21.15.177:0/904473112 wait complete. 2024-09-16T07:22:26.564 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 9bec3138-a06c-4a25-a0b2-dbe299fb2db3 5 2024-09-16T07:22:26.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.717+0000 7f4b2e600640 1 -- 172.21.15.177:0/232654866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b280cf830 msgr2=0x7f4b280b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.717+0000 7f4b2e600640 1 --2- 172.21.15.177:0/232654866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b280cf830 0x7f4b280b33b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4b280d1ce0 tx=0x7f4b1402a560 comp rx=0 tx=0).stop 2024-09-16T07:22:26.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.717+0000 7f4b2e600640 1 -- 172.21.15.177:0/232654866 shutdown_connections 2024-09-16T07:22:26.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.717+0000 7f4b2e600640 1 --2- 172.21.15.177:0/232654866 >> v2:172.21.15.71:3300/0 conn(0x7f4b280d2a00 0x7f4b280b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.717+0000 7f4b2e600640 1 --2- 172.21.15.177:0/232654866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b280cf830 0x7f4b280b33b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.717+0000 7f4b2e600640 1 -- 172.21.15.177:0/232654866 >> 172.21.15.177:0/232654866 conn(0x7f4b2809d0b0 msgr2=0x7f4b280d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:26.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.717+0000 7f4b2e600640 1 -- 172.21.15.177:0/232654866 shutdown_connections 2024-09-16T07:22:26.718 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.717+0000 7f4b2e600640 1 -- 172.21.15.177:0/232654866 wait complete. 2024-09-16T07:22:26.718 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.718+0000 7f4b2e600640 1 Processor -- start 2024-09-16T07:22:26.718 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.719+0000 7f4b2e600640 1 -- start start 2024-09-16T07:22:26.720 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.719+0000 7f4b2e600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b280cf830 0x7f4b281f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.720 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.719+0000 7f4b2e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b280d2a00 0x7f4b280b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.720 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.719+0000 7f4b2e600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b28161460 0x7f4b280b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.720 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.719+0000 7f4b2e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4b280b50c0 con 0x7f4b280d2a00 2024-09-16T07:22:26.721 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.719+0000 7f4b2e600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4b280b4f40 con 0x7f4b28161460 2024-09-16T07:22:26.721 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.719+0000 7f4b2e600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4b280b3e30 con 0x7f4b280cf830 2024-09-16T07:22:26.721 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b280cf830 0x7f4b281f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b280cf830 0x7f4b281f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46770/0 (socket says 172.21.15.177:46770) 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 -- 172.21.15.177:0/2590178117 learned_addr learned my addr 172.21.15.177:0/2590178117 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b27e00640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b280d2a00 0x7f4b280b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2dc00640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b28161460 0x7f4b280b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 -- 172.21.15.177:0/2590178117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b28161460 msgr2=0x7f4b280b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b28161460 0x7f4b280b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 -- 172.21.15.177:0/2590178117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b280d2a00 msgr2=0x7f4b280b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b280d2a00 0x7f4b280b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 -- 172.21.15.177:0/2590178117 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b280ca0e0 con 0x7f4b280cf830 2024-09-16T07:22:26.724 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.720+0000 7f4b2d200640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b280cf830 0x7f4b281f8660 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c3a0 tx=0x7f4b1c00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:26.725 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.721+0000 7f4b24c00640 1 -- 172.21.15.177:0/2590178117 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c017070 con 0x7f4b280cf830 2024-09-16T07:22:26.725 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.721+0000 7f4b24c00640 1 -- 172.21.15.177:0/2590178117 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b1c0053c0 con 0x7f4b280cf830 2024-09-16T07:22:26.725 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.721+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b281f4960 con 0x7f4b280cf830 2024-09-16T07:22:26.726 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.721+0000 7f4b24c00640 1 -- 172.21.15.177:0/2590178117 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c003850 con 0x7f4b280cf830 2024-09-16T07:22:26.726 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.721+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b281f4f20 con 0x7f4b280cf830 2024-09-16T07:22:26.726 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.721+0000 7f4b24c00640 1 -- 172.21.15.177:0/2590178117 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b1c01a070 con 0x7f4b280cf830 2024-09-16T07:22:26.726 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.721+0000 7f4b24c00640 1 -- 172.21.15.177:0/2590178117 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f4b1c011c20 con 0x7f4b280cf830 2024-09-16T07:22:26.726 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.722+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b280b42e0 con 0x7f4b280cf830 2024-09-16T07:22:26.726 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.726+0000 7f4b24c00640 1 -- 172.21.15.177:0/2590178117 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4b1c021020 con 0x7f4b280cf830 2024-09-16T07:22:26.798 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.799+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9bec3138-a06c-4a25-a0b2-dbe299fb2db3", "id": 5} v 0) -- 0x7f4b280d2580 con 0x7f4b280cf830 2024-09-16T07:22:26.803 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.803+0000 7f4b24c00640 1 -- 172.21.15.177:0/2590178117 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9bec3138-a06c-4a25-a0b2-dbe299fb2db3", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f4b1c011590 con 0x7f4b280cf830 2024-09-16T07:22:26.803 INFO:teuthology.orchestra.run.smithi177.stdout:5 2024-09-16T07:22:26.804 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.804+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b280cf830 msgr2=0x7f4b281f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.804+0000 7f4b2e600640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b280cf830 0x7f4b281f8660 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c3a0 tx=0x7f4b1c00c860 comp rx=0 tx=0).stop 2024-09-16T07:22:26.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.804+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 shutdown_connections 2024-09-16T07:22:26.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.804+0000 7f4b2e600640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b28161460 0x7f4b280b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.804+0000 7f4b2e600640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b280d2a00 0x7f4b280b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.804+0000 7f4b2e600640 1 --2- 172.21.15.177:0/2590178117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b280cf830 0x7f4b281f8660 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.804+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 >> 172.21.15.177:0/2590178117 conn(0x7f4b2809d0b0 msgr2=0x7f4b2806d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:26.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.805+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 shutdown_connections 2024-09-16T07:22:26.805 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.805+0000 7f4b2e600640 1 -- 172.21.15.177:0/2590178117 wait complete. 2024-09-16T07:22:26.818 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 12a9547a-4112-4da5-8392-a6af10dccf47 6 2024-09-16T07:22:26.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.972+0000 7f973a600640 1 -- 172.21.15.177:0/3486515658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340d2000 msgr2=0x7f97340d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.972+0000 7f973a600640 1 --2- 172.21.15.177:0/3486515658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340d2000 0x7f97340d23e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9728009860 tx=0x7f972802a670 comp rx=0 tx=0).stop 2024-09-16T07:22:26.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.972+0000 7f973a600640 1 -- 172.21.15.177:0/3486515658 shutdown_connections 2024-09-16T07:22:26.972 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.972+0000 7f973a600640 1 --2- 172.21.15.177:0/3486515658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340d2000 0x7f97340d23e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.972+0000 7f973a600640 1 -- 172.21.15.177:0/3486515658 >> 172.21.15.177:0/3486515658 conn(0x7f97340d3e20 msgr2=0x7f97340d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:26.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.972+0000 7f973a600640 1 -- 172.21.15.177:0/3486515658 shutdown_connections 2024-09-16T07:22:26.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.972+0000 7f973a600640 1 -- 172.21.15.177:0/3486515658 wait complete. 2024-09-16T07:22:26.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.974+0000 7f973a600640 1 Processor -- start 2024-09-16T07:22:26.973 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.974+0000 7f973a600640 1 -- start start 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f973a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340b4e70 0x7f97341f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f973a600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97340d2000 0x7f97341511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f973a600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97340d2920 0x7f973414e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f973a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f97341692e0 con 0x7f97340b4e70 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f973a600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9734169160 con 0x7f97340d2920 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f973a600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9734169460 con 0x7f97340d2000 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f9733e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97340d2000 0x7f97341511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f9739200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340b4e70 0x7f97341f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f9739c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97340d2920 0x7f973414e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:26.975 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f9739200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340b4e70 0x7f97341f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.177:46820/0 (socket says 172.21.15.177:46820) 2024-09-16T07:22:26.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f9733e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97340d2000 0x7f97341511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46774/0 (socket says 172.21.15.177:46774) 2024-09-16T07:22:26.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f9739200640 1 -- 172.21.15.177:0/3842835515 learned_addr learned my addr 172.21.15.177:0/3842835515 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.975+0000 7f9739c00640 1 -- 172.21.15.177:0/3842835515 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97340d2000 msgr2=0x7f97341511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9739c00640 1 --2- 172.21.15.177:0/3842835515 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97340d2000 0x7f97341511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9739c00640 1 -- 172.21.15.177:0/3842835515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340b4e70 msgr2=0x7f97341f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9739c00640 1 --2- 172.21.15.177:0/3842835515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340b4e70 0x7f97341f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9739c00640 1 -- 172.21.15.177:0/3842835515 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97341516f0 con 0x7f97340d2920 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9739200640 1 --2- 172.21.15.177:0/3842835515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340b4e70 0x7f97341f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9739c00640 1 --2- 172.21.15.177:0/3842835515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97340d2920 0x7f973414e9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f972c00caa0 tx=0x7f972c00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9730c00640 1 -- 172.21.15.177:0/3842835515 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c016020 con 0x7f97340d2920 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9730c00640 1 -- 172.21.15.177:0/3842835515 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f972c00ad10 con 0x7f97340d2920 2024-09-16T07:22:26.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9730c00640 1 -- 172.21.15.177:0/3842835515 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c012680 con 0x7f97340d2920 2024-09-16T07:22:26.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9734151980 con 0x7f97340d2920 2024-09-16T07:22:26.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9734151f40 con 0x7f97340d2920 2024-09-16T07:22:26.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.976+0000 7f9730c00640 1 -- 172.21.15.177:0/3842835515 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f972c019020 con 0x7f97340d2920 2024-09-16T07:22:26.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.977+0000 7f9730c00640 1 -- 172.21.15.177:0/3842835515 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f972c01bc20 con 0x7f97340d2920 2024-09-16T07:22:26.978 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.977+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9734152140 con 0x7f97340d2920 2024-09-16T07:22:26.979 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:26.979+0000 7f9730c00640 1 -- 172.21.15.177:0/3842835515 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9734152140 con 0x7f97340d2920 2024-09-16T07:22:27.053 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.053+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "12a9547a-4112-4da5-8392-a6af10dccf47", "id": 6} v 0) -- 0x7f97340d23e0 con 0x7f97340d2920 2024-09-16T07:22:27.057 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.058+0000 7f9730c00640 1 -- 172.21.15.177:0/3842835515 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12a9547a-4112-4da5-8392-a6af10dccf47", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f972c020020 con 0x7f97340d2920 2024-09-16T07:22:27.057 INFO:teuthology.orchestra.run.smithi177.stdout:6 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97340d2920 msgr2=0x7f973414e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 --2- 172.21.15.177:0/3842835515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97340d2920 0x7f973414e9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f972c00caa0 tx=0x7f972c00cf60 comp rx=0 tx=0).stop 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 shutdown_connections 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 --2- 172.21.15.177:0/3842835515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97340d2920 0x7f973414e9f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 --2- 172.21.15.177:0/3842835515 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97340d2000 0x7f97341511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 --2- 172.21.15.177:0/3842835515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97340b4e70 0x7f97341f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 >> 172.21.15.177:0/3842835515 conn(0x7f97340d3e20 msgr2=0x7f973415a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 shutdown_connections 2024-09-16T07:22:27.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.059+0000 7f973a600640 1 -- 172.21.15.177:0/3842835515 wait complete. 2024-09-16T07:22:27.073 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 07b5fcb7-d68d-4137-9fbd-688db353bed2 7 2024-09-16T07:22:27.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.227+0000 7f29a0200640 1 -- 172.21.15.177:0/3055948374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2998161590 msgr2=0x7f29980b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.227+0000 7f29a0200640 1 --2- 172.21.15.177:0/3055948374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2998161590 0x7f29980b6e00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f29980cf2a0 tx=0x7f298c02a5c0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.227+0000 7f29a0200640 1 -- 172.21.15.177:0/3055948374 shutdown_connections 2024-09-16T07:22:27.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.227+0000 7f29a0200640 1 --2- 172.21.15.177:0/3055948374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2998161590 0x7f29980b6e00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.227+0000 7f29a0200640 1 -- 172.21.15.177:0/3055948374 >> 172.21.15.177:0/3055948374 conn(0x7f29980d3fb0 msgr2=0x7f29980d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:27.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.228+0000 7f29a0200640 1 -- 172.21.15.177:0/3055948374 shutdown_connections 2024-09-16T07:22:27.228 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.228+0000 7f29a0200640 1 -- 172.21.15.177:0/3055948374 wait complete. 2024-09-16T07:22:27.229 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.229+0000 7f29a0200640 1 Processor -- start 2024-09-16T07:22:27.229 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.229+0000 7f29a0200640 1 -- start start 2024-09-16T07:22:27.230 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f29a0200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29980b5630 0x7f29981f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:27.230 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f29a0200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29980b7340 0x7f299815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:27.230 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f29a0200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2998161590 0x7f29980cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:27.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f29a0200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29980d2740 con 0x7f29980b5630 2024-09-16T07:22:27.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f29a0200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f29980cfed0 con 0x7f29980b7340 2024-09-16T07:22:27.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f29a0200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f29980d28c0 con 0x7f2998161590 2024-09-16T07:22:27.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f299d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29980b5630 0x7f29981f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:27.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f299ce00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29980b7340 0x7f299815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:27.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f299e200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2998161590 0x7f29980cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:27.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.230+0000 7f299d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29980b5630 0x7f29981f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.177:46822/0 (socket says 172.21.15.177:46822) 2024-09-16T07:22:27.231 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299d800640 1 -- 172.21.15.177:0/3035635793 learned_addr learned my addr 172.21.15.177:0/3035635793 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-16T07:22:27.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299d800640 1 -- 172.21.15.177:0/3035635793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2998161590 msgr2=0x7f29980cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299d800640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2998161590 0x7f29980cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299d800640 1 -- 172.21.15.177:0/3035635793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29980b7340 msgr2=0x7f299815c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299d800640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29980b7340 0x7f299815c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299d800640 1 -- 172.21.15.177:0/3035635793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29981f4ee0 con 0x7f29980b5630 2024-09-16T07:22:27.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299e200640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2998161590 0x7f29980cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:27.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299ce00640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29980b7340 0x7f299815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:27.233 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.231+0000 7f299d800640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29980b5630 0x7f29981f8970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f298c009780 tx=0x7f298c033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:27.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.232+0000 7f2995600640 1 -- 172.21.15.177:0/3035635793 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c031d60 con 0x7f29980b5630 2024-09-16T07:22:27.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.232+0000 7f2995600640 1 -- 172.21.15.177:0/3035635793 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f298c005020 con 0x7f29980b5630 2024-09-16T07:22:27.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.232+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29981f51c0 con 0x7f29980b5630 2024-09-16T07:22:27.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.232+0000 7f2995600640 1 -- 172.21.15.177:0/3035635793 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c005470 con 0x7f29980b5630 2024-09-16T07:22:27.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.232+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29981f5780 con 0x7f29980b5630 2024-09-16T07:22:27.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.232+0000 7f2995600640 1 -- 172.21.15.177:0/3035635793 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f298c004b30 con 0x7f29980b5630 2024-09-16T07:22:27.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.232+0000 7f2995600640 1 -- 172.21.15.177:0/3035635793 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f298c004e20 con 0x7f29980b5630 2024-09-16T07:22:27.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.233+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29980d2d40 con 0x7f29980b5630 2024-09-16T07:22:27.236 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.237+0000 7f2995600640 1 -- 172.21.15.177:0/3035635793 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f298c03f030 con 0x7f29980b5630 2024-09-16T07:22:27.310 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.310+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd new", "uuid": "07b5fcb7-d68d-4137-9fbd-688db353bed2", "id": 7} v 0) -- 0x7f29980caa60 con 0x7f29980b5630 2024-09-16T07:22:27.312 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.312+0000 7f2995600640 1 -- 172.21.15.177:0/3035635793 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07b5fcb7-d68d-4137-9fbd-688db353bed2", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f298c005610 con 0x7f29980b5630 2024-09-16T07:22:27.312 INFO:teuthology.orchestra.run.smithi177.stdout:7 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29980b5630 msgr2=0x7f29981f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29980b5630 0x7f29981f8970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f298c009780 tx=0x7f298c033d60 comp rx=0 tx=0).stop 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 shutdown_connections 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2998161590 0x7f29980cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29980b7340 0x7f299815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 --2- 172.21.15.177:0/3035635793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29980b5630 0x7f29981f8970 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 >> 172.21.15.177:0/3035635793 conn(0x7f29980d3fb0 msgr2=0x7f29980cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 shutdown_connections 2024-09-16T07:22:27.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:22:27.314+0000 7f29a0200640 1 -- 172.21.15.177:0/3035635793 wait complete. 2024-09-16T07:22:27.329 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T07:22:27.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T07:22:27.333 INFO:tasks.ceph.osd.0:Started 2024-09-16T07:22:27.333 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T07:22:27.334 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T07:22:27.337 INFO:tasks.ceph.osd.1:Started 2024-09-16T07:22:27.337 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T07:22:27.337 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T07:22:27.341 INFO:tasks.ceph.osd.2:Started 2024-09-16T07:22:27.341 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T07:22:27.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T07:22:27.346 INFO:tasks.ceph.osd.3:Started 2024-09-16T07:22:27.346 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T07:22:27.346 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T07:22:27.350 INFO:tasks.ceph.osd.4:Started 2024-09-16T07:22:27.350 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T07:22:27.350 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T07:22:27.354 INFO:tasks.ceph.osd.5:Started 2024-09-16T07:22:27.354 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T07:22:27.354 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T07:22:27.358 INFO:tasks.ceph.osd.6:Started 2024-09-16T07:22:27.358 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T07:22:27.358 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T07:22:27.363 INFO:tasks.ceph.osd.7:Started 2024-09-16T07:22:27.363 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:27.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.577+0000 7f6237200640 1 -- 172.21.15.71:0/2276297282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62300709e0 msgr2=0x7f6230073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.577+0000 7f6237200640 1 --2- 172.21.15.71:0/2276297282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62300709e0 0x7f6230073af0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6220005fd0 tx=0x7f622002a5d0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.577+0000 7f6237200640 1 -- 172.21.15.71:0/2276297282 shutdown_connections 2024-09-16T07:22:27.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.577+0000 7f6237200640 1 --2- 172.21.15.71:0/2276297282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62300ca4d0 0x7f623015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.577+0000 7f6237200640 1 --2- 172.21.15.71:0/2276297282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62300709e0 0x7f6230073af0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.577+0000 7f6237200640 1 -- 172.21.15.71:0/2276297282 >> 172.21.15.71:0/2276297282 conn(0x7f62300b2b90 msgr2=0x7f62300b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:27.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.578+0000 7f6237200640 1 -- 172.21.15.71:0/2276297282 shutdown_connections 2024-09-16T07:22:27.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.578+0000 7f6237200640 1 -- 172.21.15.71:0/2276297282 wait complete. 2024-09-16T07:22:27.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.579+0000 7f6237200640 1 Processor -- start 2024-09-16T07:22:27.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.579+0000 7f6237200640 1 -- start start 2024-09-16T07:22:27.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f6237200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623006cb70 0x7f6230156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:27.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f6237200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300709e0 0x7f62300b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:27.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f6237200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62300ca4d0 0x7f6230155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:27.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f6237200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62300d0250 con 0x7f623006cb70 2024-09-16T07:22:27.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f6237200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f62300d00d0 con 0x7f62300709e0 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f6237200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f62300d03d0 con 0x7f62300ca4d0 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f622fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623006cb70 0x7f6230156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f622fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623006cb70 0x7f6230156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40658/0 (socket says 172.21.15.71:40658) 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f622fe00640 1 -- 172.21.15.71:0/4006409642 learned_addr learned my addr 172.21.15.71:0/4006409642 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f6227e00640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300709e0 0x7f62300b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.580+0000 7f6235200640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62300ca4d0 0x7f6230155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622fe00640 1 -- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62300ca4d0 msgr2=0x7f6230155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622fe00640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62300ca4d0 0x7f6230155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622fe00640 1 -- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300709e0 msgr2=0x7f62300b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622fe00640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300709e0 0x7f62300b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622fe00640 1 -- 172.21.15.71:0/4006409642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62300b6980 con 0x7f623006cb70 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f6227e00640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300709e0 0x7f62300b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:27.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f6235200640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62300ca4d0 0x7f6230155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:27.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622fe00640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623006cb70 0x7f6230156f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62200099f0 tx=0x7f6220003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:27.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622cc00640 1 -- 172.21.15.71:0/4006409642 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622003f070 con 0x7f623006cb70 2024-09-16T07:22:27.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622cc00640 1 -- 172.21.15.71:0/4006409642 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6220005600 con 0x7f623006cb70 2024-09-16T07:22:27.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f622cc00640 1 -- 172.21.15.71:0/4006409642 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220035030 con 0x7f623006cb70 2024-09-16T07:22:27.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62300b6c60 con 0x7f623006cb70 2024-09-16T07:22:27.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.581+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62300cdea0 con 0x7f623006cb70 2024-09-16T07:22:27.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.582+0000 7f622cc00640 1 -- 172.21.15.71:0/4006409642 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6220042070 con 0x7f623006cb70 2024-09-16T07:22:27.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.582+0000 7f622cc00640 1 -- 172.21.15.71:0/4006409642 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f622003bb30 con 0x7f623006cb70 2024-09-16T07:22:27.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.582+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623006d2f0 con 0x7f623006cb70 2024-09-16T07:22:27.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.585+0000 7f622cc00640 1 -- 172.21.15.71:0/4006409642 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f622003bdd0 con 0x7f623006cb70 2024-09-16T07:22:27.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.647+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62300701e0 con 0x7f623006cb70 2024-09-16T07:22:27.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.648+0000 7f622cc00640 1 -- 172.21.15.71:0/4006409642 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f622003b460 con 0x7f623006cb70 2024-09-16T07:22:27.648 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:27.648 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":10,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:27.312216+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:27.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623006cb70 msgr2=0x7f6230156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:27.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623006cb70 0x7f6230156f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62200099f0 tx=0x7f6220003820 comp rx=0 tx=0).stop 2024-09-16T07:22:27.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 shutdown_connections 2024-09-16T07:22:27.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62300ca4d0 0x7f6230155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300709e0 0x7f62300b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 --2- 172.21.15.71:0/4006409642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623006cb70 0x7f6230156f80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:27.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 >> 172.21.15.71:0/4006409642 conn(0x7f62300b2b90 msgr2=0x7f623015c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:27.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 shutdown_connections 2024-09-16T07:22:27.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:27.649+0000 7f6237200640 1 -- 172.21.15.71:0/4006409642 wait complete. 2024-09-16T07:22:27.663 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T07:22:27.663 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T07:22:28.470 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:28.613 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 -- 172.21.15.71:0/243125595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200b2fa0 msgr2=0x7f96200d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:28.613 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 --2- 172.21.15.71:0/243125595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200b2fa0 0x7f96200d2950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f962008d6a0 tx=0x7f961002a610 comp rx=0 tx=0).stop 2024-09-16T07:22:28.613 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 -- 172.21.15.71:0/243125595 shutdown_connections 2024-09-16T07:22:28.613 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 --2- 172.21.15.71:0/243125595 >> v2:172.21.15.177:3300/0 conn(0x7f96200d1ba0 0x7f9620161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.613 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 --2- 172.21.15.71:0/243125595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200b2fa0 0x7f96200d2950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.613 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 --2- 172.21.15.71:0/243125595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962015de90 0x7f96200d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.614 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 -- 172.21.15.71:0/243125595 >> 172.21.15.71:0/243125595 conn(0x7f96200d3fd0 msgr2=0x7f96200d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:28.614 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 -- 172.21.15.71:0/243125595 shutdown_connections 2024-09-16T07:22:28.614 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.612+0000 7f9628800640 1 -- 172.21.15.71:0/243125595 wait complete. 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.614+0000 7f9628800640 1 Processor -- start 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.614+0000 7f9628800640 1 -- start start 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9628800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200d1ba0 0x7f96200b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9628800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962015de90 0x7f96200b8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9628800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96200b6e00 0x7f96200b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9628800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9620162070 con 0x7f96200d1ba0 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9628800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f962015e480 con 0x7f96200b6e00 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9628800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96201621f0 con 0x7f962015de90 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9625e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200d1ba0 0x7f96200b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:28.616 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9625e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200d1ba0 0x7f96200b96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40670/0 (socket says 172.21.15.71:40670) 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9625e00640 1 -- 172.21.15.71:0/1692708552 learned_addr learned my addr 172.21.15.71:0/1692708552 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9625400640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962015de90 0x7f96200b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.615+0000 7f9626800640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96200b6e00 0x7f96200b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f9625e00640 1 -- 172.21.15.71:0/1692708552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962015de90 msgr2=0x7f96200b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f9625e00640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962015de90 0x7f96200b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f9625e00640 1 -- 172.21.15.71:0/1692708552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96200b6e00 msgr2=0x7f96200b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f9625e00640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96200b6e00 0x7f96200b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f9625e00640 1 -- 172.21.15.71:0/1692708552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962015a8b0 con 0x7f96200d1ba0 2024-09-16T07:22:28.617 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f9625e00640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200d1ba0 0x7f96200b96e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f961400e7c0 tx=0x7f961400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:28.618 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f961a000640 1 -- 172.21.15.71:0/1692708552 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400cab0 con 0x7f96200d1ba0 2024-09-16T07:22:28.618 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f961a000640 1 -- 172.21.15.71:0/1692708552 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f961400cd40 con 0x7f96200d1ba0 2024-09-16T07:22:28.618 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f961a000640 1 -- 172.21.15.71:0/1692708552 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614010690 con 0x7f96200d1ba0 2024-09-16T07:22:28.618 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96200b7300 con 0x7f96200d1ba0 2024-09-16T07:22:28.618 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.616+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96200b78c0 con 0x7f96200d1ba0 2024-09-16T07:22:28.618 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.617+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9620162710 con 0x7f96200d1ba0 2024-09-16T07:22:28.619 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.618+0000 7f961a000640 1 -- 172.21.15.71:0/1692708552 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 2) ==== 50665+0+0 (secure 0 0 0) 0x7f9614010830 con 0x7f96200d1ba0 2024-09-16T07:22:28.619 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.618+0000 7f961a000640 1 -- 172.21.15.71:0/1692708552 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9614052ba0 con 0x7f96200d1ba0 2024-09-16T07:22:28.619 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.620+0000 7f961a000640 1 -- 172.21.15.71:0/1692708552 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9614017df0 con 0x7f96200d1ba0 2024-09-16T07:22:28.684 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.684+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96200cb0a0 con 0x7f96200d1ba0 2024-09-16T07:22:28.684 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.685+0000 7f961a000640 1 -- 172.21.15.71:0/1692708552 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f96140187a0 con 0x7f96200d1ba0 2024-09-16T07:22:28.684 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-16T07:22:28.685 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":10,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:27.312216+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200d1ba0 msgr2=0x7f96200b96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200d1ba0 0x7f96200b96e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f961400e7c0 tx=0x7f961400ebc0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 shutdown_connections 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96200b6e00 0x7f96200b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962015de90 0x7f96200b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 --2- 172.21.15.71:0/1692708552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96200d1ba0 0x7f96200b96e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 >> 172.21.15.71:0/1692708552 conn(0x7f96200d3fd0 msgr2=0x7f962006d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 shutdown_connections 2024-09-16T07:22:28.686 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:28.686+0000 7f9628800640 1 -- 172.21.15.71:0/1692708552 wait complete. 2024-09-16T07:22:28.697 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T07:22:28.838 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-09-16T07:22:28.838+0000 7fe80121d640 -1 Falling back to public interface 2024-09-16T07:22:28.861 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-16T07:22:28.862+0000 7f2b97125640 -1 Falling back to public interface 2024-09-16T07:22:28.925 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-09-16T07:22:28.925+0000 7fda0382e640 -1 Falling back to public interface 2024-09-16T07:22:28.939 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-16T07:22:28.940+0000 7f650c410640 -1 Falling back to public interface 2024-09-16T07:22:29.124 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-09-16T07:22:29.125+0000 7f0bfc00e640 -1 Falling back to public interface 2024-09-16T07:22:29.128 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-09-16T07:22:29.129+0000 7fd2afe0f640 -1 Falling back to public interface 2024-09-16T07:22:29.130 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-16T07:22:29.131+0000 7fa4adc0f640 -1 Falling back to public interface 2024-09-16T07:22:29.182 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-16T07:22:29.183+0000 7f49e2cb2640 -1 Falling back to public interface 2024-09-16T07:22:29.299 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:22:29.299+0000 7fe794000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T07:22:29.299 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:22:29.300+0000 7fe794000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T07:22:32.496 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-09-16T07:22:32.497+0000 7fe80121d640 -1 osd.4 0 log_to_monitors true 2024-09-16T07:22:32.818 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-16T07:22:32.819+0000 7f2b97125640 -1 osd.0 0 log_to_monitors true 2024-09-16T07:22:33.127 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-16T07:22:33.127+0000 7fa4adc0f640 -1 osd.1 0 log_to_monitors true 2024-09-16T07:22:33.154 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-16T07:22:33.155+0000 7f49e2cb2640 -1 osd.2 0 log_to_monitors true 2024-09-16T07:22:33.348 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-09-16T07:22:33.349+0000 7f0bfc00e640 -1 osd.6 0 log_to_monitors true 2024-09-16T07:22:33.361 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-09-16T07:22:33.362+0000 7fda0382e640 -1 osd.7 0 log_to_monitors true 2024-09-16T07:22:33.411 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-16T07:22:33.412+0000 7f650c410640 -1 osd.3 0 log_to_monitors true 2024-09-16T07:22:33.566 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-09-16T07:22:33.567+0000 7fd2afe0f640 -1 osd.5 0 log_to_monitors true 2024-09-16T07:22:34.583 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-09-16T07:22:34.584+0000 7fe7fc600640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T07:22:34.601 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-09-16T07:22:34.602+0000 7fe7f1c00640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T07:22:35.505 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:35.677 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.677+0000 7f4009e00640 1 -- 172.21.15.71:0/90442710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040d07c0 msgr2=0x7f40040d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:35.677 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.677+0000 7f4009e00640 1 --2- 172.21.15.71:0/90442710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040d07c0 0x7f40040d0ba0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005fd0 tx=0x7f3ff802a540 comp rx=0 tx=0).stop 2024-09-16T07:22:35.677 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.677+0000 7f4009e00640 1 -- 172.21.15.71:0/90442710 shutdown_connections 2024-09-16T07:22:35.677 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.677+0000 7f4009e00640 1 --2- 172.21.15.71:0/90442710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040b1020 0x7f40040b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:35.677 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.677+0000 7f4009e00640 1 --2- 172.21.15.71:0/90442710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040d07c0 0x7f40040d0ba0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:35.677 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.677+0000 7f4009e00640 1 -- 172.21.15.71:0/90442710 >> 172.21.15.71:0/90442710 conn(0x7f400409d070 msgr2=0x7f40040d3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:35.678 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.677+0000 7f4009e00640 1 -- 172.21.15.71:0/90442710 shutdown_connections 2024-09-16T07:22:35.678 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.677+0000 7f4009e00640 1 -- 172.21.15.71:0/90442710 wait complete. 2024-09-16T07:22:35.678 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.679+0000 7f4009e00640 1 Processor -- start 2024-09-16T07:22:35.678 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.679+0000 7f4009e00640 1 -- start start 2024-09-16T07:22:35.680 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040b1020 0x7f4004155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:35.680 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b2eb0 0x7f40040b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:35.680 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40040d07c0 0x7f40040cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f40040b1d80 con 0x7f40040d07c0 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40040b1c00 con 0x7f40040b2eb0 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f40040b1f00 con 0x7f40040b1020 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40040d07c0 0x7f40040cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40040d07c0 0x7f40040cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51762/0 (socket says 172.21.15.71:51762) 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.680+0000 7f4009400640 1 -- 172.21.15.71:0/540554597 learned_addr learned my addr 172.21.15.71:0/540554597 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.681+0000 7f4008a00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040b1020 0x7f4004155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.681+0000 7f4003e00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b2eb0 0x7f40040b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.681+0000 7f4009400640 1 -- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040b1020 msgr2=0x7f4004155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.681+0000 7f4009400640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040b1020 0x7f4004155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.681+0000 7f4009400640 1 -- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b2eb0 msgr2=0x7f40040b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.681+0000 7f4009400640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b2eb0 0x7f40040b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.681+0000 7f4009400640 1 -- 172.21.15.71:0/540554597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40040b8060 con 0x7f40040d07c0 2024-09-16T07:22:35.681 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.681+0000 7f4009400640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40040d07c0 0x7f40040cb040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00eb10 tx=0x7f3ffc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:35.695 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.696+0000 7f4000c00640 1 -- 172.21.15.71:0/540554597 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc004b20 con 0x7f40040d07c0 2024-09-16T07:22:35.695 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.696+0000 7f4000c00640 1 -- 172.21.15.71:0/540554597 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ffc004ce0 con 0x7f40040d07c0 2024-09-16T07:22:35.696 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.696+0000 7f4000c00640 1 -- 172.21.15.71:0/540554597 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc005280 con 0x7f40040d07c0 2024-09-16T07:22:35.696 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.696+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40040b8340 con 0x7f40040d07c0 2024-09-16T07:22:35.696 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.696+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40040b4120 con 0x7f40040d07c0 2024-09-16T07:22:35.696 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.697+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400406f080 con 0x7f40040d07c0 2024-09-16T07:22:35.698 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.699+0000 7f4000c00640 1 -- 172.21.15.71:0/540554597 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) ==== 101231+0+0 (secure 0 0 0) 0x7f3ffc005420 con 0x7f40040d07c0 2024-09-16T07:22:35.700 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.701+0000 7f4000c00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3fe40787c0 0x7f3fe407ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:35.700 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.701+0000 7f4000c00640 1 -- 172.21.15.71:0/540554597 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f3ffc09a760 con 0x7f40040d07c0 2024-09-16T07:22:35.702 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.703+0000 7f4008a00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3fe40787c0 0x7f3fe407ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:35.703 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.704+0000 7f4008a00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3fe40787c0 0x7f3fe407ac80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8033000 tx=0x7f3ff8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:35.707 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.707+0000 7f4000c00640 1 -- 172.21.15.71:0/540554597 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3ffc063b20 con 0x7f40040d07c0 2024-09-16T07:22:35.979 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-09-16T07:22:35.981+0000 7fd9fe600640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T07:22:35.981 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:35.981+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40040d4cc0 con 0x7f40040d07c0 2024-09-16T07:22:35.987 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-09-16T07:22:35.988+0000 7fd9f3c00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T07:22:35.991 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-09-16T07:22:35.993+0000 7f0bf7200640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T07:22:36.008 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-16T07:22:36.008 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":14,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:35.561986+0000","last_up_change":"2024-09-16T07:22:35.561986+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:36.008 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.007+0000 7f4000c00640 1 -- 172.21.15.71:0/540554597 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7567 (secure 0 0 0) 0x7f3ffc063420 con 0x7f40040d07c0 2024-09-16T07:22:36.010 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.010+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3fe40787c0 msgr2=0x7f3fe407ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:36.010 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.010+0000 7f4009e00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3fe40787c0 0x7f3fe407ac80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8033000 tx=0x7f3ff8004730 comp rx=0 tx=0).stop 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.010+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40040d07c0 msgr2=0x7f40040cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.010+0000 7f4009e00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40040d07c0 0x7f40040cb040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00eb10 tx=0x7f3ffc00c2d0 comp rx=0 tx=0).stop 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.011+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 shutdown_connections 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.011+0000 7f4009e00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3fe40787c0 0x7f3fe407ac80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.011+0000 7f4009e00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40040d07c0 0x7f40040cb040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.011+0000 7f4009e00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b2eb0 0x7f40040b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.011+0000 7f4009e00640 1 --2- 172.21.15.71:0/540554597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040b1020 0x7f4004155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.011+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 >> 172.21.15.71:0/540554597 conn(0x7f400409d070 msgr2=0x7f40040d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.011+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 shutdown_connections 2024-09-16T07:22:36.011 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:36.011+0000 7f4009e00640 1 -- 172.21.15.71:0/540554597 wait complete. 2024-09-16T07:22:36.018 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-09-16T07:22:36.020+0000 7f0bec800640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T07:22:36.030 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-16T07:22:36.743 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-09-16T07:22:36.745+0000 7fd2ab000640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T07:22:36.751 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-09-16T07:22:36.752+0000 7fd2a0600640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T07:22:36.834 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-16T07:22:36.835+0000 7f2b92000640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T07:22:36.843 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-09-16T07:22:36.843+0000 7f2b88000640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T07:22:36.866 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-16T07:22:36.866+0000 7fa4a9000640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T07:22:36.877 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-09-16T07:22:36.878+0000 7fa49e600640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T07:22:36.899 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-16T07:22:36.900+0000 7f6507600640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T07:22:36.905 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-09-16T07:22:36.905+0000 7f64fcc00640 -1 osd.3 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T07:22:36.938 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-16T07:22:36.939+0000 7f49de600640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T07:22:36.954 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-09-16T07:22:36.954+0000 7f49d3c00640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T07:22:42.837 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:42.983 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.983+0000 7f8819c00640 1 -- 172.21.15.71:0/2433019361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 msgr2=0x7f87f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:42.983 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.983+0000 7f8819c00640 1 --2- 172.21.15.71:0/2433019361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 0x7f87f800b9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f880c009f00 tx=0x7f87f8005830 comp rx=0 tx=0).stop 2024-09-16T07:22:42.983 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.983+0000 7f8819200640 1 -- 172.21.15.71:0/2433019361 reap_dead start 2024-09-16T07:22:42.984 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.983+0000 7f8819c00640 1 -- 172.21.15.71:0/2433019361 shutdown_connections 2024-09-16T07:22:42.984 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.983+0000 7f8819c00640 1 -- 172.21.15.71:0/2433019361 >> 172.21.15.71:0/2433019361 conn(0x7f88140ccca0 msgr2=0x7f88140cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:42.984 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.983+0000 7f8819c00640 1 -- 172.21.15.71:0/2433019361 shutdown_connections 2024-09-16T07:22:42.984 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.983+0000 7f8819c00640 1 -- 172.21.15.71:0/2433019361 wait complete. 2024-09-16T07:22:42.984 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.985+0000 7f8819c00640 1 Processor -- start 2024-09-16T07:22:42.984 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.985+0000 7f8819c00640 1 -- start start 2024-09-16T07:22:42.986 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8819c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 0x7f881409b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8819c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881406f170 0x7f88140c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8819c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8814152570 0x7f8814152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8819c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8814072ce0 con 0x7f87f80051c0 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8819c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8814072b60 con 0x7f8814152570 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8819c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8814072e60 con 0x7f881406f170 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8813e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 0x7f881409b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8813e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 0x7f881409b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51788/0 (socket says 172.21.15.71:51788) 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8813e00640 1 -- 172.21.15.71:0/2331037045 learned_addr learned my addr 172.21.15.71:0/2331037045 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f880be00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881406f170 0x7f88140c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.986+0000 7f8819200640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8814152570 0x7f8814152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:42.987 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8813e00640 1 -- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881406f170 msgr2=0x7f88140c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8813e00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881406f170 0x7f88140c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8813e00640 1 -- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8814152570 msgr2=0x7f8814152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8813e00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8814152570 0x7f8814152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8813e00640 1 -- 172.21.15.71:0/2331037045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8814157720 con 0x7f87f80051c0 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f880be00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881406f170 0x7f88140c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8813e00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 0x7f881409b470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8804011900 tx=0x7f8804011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8810c00640 1 -- 172.21.15.71:0/2331037045 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880400fe10 con 0x7f87f80051c0 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8810c00640 1 -- 172.21.15.71:0/2331037045 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8804004030 con 0x7f87f80051c0 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8814157a00 con 0x7f87f80051c0 2024-09-16T07:22:42.988 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8810c00640 1 -- 172.21.15.71:0/2331037045 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004480 con 0x7f87f80051c0 2024-09-16T07:22:42.989 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.987+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8814156fc0 con 0x7f87f80051c0 2024-09-16T07:22:42.989 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.988+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881409df30 con 0x7f87f80051c0 2024-09-16T07:22:42.989 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.989+0000 7f8810c00640 1 -- 172.21.15.71:0/2331037045 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f88040035f0 con 0x7f87f80051c0 2024-09-16T07:22:42.990 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.990+0000 7f8810c00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f87f0078880 0x7f87f007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:42.990 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.990+0000 7f8810c00640 1 -- 172.21.15.71:0/2331037045 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f880409b300 con 0x7f87f80051c0 2024-09-16T07:22:42.990 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.991+0000 7f880be00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f87f0078880 0x7f87f007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:42.991 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.992+0000 7f880be00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f87f0078880 0x7f87f007ad40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87fc001860 tx=0x7f87fc001110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:42.995 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:42.995+0000 7f8810c00640 1 -- 172.21.15.71:0/2331037045 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f881409df30 con 0x7f87f80051c0 2024-09-16T07:22:43.182 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.182+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88140b4180 con 0x7f87f80051c0 2024-09-16T07:22:43.182 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.183+0000 7f8810c00640 1 -- 172.21.15.71:0/2331037045 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12908 (secure 0 0 0) 0x7f8804063d50 con 0x7f87f80051c0 2024-09-16T07:22:43.183 INFO:teuthology.misc.health.smithi071.stdout: 2024-09-16T07:22:43.183 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":19,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:40.857580+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:43.185 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f87f0078880 msgr2=0x7f87f007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f87f0078880 0x7f87f007ad40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87fc001860 tx=0x7f87fc001110 comp rx=0 tx=0).stop 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 msgr2=0x7f881409b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 0x7f881409b470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8804011900 tx=0x7f8804011dc0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 shutdown_connections 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f87f0078880 0x7f87f007ad40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8814152570 0x7f8814152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881406f170 0x7f88140c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 --2- 172.21.15.71:0/2331037045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f80051c0 0x7f881409b470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 >> 172.21.15.71:0/2331037045 conn(0x7f88140ccca0 msgr2=0x7f881406fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.185+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 shutdown_connections 2024-09-16T07:22:43.186 INFO:teuthology.misc.health.smithi071.stderr:2024-09-16T07:22:43.186+0000 7f8819c00640 1 -- 172.21.15.71:0/2331037045 wait complete. 2024-09-16T07:22:43.197 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T07:22:43.197 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T07:22:43.198 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:22:43.198 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:22:43.203 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:22:43.203 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:22:43.203 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:22:43.206 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:22:43.206 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:22:43.207 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:22:43.211 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:22:43.211 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:22:43.211 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:22:43.215 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:22:43.215 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-16T07:22:43.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:22:43.257 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:22:43.264 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:22:43.334 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:22:43.336 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:22:43.344 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:22:43.347 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:22:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.387+0000 7feddc400640 1 -- 172.21.15.71:0/73929836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd40b4240 msgr2=0x7fedd40730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.387+0000 7feddc400640 1 --2- 172.21.15.71:0/73929836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd40b4240 0x7fedd40730c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fedd4076530 tx=0x7fedc002a560 comp rx=0 tx=0).stop 2024-09-16T07:22:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.387+0000 7feddc400640 1 -- 172.21.15.71:0/73929836 shutdown_connections 2024-09-16T07:22:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.387+0000 7feddc400640 1 --2- 172.21.15.71:0/73929836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd40b4240 0x7fedd40730c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.387+0000 7feddc400640 1 --2- 172.21.15.71:0/73929836 >> v2:172.21.15.177:3300/0 conn(0x7fedd415bdc0 0x7fedd40b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.387+0000 7feddc400640 1 -- 172.21.15.71:0/73929836 >> 172.21.15.71:0/73929836 conn(0x7fedd40b2230 msgr2=0x7fedd40b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.387+0000 7feddc400640 1 -- 172.21.15.71:0/73929836 shutdown_connections 2024-09-16T07:22:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.387+0000 7feddc400640 1 -- 172.21.15.71:0/73929836 wait complete. 2024-09-16T07:22:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.389+0000 7feddc400640 1 Processor -- start 2024-09-16T07:22:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.389+0000 7feddc400640 1 -- start start 2024-09-16T07:22:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7feddc400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd4073600 0x7fedd4155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7feddc400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd40b4240 0x7fedd415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7feddc400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd415bdc0 0x7fedd4145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7feddc400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4151e60 con 0x7fedd415bdc0 2024-09-16T07:22:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7feddc400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fedd415c3b0 con 0x7fedd40b4240 2024-09-16T07:22:43.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7feddc400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fedd4151fe0 con 0x7fedd4073600 2024-09-16T07:22:43.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7fedd9a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd4073600 0x7fedd4155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:43.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7fedda400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd415bdc0 0x7fedd4145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:43.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7fedd9000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd40b4240 0x7fedd415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:43.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7fedd9a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd4073600 0x7fedd4155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:42792/0 (socket says 172.21.15.71:42792) 2024-09-16T07:22:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.390+0000 7fedd9a00640 1 -- 172.21.15.71:0/3710286485 learned_addr learned my addr 172.21.15.71:0/3710286485 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedda400640 1 -- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd4073600 msgr2=0x7fedd4155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedda400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd4073600 0x7fedd4155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedda400640 1 -- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd40b4240 msgr2=0x7fedd415deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedda400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd40b4240 0x7fedd415deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedda400640 1 -- 172.21.15.71:0/3710286485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd4157e90 con 0x7fedd415bdc0 2024-09-16T07:22:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedd9a00640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd4073600 0x7fedd4155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:22:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedda400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd415bdc0 0x7fedd4145e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c780 tx=0x7fedcc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedd1600640 1 -- 172.21.15.71:0/3710286485 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc016020 con 0x7fedd415bdc0 2024-09-16T07:22:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedd1600640 1 -- 172.21.15.71:0/3710286485 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedcc00ab40 con 0x7fedd415bdc0 2024-09-16T07:22:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7fedd1600640 1 -- 172.21.15.71:0/3710286485 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00ce30 con 0x7fedd415bdc0 2024-09-16T07:22:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd4158150 con 0x7fedd415bdc0 2024-09-16T07:22:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.391+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd4158710 con 0x7fedd415bdc0 2024-09-16T07:22:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.392+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd40730c0 con 0x7fedd415bdc0 2024-09-16T07:22:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.393+0000 7fedd1600640 1 -- 172.21.15.71:0/3710286485 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fedcc019020 con 0x7fedd415bdc0 2024-09-16T07:22:43.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.395+0000 7fedd1600640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb4078850 0x7fedb407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:43.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.395+0000 7fedd1600640 1 -- 172.21.15.71:0/3710286485 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fedcc09a420 con 0x7fedd415bdc0 2024-09-16T07:22:43.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.395+0000 7fedd9a00640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb4078850 0x7fedb407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:43.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.396+0000 7fedd9a00640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb4078850 0x7fedb407ad10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fedc80058c0 tx=0x7fedc800b410 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:43.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.400+0000 7fedd1600640 1 -- 172.21.15.71:0/3710286485 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fedcc020030 con 0x7fedd415bdc0 2024-09-16T07:22:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.624+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedd415a7f0 con 0x7fedd415bdc0 2024-09-16T07:22:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.625+0000 7fedd1600640 1 -- 172.21.15.71:0/3710286485 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7fedcc0626f0 con 0x7fedd415bdc0 2024-09-16T07:22:43.625 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:43.625 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":2,"btime":"2024-09-16T07:22:43:343709+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T07:22:43.625 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 2 2024-09-16T07:22:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb4078850 msgr2=0x7fedb407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb4078850 0x7fedb407ad10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fedc80058c0 tx=0x7fedc800b410 comp rx=0 tx=0).stop 2024-09-16T07:22:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd415bdc0 msgr2=0x7fedd4145e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd415bdc0 0x7fedd4145e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c780 tx=0x7fedcc00cc40 comp rx=0 tx=0).stop 2024-09-16T07:22:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 shutdown_connections 2024-09-16T07:22:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb4078850 0x7fedb407ad10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd415bdc0 0x7fedd4145e50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd40b4240 0x7fedd415deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 --2- 172.21.15.71:0/3710286485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd4073600 0x7fedd4155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 >> 172.21.15.71:0/3710286485 conn(0x7fedd40b2230 msgr2=0x7fedd406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 shutdown_connections 2024-09-16T07:22:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:43.627+0000 7feddc400640 1 -- 172.21.15.71:0/3710286485 wait complete. 2024-09-16T07:22:45.643 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:22:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.811+0000 7f46ac000640 1 -- 172.21.15.71:0/1728278662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46a415f5a0 msgr2=0x7f46a40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.811+0000 7f46ac000640 1 --2- 172.21.15.71:0/1728278662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46a415f5a0 0x7f46a40b2440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4698005fd0 tx=0x7f469802a820 comp rx=0 tx=0).stop 2024-09-16T07:22:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.811+0000 7f46ac000640 1 -- 172.21.15.71:0/1728278662 shutdown_connections 2024-09-16T07:22:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.811+0000 7f46ac000640 1 --2- 172.21.15.71:0/1728278662 >> v2:172.21.15.177:3300/0 conn(0x7f46a4073fb0 0x7f46a4074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.811+0000 7f46ac000640 1 --2- 172.21.15.71:0/1728278662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46a415f5a0 0x7f46a40b2440 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.811+0000 7f46ac000640 1 -- 172.21.15.71:0/1728278662 >> 172.21.15.71:0/1728278662 conn(0x7f46a40b1950 msgr2=0x7f46a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.811+0000 7f46ac000640 1 -- 172.21.15.71:0/1728278662 shutdown_connections 2024-09-16T07:22:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.811+0000 7f46ac000640 1 -- 172.21.15.71:0/1728278662 wait complete. 2024-09-16T07:22:45.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.813+0000 7f46ac000640 1 Processor -- start 2024-09-16T07:22:45.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.813+0000 7f46ac000640 1 -- start start 2024-09-16T07:22:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46ac000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46a4073fb0 0x7f46a41535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46ac000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40ccf00 0x7f46a41515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46ac000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46a415f5a0 0x7f46a41520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46ac000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f46a406f2e0 con 0x7f46a40ccf00 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46ac000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46a406f160 con 0x7f46a415f5a0 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46ac000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f46a406f460 con 0x7f46a4073fb0 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40ccf00 0x7f46a41515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40ccf00 0x7f46a41515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34402/0 (socket says 172.21.15.71:34402) 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a8c00640 1 -- 172.21.15.71:0/4244490613 learned_addr learned my addr 172.21.15.71:0/4244490613 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a9600640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46a4073fb0 0x7f46a41535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46aa000640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46a415f5a0 0x7f46a41520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a8c00640 1 -- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46a4073fb0 msgr2=0x7f46a41535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a8c00640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46a4073fb0 0x7f46a41535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a8c00640 1 -- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46a415f5a0 msgr2=0x7f46a41520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a8c00640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46a415f5a0 0x7f46a41520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46a8c00640 1 -- 172.21.15.71:0/4244490613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a41f35d0 con 0x7f46a40ccf00 2024-09-16T07:22:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.814+0000 7f46aa000640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46a415f5a0 0x7f46a41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.815+0000 7f46a8c00640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40ccf00 0x7f46a41515c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f469000c460 tx=0x7f469000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.815+0000 7f46a1600640 1 -- 172.21.15.71:0/4244490613 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690016020 con 0x7f46a40ccf00 2024-09-16T07:22:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.815+0000 7f46a1600640 1 -- 172.21.15.71:0/4244490613 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4690012920 con 0x7f46a40ccf00 2024-09-16T07:22:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.815+0000 7f46a1600640 1 -- 172.21.15.71:0/4244490613 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690012d50 con 0x7f46a40ccf00 2024-09-16T07:22:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.815+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a4150ae0 con 0x7f46a40ccf00 2024-09-16T07:22:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.815+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a4151b00 con 0x7f46a40ccf00 2024-09-16T07:22:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.816+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a406f9f0 con 0x7f46a40ccf00 2024-09-16T07:22:45.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.817+0000 7f46a1600640 1 -- 172.21.15.71:0/4244490613 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4690019020 con 0x7f46a40ccf00 2024-09-16T07:22:45.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.818+0000 7f46a1600640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4684078850 0x7f468407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:45.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.818+0000 7f46a1600640 1 -- 172.21.15.71:0/4244490613 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f469009a4d0 con 0x7f46a40ccf00 2024-09-16T07:22:45.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.818+0000 7f46a9600640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4684078850 0x7f468407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:45.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.819+0000 7f46a9600640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4684078850 0x7f468407ad10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4698034000 tx=0x7f4698004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:45.822+0000 7f46a1600640 1 -- 172.21.15.71:0/4244490613 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4690020020 con 0x7f46a40ccf00 2024-09-16T07:22:46.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.040+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46a409cb50 con 0x7f46a40ccf00 2024-09-16T07:22:46.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.041+0000 7f46a1600640 1 -- 172.21.15.71:0/4244490613 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7f4690062f40 con 0x7f46a40ccf00 2024-09-16T07:22:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:46.041 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":3,"btime":"2024-09-16T07:22:44:348467+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T07:22:46.041 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 3 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4684078850 msgr2=0x7f468407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4684078850 0x7f468407ad10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4698034000 tx=0x7f4698004ab0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40ccf00 msgr2=0x7f46a41515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40ccf00 0x7f46a41515c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f469000c460 tx=0x7f469000c920 comp rx=0 tx=0).stop 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 shutdown_connections 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4684078850 0x7f468407ad10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46a415f5a0 0x7f46a41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40ccf00 0x7f46a41515c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 --2- 172.21.15.71:0/4244490613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46a4073fb0 0x7f46a41535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 >> 172.21.15.71:0/4244490613 conn(0x7f46a40b1950 msgr2=0x7f46a409c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 shutdown_connections 2024-09-16T07:22:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.043+0000 7f46ac000640 1 -- 172.21.15.71:0/4244490613 wait complete. 2024-09-16T07:22:46.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:22:46.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.215+0000 7f247ce00640 1 -- 172.21.15.71:0/3412031239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2478073fb0 msgr2=0x7f2478074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.215+0000 7f247ce00640 1 --2- 172.21.15.71:0/3412031239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2478073fb0 0x7f2478074440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2470009830 tx=0x7f247002a480 comp rx=0 tx=0).stop 2024-09-16T07:22:46.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.215+0000 7f247ce00640 1 -- 172.21.15.71:0/3412031239 shutdown_connections 2024-09-16T07:22:46.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.215+0000 7f247ce00640 1 --2- 172.21.15.71:0/3412031239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2478073fb0 0x7f2478074440 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.215+0000 7f247ce00640 1 --2- 172.21.15.71:0/3412031239 >> v2:172.21.15.177:3300/0 conn(0x7f247815f5a0 0x7f24780b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.215+0000 7f247ce00640 1 -- 172.21.15.71:0/3412031239 >> 172.21.15.71:0/3412031239 conn(0x7f24780b1950 msgr2=0x7f24780b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:46.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.215+0000 7f247ce00640 1 -- 172.21.15.71:0/3412031239 shutdown_connections 2024-09-16T07:22:46.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.215+0000 7f247ce00640 1 -- 172.21.15.71:0/3412031239 wait complete. 2024-09-16T07:22:46.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.216+0000 7f247ce00640 1 Processor -- start 2024-09-16T07:22:46.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.216+0000 7f247ce00640 1 -- start start 2024-09-16T07:22:46.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.217+0000 7f247ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478073fb0 0x7f247814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:46.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.217+0000 7f247ce00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24780ccf00 0x7f2478157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:46.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.217+0000 7f247ce00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f247815f5a0 0x7f24780afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:46.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.217+0000 7f247ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f247806f2e0 con 0x7f2478073fb0 2024-09-16T07:22:46.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.217+0000 7f247ce00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f247806f160 con 0x7f24780ccf00 2024-09-16T07:22:46.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.217+0000 7f247ce00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f247806f460 con 0x7f247815f5a0 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478073fb0 0x7f247814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478073fb0 0x7f247814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34418/0 (socket says 172.21.15.71:34418) 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 -- 172.21.15.71:0/2374341484 learned_addr learned my addr 172.21.15.71:0/2374341484 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475400640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24780ccf00 0x7f2478157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 -- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f247815f5a0 msgr2=0x7f24780afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2476800640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f247815f5a0 0x7f24780afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f247815f5a0 0x7f24780afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 -- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24780ccf00 msgr2=0x7f2478157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24780ccf00 0x7f2478157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 -- 172.21.15.71:0/2374341484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2478158240 con 0x7f2478073fb0 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2476800640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f247815f5a0 0x7f24780afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:22:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475400640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24780ccf00 0x7f2478157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:22:46.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.218+0000 7f2475e00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478073fb0 0x7f247814f660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f246800ef90 tx=0x7f246800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:46.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.219+0000 7f246e000640 1 -- 172.21.15.71:0/2374341484 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468003970 con 0x7f2478073fb0 2024-09-16T07:22:46.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.219+0000 7f246e000640 1 -- 172.21.15.71:0/2374341484 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2468003c00 con 0x7f2478073fb0 2024-09-16T07:22:46.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.219+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24781f80d0 con 0x7f2478073fb0 2024-09-16T07:22:46.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.219+0000 7f246e000640 1 -- 172.21.15.71:0/2374341484 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468004020 con 0x7f2478073fb0 2024-09-16T07:22:46.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.219+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247814fba0 con 0x7f2478073fb0 2024-09-16T07:22:46.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.220+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247806f980 con 0x7f2478073fb0 2024-09-16T07:22:46.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.221+0000 7f246e000640 1 -- 172.21.15.71:0/2374341484 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2468009780 con 0x7f2478073fb0 2024-09-16T07:22:46.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.222+0000 7f246e000640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2458078850 0x7f245807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:46.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.222+0000 7f246e000640 1 -- 172.21.15.71:0/2374341484 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f246809aeb0 con 0x7f2478073fb0 2024-09-16T07:22:46.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.222+0000 7f2475400640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2458078850 0x7f245807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:46.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.223+0000 7f2475400640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2458078850 0x7f245807ad10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f247809c200 tx=0x7f2464009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.226+0000 7f246e000640 1 -- 172.21.15.71:0/2374341484 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2468063860 con 0x7f2478073fb0 2024-09-16T07:22:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.414+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f247809dea0 con 0x7f2478073fb0 2024-09-16T07:22:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.414+0000 7f246e000640 1 -- 172.21.15.71:0/2374341484 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f2468063160 con 0x7f2478073fb0 2024-09-16T07:22:46.414 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:46.414 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:22:46.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:22:46.414 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:22:46.414 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:22:46.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:22:46.414 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:22:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.416+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2458078850 msgr2=0x7f245807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.416+0000 7f247ce00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2458078850 0x7f245807ad10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f247809c200 tx=0x7f2464009040 comp rx=0 tx=0).stop 2024-09-16T07:22:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.416+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478073fb0 msgr2=0x7f247814f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.416+0000 7f247ce00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478073fb0 0x7f247814f660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f246800ef90 tx=0x7f246800c800 comp rx=0 tx=0).stop 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.417+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 shutdown_connections 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.417+0000 7f247ce00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2458078850 0x7f245807ad10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.417+0000 7f247ce00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f247815f5a0 0x7f24780afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.417+0000 7f247ce00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24780ccf00 0x7f2478157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.417+0000 7f247ce00640 1 --2- 172.21.15.71:0/2374341484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478073fb0 0x7f247814f660 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.417+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 >> 172.21.15.71:0/2374341484 conn(0x7f24780b1950 msgr2=0x7f24780b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.417+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 shutdown_connections 2024-09-16T07:22:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.417+0000 7f247ce00640 1 -- 172.21.15.71:0/2374341484 wait complete. 2024-09-16T07:22:46.431 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:22:46.431 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:22:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.592+0000 7f4c26400640 1 -- 172.21.15.71:0/1622184482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c2015bda0 msgr2=0x7f4c20073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.592+0000 7f4c26400640 1 --2- 172.21.15.71:0/1622184482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c2015bda0 0x7f4c20073010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009830 tx=0x7f4c1402a510 comp rx=0 tx=0).stop 2024-09-16T07:22:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.592+0000 7f4c26400640 1 -- 172.21.15.71:0/1622184482 shutdown_connections 2024-09-16T07:22:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.592+0000 7f4c26400640 1 --2- 172.21.15.71:0/1622184482 >> v2:172.21.15.177:3300/0 conn(0x7f4c2006e590 0x7f4c2006fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.592+0000 7f4c26400640 1 --2- 172.21.15.71:0/1622184482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c2015bda0 0x7f4c20073010 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.592+0000 7f4c26400640 1 -- 172.21.15.71:0/1622184482 >> 172.21.15.71:0/1622184482 conn(0x7f4c200cfa70 msgr2=0x7f4c200cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.592+0000 7f4c26400640 1 -- 172.21.15.71:0/1622184482 shutdown_connections 2024-09-16T07:22:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.592+0000 7f4c26400640 1 -- 172.21.15.71:0/1622184482 wait complete. 2024-09-16T07:22:46.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.594+0000 7f4c26400640 1 Processor -- start 2024-09-16T07:22:46.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.594+0000 7f4c26400640 1 -- start start 2024-09-16T07:22:46.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c26400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2006e590 0x7f4c2015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:46.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c26400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c20070380 0x7f4c201f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c26400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c2015bda0 0x7f4c200ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c26400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c200cc350 con 0x7f4c20070380 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c26400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4c200cc1d0 con 0x7f4c2006e590 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c26400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4c200740e0 con 0x7f4c2015bda0 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c1fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c20070380 0x7f4c201f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c1fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c20070380 0x7f4c201f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34426/0 (socket says 172.21.15.71:34426) 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c1fe00640 1 -- 172.21.15.71:0/2550332550 learned_addr learned my addr 172.21.15.71:0/2550332550 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c25000640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2006e590 0x7f4c2015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c25a00640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c2015bda0 0x7f4c200ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c1fe00640 1 -- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c2015bda0 msgr2=0x7f4c200ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.595+0000 7f4c1fe00640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c2015bda0 0x7f4c200ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c1fe00640 1 -- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2006e590 msgr2=0x7f4c2015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c1fe00640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2006e590 0x7f4c2015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c1fe00640 1 -- 172.21.15.71:0/2550332550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2015c8c0 con 0x7f4c20070380 2024-09-16T07:22:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c1fe00640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c20070380 0x7f4c201f81f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00e9f0 tx=0x7f4c0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c1cc00640 1 -- 172.21.15.71:0/2550332550 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00ccc0 con 0x7f4c20070380 2024-09-16T07:22:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c1cc00640 1 -- 172.21.15.71:0/2550332550 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c0c004020 con 0x7f4c20070380 2024-09-16T07:22:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c1cc00640 1 -- 172.21.15.71:0/2550332550 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c00ce60 con 0x7f4c20070380 2024-09-16T07:22:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2015cba0 con 0x7f4c20070380 2024-09-16T07:22:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.596+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c201f7cf0 con 0x7f4c20070380 2024-09-16T07:22:46.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.597+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c201f7b00 con 0x7f4c20070380 2024-09-16T07:22:46.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.598+0000 7f4c1cc00640 1 -- 172.21.15.71:0/2550332550 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4c0c0035f0 con 0x7f4c20070380 2024-09-16T07:22:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.600+0000 7f4c1cc00640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c00078850 0x7f4c0007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.600+0000 7f4c1cc00640 1 -- 172.21.15.71:0/2550332550 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f4c0c09af20 con 0x7f4c20070380 2024-09-16T07:22:46.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.600+0000 7f4c25000640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c00078850 0x7f4c0007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:46.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.601+0000 7f4c25000640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c00078850 0x7f4c0007ad10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c14034000 tx=0x7f4c14004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:46.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.604+0000 7f4c1cc00640 1 -- 172.21.15.71:0/2550332550 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4c0c0631f0 con 0x7f4c20070380 2024-09-16T07:22:46.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:46.788+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4c20074610 con 0x7f4c20070380 2024-09-16T07:22:47.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.098+0000 7f4c1cc00640 1 -- 172.21.15.71:0/2550332550 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f4c0c067c20 con 0x7f4c20070380 2024-09-16T07:22:47.098 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:22:47.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c00078850 msgr2=0x7f4c0007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c00078850 0x7f4c0007ad10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c14034000 tx=0x7f4c14004730 comp rx=0 tx=0).stop 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c20070380 msgr2=0x7f4c201f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c20070380 0x7f4c201f81f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00e9f0 tx=0x7f4c0c00eeb0 comp rx=0 tx=0).stop 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 shutdown_connections 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c00078850 0x7f4c0007ad10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c2015bda0 0x7f4c200ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c20070380 0x7f4c201f81f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 --2- 172.21.15.71:0/2550332550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2006e590 0x7f4c2015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 >> 172.21.15.71:0/2550332550 conn(0x7f4c200cfa70 msgr2=0x7f4c20159660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:47.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.101+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 shutdown_connections 2024-09-16T07:22:47.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.102+0000 7f4c26400640 1 -- 172.21.15.71:0/2550332550 wait complete. 2024-09-16T07:22:47.122 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:22:47.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.287+0000 7fb5b7200640 1 -- 172.21.15.71:0/3644024736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5b006cb70 msgr2=0x7fb5b00741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:47.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.287+0000 7fb5b7200640 1 --2- 172.21.15.71:0/3644024736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5b006cb70 0x7fb5b00741e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb598009860 tx=0x7fb59802a530 comp rx=0 tx=0).stop 2024-09-16T07:22:47.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.287+0000 7fb5b7200640 1 -- 172.21.15.71:0/3644024736 shutdown_connections 2024-09-16T07:22:47.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.287+0000 7fb5b7200640 1 --2- 172.21.15.71:0/3644024736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5b006cb70 0x7fb5b00741e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:47.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.287+0000 7fb5b7200640 1 -- 172.21.15.71:0/3644024736 >> 172.21.15.71:0/3644024736 conn(0x7fb5b00d0d40 msgr2=0x7fb5b00d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:47.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.287+0000 7fb5b7200640 1 -- 172.21.15.71:0/3644024736 shutdown_connections 2024-09-16T07:22:47.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.287+0000 7fb5b7200640 1 -- 172.21.15.71:0/3644024736 wait complete. 2024-09-16T07:22:47.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.289+0000 7fb5b7200640 1 Processor -- start 2024-09-16T07:22:47.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.289+0000 7fb5b7200640 1 -- start start 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b7200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5b006cb70 0x7fb5b01e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b7200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5b0074720 0x7fb5b01f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b015ba90 0x7fb5b0156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00cf1b0 con 0x7fb5b015ba90 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b7200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00cf030 con 0x7fb5b006cb70 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b7200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00cf330 con 0x7fb5b0074720 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b015ba90 0x7fb5b0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b015ba90 0x7fb5b0156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34438/0 (socket says 172.21.15.71:34438) 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5b5200640 1 -- 172.21.15.71:0/3526738839 learned_addr learned my addr 172.21.15.71:0/3526738839 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5afe00640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5b006cb70 0x7fb5b01e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.290+0000 7fb5a7400640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5b0074720 0x7fb5b01f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5b5200640 1 -- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5b0074720 msgr2=0x7fb5b01f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5b5200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5b0074720 0x7fb5b01f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5b5200640 1 -- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5b006cb70 msgr2=0x7fb5b01e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5b5200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5b006cb70 0x7fb5b01e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5b5200640 1 -- 172.21.15.71:0/3526738839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b01f5b00 con 0x7fb5b015ba90 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5b5200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b015ba90 0x7fb5b0156250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800c4d0 tx=0x7fb5a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5acc00640 1 -- 172.21.15.71:0/3526738839 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8016020 con 0x7fb5b015ba90 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5acc00640 1 -- 172.21.15.71:0/3526738839 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5a8012940 con 0x7fb5b015ba90 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5acc00640 1 -- 172.21.15.71:0/3526738839 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8012d90 con 0x7fb5b015ba90 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b0156dc0 con 0x7fb5b015ba90 2024-09-16T07:22:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.291+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b01e7160 con 0x7fb5b015ba90 2024-09-16T07:22:47.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.292+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b00cc540 con 0x7fb5b015ba90 2024-09-16T07:22:47.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.294+0000 7fb5acc00640 1 -- 172.21.15.71:0/3526738839 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb5a8019020 con 0x7fb5b015ba90 2024-09-16T07:22:47.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.294+0000 7fb5acc00640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb58c078850 0x7fb58c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:47.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.295+0000 7fb5acc00640 1 -- 172.21.15.71:0/3526738839 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7fb5a809a4a0 con 0x7fb5b015ba90 2024-09-16T07:22:47.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.295+0000 7fb5afe00640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb58c078850 0x7fb58c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:47.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.296+0000 7fb5afe00640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb58c078850 0x7fb58c07ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0000f80 tx=0x7fb5a0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:47.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.299+0000 7fb5acc00640 1 -- 172.21.15.71:0/3526738839 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5b00cc540 con 0x7fb5b015ba90 2024-09-16T07:22:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:47.486+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb5b0074ba0 con 0x7fb5b015ba90 2024-09-16T07:22:48.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.108+0000 7fb5acc00640 1 -- 172.21.15.71:0/3526738839 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7fb5a8020030 con 0x7fb5b015ba90 2024-09-16T07:22:48.107 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb58c078850 msgr2=0x7fb58c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb58c078850 0x7fb58c07ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0000f80 tx=0x7fb5a0009040 comp rx=0 tx=0).stop 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b015ba90 msgr2=0x7fb5b0156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b015ba90 0x7fb5b0156250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800c4d0 tx=0x7fb5a800c990 comp rx=0 tx=0).stop 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 shutdown_connections 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb58c078850 0x7fb58c07ad10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b015ba90 0x7fb5b0156250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5b0074720 0x7fb5b01f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 --2- 172.21.15.71:0/3526738839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5b006cb70 0x7fb5b01e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.111+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 >> 172.21.15.71:0/3526738839 conn(0x7fb5b00d0d40 msgr2=0x7fb5b0070740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:48.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.112+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 shutdown_connections 2024-09-16T07:22:48.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.114+0000 7fb5b7200640 1 -- 172.21.15.71:0/3526738839 wait complete. 2024-09-16T07:22:48.145 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:22:48.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.306+0000 7fc4b5e00640 1 -- 172.21.15.71:0/3364287447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 msgr2=0x7fc494003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:48.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.306+0000 7fc4b5e00640 1 --2- 172.21.15.71:0/3364287447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 0x7fc494003e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00099d0 tx=0x7fc494005830 comp rx=0 tx=0).stop 2024-09-16T07:22:48.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.306+0000 7fc4af400640 1 -- 172.21.15.71:0/3364287447 reap_dead start 2024-09-16T07:22:48.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.306+0000 7fc4b5e00640 1 -- 172.21.15.71:0/3364287447 shutdown_connections 2024-09-16T07:22:48.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.306+0000 7fc4b5e00640 1 -- 172.21.15.71:0/3364287447 >> 172.21.15.71:0/3364287447 conn(0x7fc4b00b1840 msgr2=0x7fc4b00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:48.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.306+0000 7fc4b5e00640 1 -- 172.21.15.71:0/3364287447 shutdown_connections 2024-09-16T07:22:48.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.306+0000 7fc4b5e00640 1 -- 172.21.15.71:0/3364287447 wait complete. 2024-09-16T07:22:48.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.308+0000 7fc4b5e00640 1 Processor -- start 2024-09-16T07:22:48.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.308+0000 7fc4b5e00640 1 -- start start 2024-09-16T07:22:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4b5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 0x7fc4b01f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4b5e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4b01f28f0 0x7fc4b013bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4b5e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4b00ae6c0 0x7fc4b00aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4b5e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b00737d0 con 0x7fc494003a50 2024-09-16T07:22:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4b5e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0073650 con 0x7fc4b01f28f0 2024-09-16T07:22:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4b5e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b00ce0f0 con 0x7fc4b00ae6c0 2024-09-16T07:22:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4aea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 0x7fc4b01f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4aea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 0x7fc4b01f6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34462/0 (socket says 172.21.15.71:34462) 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4aea00640 1 -- 172.21.15.71:0/1581604045 learned_addr learned my addr 172.21.15.71:0/1581604045 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.309+0000 7fc4af400640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4b00ae6c0 0x7fc4b00aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4ae000640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4b01f28f0 0x7fc4b013bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4aea00640 1 -- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4b00ae6c0 msgr2=0x7fc4b00aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4aea00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4b00ae6c0 0x7fc4b00aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4aea00640 1 -- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4b01f28f0 msgr2=0x7fc4b013bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4aea00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4b01f28f0 0x7fc4b013bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4aea00640 1 -- 172.21.15.71:0/1581604045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b01f3060 con 0x7fc494003a50 2024-09-16T07:22:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4aea00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 0x7fc4b01f6980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0000c00 tx=0x7fc4a0032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4a7400640 1 -- 172.21.15.71:0/1581604045 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0037050 con 0x7fc494003a50 2024-09-16T07:22:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4a7400640 1 -- 172.21.15.71:0/1581604045 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4a0038040 con 0x7fc494003a50 2024-09-16T07:22:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b01f3340 con 0x7fc494003a50 2024-09-16T07:22:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4a7400640 1 -- 172.21.15.71:0/1581604045 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a003c870 con 0x7fc494003a50 2024-09-16T07:22:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.310+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b01413a0 con 0x7fc494003a50 2024-09-16T07:22:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.311+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b00cdbb0 con 0x7fc494003a50 2024-09-16T07:22:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.312+0000 7fc4a7400640 1 -- 172.21.15.71:0/1581604045 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc4a0045020 con 0x7fc494003a50 2024-09-16T07:22:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.313+0000 7fc4a7400640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc490078850 0x7fc49007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.314+0000 7fc4a7400640 1 -- 172.21.15.71:0/1581604045 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7fc4a00baa40 con 0x7fc494003a50 2024-09-16T07:22:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.314+0000 7fc4ae000640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc490078850 0x7fc49007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.315+0000 7fc4ae000640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc490078850 0x7fc49007ad10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc49c001eb0 tx=0x7fc49c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:48.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.319+0000 7fc4a7400640 1 -- 172.21.15.71:0/1581604045 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc4a0082a30 con 0x7fc494003a50 2024-09-16T07:22:48.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:48.540+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc4b00cef40 con 0x7fc494003a50 2024-09-16T07:22:49.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.122+0000 7fc4a7400640 1 -- 172.21.15.71:0/1581604045 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fc4a00bae20 con 0x7fc494003a50 2024-09-16T07:22:49.122 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T07:22:49.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc490078850 msgr2=0x7fc49007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc490078850 0x7fc49007ad10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc49c001eb0 tx=0x7fc49c009040 comp rx=0 tx=0).stop 2024-09-16T07:22:49.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 msgr2=0x7fc4b01f6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 0x7fc4b01f6980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0000c00 tx=0x7fc4a0032d70 comp rx=0 tx=0).stop 2024-09-16T07:22:49.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 shutdown_connections 2024-09-16T07:22:49.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc490078850 0x7fc49007ad10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4b00ae6c0 0x7fc4b00aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4b01f28f0 0x7fc4b013bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 --2- 172.21.15.71:0/1581604045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc494003a50 0x7fc4b01f6980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.127+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 >> 172.21.15.71:0/1581604045 conn(0x7fc4b00b1840 msgr2=0x7fc4b006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:49.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.128+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 shutdown_connections 2024-09-16T07:22:49.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.131+0000 7fc4b5e00640 1 -- 172.21.15.71:0/1581604045 wait complete. 2024-09-16T07:22:49.156 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.313+0000 7fc7a8c00640 1 -- 172.21.15.71:0/3755146593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a40b2440 msgr2=0x7fc7a406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.313+0000 7fc7a8c00640 1 --2- 172.21.15.71:0/3755146593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a40b2440 0x7fc7a406f160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc79c005fd0 tx=0x7fc79c02a820 comp rx=0 tx=0).stop 2024-09-16T07:22:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.313+0000 7fc7a8c00640 1 -- 172.21.15.71:0/3755146593 shutdown_connections 2024-09-16T07:22:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.313+0000 7fc7a8c00640 1 --2- 172.21.15.71:0/3755146593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a40b2440 0x7fc7a406f160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.313+0000 7fc7a8c00640 1 -- 172.21.15.71:0/3755146593 >> 172.21.15.71:0/3755146593 conn(0x7fc7a40b1950 msgr2=0x7fc7a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.313+0000 7fc7a8c00640 1 -- 172.21.15.71:0/3755146593 shutdown_connections 2024-09-16T07:22:49.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.313+0000 7fc7a8c00640 1 -- 172.21.15.71:0/3755146593 wait complete. 2024-09-16T07:22:49.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.315+0000 7fc7a8c00640 1 Processor -- start 2024-09-16T07:22:49.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.315+0000 7fc7a8c00640 1 -- start start 2024-09-16T07:22:49.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a8c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a40b2440 0x7fc7a409d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:49.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a8c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a40ccf00 0x7fc7a41f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:49.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a415f5a0 0x7fc7a41efb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:49.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a8c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a4074840 con 0x7fc7a415f5a0 2024-09-16T07:22:49.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a8c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a40746c0 con 0x7fc7a40ccf00 2024-09-16T07:22:49.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a8c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a406f6a0 con 0x7fc7a40b2440 2024-09-16T07:22:49.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a415f5a0 0x7fc7a41efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:49.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a415f5a0 0x7fc7a41efb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34484/0 (socket says 172.21.15.71:34484) 2024-09-16T07:22:49.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a2800640 1 -- 172.21.15.71:0/2729290694 learned_addr learned my addr 172.21.15.71:0/2729290694 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:49.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc7a1e00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a40b2440 0x7fc7a409d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:49.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.316+0000 7fc79be00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a40ccf00 0x7fc7a41f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:49.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc7a2800640 1 -- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a40b2440 msgr2=0x7fc7a409d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc7a2800640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a40b2440 0x7fc7a409d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc7a2800640 1 -- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a40ccf00 msgr2=0x7fc7a41f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc7a2800640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a40ccf00 0x7fc7a41f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc7a2800640 1 -- 172.21.15.71:0/2729290694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a4158810 con 0x7fc7a415f5a0 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc79be00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a40ccf00 0x7fc7a41f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc7a2800640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a415f5a0 0x7fc7a41efb50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009b30 tx=0x7fc79c032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc79a000640 1 -- 172.21.15.71:0/2729290694 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c037040 con 0x7fc7a415f5a0 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc79a000640 1 -- 172.21.15.71:0/2729290694 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc79c042070 con 0x7fc7a415f5a0 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc79a000640 1 -- 172.21.15.71:0/2729290694 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0042a0 con 0x7fc7a415f5a0 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a4158af0 con 0x7fc7a415f5a0 2024-09-16T07:22:49.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.317+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a41590b0 con 0x7fc7a415f5a0 2024-09-16T07:22:49.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.318+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a406f160 con 0x7fc7a415f5a0 2024-09-16T07:22:49.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.319+0000 7fc79a000640 1 -- 172.21.15.71:0/2729290694 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc79c004440 con 0x7fc7a415f5a0 2024-09-16T07:22:49.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.320+0000 7fc79a000640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078850 0x7fc78007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:49.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.320+0000 7fc79a000640 1 -- 172.21.15.71:0/2729290694 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc79c0bb650 con 0x7fc7a415f5a0 2024-09-16T07:22:49.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.320+0000 7fc7a1e00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078850 0x7fc78007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:49.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.321+0000 7fc7a1e00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078850 0x7fc78007ad10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794009060 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:49.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.325+0000 7fc79a000640 1 -- 172.21.15.71:0/2729290694 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc79c083cb0 con 0x7fc7a415f5a0 2024-09-16T07:22:49.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.509+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7a4155000 con 0x7fc7a415f5a0 2024-09-16T07:22:49.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.510+0000 7fc79a000640 1 -- 172.21.15.71:0/2729290694 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7fc79c0835b0 con 0x7fc7a415f5a0 2024-09-16T07:22:49.510 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:49.510 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":22,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:48.541551+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:49.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.512+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078850 msgr2=0x7fc78007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.512+0000 7fc7a8c00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078850 0x7fc78007ad10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794009060 comp rx=0 tx=0).stop 2024-09-16T07:22:49.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.512+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a415f5a0 msgr2=0x7fc7a41efb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.512+0000 7fc7a8c00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a415f5a0 0x7fc7a41efb50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009b30 tx=0x7fc79c032c90 comp rx=0 tx=0).stop 2024-09-16T07:22:49.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.512+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 shutdown_connections 2024-09-16T07:22:49.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.512+0000 7fc7a8c00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078850 0x7fc78007ad10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.513+0000 7fc7a8c00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7a415f5a0 0x7fc7a41efb50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.513+0000 7fc7a8c00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a40ccf00 0x7fc7a41f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.513+0000 7fc7a8c00640 1 --2- 172.21.15.71:0/2729290694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a40b2440 0x7fc7a409d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.513+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 >> 172.21.15.71:0/2729290694 conn(0x7fc7a40b1950 msgr2=0x7fc7a41543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.513+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 shutdown_connections 2024-09-16T07:22:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.513+0000 7fc7a8c00640 1 -- 172.21.15.71:0/2729290694 wait complete. 2024-09-16T07:22:49.527 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.684+0000 7f9880600640 1 -- 172.21.15.71:0/1894513726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 msgr2=0x7f98780b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.684+0000 7f9880600640 1 --2- 172.21.15.71:0/1894513726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 0x7f98780b2440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f986c005fd0 tx=0x7f986c02a540 comp rx=0 tx=0).stop 2024-09-16T07:22:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.684+0000 7f9880600640 1 -- 172.21.15.71:0/1894513726 shutdown_connections 2024-09-16T07:22:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.684+0000 7f9880600640 1 --2- 172.21.15.71:0/1894513726 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98780ccf00 0x7f98780acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.684+0000 7f9880600640 1 --2- 172.21.15.71:0/1894513726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 0x7f98780b2440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.684+0000 7f9880600640 1 -- 172.21.15.71:0/1894513726 >> 172.21.15.71:0/1894513726 conn(0x7f98780b1950 msgr2=0x7f98780b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.684+0000 7f9880600640 1 -- 172.21.15.71:0/1894513726 shutdown_connections 2024-09-16T07:22:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.684+0000 7f9880600640 1 -- 172.21.15.71:0/1894513726 wait complete. 2024-09-16T07:22:49.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.686+0000 7f9880600640 1 Processor -- start 2024-09-16T07:22:49.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.686+0000 7f9880600640 1 -- start start 2024-09-16T07:22:49.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f9880600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98780ad270 0x7f98781464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:49.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f9880600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98780ccf00 0x7f98780cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f9880600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 0x7f98780b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f9880600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f987806f2e0 con 0x7f987815f5a0 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f9880600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f987806f160 con 0x7f98780ad270 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f9880600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f987806f460 con 0x7f98780ccf00 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 0x7f98780b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 0x7f98780b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34510/0 (socket says 172.21.15.71:34510) 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987e600640 1 -- 172.21.15.71:0/3921159665 learned_addr learned my addr 172.21.15.71:0/3921159665 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987d200640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98780ccf00 0x7f98780cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987dc00640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98780ad270 0x7f98781464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987e600640 1 -- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98780ccf00 msgr2=0x7f98780cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987e600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98780ccf00 0x7f98780cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987e600640 1 -- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98780ad270 msgr2=0x7f98781464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987e600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98780ad270 0x7f98781464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987e600640 1 -- 172.21.15.71:0/3921159665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98780b3800 con 0x7f987815f5a0 2024-09-16T07:22:49.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.687+0000 7f987dc00640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98780ad270 0x7f98781464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:49.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.688+0000 7f987e600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 0x7f98780b3ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f987000c7d0 tx=0x7f987000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:49.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.688+0000 7f9875600640 1 -- 172.21.15.71:0/3921159665 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870003b70 con 0x7f987815f5a0 2024-09-16T07:22:49.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.688+0000 7f9875600640 1 -- 172.21.15.71:0/3921159665 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9870003d10 con 0x7f987815f5a0 2024-09-16T07:22:49.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.688+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98781469e0 con 0x7f987815f5a0 2024-09-16T07:22:49.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.688+0000 7f9875600640 1 -- 172.21.15.71:0/3921159665 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987000aab0 con 0x7f987815f5a0 2024-09-16T07:22:49.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.688+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9878150c70 con 0x7f987815f5a0 2024-09-16T07:22:49.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.689+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98780b39d0 con 0x7f987815f5a0 2024-09-16T07:22:49.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.690+0000 7f9875600640 1 -- 172.21.15.71:0/3921159665 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f987000ac50 con 0x7f987815f5a0 2024-09-16T07:22:49.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.691+0000 7f9875600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9858078850 0x7f985807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:49.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.691+0000 7f9875600640 1 -- 172.21.15.71:0/3921159665 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f987009a810 con 0x7f987815f5a0 2024-09-16T07:22:49.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.692+0000 7f987dc00640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9858078850 0x7f985807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:49.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.693+0000 7f987dc00640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9858078850 0x7f985807ad10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f986c033000 tx=0x7f986c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:49.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.695+0000 7f9875600640 1 -- 172.21.15.71:0/3921159665 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9870062e70 con 0x7f987815f5a0 2024-09-16T07:22:49.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.879+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f987806f9f0 con 0x7f987815f5a0 2024-09-16T07:22:49.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.880+0000 7f9875600640 1 -- 172.21.15.71:0/3921159665 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f9870062770 con 0x7f987815f5a0 2024-09-16T07:22:49.880 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:49.880 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":22,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:48.541551+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:49.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9858078850 msgr2=0x7f985807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9858078850 0x7f985807ad10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f986c033000 tx=0x7f986c004730 comp rx=0 tx=0).stop 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 msgr2=0x7f98780b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 0x7f98780b3ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f987000c7d0 tx=0x7f987000cc90 comp rx=0 tx=0).stop 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 shutdown_connections 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9858078850 0x7f985807ad10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987815f5a0 0x7f98780b3ce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98780ccf00 0x7f98780cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 --2- 172.21.15.71:0/3921159665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98780ad270 0x7f98781464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 >> 172.21.15.71:0/3921159665 conn(0x7f98780b1950 msgr2=0x7f98780b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 shutdown_connections 2024-09-16T07:22:49.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:49.883+0000 7f9880600640 1 -- 172.21.15.71:0/3921159665 wait complete. 2024-09-16T07:22:49.898 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:22:50.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.058+0000 7fdf35200640 1 -- 172.21.15.71:0/1512906522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf30145750 msgr2=0x7fdf30073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.058+0000 7fdf35200640 1 --2- 172.21.15.71:0/1512906522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf30145750 0x7fdf30073440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009830 tx=0x7fdf2002a530 comp rx=0 tx=0).stop 2024-09-16T07:22:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.058+0000 7fdf35200640 1 -- 172.21.15.71:0/1512906522 shutdown_connections 2024-09-16T07:22:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.058+0000 7fdf35200640 1 --2- 172.21.15.71:0/1512906522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf30145750 0x7fdf30073440 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.058+0000 7fdf35200640 1 -- 172.21.15.71:0/1512906522 >> 172.21.15.71:0/1512906522 conn(0x7fdf300b1840 msgr2=0x7fdf300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.059+0000 7fdf35200640 1 -- 172.21.15.71:0/1512906522 shutdown_connections 2024-09-16T07:22:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.059+0000 7fdf35200640 1 -- 172.21.15.71:0/1512906522 wait complete. 2024-09-16T07:22:50.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.060+0000 7fdf35200640 1 Processor -- start 2024-09-16T07:22:50.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.060+0000 7fdf35200640 1 -- start start 2024-09-16T07:22:50.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf35200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3006efa0 0x7fdf301364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf35200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf300cccd0 0x7fdf300ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf35200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf30145750 0x7fdf300ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf35200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf300ce6a0 con 0x7fdf300cccd0 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf35200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdf300ce520 con 0x7fdf30145750 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf35200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdf300ce820 con 0x7fdf3006efa0 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf300cccd0 0x7fdf300ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf27e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf300cccd0 0x7fdf300ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34524/0 (socket says 172.21.15.71:34524) 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf27e00640 1 -- 172.21.15.71:0/2542422945 learned_addr learned my addr 172.21.15.71:0/2542422945 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf2de00640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3006efa0 0x7fdf301364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.061+0000 7fdf2e800640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf30145750 0x7fdf300ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf27e00640 1 -- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3006efa0 msgr2=0x7fdf301364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf27e00640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3006efa0 0x7fdf301364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf27e00640 1 -- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf30145750 msgr2=0x7fdf300ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf27e00640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf30145750 0x7fdf300ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf27e00640 1 -- 172.21.15.71:0/2542422945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf301f5600 con 0x7fdf300cccd0 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf27e00640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf300cccd0 0x7fdf300ad8c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c530 tx=0x7fdf1800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf26000640 1 -- 172.21.15.71:0/2542422945 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180052d0 con 0x7fdf300cccd0 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf26000640 1 -- 172.21.15.71:0/2542422945 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf18005470 con 0x7fdf300cccd0 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf26000640 1 -- 172.21.15.71:0/2542422945 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18012870 con 0x7fdf300cccd0 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf301f8010 con 0x7fdf300cccd0 2024-09-16T07:22:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.062+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf301f85d0 con 0x7fdf300cccd0 2024-09-16T07:22:50.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.064+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3006e690 con 0x7fdf300cccd0 2024-09-16T07:22:50.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.065+0000 7fdf26000640 1 -- 172.21.15.71:0/2542422945 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fdf180035f0 con 0x7fdf300cccd0 2024-09-16T07:22:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.066+0000 7fdf26000640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf0c078850 0x7fdf0c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.066+0000 7fdf26000640 1 -- 172.21.15.71:0/2542422945 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdf1809a3b0 con 0x7fdf300cccd0 2024-09-16T07:22:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.067+0000 7fdf2de00640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf0c078850 0x7fdf0c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.067+0000 7fdf2de00640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf0c078850 0x7fdf0c07ad10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf20034000 tx=0x7fdf20004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:50.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.070+0000 7fdf26000640 1 -- 172.21.15.71:0/2542422945 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdf3006e690 con 0x7fdf300cccd0 2024-09-16T07:22:50.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:50.296+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdf300cecd0 con 0x7fdf300cccd0 2024-09-16T07:22:51.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.133+0000 7fdf26000640 1 -- 172.21.15.71:0/2542422945 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fdf180623e0 con 0x7fdf300cccd0 2024-09-16T07:22:51.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf0c078850 msgr2=0x7fdf0c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf0c078850 0x7fdf0c07ad10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf20034000 tx=0x7fdf20004730 comp rx=0 tx=0).stop 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf300cccd0 msgr2=0x7fdf300ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf300cccd0 0x7fdf300ad8c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c530 tx=0x7fdf1800c9f0 comp rx=0 tx=0).stop 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 shutdown_connections 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf0c078850 0x7fdf0c07ad10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf30145750 0x7fdf300ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf300cccd0 0x7fdf300ad8c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 --2- 172.21.15.71:0/2542422945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3006efa0 0x7fdf301364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 >> 172.21.15.71:0/2542422945 conn(0x7fdf300b1840 msgr2=0x7fdf300b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 shutdown_connections 2024-09-16T07:22:51.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.136+0000 7fdf35200640 1 -- 172.21.15.71:0/2542422945 wait complete. 2024-09-16T07:22:51.154 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:22:51.155 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:22:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.319+0000 7fe910200640 1 -- 172.21.15.71:0/2341073470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 msgr2=0x7fe8f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.319+0000 7fe910200640 1 --2- 172.21.15.71:0/2341073470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 0x7fe8f0004c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009b50 tx=0x7fe8f0005830 comp rx=0 tx=0).stop 2024-09-16T07:22:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.319+0000 7fe90e200640 1 -- 172.21.15.71:0/2341073470 reap_dead start 2024-09-16T07:22:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.319+0000 7fe910200640 1 -- 172.21.15.71:0/2341073470 shutdown_connections 2024-09-16T07:22:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.319+0000 7fe910200640 1 -- 172.21.15.71:0/2341073470 >> 172.21.15.71:0/2341073470 conn(0x7fe9080b1840 msgr2=0x7fe9080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.319+0000 7fe910200640 1 -- 172.21.15.71:0/2341073470 shutdown_connections 2024-09-16T07:22:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.319+0000 7fe910200640 1 -- 172.21.15.71:0/2341073470 wait complete. 2024-09-16T07:22:51.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.321+0000 7fe910200640 1 Processor -- start 2024-09-16T07:22:51.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.321+0000 7fe910200640 1 -- start start 2024-09-16T07:22:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe910200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 0x7fe9080b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe910200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9081f8fa0 0x7fe90815e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe910200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe90812e2a0 0x7fe9081f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe910200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9080ce2c0 con 0x7fe8f0004800 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe910200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9080736d0 con 0x7fe90812e2a0 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe910200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9080ce440 con 0x7fe9081f8fa0 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 0x7fe9080b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 0x7fe9080b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34542/0 (socket says 172.21.15.71:34542) 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90d800640 1 -- 172.21.15.71:0/3348298881 learned_addr learned my addr 172.21.15.71:0/3348298881 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90ce00640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9081f8fa0 0x7fe90815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90e200640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe90812e2a0 0x7fe9081f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90d800640 1 -- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9081f8fa0 msgr2=0x7fe90815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90d800640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9081f8fa0 0x7fe90815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90d800640 1 -- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe90812e2a0 msgr2=0x7fe9081f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90d800640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe90812e2a0 0x7fe9081f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90d800640 1 -- 172.21.15.71:0/3348298881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90812d200 con 0x7fe8f0004800 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.322+0000 7fe90ce00640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9081f8fa0 0x7fe90815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.323+0000 7fe90d800640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 0x7fe9080b3d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc001860 tx=0x7fe8fc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.323+0000 7fe905600640 1 -- 172.21.15.71:0/3348298881 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc015070 con 0x7fe8f0004800 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.323+0000 7fe905600640 1 -- 172.21.15.71:0/3348298881 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8fc011840 con 0x7fe8f0004800 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.323+0000 7fe905600640 1 -- 172.21.15.71:0/3348298881 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc011c90 con 0x7fe8f0004800 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.323+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90812d4e0 con 0x7fe8f0004800 2024-09-16T07:22:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.323+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90812daa0 con 0x7fe8f0004800 2024-09-16T07:22:51.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.324+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9080ce9d0 con 0x7fe8f0004800 2024-09-16T07:22:51.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.325+0000 7fe905600640 1 -- 172.21.15.71:0/3348298881 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe8fc018070 con 0x7fe8f0004800 2024-09-16T07:22:51.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.326+0000 7fe905600640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8e8078880 0x7fe8e807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:51.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.326+0000 7fe905600640 1 -- 172.21.15.71:0/3348298881 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe8fc09b520 con 0x7fe8f0004800 2024-09-16T07:22:51.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.326+0000 7fe90ce00640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8e8078880 0x7fe8e807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:51.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.327+0000 7fe90ce00640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8e8078880 0x7fe8e807ad40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4035000 tx=0x7fe8f402d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:51.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.330+0000 7fe905600640 1 -- 172.21.15.71:0/3348298881 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe8fc063ba0 con 0x7fe8f0004800 2024-09-16T07:22:51.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:51.550+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe9080afa20 con 0x7fe8f0004800 2024-09-16T07:22:52.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.135+0000 7fe905600640 1 -- 172.21.15.71:0/3348298881 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fe8fc0634a0 con 0x7fe8f0004800 2024-09-16T07:22:52.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8e8078880 msgr2=0x7fe8e807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8e8078880 0x7fe8e807ad40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4035000 tx=0x7fe8f402d040 comp rx=0 tx=0).stop 2024-09-16T07:22:52.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 msgr2=0x7fe9080b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 0x7fe9080b3d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc001860 tx=0x7fe8fc0039c0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 shutdown_connections 2024-09-16T07:22:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8e8078880 0x7fe8e807ad40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe90812e2a0 0x7fe9081f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9081f8fa0 0x7fe90815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 --2- 172.21.15.71:0/3348298881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8f0004800 0x7fe9080b3d60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 >> 172.21.15.71:0/3348298881 conn(0x7fe9080b1840 msgr2=0x7fe9080aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 shutdown_connections 2024-09-16T07:22:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.137+0000 7fe910200640 1 -- 172.21.15.71:0/3348298881 wait complete. 2024-09-16T07:22:52.153 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:22:52.154 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:22:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.310+0000 7f01f7000640 1 -- 172.21.15.71:0/4294386735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 msgr2=0x7f01f00acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.310+0000 7f01f7000640 1 --2- 172.21.15.71:0/4294386735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 0x7f01f00acd30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01d8005fd0 tx=0x7f01d802a540 comp rx=0 tx=0).stop 2024-09-16T07:22:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.310+0000 7f01f7000640 1 -- 172.21.15.71:0/4294386735 shutdown_connections 2024-09-16T07:22:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.310+0000 7f01f7000640 1 --2- 172.21.15.71:0/4294386735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 0x7f01f00acd30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.310+0000 7f01f7000640 1 --2- 172.21.15.71:0/4294386735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f015f5a0 0x7f01f00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.310+0000 7f01f7000640 1 -- 172.21.15.71:0/4294386735 >> 172.21.15.71:0/4294386735 conn(0x7f01f00b1950 msgr2=0x7f01f00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.310+0000 7f01f7000640 1 -- 172.21.15.71:0/4294386735 shutdown_connections 2024-09-16T07:22:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.310+0000 7f01f7000640 1 -- 172.21.15.71:0/4294386735 wait complete. 2024-09-16T07:22:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.312+0000 7f01f7000640 1 Processor -- start 2024-09-16T07:22:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.312+0000 7f01f7000640 1 -- start start 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01f7000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f00ad270 0x7f01f00cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01f7000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 0x7f01f0154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01f7000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f015f5a0 0x7f01f015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01f7000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01f006f2e0 con 0x7f01f00ccf00 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01f7000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01f006f160 con 0x7f01f00ad270 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01f7000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01f006f460 con 0x7f01f015f5a0 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 0x7f01f0154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01efe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f00ad270 0x7f01f00cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01f5000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f015f5a0 0x7f01f015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 0x7f01f0154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34550/0 (socket says 172.21.15.71:34550) 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01e7e00640 1 -- 172.21.15.71:0/2845363476 learned_addr learned my addr 172.21.15.71:0/2845363476 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:52.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.313+0000 7f01efe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f00ad270 0x7f01f00cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:39872/0 (socket says 172.21.15.71:39872) 2024-09-16T07:22:52.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01e7e00640 1 -- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f015f5a0 msgr2=0x7f01f015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01e7e00640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f015f5a0 0x7f01f015b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01e7e00640 1 -- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f00ad270 msgr2=0x7f01f00cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01e7e00640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f00ad270 0x7f01f00cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01e7e00640 1 -- 172.21.15.71:0/2845363476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f01f4830 con 0x7f01f00ccf00 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01e7e00640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 0x7f01f0154e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009850 tx=0x7f01d80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01ecc00640 1 -- 172.21.15.71:0/2845363476 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d803f070 con 0x7f01f00ccf00 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01ecc00640 1 -- 172.21.15.71:0/2845363476 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01d80047a0 con 0x7f01f00ccf00 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f00cb5b0 con 0x7f01f00ccf00 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01ecc00640 1 -- 172.21.15.71:0/2845363476 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8035030 con 0x7f01f00ccf00 2024-09-16T07:22:52.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.314+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01f4dd0 con 0x7f01f00ccf00 2024-09-16T07:22:52.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.315+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f006f980 con 0x7f01f00ccf00 2024-09-16T07:22:52.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.317+0000 7f01ecc00640 1 -- 172.21.15.71:0/2845363476 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f01d8042070 con 0x7f01f00ccf00 2024-09-16T07:22:52.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.318+0000 7f01ecc00640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01cc078850 0x7f01cc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:52.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.318+0000 7f01ecc00640 1 -- 172.21.15.71:0/2845363476 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f01d80bb770 con 0x7f01f00ccf00 2024-09-16T07:22:52.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.319+0000 7f01efe00640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01cc078850 0x7f01cc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:52.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.319+0000 7f01efe00640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01cc078850 0x7f01cc07ad10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f01e0004630 tx=0x7f01e000b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.322+0000 7f01ecc00640 1 -- 172.21.15.71:0/2845363476 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f01d8083780 con 0x7f01f00ccf00 2024-09-16T07:22:52.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.540+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01f0069ac0 con 0x7f01f00ccf00 2024-09-16T07:22:52.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.541+0000 7f01ecc00640 1 -- 172.21.15.71:0/2845363476 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f01d80881b0 con 0x7f01f00ccf00 2024-09-16T07:22:52.540 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:52.540 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:22:48.541520+0000","modified":"2024-09-16T07:22:51.551572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4404":{"gid":4404,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4404,"qdb_cluster":[4404]},"id":1}]} 2024-09-16T07:22:52.541 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-09-16T07:22:52.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01cc078850 msgr2=0x7f01cc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01cc078850 0x7f01cc07ad10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f01e0004630 tx=0x7f01e000b2e0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 msgr2=0x7f01f0154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 0x7f01f0154e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009850 tx=0x7f01d80038a0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 shutdown_connections 2024-09-16T07:22:52.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01cc078850 0x7f01cc07ad10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f015f5a0 0x7f01f015b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f00ccf00 0x7f01f0154e60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 --2- 172.21.15.71:0/2845363476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f00ad270 0x7f01f00cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 >> 172.21.15.71:0/2845363476 conn(0x7f01f00b1950 msgr2=0x7f01f015e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:52.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 shutdown_connections 2024-09-16T07:22:52.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.543+0000 7f01f7000640 1 -- 172.21.15.71:0/2845363476 wait complete. 2024-09-16T07:22:52.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:52.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.714+0000 7f4174200640 1 -- 172.21.15.71:0/570711038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f416c073fb0 msgr2=0x7f416c074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.714+0000 7f4174200640 1 --2- 172.21.15.71:0/570711038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f416c073fb0 0x7f416c074440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4164005fd0 tx=0x7f416402aa10 comp rx=0 tx=0).stop 2024-09-16T07:22:52.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.714+0000 7f4174200640 1 -- 172.21.15.71:0/570711038 shutdown_connections 2024-09-16T07:22:52.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.714+0000 7f4174200640 1 --2- 172.21.15.71:0/570711038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f416c073fb0 0x7f416c074440 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.714+0000 7f4174200640 1 --2- 172.21.15.71:0/570711038 >> v2:172.21.15.177:3300/0 conn(0x7f416c15f5a0 0x7f416c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.714+0000 7f4174200640 1 -- 172.21.15.71:0/570711038 >> 172.21.15.71:0/570711038 conn(0x7f416c0b1950 msgr2=0x7f416c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:52.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.714+0000 7f4174200640 1 -- 172.21.15.71:0/570711038 shutdown_connections 2024-09-16T07:22:52.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.714+0000 7f4174200640 1 -- 172.21.15.71:0/570711038 wait complete. 2024-09-16T07:22:52.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.716+0000 7f4174200640 1 Processor -- start 2024-09-16T07:22:52.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.716+0000 7f4174200640 1 -- start start 2024-09-16T07:22:52.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4174200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f416c073fb0 0x7f416c1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:52.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4174200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416c0ccf00 0x7f416c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:52.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4174200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f416c15f5a0 0x7f416c0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:52.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4174200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f416c06f2e0 con 0x7f416c073fb0 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4174200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f416c06f160 con 0x7f416c0ccf00 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4174200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f416c06f460 con 0x7f416c15f5a0 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4171800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f416c073fb0 0x7f416c1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4171800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f416c073fb0 0x7f416c1f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34552/0 (socket says 172.21.15.71:34552) 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4171800640 1 -- 172.21.15.71:0/1888256829 learned_addr learned my addr 172.21.15.71:0/1888256829 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4172200640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f416c15f5a0 0x7f416c0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.717+0000 7f4170e00640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416c0ccf00 0x7f416c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4171800640 1 -- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f416c15f5a0 msgr2=0x7f416c0ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4171800640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f416c15f5a0 0x7f416c0ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4171800640 1 -- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416c0ccf00 msgr2=0x7f416c0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4171800640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416c0ccf00 0x7f416c0cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4171800640 1 -- 172.21.15.71:0/1888256829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1f5b90 con 0x7f416c073fb0 2024-09-16T07:22:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4171800640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f416c073fb0 0x7f416c1f5650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f416000e7c0 tx=0x7f416000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4169600640 1 -- 172.21.15.71:0/1888256829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416000ccc0 con 0x7f416c073fb0 2024-09-16T07:22:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4169600640 1 -- 172.21.15.71:0/1888256829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4160010430 con 0x7f416c073fb0 2024-09-16T07:22:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4169600640 1 -- 172.21.15.71:0/1888256829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160010860 con 0x7f416c073fb0 2024-09-16T07:22:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.718+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1564c0 con 0x7f416c073fb0 2024-09-16T07:22:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.719+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c156a80 con 0x7f416c073fb0 2024-09-16T07:22:52.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.720+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c06f980 con 0x7f416c073fb0 2024-09-16T07:22:52.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.720+0000 7f4169600640 1 -- 172.21.15.71:0/1888256829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4160010a00 con 0x7f416c073fb0 2024-09-16T07:22:52.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.721+0000 7f4169600640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f414c078850 0x7f414c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:52.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.722+0000 7f4169600640 1 -- 172.21.15.71:0/1888256829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f416009b1a0 con 0x7f416c073fb0 2024-09-16T07:22:52.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.722+0000 7f4170e00640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f414c078850 0x7f414c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:52.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.722+0000 7f4170e00640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f414c078850 0x7f414c07ad10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f416c06add0 tx=0x7f4158009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:52.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.725+0000 7f4169600640 1 -- 172.21.15.71:0/1888256829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f416c1f5b90 con 0x7f416c073fb0 2024-09-16T07:22:52.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.908+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f416c0ccc60 con 0x7f416c073fb0 2024-09-16T07:22:52.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.910+0000 7f4169600640 1 -- 172.21.15.71:0/1888256829 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7f4160063250 con 0x7f416c073fb0 2024-09-16T07:22:52.909 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:52.909 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":23,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:50.122046+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:52.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f414c078850 msgr2=0x7f414c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f414c078850 0x7f414c07ad10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f416c06add0 tx=0x7f4158009040 comp rx=0 tx=0).stop 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f416c073fb0 msgr2=0x7f416c1f5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f416c073fb0 0x7f416c1f5650 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f416000e7c0 tx=0x7f416000ec80 comp rx=0 tx=0).stop 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 shutdown_connections 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f414c078850 0x7f414c07ad10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f416c15f5a0 0x7f416c0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416c0ccf00 0x7f416c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 --2- 172.21.15.71:0/1888256829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f416c073fb0 0x7f416c1f5650 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 >> 172.21.15.71:0/1888256829 conn(0x7f416c0b1950 msgr2=0x7f416c0afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 shutdown_connections 2024-09-16T07:22:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:52.912+0000 7f4174200640 1 -- 172.21.15.71:0/1888256829 wait complete. 2024-09-16T07:22:52.928 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:22:52.928 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:22:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.085+0000 7f4051000640 1 -- 172.21.15.71:0/3296140336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c0725d0 msgr2=0x7f404c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.085+0000 7f4051000640 1 --2- 172.21.15.71:0/3296140336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c0725d0 0x7f404c0b28f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4038009830 tx=0x7f403802a510 comp rx=0 tx=0).stop 2024-09-16T07:22:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.085+0000 7f4051000640 1 -- 172.21.15.71:0/3296140336 shutdown_connections 2024-09-16T07:22:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.085+0000 7f4051000640 1 --2- 172.21.15.71:0/3296140336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c0725d0 0x7f404c0b28f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.085+0000 7f4051000640 1 -- 172.21.15.71:0/3296140336 >> 172.21.15.71:0/3296140336 conn(0x7f404c0cfa20 msgr2=0x7f404c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.085+0000 7f4051000640 1 -- 172.21.15.71:0/3296140336 shutdown_connections 2024-09-16T07:22:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.085+0000 7f4051000640 1 -- 172.21.15.71:0/3296140336 wait complete. 2024-09-16T07:22:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.086+0000 7f4051000640 1 Processor -- start 2024-09-16T07:22:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.087+0000 7f4051000640 1 -- start start 2024-09-16T07:22:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.087+0000 7f4051000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f404c0d0540 0x7f404c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.087+0000 7f4051000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f404c15bc30 0x7f404c0c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.087+0000 7f4051000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c151250 0x7f404c150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.087+0000 7f4051000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f404c074660 con 0x7f404c151250 2024-09-16T07:22:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.087+0000 7f4051000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f404c06fa30 con 0x7f404c15bc30 2024-09-16T07:22:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.087+0000 7f4051000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f404c150a50 con 0x7f404c0d0540 2024-09-16T07:22:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c151250 0x7f404c150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c151250 0x7f404c150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34574/0 (socket says 172.21.15.71:34574) 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 -- 172.21.15.71:0/583399446 learned_addr learned my addr 172.21.15.71:0/583399446 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404aa00640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f404c15bc30 0x7f404c0c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 -- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f404c0d0540 msgr2=0x7f404c1563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f404c0d0540 0x7f404c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 -- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f404c15bc30 msgr2=0x7f404c0c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f404c15bc30 0x7f404c0c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 -- 172.21.15.71:0/583399446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c1f27c0 con 0x7f404c151250 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404aa00640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f404c15bc30 0x7f404c0c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f404be00640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c151250 0x7f404c150510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f404400cbc0 tx=0x7f404400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.088+0000 7f4043400640 1 -- 172.21.15.71:0/583399446 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044016020 con 0x7f404c151250 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.089+0000 7f4043400640 1 -- 172.21.15.71:0/583399446 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f404400ae30 con 0x7f404c151250 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.089+0000 7f4043400640 1 -- 172.21.15.71:0/583399446 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044012660 con 0x7f404c151250 2024-09-16T07:22:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.089+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c1f2aa0 con 0x7f404c151250 2024-09-16T07:22:53.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.089+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c1f3060 con 0x7f404c151250 2024-09-16T07:22:53.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.090+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c0b28f0 con 0x7f404c151250 2024-09-16T07:22:53.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.091+0000 7f4043400640 1 -- 172.21.15.71:0/583399446 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4044019020 con 0x7f404c151250 2024-09-16T07:22:53.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.092+0000 7f4043400640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f402c078850 0x7f402c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.092+0000 7f4043400640 1 -- 172.21.15.71:0/583399446 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f404409afb0 con 0x7f404c151250 2024-09-16T07:22:53.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.092+0000 7f404b400640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f402c078850 0x7f402c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.093+0000 7f404b400640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f402c078850 0x7f402c07ad10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c006bc0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:53.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.096+0000 7f4043400640 1 -- 172.21.15.71:0/583399446 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4044020020 con 0x7f404c151250 2024-09-16T07:22:53.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.282+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f404c0d0180 con 0x7f402c078850 2024-09-16T07:22:53.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.288+0000 7f4043400640 1 -- 172.21.15.71:0/583399446 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245009 (secure 0 0 0) 0x7f404c0d0180 con 0x7f402c078850 2024-09-16T07:22:53.288 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:53.292 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:22:53.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f402c078850 msgr2=0x7f402c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f402c078850 0x7f402c07ad10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c006bc0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c151250 msgr2=0x7f404c150510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c151250 0x7f404c150510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f404400cbc0 tx=0x7f404400a430 comp rx=0 tx=0).stop 2024-09-16T07:22:53.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 shutdown_connections 2024-09-16T07:22:53.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f402c078850 0x7f402c07ad10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f404c151250 0x7f404c150510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f404c15bc30 0x7f404c0c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 --2- 172.21.15.71:0/583399446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f404c0d0540 0x7f404c1563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 >> 172.21.15.71:0/583399446 conn(0x7f404c0cfa20 msgr2=0x7f404c0b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:53.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 shutdown_connections 2024-09-16T07:22:53.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.292+0000 7f4051000640 1 -- 172.21.15.71:0/583399446 wait complete. 2024-09-16T07:22:53.307 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T07:22:52.294287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223116,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751604,"statfs":{"total":773094113280,"available":772865642496,"internally_reserved":0,"allocated":2797568,"data_stored":1699501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.193862"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123907+0000","last_change":"2024-09-16T07:22:48.123907+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123907+0000","last_undegraded":"2024-09-16T07:22:48.123907+0000","last_fullsized":"2024-09-16T07:22:48.123907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114075+0000","last_change":"2024-09-16T07:22:48.114075+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114075+0000","last_undegraded":"2024-09-16T07:22:48.114075+0000","last_fullsized":"2024-09-16T07:22:48.114075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119393+0000","last_change":"2024-09-16T07:22:48.119393+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119393+0000","last_undegraded":"2024-09-16T07:22:48.119393+0000","last_fullsized":"2024-09-16T07:22:48.119393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114916+0000","last_change":"2024-09-16T07:22:48.114916+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114916+0000","last_undegraded":"2024-09-16T07:22:48.114916+0000","last_fullsized":"2024-09-16T07:22:48.114916+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.122187+0000","last_change":"2024-09-16T07:22:48.122187+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.122187+0000","last_undegraded":"2024-09-16T07:22:48.122187+0000","last_fullsized":"2024-09-16T07:22:48.122187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115005+0000","last_change":"2024-09-16T07:22:48.115005+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115005+0000","last_undegraded":"2024-09-16T07:22:48.115005+0000","last_fullsized":"2024-09-16T07:22:48.115005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.120889+0000","last_change":"2024-09-16T07:22:48.120889+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.120889+0000","last_undegraded":"2024-09-16T07:22:48.120889+0000","last_fullsized":"2024-09-16T07:22:48.120889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.124292+0000","last_change":"2024-09-16T07:22:48.124292+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.124292+0000","last_undegraded":"2024-09-16T07:22:48.124292+0000","last_fullsized":"2024-09-16T07:22:48.124292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.122392+0000","last_change":"2024-09-16T07:22:48.122392+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.122392+0000","last_undegraded":"2024-09-16T07:22:48.122392+0000","last_fullsized":"2024-09-16T07:22:48.122392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.127288+0000","last_change":"2024-09-16T07:22:48.127288+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.127288+0000","last_undegraded":"2024-09-16T07:22:48.127288+0000","last_fullsized":"2024-09-16T07:22:48.127288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.117320+0000","last_change":"2024-09-16T07:22:48.117320+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.117320+0000","last_undegraded":"2024-09-16T07:22:48.117320+0000","last_fullsized":"2024-09-16T07:22:48.117320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119755+0000","last_change":"2024-09-16T07:22:48.119755+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119755+0000","last_undegraded":"2024-09-16T07:22:48.119755+0000","last_fullsized":"2024-09-16T07:22:48.119755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121828+0000","last_change":"2024-09-16T07:22:48.121828+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121828+0000","last_undegraded":"2024-09-16T07:22:48.121828+0000","last_fullsized":"2024-09-16T07:22:48.121828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132192+0000","last_change":"2024-09-16T07:22:49.133568+0000","last_active":"2024-09-16T07:22:50.132192+0000","last_peered":"2024-09-16T07:22:50.132192+0000","last_clean":"2024-09-16T07:22:50.132192+0000","last_became_active":"2024-09-16T07:22:49.133323+0000","last_became_peered":"2024-09-16T07:22:49.133323+0000","last_unstale":"2024-09-16T07:22:50.132192+0000","last_undegraded":"2024-09-16T07:22:50.132192+0000","last_fullsized":"2024-09-16T07:22:50.132192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:45.369698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114307+0000","last_change":"2024-09-16T07:22:48.114307+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114307+0000","last_undegraded":"2024-09-16T07:22:48.114307+0000","last_fullsized":"2024-09-16T07:22:48.114307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.117490+0000","last_change":"2024-09-16T07:22:48.117490+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.117490+0000","last_undegraded":"2024-09-16T07:22:48.117490+0000","last_fullsized":"2024-09-16T07:22:48.117490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115221+0000","last_change":"2024-09-16T07:22:48.115221+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115221+0000","last_undegraded":"2024-09-16T07:22:48.115221+0000","last_fullsized":"2024-09-16T07:22:48.115221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123978+0000","last_change":"2024-09-16T07:22:48.123978+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123978+0000","last_undegraded":"2024-09-16T07:22:48.123978+0000","last_fullsized":"2024-09-16T07:22:48.123978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115447+0000","last_change":"2024-09-16T07:22:48.115447+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115447+0000","last_undegraded":"2024-09-16T07:22:48.115447+0000","last_fullsized":"2024-09-16T07:22:48.115447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113455+0000","last_change":"2024-09-16T07:22:48.113455+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113455+0000","last_undegraded":"2024-09-16T07:22:48.113455+0000","last_fullsized":"2024-09-16T07:22:48.113455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.103688+0000","last_change":"2024-09-16T07:22:47.103688+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.103688+0000","last_undegraded":"2024-09-16T07:22:47.103688+0000","last_fullsized":"2024-09-16T07:22:47.103688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132215+0000","last_change":"2024-09-16T07:22:49.132936+0000","last_active":"2024-09-16T07:22:50.132215+0000","last_peered":"2024-09-16T07:22:50.132215+0000","last_clean":"2024-09-16T07:22:50.132215+0000","last_became_active":"2024-09-16T07:22:49.132677+0000","last_became_peered":"2024-09-16T07:22:49.132677+0000","last_unstale":"2024-09-16T07:22:50.132215+0000","last_undegraded":"2024-09-16T07:22:50.132215+0000","last_fullsized":"2024-09-16T07:22:50.132215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:16.641320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146411+0000","last_change":"2024-09-16T07:22:48.140563+0000","last_active":"2024-09-16T07:22:48.146411+0000","last_peered":"2024-09-16T07:22:48.146411+0000","last_clean":"2024-09-16T07:22:48.146411+0000","last_became_active":"2024-09-16T07:22:48.139864+0000","last_became_peered":"2024-09-16T07:22:48.139864+0000","last_unstale":"2024-09-16T07:22:48.146411+0000","last_undegraded":"2024-09-16T07:22:48.146411+0000","last_fullsized":"2024-09-16T07:22:48.146411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:46.703518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108684+0000","last_change":"2024-09-16T07:22:47.108684+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108684+0000","last_undegraded":"2024-09-16T07:22:47.108684+0000","last_fullsized":"2024-09-16T07:22:47.108684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142295+0000","last_change":"2024-09-16T07:22:48.135129+0000","last_active":"2024-09-16T07:22:48.142295+0000","last_peered":"2024-09-16T07:22:48.142295+0000","last_clean":"2024-09-16T07:22:48.142295+0000","last_became_active":"2024-09-16T07:22:48.134665+0000","last_became_peered":"2024-09-16T07:22:48.134665+0000","last_unstale":"2024-09-16T07:22:48.142295+0000","last_undegraded":"2024-09-16T07:22:48.142295+0000","last_fullsized":"2024-09-16T07:22:48.142295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130571+0000","last_change":"2024-09-16T07:22:49.129262+0000","last_active":"2024-09-16T07:22:50.130571+0000","last_peered":"2024-09-16T07:22:50.130571+0000","last_clean":"2024-09-16T07:22:50.130571+0000","last_became_active":"2024-09-16T07:22:49.128955+0000","last_became_peered":"2024-09-16T07:22:49.128955+0000","last_unstale":"2024-09-16T07:22:50.130571+0000","last_undegraded":"2024-09-16T07:22:50.130571+0000","last_fullsized":"2024-09-16T07:22:50.130571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:35.169775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.141837+0000","last_change":"2024-09-16T07:22:48.139495+0000","last_active":"2024-09-16T07:22:48.141837+0000","last_peered":"2024-09-16T07:22:48.141837+0000","last_clean":"2024-09-16T07:22:48.141837+0000","last_became_active":"2024-09-16T07:22:48.139004+0000","last_became_peered":"2024-09-16T07:22:48.139004+0000","last_unstale":"2024-09-16T07:22:48.141837+0000","last_undegraded":"2024-09-16T07:22:48.141837+0000","last_fullsized":"2024-09-16T07:22:48.141837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:57.410149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114809+0000","last_change":"2024-09-16T07:22:48.114809+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114809+0000","last_undegraded":"2024-09-16T07:22:48.114809+0000","last_fullsized":"2024-09-16T07:22:48.114809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.135332+0000","last_change":"2024-09-16T07:22:48.130409+0000","last_active":"2024-09-16T07:22:48.135332+0000","last_peered":"2024-09-16T07:22:48.135332+0000","last_clean":"2024-09-16T07:22:48.135332+0000","last_became_active":"2024-09-16T07:22:48.130159+0000","last_became_peered":"2024-09-16T07:22:48.130159+0000","last_unstale":"2024-09-16T07:22:48.135332+0000","last_undegraded":"2024-09-16T07:22:48.135332+0000","last_fullsized":"2024-09-16T07:22:48.135332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:34.669437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.110294+0000","last_change":"2024-09-16T07:22:47.110294+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.110294+0000","last_undegraded":"2024-09-16T07:22:47.110294+0000","last_fullsized":"2024-09-16T07:22:47.110294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.120521+0000","last_change":"2024-09-16T07:22:48.120521+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.120521+0000","last_undegraded":"2024-09-16T07:22:48.120521+0000","last_fullsized":"2024-09-16T07:22:48.120521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.136821+0000","last_change":"2024-09-16T07:22:48.130827+0000","last_active":"2024-09-16T07:22:48.136821+0000","last_peered":"2024-09-16T07:22:48.136821+0000","last_clean":"2024-09-16T07:22:48.136821+0000","last_became_active":"2024-09-16T07:22:48.129479+0000","last_became_peered":"2024-09-16T07:22:48.129479+0000","last_unstale":"2024-09-16T07:22:48.136821+0000","last_undegraded":"2024-09-16T07:22:48.136821+0000","last_fullsized":"2024-09-16T07:22:48.136821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119466+0000","last_change":"2024-09-16T07:22:48.119466+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119466+0000","last_undegraded":"2024-09-16T07:22:48.119466+0000","last_fullsized":"2024-09-16T07:22:48.119466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.109996+0000","last_change":"2024-09-16T07:22:47.109996+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.109996+0000","last_undegraded":"2024-09-16T07:22:47.109996+0000","last_fullsized":"2024-09-16T07:22:47.109996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142372+0000","last_change":"2024-09-16T07:22:48.137679+0000","last_active":"2024-09-16T07:22:48.142372+0000","last_peered":"2024-09-16T07:22:48.142372+0000","last_clean":"2024-09-16T07:22:48.142372+0000","last_became_active":"2024-09-16T07:22:48.134910+0000","last_became_peered":"2024-09-16T07:22:48.134910+0000","last_unstale":"2024-09-16T07:22:48.142372+0000","last_undegraded":"2024-09-16T07:22:48.142372+0000","last_fullsized":"2024-09-16T07:22:48.142372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.126214+0000","last_change":"2024-09-16T07:22:48.126214+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.126214+0000","last_undegraded":"2024-09-16T07:22:48.126214+0000","last_fullsized":"2024-09-16T07:22:48.126214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.144498+0000","last_change":"2024-09-16T07:22:48.139812+0000","last_active":"2024-09-16T07:22:48.144498+0000","last_peered":"2024-09-16T07:22:48.144498+0000","last_clean":"2024-09-16T07:22:48.144498+0000","last_became_active":"2024-09-16T07:22:48.139509+0000","last_became_peered":"2024-09-16T07:22:48.139509+0000","last_unstale":"2024-09-16T07:22:48.144498+0000","last_undegraded":"2024-09-16T07:22:48.144498+0000","last_fullsized":"2024-09-16T07:22:48.144498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113301+0000","last_change":"2024-09-16T07:22:48.113301+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113301+0000","last_undegraded":"2024-09-16T07:22:48.113301+0000","last_fullsized":"2024-09-16T07:22:48.113301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130773+0000","last_change":"2024-09-16T07:22:48.134633+0000","last_active":"2024-09-16T07:22:50.130773+0000","last_peered":"2024-09-16T07:22:50.130773+0000","last_clean":"2024-09-16T07:22:50.130773+0000","last_became_active":"2024-09-16T07:22:48.134291+0000","last_became_peered":"2024-09-16T07:22:48.134291+0000","last_unstale":"2024-09-16T07:22:50.130773+0000","last_undegraded":"2024-09-16T07:22:50.130773+0000","last_fullsized":"2024-09-16T07:22:50.130773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:35.346741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121798+0000","last_change":"2024-09-16T07:22:48.121798+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121798+0000","last_undegraded":"2024-09-16T07:22:48.121798+0000","last_fullsized":"2024-09-16T07:22:48.121798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.141898+0000","last_change":"2024-09-16T07:22:48.138039+0000","last_active":"2024-09-16T07:22:48.141898+0000","last_peered":"2024-09-16T07:22:48.141898+0000","last_clean":"2024-09-16T07:22:48.141898+0000","last_became_active":"2024-09-16T07:22:48.137746+0000","last_became_peered":"2024-09-16T07:22:48.137746+0000","last_unstale":"2024-09-16T07:22:48.141898+0000","last_undegraded":"2024-09-16T07:22:48.141898+0000","last_fullsized":"2024-09-16T07:22:48.141898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114410+0000","last_change":"2024-09-16T07:22:48.114410+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114410+0000","last_undegraded":"2024-09-16T07:22:48.114410+0000","last_fullsized":"2024-09-16T07:22:48.114410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132541+0000","last_change":"2024-09-16T07:22:48.137778+0000","last_active":"2024-09-16T07:22:50.132541+0000","last_peered":"2024-09-16T07:22:50.132541+0000","last_clean":"2024-09-16T07:22:50.132541+0000","last_became_active":"2024-09-16T07:22:48.137159+0000","last_became_peered":"2024-09-16T07:22:48.137159+0000","last_unstale":"2024-09-16T07:22:50.132541+0000","last_undegraded":"2024-09-16T07:22:50.132541+0000","last_fullsized":"2024-09-16T07:22:50.132541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:37.703838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123557+0000","last_change":"2024-09-16T07:22:48.123557+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123557+0000","last_undegraded":"2024-09-16T07:22:48.123557+0000","last_fullsized":"2024-09-16T07:22:48.123557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.104123+0000","last_change":"2024-09-16T07:22:47.104123+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.104123+0000","last_undegraded":"2024-09-16T07:22:47.104123+0000","last_fullsized":"2024-09-16T07:22:47.104123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121319+0000","last_change":"2024-09-16T07:22:48.121319+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121319+0000","last_undegraded":"2024-09-16T07:22:48.121319+0000","last_fullsized":"2024-09-16T07:22:48.121319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108513+0000","last_change":"2024-09-16T07:22:47.108513+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108513+0000","last_undegraded":"2024-09-16T07:22:47.108513+0000","last_fullsized":"2024-09-16T07:22:47.108513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.128400+0000","last_change":"2024-09-16T07:22:48.128400+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.128400+0000","last_undegraded":"2024-09-16T07:22:48.128400+0000","last_fullsized":"2024-09-16T07:22:48.128400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":107,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.111659+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:22:48.111659+0000","last_peered":"2024-09-16T07:22:48.111659+0000","last_clean":"2024-09-16T07:22:48.111659+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:22:48.111659+0000","last_undegraded":"2024-09-16T07:22:48.111659+0000","last_fullsized":"2024-09-16T07:22:48.111659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146419+0000","last_change":"2024-09-16T07:22:48.139675+0000","last_active":"2024-09-16T07:22:48.146419+0000","last_peered":"2024-09-16T07:22:48.146419+0000","last_clean":"2024-09-16T07:22:48.146419+0000","last_became_active":"2024-09-16T07:22:48.137989+0000","last_became_peered":"2024-09-16T07:22:48.137989+0000","last_unstale":"2024-09-16T07:22:48.146419+0000","last_undegraded":"2024-09-16T07:22:48.146419+0000","last_fullsized":"2024-09-16T07:22:48.146419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:08.500032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132139+0000","last_change":"2024-09-16T07:22:49.137102+0000","last_active":"2024-09-16T07:22:50.132139+0000","last_peered":"2024-09-16T07:22:50.132139+0000","last_clean":"2024-09-16T07:22:50.132139+0000","last_became_active":"2024-09-16T07:22:49.136755+0000","last_became_peered":"2024-09-16T07:22:49.136755+0000","last_unstale":"2024-09-16T07:22:50.132139+0000","last_undegraded":"2024-09-16T07:22:50.132139+0000","last_fullsized":"2024-09-16T07:22:50.132139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:26.935160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.139652+0000","last_change":"2024-09-16T07:22:48.133273+0000","last_active":"2024-09-16T07:22:48.139652+0000","last_peered":"2024-09-16T07:22:48.139652+0000","last_clean":"2024-09-16T07:22:48.139652+0000","last_became_active":"2024-09-16T07:22:48.132984+0000","last_became_peered":"2024-09-16T07:22:48.132984+0000","last_unstale":"2024-09-16T07:22:48.139652+0000","last_undegraded":"2024-09-16T07:22:48.139652+0000","last_fullsized":"2024-09-16T07:22:48.139652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:12.007091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.110270+0000","last_change":"2024-09-16T07:22:47.110270+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.110270+0000","last_undegraded":"2024-09-16T07:22:47.110270+0000","last_fullsized":"2024-09-16T07:22:47.110270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146385+0000","last_change":"2024-09-16T07:22:48.139380+0000","last_active":"2024-09-16T07:22:48.146385+0000","last_peered":"2024-09-16T07:22:48.146385+0000","last_clean":"2024-09-16T07:22:48.146385+0000","last_became_active":"2024-09-16T07:22:48.138052+0000","last_became_peered":"2024-09-16T07:22:48.138052+0000","last_unstale":"2024-09-16T07:22:48.146385+0000","last_undegraded":"2024-09-16T07:22:48.146385+0000","last_fullsized":"2024-09-16T07:22:48.146385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:07.509813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119932+0000","last_change":"2024-09-16T07:22:48.119932+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119932+0000","last_undegraded":"2024-09-16T07:22:48.119932+0000","last_fullsized":"2024-09-16T07:22:48.119932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.103481+0000","last_change":"2024-09-16T07:22:47.103481+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.103481+0000","last_undegraded":"2024-09-16T07:22:47.103481+0000","last_fullsized":"2024-09-16T07:22:47.103481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115959+0000","last_change":"2024-09-16T07:22:48.115959+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115959+0000","last_undegraded":"2024-09-16T07:22:48.115959+0000","last_fullsized":"2024-09-16T07:22:48.115959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142306+0000","last_change":"2024-09-16T07:22:48.135598+0000","last_active":"2024-09-16T07:22:48.142306+0000","last_peered":"2024-09-16T07:22:48.142306+0000","last_clean":"2024-09-16T07:22:48.142306+0000","last_became_active":"2024-09-16T07:22:48.135179+0000","last_became_peered":"2024-09-16T07:22:48.135179+0000","last_unstale":"2024-09-16T07:22:48.142306+0000","last_undegraded":"2024-09-16T07:22:48.142306+0000","last_fullsized":"2024-09-16T07:22:48.142306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138704+0000","last_change":"2024-09-16T07:22:48.137476+0000","last_active":"2024-09-16T07:22:50.138704+0000","last_peered":"2024-09-16T07:22:50.138704+0000","last_clean":"2024-09-16T07:22:50.138704+0000","last_became_active":"2024-09-16T07:22:48.136772+0000","last_became_peered":"2024-09-16T07:22:48.136772+0000","last_unstale":"2024-09-16T07:22:50.138704+0000","last_undegraded":"2024-09-16T07:22:50.138704+0000","last_fullsized":"2024-09-16T07:22:50.138704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.906034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132443+0000","last_change":"2024-09-16T07:22:49.144331+0000","last_active":"2024-09-16T07:22:50.132443+0000","last_peered":"2024-09-16T07:22:50.132443+0000","last_clean":"2024-09-16T07:22:50.132443+0000","last_became_active":"2024-09-16T07:22:49.144071+0000","last_became_peered":"2024-09-16T07:22:49.144071+0000","last_unstale":"2024-09-16T07:22:50.132443+0000","last_undegraded":"2024-09-16T07:22:50.132443+0000","last_fullsized":"2024-09-16T07:22:50.132443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.753793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.139583+0000","last_change":"2024-09-16T07:22:48.132847+0000","last_active":"2024-09-16T07:22:48.139583+0000","last_peered":"2024-09-16T07:22:48.139583+0000","last_clean":"2024-09-16T07:22:48.139583+0000","last_became_active":"2024-09-16T07:22:48.132264+0000","last_became_peered":"2024-09-16T07:22:48.132264+0000","last_unstale":"2024-09-16T07:22:48.139583+0000","last_undegraded":"2024-09-16T07:22:48.139583+0000","last_fullsized":"2024-09-16T07:22:48.139583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:52.735822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:49.138949+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:49.138580+0000","last_became_peered":"2024-09-16T07:22:49.138580+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:30.446387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.134073+0000","last_change":"2024-09-16T07:22:48.129412+0000","last_active":"2024-09-16T07:22:48.134073+0000","last_peered":"2024-09-16T07:22:48.134073+0000","last_clean":"2024-09-16T07:22:48.134073+0000","last_became_active":"2024-09-16T07:22:48.129137+0000","last_became_peered":"2024-09-16T07:22:48.129137+0000","last_unstale":"2024-09-16T07:22:48.134073+0000","last_undegraded":"2024-09-16T07:22:48.134073+0000","last_fullsized":"2024-09-16T07:22:48.134073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121359+0000","last_change":"2024-09-16T07:22:48.121359+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121359+0000","last_undegraded":"2024-09-16T07:22:48.121359+0000","last_fullsized":"2024-09-16T07:22:48.121359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130669+0000","last_change":"2024-09-16T07:22:48.134437+0000","last_active":"2024-09-16T07:22:50.130669+0000","last_peered":"2024-09-16T07:22:50.130669+0000","last_clean":"2024-09-16T07:22:50.130669+0000","last_became_active":"2024-09-16T07:22:48.134144+0000","last_became_peered":"2024-09-16T07:22:48.134144+0000","last_unstale":"2024-09-16T07:22:50.130669+0000","last_undegraded":"2024-09-16T07:22:50.130669+0000","last_fullsized":"2024-09-16T07:22:50.130669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:04.051560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130660+0000","last_change":"2024-09-16T07:22:49.141869+0000","last_active":"2024-09-16T07:22:50.130660+0000","last_peered":"2024-09-16T07:22:50.130660+0000","last_clean":"2024-09-16T07:22:50.130660+0000","last_became_active":"2024-09-16T07:22:49.141602+0000","last_became_peered":"2024-09-16T07:22:49.141602+0000","last_unstale":"2024-09-16T07:22:50.130660+0000","last_undegraded":"2024-09-16T07:22:50.130660+0000","last_fullsized":"2024-09-16T07:22:50.130660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:44.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108381+0000","last_change":"2024-09-16T07:22:47.108381+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108381+0000","last_undegraded":"2024-09-16T07:22:47.108381+0000","last_fullsized":"2024-09-16T07:22:47.108381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.140648+0000","last_change":"2024-09-16T07:22:48.131412+0000","last_active":"2024-09-16T07:22:48.140648+0000","last_peered":"2024-09-16T07:22:48.140648+0000","last_clean":"2024-09-16T07:22:48.140648+0000","last_became_active":"2024-09-16T07:22:48.131167+0000","last_became_peered":"2024-09-16T07:22:48.131167+0000","last_unstale":"2024-09-16T07:22:48.140648+0000","last_undegraded":"2024-09-16T07:22:48.140648+0000","last_fullsized":"2024-09-16T07:22:48.140648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123529+0000","last_change":"2024-09-16T07:22:48.123529+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123529+0000","last_undegraded":"2024-09-16T07:22:48.123529+0000","last_fullsized":"2024-09-16T07:22:48.123529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.128864+0000","last_change":"2024-09-16T07:22:48.128864+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.128864+0000","last_undegraded":"2024-09-16T07:22:48.128864+0000","last_fullsized":"2024-09-16T07:22:48.128864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.135197+0000","last_change":"2024-09-16T07:22:48.130394+0000","last_active":"2024-09-16T07:22:48.135197+0000","last_peered":"2024-09-16T07:22:48.135197+0000","last_clean":"2024-09-16T07:22:48.135197+0000","last_became_active":"2024-09-16T07:22:48.130013+0000","last_became_peered":"2024-09-16T07:22:48.130013+0000","last_unstale":"2024-09-16T07:22:48.135197+0000","last_undegraded":"2024-09-16T07:22:48.135197+0000","last_fullsized":"2024-09-16T07:22:48.135197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:52.735822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113994+0000","last_change":"2024-09-16T07:22:48.113994+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113994+0000","last_undegraded":"2024-09-16T07:22:48.113994+0000","last_fullsized":"2024-09-16T07:22:48.113994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.140231+0000","last_change":"2024-09-16T07:22:48.135403+0000","last_active":"2024-09-16T07:22:48.140231+0000","last_peered":"2024-09-16T07:22:48.140231+0000","last_clean":"2024-09-16T07:22:48.140231+0000","last_became_active":"2024-09-16T07:22:48.135142+0000","last_became_peered":"2024-09-16T07:22:48.135142+0000","last_unstale":"2024-09-16T07:22:48.140231+0000","last_undegraded":"2024-09-16T07:22:48.140231+0000","last_fullsized":"2024-09-16T07:22:48.140231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.104041+0000","last_change":"2024-09-16T07:22:47.104041+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.104041+0000","last_undegraded":"2024-09-16T07:22:47.104041+0000","last_fullsized":"2024-09-16T07:22:47.104041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132622+0000","last_change":"2024-09-16T07:22:48.141242+0000","last_active":"2024-09-16T07:22:50.132622+0000","last_peered":"2024-09-16T07:22:50.132622+0000","last_clean":"2024-09-16T07:22:50.132622+0000","last_became_active":"2024-09-16T07:22:48.139871+0000","last_became_peered":"2024-09-16T07:22:48.139871+0000","last_unstale":"2024-09-16T07:22:50.132622+0000","last_undegraded":"2024-09-16T07:22:50.132622+0000","last_fullsized":"2024-09-16T07:22:50.132622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:32.916879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":651725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":651725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:22:53.313 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:22:53.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.505+0000 7f59aea00640 1 -- 172.21.15.71:0/1583066417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598c003a00 msgr2=0x7f598c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.505+0000 7f59aea00640 1 --2- 172.21.15.71:0/1583066417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598c003a00 0x7f598c003e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5998002230 tx=0x7f598c005830 comp rx=0 tx=0).stop 2024-09-16T07:22:53.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.505+0000 7f59aca00640 1 -- 172.21.15.71:0/1583066417 reap_dead start 2024-09-16T07:22:53.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.505+0000 7f59aea00640 1 -- 172.21.15.71:0/1583066417 shutdown_connections 2024-09-16T07:22:53.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.505+0000 7f59aea00640 1 -- 172.21.15.71:0/1583066417 >> 172.21.15.71:0/1583066417 conn(0x7f59a80b1950 msgr2=0x7f59a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:53.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.505+0000 7f59aea00640 1 -- 172.21.15.71:0/1583066417 shutdown_connections 2024-09-16T07:22:53.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.505+0000 7f59aea00640 1 -- 172.21.15.71:0/1583066417 wait complete. 2024-09-16T07:22:53.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.506+0000 7f59aea00640 1 Processor -- start 2024-09-16T07:22:53.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.507+0000 7f59aea00640 1 -- start start 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.507+0000 7f59aea00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f598c003a00 0x7f59a80d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.507+0000 7f59aea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a8153a80 0x7f59a80cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.507+0000 7f59aea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59a80cf610 0x7f59a80cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.507+0000 7f59aea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f59a80745c0 con 0x7f59a8153a80 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.507+0000 7f59aea00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8074440 con 0x7f59a80cf610 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.507+0000 7f59aea00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f59a8074740 con 0x7f598c003a00 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a8153a80 0x7f59a80cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a8153a80 0x7f59a80cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34588/0 (socket says 172.21.15.71:34588) 2024-09-16T07:22:53.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 -- 172.21.15.71:0/1207339325 learned_addr learned my addr 172.21.15.71:0/1207339325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7e00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f598c003a00 0x7f59a80d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 -- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f598c003a00 msgr2=0x7f59a80d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f598c003a00 0x7f59a80d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 -- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59a80cf610 msgr2=0x7f59a80cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59a80cf610 0x7f59a80cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 -- 172.21.15.71:0/1207339325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a80b3d90 con 0x7f59a8153a80 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7e00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f598c003a00 0x7f59a80d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f59a7400640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a8153a80 0x7f59a80cf0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5994011b40 tx=0x7f5994009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.508+0000 7f599fe00640 1 -- 172.21.15.71:0/1207339325 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599400fc30 con 0x7f59a8153a80 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.509+0000 7f599fe00640 1 -- 172.21.15.71:0/1207339325 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599400fdd0 con 0x7f59a8153a80 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.509+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a80b4030 con 0x7f59a8153a80 2024-09-16T07:22:53.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.509+0000 7f599fe00640 1 -- 172.21.15.71:0/1207339325 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994004280 con 0x7f59a8153a80 2024-09-16T07:22:53.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.509+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a80c9070 con 0x7f59a8153a80 2024-09-16T07:22:53.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.510+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a80b2440 con 0x7f59a8153a80 2024-09-16T07:22:53.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.511+0000 7f599fe00640 1 -- 172.21.15.71:0/1207339325 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f59940035f0 con 0x7f59a8153a80 2024-09-16T07:22:53.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.512+0000 7f599fe00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5988078880 0x7f598807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.512+0000 7f599fe00640 1 -- 172.21.15.71:0/1207339325 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f599409b2d0 con 0x7f59a8153a80 2024-09-16T07:22:53.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.512+0000 7f59a7e00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5988078880 0x7f598807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.513+0000 7f59a7e00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5988078880 0x7f598807ad40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5998033000 tx=0x7f5998038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:53.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.516+0000 7f599fe00640 1 -- 172.21.15.71:0/1207339325 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f59940638a0 con 0x7f59a8153a80 2024-09-16T07:22:53.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.698+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f59a815bd90 con 0x7f5988078880 2024-09-16T07:22:53.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.703+0000 7f599fe00640 1 -- 172.21.15.71:0/1207339325 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245009 (secure 0 0 0) 0x7f599807fd30 con 0x7f5988078880 2024-09-16T07:22:53.705 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:53.706 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:22:53.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5988078880 msgr2=0x7f598807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5988078880 0x7f598807ad40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5998033000 tx=0x7f5998038040 comp rx=0 tx=0).stop 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a8153a80 msgr2=0x7f59a80cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a8153a80 0x7f59a80cf0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5994011b40 tx=0x7f5994009f10 comp rx=0 tx=0).stop 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 shutdown_connections 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5988078880 0x7f598807ad40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59a80cf610 0x7f59a80cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a8153a80 0x7f59a80cf0d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 --2- 172.21.15.71:0/1207339325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f598c003a00 0x7f59a80d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 >> 172.21.15.71:0/1207339325 conn(0x7f59a80b1950 msgr2=0x7f59a809ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 shutdown_connections 2024-09-16T07:22:53.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.707+0000 7f59aea00640 1 -- 172.21.15.71:0/1207339325 wait complete. 2024-09-16T07:22:53.722 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T07:22:52.294287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223116,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751604,"statfs":{"total":773094113280,"available":772865642496,"internally_reserved":0,"allocated":2797568,"data_stored":1699501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.193862"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123907+0000","last_change":"2024-09-16T07:22:48.123907+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123907+0000","last_undegraded":"2024-09-16T07:22:48.123907+0000","last_fullsized":"2024-09-16T07:22:48.123907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114075+0000","last_change":"2024-09-16T07:22:48.114075+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114075+0000","last_undegraded":"2024-09-16T07:22:48.114075+0000","last_fullsized":"2024-09-16T07:22:48.114075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119393+0000","last_change":"2024-09-16T07:22:48.119393+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119393+0000","last_undegraded":"2024-09-16T07:22:48.119393+0000","last_fullsized":"2024-09-16T07:22:48.119393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114916+0000","last_change":"2024-09-16T07:22:48.114916+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114916+0000","last_undegraded":"2024-09-16T07:22:48.114916+0000","last_fullsized":"2024-09-16T07:22:48.114916+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.122187+0000","last_change":"2024-09-16T07:22:48.122187+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.122187+0000","last_undegraded":"2024-09-16T07:22:48.122187+0000","last_fullsized":"2024-09-16T07:22:48.122187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115005+0000","last_change":"2024-09-16T07:22:48.115005+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115005+0000","last_undegraded":"2024-09-16T07:22:48.115005+0000","last_fullsized":"2024-09-16T07:22:48.115005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.120889+0000","last_change":"2024-09-16T07:22:48.120889+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.120889+0000","last_undegraded":"2024-09-16T07:22:48.120889+0000","last_fullsized":"2024-09-16T07:22:48.120889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.124292+0000","last_change":"2024-09-16T07:22:48.124292+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.124292+0000","last_undegraded":"2024-09-16T07:22:48.124292+0000","last_fullsized":"2024-09-16T07:22:48.124292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.122392+0000","last_change":"2024-09-16T07:22:48.122392+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.122392+0000","last_undegraded":"2024-09-16T07:22:48.122392+0000","last_fullsized":"2024-09-16T07:22:48.122392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.127288+0000","last_change":"2024-09-16T07:22:48.127288+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.127288+0000","last_undegraded":"2024-09-16T07:22:48.127288+0000","last_fullsized":"2024-09-16T07:22:48.127288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.117320+0000","last_change":"2024-09-16T07:22:48.117320+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.117320+0000","last_undegraded":"2024-09-16T07:22:48.117320+0000","last_fullsized":"2024-09-16T07:22:48.117320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119755+0000","last_change":"2024-09-16T07:22:48.119755+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119755+0000","last_undegraded":"2024-09-16T07:22:48.119755+0000","last_fullsized":"2024-09-16T07:22:48.119755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121828+0000","last_change":"2024-09-16T07:22:48.121828+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121828+0000","last_undegraded":"2024-09-16T07:22:48.121828+0000","last_fullsized":"2024-09-16T07:22:48.121828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132192+0000","last_change":"2024-09-16T07:22:49.133568+0000","last_active":"2024-09-16T07:22:50.132192+0000","last_peered":"2024-09-16T07:22:50.132192+0000","last_clean":"2024-09-16T07:22:50.132192+0000","last_became_active":"2024-09-16T07:22:49.133323+0000","last_became_peered":"2024-09-16T07:22:49.133323+0000","last_unstale":"2024-09-16T07:22:50.132192+0000","last_undegraded":"2024-09-16T07:22:50.132192+0000","last_fullsized":"2024-09-16T07:22:50.132192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:45.369698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114307+0000","last_change":"2024-09-16T07:22:48.114307+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114307+0000","last_undegraded":"2024-09-16T07:22:48.114307+0000","last_fullsized":"2024-09-16T07:22:48.114307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.117490+0000","last_change":"2024-09-16T07:22:48.117490+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.117490+0000","last_undegraded":"2024-09-16T07:22:48.117490+0000","last_fullsized":"2024-09-16T07:22:48.117490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115221+0000","last_change":"2024-09-16T07:22:48.115221+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115221+0000","last_undegraded":"2024-09-16T07:22:48.115221+0000","last_fullsized":"2024-09-16T07:22:48.115221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123978+0000","last_change":"2024-09-16T07:22:48.123978+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123978+0000","last_undegraded":"2024-09-16T07:22:48.123978+0000","last_fullsized":"2024-09-16T07:22:48.123978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115447+0000","last_change":"2024-09-16T07:22:48.115447+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115447+0000","last_undegraded":"2024-09-16T07:22:48.115447+0000","last_fullsized":"2024-09-16T07:22:48.115447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113455+0000","last_change":"2024-09-16T07:22:48.113455+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113455+0000","last_undegraded":"2024-09-16T07:22:48.113455+0000","last_fullsized":"2024-09-16T07:22:48.113455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.103688+0000","last_change":"2024-09-16T07:22:47.103688+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.103688+0000","last_undegraded":"2024-09-16T07:22:47.103688+0000","last_fullsized":"2024-09-16T07:22:47.103688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132215+0000","last_change":"2024-09-16T07:22:49.132936+0000","last_active":"2024-09-16T07:22:50.132215+0000","last_peered":"2024-09-16T07:22:50.132215+0000","last_clean":"2024-09-16T07:22:50.132215+0000","last_became_active":"2024-09-16T07:22:49.132677+0000","last_became_peered":"2024-09-16T07:22:49.132677+0000","last_unstale":"2024-09-16T07:22:50.132215+0000","last_undegraded":"2024-09-16T07:22:50.132215+0000","last_fullsized":"2024-09-16T07:22:50.132215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:16.641320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146411+0000","last_change":"2024-09-16T07:22:48.140563+0000","last_active":"2024-09-16T07:22:48.146411+0000","last_peered":"2024-09-16T07:22:48.146411+0000","last_clean":"2024-09-16T07:22:48.146411+0000","last_became_active":"2024-09-16T07:22:48.139864+0000","last_became_peered":"2024-09-16T07:22:48.139864+0000","last_unstale":"2024-09-16T07:22:48.146411+0000","last_undegraded":"2024-09-16T07:22:48.146411+0000","last_fullsized":"2024-09-16T07:22:48.146411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:46.703518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108684+0000","last_change":"2024-09-16T07:22:47.108684+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108684+0000","last_undegraded":"2024-09-16T07:22:47.108684+0000","last_fullsized":"2024-09-16T07:22:47.108684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142295+0000","last_change":"2024-09-16T07:22:48.135129+0000","last_active":"2024-09-16T07:22:48.142295+0000","last_peered":"2024-09-16T07:22:48.142295+0000","last_clean":"2024-09-16T07:22:48.142295+0000","last_became_active":"2024-09-16T07:22:48.134665+0000","last_became_peered":"2024-09-16T07:22:48.134665+0000","last_unstale":"2024-09-16T07:22:48.142295+0000","last_undegraded":"2024-09-16T07:22:48.142295+0000","last_fullsized":"2024-09-16T07:22:48.142295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130571+0000","last_change":"2024-09-16T07:22:49.129262+0000","last_active":"2024-09-16T07:22:50.130571+0000","last_peered":"2024-09-16T07:22:50.130571+0000","last_clean":"2024-09-16T07:22:50.130571+0000","last_became_active":"2024-09-16T07:22:49.128955+0000","last_became_peered":"2024-09-16T07:22:49.128955+0000","last_unstale":"2024-09-16T07:22:50.130571+0000","last_undegraded":"2024-09-16T07:22:50.130571+0000","last_fullsized":"2024-09-16T07:22:50.130571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:35.169775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.141837+0000","last_change":"2024-09-16T07:22:48.139495+0000","last_active":"2024-09-16T07:22:48.141837+0000","last_peered":"2024-09-16T07:22:48.141837+0000","last_clean":"2024-09-16T07:22:48.141837+0000","last_became_active":"2024-09-16T07:22:48.139004+0000","last_became_peered":"2024-09-16T07:22:48.139004+0000","last_unstale":"2024-09-16T07:22:48.141837+0000","last_undegraded":"2024-09-16T07:22:48.141837+0000","last_fullsized":"2024-09-16T07:22:48.141837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:57.410149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114809+0000","last_change":"2024-09-16T07:22:48.114809+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114809+0000","last_undegraded":"2024-09-16T07:22:48.114809+0000","last_fullsized":"2024-09-16T07:22:48.114809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.135332+0000","last_change":"2024-09-16T07:22:48.130409+0000","last_active":"2024-09-16T07:22:48.135332+0000","last_peered":"2024-09-16T07:22:48.135332+0000","last_clean":"2024-09-16T07:22:48.135332+0000","last_became_active":"2024-09-16T07:22:48.130159+0000","last_became_peered":"2024-09-16T07:22:48.130159+0000","last_unstale":"2024-09-16T07:22:48.135332+0000","last_undegraded":"2024-09-16T07:22:48.135332+0000","last_fullsized":"2024-09-16T07:22:48.135332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:34.669437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.110294+0000","last_change":"2024-09-16T07:22:47.110294+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.110294+0000","last_undegraded":"2024-09-16T07:22:47.110294+0000","last_fullsized":"2024-09-16T07:22:47.110294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.120521+0000","last_change":"2024-09-16T07:22:48.120521+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.120521+0000","last_undegraded":"2024-09-16T07:22:48.120521+0000","last_fullsized":"2024-09-16T07:22:48.120521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.136821+0000","last_change":"2024-09-16T07:22:48.130827+0000","last_active":"2024-09-16T07:22:48.136821+0000","last_peered":"2024-09-16T07:22:48.136821+0000","last_clean":"2024-09-16T07:22:48.136821+0000","last_became_active":"2024-09-16T07:22:48.129479+0000","last_became_peered":"2024-09-16T07:22:48.129479+0000","last_unstale":"2024-09-16T07:22:48.136821+0000","last_undegraded":"2024-09-16T07:22:48.136821+0000","last_fullsized":"2024-09-16T07:22:48.136821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119466+0000","last_change":"2024-09-16T07:22:48.119466+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119466+0000","last_undegraded":"2024-09-16T07:22:48.119466+0000","last_fullsized":"2024-09-16T07:22:48.119466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.109996+0000","last_change":"2024-09-16T07:22:47.109996+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.109996+0000","last_undegraded":"2024-09-16T07:22:47.109996+0000","last_fullsized":"2024-09-16T07:22:47.109996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142372+0000","last_change":"2024-09-16T07:22:48.137679+0000","last_active":"2024-09-16T07:22:48.142372+0000","last_peered":"2024-09-16T07:22:48.142372+0000","last_clean":"2024-09-16T07:22:48.142372+0000","last_became_active":"2024-09-16T07:22:48.134910+0000","last_became_peered":"2024-09-16T07:22:48.134910+0000","last_unstale":"2024-09-16T07:22:48.142372+0000","last_undegraded":"2024-09-16T07:22:48.142372+0000","last_fullsized":"2024-09-16T07:22:48.142372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.126214+0000","last_change":"2024-09-16T07:22:48.126214+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.126214+0000","last_undegraded":"2024-09-16T07:22:48.126214+0000","last_fullsized":"2024-09-16T07:22:48.126214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.144498+0000","last_change":"2024-09-16T07:22:48.139812+0000","last_active":"2024-09-16T07:22:48.144498+0000","last_peered":"2024-09-16T07:22:48.144498+0000","last_clean":"2024-09-16T07:22:48.144498+0000","last_became_active":"2024-09-16T07:22:48.139509+0000","last_became_peered":"2024-09-16T07:22:48.139509+0000","last_unstale":"2024-09-16T07:22:48.144498+0000","last_undegraded":"2024-09-16T07:22:48.144498+0000","last_fullsized":"2024-09-16T07:22:48.144498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113301+0000","last_change":"2024-09-16T07:22:48.113301+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113301+0000","last_undegraded":"2024-09-16T07:22:48.113301+0000","last_fullsized":"2024-09-16T07:22:48.113301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130773+0000","last_change":"2024-09-16T07:22:48.134633+0000","last_active":"2024-09-16T07:22:50.130773+0000","last_peered":"2024-09-16T07:22:50.130773+0000","last_clean":"2024-09-16T07:22:50.130773+0000","last_became_active":"2024-09-16T07:22:48.134291+0000","last_became_peered":"2024-09-16T07:22:48.134291+0000","last_unstale":"2024-09-16T07:22:50.130773+0000","last_undegraded":"2024-09-16T07:22:50.130773+0000","last_fullsized":"2024-09-16T07:22:50.130773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:35.346741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121798+0000","last_change":"2024-09-16T07:22:48.121798+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121798+0000","last_undegraded":"2024-09-16T07:22:48.121798+0000","last_fullsized":"2024-09-16T07:22:48.121798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.141898+0000","last_change":"2024-09-16T07:22:48.138039+0000","last_active":"2024-09-16T07:22:48.141898+0000","last_peered":"2024-09-16T07:22:48.141898+0000","last_clean":"2024-09-16T07:22:48.141898+0000","last_became_active":"2024-09-16T07:22:48.137746+0000","last_became_peered":"2024-09-16T07:22:48.137746+0000","last_unstale":"2024-09-16T07:22:48.141898+0000","last_undegraded":"2024-09-16T07:22:48.141898+0000","last_fullsized":"2024-09-16T07:22:48.141898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114410+0000","last_change":"2024-09-16T07:22:48.114410+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114410+0000","last_undegraded":"2024-09-16T07:22:48.114410+0000","last_fullsized":"2024-09-16T07:22:48.114410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132541+0000","last_change":"2024-09-16T07:22:48.137778+0000","last_active":"2024-09-16T07:22:50.132541+0000","last_peered":"2024-09-16T07:22:50.132541+0000","last_clean":"2024-09-16T07:22:50.132541+0000","last_became_active":"2024-09-16T07:22:48.137159+0000","last_became_peered":"2024-09-16T07:22:48.137159+0000","last_unstale":"2024-09-16T07:22:50.132541+0000","last_undegraded":"2024-09-16T07:22:50.132541+0000","last_fullsized":"2024-09-16T07:22:50.132541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:37.703838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123557+0000","last_change":"2024-09-16T07:22:48.123557+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123557+0000","last_undegraded":"2024-09-16T07:22:48.123557+0000","last_fullsized":"2024-09-16T07:22:48.123557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.104123+0000","last_change":"2024-09-16T07:22:47.104123+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.104123+0000","last_undegraded":"2024-09-16T07:22:47.104123+0000","last_fullsized":"2024-09-16T07:22:47.104123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121319+0000","last_change":"2024-09-16T07:22:48.121319+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121319+0000","last_undegraded":"2024-09-16T07:22:48.121319+0000","last_fullsized":"2024-09-16T07:22:48.121319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108513+0000","last_change":"2024-09-16T07:22:47.108513+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108513+0000","last_undegraded":"2024-09-16T07:22:47.108513+0000","last_fullsized":"2024-09-16T07:22:47.108513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.128400+0000","last_change":"2024-09-16T07:22:48.128400+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.128400+0000","last_undegraded":"2024-09-16T07:22:48.128400+0000","last_fullsized":"2024-09-16T07:22:48.128400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":107,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.111659+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:22:48.111659+0000","last_peered":"2024-09-16T07:22:48.111659+0000","last_clean":"2024-09-16T07:22:48.111659+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:22:48.111659+0000","last_undegraded":"2024-09-16T07:22:48.111659+0000","last_fullsized":"2024-09-16T07:22:48.111659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146419+0000","last_change":"2024-09-16T07:22:48.139675+0000","last_active":"2024-09-16T07:22:48.146419+0000","last_peered":"2024-09-16T07:22:48.146419+0000","last_clean":"2024-09-16T07:22:48.146419+0000","last_became_active":"2024-09-16T07:22:48.137989+0000","last_became_peered":"2024-09-16T07:22:48.137989+0000","last_unstale":"2024-09-16T07:22:48.146419+0000","last_undegraded":"2024-09-16T07:22:48.146419+0000","last_fullsized":"2024-09-16T07:22:48.146419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:08.500032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132139+0000","last_change":"2024-09-16T07:22:49.137102+0000","last_active":"2024-09-16T07:22:50.132139+0000","last_peered":"2024-09-16T07:22:50.132139+0000","last_clean":"2024-09-16T07:22:50.132139+0000","last_became_active":"2024-09-16T07:22:49.136755+0000","last_became_peered":"2024-09-16T07:22:49.136755+0000","last_unstale":"2024-09-16T07:22:50.132139+0000","last_undegraded":"2024-09-16T07:22:50.132139+0000","last_fullsized":"2024-09-16T07:22:50.132139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:26.935160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.139652+0000","last_change":"2024-09-16T07:22:48.133273+0000","last_active":"2024-09-16T07:22:48.139652+0000","last_peered":"2024-09-16T07:22:48.139652+0000","last_clean":"2024-09-16T07:22:48.139652+0000","last_became_active":"2024-09-16T07:22:48.132984+0000","last_became_peered":"2024-09-16T07:22:48.132984+0000","last_unstale":"2024-09-16T07:22:48.139652+0000","last_undegraded":"2024-09-16T07:22:48.139652+0000","last_fullsized":"2024-09-16T07:22:48.139652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:12.007091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.110270+0000","last_change":"2024-09-16T07:22:47.110270+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.110270+0000","last_undegraded":"2024-09-16T07:22:47.110270+0000","last_fullsized":"2024-09-16T07:22:47.110270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146385+0000","last_change":"2024-09-16T07:22:48.139380+0000","last_active":"2024-09-16T07:22:48.146385+0000","last_peered":"2024-09-16T07:22:48.146385+0000","last_clean":"2024-09-16T07:22:48.146385+0000","last_became_active":"2024-09-16T07:22:48.138052+0000","last_became_peered":"2024-09-16T07:22:48.138052+0000","last_unstale":"2024-09-16T07:22:48.146385+0000","last_undegraded":"2024-09-16T07:22:48.146385+0000","last_fullsized":"2024-09-16T07:22:48.146385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:07.509813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119932+0000","last_change":"2024-09-16T07:22:48.119932+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119932+0000","last_undegraded":"2024-09-16T07:22:48.119932+0000","last_fullsized":"2024-09-16T07:22:48.119932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.103481+0000","last_change":"2024-09-16T07:22:47.103481+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.103481+0000","last_undegraded":"2024-09-16T07:22:47.103481+0000","last_fullsized":"2024-09-16T07:22:47.103481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115959+0000","last_change":"2024-09-16T07:22:48.115959+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115959+0000","last_undegraded":"2024-09-16T07:22:48.115959+0000","last_fullsized":"2024-09-16T07:22:48.115959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142306+0000","last_change":"2024-09-16T07:22:48.135598+0000","last_active":"2024-09-16T07:22:48.142306+0000","last_peered":"2024-09-16T07:22:48.142306+0000","last_clean":"2024-09-16T07:22:48.142306+0000","last_became_active":"2024-09-16T07:22:48.135179+0000","last_became_peered":"2024-09-16T07:22:48.135179+0000","last_unstale":"2024-09-16T07:22:48.142306+0000","last_undegraded":"2024-09-16T07:22:48.142306+0000","last_fullsized":"2024-09-16T07:22:48.142306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138704+0000","last_change":"2024-09-16T07:22:48.137476+0000","last_active":"2024-09-16T07:22:50.138704+0000","last_peered":"2024-09-16T07:22:50.138704+0000","last_clean":"2024-09-16T07:22:50.138704+0000","last_became_active":"2024-09-16T07:22:48.136772+0000","last_became_peered":"2024-09-16T07:22:48.136772+0000","last_unstale":"2024-09-16T07:22:50.138704+0000","last_undegraded":"2024-09-16T07:22:50.138704+0000","last_fullsized":"2024-09-16T07:22:50.138704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.906034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132443+0000","last_change":"2024-09-16T07:22:49.144331+0000","last_active":"2024-09-16T07:22:50.132443+0000","last_peered":"2024-09-16T07:22:50.132443+0000","last_clean":"2024-09-16T07:22:50.132443+0000","last_became_active":"2024-09-16T07:22:49.144071+0000","last_became_peered":"2024-09-16T07:22:49.144071+0000","last_unstale":"2024-09-16T07:22:50.132443+0000","last_undegraded":"2024-09-16T07:22:50.132443+0000","last_fullsized":"2024-09-16T07:22:50.132443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.753793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.139583+0000","last_change":"2024-09-16T07:22:48.132847+0000","last_active":"2024-09-16T07:22:48.139583+0000","last_peered":"2024-09-16T07:22:48.139583+0000","last_clean":"2024-09-16T07:22:48.139583+0000","last_became_active":"2024-09-16T07:22:48.132264+0000","last_became_peered":"2024-09-16T07:22:48.132264+0000","last_unstale":"2024-09-16T07:22:48.139583+0000","last_undegraded":"2024-09-16T07:22:48.139583+0000","last_fullsized":"2024-09-16T07:22:48.139583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:52.735822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:49.138949+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:49.138580+0000","last_became_peered":"2024-09-16T07:22:49.138580+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:30.446387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.134073+0000","last_change":"2024-09-16T07:22:48.129412+0000","last_active":"2024-09-16T07:22:48.134073+0000","last_peered":"2024-09-16T07:22:48.134073+0000","last_clean":"2024-09-16T07:22:48.134073+0000","last_became_active":"2024-09-16T07:22:48.129137+0000","last_became_peered":"2024-09-16T07:22:48.129137+0000","last_unstale":"2024-09-16T07:22:48.134073+0000","last_undegraded":"2024-09-16T07:22:48.134073+0000","last_fullsized":"2024-09-16T07:22:48.134073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121359+0000","last_change":"2024-09-16T07:22:48.121359+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121359+0000","last_undegraded":"2024-09-16T07:22:48.121359+0000","last_fullsized":"2024-09-16T07:22:48.121359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130669+0000","last_change":"2024-09-16T07:22:48.134437+0000","last_active":"2024-09-16T07:22:50.130669+0000","last_peered":"2024-09-16T07:22:50.130669+0000","last_clean":"2024-09-16T07:22:50.130669+0000","last_became_active":"2024-09-16T07:22:48.134144+0000","last_became_peered":"2024-09-16T07:22:48.134144+0000","last_unstale":"2024-09-16T07:22:50.130669+0000","last_undegraded":"2024-09-16T07:22:50.130669+0000","last_fullsized":"2024-09-16T07:22:50.130669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:04.051560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130660+0000","last_change":"2024-09-16T07:22:49.141869+0000","last_active":"2024-09-16T07:22:50.130660+0000","last_peered":"2024-09-16T07:22:50.130660+0000","last_clean":"2024-09-16T07:22:50.130660+0000","last_became_active":"2024-09-16T07:22:49.141602+0000","last_became_peered":"2024-09-16T07:22:49.141602+0000","last_unstale":"2024-09-16T07:22:50.130660+0000","last_undegraded":"2024-09-16T07:22:50.130660+0000","last_fullsized":"2024-09-16T07:22:50.130660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:44.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108381+0000","last_change":"2024-09-16T07:22:47.108381+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108381+0000","last_undegraded":"2024-09-16T07:22:47.108381+0000","last_fullsized":"2024-09-16T07:22:47.108381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.140648+0000","last_change":"2024-09-16T07:22:48.131412+0000","last_active":"2024-09-16T07:22:48.140648+0000","last_peered":"2024-09-16T07:22:48.140648+0000","last_clean":"2024-09-16T07:22:48.140648+0000","last_became_active":"2024-09-16T07:22:48.131167+0000","last_became_peered":"2024-09-16T07:22:48.131167+0000","last_unstale":"2024-09-16T07:22:48.140648+0000","last_undegraded":"2024-09-16T07:22:48.140648+0000","last_fullsized":"2024-09-16T07:22:48.140648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123529+0000","last_change":"2024-09-16T07:22:48.123529+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123529+0000","last_undegraded":"2024-09-16T07:22:48.123529+0000","last_fullsized":"2024-09-16T07:22:48.123529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.128864+0000","last_change":"2024-09-16T07:22:48.128864+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.128864+0000","last_undegraded":"2024-09-16T07:22:48.128864+0000","last_fullsized":"2024-09-16T07:22:48.128864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.135197+0000","last_change":"2024-09-16T07:22:48.130394+0000","last_active":"2024-09-16T07:22:48.135197+0000","last_peered":"2024-09-16T07:22:48.135197+0000","last_clean":"2024-09-16T07:22:48.135197+0000","last_became_active":"2024-09-16T07:22:48.130013+0000","last_became_peered":"2024-09-16T07:22:48.130013+0000","last_unstale":"2024-09-16T07:22:48.135197+0000","last_undegraded":"2024-09-16T07:22:48.135197+0000","last_fullsized":"2024-09-16T07:22:48.135197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:52.735822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113994+0000","last_change":"2024-09-16T07:22:48.113994+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113994+0000","last_undegraded":"2024-09-16T07:22:48.113994+0000","last_fullsized":"2024-09-16T07:22:48.113994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.140231+0000","last_change":"2024-09-16T07:22:48.135403+0000","last_active":"2024-09-16T07:22:48.140231+0000","last_peered":"2024-09-16T07:22:48.140231+0000","last_clean":"2024-09-16T07:22:48.140231+0000","last_became_active":"2024-09-16T07:22:48.135142+0000","last_became_peered":"2024-09-16T07:22:48.135142+0000","last_unstale":"2024-09-16T07:22:48.140231+0000","last_undegraded":"2024-09-16T07:22:48.140231+0000","last_fullsized":"2024-09-16T07:22:48.140231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.104041+0000","last_change":"2024-09-16T07:22:47.104041+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.104041+0000","last_undegraded":"2024-09-16T07:22:47.104041+0000","last_fullsized":"2024-09-16T07:22:47.104041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132622+0000","last_change":"2024-09-16T07:22:48.141242+0000","last_active":"2024-09-16T07:22:50.132622+0000","last_peered":"2024-09-16T07:22:50.132622+0000","last_clean":"2024-09-16T07:22:50.132622+0000","last_became_active":"2024-09-16T07:22:48.139871+0000","last_became_peered":"2024-09-16T07:22:48.139871+0000","last_unstale":"2024-09-16T07:22:50.132622+0000","last_undegraded":"2024-09-16T07:22:50.132622+0000","last_fullsized":"2024-09-16T07:22:50.132622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:32.916879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":651725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":651725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:22:53.726 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:22:53.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.919+0000 7f6a3b600640 1 -- 172.21.15.71:0/2590738766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ccf00 msgr2=0x7f6a340b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.919+0000 7f6a3b600640 1 --2- 172.21.15.71:0/2590738766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ccf00 0x7f6a340b2440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009830 tx=0x7f6a2002a480 comp rx=0 tx=0).stop 2024-09-16T07:22:53.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.919+0000 7f6a3b600640 1 -- 172.21.15.71:0/2590738766 shutdown_connections 2024-09-16T07:22:53.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.919+0000 7f6a3b600640 1 --2- 172.21.15.71:0/2590738766 >> v2:172.21.15.71:3300/0 conn(0x7f6a34073fb0 0x7f6a34074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.919+0000 7f6a3b600640 1 --2- 172.21.15.71:0/2590738766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ccf00 0x7f6a340b2440 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.919+0000 7f6a3b600640 1 -- 172.21.15.71:0/2590738766 >> 172.21.15.71:0/2590738766 conn(0x7f6a340b1950 msgr2=0x7f6a340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.920+0000 7f6a3b600640 1 -- 172.21.15.71:0/2590738766 shutdown_connections 2024-09-16T07:22:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.920+0000 7f6a3b600640 1 -- 172.21.15.71:0/2590738766 wait complete. 2024-09-16T07:22:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.921+0000 7f6a3b600640 1 Processor -- start 2024-09-16T07:22:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.921+0000 7f6a3b600640 1 -- start start 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a3b600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a34073fb0 0x7f6a3415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a3b600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a340ccf00 0x7f6a341f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a3b600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3415f5a0 0x7f6a3415cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a3b600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3406f2e0 con 0x7f6a34073fb0 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a3b600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3406f160 con 0x7f6a340ccf00 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a3b600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3406f460 con 0x7f6a3415f5a0 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a38c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a34073fb0 0x7f6a3415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a38c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a34073fb0 0x7f6a3415c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34608/0 (socket says 172.21.15.71:34608) 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.922+0000 7f6a38c00640 1 -- 172.21.15.71:0/3066860667 learned_addr learned my addr 172.21.15.71:0/3066860667 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a33e00640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a340ccf00 0x7f6a341f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a39600640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3415f5a0 0x7f6a3415cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a38c00640 1 -- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3415f5a0 msgr2=0x7f6a3415cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a38c00640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3415f5a0 0x7f6a3415cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a38c00640 1 -- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a340ccf00 msgr2=0x7f6a341f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a38c00640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a340ccf00 0x7f6a341f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a38c00640 1 -- 172.21.15.71:0/3066860667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a340cf850 con 0x7f6a34073fb0 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a38c00640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a34073fb0 0x7f6a3415c170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c460 tx=0x7f6a2400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a30c00640 1 -- 172.21.15.71:0/3066860667 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24011790 con 0x7f6a34073fb0 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a30c00640 1 -- 172.21.15.71:0/3066860667 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a24017070 con 0x7f6a34073fb0 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a340cdfc0 con 0x7f6a34073fb0 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a30c00640 1 -- 172.21.15.71:0/3066860667 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24011ce0 con 0x7f6a34073fb0 2024-09-16T07:22:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.923+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a340ce580 con 0x7f6a34073fb0 2024-09-16T07:22:53.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.924+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3406f9f0 con 0x7f6a34073fb0 2024-09-16T07:22:53.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.925+0000 7f6a30c00640 1 -- 172.21.15.71:0/3066860667 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6a2401c460 con 0x7f6a34073fb0 2024-09-16T07:22:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.926+0000 7f6a30c00640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a14078880 0x7f6a1407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.926+0000 7f6a30c00640 1 -- 172.21.15.71:0/3066860667 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6a2409a470 con 0x7f6a34073fb0 2024-09-16T07:22:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.927+0000 7f6a33e00640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a14078880 0x7f6a1407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.927+0000 7f6a33e00640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a14078880 0x7f6a1407ad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6a20034000 tx=0x7f6a20004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:53.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:53.930+0000 7f6a30c00640 1 -- 172.21.15.71:0/3066860667 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6a24062b70 con 0x7f6a34073fb0 2024-09-16T07:22:54.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.115+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a34156090 con 0x7f6a14078880 2024-09-16T07:22:54.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.120+0000 7f6a30c00640 1 -- 172.21.15.71:0/3066860667 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245009 (secure 0 0 0) 0x7f6a34156090 con 0x7f6a14078880 2024-09-16T07:22:54.122 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:54.123 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a14078880 msgr2=0x7f6a1407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a14078880 0x7f6a1407ad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6a20034000 tx=0x7f6a20004730 comp rx=0 tx=0).stop 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a34073fb0 msgr2=0x7f6a3415c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a34073fb0 0x7f6a3415c170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c460 tx=0x7f6a2400c860 comp rx=0 tx=0).stop 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 shutdown_connections 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a14078880 0x7f6a1407ad40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3415f5a0 0x7f6a3415cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a340ccf00 0x7f6a341f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 --2- 172.21.15.71:0/3066860667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a34073fb0 0x7f6a3415c170 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 >> 172.21.15.71:0/3066860667 conn(0x7f6a340b1950 msgr2=0x7f6a340b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:54.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 shutdown_connections 2024-09-16T07:22:54.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.124+0000 7f6a3b600640 1 -- 172.21.15.71:0/3066860667 wait complete. 2024-09-16T07:22:54.138 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T07:22:52.294287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223116,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751604,"statfs":{"total":773094113280,"available":772865642496,"internally_reserved":0,"allocated":2797568,"data_stored":1699501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.193862"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123907+0000","last_change":"2024-09-16T07:22:48.123907+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123907+0000","last_undegraded":"2024-09-16T07:22:48.123907+0000","last_fullsized":"2024-09-16T07:22:48.123907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114075+0000","last_change":"2024-09-16T07:22:48.114075+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114075+0000","last_undegraded":"2024-09-16T07:22:48.114075+0000","last_fullsized":"2024-09-16T07:22:48.114075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119393+0000","last_change":"2024-09-16T07:22:48.119393+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119393+0000","last_undegraded":"2024-09-16T07:22:48.119393+0000","last_fullsized":"2024-09-16T07:22:48.119393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114916+0000","last_change":"2024-09-16T07:22:48.114916+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114916+0000","last_undegraded":"2024-09-16T07:22:48.114916+0000","last_fullsized":"2024-09-16T07:22:48.114916+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.122187+0000","last_change":"2024-09-16T07:22:48.122187+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.122187+0000","last_undegraded":"2024-09-16T07:22:48.122187+0000","last_fullsized":"2024-09-16T07:22:48.122187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115005+0000","last_change":"2024-09-16T07:22:48.115005+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115005+0000","last_undegraded":"2024-09-16T07:22:48.115005+0000","last_fullsized":"2024-09-16T07:22:48.115005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.120889+0000","last_change":"2024-09-16T07:22:48.120889+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.120889+0000","last_undegraded":"2024-09-16T07:22:48.120889+0000","last_fullsized":"2024-09-16T07:22:48.120889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.124292+0000","last_change":"2024-09-16T07:22:48.124292+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.124292+0000","last_undegraded":"2024-09-16T07:22:48.124292+0000","last_fullsized":"2024-09-16T07:22:48.124292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.122392+0000","last_change":"2024-09-16T07:22:48.122392+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.122392+0000","last_undegraded":"2024-09-16T07:22:48.122392+0000","last_fullsized":"2024-09-16T07:22:48.122392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.127288+0000","last_change":"2024-09-16T07:22:48.127288+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.127288+0000","last_undegraded":"2024-09-16T07:22:48.127288+0000","last_fullsized":"2024-09-16T07:22:48.127288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.117320+0000","last_change":"2024-09-16T07:22:48.117320+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.117320+0000","last_undegraded":"2024-09-16T07:22:48.117320+0000","last_fullsized":"2024-09-16T07:22:48.117320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119755+0000","last_change":"2024-09-16T07:22:48.119755+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119755+0000","last_undegraded":"2024-09-16T07:22:48.119755+0000","last_fullsized":"2024-09-16T07:22:48.119755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121828+0000","last_change":"2024-09-16T07:22:48.121828+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121828+0000","last_undegraded":"2024-09-16T07:22:48.121828+0000","last_fullsized":"2024-09-16T07:22:48.121828+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132192+0000","last_change":"2024-09-16T07:22:49.133568+0000","last_active":"2024-09-16T07:22:50.132192+0000","last_peered":"2024-09-16T07:22:50.132192+0000","last_clean":"2024-09-16T07:22:50.132192+0000","last_became_active":"2024-09-16T07:22:49.133323+0000","last_became_peered":"2024-09-16T07:22:49.133323+0000","last_unstale":"2024-09-16T07:22:50.132192+0000","last_undegraded":"2024-09-16T07:22:50.132192+0000","last_fullsized":"2024-09-16T07:22:50.132192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:45.369698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114307+0000","last_change":"2024-09-16T07:22:48.114307+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114307+0000","last_undegraded":"2024-09-16T07:22:48.114307+0000","last_fullsized":"2024-09-16T07:22:48.114307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.117490+0000","last_change":"2024-09-16T07:22:48.117490+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.117490+0000","last_undegraded":"2024-09-16T07:22:48.117490+0000","last_fullsized":"2024-09-16T07:22:48.117490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115221+0000","last_change":"2024-09-16T07:22:48.115221+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115221+0000","last_undegraded":"2024-09-16T07:22:48.115221+0000","last_fullsized":"2024-09-16T07:22:48.115221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123978+0000","last_change":"2024-09-16T07:22:48.123978+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123978+0000","last_undegraded":"2024-09-16T07:22:48.123978+0000","last_fullsized":"2024-09-16T07:22:48.123978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115447+0000","last_change":"2024-09-16T07:22:48.115447+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115447+0000","last_undegraded":"2024-09-16T07:22:48.115447+0000","last_fullsized":"2024-09-16T07:22:48.115447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113455+0000","last_change":"2024-09-16T07:22:48.113455+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113455+0000","last_undegraded":"2024-09-16T07:22:48.113455+0000","last_fullsized":"2024-09-16T07:22:48.113455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.103688+0000","last_change":"2024-09-16T07:22:47.103688+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.103688+0000","last_undegraded":"2024-09-16T07:22:47.103688+0000","last_fullsized":"2024-09-16T07:22:47.103688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132215+0000","last_change":"2024-09-16T07:22:49.132936+0000","last_active":"2024-09-16T07:22:50.132215+0000","last_peered":"2024-09-16T07:22:50.132215+0000","last_clean":"2024-09-16T07:22:50.132215+0000","last_became_active":"2024-09-16T07:22:49.132677+0000","last_became_peered":"2024-09-16T07:22:49.132677+0000","last_unstale":"2024-09-16T07:22:50.132215+0000","last_undegraded":"2024-09-16T07:22:50.132215+0000","last_fullsized":"2024-09-16T07:22:50.132215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:16.641320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146411+0000","last_change":"2024-09-16T07:22:48.140563+0000","last_active":"2024-09-16T07:22:48.146411+0000","last_peered":"2024-09-16T07:22:48.146411+0000","last_clean":"2024-09-16T07:22:48.146411+0000","last_became_active":"2024-09-16T07:22:48.139864+0000","last_became_peered":"2024-09-16T07:22:48.139864+0000","last_unstale":"2024-09-16T07:22:48.146411+0000","last_undegraded":"2024-09-16T07:22:48.146411+0000","last_fullsized":"2024-09-16T07:22:48.146411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:46.703518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108684+0000","last_change":"2024-09-16T07:22:47.108684+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108684+0000","last_undegraded":"2024-09-16T07:22:47.108684+0000","last_fullsized":"2024-09-16T07:22:47.108684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142295+0000","last_change":"2024-09-16T07:22:48.135129+0000","last_active":"2024-09-16T07:22:48.142295+0000","last_peered":"2024-09-16T07:22:48.142295+0000","last_clean":"2024-09-16T07:22:48.142295+0000","last_became_active":"2024-09-16T07:22:48.134665+0000","last_became_peered":"2024-09-16T07:22:48.134665+0000","last_unstale":"2024-09-16T07:22:48.142295+0000","last_undegraded":"2024-09-16T07:22:48.142295+0000","last_fullsized":"2024-09-16T07:22:48.142295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130571+0000","last_change":"2024-09-16T07:22:49.129262+0000","last_active":"2024-09-16T07:22:50.130571+0000","last_peered":"2024-09-16T07:22:50.130571+0000","last_clean":"2024-09-16T07:22:50.130571+0000","last_became_active":"2024-09-16T07:22:49.128955+0000","last_became_peered":"2024-09-16T07:22:49.128955+0000","last_unstale":"2024-09-16T07:22:50.130571+0000","last_undegraded":"2024-09-16T07:22:50.130571+0000","last_fullsized":"2024-09-16T07:22:50.130571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:35.169775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.141837+0000","last_change":"2024-09-16T07:22:48.139495+0000","last_active":"2024-09-16T07:22:48.141837+0000","last_peered":"2024-09-16T07:22:48.141837+0000","last_clean":"2024-09-16T07:22:48.141837+0000","last_became_active":"2024-09-16T07:22:48.139004+0000","last_became_peered":"2024-09-16T07:22:48.139004+0000","last_unstale":"2024-09-16T07:22:48.141837+0000","last_undegraded":"2024-09-16T07:22:48.141837+0000","last_fullsized":"2024-09-16T07:22:48.141837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:57.410149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114809+0000","last_change":"2024-09-16T07:22:48.114809+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114809+0000","last_undegraded":"2024-09-16T07:22:48.114809+0000","last_fullsized":"2024-09-16T07:22:48.114809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.135332+0000","last_change":"2024-09-16T07:22:48.130409+0000","last_active":"2024-09-16T07:22:48.135332+0000","last_peered":"2024-09-16T07:22:48.135332+0000","last_clean":"2024-09-16T07:22:48.135332+0000","last_became_active":"2024-09-16T07:22:48.130159+0000","last_became_peered":"2024-09-16T07:22:48.130159+0000","last_unstale":"2024-09-16T07:22:48.135332+0000","last_undegraded":"2024-09-16T07:22:48.135332+0000","last_fullsized":"2024-09-16T07:22:48.135332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:34.669437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.110294+0000","last_change":"2024-09-16T07:22:47.110294+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.110294+0000","last_undegraded":"2024-09-16T07:22:47.110294+0000","last_fullsized":"2024-09-16T07:22:47.110294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.120521+0000","last_change":"2024-09-16T07:22:48.120521+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.120521+0000","last_undegraded":"2024-09-16T07:22:48.120521+0000","last_fullsized":"2024-09-16T07:22:48.120521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.136821+0000","last_change":"2024-09-16T07:22:48.130827+0000","last_active":"2024-09-16T07:22:48.136821+0000","last_peered":"2024-09-16T07:22:48.136821+0000","last_clean":"2024-09-16T07:22:48.136821+0000","last_became_active":"2024-09-16T07:22:48.129479+0000","last_became_peered":"2024-09-16T07:22:48.129479+0000","last_unstale":"2024-09-16T07:22:48.136821+0000","last_undegraded":"2024-09-16T07:22:48.136821+0000","last_fullsized":"2024-09-16T07:22:48.136821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119466+0000","last_change":"2024-09-16T07:22:48.119466+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119466+0000","last_undegraded":"2024-09-16T07:22:48.119466+0000","last_fullsized":"2024-09-16T07:22:48.119466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.109996+0000","last_change":"2024-09-16T07:22:47.109996+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.109996+0000","last_undegraded":"2024-09-16T07:22:47.109996+0000","last_fullsized":"2024-09-16T07:22:47.109996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142372+0000","last_change":"2024-09-16T07:22:48.137679+0000","last_active":"2024-09-16T07:22:48.142372+0000","last_peered":"2024-09-16T07:22:48.142372+0000","last_clean":"2024-09-16T07:22:48.142372+0000","last_became_active":"2024-09-16T07:22:48.134910+0000","last_became_peered":"2024-09-16T07:22:48.134910+0000","last_unstale":"2024-09-16T07:22:48.142372+0000","last_undegraded":"2024-09-16T07:22:48.142372+0000","last_fullsized":"2024-09-16T07:22:48.142372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.126214+0000","last_change":"2024-09-16T07:22:48.126214+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.126214+0000","last_undegraded":"2024-09-16T07:22:48.126214+0000","last_fullsized":"2024-09-16T07:22:48.126214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.144498+0000","last_change":"2024-09-16T07:22:48.139812+0000","last_active":"2024-09-16T07:22:48.144498+0000","last_peered":"2024-09-16T07:22:48.144498+0000","last_clean":"2024-09-16T07:22:48.144498+0000","last_became_active":"2024-09-16T07:22:48.139509+0000","last_became_peered":"2024-09-16T07:22:48.139509+0000","last_unstale":"2024-09-16T07:22:48.144498+0000","last_undegraded":"2024-09-16T07:22:48.144498+0000","last_fullsized":"2024-09-16T07:22:48.144498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113301+0000","last_change":"2024-09-16T07:22:48.113301+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113301+0000","last_undegraded":"2024-09-16T07:22:48.113301+0000","last_fullsized":"2024-09-16T07:22:48.113301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130773+0000","last_change":"2024-09-16T07:22:48.134633+0000","last_active":"2024-09-16T07:22:50.130773+0000","last_peered":"2024-09-16T07:22:50.130773+0000","last_clean":"2024-09-16T07:22:50.130773+0000","last_became_active":"2024-09-16T07:22:48.134291+0000","last_became_peered":"2024-09-16T07:22:48.134291+0000","last_unstale":"2024-09-16T07:22:50.130773+0000","last_undegraded":"2024-09-16T07:22:50.130773+0000","last_fullsized":"2024-09-16T07:22:50.130773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:35.346741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121798+0000","last_change":"2024-09-16T07:22:48.121798+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121798+0000","last_undegraded":"2024-09-16T07:22:48.121798+0000","last_fullsized":"2024-09-16T07:22:48.121798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.141898+0000","last_change":"2024-09-16T07:22:48.138039+0000","last_active":"2024-09-16T07:22:48.141898+0000","last_peered":"2024-09-16T07:22:48.141898+0000","last_clean":"2024-09-16T07:22:48.141898+0000","last_became_active":"2024-09-16T07:22:48.137746+0000","last_became_peered":"2024-09-16T07:22:48.137746+0000","last_unstale":"2024-09-16T07:22:48.141898+0000","last_undegraded":"2024-09-16T07:22:48.141898+0000","last_fullsized":"2024-09-16T07:22:48.141898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.114410+0000","last_change":"2024-09-16T07:22:48.114410+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.114410+0000","last_undegraded":"2024-09-16T07:22:48.114410+0000","last_fullsized":"2024-09-16T07:22:48.114410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132541+0000","last_change":"2024-09-16T07:22:48.137778+0000","last_active":"2024-09-16T07:22:50.132541+0000","last_peered":"2024-09-16T07:22:50.132541+0000","last_clean":"2024-09-16T07:22:50.132541+0000","last_became_active":"2024-09-16T07:22:48.137159+0000","last_became_peered":"2024-09-16T07:22:48.137159+0000","last_unstale":"2024-09-16T07:22:50.132541+0000","last_undegraded":"2024-09-16T07:22:50.132541+0000","last_fullsized":"2024-09-16T07:22:50.132541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:37.703838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123557+0000","last_change":"2024-09-16T07:22:48.123557+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123557+0000","last_undegraded":"2024-09-16T07:22:48.123557+0000","last_fullsized":"2024-09-16T07:22:48.123557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.104123+0000","last_change":"2024-09-16T07:22:47.104123+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.104123+0000","last_undegraded":"2024-09-16T07:22:47.104123+0000","last_fullsized":"2024-09-16T07:22:47.104123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121319+0000","last_change":"2024-09-16T07:22:48.121319+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121319+0000","last_undegraded":"2024-09-16T07:22:48.121319+0000","last_fullsized":"2024-09-16T07:22:48.121319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108513+0000","last_change":"2024-09-16T07:22:47.108513+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108513+0000","last_undegraded":"2024-09-16T07:22:47.108513+0000","last_fullsized":"2024-09-16T07:22:47.108513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.128400+0000","last_change":"2024-09-16T07:22:48.128400+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.128400+0000","last_undegraded":"2024-09-16T07:22:48.128400+0000","last_fullsized":"2024-09-16T07:22:48.128400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":107,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.111659+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:22:48.111659+0000","last_peered":"2024-09-16T07:22:48.111659+0000","last_clean":"2024-09-16T07:22:48.111659+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:22:48.111659+0000","last_undegraded":"2024-09-16T07:22:48.111659+0000","last_fullsized":"2024-09-16T07:22:48.111659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146419+0000","last_change":"2024-09-16T07:22:48.139675+0000","last_active":"2024-09-16T07:22:48.146419+0000","last_peered":"2024-09-16T07:22:48.146419+0000","last_clean":"2024-09-16T07:22:48.146419+0000","last_became_active":"2024-09-16T07:22:48.137989+0000","last_became_peered":"2024-09-16T07:22:48.137989+0000","last_unstale":"2024-09-16T07:22:48.146419+0000","last_undegraded":"2024-09-16T07:22:48.146419+0000","last_fullsized":"2024-09-16T07:22:48.146419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:08.500032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132139+0000","last_change":"2024-09-16T07:22:49.137102+0000","last_active":"2024-09-16T07:22:50.132139+0000","last_peered":"2024-09-16T07:22:50.132139+0000","last_clean":"2024-09-16T07:22:50.132139+0000","last_became_active":"2024-09-16T07:22:49.136755+0000","last_became_peered":"2024-09-16T07:22:49.136755+0000","last_unstale":"2024-09-16T07:22:50.132139+0000","last_undegraded":"2024-09-16T07:22:50.132139+0000","last_fullsized":"2024-09-16T07:22:50.132139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:26.935160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.139652+0000","last_change":"2024-09-16T07:22:48.133273+0000","last_active":"2024-09-16T07:22:48.139652+0000","last_peered":"2024-09-16T07:22:48.139652+0000","last_clean":"2024-09-16T07:22:48.139652+0000","last_became_active":"2024-09-16T07:22:48.132984+0000","last_became_peered":"2024-09-16T07:22:48.132984+0000","last_unstale":"2024-09-16T07:22:48.139652+0000","last_undegraded":"2024-09-16T07:22:48.139652+0000","last_fullsized":"2024-09-16T07:22:48.139652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:12.007091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.110270+0000","last_change":"2024-09-16T07:22:47.110270+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.110270+0000","last_undegraded":"2024-09-16T07:22:47.110270+0000","last_fullsized":"2024-09-16T07:22:47.110270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.146385+0000","last_change":"2024-09-16T07:22:48.139380+0000","last_active":"2024-09-16T07:22:48.146385+0000","last_peered":"2024-09-16T07:22:48.146385+0000","last_clean":"2024-09-16T07:22:48.146385+0000","last_became_active":"2024-09-16T07:22:48.138052+0000","last_became_peered":"2024-09-16T07:22:48.138052+0000","last_unstale":"2024-09-16T07:22:48.146385+0000","last_undegraded":"2024-09-16T07:22:48.146385+0000","last_fullsized":"2024-09-16T07:22:48.146385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:07.509813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.119932+0000","last_change":"2024-09-16T07:22:48.119932+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.119932+0000","last_undegraded":"2024-09-16T07:22:48.119932+0000","last_fullsized":"2024-09-16T07:22:48.119932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.103481+0000","last_change":"2024-09-16T07:22:47.103481+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.103481+0000","last_undegraded":"2024-09-16T07:22:47.103481+0000","last_fullsized":"2024-09-16T07:22:47.103481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.115959+0000","last_change":"2024-09-16T07:22:48.115959+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.115959+0000","last_undegraded":"2024-09-16T07:22:48.115959+0000","last_fullsized":"2024-09-16T07:22:48.115959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.142306+0000","last_change":"2024-09-16T07:22:48.135598+0000","last_active":"2024-09-16T07:22:48.142306+0000","last_peered":"2024-09-16T07:22:48.142306+0000","last_clean":"2024-09-16T07:22:48.142306+0000","last_became_active":"2024-09-16T07:22:48.135179+0000","last_became_peered":"2024-09-16T07:22:48.135179+0000","last_unstale":"2024-09-16T07:22:48.142306+0000","last_undegraded":"2024-09-16T07:22:48.142306+0000","last_fullsized":"2024-09-16T07:22:48.142306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138704+0000","last_change":"2024-09-16T07:22:48.137476+0000","last_active":"2024-09-16T07:22:50.138704+0000","last_peered":"2024-09-16T07:22:50.138704+0000","last_clean":"2024-09-16T07:22:50.138704+0000","last_became_active":"2024-09-16T07:22:48.136772+0000","last_became_peered":"2024-09-16T07:22:48.136772+0000","last_unstale":"2024-09-16T07:22:50.138704+0000","last_undegraded":"2024-09-16T07:22:50.138704+0000","last_fullsized":"2024-09-16T07:22:50.138704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.906034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132443+0000","last_change":"2024-09-16T07:22:49.144331+0000","last_active":"2024-09-16T07:22:50.132443+0000","last_peered":"2024-09-16T07:22:50.132443+0000","last_clean":"2024-09-16T07:22:50.132443+0000","last_became_active":"2024-09-16T07:22:49.144071+0000","last_became_peered":"2024-09-16T07:22:49.144071+0000","last_unstale":"2024-09-16T07:22:50.132443+0000","last_undegraded":"2024-09-16T07:22:50.132443+0000","last_fullsized":"2024-09-16T07:22:50.132443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.753793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.139583+0000","last_change":"2024-09-16T07:22:48.132847+0000","last_active":"2024-09-16T07:22:48.139583+0000","last_peered":"2024-09-16T07:22:48.139583+0000","last_clean":"2024-09-16T07:22:48.139583+0000","last_became_active":"2024-09-16T07:22:48.132264+0000","last_became_peered":"2024-09-16T07:22:48.132264+0000","last_unstale":"2024-09-16T07:22:48.139583+0000","last_undegraded":"2024-09-16T07:22:48.139583+0000","last_fullsized":"2024-09-16T07:22:48.139583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:52.735822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:49.138949+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:49.138580+0000","last_became_peered":"2024-09-16T07:22:49.138580+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:30.446387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.134073+0000","last_change":"2024-09-16T07:22:48.129412+0000","last_active":"2024-09-16T07:22:48.134073+0000","last_peered":"2024-09-16T07:22:48.134073+0000","last_clean":"2024-09-16T07:22:48.134073+0000","last_became_active":"2024-09-16T07:22:48.129137+0000","last_became_peered":"2024-09-16T07:22:48.129137+0000","last_unstale":"2024-09-16T07:22:48.134073+0000","last_undegraded":"2024-09-16T07:22:48.134073+0000","last_fullsized":"2024-09-16T07:22:48.134073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.121359+0000","last_change":"2024-09-16T07:22:48.121359+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.121359+0000","last_undegraded":"2024-09-16T07:22:48.121359+0000","last_fullsized":"2024-09-16T07:22:48.121359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130669+0000","last_change":"2024-09-16T07:22:48.134437+0000","last_active":"2024-09-16T07:22:50.130669+0000","last_peered":"2024-09-16T07:22:50.130669+0000","last_clean":"2024-09-16T07:22:50.130669+0000","last_became_active":"2024-09-16T07:22:48.134144+0000","last_became_peered":"2024-09-16T07:22:48.134144+0000","last_unstale":"2024-09-16T07:22:50.130669+0000","last_undegraded":"2024-09-16T07:22:50.130669+0000","last_fullsized":"2024-09-16T07:22:50.130669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:04.051560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130660+0000","last_change":"2024-09-16T07:22:49.141869+0000","last_active":"2024-09-16T07:22:50.130660+0000","last_peered":"2024-09-16T07:22:50.130660+0000","last_clean":"2024-09-16T07:22:50.130660+0000","last_became_active":"2024-09-16T07:22:49.141602+0000","last_became_peered":"2024-09-16T07:22:49.141602+0000","last_unstale":"2024-09-16T07:22:50.130660+0000","last_undegraded":"2024-09-16T07:22:50.130660+0000","last_fullsized":"2024-09-16T07:22:50.130660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:44.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.108381+0000","last_change":"2024-09-16T07:22:47.108381+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.108381+0000","last_undegraded":"2024-09-16T07:22:47.108381+0000","last_fullsized":"2024-09-16T07:22:47.108381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.140648+0000","last_change":"2024-09-16T07:22:48.131412+0000","last_active":"2024-09-16T07:22:48.140648+0000","last_peered":"2024-09-16T07:22:48.140648+0000","last_clean":"2024-09-16T07:22:48.140648+0000","last_became_active":"2024-09-16T07:22:48.131167+0000","last_became_peered":"2024-09-16T07:22:48.131167+0000","last_unstale":"2024-09-16T07:22:48.140648+0000","last_undegraded":"2024-09-16T07:22:48.140648+0000","last_fullsized":"2024-09-16T07:22:48.140648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:41:46.947628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.123529+0000","last_change":"2024-09-16T07:22:48.123529+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.123529+0000","last_undegraded":"2024-09-16T07:22:48.123529+0000","last_fullsized":"2024-09-16T07:22:48.123529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.128864+0000","last_change":"2024-09-16T07:22:48.128864+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.128864+0000","last_undegraded":"2024-09-16T07:22:48.128864+0000","last_fullsized":"2024-09-16T07:22:48.128864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.135197+0000","last_change":"2024-09-16T07:22:48.130394+0000","last_active":"2024-09-16T07:22:48.135197+0000","last_peered":"2024-09-16T07:22:48.135197+0000","last_clean":"2024-09-16T07:22:48.135197+0000","last_became_active":"2024-09-16T07:22:48.130013+0000","last_became_peered":"2024-09-16T07:22:48.130013+0000","last_unstale":"2024-09-16T07:22:48.135197+0000","last_undegraded":"2024-09-16T07:22:48.135197+0000","last_fullsized":"2024-09-16T07:22:48.135197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:52.735822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-16T07:22:48.113994+0000","last_change":"2024-09-16T07:22:48.113994+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.113994+0000","last_undegraded":"2024-09-16T07:22:48.113994+0000","last_fullsized":"2024-09-16T07:22:48.113994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T07:22:48.140231+0000","last_change":"2024-09-16T07:22:48.135403+0000","last_active":"2024-09-16T07:22:48.140231+0000","last_peered":"2024-09-16T07:22:48.140231+0000","last_clean":"2024-09-16T07:22:48.140231+0000","last_became_active":"2024-09-16T07:22:48.135142+0000","last_became_peered":"2024-09-16T07:22:48.135142+0000","last_unstale":"2024-09-16T07:22:48.140231+0000","last_undegraded":"2024-09-16T07:22:48.140231+0000","last_fullsized":"2024-09-16T07:22:48.140231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T07:22:47.104041+0000","last_change":"2024-09-16T07:22:47.104041+0000","last_active":"2024-09-16T07:22:47.093938+0000","last_peered":"2024-09-16T07:22:47.093938+0000","last_clean":"2024-09-16T07:22:47.093938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:47.104041+0000","last_undegraded":"2024-09-16T07:22:47.104041+0000","last_fullsized":"2024-09-16T07:22:47.104041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:22:48.099589+0000","last_change":"2024-09-16T07:22:48.099589+0000","last_active":"2024-09-16T07:22:48.099589+0000","last_peered":"2024-09-16T07:22:48.099589+0000","last_clean":"2024-09-16T07:22:48.099589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:22:48.099589+0000","last_undegraded":"2024-09-16T07:22:48.099589+0000","last_fullsized":"2024-09-16T07:22:48.099589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132622+0000","last_change":"2024-09-16T07:22:48.141242+0000","last_active":"2024-09-16T07:22:50.132622+0000","last_peered":"2024-09-16T07:22:50.132622+0000","last_clean":"2024-09-16T07:22:50.132622+0000","last_became_active":"2024-09-16T07:22:48.139871+0000","last_became_peered":"2024-09-16T07:22:48.139871+0000","last_unstale":"2024-09-16T07:22:50.132622+0000","last_undegraded":"2024-09-16T07:22:50.132622+0000","last_fullsized":"2024-09-16T07:22:50.132622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:32.916879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":5},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476742,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":786432,"data_stored":651725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":61357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":786432,"data_stored":651725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:22:54.142 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-16T07:22:54.142 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.123907+0000', 'last_change': '2024-09-16T07:22:48.123907+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.123907+0000', 'last_undegraded': '2024-09-16T07:22:48.123907+0000', 'last_fullsized': '2024-09-16T07:22:48.123907+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.142 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-16T07:22:54.142 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.114075+0000', 'last_change': '2024-09-16T07:22:48.114075+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.114075+0000', 'last_undegraded': '2024-09-16T07:22:48.114075+0000', 'last_fullsized': '2024-09-16T07:22:48.114075+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.119393+0000', 'last_change': '2024-09-16T07:22:48.119393+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.119393+0000', 'last_undegraded': '2024-09-16T07:22:48.119393+0000', 'last_fullsized': '2024-09-16T07:22:48.119393+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.114916+0000', 'last_change': '2024-09-16T07:22:48.114916+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.114916+0000', 'last_undegraded': '2024-09-16T07:22:48.114916+0000', 'last_fullsized': '2024-09-16T07:22:48.114916+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.122187+0000', 'last_change': '2024-09-16T07:22:48.122187+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.122187+0000', 'last_undegraded': '2024-09-16T07:22:48.122187+0000', 'last_fullsized': '2024-09-16T07:22:48.122187+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-16T07:22:54.143 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.115005+0000', 'last_change': '2024-09-16T07:22:48.115005+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.115005+0000', 'last_undegraded': '2024-09-16T07:22:48.115005+0000', 'last_fullsized': '2024-09-16T07:22:48.115005+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.120889+0000', 'last_change': '2024-09-16T07:22:48.120889+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.120889+0000', 'last_undegraded': '2024-09-16T07:22:48.120889+0000', 'last_fullsized': '2024-09-16T07:22:48.120889+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.124292+0000', 'last_change': '2024-09-16T07:22:48.124292+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.124292+0000', 'last_undegraded': '2024-09-16T07:22:48.124292+0000', 'last_fullsized': '2024-09-16T07:22:48.124292+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.122392+0000', 'last_change': '2024-09-16T07:22:48.122392+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.122392+0000', 'last_undegraded': '2024-09-16T07:22:48.122392+0000', 'last_fullsized': '2024-09-16T07:22:48.122392+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-16T07:22:54.144 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.127288+0000', 'last_change': '2024-09-16T07:22:48.127288+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.127288+0000', 'last_undegraded': '2024-09-16T07:22:48.127288+0000', 'last_fullsized': '2024-09-16T07:22:48.127288+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.117320+0000', 'last_change': '2024-09-16T07:22:48.117320+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.117320+0000', 'last_undegraded': '2024-09-16T07:22:48.117320+0000', 'last_fullsized': '2024-09-16T07:22:48.117320+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.145 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.119755+0000', 'last_change': '2024-09-16T07:22:48.119755+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.119755+0000', 'last_undegraded': '2024-09-16T07:22:48.119755+0000', 'last_fullsized': '2024-09-16T07:22:48.119755+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.121828+0000', 'last_change': '2024-09-16T07:22:48.121828+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.121828+0000', 'last_undegraded': '2024-09-16T07:22:48.121828+0000', 'last_fullsized': '2024-09-16T07:22:48.121828+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.114307+0000', 'last_change': '2024-09-16T07:22:48.114307+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.114307+0000', 'last_undegraded': '2024-09-16T07:22:48.114307+0000', 'last_fullsized': '2024-09-16T07:22:48.114307+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.117490+0000', 'last_change': '2024-09-16T07:22:48.117490+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.117490+0000', 'last_undegraded': '2024-09-16T07:22:48.117490+0000', 'last_fullsized': '2024-09-16T07:22:48.117490+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-16T07:22:54.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.115221+0000', 'last_change': '2024-09-16T07:22:48.115221+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.115221+0000', 'last_undegraded': '2024-09-16T07:22:48.115221+0000', 'last_fullsized': '2024-09-16T07:22:48.115221+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-16T07:22:54.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.123978+0000', 'last_change': '2024-09-16T07:22:48.123978+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.123978+0000', 'last_undegraded': '2024-09-16T07:22:48.123978+0000', 'last_fullsized': '2024-09-16T07:22:48.123978+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.115447+0000', 'last_change': '2024-09-16T07:22:48.115447+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.115447+0000', 'last_undegraded': '2024-09-16T07:22:48.115447+0000', 'last_fullsized': '2024-09-16T07:22:48.115447+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.113455+0000', 'last_change': '2024-09-16T07:22:48.113455+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.113455+0000', 'last_undegraded': '2024-09-16T07:22:48.113455+0000', 'last_fullsized': '2024-09-16T07:22:48.113455+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.103688+0000', 'last_change': '2024-09-16T07:22:47.103688+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.103688+0000', 'last_undegraded': '2024-09-16T07:22:47.103688+0000', 'last_fullsized': '2024-09-16T07:22:47.103688+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-16T07:22:54.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.108684+0000', 'last_change': '2024-09-16T07:22:47.108684+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.108684+0000', 'last_undegraded': '2024-09-16T07:22:47.108684+0000', 'last_fullsized': '2024-09-16T07:22:47.108684+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.114809+0000', 'last_change': '2024-09-16T07:22:48.114809+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.114809+0000', 'last_undegraded': '2024-09-16T07:22:48.114809+0000', 'last_fullsized': '2024-09-16T07:22:48.114809+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:PG 2.15 is not active+clean 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.110294+0000', 'last_change': '2024-09-16T07:22:47.110294+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.110294+0000', 'last_undegraded': '2024-09-16T07:22:47.110294+0000', 'last_fullsized': '2024-09-16T07:22:47.110294+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-16T07:22:54.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.120521+0000', 'last_change': '2024-09-16T07:22:48.120521+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.120521+0000', 'last_undegraded': '2024-09-16T07:22:48.120521+0000', 'last_fullsized': '2024-09-16T07:22:48.120521+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.150 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-16T07:22:54.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.119466+0000', 'last_change': '2024-09-16T07:22:48.119466+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.119466+0000', 'last_undegraded': '2024-09-16T07:22:48.119466+0000', 'last_fullsized': '2024-09-16T07:22:48.119466+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.150 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-16T07:22:54.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.109996+0000', 'last_change': '2024-09-16T07:22:47.109996+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.109996+0000', 'last_undegraded': '2024-09-16T07:22:47.109996+0000', 'last_fullsized': '2024-09-16T07:22:47.109996+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:22:54.150 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-16T07:22:54.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.150 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-16T07:22:54.151 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.126214+0000', 'last_change': '2024-09-16T07:22:48.126214+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.126214+0000', 'last_undegraded': '2024-09-16T07:22:48.126214+0000', 'last_fullsized': '2024-09-16T07:22:48.126214+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.151 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-16T07:22:54.151 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.113301+0000', 'last_change': '2024-09-16T07:22:48.113301+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.113301+0000', 'last_undegraded': '2024-09-16T07:22:48.113301+0000', 'last_fullsized': '2024-09-16T07:22:48.113301+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:22:54.151 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-16T07:22:54.151 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.121798+0000', 'last_change': '2024-09-16T07:22:48.121798+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.121798+0000', 'last_undegraded': '2024-09-16T07:22:48.121798+0000', 'last_fullsized': '2024-09-16T07:22:48.121798+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.151 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.114410+0000', 'last_change': '2024-09-16T07:22:48.114410+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.114410+0000', 'last_undegraded': '2024-09-16T07:22:48.114410+0000', 'last_fullsized': '2024-09-16T07:22:48.114410+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.123557+0000', 'last_change': '2024-09-16T07:22:48.123557+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.123557+0000', 'last_undegraded': '2024-09-16T07:22:48.123557+0000', 'last_fullsized': '2024-09-16T07:22:48.123557+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.104123+0000', 'last_change': '2024-09-16T07:22:47.104123+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.104123+0000', 'last_undegraded': '2024-09-16T07:22:47.104123+0000', 'last_fullsized': '2024-09-16T07:22:47.104123+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.121319+0000', 'last_change': '2024-09-16T07:22:48.121319+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.121319+0000', 'last_undegraded': '2024-09-16T07:22:48.121319+0000', 'last_fullsized': '2024-09-16T07:22:48.121319+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.108513+0000', 'last_change': '2024-09-16T07:22:47.108513+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.108513+0000', 'last_undegraded': '2024-09-16T07:22:47.108513+0000', 'last_fullsized': '2024-09-16T07:22:47.108513+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-16T07:22:54.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.128400+0000', 'last_change': '2024-09-16T07:22:48.128400+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.128400+0000', 'last_undegraded': '2024-09-16T07:22:48.128400+0000', 'last_fullsized': '2024-09-16T07:22:48.128400+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.110270+0000', 'last_change': '2024-09-16T07:22:47.110270+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.110270+0000', 'last_undegraded': '2024-09-16T07:22:47.110270+0000', 'last_fullsized': '2024-09-16T07:22:47.110270+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.119932+0000', 'last_change': '2024-09-16T07:22:48.119932+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.119932+0000', 'last_undegraded': '2024-09-16T07:22:48.119932+0000', 'last_fullsized': '2024-09-16T07:22:48.119932+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.103481+0000', 'last_change': '2024-09-16T07:22:47.103481+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.103481+0000', 'last_undegraded': '2024-09-16T07:22:47.103481+0000', 'last_fullsized': '2024-09-16T07:22:47.103481+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-16T07:22:54.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.115959+0000', 'last_change': '2024-09-16T07:22:48.115959+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.115959+0000', 'last_undegraded': '2024-09-16T07:22:48.115959+0000', 'last_fullsized': '2024-09-16T07:22:48.115959+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:22:54.154 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-16T07:22:54.154 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.154 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-16T07:22:54.154 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.121359+0000', 'last_change': '2024-09-16T07:22:48.121359+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.121359+0000', 'last_undegraded': '2024-09-16T07:22:48.121359+0000', 'last_fullsized': '2024-09-16T07:22:48.121359+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.108381+0000', 'last_change': '2024-09-16T07:22:47.108381+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.108381+0000', 'last_undegraded': '2024-09-16T07:22:47.108381+0000', 'last_fullsized': '2024-09-16T07:22:47.108381+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.123529+0000', 'last_change': '2024-09-16T07:22:48.123529+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.123529+0000', 'last_undegraded': '2024-09-16T07:22:48.123529+0000', 'last_fullsized': '2024-09-16T07:22:48.123529+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.128864+0000', 'last_change': '2024-09-16T07:22:48.128864+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.128864+0000', 'last_undegraded': '2024-09-16T07:22:48.128864+0000', 'last_fullsized': '2024-09-16T07:22:48.128864+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-16T07:22:54.155 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:48.113994+0000', 'last_change': '2024-09-16T07:22:48.113994+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.113994+0000', 'last_undegraded': '2024-09-16T07:22:48.113994+0000', 'last_fullsized': '2024-09-16T07:22:48.113994+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:22:54.156 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-16T07:22:54.156 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.156 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-16T07:22:54.156 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:22:47.104041+0000', 'last_change': '2024-09-16T07:22:47.104041+0000', 'last_active': '2024-09-16T07:22:47.093938+0000', 'last_peered': '2024-09-16T07:22:47.093938+0000', 'last_clean': '2024-09-16T07:22:47.093938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:47.104041+0000', 'last_undegraded': '2024-09-16T07:22:47.104041+0000', 'last_fullsized': '2024-09-16T07:22:47.104041+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:47.093938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:22:54.156 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-16T07:22:54.156 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:22:48.099589+0000', 'last_change': '2024-09-16T07:22:48.099589+0000', 'last_active': '2024-09-16T07:22:48.099589+0000', 'last_peered': '2024-09-16T07:22:48.099589+0000', 'last_clean': '2024-09-16T07:22:48.099589+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:22:48.099589+0000', 'last_undegraded': '2024-09-16T07:22:48.099589+0000', 'last_fullsized': '2024-09-16T07:22:48.099589+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'last_clean_scrub_stamp': '2024-09-16T07:22:48.099589+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:22:54.157 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T07:22:54.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.335+0000 7fdad2c00640 1 -- 172.21.15.71:0/875611871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0003c60 msgr2=0x7fdab0004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.335+0000 7fdad2c00640 1 --2- 172.21.15.71:0/875611871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0003c60 0x7fdab0004060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdabc006ea0 tx=0x7fdab0005830 comp rx=0 tx=0).stop 2024-09-16T07:22:54.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.335+0000 7fdad0c00640 1 -- 172.21.15.71:0/875611871 reap_dead start 2024-09-16T07:22:54.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.335+0000 7fdad2c00640 1 -- 172.21.15.71:0/875611871 shutdown_connections 2024-09-16T07:22:54.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.335+0000 7fdad2c00640 1 -- 172.21.15.71:0/875611871 >> 172.21.15.71:0/875611871 conn(0x7fdacc0b18e0 msgr2=0x7fdacc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:54.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.335+0000 7fdad2c00640 1 -- 172.21.15.71:0/875611871 shutdown_connections 2024-09-16T07:22:54.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.335+0000 7fdad2c00640 1 -- 172.21.15.71:0/875611871 wait complete. 2024-09-16T07:22:54.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.336+0000 7fdad2c00640 1 Processor -- start 2024-09-16T07:22:54.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.337+0000 7fdad2c00640 1 -- start start 2024-09-16T07:22:54.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad2c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdab0003c60 0x7fdacc09c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:54.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad2c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc1546d0 0x7fdacc147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad2c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdacc0ad660 0x7fdacc1e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad2c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc0cc190 con 0x7fdacc0ad660 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad2c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc06fb90 con 0x7fdacc1546d0 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad2c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdacc0cc310 con 0x7fdab0003c60 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdacc0ad660 0x7fdacc1e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdacc0ad660 0x7fdacc1e8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34624/0 (socket says 172.21.15.71:34624) 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 -- 172.21.15.71:0/2848156807 learned_addr learned my addr 172.21.15.71:0/2848156807 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdacb400640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc1546d0 0x7fdacc147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdacbe00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdab0003c60 0x7fdacc09c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 -- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdab0003c60 msgr2=0x7fdacc09c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdab0003c60 0x7fdacc09c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 -- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc1546d0 msgr2=0x7fdacc147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc1546d0 0x7fdacc147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 -- 172.21.15.71:0/2848156807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc09c7b0 con 0x7fdacc0ad660 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdacbe00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdab0003c60 0x7fdacc09c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdacb400640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc1546d0 0x7fdacc147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.338+0000 7fdad0c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdacc0ad660 0x7fdacc1e8810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0af550 tx=0x7fdac4010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.339+0000 7fdac3e00640 1 -- 172.21.15.71:0/2848156807 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4003970 con 0x7fdacc0ad660 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.339+0000 7fdac3e00640 1 -- 172.21.15.71:0/2848156807 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdac4015070 con 0x7fdacc0ad660 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.339+0000 7fdac3e00640 1 -- 172.21.15.71:0/2848156807 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4004030 con 0x7fdacc0ad660 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.339+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc09ca30 con 0x7fdacc0ad660 2024-09-16T07:22:54.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.339+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc0ace60 con 0x7fdacc0ad660 2024-09-16T07:22:54.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.340+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdacc0cc8a0 con 0x7fdacc0ad660 2024-09-16T07:22:54.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.341+0000 7fdac3e00640 1 -- 172.21.15.71:0/2848156807 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fdac40041d0 con 0x7fdacc0ad660 2024-09-16T07:22:54.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.342+0000 7fdac3e00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdaac078850 0x7fdaac07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:54.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.342+0000 7fdac3e00640 1 -- 172.21.15.71:0/2848156807 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdac409b0d0 con 0x7fdacc0ad660 2024-09-16T07:22:54.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.342+0000 7fdacbe00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdaac078850 0x7fdaac07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:54.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.343+0000 7fdacbe00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdaac078850 0x7fdaac07ad10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdabc033000 tx=0x7fdabc001850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:54.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.346+0000 7fdac3e00640 1 -- 172.21.15.71:0/2848156807 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdacc09c7b0 con 0x7fdacc0ad660 2024-09-16T07:22:54.358 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-09-16T07:22:54.358+0000 7f6fe6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:22:54.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.602+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fdacc15fee0 con 0x7fdacc0ad660 2024-09-16T07:22:54.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.603+0000 7fdac3e00640 1 -- 172.21.15.71:0/2848156807 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1214 (secure 0 0 0) 0x7fdac4063730 con 0x7fdacc0ad660 2024-09-16T07:22:54.603 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:54.603 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1726471357,"num_in_osds":8,"osd_in_since":1726471347,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":45},{"state_name":"active+clean","count":30},{"state_name":"unknown","count":22}],"num_pgs":97,"num_pools":3,"num_objects":10,"data_bytes":590950,"bytes_used":228470784,"bytes_avail":772865642496,"bytes_total":773094113280,"unknown_pgs_ratio":0.22680412232875824,"inactive_pgs_ratio":0.46391752362251282,"write_bytes_sec":197,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4404}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-16T07:22:23.113911+0000","services":{}},"progress_events":{}} 2024-09-16T07:22:54.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdaac078850 msgr2=0x7fdaac07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdaac078850 0x7fdaac07ad10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdabc033000 tx=0x7fdabc001850 comp rx=0 tx=0).stop 2024-09-16T07:22:54.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdacc0ad660 msgr2=0x7fdacc1e8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdacc0ad660 0x7fdacc1e8810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0af550 tx=0x7fdac4010f40 comp rx=0 tx=0).stop 2024-09-16T07:22:54.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 shutdown_connections 2024-09-16T07:22:54.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdaac078850 0x7fdaac07ad10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdacc0ad660 0x7fdacc1e8810 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc1546d0 0x7fdacc147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 --2- 172.21.15.71:0/2848156807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdab0003c60 0x7fdacc09c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 >> 172.21.15.71:0/2848156807 conn(0x7fdacc0b18e0 msgr2=0x7fdacc15c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:54.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 shutdown_connections 2024-09-16T07:22:54.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.606+0000 7fdad2c00640 1 -- 172.21.15.71:0/2848156807 wait complete. 2024-09-16T07:22:54.622 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T07:22:54.622 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:22:54.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.778+0000 7f1aa8c00640 1 -- 172.21.15.71:0/47345641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 msgr2=0x7f1aa40b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.778+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/47345641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 0x7f1aa40b5370 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40c7470 tx=0x7f1a9402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.778+0000 7f1aa8c00640 1 -- 172.21.15.71:0/47345641 shutdown_connections 2024-09-16T07:22:54.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.778+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/47345641 >> v2:172.21.15.177:3300/0 conn(0x7f1aa40b34b0 0x7f1aa40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.778+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/47345641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 0x7f1aa40b5370 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.778+0000 7f1aa8c00640 1 -- 172.21.15.71:0/47345641 >> 172.21.15.71:0/47345641 conn(0x7f1aa40d3df0 msgr2=0x7f1aa40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:54.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.779+0000 7f1aa8c00640 1 -- 172.21.15.71:0/47345641 shutdown_connections 2024-09-16T07:22:54.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.779+0000 7f1aa8c00640 1 -- 172.21.15.71:0/47345641 wait complete. 2024-09-16T07:22:54.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.780+0000 7f1aa8c00640 1 Processor -- start 2024-09-16T07:22:54.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.780+0000 7f1aa8c00640 1 -- start start 2024-09-16T07:22:54.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa8c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1aa40b34b0 0x7f1aa40b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:54.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa8c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa40b3e80 0x7f1aa41562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:54.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 0x7f1aa40c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:54.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa8c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa40b2900 con 0x7f1aa4161440 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa8c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa40b2780 con 0x7f1aa40b3e80 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa8c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa40b2a80 con 0x7f1aa40b34b0 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 0x7f1aa40c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 0x7f1aa40c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50524/0 (socket says 172.21.15.71:50524) 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa3e00640 1 -- 172.21.15.71:0/1211552715 learned_addr learned my addr 172.21.15.71:0/1211552715 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.781+0000 7f1aa2a00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa40b3e80 0x7f1aa41562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa3400640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1aa40b34b0 0x7f1aa40b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa3e00640 1 -- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1aa40b34b0 msgr2=0x7f1aa40b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa3e00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1aa40b34b0 0x7f1aa40b96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa3e00640 1 -- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa40b3e80 msgr2=0x7f1aa41562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa3e00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa40b3e80 0x7f1aa41562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa3e00640 1 -- 172.21.15.71:0/1211552715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa40ce200 con 0x7f1aa4161440 2024-09-16T07:22:54.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa3e00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 0x7f1aa40c9d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00c780 tx=0x7f1a9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:54.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1a9b400640 1 -- 172.21.15.71:0/1211552715 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c004b60 con 0x7f1aa4161440 2024-09-16T07:22:54.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1a9b400640 1 -- 172.21.15.71:0/1211552715 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a9c004df0 con 0x7f1aa4161440 2024-09-16T07:22:54.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1a9b400640 1 -- 172.21.15.71:0/1211552715 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00aa70 con 0x7f1aa4161440 2024-09-16T07:22:54.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa40c83d0 con 0x7f1aa4161440 2024-09-16T07:22:54.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.782+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa40c8990 con 0x7f1aa4161440 2024-09-16T07:22:54.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.784+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa40b3940 con 0x7f1aa4161440 2024-09-16T07:22:54.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.784+0000 7f1a9b400640 1 -- 172.21.15.71:0/1211552715 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1a9c00ac10 con 0x7f1aa4161440 2024-09-16T07:22:54.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.785+0000 7f1a9b400640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a84078850 0x7f1a8407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:54.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.786+0000 7f1a9b400640 1 -- 172.21.15.71:0/1211552715 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1a9c09ab30 con 0x7f1aa4161440 2024-09-16T07:22:54.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.786+0000 7f1aa3400640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a84078850 0x7f1a8407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:54.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.787+0000 7f1aa3400640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a84078850 0x7f1a8407ad10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1a94033000 tx=0x7f1a94004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:54.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.789+0000 7f1a9b400640 1 -- 172.21.15.71:0/1211552715 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1a9c020070 con 0x7f1aa4161440 2024-09-16T07:22:54.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.972+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1aa40b5320 con 0x7f1a84078850 2024-09-16T07:22:54.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.978+0000 7f1a9b400640 1 -- 172.21.15.71:0/1211552715 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f1aa40b5320 con 0x7f1a84078850 2024-09-16T07:22:54.981 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:22:54.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a84078850 msgr2=0x7f1a8407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a84078850 0x7f1a8407ad10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1a94033000 tx=0x7f1a94004730 comp rx=0 tx=0).stop 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 msgr2=0x7f1aa40c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 0x7f1aa40c9d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00c780 tx=0x7f1a9c00cc40 comp rx=0 tx=0).stop 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 shutdown_connections 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a84078850 0x7f1a8407ad10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa4161440 0x7f1aa40c9d10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa40b3e80 0x7f1aa41562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 --2- 172.21.15.71:0/1211552715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1aa40b34b0 0x7f1aa40b96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 >> 172.21.15.71:0/1211552715 conn(0x7f1aa40d3df0 msgr2=0x7f1aa40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 shutdown_connections 2024-09-16T07:22:54.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:54.982+0000 7f1aa8c00640 1 -- 172.21.15.71:0/1211552715 wait complete. 2024-09-16T07:22:54.997 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-16T07:22:54.294670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223448,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751272,"statfs":{"total":773094113280,"available":772865302528,"internally_reserved":0,"allocated":3088384,"data_stored":1831212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.179712"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139414+0000","last_change":"2024-09-16T07:22:49.144943+0000","last_active":"2024-09-16T07:22:50.139414+0000","last_peered":"2024-09-16T07:22:50.139414+0000","last_clean":"2024-09-16T07:22:50.139414+0000","last_became_active":"2024-09-16T07:22:49.144609+0000","last_became_peered":"2024-09-16T07:22:49.144609+0000","last_unstale":"2024-09-16T07:22:50.139414+0000","last_undegraded":"2024-09-16T07:22:50.139414+0000","last_fullsized":"2024-09-16T07:22:50.139414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:21.053710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132260+0000","last_change":"2024-09-16T07:22:49.142860+0000","last_active":"2024-09-16T07:22:50.132260+0000","last_peered":"2024-09-16T07:22:50.132260+0000","last_clean":"2024-09-16T07:22:50.132260+0000","last_became_active":"2024-09-16T07:22:49.142555+0000","last_became_peered":"2024-09-16T07:22:49.142555+0000","last_unstale":"2024-09-16T07:22:50.132260+0000","last_undegraded":"2024-09-16T07:22:50.132260+0000","last_fullsized":"2024-09-16T07:22:50.132260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:20.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130865+0000","last_change":"2024-09-16T07:22:49.141351+0000","last_active":"2024-09-16T07:22:50.130865+0000","last_peered":"2024-09-16T07:22:50.130865+0000","last_clean":"2024-09-16T07:22:50.130865+0000","last_became_active":"2024-09-16T07:22:49.141078+0000","last_became_peered":"2024-09-16T07:22:49.141078+0000","last_unstale":"2024-09-16T07:22:50.130865+0000","last_undegraded":"2024-09-16T07:22:50.130865+0000","last_fullsized":"2024-09-16T07:22:50.130865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:54.707093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132009+0000","last_change":"2024-09-16T07:22:49.130699+0000","last_active":"2024-09-16T07:22:50.132009+0000","last_peered":"2024-09-16T07:22:50.132009+0000","last_clean":"2024-09-16T07:22:50.132009+0000","last_became_active":"2024-09-16T07:22:49.130435+0000","last_became_peered":"2024-09-16T07:22:49.130435+0000","last_unstale":"2024-09-16T07:22:50.132009+0000","last_undegraded":"2024-09-16T07:22:50.132009+0000","last_fullsized":"2024-09-16T07:22:50.132009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:37.684357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139429+0000","last_change":"2024-09-16T07:22:49.139132+0000","last_active":"2024-09-16T07:22:50.139429+0000","last_peered":"2024-09-16T07:22:50.139429+0000","last_clean":"2024-09-16T07:22:50.139429+0000","last_became_active":"2024-09-16T07:22:49.134577+0000","last_became_peered":"2024-09-16T07:22:49.134577+0000","last_unstale":"2024-09-16T07:22:50.139429+0000","last_undegraded":"2024-09-16T07:22:50.139429+0000","last_fullsized":"2024-09-16T07:22:50.139429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:28.720254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131882+0000","last_change":"2024-09-16T07:22:49.130577+0000","last_active":"2024-09-16T07:22:50.131882+0000","last_peered":"2024-09-16T07:22:50.131882+0000","last_clean":"2024-09-16T07:22:50.131882+0000","last_became_active":"2024-09-16T07:22:49.130338+0000","last_became_peered":"2024-09-16T07:22:49.130338+0000","last_unstale":"2024-09-16T07:22:50.131882+0000","last_undegraded":"2024-09-16T07:22:50.131882+0000","last_fullsized":"2024-09-16T07:22:50.131882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139192+0000","last_change":"2024-09-16T07:22:49.142202+0000","last_active":"2024-09-16T07:22:50.139192+0000","last_peered":"2024-09-16T07:22:50.139192+0000","last_clean":"2024-09-16T07:22:50.139192+0000","last_became_active":"2024-09-16T07:22:49.141956+0000","last_became_peered":"2024-09-16T07:22:49.141956+0000","last_unstale":"2024-09-16T07:22:50.139192+0000","last_undegraded":"2024-09-16T07:22:50.139192+0000","last_fullsized":"2024-09-16T07:22:50.139192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:35.827940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130543+0000","last_change":"2024-09-16T07:22:49.130201+0000","last_active":"2024-09-16T07:22:50.130543+0000","last_peered":"2024-09-16T07:22:50.130543+0000","last_clean":"2024-09-16T07:22:50.130543+0000","last_became_active":"2024-09-16T07:22:49.129912+0000","last_became_peered":"2024-09-16T07:22:49.129912+0000","last_unstale":"2024-09-16T07:22:50.130543+0000","last_undegraded":"2024-09-16T07:22:50.130543+0000","last_fullsized":"2024-09-16T07:22:50.130543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:21.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279771+0000","last_change":"2024-09-16T07:22:49.141365+0000","last_active":"2024-09-16T07:22:50.279771+0000","last_peered":"2024-09-16T07:22:50.279771+0000","last_clean":"2024-09-16T07:22:50.279771+0000","last_became_active":"2024-09-16T07:22:49.141041+0000","last_became_peered":"2024-09-16T07:22:49.141041+0000","last_unstale":"2024-09-16T07:22:50.279771+0000","last_undegraded":"2024-09-16T07:22:50.279771+0000","last_fullsized":"2024-09-16T07:22:50.279771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:37.684357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139441+0000","last_change":"2024-09-16T07:22:49.142086+0000","last_active":"2024-09-16T07:22:50.139441+0000","last_peered":"2024-09-16T07:22:50.139441+0000","last_clean":"2024-09-16T07:22:50.139441+0000","last_became_active":"2024-09-16T07:22:49.141832+0000","last_became_peered":"2024-09-16T07:22:49.141832+0000","last_unstale":"2024-09-16T07:22:50.139441+0000","last_undegraded":"2024-09-16T07:22:50.139441+0000","last_fullsized":"2024-09-16T07:22:50.139441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:05.967417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139585+0000","last_change":"2024-09-16T07:22:49.144692+0000","last_active":"2024-09-16T07:22:50.139585+0000","last_peered":"2024-09-16T07:22:50.139585+0000","last_clean":"2024-09-16T07:22:50.139585+0000","last_became_active":"2024-09-16T07:22:49.144396+0000","last_became_peered":"2024-09-16T07:22:49.144396+0000","last_unstale":"2024-09-16T07:22:50.139585+0000","last_undegraded":"2024-09-16T07:22:50.139585+0000","last_fullsized":"2024-09-16T07:22:50.139585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:47.741254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139742+0000","last_change":"2024-09-16T07:22:49.130387+0000","last_active":"2024-09-16T07:22:50.139742+0000","last_peered":"2024-09-16T07:22:50.139742+0000","last_clean":"2024-09-16T07:22:50.139742+0000","last_became_active":"2024-09-16T07:22:49.129994+0000","last_became_peered":"2024-09-16T07:22:49.129994+0000","last_unstale":"2024-09-16T07:22:50.139742+0000","last_undegraded":"2024-09-16T07:22:50.139742+0000","last_fullsized":"2024-09-16T07:22:50.139742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:11.870228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279379+0000","last_change":"2024-09-16T07:22:49.133371+0000","last_active":"2024-09-16T07:22:50.279379+0000","last_peered":"2024-09-16T07:22:50.279379+0000","last_clean":"2024-09-16T07:22:50.279379+0000","last_became_active":"2024-09-16T07:22:49.133088+0000","last_became_peered":"2024-09-16T07:22:49.133088+0000","last_unstale":"2024-09-16T07:22:50.279379+0000","last_undegraded":"2024-09-16T07:22:50.279379+0000","last_fullsized":"2024-09-16T07:22:50.279379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:17.196275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279291+0000","last_change":"2024-09-16T07:22:49.140290+0000","last_active":"2024-09-16T07:22:50.279291+0000","last_peered":"2024-09-16T07:22:50.279291+0000","last_clean":"2024-09-16T07:22:50.279291+0000","last_became_active":"2024-09-16T07:22:49.139999+0000","last_became_peered":"2024-09-16T07:22:49.139999+0000","last_unstale":"2024-09-16T07:22:50.279291+0000","last_undegraded":"2024-09-16T07:22:50.279291+0000","last_fullsized":"2024-09-16T07:22:50.279291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131078+0000","last_change":"2024-09-16T07:22:49.131493+0000","last_active":"2024-09-16T07:22:50.131078+0000","last_peered":"2024-09-16T07:22:50.131078+0000","last_clean":"2024-09-16T07:22:50.131078+0000","last_became_active":"2024-09-16T07:22:49.131241+0000","last_became_peered":"2024-09-16T07:22:49.131241+0000","last_unstale":"2024-09-16T07:22:50.131078+0000","last_undegraded":"2024-09-16T07:22:50.131078+0000","last_fullsized":"2024-09-16T07:22:50.131078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:24.470314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278778+0000","last_change":"2024-09-16T07:22:49.126099+0000","last_active":"2024-09-16T07:22:50.278778+0000","last_peered":"2024-09-16T07:22:50.278778+0000","last_clean":"2024-09-16T07:22:50.278778+0000","last_became_active":"2024-09-16T07:22:49.125826+0000","last_became_peered":"2024-09-16T07:22:49.125826+0000","last_unstale":"2024-09-16T07:22:50.278778+0000","last_undegraded":"2024-09-16T07:22:50.278778+0000","last_fullsized":"2024-09-16T07:22:50.278778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:41.565481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280232+0000","last_change":"2024-09-16T07:22:49.132617+0000","last_active":"2024-09-16T07:22:50.280232+0000","last_peered":"2024-09-16T07:22:50.280232+0000","last_clean":"2024-09-16T07:22:50.280232+0000","last_became_active":"2024-09-16T07:22:49.132278+0000","last_became_peered":"2024-09-16T07:22:49.132278+0000","last_unstale":"2024-09-16T07:22:50.280232+0000","last_undegraded":"2024-09-16T07:22:50.280232+0000","last_fullsized":"2024-09-16T07:22:50.280232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:10.129478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280258+0000","last_change":"2024-09-16T07:22:49.143862+0000","last_active":"2024-09-16T07:22:50.280258+0000","last_peered":"2024-09-16T07:22:50.280258+0000","last_clean":"2024-09-16T07:22:50.280258+0000","last_became_active":"2024-09-16T07:22:49.143457+0000","last_became_peered":"2024-09-16T07:22:49.143457+0000","last_unstale":"2024-09-16T07:22:50.280258+0000","last_undegraded":"2024-09-16T07:22:50.280258+0000","last_fullsized":"2024-09-16T07:22:50.280258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279691+0000","last_change":"2024-09-16T07:22:49.130309+0000","last_active":"2024-09-16T07:22:50.279691+0000","last_peered":"2024-09-16T07:22:50.279691+0000","last_clean":"2024-09-16T07:22:50.279691+0000","last_became_active":"2024-09-16T07:22:49.130069+0000","last_became_peered":"2024-09-16T07:22:49.130069+0000","last_unstale":"2024-09-16T07:22:50.279691+0000","last_undegraded":"2024-09-16T07:22:50.279691+0000","last_fullsized":"2024-09-16T07:22:50.279691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:20.784563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139554+0000","last_change":"2024-09-16T07:22:49.129763+0000","last_active":"2024-09-16T07:22:50.139554+0000","last_peered":"2024-09-16T07:22:50.139554+0000","last_clean":"2024-09-16T07:22:50.139554+0000","last_became_active":"2024-09-16T07:22:49.129067+0000","last_became_peered":"2024-09-16T07:22:49.129067+0000","last_unstale":"2024-09-16T07:22:50.139554+0000","last_undegraded":"2024-09-16T07:22:50.139554+0000","last_fullsized":"2024-09-16T07:22:50.139554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:24.919949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132192+0000","last_change":"2024-09-16T07:22:49.133568+0000","last_active":"2024-09-16T07:22:50.132192+0000","last_peered":"2024-09-16T07:22:50.132192+0000","last_clean":"2024-09-16T07:22:50.132192+0000","last_became_active":"2024-09-16T07:22:49.133323+0000","last_became_peered":"2024-09-16T07:22:49.133323+0000","last_unstale":"2024-09-16T07:22:50.132192+0000","last_undegraded":"2024-09-16T07:22:50.132192+0000","last_fullsized":"2024-09-16T07:22:50.132192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:45.369698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279803+0000","last_change":"2024-09-16T07:22:49.137697+0000","last_active":"2024-09-16T07:22:50.279803+0000","last_peered":"2024-09-16T07:22:50.279803+0000","last_clean":"2024-09-16T07:22:50.279803+0000","last_became_active":"2024-09-16T07:22:49.137443+0000","last_became_peered":"2024-09-16T07:22:49.137443+0000","last_unstale":"2024-09-16T07:22:50.279803+0000","last_undegraded":"2024-09-16T07:22:50.279803+0000","last_fullsized":"2024-09-16T07:22:50.279803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130836+0000","last_change":"2024-09-16T07:22:49.139909+0000","last_active":"2024-09-16T07:22:50.130836+0000","last_peered":"2024-09-16T07:22:50.130836+0000","last_clean":"2024-09-16T07:22:50.130836+0000","last_became_active":"2024-09-16T07:22:49.139628+0000","last_became_peered":"2024-09-16T07:22:49.139628+0000","last_unstale":"2024-09-16T07:22:50.130836+0000","last_undegraded":"2024-09-16T07:22:50.130836+0000","last_fullsized":"2024-09-16T07:22:50.130836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:02.467243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278810+0000","last_change":"2024-09-16T07:22:49.134680+0000","last_active":"2024-09-16T07:22:50.278810+0000","last_peered":"2024-09-16T07:22:50.278810+0000","last_clean":"2024-09-16T07:22:50.278810+0000","last_became_active":"2024-09-16T07:22:49.134374+0000","last_became_peered":"2024-09-16T07:22:49.134374+0000","last_unstale":"2024-09-16T07:22:50.278810+0000","last_undegraded":"2024-09-16T07:22:50.278810+0000","last_fullsized":"2024-09-16T07:22:50.278810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:16.837921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132879+0000","last_change":"2024-09-16T07:22:49.133787+0000","last_active":"2024-09-16T07:22:50.132879+0000","last_peered":"2024-09-16T07:22:50.132879+0000","last_clean":"2024-09-16T07:22:50.132879+0000","last_became_active":"2024-09-16T07:22:49.133552+0000","last_became_peered":"2024-09-16T07:22:49.133552+0000","last_unstale":"2024-09-16T07:22:50.132879+0000","last_undegraded":"2024-09-16T07:22:50.132879+0000","last_fullsized":"2024-09-16T07:22:50.132879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.930728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132323+0000","last_change":"2024-09-16T07:22:49.135809+0000","last_active":"2024-09-16T07:22:50.132323+0000","last_peered":"2024-09-16T07:22:50.132323+0000","last_clean":"2024-09-16T07:22:50.132323+0000","last_became_active":"2024-09-16T07:22:49.135527+0000","last_became_peered":"2024-09-16T07:22:49.135527+0000","last_unstale":"2024-09-16T07:22:50.132323+0000","last_undegraded":"2024-09-16T07:22:50.132323+0000","last_fullsized":"2024-09-16T07:22:50.132323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279729+0000","last_change":"2024-09-16T07:22:49.140506+0000","last_active":"2024-09-16T07:22:50.279729+0000","last_peered":"2024-09-16T07:22:50.279729+0000","last_clean":"2024-09-16T07:22:50.279729+0000","last_became_active":"2024-09-16T07:22:49.140006+0000","last_became_peered":"2024-09-16T07:22:49.140006+0000","last_unstale":"2024-09-16T07:22:50.279729+0000","last_undegraded":"2024-09-16T07:22:50.279729+0000","last_fullsized":"2024-09-16T07:22:50.279729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:29.077071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279953+0000","last_change":"2024-09-16T07:22:49.136459+0000","last_active":"2024-09-16T07:22:50.279953+0000","last_peered":"2024-09-16T07:22:50.279953+0000","last_clean":"2024-09-16T07:22:50.279953+0000","last_became_active":"2024-09-16T07:22:49.136183+0000","last_became_peered":"2024-09-16T07:22:49.136183+0000","last_unstale":"2024-09-16T07:22:50.279953+0000","last_undegraded":"2024-09-16T07:22:50.279953+0000","last_fullsized":"2024-09-16T07:22:50.279953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:33.154370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130793+0000","last_change":"2024-09-16T07:22:49.136171+0000","last_active":"2024-09-16T07:22:50.130793+0000","last_peered":"2024-09-16T07:22:50.130793+0000","last_clean":"2024-09-16T07:22:50.130793+0000","last_became_active":"2024-09-16T07:22:49.135900+0000","last_became_peered":"2024-09-16T07:22:49.135900+0000","last_unstale":"2024-09-16T07:22:50.130793+0000","last_undegraded":"2024-09-16T07:22:50.130793+0000","last_fullsized":"2024-09-16T07:22:50.130793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:01.273769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279131+0000","last_change":"2024-09-16T07:22:49.133629+0000","last_active":"2024-09-16T07:22:50.279131+0000","last_peered":"2024-09-16T07:22:50.279131+0000","last_clean":"2024-09-16T07:22:50.279131+0000","last_became_active":"2024-09-16T07:22:49.133394+0000","last_became_peered":"2024-09-16T07:22:49.133394+0000","last_unstale":"2024-09-16T07:22:50.279131+0000","last_undegraded":"2024-09-16T07:22:50.279131+0000","last_fullsized":"2024-09-16T07:22:50.279131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:41.976217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279399+0000","last_change":"2024-09-16T07:22:49.144945+0000","last_active":"2024-09-16T07:22:50.279399+0000","last_peered":"2024-09-16T07:22:50.279399+0000","last_clean":"2024-09-16T07:22:50.279399+0000","last_became_active":"2024-09-16T07:22:49.144649+0000","last_became_peered":"2024-09-16T07:22:49.144649+0000","last_unstale":"2024-09-16T07:22:50.279399+0000","last_undegraded":"2024-09-16T07:22:50.279399+0000","last_fullsized":"2024-09-16T07:22:50.279399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:59.994632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131706+0000","last_change":"2024-09-16T07:22:49.136574+0000","last_active":"2024-09-16T07:22:50.131706+0000","last_peered":"2024-09-16T07:22:50.131706+0000","last_clean":"2024-09-16T07:22:50.131706+0000","last_became_active":"2024-09-16T07:22:49.136412+0000","last_became_peered":"2024-09-16T07:22:49.136412+0000","last_unstale":"2024-09-16T07:22:50.131706+0000","last_undegraded":"2024-09-16T07:22:50.131706+0000","last_fullsized":"2024-09-16T07:22:50.131706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280014+0000","last_change":"2024-09-16T07:22:49.141959+0000","last_active":"2024-09-16T07:22:50.280014+0000","last_peered":"2024-09-16T07:22:50.280014+0000","last_clean":"2024-09-16T07:22:50.280014+0000","last_became_active":"2024-09-16T07:22:49.140661+0000","last_became_peered":"2024-09-16T07:22:49.140661+0000","last_unstale":"2024-09-16T07:22:50.280014+0000","last_undegraded":"2024-09-16T07:22:50.280014+0000","last_fullsized":"2024-09-16T07:22:50.280014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130863+0000","last_change":"2024-09-16T07:22:48.137796+0000","last_active":"2024-09-16T07:22:50.130863+0000","last_peered":"2024-09-16T07:22:50.130863+0000","last_clean":"2024-09-16T07:22:50.130863+0000","last_became_active":"2024-09-16T07:22:48.136473+0000","last_became_peered":"2024-09-16T07:22:48.136473+0000","last_unstale":"2024-09-16T07:22:50.130863+0000","last_undegraded":"2024-09-16T07:22:50.130863+0000","last_fullsized":"2024-09-16T07:22:50.130863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:16.721715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132215+0000","last_change":"2024-09-16T07:22:49.132936+0000","last_active":"2024-09-16T07:22:50.132215+0000","last_peered":"2024-09-16T07:22:50.132215+0000","last_clean":"2024-09-16T07:22:50.132215+0000","last_became_active":"2024-09-16T07:22:49.132677+0000","last_became_peered":"2024-09-16T07:22:49.132677+0000","last_unstale":"2024-09-16T07:22:50.132215+0000","last_undegraded":"2024-09-16T07:22:50.132215+0000","last_fullsized":"2024-09-16T07:22:50.132215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:16.641320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279272+0000","last_change":"2024-09-16T07:22:48.140563+0000","last_active":"2024-09-16T07:22:50.279272+0000","last_peered":"2024-09-16T07:22:50.279272+0000","last_clean":"2024-09-16T07:22:50.279272+0000","last_became_active":"2024-09-16T07:22:48.139864+0000","last_became_peered":"2024-09-16T07:22:48.139864+0000","last_unstale":"2024-09-16T07:22:50.279272+0000","last_undegraded":"2024-09-16T07:22:50.279272+0000","last_fullsized":"2024-09-16T07:22:50.279272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:36.678707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279310+0000","last_change":"2024-09-16T07:22:48.135463+0000","last_active":"2024-09-16T07:22:50.279310+0000","last_peered":"2024-09-16T07:22:50.279310+0000","last_clean":"2024-09-16T07:22:50.279310+0000","last_became_active":"2024-09-16T07:22:48.135133+0000","last_became_peered":"2024-09-16T07:22:48.135133+0000","last_unstale":"2024-09-16T07:22:50.279310+0000","last_undegraded":"2024-09-16T07:22:50.279310+0000","last_fullsized":"2024-09-16T07:22:50.279310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130763+0000","last_change":"2024-09-16T07:22:49.138488+0000","last_active":"2024-09-16T07:22:50.130763+0000","last_peered":"2024-09-16T07:22:50.130763+0000","last_clean":"2024-09-16T07:22:50.130763+0000","last_became_active":"2024-09-16T07:22:49.138042+0000","last_became_peered":"2024-09-16T07:22:49.138042+0000","last_unstale":"2024-09-16T07:22:50.130763+0000","last_undegraded":"2024-09-16T07:22:50.130763+0000","last_fullsized":"2024-09-16T07:22:50.130763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:20.696268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279074+0000","last_change":"2024-09-16T07:22:48.135129+0000","last_active":"2024-09-16T07:22:50.279074+0000","last_peered":"2024-09-16T07:22:50.279074+0000","last_clean":"2024-09-16T07:22:50.279074+0000","last_became_active":"2024-09-16T07:22:48.134665+0000","last_became_peered":"2024-09-16T07:22:48.134665+0000","last_unstale":"2024-09-16T07:22:50.279074+0000","last_undegraded":"2024-09-16T07:22:50.279074+0000","last_fullsized":"2024-09-16T07:22:50.279074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:58.988982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130571+0000","last_change":"2024-09-16T07:22:49.129262+0000","last_active":"2024-09-16T07:22:50.130571+0000","last_peered":"2024-09-16T07:22:50.130571+0000","last_clean":"2024-09-16T07:22:50.130571+0000","last_became_active":"2024-09-16T07:22:49.128955+0000","last_became_peered":"2024-09-16T07:22:49.128955+0000","last_unstale":"2024-09-16T07:22:50.130571+0000","last_undegraded":"2024-09-16T07:22:50.130571+0000","last_fullsized":"2024-09-16T07:22:50.130571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:35.169775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279936+0000","last_change":"2024-09-16T07:22:48.139495+0000","last_active":"2024-09-16T07:22:50.279936+0000","last_peered":"2024-09-16T07:22:50.279936+0000","last_clean":"2024-09-16T07:22:50.279936+0000","last_became_active":"2024-09-16T07:22:48.139004+0000","last_became_peered":"2024-09-16T07:22:48.139004+0000","last_unstale":"2024-09-16T07:22:50.279936+0000","last_undegraded":"2024-09-16T07:22:50.279936+0000","last_fullsized":"2024-09-16T07:22:50.279936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:22.689639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132362+0000","last_change":"2024-09-16T07:22:49.134719+0000","last_active":"2024-09-16T07:22:50.132362+0000","last_peered":"2024-09-16T07:22:50.132362+0000","last_clean":"2024-09-16T07:22:50.132362+0000","last_became_active":"2024-09-16T07:22:49.134471+0000","last_became_peered":"2024-09-16T07:22:49.134471+0000","last_unstale":"2024-09-16T07:22:50.132362+0000","last_undegraded":"2024-09-16T07:22:50.132362+0000","last_fullsized":"2024-09-16T07:22:50.132362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:15.125396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139537+0000","last_change":"2024-09-16T07:22:48.130409+0000","last_active":"2024-09-16T07:22:50.139537+0000","last_peered":"2024-09-16T07:22:50.139537+0000","last_clean":"2024-09-16T07:22:50.139537+0000","last_became_active":"2024-09-16T07:22:48.130159+0000","last_became_peered":"2024-09-16T07:22:48.130159+0000","last_unstale":"2024-09-16T07:22:50.139537+0000","last_undegraded":"2024-09-16T07:22:50.139537+0000","last_fullsized":"2024-09-16T07:22:50.139537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:08.500032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.133118+0000","last_change":"2024-09-16T07:22:49.140179+0000","last_active":"2024-09-16T07:22:50.133118+0000","last_peered":"2024-09-16T07:22:50.133118+0000","last_clean":"2024-09-16T07:22:50.133118+0000","last_became_active":"2024-09-16T07:22:49.137012+0000","last_became_peered":"2024-09-16T07:22:49.137012+0000","last_unstale":"2024-09-16T07:22:50.133118+0000","last_undegraded":"2024-09-16T07:22:50.133118+0000","last_fullsized":"2024-09-16T07:22:50.133118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:48.929979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280281+0000","last_change":"2024-09-16T07:22:48.140081+0000","last_active":"2024-09-16T07:22:50.280281+0000","last_peered":"2024-09-16T07:22:50.280281+0000","last_clean":"2024-09-16T07:22:50.280281+0000","last_became_active":"2024-09-16T07:22:48.139786+0000","last_became_peered":"2024-09-16T07:22:48.139786+0000","last_unstale":"2024-09-16T07:22:50.280281+0000","last_undegraded":"2024-09-16T07:22:50.280281+0000","last_fullsized":"2024-09-16T07:22:50.280281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280002+0000","last_change":"2024-09-16T07:22:49.135775+0000","last_active":"2024-09-16T07:22:50.280002+0000","last_peered":"2024-09-16T07:22:50.280002+0000","last_clean":"2024-09-16T07:22:50.280002+0000","last_became_active":"2024-09-16T07:22:49.135501+0000","last_became_peered":"2024-09-16T07:22:49.135501+0000","last_unstale":"2024-09-16T07:22:50.280002+0000","last_undegraded":"2024-09-16T07:22:50.280002+0000","last_fullsized":"2024-09-16T07:22:50.280002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:03:48.213796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132027+0000","last_change":"2024-09-16T07:22:48.130827+0000","last_active":"2024-09-16T07:22:50.132027+0000","last_peered":"2024-09-16T07:22:50.132027+0000","last_clean":"2024-09-16T07:22:50.132027+0000","last_became_active":"2024-09-16T07:22:48.129479+0000","last_became_peered":"2024-09-16T07:22:48.129479+0000","last_unstale":"2024-09-16T07:22:50.132027+0000","last_undegraded":"2024-09-16T07:22:50.132027+0000","last_fullsized":"2024-09-16T07:22:50.132027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:34.669437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139797+0000","last_change":"2024-09-16T07:22:49.141394+0000","last_active":"2024-09-16T07:22:50.139797+0000","last_peered":"2024-09-16T07:22:50.139797+0000","last_clean":"2024-09-16T07:22:50.139797+0000","last_became_active":"2024-09-16T07:22:49.141098+0000","last_became_peered":"2024-09-16T07:22:49.141098+0000","last_unstale":"2024-09-16T07:22:50.139797+0000","last_undegraded":"2024-09-16T07:22:50.139797+0000","last_fullsized":"2024-09-16T07:22:50.139797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:52.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279971+0000","last_change":"2024-09-16T07:22:48.137959+0000","last_active":"2024-09-16T07:22:50.279971+0000","last_peered":"2024-09-16T07:22:50.279971+0000","last_clean":"2024-09-16T07:22:50.279971+0000","last_became_active":"2024-09-16T07:22:48.137703+0000","last_became_peered":"2024-09-16T07:22:48.137703+0000","last_unstale":"2024-09-16T07:22:50.279971+0000","last_undegraded":"2024-09-16T07:22:50.279971+0000","last_fullsized":"2024-09-16T07:22:50.279971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279516+0000","last_change":"2024-09-16T07:22:49.133885+0000","last_active":"2024-09-16T07:22:50.279516+0000","last_peered":"2024-09-16T07:22:50.279516+0000","last_clean":"2024-09-16T07:22:50.279516+0000","last_became_active":"2024-09-16T07:22:49.133562+0000","last_became_peered":"2024-09-16T07:22:49.133562+0000","last_unstale":"2024-09-16T07:22:50.279516+0000","last_undegraded":"2024-09-16T07:22:50.279516+0000","last_fullsized":"2024-09-16T07:22:50.279516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:43.685739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279361+0000","last_change":"2024-09-16T07:22:48.137679+0000","last_active":"2024-09-16T07:22:50.279361+0000","last_peered":"2024-09-16T07:22:50.279361+0000","last_clean":"2024-09-16T07:22:50.279361+0000","last_became_active":"2024-09-16T07:22:48.134910+0000","last_became_peered":"2024-09-16T07:22:48.134910+0000","last_unstale":"2024-09-16T07:22:50.279361+0000","last_undegraded":"2024-09-16T07:22:50.279361+0000","last_fullsized":"2024-09-16T07:22:50.279361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:06.837305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279526+0000","last_change":"2024-09-16T07:22:49.131231+0000","last_active":"2024-09-16T07:22:50.279526+0000","last_peered":"2024-09-16T07:22:50.279526+0000","last_clean":"2024-09-16T07:22:50.279526+0000","last_became_active":"2024-09-16T07:22:49.130991+0000","last_became_peered":"2024-09-16T07:22:49.130991+0000","last_unstale":"2024-09-16T07:22:50.279526+0000","last_undegraded":"2024-09-16T07:22:50.279526+0000","last_fullsized":"2024-09-16T07:22:50.279526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:14:00.862069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139154+0000","last_change":"2024-09-16T07:22:48.139812+0000","last_active":"2024-09-16T07:22:50.139154+0000","last_peered":"2024-09-16T07:22:50.139154+0000","last_clean":"2024-09-16T07:22:50.139154+0000","last_became_active":"2024-09-16T07:22:48.139509+0000","last_became_peered":"2024-09-16T07:22:48.139509+0000","last_unstale":"2024-09-16T07:22:50.139154+0000","last_undegraded":"2024-09-16T07:22:50.139154+0000","last_fullsized":"2024-09-16T07:22:50.139154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:46.703518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278833+0000","last_change":"2024-09-16T07:22:49.143207+0000","last_active":"2024-09-16T07:22:50.278833+0000","last_peered":"2024-09-16T07:22:50.278833+0000","last_clean":"2024-09-16T07:22:50.278833+0000","last_became_active":"2024-09-16T07:22:49.139696+0000","last_became_peered":"2024-09-16T07:22:49.139696+0000","last_unstale":"2024-09-16T07:22:50.278833+0000","last_undegraded":"2024-09-16T07:22:50.278833+0000","last_fullsized":"2024-09-16T07:22:50.278833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130773+0000","last_change":"2024-09-16T07:22:48.134633+0000","last_active":"2024-09-16T07:22:50.130773+0000","last_peered":"2024-09-16T07:22:50.130773+0000","last_clean":"2024-09-16T07:22:50.130773+0000","last_became_active":"2024-09-16T07:22:48.134291+0000","last_became_peered":"2024-09-16T07:22:48.134291+0000","last_unstale":"2024-09-16T07:22:50.130773+0000","last_undegraded":"2024-09-16T07:22:50.130773+0000","last_fullsized":"2024-09-16T07:22:50.130773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:35.346741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278836+0000","last_change":"2024-09-16T07:22:49.127175+0000","last_active":"2024-09-16T07:22:50.278836+0000","last_peered":"2024-09-16T07:22:50.278836+0000","last_clean":"2024-09-16T07:22:50.278836+0000","last_became_active":"2024-09-16T07:22:49.126846+0000","last_became_peered":"2024-09-16T07:22:49.126846+0000","last_unstale":"2024-09-16T07:22:50.278836+0000","last_undegraded":"2024-09-16T07:22:50.278836+0000","last_fullsized":"2024-09-16T07:22:50.278836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:51.576491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139738+0000","last_change":"2024-09-16T07:22:48.138039+0000","last_active":"2024-09-16T07:22:50.139738+0000","last_peered":"2024-09-16T07:22:50.139738+0000","last_clean":"2024-09-16T07:22:50.139738+0000","last_became_active":"2024-09-16T07:22:48.137746+0000","last_became_peered":"2024-09-16T07:22:48.137746+0000","last_unstale":"2024-09-16T07:22:50.139738+0000","last_undegraded":"2024-09-16T07:22:50.139738+0000","last_fullsized":"2024-09-16T07:22:50.139738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:09.846796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132500+0000","last_change":"2024-09-16T07:22:49.132362+0000","last_active":"2024-09-16T07:22:50.132500+0000","last_peered":"2024-09-16T07:22:50.132500+0000","last_clean":"2024-09-16T07:22:50.132500+0000","last_became_active":"2024-09-16T07:22:49.131756+0000","last_became_peered":"2024-09-16T07:22:49.131756+0000","last_unstale":"2024-09-16T07:22:50.132500+0000","last_undegraded":"2024-09-16T07:22:50.132500+0000","last_fullsized":"2024-09-16T07:22:50.132500+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:43.685739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132541+0000","last_change":"2024-09-16T07:22:48.137778+0000","last_active":"2024-09-16T07:22:50.132541+0000","last_peered":"2024-09-16T07:22:50.132541+0000","last_clean":"2024-09-16T07:22:50.132541+0000","last_became_active":"2024-09-16T07:22:48.137159+0000","last_became_peered":"2024-09-16T07:22:48.137159+0000","last_unstale":"2024-09-16T07:22:50.132541+0000","last_undegraded":"2024-09-16T07:22:50.132541+0000","last_fullsized":"2024-09-16T07:22:50.132541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:37.703838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139626+0000","last_change":"2024-09-16T07:22:49.130481+0000","last_active":"2024-09-16T07:22:50.139626+0000","last_peered":"2024-09-16T07:22:50.139626+0000","last_clean":"2024-09-16T07:22:50.139626+0000","last_became_active":"2024-09-16T07:22:49.130181+0000","last_became_peered":"2024-09-16T07:22:49.130181+0000","last_unstale":"2024-09-16T07:22:50.139626+0000","last_undegraded":"2024-09-16T07:22:50.139626+0000","last_fullsized":"2024-09-16T07:22:50.139626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:41.565481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130923+0000","last_change":"2024-09-16T07:22:48.135490+0000","last_active":"2024-09-16T07:22:50.130923+0000","last_peered":"2024-09-16T07:22:50.130923+0000","last_clean":"2024-09-16T07:22:50.130923+0000","last_became_active":"2024-09-16T07:22:48.134343+0000","last_became_peered":"2024-09-16T07:22:48.134343+0000","last_unstale":"2024-09-16T07:22:50.130923+0000","last_undegraded":"2024-09-16T07:22:50.130923+0000","last_fullsized":"2024-09-16T07:22:50.130923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:45.020349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139776+0000","last_change":"2024-09-16T07:22:49.144669+0000","last_active":"2024-09-16T07:22:50.139776+0000","last_peered":"2024-09-16T07:22:50.139776+0000","last_clean":"2024-09-16T07:22:50.139776+0000","last_became_active":"2024-09-16T07:22:49.144415+0000","last_became_peered":"2024-09-16T07:22:49.144415+0000","last_unstale":"2024-09-16T07:22:50.139776+0000","last_undegraded":"2024-09-16T07:22:50.139776+0000","last_fullsized":"2024-09-16T07:22:50.139776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:51.576491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280181+0000","last_change":"2024-09-16T07:22:48.135506+0000","last_active":"2024-09-16T07:22:50.280181+0000","last_peered":"2024-09-16T07:22:50.280181+0000","last_clean":"2024-09-16T07:22:50.280181+0000","last_became_active":"2024-09-16T07:22:48.134852+0000","last_became_peered":"2024-09-16T07:22:48.134852+0000","last_unstale":"2024-09-16T07:22:50.280181+0000","last_undegraded":"2024-09-16T07:22:50.280181+0000","last_fullsized":"2024-09-16T07:22:50.280181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279526+0000","last_change":"2024-09-16T07:22:49.134414+0000","last_active":"2024-09-16T07:22:50.279526+0000","last_peered":"2024-09-16T07:22:50.279526+0000","last_clean":"2024-09-16T07:22:50.279526+0000","last_became_active":"2024-09-16T07:22:49.130571+0000","last_became_peered":"2024-09-16T07:22:49.130571+0000","last_unstale":"2024-09-16T07:22:50.279526+0000","last_undegraded":"2024-09-16T07:22:50.279526+0000","last_fullsized":"2024-09-16T07:22:50.279526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:02.699542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279112+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:22:50.279112+0000","last_peered":"2024-09-16T07:22:50.279112+0000","last_clean":"2024-09-16T07:22:50.279112+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:22:50.279112+0000","last_undegraded":"2024-09-16T07:22:50.279112+0000","last_fullsized":"2024-09-16T07:22:50.279112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279775+0000","last_change":"2024-09-16T07:22:48.139675+0000","last_active":"2024-09-16T07:22:50.279775+0000","last_peered":"2024-09-16T07:22:50.279775+0000","last_clean":"2024-09-16T07:22:50.279775+0000","last_became_active":"2024-09-16T07:22:48.137989+0000","last_became_peered":"2024-09-16T07:22:48.137989+0000","last_unstale":"2024-09-16T07:22:50.279775+0000","last_undegraded":"2024-09-16T07:22:50.279775+0000","last_fullsized":"2024-09-16T07:22:50.279775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:16.190625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132139+0000","last_change":"2024-09-16T07:22:49.137102+0000","last_active":"2024-09-16T07:22:50.132139+0000","last_peered":"2024-09-16T07:22:50.132139+0000","last_clean":"2024-09-16T07:22:50.132139+0000","last_became_active":"2024-09-16T07:22:49.136755+0000","last_became_peered":"2024-09-16T07:22:49.136755+0000","last_unstale":"2024-09-16T07:22:50.132139+0000","last_undegraded":"2024-09-16T07:22:50.132139+0000","last_fullsized":"2024-09-16T07:22:50.132139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:26.935160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278962+0000","last_change":"2024-09-16T07:22:48.133273+0000","last_active":"2024-09-16T07:22:50.278962+0000","last_peered":"2024-09-16T07:22:50.278962+0000","last_clean":"2024-09-16T07:22:50.278962+0000","last_became_active":"2024-09-16T07:22:48.132984+0000","last_became_peered":"2024-09-16T07:22:48.132984+0000","last_unstale":"2024-09-16T07:22:50.278962+0000","last_undegraded":"2024-09-16T07:22:50.278962+0000","last_fullsized":"2024-09-16T07:22:50.278962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.657246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279239+0000","last_change":"2024-09-16T07:22:49.134936+0000","last_active":"2024-09-16T07:22:50.279239+0000","last_peered":"2024-09-16T07:22:50.279239+0000","last_clean":"2024-09-16T07:22:50.279239+0000","last_became_active":"2024-09-16T07:22:49.134575+0000","last_became_peered":"2024-09-16T07:22:49.134575+0000","last_unstale":"2024-09-16T07:22:50.279239+0000","last_undegraded":"2024-09-16T07:22:50.279239+0000","last_fullsized":"2024-09-16T07:22:50.279239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279457+0000","last_change":"2024-09-16T07:22:48.140454+0000","last_active":"2024-09-16T07:22:50.279457+0000","last_peered":"2024-09-16T07:22:50.279457+0000","last_clean":"2024-09-16T07:22:50.279457+0000","last_became_active":"2024-09-16T07:22:48.140025+0000","last_became_peered":"2024-09-16T07:22:48.140025+0000","last_unstale":"2024-09-16T07:22:50.279457+0000","last_undegraded":"2024-09-16T07:22:50.279457+0000","last_fullsized":"2024-09-16T07:22:50.279457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130545+0000","last_change":"2024-09-16T07:22:49.138294+0000","last_active":"2024-09-16T07:22:50.130545+0000","last_peered":"2024-09-16T07:22:50.130545+0000","last_clean":"2024-09-16T07:22:50.130545+0000","last_became_active":"2024-09-16T07:22:49.137685+0000","last_became_peered":"2024-09-16T07:22:49.137685+0000","last_unstale":"2024-09-16T07:22:50.130545+0000","last_undegraded":"2024-09-16T07:22:50.130545+0000","last_fullsized":"2024-09-16T07:22:50.130545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:51.068492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279367+0000","last_change":"2024-09-16T07:22:48.139380+0000","last_active":"2024-09-16T07:22:50.279367+0000","last_peered":"2024-09-16T07:22:50.279367+0000","last_clean":"2024-09-16T07:22:50.279367+0000","last_became_active":"2024-09-16T07:22:48.138052+0000","last_became_peered":"2024-09-16T07:22:48.138052+0000","last_unstale":"2024-09-16T07:22:50.279367+0000","last_undegraded":"2024-09-16T07:22:50.279367+0000","last_fullsized":"2024-09-16T07:22:50.279367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:28.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139289+0000","last_change":"2024-09-16T07:22:49.133663+0000","last_active":"2024-09-16T07:22:50.139289+0000","last_peered":"2024-09-16T07:22:50.139289+0000","last_clean":"2024-09-16T07:22:50.139289+0000","last_became_active":"2024-09-16T07:22:49.133342+0000","last_became_peered":"2024-09-16T07:22:49.133342+0000","last_unstale":"2024-09-16T07:22:50.139289+0000","last_undegraded":"2024-09-16T07:22:50.139289+0000","last_fullsized":"2024-09-16T07:22:50.139289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:04.490402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130957+0000","last_change":"2024-09-16T07:22:48.141538+0000","last_active":"2024-09-16T07:22:50.130957+0000","last_peered":"2024-09-16T07:22:50.130957+0000","last_clean":"2024-09-16T07:22:50.130957+0000","last_became_active":"2024-09-16T07:22:48.138174+0000","last_became_peered":"2024-09-16T07:22:48.138174+0000","last_unstale":"2024-09-16T07:22:50.130957+0000","last_undegraded":"2024-09-16T07:22:50.130957+0000","last_fullsized":"2024-09-16T07:22:50.130957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:29.651356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132217+0000","last_change":"2024-09-16T07:22:49.136727+0000","last_active":"2024-09-16T07:22:50.132217+0000","last_peered":"2024-09-16T07:22:50.132217+0000","last_clean":"2024-09-16T07:22:50.132217+0000","last_became_active":"2024-09-16T07:22:49.136064+0000","last_became_peered":"2024-09-16T07:22:49.136064+0000","last_unstale":"2024-09-16T07:22:50.132217+0000","last_undegraded":"2024-09-16T07:22:50.132217+0000","last_fullsized":"2024-09-16T07:22:50.132217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279830+0000","last_change":"2024-09-16T07:22:48.135598+0000","last_active":"2024-09-16T07:22:50.279830+0000","last_peered":"2024-09-16T07:22:50.279830+0000","last_clean":"2024-09-16T07:22:50.279830+0000","last_became_active":"2024-09-16T07:22:48.135179+0000","last_became_peered":"2024-09-16T07:22:48.135179+0000","last_unstale":"2024-09-16T07:22:50.279830+0000","last_undegraded":"2024-09-16T07:22:50.279830+0000","last_fullsized":"2024-09-16T07:22:50.279830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:09.846796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280111+0000","last_change":"2024-09-16T07:22:49.137714+0000","last_active":"2024-09-16T07:22:50.280111+0000","last_peered":"2024-09-16T07:22:50.280111+0000","last_clean":"2024-09-16T07:22:50.280111+0000","last_became_active":"2024-09-16T07:22:49.134740+0000","last_became_peered":"2024-09-16T07:22:49.134740+0000","last_unstale":"2024-09-16T07:22:50.280111+0000","last_undegraded":"2024-09-16T07:22:50.280111+0000","last_fullsized":"2024-09-16T07:22:50.280111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:44.194572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138704+0000","last_change":"2024-09-16T07:22:48.137476+0000","last_active":"2024-09-16T07:22:50.138704+0000","last_peered":"2024-09-16T07:22:50.138704+0000","last_clean":"2024-09-16T07:22:50.138704+0000","last_became_active":"2024-09-16T07:22:48.136772+0000","last_became_peered":"2024-09-16T07:22:48.136772+0000","last_unstale":"2024-09-16T07:22:50.138704+0000","last_undegraded":"2024-09-16T07:22:50.138704+0000","last_fullsized":"2024-09-16T07:22:50.138704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.906034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132443+0000","last_change":"2024-09-16T07:22:49.144331+0000","last_active":"2024-09-16T07:22:50.132443+0000","last_peered":"2024-09-16T07:22:50.132443+0000","last_clean":"2024-09-16T07:22:50.132443+0000","last_became_active":"2024-09-16T07:22:49.144071+0000","last_became_peered":"2024-09-16T07:22:49.144071+0000","last_unstale":"2024-09-16T07:22:50.132443+0000","last_undegraded":"2024-09-16T07:22:50.132443+0000","last_fullsized":"2024-09-16T07:22:50.132443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.753793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280211+0000","last_change":"2024-09-16T07:22:48.132847+0000","last_active":"2024-09-16T07:22:50.280211+0000","last_peered":"2024-09-16T07:22:50.280211+0000","last_clean":"2024-09-16T07:22:50.280211+0000","last_became_active":"2024-09-16T07:22:48.132264+0000","last_became_peered":"2024-09-16T07:22:48.132264+0000","last_unstale":"2024-09-16T07:22:50.280211+0000","last_undegraded":"2024-09-16T07:22:50.280211+0000","last_fullsized":"2024-09-16T07:22:50.280211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:42.680089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:49.138949+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:49.138580+0000","last_became_peered":"2024-09-16T07:22:49.138580+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:30.446387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138995+0000","last_change":"2024-09-16T07:22:48.129412+0000","last_active":"2024-09-16T07:22:50.138995+0000","last_peered":"2024-09-16T07:22:50.138995+0000","last_clean":"2024-09-16T07:22:50.138995+0000","last_became_active":"2024-09-16T07:22:48.129137+0000","last_became_peered":"2024-09-16T07:22:48.129137+0000","last_unstale":"2024-09-16T07:22:50.138995+0000","last_undegraded":"2024-09-16T07:22:50.138995+0000","last_fullsized":"2024-09-16T07:22:50.138995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:07.509813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279051+0000","last_change":"2024-09-16T07:22:49.129827+0000","last_active":"2024-09-16T07:22:50.279051+0000","last_peered":"2024-09-16T07:22:50.279051+0000","last_clean":"2024-09-16T07:22:50.279051+0000","last_became_active":"2024-09-16T07:22:49.129474+0000","last_became_peered":"2024-09-16T07:22:49.129474+0000","last_unstale":"2024-09-16T07:22:50.279051+0000","last_undegraded":"2024-09-16T07:22:50.279051+0000","last_fullsized":"2024-09-16T07:22:50.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:57.650915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130669+0000","last_change":"2024-09-16T07:22:48.134437+0000","last_active":"2024-09-16T07:22:50.130669+0000","last_peered":"2024-09-16T07:22:50.130669+0000","last_clean":"2024-09-16T07:22:50.130669+0000","last_became_active":"2024-09-16T07:22:48.134144+0000","last_became_peered":"2024-09-16T07:22:48.134144+0000","last_unstale":"2024-09-16T07:22:50.130669+0000","last_undegraded":"2024-09-16T07:22:50.130669+0000","last_fullsized":"2024-09-16T07:22:50.130669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:04.051560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130660+0000","last_change":"2024-09-16T07:22:49.141869+0000","last_active":"2024-09-16T07:22:50.130660+0000","last_peered":"2024-09-16T07:22:50.130660+0000","last_clean":"2024-09-16T07:22:50.130660+0000","last_became_active":"2024-09-16T07:22:49.141602+0000","last_became_peered":"2024-09-16T07:22:49.141602+0000","last_unstale":"2024-09-16T07:22:50.130660+0000","last_undegraded":"2024-09-16T07:22:50.130660+0000","last_fullsized":"2024-09-16T07:22:50.130660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:44.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279669+0000","last_change":"2024-09-16T07:22:48.137499+0000","last_active":"2024-09-16T07:22:50.279669+0000","last_peered":"2024-09-16T07:22:50.279669+0000","last_clean":"2024-09-16T07:22:50.279669+0000","last_became_active":"2024-09-16T07:22:48.137177+0000","last_became_peered":"2024-09-16T07:22:48.137177+0000","last_unstale":"2024-09-16T07:22:50.279669+0000","last_undegraded":"2024-09-16T07:22:50.279669+0000","last_fullsized":"2024-09-16T07:22:50.279669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280080+0000","last_change":"2024-09-16T07:22:49.143083+0000","last_active":"2024-09-16T07:22:50.280080+0000","last_peered":"2024-09-16T07:22:50.280080+0000","last_clean":"2024-09-16T07:22:50.280080+0000","last_became_active":"2024-09-16T07:22:49.142817+0000","last_became_peered":"2024-09-16T07:22:49.142817+0000","last_unstale":"2024-09-16T07:22:50.280080+0000","last_undegraded":"2024-09-16T07:22:50.280080+0000","last_fullsized":"2024-09-16T07:22:50.280080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279682+0000","last_change":"2024-09-16T07:22:48.131412+0000","last_active":"2024-09-16T07:22:50.279682+0000","last_peered":"2024-09-16T07:22:50.279682+0000","last_clean":"2024-09-16T07:22:50.279682+0000","last_became_active":"2024-09-16T07:22:48.131167+0000","last_became_peered":"2024-09-16T07:22:48.131167+0000","last_unstale":"2024-09-16T07:22:50.279682+0000","last_undegraded":"2024-09-16T07:22:50.279682+0000","last_fullsized":"2024-09-16T07:22:50.279682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:40.642232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279172+0000","last_change":"2024-09-16T07:22:49.129812+0000","last_active":"2024-09-16T07:22:50.279172+0000","last_peered":"2024-09-16T07:22:50.279172+0000","last_clean":"2024-09-16T07:22:50.279172+0000","last_became_active":"2024-09-16T07:22:49.129511+0000","last_became_peered":"2024-09-16T07:22:49.129511+0000","last_unstale":"2024-09-16T07:22:50.279172+0000","last_undegraded":"2024-09-16T07:22:50.279172+0000","last_fullsized":"2024-09-16T07:22:50.279172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:12.678514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279340+0000","last_change":"2024-09-16T07:22:49.137458+0000","last_active":"2024-09-16T07:22:50.279340+0000","last_peered":"2024-09-16T07:22:50.279340+0000","last_clean":"2024-09-16T07:22:50.279340+0000","last_became_active":"2024-09-16T07:22:49.137122+0000","last_became_peered":"2024-09-16T07:22:49.137122+0000","last_unstale":"2024-09-16T07:22:50.279340+0000","last_undegraded":"2024-09-16T07:22:50.279340+0000","last_fullsized":"2024-09-16T07:22:50.279340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:52.744207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139907+0000","last_change":"2024-09-16T07:22:48.130394+0000","last_active":"2024-09-16T07:22:50.139907+0000","last_peered":"2024-09-16T07:22:50.139907+0000","last_clean":"2024-09-16T07:22:50.139907+0000","last_became_active":"2024-09-16T07:22:48.130013+0000","last_became_peered":"2024-09-16T07:22:48.130013+0000","last_unstale":"2024-09-16T07:22:50.139907+0000","last_undegraded":"2024-09-16T07:22:50.139907+0000","last_fullsized":"2024-09-16T07:22:50.139907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.657246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279527+0000","last_change":"2024-09-16T07:22:49.136594+0000","last_active":"2024-09-16T07:22:50.279527+0000","last_peered":"2024-09-16T07:22:50.279527+0000","last_clean":"2024-09-16T07:22:50.279527+0000","last_became_active":"2024-09-16T07:22:49.136266+0000","last_became_peered":"2024-09-16T07:22:49.136266+0000","last_unstale":"2024-09-16T07:22:50.279527+0000","last_undegraded":"2024-09-16T07:22:50.279527+0000","last_fullsized":"2024-09-16T07:22:50.279527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139950+0000","last_change":"2024-09-16T07:22:48.135403+0000","last_active":"2024-09-16T07:22:50.139950+0000","last_peered":"2024-09-16T07:22:50.139950+0000","last_clean":"2024-09-16T07:22:50.139950+0000","last_became_active":"2024-09-16T07:22:48.135142+0000","last_became_peered":"2024-09-16T07:22:48.135142+0000","last_unstale":"2024-09-16T07:22:50.139950+0000","last_undegraded":"2024-09-16T07:22:50.139950+0000","last_fullsized":"2024-09-16T07:22:50.139950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:58.988982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.133007+0000","last_change":"2024-09-16T07:22:49.129289+0000","last_active":"2024-09-16T07:22:50.133007+0000","last_peered":"2024-09-16T07:22:50.133007+0000","last_clean":"2024-09-16T07:22:50.133007+0000","last_became_active":"2024-09-16T07:22:49.129000+0000","last_became_peered":"2024-09-16T07:22:49.129000+0000","last_unstale":"2024-09-16T07:22:50.133007+0000","last_undegraded":"2024-09-16T07:22:50.133007+0000","last_fullsized":"2024-09-16T07:22:50.133007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:31.607464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:48.132860+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:48.132544+0000","last_became_peered":"2024-09-16T07:22:48.132544+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:52:05.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132482+0000","last_change":"2024-09-16T07:22:49.142356+0000","last_active":"2024-09-16T07:22:50.132482+0000","last_peered":"2024-09-16T07:22:50.132482+0000","last_clean":"2024-09-16T07:22:50.132482+0000","last_became_active":"2024-09-16T07:22:49.141639+0000","last_became_peered":"2024-09-16T07:22:49.141639+0000","last_unstale":"2024-09-16T07:22:50.132482+0000","last_undegraded":"2024-09-16T07:22:50.132482+0000","last_fullsized":"2024-09-16T07:22:50.132482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:57.949375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132622+0000","last_change":"2024-09-16T07:22:48.141242+0000","last_active":"2024-09-16T07:22:50.132622+0000","last_peered":"2024-09-16T07:22:50.132622+0000","last_clean":"2024-09-16T07:22:50.132622+0000","last_became_active":"2024-09-16T07:22:48.139871+0000","last_became_peered":"2024-09-16T07:22:48.139871+0000","last_unstale":"2024-09-16T07:22:50.132622+0000","last_undegraded":"2024-09-16T07:22:50.132622+0000","last_fullsized":"2024-09-16T07:22:50.132622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:32.916879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476743,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":81289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476743,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":671145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":80763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":80831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":80707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":831488,"data_stored":671167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:22:58.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:22:58.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.169+0000 7f552fe00640 1 -- 172.21.15.71:0/2858829171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5528073fb0 msgr2=0x7f5528074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.169+0000 7f552fe00640 1 --2- 172.21.15.71:0/2858829171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5528073fb0 0x7f5528074420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55280cfbb0 tx=0x7f552002a560 comp rx=0 tx=0).stop 2024-09-16T07:22:58.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.169+0000 7f552fe00640 1 -- 172.21.15.71:0/2858829171 shutdown_connections 2024-09-16T07:22:58.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.169+0000 7f552fe00640 1 --2- 172.21.15.71:0/2858829171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5528073fb0 0x7f5528074420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.169+0000 7f552fe00640 1 --2- 172.21.15.71:0/2858829171 >> v2:172.21.15.177:3300/0 conn(0x7f55280ccf00 0x7f55280b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.169+0000 7f552fe00640 1 -- 172.21.15.71:0/2858829171 >> 172.21.15.71:0/2858829171 conn(0x7f55280b1950 msgr2=0x7f55280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:58.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.169+0000 7f552fe00640 1 -- 172.21.15.71:0/2858829171 shutdown_connections 2024-09-16T07:22:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.169+0000 7f552fe00640 1 -- 172.21.15.71:0/2858829171 wait complete. 2024-09-16T07:22:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.171+0000 7f552fe00640 1 Processor -- start 2024-09-16T07:22:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.171+0000 7f552fe00640 1 -- start start 2024-09-16T07:22:58.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5528073fb0 0x7f5528149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55280ccf00 0x7f55280cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f552815f5a0 0x7f552809cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f552806f2e0 con 0x7f552815f5a0 2024-09-16T07:22:58.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f552806f160 con 0x7f5528073fb0 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f552806f460 con 0x7f55280ccf00 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f552815f5a0 0x7f552809cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f552815f5a0 0x7f552809cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50546/0 (socket says 172.21.15.71:50546) 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552de00640 1 -- 172.21.15.71:0/3879541539 learned_addr learned my addr 172.21.15.71:0/3879541539 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552d400640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5528073fb0 0x7f5528149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.172+0000 7f552ca00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55280ccf00 0x7f55280cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f552de00640 1 -- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55280ccf00 msgr2=0x7f55280cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f552de00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55280ccf00 0x7f55280cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f552de00640 1 -- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5528073fb0 msgr2=0x7f5528149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f552de00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5528073fb0 0x7f5528149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f552de00640 1 -- 172.21.15.71:0/3879541539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55280ce9d0 con 0x7f552815f5a0 2024-09-16T07:22:58.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f552de00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f552815f5a0 0x7f552809cb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55200099c0 tx=0x7f5520003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:58.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f5525600640 1 -- 172.21.15.71:0/3879541539 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552003f070 con 0x7f552815f5a0 2024-09-16T07:22:58.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f5525600640 1 -- 172.21.15.71:0/3879541539 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55200047a0 con 0x7f552815f5a0 2024-09-16T07:22:58.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f5525600640 1 -- 172.21.15.71:0/3879541539 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520035030 con 0x7f552815f5a0 2024-09-16T07:22:58.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55280cecb0 con 0x7f552815f5a0 2024-09-16T07:22:58.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.173+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55280cf270 con 0x7f552815f5a0 2024-09-16T07:22:58.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.174+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552806f9f0 con 0x7f552815f5a0 2024-09-16T07:22:58.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.175+0000 7f5525600640 1 -- 172.21.15.71:0/3879541539 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5520042070 con 0x7f552815f5a0 2024-09-16T07:22:58.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.176+0000 7f5525600640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5508078850 0x7f550807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.176+0000 7f5525600640 1 -- 172.21.15.71:0/3879541539 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f55200bb740 con 0x7f552815f5a0 2024-09-16T07:22:58.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.177+0000 7f552d400640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5508078850 0x7f550807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.178+0000 7f552d400640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5508078850 0x7f550807ad10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f551c001e20 tx=0x7f551c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:58.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.180+0000 7f5525600640 1 -- 172.21.15.71:0/3879541539 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f55200836c0 con 0x7f552815f5a0 2024-09-16T07:22:58.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.365+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55280afc40 con 0x7f5508078850 2024-09-16T07:22:58.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.370+0000 7f5525600640 1 -- 172.21.15.71:0/3879541539 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251885 (secure 0 0 0) 0x7f55280afc40 con 0x7f5508078850 2024-09-16T07:22:58.373 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5508078850 msgr2=0x7f550807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5508078850 0x7f550807ad10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f551c001e20 tx=0x7f551c009040 comp rx=0 tx=0).stop 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f552815f5a0 msgr2=0x7f552809cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f552815f5a0 0x7f552809cb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55200099c0 tx=0x7f5520003960 comp rx=0 tx=0).stop 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 shutdown_connections 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5508078850 0x7f550807ad10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f552815f5a0 0x7f552809cb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55280ccf00 0x7f55280cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 --2- 172.21.15.71:0/3879541539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5528073fb0 0x7f5528149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 >> 172.21.15.71:0/3879541539 conn(0x7f55280b1950 msgr2=0x7f55280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 shutdown_connections 2024-09-16T07:22:58.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.374+0000 7f552fe00640 1 -- 172.21.15.71:0/3879541539 wait complete. 2024-09-16T07:22:58.389 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-16T07:22:58.295739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223448,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751272,"statfs":{"total":773094113280,"available":772865302528,"internally_reserved":0,"allocated":3088384,"data_stored":1831212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.180930"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139414+0000","last_change":"2024-09-16T07:22:49.144943+0000","last_active":"2024-09-16T07:22:50.139414+0000","last_peered":"2024-09-16T07:22:50.139414+0000","last_clean":"2024-09-16T07:22:50.139414+0000","last_became_active":"2024-09-16T07:22:49.144609+0000","last_became_peered":"2024-09-16T07:22:49.144609+0000","last_unstale":"2024-09-16T07:22:50.139414+0000","last_undegraded":"2024-09-16T07:22:50.139414+0000","last_fullsized":"2024-09-16T07:22:50.139414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:21.053710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132260+0000","last_change":"2024-09-16T07:22:49.142860+0000","last_active":"2024-09-16T07:22:50.132260+0000","last_peered":"2024-09-16T07:22:50.132260+0000","last_clean":"2024-09-16T07:22:50.132260+0000","last_became_active":"2024-09-16T07:22:49.142555+0000","last_became_peered":"2024-09-16T07:22:49.142555+0000","last_unstale":"2024-09-16T07:22:50.132260+0000","last_undegraded":"2024-09-16T07:22:50.132260+0000","last_fullsized":"2024-09-16T07:22:50.132260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:20.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130865+0000","last_change":"2024-09-16T07:22:49.141351+0000","last_active":"2024-09-16T07:22:50.130865+0000","last_peered":"2024-09-16T07:22:50.130865+0000","last_clean":"2024-09-16T07:22:50.130865+0000","last_became_active":"2024-09-16T07:22:49.141078+0000","last_became_peered":"2024-09-16T07:22:49.141078+0000","last_unstale":"2024-09-16T07:22:50.130865+0000","last_undegraded":"2024-09-16T07:22:50.130865+0000","last_fullsized":"2024-09-16T07:22:50.130865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:54.707093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132009+0000","last_change":"2024-09-16T07:22:49.130699+0000","last_active":"2024-09-16T07:22:50.132009+0000","last_peered":"2024-09-16T07:22:50.132009+0000","last_clean":"2024-09-16T07:22:50.132009+0000","last_became_active":"2024-09-16T07:22:49.130435+0000","last_became_peered":"2024-09-16T07:22:49.130435+0000","last_unstale":"2024-09-16T07:22:50.132009+0000","last_undegraded":"2024-09-16T07:22:50.132009+0000","last_fullsized":"2024-09-16T07:22:50.132009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:37.684357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139429+0000","last_change":"2024-09-16T07:22:49.139132+0000","last_active":"2024-09-16T07:22:50.139429+0000","last_peered":"2024-09-16T07:22:50.139429+0000","last_clean":"2024-09-16T07:22:50.139429+0000","last_became_active":"2024-09-16T07:22:49.134577+0000","last_became_peered":"2024-09-16T07:22:49.134577+0000","last_unstale":"2024-09-16T07:22:50.139429+0000","last_undegraded":"2024-09-16T07:22:50.139429+0000","last_fullsized":"2024-09-16T07:22:50.139429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:28.720254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131882+0000","last_change":"2024-09-16T07:22:49.130577+0000","last_active":"2024-09-16T07:22:50.131882+0000","last_peered":"2024-09-16T07:22:50.131882+0000","last_clean":"2024-09-16T07:22:50.131882+0000","last_became_active":"2024-09-16T07:22:49.130338+0000","last_became_peered":"2024-09-16T07:22:49.130338+0000","last_unstale":"2024-09-16T07:22:50.131882+0000","last_undegraded":"2024-09-16T07:22:50.131882+0000","last_fullsized":"2024-09-16T07:22:50.131882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139192+0000","last_change":"2024-09-16T07:22:49.142202+0000","last_active":"2024-09-16T07:22:50.139192+0000","last_peered":"2024-09-16T07:22:50.139192+0000","last_clean":"2024-09-16T07:22:50.139192+0000","last_became_active":"2024-09-16T07:22:49.141956+0000","last_became_peered":"2024-09-16T07:22:49.141956+0000","last_unstale":"2024-09-16T07:22:50.139192+0000","last_undegraded":"2024-09-16T07:22:50.139192+0000","last_fullsized":"2024-09-16T07:22:50.139192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:35.827940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130543+0000","last_change":"2024-09-16T07:22:49.130201+0000","last_active":"2024-09-16T07:22:50.130543+0000","last_peered":"2024-09-16T07:22:50.130543+0000","last_clean":"2024-09-16T07:22:50.130543+0000","last_became_active":"2024-09-16T07:22:49.129912+0000","last_became_peered":"2024-09-16T07:22:49.129912+0000","last_unstale":"2024-09-16T07:22:50.130543+0000","last_undegraded":"2024-09-16T07:22:50.130543+0000","last_fullsized":"2024-09-16T07:22:50.130543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:21.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279771+0000","last_change":"2024-09-16T07:22:49.141365+0000","last_active":"2024-09-16T07:22:50.279771+0000","last_peered":"2024-09-16T07:22:50.279771+0000","last_clean":"2024-09-16T07:22:50.279771+0000","last_became_active":"2024-09-16T07:22:49.141041+0000","last_became_peered":"2024-09-16T07:22:49.141041+0000","last_unstale":"2024-09-16T07:22:50.279771+0000","last_undegraded":"2024-09-16T07:22:50.279771+0000","last_fullsized":"2024-09-16T07:22:50.279771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:37.684357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139441+0000","last_change":"2024-09-16T07:22:49.142086+0000","last_active":"2024-09-16T07:22:50.139441+0000","last_peered":"2024-09-16T07:22:50.139441+0000","last_clean":"2024-09-16T07:22:50.139441+0000","last_became_active":"2024-09-16T07:22:49.141832+0000","last_became_peered":"2024-09-16T07:22:49.141832+0000","last_unstale":"2024-09-16T07:22:50.139441+0000","last_undegraded":"2024-09-16T07:22:50.139441+0000","last_fullsized":"2024-09-16T07:22:50.139441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:05.967417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139585+0000","last_change":"2024-09-16T07:22:49.144692+0000","last_active":"2024-09-16T07:22:50.139585+0000","last_peered":"2024-09-16T07:22:50.139585+0000","last_clean":"2024-09-16T07:22:50.139585+0000","last_became_active":"2024-09-16T07:22:49.144396+0000","last_became_peered":"2024-09-16T07:22:49.144396+0000","last_unstale":"2024-09-16T07:22:50.139585+0000","last_undegraded":"2024-09-16T07:22:50.139585+0000","last_fullsized":"2024-09-16T07:22:50.139585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:47.741254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139742+0000","last_change":"2024-09-16T07:22:49.130387+0000","last_active":"2024-09-16T07:22:50.139742+0000","last_peered":"2024-09-16T07:22:50.139742+0000","last_clean":"2024-09-16T07:22:50.139742+0000","last_became_active":"2024-09-16T07:22:49.129994+0000","last_became_peered":"2024-09-16T07:22:49.129994+0000","last_unstale":"2024-09-16T07:22:50.139742+0000","last_undegraded":"2024-09-16T07:22:50.139742+0000","last_fullsized":"2024-09-16T07:22:50.139742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:11.870228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279379+0000","last_change":"2024-09-16T07:22:49.133371+0000","last_active":"2024-09-16T07:22:50.279379+0000","last_peered":"2024-09-16T07:22:50.279379+0000","last_clean":"2024-09-16T07:22:50.279379+0000","last_became_active":"2024-09-16T07:22:49.133088+0000","last_became_peered":"2024-09-16T07:22:49.133088+0000","last_unstale":"2024-09-16T07:22:50.279379+0000","last_undegraded":"2024-09-16T07:22:50.279379+0000","last_fullsized":"2024-09-16T07:22:50.279379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:17.196275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279291+0000","last_change":"2024-09-16T07:22:49.140290+0000","last_active":"2024-09-16T07:22:50.279291+0000","last_peered":"2024-09-16T07:22:50.279291+0000","last_clean":"2024-09-16T07:22:50.279291+0000","last_became_active":"2024-09-16T07:22:49.139999+0000","last_became_peered":"2024-09-16T07:22:49.139999+0000","last_unstale":"2024-09-16T07:22:50.279291+0000","last_undegraded":"2024-09-16T07:22:50.279291+0000","last_fullsized":"2024-09-16T07:22:50.279291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131078+0000","last_change":"2024-09-16T07:22:49.131493+0000","last_active":"2024-09-16T07:22:50.131078+0000","last_peered":"2024-09-16T07:22:50.131078+0000","last_clean":"2024-09-16T07:22:50.131078+0000","last_became_active":"2024-09-16T07:22:49.131241+0000","last_became_peered":"2024-09-16T07:22:49.131241+0000","last_unstale":"2024-09-16T07:22:50.131078+0000","last_undegraded":"2024-09-16T07:22:50.131078+0000","last_fullsized":"2024-09-16T07:22:50.131078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:24.470314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278778+0000","last_change":"2024-09-16T07:22:49.126099+0000","last_active":"2024-09-16T07:22:50.278778+0000","last_peered":"2024-09-16T07:22:50.278778+0000","last_clean":"2024-09-16T07:22:50.278778+0000","last_became_active":"2024-09-16T07:22:49.125826+0000","last_became_peered":"2024-09-16T07:22:49.125826+0000","last_unstale":"2024-09-16T07:22:50.278778+0000","last_undegraded":"2024-09-16T07:22:50.278778+0000","last_fullsized":"2024-09-16T07:22:50.278778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:41.565481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280232+0000","last_change":"2024-09-16T07:22:49.132617+0000","last_active":"2024-09-16T07:22:50.280232+0000","last_peered":"2024-09-16T07:22:50.280232+0000","last_clean":"2024-09-16T07:22:50.280232+0000","last_became_active":"2024-09-16T07:22:49.132278+0000","last_became_peered":"2024-09-16T07:22:49.132278+0000","last_unstale":"2024-09-16T07:22:50.280232+0000","last_undegraded":"2024-09-16T07:22:50.280232+0000","last_fullsized":"2024-09-16T07:22:50.280232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:10.129478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280258+0000","last_change":"2024-09-16T07:22:49.143862+0000","last_active":"2024-09-16T07:22:50.280258+0000","last_peered":"2024-09-16T07:22:50.280258+0000","last_clean":"2024-09-16T07:22:50.280258+0000","last_became_active":"2024-09-16T07:22:49.143457+0000","last_became_peered":"2024-09-16T07:22:49.143457+0000","last_unstale":"2024-09-16T07:22:50.280258+0000","last_undegraded":"2024-09-16T07:22:50.280258+0000","last_fullsized":"2024-09-16T07:22:50.280258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279691+0000","last_change":"2024-09-16T07:22:49.130309+0000","last_active":"2024-09-16T07:22:50.279691+0000","last_peered":"2024-09-16T07:22:50.279691+0000","last_clean":"2024-09-16T07:22:50.279691+0000","last_became_active":"2024-09-16T07:22:49.130069+0000","last_became_peered":"2024-09-16T07:22:49.130069+0000","last_unstale":"2024-09-16T07:22:50.279691+0000","last_undegraded":"2024-09-16T07:22:50.279691+0000","last_fullsized":"2024-09-16T07:22:50.279691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:20.784563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139554+0000","last_change":"2024-09-16T07:22:49.129763+0000","last_active":"2024-09-16T07:22:50.139554+0000","last_peered":"2024-09-16T07:22:50.139554+0000","last_clean":"2024-09-16T07:22:50.139554+0000","last_became_active":"2024-09-16T07:22:49.129067+0000","last_became_peered":"2024-09-16T07:22:49.129067+0000","last_unstale":"2024-09-16T07:22:50.139554+0000","last_undegraded":"2024-09-16T07:22:50.139554+0000","last_fullsized":"2024-09-16T07:22:50.139554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:24.919949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132192+0000","last_change":"2024-09-16T07:22:49.133568+0000","last_active":"2024-09-16T07:22:50.132192+0000","last_peered":"2024-09-16T07:22:50.132192+0000","last_clean":"2024-09-16T07:22:50.132192+0000","last_became_active":"2024-09-16T07:22:49.133323+0000","last_became_peered":"2024-09-16T07:22:49.133323+0000","last_unstale":"2024-09-16T07:22:50.132192+0000","last_undegraded":"2024-09-16T07:22:50.132192+0000","last_fullsized":"2024-09-16T07:22:50.132192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:45.369698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279803+0000","last_change":"2024-09-16T07:22:49.137697+0000","last_active":"2024-09-16T07:22:50.279803+0000","last_peered":"2024-09-16T07:22:50.279803+0000","last_clean":"2024-09-16T07:22:50.279803+0000","last_became_active":"2024-09-16T07:22:49.137443+0000","last_became_peered":"2024-09-16T07:22:49.137443+0000","last_unstale":"2024-09-16T07:22:50.279803+0000","last_undegraded":"2024-09-16T07:22:50.279803+0000","last_fullsized":"2024-09-16T07:22:50.279803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130836+0000","last_change":"2024-09-16T07:22:49.139909+0000","last_active":"2024-09-16T07:22:50.130836+0000","last_peered":"2024-09-16T07:22:50.130836+0000","last_clean":"2024-09-16T07:22:50.130836+0000","last_became_active":"2024-09-16T07:22:49.139628+0000","last_became_peered":"2024-09-16T07:22:49.139628+0000","last_unstale":"2024-09-16T07:22:50.130836+0000","last_undegraded":"2024-09-16T07:22:50.130836+0000","last_fullsized":"2024-09-16T07:22:50.130836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:02.467243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278810+0000","last_change":"2024-09-16T07:22:49.134680+0000","last_active":"2024-09-16T07:22:50.278810+0000","last_peered":"2024-09-16T07:22:50.278810+0000","last_clean":"2024-09-16T07:22:50.278810+0000","last_became_active":"2024-09-16T07:22:49.134374+0000","last_became_peered":"2024-09-16T07:22:49.134374+0000","last_unstale":"2024-09-16T07:22:50.278810+0000","last_undegraded":"2024-09-16T07:22:50.278810+0000","last_fullsized":"2024-09-16T07:22:50.278810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:16.837921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132879+0000","last_change":"2024-09-16T07:22:49.133787+0000","last_active":"2024-09-16T07:22:50.132879+0000","last_peered":"2024-09-16T07:22:50.132879+0000","last_clean":"2024-09-16T07:22:50.132879+0000","last_became_active":"2024-09-16T07:22:49.133552+0000","last_became_peered":"2024-09-16T07:22:49.133552+0000","last_unstale":"2024-09-16T07:22:50.132879+0000","last_undegraded":"2024-09-16T07:22:50.132879+0000","last_fullsized":"2024-09-16T07:22:50.132879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.930728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132323+0000","last_change":"2024-09-16T07:22:49.135809+0000","last_active":"2024-09-16T07:22:50.132323+0000","last_peered":"2024-09-16T07:22:50.132323+0000","last_clean":"2024-09-16T07:22:50.132323+0000","last_became_active":"2024-09-16T07:22:49.135527+0000","last_became_peered":"2024-09-16T07:22:49.135527+0000","last_unstale":"2024-09-16T07:22:50.132323+0000","last_undegraded":"2024-09-16T07:22:50.132323+0000","last_fullsized":"2024-09-16T07:22:50.132323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279729+0000","last_change":"2024-09-16T07:22:49.140506+0000","last_active":"2024-09-16T07:22:50.279729+0000","last_peered":"2024-09-16T07:22:50.279729+0000","last_clean":"2024-09-16T07:22:50.279729+0000","last_became_active":"2024-09-16T07:22:49.140006+0000","last_became_peered":"2024-09-16T07:22:49.140006+0000","last_unstale":"2024-09-16T07:22:50.279729+0000","last_undegraded":"2024-09-16T07:22:50.279729+0000","last_fullsized":"2024-09-16T07:22:50.279729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:29.077071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279953+0000","last_change":"2024-09-16T07:22:49.136459+0000","last_active":"2024-09-16T07:22:50.279953+0000","last_peered":"2024-09-16T07:22:50.279953+0000","last_clean":"2024-09-16T07:22:50.279953+0000","last_became_active":"2024-09-16T07:22:49.136183+0000","last_became_peered":"2024-09-16T07:22:49.136183+0000","last_unstale":"2024-09-16T07:22:50.279953+0000","last_undegraded":"2024-09-16T07:22:50.279953+0000","last_fullsized":"2024-09-16T07:22:50.279953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:33.154370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130793+0000","last_change":"2024-09-16T07:22:49.136171+0000","last_active":"2024-09-16T07:22:50.130793+0000","last_peered":"2024-09-16T07:22:50.130793+0000","last_clean":"2024-09-16T07:22:50.130793+0000","last_became_active":"2024-09-16T07:22:49.135900+0000","last_became_peered":"2024-09-16T07:22:49.135900+0000","last_unstale":"2024-09-16T07:22:50.130793+0000","last_undegraded":"2024-09-16T07:22:50.130793+0000","last_fullsized":"2024-09-16T07:22:50.130793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:01.273769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279131+0000","last_change":"2024-09-16T07:22:49.133629+0000","last_active":"2024-09-16T07:22:50.279131+0000","last_peered":"2024-09-16T07:22:50.279131+0000","last_clean":"2024-09-16T07:22:50.279131+0000","last_became_active":"2024-09-16T07:22:49.133394+0000","last_became_peered":"2024-09-16T07:22:49.133394+0000","last_unstale":"2024-09-16T07:22:50.279131+0000","last_undegraded":"2024-09-16T07:22:50.279131+0000","last_fullsized":"2024-09-16T07:22:50.279131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:41.976217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279399+0000","last_change":"2024-09-16T07:22:49.144945+0000","last_active":"2024-09-16T07:22:50.279399+0000","last_peered":"2024-09-16T07:22:50.279399+0000","last_clean":"2024-09-16T07:22:50.279399+0000","last_became_active":"2024-09-16T07:22:49.144649+0000","last_became_peered":"2024-09-16T07:22:49.144649+0000","last_unstale":"2024-09-16T07:22:50.279399+0000","last_undegraded":"2024-09-16T07:22:50.279399+0000","last_fullsized":"2024-09-16T07:22:50.279399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:59.994632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131706+0000","last_change":"2024-09-16T07:22:49.136574+0000","last_active":"2024-09-16T07:22:50.131706+0000","last_peered":"2024-09-16T07:22:50.131706+0000","last_clean":"2024-09-16T07:22:50.131706+0000","last_became_active":"2024-09-16T07:22:49.136412+0000","last_became_peered":"2024-09-16T07:22:49.136412+0000","last_unstale":"2024-09-16T07:22:50.131706+0000","last_undegraded":"2024-09-16T07:22:50.131706+0000","last_fullsized":"2024-09-16T07:22:50.131706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280014+0000","last_change":"2024-09-16T07:22:49.141959+0000","last_active":"2024-09-16T07:22:50.280014+0000","last_peered":"2024-09-16T07:22:50.280014+0000","last_clean":"2024-09-16T07:22:50.280014+0000","last_became_active":"2024-09-16T07:22:49.140661+0000","last_became_peered":"2024-09-16T07:22:49.140661+0000","last_unstale":"2024-09-16T07:22:50.280014+0000","last_undegraded":"2024-09-16T07:22:50.280014+0000","last_fullsized":"2024-09-16T07:22:50.280014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130863+0000","last_change":"2024-09-16T07:22:48.137796+0000","last_active":"2024-09-16T07:22:50.130863+0000","last_peered":"2024-09-16T07:22:50.130863+0000","last_clean":"2024-09-16T07:22:50.130863+0000","last_became_active":"2024-09-16T07:22:48.136473+0000","last_became_peered":"2024-09-16T07:22:48.136473+0000","last_unstale":"2024-09-16T07:22:50.130863+0000","last_undegraded":"2024-09-16T07:22:50.130863+0000","last_fullsized":"2024-09-16T07:22:50.130863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:16.721715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132215+0000","last_change":"2024-09-16T07:22:49.132936+0000","last_active":"2024-09-16T07:22:50.132215+0000","last_peered":"2024-09-16T07:22:50.132215+0000","last_clean":"2024-09-16T07:22:50.132215+0000","last_became_active":"2024-09-16T07:22:49.132677+0000","last_became_peered":"2024-09-16T07:22:49.132677+0000","last_unstale":"2024-09-16T07:22:50.132215+0000","last_undegraded":"2024-09-16T07:22:50.132215+0000","last_fullsized":"2024-09-16T07:22:50.132215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:16.641320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279272+0000","last_change":"2024-09-16T07:22:48.140563+0000","last_active":"2024-09-16T07:22:50.279272+0000","last_peered":"2024-09-16T07:22:50.279272+0000","last_clean":"2024-09-16T07:22:50.279272+0000","last_became_active":"2024-09-16T07:22:48.139864+0000","last_became_peered":"2024-09-16T07:22:48.139864+0000","last_unstale":"2024-09-16T07:22:50.279272+0000","last_undegraded":"2024-09-16T07:22:50.279272+0000","last_fullsized":"2024-09-16T07:22:50.279272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:36.678707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279310+0000","last_change":"2024-09-16T07:22:48.135463+0000","last_active":"2024-09-16T07:22:50.279310+0000","last_peered":"2024-09-16T07:22:50.279310+0000","last_clean":"2024-09-16T07:22:50.279310+0000","last_became_active":"2024-09-16T07:22:48.135133+0000","last_became_peered":"2024-09-16T07:22:48.135133+0000","last_unstale":"2024-09-16T07:22:50.279310+0000","last_undegraded":"2024-09-16T07:22:50.279310+0000","last_fullsized":"2024-09-16T07:22:50.279310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130763+0000","last_change":"2024-09-16T07:22:49.138488+0000","last_active":"2024-09-16T07:22:50.130763+0000","last_peered":"2024-09-16T07:22:50.130763+0000","last_clean":"2024-09-16T07:22:50.130763+0000","last_became_active":"2024-09-16T07:22:49.138042+0000","last_became_peered":"2024-09-16T07:22:49.138042+0000","last_unstale":"2024-09-16T07:22:50.130763+0000","last_undegraded":"2024-09-16T07:22:50.130763+0000","last_fullsized":"2024-09-16T07:22:50.130763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:20.696268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279074+0000","last_change":"2024-09-16T07:22:48.135129+0000","last_active":"2024-09-16T07:22:50.279074+0000","last_peered":"2024-09-16T07:22:50.279074+0000","last_clean":"2024-09-16T07:22:50.279074+0000","last_became_active":"2024-09-16T07:22:48.134665+0000","last_became_peered":"2024-09-16T07:22:48.134665+0000","last_unstale":"2024-09-16T07:22:50.279074+0000","last_undegraded":"2024-09-16T07:22:50.279074+0000","last_fullsized":"2024-09-16T07:22:50.279074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:58.988982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130571+0000","last_change":"2024-09-16T07:22:49.129262+0000","last_active":"2024-09-16T07:22:50.130571+0000","last_peered":"2024-09-16T07:22:50.130571+0000","last_clean":"2024-09-16T07:22:50.130571+0000","last_became_active":"2024-09-16T07:22:49.128955+0000","last_became_peered":"2024-09-16T07:22:49.128955+0000","last_unstale":"2024-09-16T07:22:50.130571+0000","last_undegraded":"2024-09-16T07:22:50.130571+0000","last_fullsized":"2024-09-16T07:22:50.130571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:35.169775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279936+0000","last_change":"2024-09-16T07:22:48.139495+0000","last_active":"2024-09-16T07:22:50.279936+0000","last_peered":"2024-09-16T07:22:50.279936+0000","last_clean":"2024-09-16T07:22:50.279936+0000","last_became_active":"2024-09-16T07:22:48.139004+0000","last_became_peered":"2024-09-16T07:22:48.139004+0000","last_unstale":"2024-09-16T07:22:50.279936+0000","last_undegraded":"2024-09-16T07:22:50.279936+0000","last_fullsized":"2024-09-16T07:22:50.279936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:22.689639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132362+0000","last_change":"2024-09-16T07:22:49.134719+0000","last_active":"2024-09-16T07:22:50.132362+0000","last_peered":"2024-09-16T07:22:50.132362+0000","last_clean":"2024-09-16T07:22:50.132362+0000","last_became_active":"2024-09-16T07:22:49.134471+0000","last_became_peered":"2024-09-16T07:22:49.134471+0000","last_unstale":"2024-09-16T07:22:50.132362+0000","last_undegraded":"2024-09-16T07:22:50.132362+0000","last_fullsized":"2024-09-16T07:22:50.132362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:15.125396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139537+0000","last_change":"2024-09-16T07:22:48.130409+0000","last_active":"2024-09-16T07:22:50.139537+0000","last_peered":"2024-09-16T07:22:50.139537+0000","last_clean":"2024-09-16T07:22:50.139537+0000","last_became_active":"2024-09-16T07:22:48.130159+0000","last_became_peered":"2024-09-16T07:22:48.130159+0000","last_unstale":"2024-09-16T07:22:50.139537+0000","last_undegraded":"2024-09-16T07:22:50.139537+0000","last_fullsized":"2024-09-16T07:22:50.139537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:08.500032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.133118+0000","last_change":"2024-09-16T07:22:49.140179+0000","last_active":"2024-09-16T07:22:50.133118+0000","last_peered":"2024-09-16T07:22:50.133118+0000","last_clean":"2024-09-16T07:22:50.133118+0000","last_became_active":"2024-09-16T07:22:49.137012+0000","last_became_peered":"2024-09-16T07:22:49.137012+0000","last_unstale":"2024-09-16T07:22:50.133118+0000","last_undegraded":"2024-09-16T07:22:50.133118+0000","last_fullsized":"2024-09-16T07:22:50.133118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:48.929979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280281+0000","last_change":"2024-09-16T07:22:48.140081+0000","last_active":"2024-09-16T07:22:50.280281+0000","last_peered":"2024-09-16T07:22:50.280281+0000","last_clean":"2024-09-16T07:22:50.280281+0000","last_became_active":"2024-09-16T07:22:48.139786+0000","last_became_peered":"2024-09-16T07:22:48.139786+0000","last_unstale":"2024-09-16T07:22:50.280281+0000","last_undegraded":"2024-09-16T07:22:50.280281+0000","last_fullsized":"2024-09-16T07:22:50.280281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280002+0000","last_change":"2024-09-16T07:22:49.135775+0000","last_active":"2024-09-16T07:22:50.280002+0000","last_peered":"2024-09-16T07:22:50.280002+0000","last_clean":"2024-09-16T07:22:50.280002+0000","last_became_active":"2024-09-16T07:22:49.135501+0000","last_became_peered":"2024-09-16T07:22:49.135501+0000","last_unstale":"2024-09-16T07:22:50.280002+0000","last_undegraded":"2024-09-16T07:22:50.280002+0000","last_fullsized":"2024-09-16T07:22:50.280002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:03:48.213796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132027+0000","last_change":"2024-09-16T07:22:48.130827+0000","last_active":"2024-09-16T07:22:50.132027+0000","last_peered":"2024-09-16T07:22:50.132027+0000","last_clean":"2024-09-16T07:22:50.132027+0000","last_became_active":"2024-09-16T07:22:48.129479+0000","last_became_peered":"2024-09-16T07:22:48.129479+0000","last_unstale":"2024-09-16T07:22:50.132027+0000","last_undegraded":"2024-09-16T07:22:50.132027+0000","last_fullsized":"2024-09-16T07:22:50.132027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:34.669437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139797+0000","last_change":"2024-09-16T07:22:49.141394+0000","last_active":"2024-09-16T07:22:50.139797+0000","last_peered":"2024-09-16T07:22:50.139797+0000","last_clean":"2024-09-16T07:22:50.139797+0000","last_became_active":"2024-09-16T07:22:49.141098+0000","last_became_peered":"2024-09-16T07:22:49.141098+0000","last_unstale":"2024-09-16T07:22:50.139797+0000","last_undegraded":"2024-09-16T07:22:50.139797+0000","last_fullsized":"2024-09-16T07:22:50.139797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:52.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279971+0000","last_change":"2024-09-16T07:22:48.137959+0000","last_active":"2024-09-16T07:22:50.279971+0000","last_peered":"2024-09-16T07:22:50.279971+0000","last_clean":"2024-09-16T07:22:50.279971+0000","last_became_active":"2024-09-16T07:22:48.137703+0000","last_became_peered":"2024-09-16T07:22:48.137703+0000","last_unstale":"2024-09-16T07:22:50.279971+0000","last_undegraded":"2024-09-16T07:22:50.279971+0000","last_fullsized":"2024-09-16T07:22:50.279971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279516+0000","last_change":"2024-09-16T07:22:49.133885+0000","last_active":"2024-09-16T07:22:50.279516+0000","last_peered":"2024-09-16T07:22:50.279516+0000","last_clean":"2024-09-16T07:22:50.279516+0000","last_became_active":"2024-09-16T07:22:49.133562+0000","last_became_peered":"2024-09-16T07:22:49.133562+0000","last_unstale":"2024-09-16T07:22:50.279516+0000","last_undegraded":"2024-09-16T07:22:50.279516+0000","last_fullsized":"2024-09-16T07:22:50.279516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:43.685739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279361+0000","last_change":"2024-09-16T07:22:48.137679+0000","last_active":"2024-09-16T07:22:50.279361+0000","last_peered":"2024-09-16T07:22:50.279361+0000","last_clean":"2024-09-16T07:22:50.279361+0000","last_became_active":"2024-09-16T07:22:48.134910+0000","last_became_peered":"2024-09-16T07:22:48.134910+0000","last_unstale":"2024-09-16T07:22:50.279361+0000","last_undegraded":"2024-09-16T07:22:50.279361+0000","last_fullsized":"2024-09-16T07:22:50.279361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:06.837305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279526+0000","last_change":"2024-09-16T07:22:49.131231+0000","last_active":"2024-09-16T07:22:50.279526+0000","last_peered":"2024-09-16T07:22:50.279526+0000","last_clean":"2024-09-16T07:22:50.279526+0000","last_became_active":"2024-09-16T07:22:49.130991+0000","last_became_peered":"2024-09-16T07:22:49.130991+0000","last_unstale":"2024-09-16T07:22:50.279526+0000","last_undegraded":"2024-09-16T07:22:50.279526+0000","last_fullsized":"2024-09-16T07:22:50.279526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:14:00.862069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139154+0000","last_change":"2024-09-16T07:22:48.139812+0000","last_active":"2024-09-16T07:22:50.139154+0000","last_peered":"2024-09-16T07:22:50.139154+0000","last_clean":"2024-09-16T07:22:50.139154+0000","last_became_active":"2024-09-16T07:22:48.139509+0000","last_became_peered":"2024-09-16T07:22:48.139509+0000","last_unstale":"2024-09-16T07:22:50.139154+0000","last_undegraded":"2024-09-16T07:22:50.139154+0000","last_fullsized":"2024-09-16T07:22:50.139154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:46.703518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278833+0000","last_change":"2024-09-16T07:22:49.143207+0000","last_active":"2024-09-16T07:22:50.278833+0000","last_peered":"2024-09-16T07:22:50.278833+0000","last_clean":"2024-09-16T07:22:50.278833+0000","last_became_active":"2024-09-16T07:22:49.139696+0000","last_became_peered":"2024-09-16T07:22:49.139696+0000","last_unstale":"2024-09-16T07:22:50.278833+0000","last_undegraded":"2024-09-16T07:22:50.278833+0000","last_fullsized":"2024-09-16T07:22:50.278833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130773+0000","last_change":"2024-09-16T07:22:48.134633+0000","last_active":"2024-09-16T07:22:50.130773+0000","last_peered":"2024-09-16T07:22:50.130773+0000","last_clean":"2024-09-16T07:22:50.130773+0000","last_became_active":"2024-09-16T07:22:48.134291+0000","last_became_peered":"2024-09-16T07:22:48.134291+0000","last_unstale":"2024-09-16T07:22:50.130773+0000","last_undegraded":"2024-09-16T07:22:50.130773+0000","last_fullsized":"2024-09-16T07:22:50.130773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:35.346741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278836+0000","last_change":"2024-09-16T07:22:49.127175+0000","last_active":"2024-09-16T07:22:50.278836+0000","last_peered":"2024-09-16T07:22:50.278836+0000","last_clean":"2024-09-16T07:22:50.278836+0000","last_became_active":"2024-09-16T07:22:49.126846+0000","last_became_peered":"2024-09-16T07:22:49.126846+0000","last_unstale":"2024-09-16T07:22:50.278836+0000","last_undegraded":"2024-09-16T07:22:50.278836+0000","last_fullsized":"2024-09-16T07:22:50.278836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:51.576491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139738+0000","last_change":"2024-09-16T07:22:48.138039+0000","last_active":"2024-09-16T07:22:50.139738+0000","last_peered":"2024-09-16T07:22:50.139738+0000","last_clean":"2024-09-16T07:22:50.139738+0000","last_became_active":"2024-09-16T07:22:48.137746+0000","last_became_peered":"2024-09-16T07:22:48.137746+0000","last_unstale":"2024-09-16T07:22:50.139738+0000","last_undegraded":"2024-09-16T07:22:50.139738+0000","last_fullsized":"2024-09-16T07:22:50.139738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:09.846796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132500+0000","last_change":"2024-09-16T07:22:49.132362+0000","last_active":"2024-09-16T07:22:50.132500+0000","last_peered":"2024-09-16T07:22:50.132500+0000","last_clean":"2024-09-16T07:22:50.132500+0000","last_became_active":"2024-09-16T07:22:49.131756+0000","last_became_peered":"2024-09-16T07:22:49.131756+0000","last_unstale":"2024-09-16T07:22:50.132500+0000","last_undegraded":"2024-09-16T07:22:50.132500+0000","last_fullsized":"2024-09-16T07:22:50.132500+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:43.685739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132541+0000","last_change":"2024-09-16T07:22:48.137778+0000","last_active":"2024-09-16T07:22:50.132541+0000","last_peered":"2024-09-16T07:22:50.132541+0000","last_clean":"2024-09-16T07:22:50.132541+0000","last_became_active":"2024-09-16T07:22:48.137159+0000","last_became_peered":"2024-09-16T07:22:48.137159+0000","last_unstale":"2024-09-16T07:22:50.132541+0000","last_undegraded":"2024-09-16T07:22:50.132541+0000","last_fullsized":"2024-09-16T07:22:50.132541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:37.703838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139626+0000","last_change":"2024-09-16T07:22:49.130481+0000","last_active":"2024-09-16T07:22:50.139626+0000","last_peered":"2024-09-16T07:22:50.139626+0000","last_clean":"2024-09-16T07:22:50.139626+0000","last_became_active":"2024-09-16T07:22:49.130181+0000","last_became_peered":"2024-09-16T07:22:49.130181+0000","last_unstale":"2024-09-16T07:22:50.139626+0000","last_undegraded":"2024-09-16T07:22:50.139626+0000","last_fullsized":"2024-09-16T07:22:50.139626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:41.565481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130923+0000","last_change":"2024-09-16T07:22:48.135490+0000","last_active":"2024-09-16T07:22:50.130923+0000","last_peered":"2024-09-16T07:22:50.130923+0000","last_clean":"2024-09-16T07:22:50.130923+0000","last_became_active":"2024-09-16T07:22:48.134343+0000","last_became_peered":"2024-09-16T07:22:48.134343+0000","last_unstale":"2024-09-16T07:22:50.130923+0000","last_undegraded":"2024-09-16T07:22:50.130923+0000","last_fullsized":"2024-09-16T07:22:50.130923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:45.020349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139776+0000","last_change":"2024-09-16T07:22:49.144669+0000","last_active":"2024-09-16T07:22:50.139776+0000","last_peered":"2024-09-16T07:22:50.139776+0000","last_clean":"2024-09-16T07:22:50.139776+0000","last_became_active":"2024-09-16T07:22:49.144415+0000","last_became_peered":"2024-09-16T07:22:49.144415+0000","last_unstale":"2024-09-16T07:22:50.139776+0000","last_undegraded":"2024-09-16T07:22:50.139776+0000","last_fullsized":"2024-09-16T07:22:50.139776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:51.576491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280181+0000","last_change":"2024-09-16T07:22:48.135506+0000","last_active":"2024-09-16T07:22:50.280181+0000","last_peered":"2024-09-16T07:22:50.280181+0000","last_clean":"2024-09-16T07:22:50.280181+0000","last_became_active":"2024-09-16T07:22:48.134852+0000","last_became_peered":"2024-09-16T07:22:48.134852+0000","last_unstale":"2024-09-16T07:22:50.280181+0000","last_undegraded":"2024-09-16T07:22:50.280181+0000","last_fullsized":"2024-09-16T07:22:50.280181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279526+0000","last_change":"2024-09-16T07:22:49.134414+0000","last_active":"2024-09-16T07:22:50.279526+0000","last_peered":"2024-09-16T07:22:50.279526+0000","last_clean":"2024-09-16T07:22:50.279526+0000","last_became_active":"2024-09-16T07:22:49.130571+0000","last_became_peered":"2024-09-16T07:22:49.130571+0000","last_unstale":"2024-09-16T07:22:50.279526+0000","last_undegraded":"2024-09-16T07:22:50.279526+0000","last_fullsized":"2024-09-16T07:22:50.279526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:02.699542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279112+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:22:50.279112+0000","last_peered":"2024-09-16T07:22:50.279112+0000","last_clean":"2024-09-16T07:22:50.279112+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:22:50.279112+0000","last_undegraded":"2024-09-16T07:22:50.279112+0000","last_fullsized":"2024-09-16T07:22:50.279112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279775+0000","last_change":"2024-09-16T07:22:48.139675+0000","last_active":"2024-09-16T07:22:50.279775+0000","last_peered":"2024-09-16T07:22:50.279775+0000","last_clean":"2024-09-16T07:22:50.279775+0000","last_became_active":"2024-09-16T07:22:48.137989+0000","last_became_peered":"2024-09-16T07:22:48.137989+0000","last_unstale":"2024-09-16T07:22:50.279775+0000","last_undegraded":"2024-09-16T07:22:50.279775+0000","last_fullsized":"2024-09-16T07:22:50.279775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:16.190625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132139+0000","last_change":"2024-09-16T07:22:49.137102+0000","last_active":"2024-09-16T07:22:50.132139+0000","last_peered":"2024-09-16T07:22:50.132139+0000","last_clean":"2024-09-16T07:22:50.132139+0000","last_became_active":"2024-09-16T07:22:49.136755+0000","last_became_peered":"2024-09-16T07:22:49.136755+0000","last_unstale":"2024-09-16T07:22:50.132139+0000","last_undegraded":"2024-09-16T07:22:50.132139+0000","last_fullsized":"2024-09-16T07:22:50.132139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:26.935160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278962+0000","last_change":"2024-09-16T07:22:48.133273+0000","last_active":"2024-09-16T07:22:50.278962+0000","last_peered":"2024-09-16T07:22:50.278962+0000","last_clean":"2024-09-16T07:22:50.278962+0000","last_became_active":"2024-09-16T07:22:48.132984+0000","last_became_peered":"2024-09-16T07:22:48.132984+0000","last_unstale":"2024-09-16T07:22:50.278962+0000","last_undegraded":"2024-09-16T07:22:50.278962+0000","last_fullsized":"2024-09-16T07:22:50.278962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.657246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279239+0000","last_change":"2024-09-16T07:22:49.134936+0000","last_active":"2024-09-16T07:22:50.279239+0000","last_peered":"2024-09-16T07:22:50.279239+0000","last_clean":"2024-09-16T07:22:50.279239+0000","last_became_active":"2024-09-16T07:22:49.134575+0000","last_became_peered":"2024-09-16T07:22:49.134575+0000","last_unstale":"2024-09-16T07:22:50.279239+0000","last_undegraded":"2024-09-16T07:22:50.279239+0000","last_fullsized":"2024-09-16T07:22:50.279239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279457+0000","last_change":"2024-09-16T07:22:48.140454+0000","last_active":"2024-09-16T07:22:50.279457+0000","last_peered":"2024-09-16T07:22:50.279457+0000","last_clean":"2024-09-16T07:22:50.279457+0000","last_became_active":"2024-09-16T07:22:48.140025+0000","last_became_peered":"2024-09-16T07:22:48.140025+0000","last_unstale":"2024-09-16T07:22:50.279457+0000","last_undegraded":"2024-09-16T07:22:50.279457+0000","last_fullsized":"2024-09-16T07:22:50.279457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130545+0000","last_change":"2024-09-16T07:22:49.138294+0000","last_active":"2024-09-16T07:22:50.130545+0000","last_peered":"2024-09-16T07:22:50.130545+0000","last_clean":"2024-09-16T07:22:50.130545+0000","last_became_active":"2024-09-16T07:22:49.137685+0000","last_became_peered":"2024-09-16T07:22:49.137685+0000","last_unstale":"2024-09-16T07:22:50.130545+0000","last_undegraded":"2024-09-16T07:22:50.130545+0000","last_fullsized":"2024-09-16T07:22:50.130545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:51.068492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279367+0000","last_change":"2024-09-16T07:22:48.139380+0000","last_active":"2024-09-16T07:22:50.279367+0000","last_peered":"2024-09-16T07:22:50.279367+0000","last_clean":"2024-09-16T07:22:50.279367+0000","last_became_active":"2024-09-16T07:22:48.138052+0000","last_became_peered":"2024-09-16T07:22:48.138052+0000","last_unstale":"2024-09-16T07:22:50.279367+0000","last_undegraded":"2024-09-16T07:22:50.279367+0000","last_fullsized":"2024-09-16T07:22:50.279367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:28.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139289+0000","last_change":"2024-09-16T07:22:49.133663+0000","last_active":"2024-09-16T07:22:50.139289+0000","last_peered":"2024-09-16T07:22:50.139289+0000","last_clean":"2024-09-16T07:22:50.139289+0000","last_became_active":"2024-09-16T07:22:49.133342+0000","last_became_peered":"2024-09-16T07:22:49.133342+0000","last_unstale":"2024-09-16T07:22:50.139289+0000","last_undegraded":"2024-09-16T07:22:50.139289+0000","last_fullsized":"2024-09-16T07:22:50.139289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:04.490402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130957+0000","last_change":"2024-09-16T07:22:48.141538+0000","last_active":"2024-09-16T07:22:50.130957+0000","last_peered":"2024-09-16T07:22:50.130957+0000","last_clean":"2024-09-16T07:22:50.130957+0000","last_became_active":"2024-09-16T07:22:48.138174+0000","last_became_peered":"2024-09-16T07:22:48.138174+0000","last_unstale":"2024-09-16T07:22:50.130957+0000","last_undegraded":"2024-09-16T07:22:50.130957+0000","last_fullsized":"2024-09-16T07:22:50.130957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:29.651356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132217+0000","last_change":"2024-09-16T07:22:49.136727+0000","last_active":"2024-09-16T07:22:50.132217+0000","last_peered":"2024-09-16T07:22:50.132217+0000","last_clean":"2024-09-16T07:22:50.132217+0000","last_became_active":"2024-09-16T07:22:49.136064+0000","last_became_peered":"2024-09-16T07:22:49.136064+0000","last_unstale":"2024-09-16T07:22:50.132217+0000","last_undegraded":"2024-09-16T07:22:50.132217+0000","last_fullsized":"2024-09-16T07:22:50.132217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279830+0000","last_change":"2024-09-16T07:22:48.135598+0000","last_active":"2024-09-16T07:22:50.279830+0000","last_peered":"2024-09-16T07:22:50.279830+0000","last_clean":"2024-09-16T07:22:50.279830+0000","last_became_active":"2024-09-16T07:22:48.135179+0000","last_became_peered":"2024-09-16T07:22:48.135179+0000","last_unstale":"2024-09-16T07:22:50.279830+0000","last_undegraded":"2024-09-16T07:22:50.279830+0000","last_fullsized":"2024-09-16T07:22:50.279830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:09.846796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280111+0000","last_change":"2024-09-16T07:22:49.137714+0000","last_active":"2024-09-16T07:22:50.280111+0000","last_peered":"2024-09-16T07:22:50.280111+0000","last_clean":"2024-09-16T07:22:50.280111+0000","last_became_active":"2024-09-16T07:22:49.134740+0000","last_became_peered":"2024-09-16T07:22:49.134740+0000","last_unstale":"2024-09-16T07:22:50.280111+0000","last_undegraded":"2024-09-16T07:22:50.280111+0000","last_fullsized":"2024-09-16T07:22:50.280111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:44.194572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138704+0000","last_change":"2024-09-16T07:22:48.137476+0000","last_active":"2024-09-16T07:22:50.138704+0000","last_peered":"2024-09-16T07:22:50.138704+0000","last_clean":"2024-09-16T07:22:50.138704+0000","last_became_active":"2024-09-16T07:22:48.136772+0000","last_became_peered":"2024-09-16T07:22:48.136772+0000","last_unstale":"2024-09-16T07:22:50.138704+0000","last_undegraded":"2024-09-16T07:22:50.138704+0000","last_fullsized":"2024-09-16T07:22:50.138704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.906034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132443+0000","last_change":"2024-09-16T07:22:49.144331+0000","last_active":"2024-09-16T07:22:50.132443+0000","last_peered":"2024-09-16T07:22:50.132443+0000","last_clean":"2024-09-16T07:22:50.132443+0000","last_became_active":"2024-09-16T07:22:49.144071+0000","last_became_peered":"2024-09-16T07:22:49.144071+0000","last_unstale":"2024-09-16T07:22:50.132443+0000","last_undegraded":"2024-09-16T07:22:50.132443+0000","last_fullsized":"2024-09-16T07:22:50.132443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.753793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280211+0000","last_change":"2024-09-16T07:22:48.132847+0000","last_active":"2024-09-16T07:22:50.280211+0000","last_peered":"2024-09-16T07:22:50.280211+0000","last_clean":"2024-09-16T07:22:50.280211+0000","last_became_active":"2024-09-16T07:22:48.132264+0000","last_became_peered":"2024-09-16T07:22:48.132264+0000","last_unstale":"2024-09-16T07:22:50.280211+0000","last_undegraded":"2024-09-16T07:22:50.280211+0000","last_fullsized":"2024-09-16T07:22:50.280211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:42.680089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:49.138949+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:49.138580+0000","last_became_peered":"2024-09-16T07:22:49.138580+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:30.446387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138995+0000","last_change":"2024-09-16T07:22:48.129412+0000","last_active":"2024-09-16T07:22:50.138995+0000","last_peered":"2024-09-16T07:22:50.138995+0000","last_clean":"2024-09-16T07:22:50.138995+0000","last_became_active":"2024-09-16T07:22:48.129137+0000","last_became_peered":"2024-09-16T07:22:48.129137+0000","last_unstale":"2024-09-16T07:22:50.138995+0000","last_undegraded":"2024-09-16T07:22:50.138995+0000","last_fullsized":"2024-09-16T07:22:50.138995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:07.509813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279051+0000","last_change":"2024-09-16T07:22:49.129827+0000","last_active":"2024-09-16T07:22:50.279051+0000","last_peered":"2024-09-16T07:22:50.279051+0000","last_clean":"2024-09-16T07:22:50.279051+0000","last_became_active":"2024-09-16T07:22:49.129474+0000","last_became_peered":"2024-09-16T07:22:49.129474+0000","last_unstale":"2024-09-16T07:22:50.279051+0000","last_undegraded":"2024-09-16T07:22:50.279051+0000","last_fullsized":"2024-09-16T07:22:50.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:57.650915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130669+0000","last_change":"2024-09-16T07:22:48.134437+0000","last_active":"2024-09-16T07:22:50.130669+0000","last_peered":"2024-09-16T07:22:50.130669+0000","last_clean":"2024-09-16T07:22:50.130669+0000","last_became_active":"2024-09-16T07:22:48.134144+0000","last_became_peered":"2024-09-16T07:22:48.134144+0000","last_unstale":"2024-09-16T07:22:50.130669+0000","last_undegraded":"2024-09-16T07:22:50.130669+0000","last_fullsized":"2024-09-16T07:22:50.130669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:04.051560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130660+0000","last_change":"2024-09-16T07:22:49.141869+0000","last_active":"2024-09-16T07:22:50.130660+0000","last_peered":"2024-09-16T07:22:50.130660+0000","last_clean":"2024-09-16T07:22:50.130660+0000","last_became_active":"2024-09-16T07:22:49.141602+0000","last_became_peered":"2024-09-16T07:22:49.141602+0000","last_unstale":"2024-09-16T07:22:50.130660+0000","last_undegraded":"2024-09-16T07:22:50.130660+0000","last_fullsized":"2024-09-16T07:22:50.130660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:44.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279669+0000","last_change":"2024-09-16T07:22:48.137499+0000","last_active":"2024-09-16T07:22:50.279669+0000","last_peered":"2024-09-16T07:22:50.279669+0000","last_clean":"2024-09-16T07:22:50.279669+0000","last_became_active":"2024-09-16T07:22:48.137177+0000","last_became_peered":"2024-09-16T07:22:48.137177+0000","last_unstale":"2024-09-16T07:22:50.279669+0000","last_undegraded":"2024-09-16T07:22:50.279669+0000","last_fullsized":"2024-09-16T07:22:50.279669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280080+0000","last_change":"2024-09-16T07:22:49.143083+0000","last_active":"2024-09-16T07:22:50.280080+0000","last_peered":"2024-09-16T07:22:50.280080+0000","last_clean":"2024-09-16T07:22:50.280080+0000","last_became_active":"2024-09-16T07:22:49.142817+0000","last_became_peered":"2024-09-16T07:22:49.142817+0000","last_unstale":"2024-09-16T07:22:50.280080+0000","last_undegraded":"2024-09-16T07:22:50.280080+0000","last_fullsized":"2024-09-16T07:22:50.280080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279682+0000","last_change":"2024-09-16T07:22:48.131412+0000","last_active":"2024-09-16T07:22:50.279682+0000","last_peered":"2024-09-16T07:22:50.279682+0000","last_clean":"2024-09-16T07:22:50.279682+0000","last_became_active":"2024-09-16T07:22:48.131167+0000","last_became_peered":"2024-09-16T07:22:48.131167+0000","last_unstale":"2024-09-16T07:22:50.279682+0000","last_undegraded":"2024-09-16T07:22:50.279682+0000","last_fullsized":"2024-09-16T07:22:50.279682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:40.642232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279172+0000","last_change":"2024-09-16T07:22:49.129812+0000","last_active":"2024-09-16T07:22:50.279172+0000","last_peered":"2024-09-16T07:22:50.279172+0000","last_clean":"2024-09-16T07:22:50.279172+0000","last_became_active":"2024-09-16T07:22:49.129511+0000","last_became_peered":"2024-09-16T07:22:49.129511+0000","last_unstale":"2024-09-16T07:22:50.279172+0000","last_undegraded":"2024-09-16T07:22:50.279172+0000","last_fullsized":"2024-09-16T07:22:50.279172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:12.678514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279340+0000","last_change":"2024-09-16T07:22:49.137458+0000","last_active":"2024-09-16T07:22:50.279340+0000","last_peered":"2024-09-16T07:22:50.279340+0000","last_clean":"2024-09-16T07:22:50.279340+0000","last_became_active":"2024-09-16T07:22:49.137122+0000","last_became_peered":"2024-09-16T07:22:49.137122+0000","last_unstale":"2024-09-16T07:22:50.279340+0000","last_undegraded":"2024-09-16T07:22:50.279340+0000","last_fullsized":"2024-09-16T07:22:50.279340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:52.744207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139907+0000","last_change":"2024-09-16T07:22:48.130394+0000","last_active":"2024-09-16T07:22:50.139907+0000","last_peered":"2024-09-16T07:22:50.139907+0000","last_clean":"2024-09-16T07:22:50.139907+0000","last_became_active":"2024-09-16T07:22:48.130013+0000","last_became_peered":"2024-09-16T07:22:48.130013+0000","last_unstale":"2024-09-16T07:22:50.139907+0000","last_undegraded":"2024-09-16T07:22:50.139907+0000","last_fullsized":"2024-09-16T07:22:50.139907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.657246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279527+0000","last_change":"2024-09-16T07:22:49.136594+0000","last_active":"2024-09-16T07:22:50.279527+0000","last_peered":"2024-09-16T07:22:50.279527+0000","last_clean":"2024-09-16T07:22:50.279527+0000","last_became_active":"2024-09-16T07:22:49.136266+0000","last_became_peered":"2024-09-16T07:22:49.136266+0000","last_unstale":"2024-09-16T07:22:50.279527+0000","last_undegraded":"2024-09-16T07:22:50.279527+0000","last_fullsized":"2024-09-16T07:22:50.279527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139950+0000","last_change":"2024-09-16T07:22:48.135403+0000","last_active":"2024-09-16T07:22:50.139950+0000","last_peered":"2024-09-16T07:22:50.139950+0000","last_clean":"2024-09-16T07:22:50.139950+0000","last_became_active":"2024-09-16T07:22:48.135142+0000","last_became_peered":"2024-09-16T07:22:48.135142+0000","last_unstale":"2024-09-16T07:22:50.139950+0000","last_undegraded":"2024-09-16T07:22:50.139950+0000","last_fullsized":"2024-09-16T07:22:50.139950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:58.988982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.133007+0000","last_change":"2024-09-16T07:22:49.129289+0000","last_active":"2024-09-16T07:22:50.133007+0000","last_peered":"2024-09-16T07:22:50.133007+0000","last_clean":"2024-09-16T07:22:50.133007+0000","last_became_active":"2024-09-16T07:22:49.129000+0000","last_became_peered":"2024-09-16T07:22:49.129000+0000","last_unstale":"2024-09-16T07:22:50.133007+0000","last_undegraded":"2024-09-16T07:22:50.133007+0000","last_fullsized":"2024-09-16T07:22:50.133007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:31.607464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:48.132860+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:48.132544+0000","last_became_peered":"2024-09-16T07:22:48.132544+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:52:05.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132482+0000","last_change":"2024-09-16T07:22:49.142356+0000","last_active":"2024-09-16T07:22:50.132482+0000","last_peered":"2024-09-16T07:22:50.132482+0000","last_clean":"2024-09-16T07:22:50.132482+0000","last_became_active":"2024-09-16T07:22:49.141639+0000","last_became_peered":"2024-09-16T07:22:49.141639+0000","last_unstale":"2024-09-16T07:22:50.132482+0000","last_undegraded":"2024-09-16T07:22:50.132482+0000","last_fullsized":"2024-09-16T07:22:50.132482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:57.949375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132622+0000","last_change":"2024-09-16T07:22:48.141242+0000","last_active":"2024-09-16T07:22:50.132622+0000","last_peered":"2024-09-16T07:22:50.132622+0000","last_clean":"2024-09-16T07:22:50.132622+0000","last_became_active":"2024-09-16T07:22:48.139871+0000","last_became_peered":"2024-09-16T07:22:48.139871+0000","last_unstale":"2024-09-16T07:22:50.132622+0000","last_undegraded":"2024-09-16T07:22:50.132622+0000","last_fullsized":"2024-09-16T07:22:50.132622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:32.916879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476743,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":81289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476743,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":671145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":80763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":80831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":80707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":831488,"data_stored":671167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:22:58.394 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:22:58.395 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-16T07:22:58.395 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T07:22:58.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 -- 172.21.15.71:0/3912413114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ccf00 msgr2=0x7fac4c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 --2- 172.21.15.71:0/3912413114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ccf00 0x7fac4c0acd30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fac4c076720 tx=0x7fac3802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 -- 172.21.15.71:0/3912413114 shutdown_connections 2024-09-16T07:22:58.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 --2- 172.21.15.71:0/3912413114 >> v2:172.21.15.177:3300/0 conn(0x7fac4c0ad270 0x7fac4c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 --2- 172.21.15.71:0/3912413114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ccf00 0x7fac4c0acd30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 --2- 172.21.15.71:0/3912413114 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac4c15f5a0 0x7fac4c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 -- 172.21.15.71:0/3912413114 >> 172.21.15.71:0/3912413114 conn(0x7fac4c0b1950 msgr2=0x7fac4c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 -- 172.21.15.71:0/3912413114 shutdown_connections 2024-09-16T07:22:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.587+0000 7fac53000640 1 -- 172.21.15.71:0/3912413114 wait complete. 2024-09-16T07:22:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.589+0000 7fac53000640 1 Processor -- start 2024-09-16T07:22:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.589+0000 7fac53000640 1 -- start start 2024-09-16T07:22:58.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac53000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ad270 0x7fac4c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac53000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac4c0ccf00 0x7fac4c14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac53000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac4c15f5a0 0x7fac4c14e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac53000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c06f160 con 0x7fac4c0ad270 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac53000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c074780 con 0x7fac4c15f5a0 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac53000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fac4c06f2e0 con 0x7fac4c0ccf00 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ad270 0x7fac4c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ad270 0x7fac4c14b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50566/0 (socket says 172.21.15.71:50566) 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4be00640 1 -- 172.21.15.71:0/352872068 learned_addr learned my addr 172.21.15.71:0/352872068 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac51000640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac4c15f5a0 0x7fac4c14e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4b400640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac4c0ccf00 0x7fac4c14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4be00640 1 -- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac4c0ccf00 msgr2=0x7fac4c14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4be00640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac4c0ccf00 0x7fac4c14ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4be00640 1 -- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac4c15f5a0 msgr2=0x7fac4c14e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4be00640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac4c15f5a0 0x7fac4c14e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4be00640 1 -- 172.21.15.71:0/352872068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c0cff70 con 0x7fac4c0ad270 2024-09-16T07:22:58.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac4b400640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac4c0ccf00 0x7fac4c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:58.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.590+0000 7fac51000640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac4c15f5a0 0x7fac4c14e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.591+0000 7fac4be00640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ad270 0x7fac4c14b480 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00c9a0 tx=0x7fac3c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.591+0000 7fac43e00640 1 -- 172.21.15.71:0/352872068 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c007bb0 con 0x7fac4c0ad270 2024-09-16T07:22:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.591+0000 7fac43e00640 1 -- 172.21.15.71:0/352872068 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac3c007e40 con 0x7fac4c0ad270 2024-09-16T07:22:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.591+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c0d0250 con 0x7fac4c0ad270 2024-09-16T07:22:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.591+0000 7fac43e00640 1 -- 172.21.15.71:0/352872068 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c00f690 con 0x7fac4c0ad270 2024-09-16T07:22:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.591+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c14c1c0 con 0x7fac4c0ad270 2024-09-16T07:22:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.592+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4c0d0490 con 0x7fac4c0ad270 2024-09-16T07:22:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.593+0000 7fac43e00640 1 -- 172.21.15.71:0/352872068 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fac3c00f830 con 0x7fac4c0ad270 2024-09-16T07:22:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.594+0000 7fac43e00640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fac2c078850 0x7fac2c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.594+0000 7fac43e00640 1 -- 172.21.15.71:0/352872068 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fac3c09b0f0 con 0x7fac4c0ad270 2024-09-16T07:22:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.595+0000 7fac4b400640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fac2c078850 0x7fac2c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.595+0000 7fac4b400640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fac2c078850 0x7fac2c07ad10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac38033000 tx=0x7fac38004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.598+0000 7fac43e00640 1 -- 172.21.15.71:0/352872068 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fac4c0d0490 con 0x7fac4c0ad270 2024-09-16T07:22:58.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.783+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fac4c06f870 con 0x7fac4c0ad270 2024-09-16T07:22:58.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.784+0000 7fac43e00640 1 -- 172.21.15.71:0/352872068 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7fac3c014030 con 0x7fac4c0ad270 2024-09-16T07:22:58.783 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:58.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fac2c078850 msgr2=0x7fac2c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fac2c078850 0x7fac2c07ad10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac38033000 tx=0x7fac38004730 comp rx=0 tx=0).stop 2024-09-16T07:22:58.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ad270 msgr2=0x7fac4c14b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ad270 0x7fac4c14b480 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00c9a0 tx=0x7fac3c00ce60 comp rx=0 tx=0).stop 2024-09-16T07:22:58.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 shutdown_connections 2024-09-16T07:22:58.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fac2c078850 0x7fac2c07ad10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac4c15f5a0 0x7fac4c14e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac4c0ccf00 0x7fac4c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 --2- 172.21.15.71:0/352872068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4c0ad270 0x7fac4c14b480 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 >> 172.21.15.71:0/352872068 conn(0x7fac4c0b1950 msgr2=0x7fac4c09d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:58.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 shutdown_connections 2024-09-16T07:22:58.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.786+0000 7fac53000640 1 -- 172.21.15.71:0/352872068 wait complete. 2024-09-16T07:22:58.801 INFO:teuthology.orchestra.run.smithi071.stdout:[{"version":27,"timestamp":"2024-09-16T07:22:36.940503+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"3785.239405"}]}] 2024-09-16T07:22:58.801 INFO:tasks.ceph_manager:config epoch is 27 2024-09-16T07:22:58.801 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T07:22:58.801 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T07:22:58.802 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T07:22:58.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.958+0000 7f8334000640 1 -- 172.21.15.71:0/2208886415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 msgr2=0x7f832c074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.958+0000 7f8334000640 1 --2- 172.21.15.71:0/2208886415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 0x7f832c074460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8324005fd0 tx=0x7f832402aa10 comp rx=0 tx=0).stop 2024-09-16T07:22:58.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.958+0000 7f8334000640 1 -- 172.21.15.71:0/2208886415 shutdown_connections 2024-09-16T07:22:58.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.958+0000 7f8334000640 1 --2- 172.21.15.71:0/2208886415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 0x7f832c074460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.958+0000 7f8334000640 1 --2- 172.21.15.71:0/2208886415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832c15f5a0 0x7f832c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.958+0000 7f8334000640 1 -- 172.21.15.71:0/2208886415 >> 172.21.15.71:0/2208886415 conn(0x7f832c0b1950 msgr2=0x7f832c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:58.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.958+0000 7f8334000640 1 -- 172.21.15.71:0/2208886415 shutdown_connections 2024-09-16T07:22:58.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.958+0000 7f8334000640 1 -- 172.21.15.71:0/2208886415 wait complete. 2024-09-16T07:22:58.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.960+0000 7f8334000640 1 Processor -- start 2024-09-16T07:22:58.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.960+0000 7f8334000640 1 -- start start 2024-09-16T07:22:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8334000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 0x7f832c1f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8334000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832c0ccf00 0x7f832c0cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8334000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832c15f5a0 0x7f832c0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8334000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f832c06f2e0 con 0x7f832c073fb0 2024-09-16T07:22:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8334000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f832c06f160 con 0x7f832c15f5a0 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8334000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f832c06f460 con 0x7f832c0ccf00 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8331600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 0x7f832c1f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8331600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 0x7f832c1f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50588/0 (socket says 172.21.15.71:50588) 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8331600640 1 -- 172.21.15.71:0/4218145457 learned_addr learned my addr 172.21.15.71:0/4218145457 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8330c00640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832c0ccf00 0x7f832c0cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8332000640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832c15f5a0 0x7f832c0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8331600640 1 -- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832c0ccf00 msgr2=0x7f832c0cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8331600640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832c0ccf00 0x7f832c0cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8331600640 1 -- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832c15f5a0 msgr2=0x7f832c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8331600640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832c15f5a0 0x7f832c0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8331600640 1 -- 172.21.15.71:0/4218145457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c1f5990 con 0x7f832c073fb0 2024-09-16T07:22:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.961+0000 7f8332000640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832c15f5a0 0x7f832c0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:22:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.962+0000 7f8331600640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 0x7f832c1f5450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f832000d6d0 tx=0x7f832000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.962+0000 7f8329600640 1 -- 172.21.15.71:0/4218145457 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320017070 con 0x7f832c073fb0 2024-09-16T07:22:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.962+0000 7f8329600640 1 -- 172.21.15.71:0/4218145457 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8320005330 con 0x7f832c073fb0 2024-09-16T07:22:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.962+0000 7f8329600640 1 -- 172.21.15.71:0/4218145457 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320009820 con 0x7f832c073fb0 2024-09-16T07:22:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.962+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c0cee60 con 0x7f832c073fb0 2024-09-16T07:22:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.962+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c0cf420 con 0x7f832c073fb0 2024-09-16T07:22:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.963+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832c06f9f0 con 0x7f832c073fb0 2024-09-16T07:22:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.964+0000 7f8329600640 1 -- 172.21.15.71:0/4218145457 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8320009bd0 con 0x7f832c073fb0 2024-09-16T07:22:58.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.965+0000 7f8329600640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f830c078880 0x7f830c07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:58.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.965+0000 7f8329600640 1 -- 172.21.15.71:0/4218145457 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f832009aca0 con 0x7f832c073fb0 2024-09-16T07:22:58.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.965+0000 7f8330c00640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f830c078880 0x7f830c07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:58.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.966+0000 7f8330c00640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f830c078880 0x7f830c07ad40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f832c1565c0 tx=0x7f8318009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:58.969+0000 7f8329600640 1 -- 172.21.15.71:0/4218145457 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8320063270 con 0x7f832c073fb0 2024-09-16T07:22:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.201+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f832c09dea0 con 0x7f832c073fb0 2024-09-16T07:22:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.204+0000 7f8329600640 1 -- 172.21.15.71:0/4218145457 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194345 (secure 0 0 0) 0x7f8320062b70 con 0x7f832c073fb0 2024-09-16T07:22:59.206 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:59.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f830c078880 msgr2=0x7f830c07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f830c078880 0x7f830c07ad40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f832c1565c0 tx=0x7f8318009040 comp rx=0 tx=0).stop 2024-09-16T07:22:59.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 msgr2=0x7f832c1f5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 0x7f832c1f5450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f832000d6d0 tx=0x7f832000db90 comp rx=0 tx=0).stop 2024-09-16T07:22:59.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 shutdown_connections 2024-09-16T07:22:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f830c078880 0x7f830c07ad40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f832c15f5a0 0x7f832c0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832c0ccf00 0x7f832c0cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 --2- 172.21.15.71:0/4218145457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c073fb0 0x7f832c1f5450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.207+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 >> 172.21.15.71:0/4218145457 conn(0x7f832c0b1950 msgr2=0x7f832c09ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.208+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 shutdown_connections 2024-09-16T07:22:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.208+0000 7f8334000640 1 -- 172.21.15.71:0/4218145457 wait complete. 2024-09-16T07:22:59.223 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":5,"flags":0,"active_gid":4120,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":143959246},{"type":"v1","addr":"172.21.15.177:6801","nonce":143959246}]},"active_addr":"172.21.15.177:6801/143959246","active_change":"2024-09-16T07:22:28.282582+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4131,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2311908324}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3578962374}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3549818430}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3989964090}]}]} 2024-09-16T07:22:59.227 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T07:22:59.227 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T07:22:59.227 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:59.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.388+0000 7f4cc4400640 1 -- 172.21.15.71:0/976288763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc073fb0 msgr2=0x7f4cbc074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.388+0000 7f4cc4400640 1 --2- 172.21.15.71:0/976288763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc073fb0 0x7f4cbc074440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4005fd0 tx=0x7f4cb402a820 comp rx=0 tx=0).stop 2024-09-16T07:22:59.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.388+0000 7f4cc4400640 1 -- 172.21.15.71:0/976288763 shutdown_connections 2024-09-16T07:22:59.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.388+0000 7f4cc4400640 1 --2- 172.21.15.71:0/976288763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc073fb0 0x7f4cbc074440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.388+0000 7f4cc4400640 1 --2- 172.21.15.71:0/976288763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cbc0ccf00 0x7f4cbc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.388+0000 7f4cc4400640 1 -- 172.21.15.71:0/976288763 >> 172.21.15.71:0/976288763 conn(0x7f4cbc0b1950 msgr2=0x7f4cbc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:59.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.388+0000 7f4cc4400640 1 -- 172.21.15.71:0/976288763 shutdown_connections 2024-09-16T07:22:59.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.388+0000 7f4cc4400640 1 -- 172.21.15.71:0/976288763 wait complete. 2024-09-16T07:22:59.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.390+0000 7f4cc4400640 1 Processor -- start 2024-09-16T07:22:59.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.390+0000 7f4cc4400640 1 -- start start 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc4400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc073fb0 0x7f4cbc0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc4400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cbc0ccf00 0x7f4cbc1f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc4400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc15f5a0 0x7f4cbc153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc4400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc06f2e0 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc4400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc06f160 con 0x7f4cbc073fb0 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc4400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4cbc06f460 con 0x7f4cbc0ccf00 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc15f5a0 0x7f4cbc153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc15f5a0 0x7f4cbc153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50592/0 (socket says 172.21.15.71:50592) 2024-09-16T07:22:59.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc2400640 1 -- 172.21.15.71:0/3219370583 learned_addr learned my addr 172.21.15.71:0/3219370583 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc1000640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cbc0ccf00 0x7f4cbc1f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc1a00640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc073fb0 0x7f4cbc0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc2400640 1 -- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cbc0ccf00 msgr2=0x7f4cbc1f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc2400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cbc0ccf00 0x7f4cbc1f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc2400640 1 -- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc073fb0 msgr2=0x7f4cbc0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc2400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc073fb0 0x7f4cbc0cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc2400640 1 -- 172.21.15.71:0/3219370583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cbc1f8550 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.391+0000 7f4cc1a00640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc073fb0 0x7f4cbc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:59.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.392+0000 7f4cc2400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc15f5a0 0x7f4cbc153e50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009cd0 tx=0x7f4cb4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:59.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.392+0000 7f4cb9600640 1 -- 172.21.15.71:0/3219370583 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4037040 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.392+0000 7f4cb9600640 1 -- 172.21.15.71:0/3219370583 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cb4042070 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.392+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cbc1f8830 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.392+0000 7f4cb9600640 1 -- 172.21.15.71:0/3219370583 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4003e60 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.392+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cbc0b3ff0 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.393+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cbc0b3e10 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.394+0000 7f4cb9600640 1 -- 172.21.15.71:0/3219370583 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4cb4004260 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.395+0000 7f4cb9600640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c9c078850 0x7f4c9c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.395+0000 7f4cb9600640 1 -- 172.21.15.71:0/3219370583 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4cb40bb540 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.396+0000 7f4cc1a00640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c9c078850 0x7f4c9c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.396+0000 7f4cc1a00640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c9c078850 0x7f4c9c07ad10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0007a50 tx=0x7f4cb0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:59.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.400+0000 7f4cb9600640 1 -- 172.21.15.71:0/3219370583 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4cbc0b3e10 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.584+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cbc06f9f0 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.586+0000 7f4cb9600640 1 -- 172.21.15.71:0/3219370583 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7f4cb4032450 con 0x7f4cbc15f5a0 2024-09-16T07:22:59.585 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:59.585 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":23,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:50.122046+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c9c078850 msgr2=0x7f4c9c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c9c078850 0x7f4c9c07ad10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0007a50 tx=0x7f4cb0009040 comp rx=0 tx=0).stop 2024-09-16T07:22:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc15f5a0 msgr2=0x7f4cbc153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc15f5a0 0x7f4cbc153e50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009cd0 tx=0x7f4cb4032d10 comp rx=0 tx=0).stop 2024-09-16T07:22:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 shutdown_connections 2024-09-16T07:22:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4c9c078850 0x7f4c9c07ad10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cbc15f5a0 0x7f4cbc153e50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cbc0ccf00 0x7f4cbc1f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 --2- 172.21.15.71:0/3219370583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc073fb0 0x7f4cbc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 >> 172.21.15.71:0/3219370583 conn(0x7f4cbc0b1950 msgr2=0x7f4cbc15e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:59.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 shutdown_connections 2024-09-16T07:22:59.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.588+0000 7f4cc4400640 1 -- 172.21.15.71:0/3219370583 wait complete. 2024-09-16T07:22:59.604 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T07:22:59.604 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:22:59.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.761+0000 7f18df200640 1 -- 172.21.15.71:0/3107505585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 msgr2=0x7f18e00d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.761+0000 7f18df200640 1 --2- 172.21.15.71:0/3107505585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 0x7f18e00d22a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f18d8005fd0 tx=0x7f18d802ab40 comp rx=0 tx=0).stop 2024-09-16T07:22:59.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.761+0000 7f18df200640 1 -- 172.21.15.71:0/3107505585 shutdown_connections 2024-09-16T07:22:59.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.761+0000 7f18df200640 1 --2- 172.21.15.71:0/3107505585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 0x7f18e00d22a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.761+0000 7f18df200640 1 -- 172.21.15.71:0/3107505585 >> 172.21.15.71:0/3107505585 conn(0x7f18e00d3de0 msgr2=0x7f18e00d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:59.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.762+0000 7f18df200640 1 -- 172.21.15.71:0/3107505585 shutdown_connections 2024-09-16T07:22:59.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.762+0000 7f18df200640 1 -- 172.21.15.71:0/3107505585 wait complete. 2024-09-16T07:22:59.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.763+0000 7f18df200640 1 Processor -- start 2024-09-16T07:22:59.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.763+0000 7f18df200640 1 -- start start 2024-09-16T07:22:59.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.764+0000 7f18df200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b28d0 0x7f18e0155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:59.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.764+0000 7f18df200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 0x7f18e00cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:59.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.764+0000 7f18df200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00d1ec0 0x7f18e00cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:59.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.764+0000 7f18df200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01692e0 con 0x7f18e00b4d10 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.764+0000 7f18df200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18e00b4a50 con 0x7f18e00d1ec0 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.764+0000 7f18df200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f18e0169460 con 0x7f18e00b28d0 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.764+0000 7f18d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 0x7f18e00cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 0x7f18e00cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50616/0 (socket says 172.21.15.71:50616) 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d7e00640 1 -- 172.21.15.71:0/1095161368 learned_addr learned my addr 172.21.15.71:0/1095161368 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18de800640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00d1ec0 0x7f18e00cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18dde00640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b28d0 0x7f18e0155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d7e00640 1 -- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b28d0 msgr2=0x7f18e0155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d7e00640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b28d0 0x7f18e0155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d7e00640 1 -- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00d1ec0 msgr2=0x7f18e00cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d7e00640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00d1ec0 0x7f18e00cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d7e00640 1 -- 172.21.15.71:0/1095161368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e0157f90 con 0x7f18e00b4d10 2024-09-16T07:22:59.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18de800640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00d1ec0 0x7f18e00cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:22:59.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d7e00640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 0x7f18e00cbf00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f18c800c450 tx=0x7f18c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:59.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d6000640 1 -- 172.21.15.71:0/1095161368 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c80052d0 con 0x7f18e00b4d10 2024-09-16T07:22:59.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d6000640 1 -- 172.21.15.71:0/1095161368 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18c8005470 con 0x7f18e00b4d10 2024-09-16T07:22:59.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.765+0000 7f18d6000640 1 -- 172.21.15.71:0/1095161368 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8012870 con 0x7f18e00b4d10 2024-09-16T07:22:59.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.766+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e0155870 con 0x7f18e00b4d10 2024-09-16T07:22:59.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.766+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e0155e30 con 0x7f18e00b4d10 2024-09-16T07:22:59.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.767+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e00d22a0 con 0x7f18e00b4d10 2024-09-16T07:22:59.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.768+0000 7f18d6000640 1 -- 172.21.15.71:0/1095161368 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f18c80035f0 con 0x7f18e00b4d10 2024-09-16T07:22:59.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.769+0000 7f18d6000640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18bc078850 0x7f18bc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:22:59.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.769+0000 7f18d6000640 1 -- 172.21.15.71:0/1095161368 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f18c809a530 con 0x7f18e00b4d10 2024-09-16T07:22:59.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.769+0000 7f18dde00640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18bc078850 0x7f18bc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:22:59.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.770+0000 7f18dde00640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18bc078850 0x7f18bc07ad10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18d0000f60 tx=0x7f18d0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:22:59.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.773+0000 7f18d6000640 1 -- 172.21.15.71:0/1095161368 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f18e0157f90 con 0x7f18e00b4d10 2024-09-16T07:22:59.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.958+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18e006d980 con 0x7f18e00b4d10 2024-09-16T07:22:59.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.959+0000 7f18d6000640 1 -- 172.21.15.71:0/1095161368 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7f18c8062b00 con 0x7f18e00b4d10 2024-09-16T07:22:59.958 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:22:59.959 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":23,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:50.122046+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:22:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18bc078850 msgr2=0x7f18bc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18bc078850 0x7f18bc07ad10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18d0000f60 tx=0x7f18d0009040 comp rx=0 tx=0).stop 2024-09-16T07:22:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 msgr2=0x7f18e00cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:22:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 0x7f18e00cbf00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f18c800c450 tx=0x7f18c800c910 comp rx=0 tx=0).stop 2024-09-16T07:22:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 shutdown_connections 2024-09-16T07:22:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18bc078850 0x7f18bc07ad10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00d1ec0 0x7f18e00cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e00b4d10 0x7f18e00cbf00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 --2- 172.21.15.71:0/1095161368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b28d0 0x7f18e0155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:22:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 >> 172.21.15.71:0/1095161368 conn(0x7f18e00d3de0 msgr2=0x7f18e00b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:22:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 shutdown_connections 2024-09-16T07:22:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:22:59.961+0000 7f18df200640 1 -- 172.21.15.71:0/1095161368 wait complete. 2024-09-16T07:22:59.978 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T07:22:59.979 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T07:22:59.979 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T07:22:59.980 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T07:22:59.980 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T07:22:59.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T07:22:59.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T07:22:59.982 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T07:23:00.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.255+0000 7f4a02800640 1 -- 172.21.15.71:0/2557807297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc0b5370 msgr2=0x7f49fc0b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.255+0000 7f4a02800640 1 --2- 172.21.15.71:0/2557807297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc0b5370 0x7f49fc0b3bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ce3d0 tx=0x7f49f0032860 comp rx=0 tx=0).stop 2024-09-16T07:23:00.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.255+0000 7f4a02800640 1 -- 172.21.15.71:0/2557807297 shutdown_connections 2024-09-16T07:23:00.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.255+0000 7f4a02800640 1 --2- 172.21.15.71:0/2557807297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc0b5370 0x7f49fc0b3bc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.255+0000 7f4a02800640 1 --2- 172.21.15.71:0/2557807297 >> v2:172.21.15.71:3300/0 conn(0x7f49fc0b4100 0x7f49fc0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.255+0000 7f4a02800640 1 -- 172.21.15.71:0/2557807297 >> 172.21.15.71:0/2557807297 conn(0x7f49fc0d3df0 msgr2=0x7f49fc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.256+0000 7f4a02800640 1 -- 172.21.15.71:0/2557807297 shutdown_connections 2024-09-16T07:23:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.256+0000 7f4a02800640 1 -- 172.21.15.71:0/2557807297 wait complete. 2024-09-16T07:23:00.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.258+0000 7fb2cee00640 1 -- 172.21.15.71:0/30917075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c0054860 msgr2=0x7fb2c0054c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.258+0000 7fb2cee00640 1 --2- 172.21.15.71:0/30917075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c0054860 0x7fb2c0054c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009830 tx=0x7fb2b402a480 comp rx=0 tx=0).stop 2024-09-16T07:23:00.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.259+0000 7f4a02800640 1 Processor -- start 2024-09-16T07:23:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.258+0000 7fb2cee00640 1 -- 172.21.15.71:0/30917075 shutdown_connections 2024-09-16T07:23:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.258+0000 7fb2cee00640 1 --2- 172.21.15.71:0/30917075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c0034ba0 0x7fb2c0008640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.258+0000 7fb2cee00640 1 --2- 172.21.15.71:0/30917075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c0054860 0x7fb2c0054c40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.258+0000 7fb2cee00640 1 -- 172.21.15.71:0/30917075 >> 172.21.15.71:0/30917075 conn(0x7fb2c001e890 msgr2=0x7fb2c0055640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.259+0000 7f4a02800640 1 -- start start 2024-09-16T07:23:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.259+0000 7fb2cee00640 1 -- 172.21.15.71:0/30917075 shutdown_connections 2024-09-16T07:23:00.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.259+0000 7fb2cee00640 1 -- 172.21.15.71:0/30917075 wait complete. 2024-09-16T07:23:00.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.261+0000 7f4a02800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc0b4100 0x7f49fc156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.261+0000 7f4a02800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49fc0b5370 0x7f49fc200630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.262+0000 7f4a02800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc161440 0x7f49fc200b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.262+0000 7fb2cee00640 1 Processor -- start 2024-09-16T07:23:00.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.262+0000 7fb2cee00640 1 -- start start 2024-09-16T07:23:00.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.262+0000 7f4a02800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc0b2040 con 0x7f49fc161440 2024-09-16T07:23:00.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.262+0000 7f4a01400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc0b4100 0x7f49fc156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.262+0000 7f4a01e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc161440 0x7f49fc200b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.263+0000 7f4a01e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc161440 0x7f49fc200b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50632/0 (socket says 172.21.15.71:50632) 2024-09-16T07:23:00.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.263+0000 7f4a01e00640 1 -- 172.21.15.71:0/491473354 learned_addr learned my addr 172.21.15.71:0/491473354 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.263+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc0b1ec0 con 0x7f49fc0b4100 2024-09-16T07:23:00.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.263+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc0b21c0 con 0x7f49fc0b5370 2024-09-16T07:23:00.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.263+0000 7f4a00a00640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49fc0b5370 0x7f49fc200630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.264+0000 7f4a01e00640 1 -- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49fc0b5370 msgr2=0x7f49fc200630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.264+0000 7f4a01e00640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49fc0b5370 0x7f49fc200630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.264+0000 7f4a01e00640 1 -- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc0b4100 msgr2=0x7f49fc156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.264+0000 7f4a01e00640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc0b4100 0x7f49fc156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.264+0000 7f4a01e00640 1 -- 172.21.15.71:0/491473354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc1578f0 con 0x7f49fc161440 2024-09-16T07:23:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.264+0000 7f4a01e00640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc161440 0x7f49fc200b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f49f400eaa0 tx=0x7f49f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.265+0000 7fb2cee00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c0034650 0x7fb2c00e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.265+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f400cdb0 con 0x7f49fc161440 2024-09-16T07:23:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.265+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49f4004c10 con 0x7f49fc161440 2024-09-16T07:23:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.265+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4005020 con 0x7f49fc161440 2024-09-16T07:23:00.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.266+0000 7fb2cee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c0034ba0 0x7fb2c00d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.265+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc157050 con 0x7f49fc161440 2024-09-16T07:23:00.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.266+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc157610 con 0x7f49fc161440 2024-09-16T07:23:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.266+0000 7fb2cee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c0054860 0x7fb2c017e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.267+0000 7fb2cce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c0054860 0x7fb2c017e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.267+0000 7fb2cce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c0054860 0x7fb2c017e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50636/0 (socket says 172.21.15.71:50636) 2024-09-16T07:23:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.267+0000 7fb2cce00640 1 -- 172.21.15.71:0/2370494681 learned_addr learned my addr 172.21.15.71:0/2370494681 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.267+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c00e6840 con 0x7fb2c0054860 2024-09-16T07:23:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.267+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c00e7780 con 0x7fb2c0034ba0 2024-09-16T07:23:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.267+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c00e69c0 con 0x7fb2c0034650 2024-09-16T07:23:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.268+0000 7fb2cce00640 1 -- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c0034650 msgr2=0x7fb2c00e5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.268+0000 7fb2cce00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c0034650 0x7fb2c00e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.268+0000 7fb2cce00640 1 -- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c0034ba0 msgr2=0x7fb2c00d2b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:23:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.268+0000 7fb2cce00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c0034ba0 0x7fb2c00d2b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.268+0000 7fb2cce00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c017e640 con 0x7fb2c0054860 2024-09-16T07:23:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.267+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49fc0b1770 con 0x7f49fc161440 2024-09-16T07:23:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.269+0000 7fb2cce00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c0054860 0x7fb2c017e100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb2c806b4b0 tx=0x7fb2c806a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.269+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8090070 con 0x7fb2c0054860 2024-09-16T07:23:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.269+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2c8072630 con 0x7fb2c0054860 2024-09-16T07:23:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.269+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8072b00 con 0x7fb2c0054860 2024-09-16T07:23:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.270+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c000c110 con 0x7fb2c0054860 2024-09-16T07:23:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.270+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c000bb50 con 0x7fb2c0054860 2024-09-16T07:23:00.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.273+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2a4001030 con 0x7fb2c0054860 2024-09-16T07:23:00.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.273+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f49f40051c0 con 0x7f49fc161440 2024-09-16T07:23:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb2c8093070 con 0x7fb2c0054860 2024-09-16T07:23:00.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7f5578e00640 1 -- 172.21.15.71:0/2699799287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5574161440 msgr2=0x7f55740b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7f5578e00640 1 --2- 172.21.15.71:0/2699799287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5574161440 0x7f55740b5370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5564009be0 tx=0x7f556402a800 comp rx=0 tx=0).stop 2024-09-16T07:23:00.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7f5578e00640 1 -- 172.21.15.71:0/2699799287 shutdown_connections 2024-09-16T07:23:00.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7f5578e00640 1 --2- 172.21.15.71:0/2699799287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5574161440 0x7f55740b5370 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7f5578e00640 1 -- 172.21.15.71:0/2699799287 >> 172.21.15.71:0/2699799287 conn(0x7f55740d3df0 msgr2=0x7f55740d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7f817ca00640 1 -- 172.21.15.71:0/2566274486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8178161440 msgr2=0x7f81780b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7f817ca00640 1 --2- 172.21.15.71:0/2566274486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8178161440 0x7f81780b5370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8160009860 tx=0x7f816002a460 comp rx=0 tx=0).stop 2024-09-16T07:23:00.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7f5578e00640 1 -- 172.21.15.71:0/2699799287 shutdown_connections 2024-09-16T07:23:00.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7f817ca00640 1 -- 172.21.15.71:0/2566274486 shutdown_connections 2024-09-16T07:23:00.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7f817ca00640 1 --2- 172.21.15.71:0/2566274486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8178161440 0x7f81780b5370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7f817ca00640 1 -- 172.21.15.71:0/2566274486 >> 172.21.15.71:0/2566274486 conn(0x7f81780d3df0 msgr2=0x7f81780d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.274+0000 7f49f9600640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49dc0788a0 0x7f49dc07ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7f817ca00640 1 -- 172.21.15.71:0/2566274486 shutdown_connections 2024-09-16T07:23:00.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7f817ca00640 1 -- 172.21.15.71:0/2566274486 wait complete. 2024-09-16T07:23:00.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7fb2bbe00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2a8078850 0x7fb2a807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb2c8115810 con 0x7fb2c0054860 2024-09-16T07:23:00.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7f5578e00640 1 -- 172.21.15.71:0/2699799287 wait complete. 2024-09-16T07:23:00.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7fb2bbe00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7fb2a8080a70 0x7fb2a8082e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2a8083540 con 0x7fb2a8080a70 2024-09-16T07:23:00.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb2c811a2a0 con 0x7fb2c0054860 2024-09-16T07:23:00.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.275+0000 7f4a01400640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49dc0788a0 0x7f49dc07ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.276+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f49f405fa10 con 0x7f49fc161440 2024-09-16T07:23:00.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.276+0000 7fb2c7400640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7fb2a8080a70 0x7fb2a8082e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.276+0000 7f49f9600640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f49dc080ac0 0x7f49dc082ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.276+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 --> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49f4021e50 con 0x7f49dc080ac0 2024-09-16T07:23:00.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.276+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f49f405f640 con 0x7f49fc161440 2024-09-16T07:23:00.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.276+0000 7fb2c7e00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2a8078850 0x7fb2a807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.277+0000 7fb2c7400640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7fb2a8080a70 0x7fb2a8082e90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.277+0000 7f4a00a00640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f49dc080ac0 0x7f49dc082ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.278+0000 7f817ca00640 1 Processor -- start 2024-09-16T07:23:00.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.278+0000 7fb2c7e00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2a8078850 0x7fb2a807ad10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4034000 tx=0x7fb2b40046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.278+0000 7f4a00a00640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f49dc080ac0 0x7f49dc082ee0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.278+0000 7f4a01400640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49dc0788a0 0x7f49dc07ad60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49e8007fd0 tx=0x7f49e8001370 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.278+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 <== osd.0 v2:172.21.15.71:6800/207466128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2a8083540 con 0x7fb2a8080a70 2024-09-16T07:23:00.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.278+0000 7f5578e00640 1 Processor -- start 2024-09-16T07:23:00.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.278+0000 7f817ca00640 1 -- start start 2024-09-16T07:23:00.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7f5578e00640 1 -- start start 2024-09-16T07:23:00.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7f817ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81780b34b0 0x7f81780d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f817ca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81780b4020 0x7f81780b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f8177400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81780b34b0 0x7f81780d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f817ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178161440 0x7f81780b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f817ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f81780b2210 con 0x7f8178161440 2024-09-16T07:23:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f817ca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81780b2090 con 0x7f81780b34b0 2024-09-16T07:23:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f817ca00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f81780b2390 con 0x7f81780b4020 2024-09-16T07:23:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f8177400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81780b34b0 0x7f81780d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55558/0 (socket says 172.21.15.71:55558) 2024-09-16T07:23:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f8177400640 1 -- 172.21.15.71:0/2794501909 learned_addr learned my addr 172.21.15.71:0/2794501909 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f8176a00640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81780b4020 0x7f81780b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 <== osd.1 v2:172.21.15.71:6816/1940414636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49f4021e50 con 0x7f49dc080ac0 2024-09-16T07:23:00.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.281+0000 7f5578e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55740b34b0 0x7f55740caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.281+0000 7f8177400640 1 -- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81780b4020 msgr2=0x7f81780b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.281+0000 7f8177400640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81780b4020 0x7f81780b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.281+0000 7f8177400640 1 -- 172.21.15.71:0/2794501909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178161440 msgr2=0x7f81780b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.281+0000 7f8177400640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178161440 0x7f81780b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.281+0000 7f8177400640 1 -- 172.21.15.71:0/2794501909 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81780b8f60 con 0x7f81780b34b0 2024-09-16T07:23:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7fb796800640 1 -- 172.21.15.71:0/3535024785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7900d1610 msgr2=0x7fb7900cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7fb796800640 1 --2- 172.21.15.71:0/3535024785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7900d1610 0x7fb7900cecc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb788005fd0 tx=0x7fb78802aaa0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7fb796800640 1 -- 172.21.15.71:0/3535024785 shutdown_connections 2024-09-16T07:23:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7fb796800640 1 --2- 172.21.15.71:0/3535024785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7900d1610 0x7fb7900cecc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7fb796800640 1 --2- 172.21.15.71:0/3535024785 >> v2:172.21.15.71:3300/0 conn(0x7fb79015e0d0 0x7fb7900d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7fb796800640 1 -- 172.21.15.71:0/3535024785 >> 172.21.15.71:0/3535024785 conn(0x7fb7900d3fe0 msgr2=0x7fb7900d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.279+0000 7fb796800640 1 -- 172.21.15.71:0/3535024785 shutdown_connections 2024-09-16T07:23:00.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.280+0000 7fb796800640 1 -- 172.21.15.71:0/3535024785 wait complete. 2024-09-16T07:23:00.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.281+0000 7f5573400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55740b34b0 0x7f55740caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f007ba00640 1 -- 172.21.15.71:0/400146334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740d09c0 msgr2=0x7f00740d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f5578e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55740b4020 0x7f55740cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f5578e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5574161440 0x7f55741f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f5578e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55740b2210 con 0x7f55740b34b0 2024-09-16T07:23:00.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f5578e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55740b2090 con 0x7f5574161440 2024-09-16T07:23:00.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f5578e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55740b2390 con 0x7f55740b4020 2024-09-16T07:23:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f5573400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55740b34b0 0x7f55740caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50662/0 (socket says 172.21.15.71:50662) 2024-09-16T07:23:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f5573400640 1 -- 172.21.15.71:0/1726356611 learned_addr learned my addr 172.21.15.71:0/1726356611 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f5573e00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5574161440 0x7f55741f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f8177400640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81780b34b0 0x7f81780d12e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8160009980 tx=0x7f8160033ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160031b50 con 0x7f81780b34b0 2024-09-16T07:23:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8160031de0 con 0x7f81780b34b0 2024-09-16T07:23:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160005280 con 0x7f81780b34b0 2024-09-16T07:23:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81780b9240 con 0x7f81780b34b0 2024-09-16T07:23:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81780cecb0 con 0x7f81780b34b0 2024-09-16T07:23:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7f007ba00640 1 --2- 172.21.15.71:0/400146334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740d09c0 0x7f00740d0da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0064005fd0 tx=0x7f006402a530 comp rx=0 tx=0).stop 2024-09-16T07:23:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7f5573e00640 1 -- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55740b4020 msgr2=0x7f55740cb400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:23:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7f5573e00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55740b4020 0x7f55740cb400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7f5573e00640 1 -- 172.21.15.71:0/1726356611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55740b34b0 msgr2=0x7f55740caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.282+0000 7fb796800640 1 Processor -- start 2024-09-16T07:23:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7fb796800640 1 -- start start 2024-09-16T07:23:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7f5573e00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55740b34b0 0x7f55740caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.283+0000 7f5573e00640 1 -- 172.21.15.71:0/1726356611 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5574158070 con 0x7f5574161440 2024-09-16T07:23:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.284+0000 7f5573400640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55740b34b0 0x7f55740caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.284+0000 7f007ba00640 1 -- 172.21.15.71:0/400146334 shutdown_connections 2024-09-16T07:23:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.284+0000 7f007ba00640 1 --2- 172.21.15.71:0/400146334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740d2050 0x7f00740d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.284+0000 7f007ba00640 1 --2- 172.21.15.71:0/400146334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740d09c0 0x7f00740d0da0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.284+0000 7f007ba00640 1 -- 172.21.15.71:0/400146334 >> 172.21.15.71:0/400146334 conn(0x7f00740d3fe0 msgr2=0x7f00740d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.284+0000 7f5573e00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5574161440 0x7f55741f3b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c3c0 tx=0x7f556c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.284+0000 7f007ba00640 1 -- 172.21.15.71:0/400146334 shutdown_connections 2024-09-16T07:23:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.284+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c011790 con 0x7f5574161440 2024-09-16T07:23:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.285+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f556c017070 con 0x7f5574161440 2024-09-16T07:23:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.285+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c011d20 con 0x7f5574161440 2024-09-16T07:23:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.285+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55741471e0 con 0x7f5574161440 2024-09-16T07:23:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.285+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55741477a0 con 0x7f5574161440 2024-09-16T07:23:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.285+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81780b27c0 con 0x7f81780b34b0 2024-09-16T07:23:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.285+0000 7fb796800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7900d1610 0x7fb7901fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.285+0000 7f007ba00640 1 -- 172.21.15.71:0/400146334 wait complete. 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7fb796800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7900d2050 0x7fb7900a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7fb796800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb79015e0d0 0x7fb79009e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7fb796800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7900cf610 con 0x7fb7900d2050 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7fb796800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb7900cf490 con 0x7fb7900d1610 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7fb796800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb7900cf790 con 0x7fb79015e0d0 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7fb78f400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7900d1610 0x7fb7901fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7fb78f400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7900d1610 0x7fb7901fbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55598/0 (socket says 172.21.15.71:55598) 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7fb78f400640 1 -- 172.21.15.71:0/270888785 learned_addr learned my addr 172.21.15.71:0/270888785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.287+0000 7fb78ea00640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7900d2050 0x7fb7900a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7fb78fe00640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb79015e0d0 0x7fb79009e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7fb78ea00640 1 -- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb79015e0d0 msgr2=0x7fb79009e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7fb78ea00640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb79015e0d0 0x7fb79009e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7fb78ea00640 1 -- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7900d1610 msgr2=0x7fb7901fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7fb78ea00640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7900d1610 0x7fb7901fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7fb78ea00640 1 -- 172.21.15.71:0/270888785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb790203030 con 0x7fb7900d2050 2024-09-16T07:23:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f556c01c460 con 0x7f5574161440 2024-09-16T07:23:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7f7899400640 1 -- 172.21.15.71:0/339205523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f787c004800 msgr2=0x7f787c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.286+0000 7f7899400640 1 --2- 172.21.15.71:0/339205523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f787c004800 0x7f787c004c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e610 tx=0x7f787c005830 comp rx=0 tx=0).stop 2024-09-16T07:23:00.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7f7898a00640 1 -- 172.21.15.71:0/339205523 reap_dead start 2024-09-16T07:23:00.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7f007ba00640 1 Processor -- start 2024-09-16T07:23:00.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7f007ba00640 1 -- start start 2024-09-16T07:23:00.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7f7899400640 1 -- 172.21.15.71:0/339205523 shutdown_connections 2024-09-16T07:23:00.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7f7899400640 1 -- 172.21.15.71:0/339205523 >> 172.21.15.71:0/339205523 conn(0x7f788c056770 msgr2=0x7f788c056b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.288+0000 7fb78ea00640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7900d2050 0x7fb7900a0b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c3f0 tx=0x7fb77c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c017070 con 0x7fb7900d2050 2024-09-16T07:23:00.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f556b400640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5554078850 0x7f555407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f556c09a570 con 0x7f5574161440 2024-09-16T07:23:00.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55740b27c0 con 0x7f5574161440 2024-09-16T07:23:00.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f7899400640 1 -- 172.21.15.71:0/339205523 shutdown_connections 2024-09-16T07:23:00.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f7b3f200640 1 -- 172.21.15.71:0/829594262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b4015ebe0 msgr2=0x7f7b400d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f7b3f200640 1 --2- 172.21.15.71:0/829594262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b4015ebe0 0x7f7b400d1060 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7b30005fd0 tx=0x7f7b3002a8c0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f5573400640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5554078850 0x7f555407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb77c0053c0 con 0x7fb7900d2050 2024-09-16T07:23:00.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c017070 con 0x7fb7900d2050 2024-09-16T07:23:00.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f007ba00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cea10 0x7f00740b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.289+0000 7f007ba00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740d09c0 0x7f007415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f007ba00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00740d2050 0x7f00741fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f007ba00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00740cf960 con 0x7f00740d2050 2024-09-16T07:23:00.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f007ba00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00740cf7e0 con 0x7f00740cea10 2024-09-16T07:23:00.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f007ba00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f00740cfae0 con 0x7f00740d09c0 2024-09-16T07:23:00.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f556b400640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7f5554080a50 0x7f5554082e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f7b3f200640 1 -- 172.21.15.71:0/829594262 shutdown_connections 2024-09-16T07:23:00.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f7b3f200640 1 --2- 172.21.15.71:0/829594262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b4015ebe0 0x7f7b400d1060 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f7b3f200640 1 --2- 172.21.15.71:0/829594262 >> v2:172.21.15.71:3300/0 conn(0x7f7b400d0600 0x7f7b400d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8160005420 con 0x7f81780b34b0 2024-09-16T07:23:00.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f7b3f200640 1 -- 172.21.15.71:0/829594262 >> 172.21.15.71:0/829594262 conn(0x7f7b4006dc80 msgr2=0x7f7b4006e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 --> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f556c009e50 con 0x7f5554080a50 2024-09-16T07:23:00.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f556c05eb20 con 0x7f5574161440 2024-09-16T07:23:00.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f0073e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740d09c0 0x7f007415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f0079000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cea10 0x7f00740b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f7899400640 1 -- 172.21.15.71:0/339205523 wait complete. 2024-09-16T07:23:00.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f5572a00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7f5554080a50 0x7f5554082e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.290+0000 7f0079000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cea10 0x7f00740b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55604/0 (socket says 172.21.15.71:55604) 2024-09-16T07:23:00.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.291+0000 7f7b3f200640 1 -- 172.21.15.71:0/829594262 shutdown_connections 2024-09-16T07:23:00.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.291+0000 7f0079000640 1 -- 172.21.15.71:0/2396636280 learned_addr learned my addr 172.21.15.71:0/2396636280 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.291+0000 7f7b3f200640 1 -- 172.21.15.71:0/829594262 wait complete. 2024-09-16T07:23:00.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.291+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7900a12d0 con 0x7fb7900d2050 2024-09-16T07:23:00.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.291+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7900a1890 con 0x7fb7900d2050 2024-09-16T07:23:00.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f816f400640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8154078920 0x7f815407ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f81600bb770 con 0x7f81780b34b0 2024-09-16T07:23:00.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f816f400640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8154080ba0 0x7f8154082fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f0079a00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00740d2050 0x7f00741fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 --> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8160032e50 con 0x7f8154080ba0 2024-09-16T07:23:00.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f5573400640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5554078850 0x7f555407ad10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5564000f80 tx=0x7f5564004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f5572a00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7f5554080a50 0x7f5554082e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f7899400640 1 Processor -- start 2024-09-16T07:23:00.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.292+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f81600c2050 con 0x7f81780b34b0 2024-09-16T07:23:00.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f0073e00640 1 -- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cea10 msgr2=0x7f00740b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f0073e00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cea10 0x7f00740b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f0073e00640 1 -- 172.21.15.71:0/2396636280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00740d2050 msgr2=0x7f00741fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f8177e00640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8154080ba0 0x7f8154082fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f0073e00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00740d2050 0x7f00741fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f0073e00640 1 -- 172.21.15.71:0/2396636280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00741f8cb0 con 0x7f00740d09c0 2024-09-16T07:23:00.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f0079a00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00740d2050 0x7f00741fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:00.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 <== osd.3 v2:172.21.15.71:6808/1022096187 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f556c009e50 con 0x7f5554080a50 2024-09-16T07:23:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.294+0000 7f0073e00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740d09c0 0x7f007415c340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f006000e6a0 tx=0x7f006000eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.294+0000 7f8176a00640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8154078920 0x7f815407ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.294+0000 7f7b3f200640 1 Processor -- start 2024-09-16T07:23:00.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.293+0000 7f7899400640 1 -- start start 2024-09-16T07:23:00.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.294+0000 7f7899400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f787c004800 0x7f788c04e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f8177e00640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8154080ba0 0x7f8154082fc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060004220 con 0x7f00740d09c0 2024-09-16T07:23:00.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00600043c0 con 0x7f00740d09c0 2024-09-16T07:23:00.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000cb10 con 0x7f00740d09c0 2024-09-16T07:23:00.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7900d3690 con 0x7fb7900d2050 2024-09-16T07:23:00.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f7899400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c0ddea0 0x7f788c04f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb77c01a070 con 0x7fb7900d2050 2024-09-16T07:23:00.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7fb787400640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb770078a50 0x7fb77007af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb77c0997c0 con 0x7fb7900d2050 2024-09-16T07:23:00.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7fb787400640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb770080c70 0x7fb770083090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 --> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb770083740 con 0x7fb770080c70 2024-09-16T07:23:00.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb77c0a0050 con 0x7fb7900d2050 2024-09-16T07:23:00.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7fb78f400640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb770078a50 0x7fb77007af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7fb78fe00640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb770080c70 0x7fb770083090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f8176a00640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8154078920 0x7f815407ade0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8168007fd0 tx=0x7f8168007e50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f7899400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f788c04fcf0 0x7f788c050170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f7893e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f787c004800 0x7f788c04e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f7893e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f787c004800 0x7f788c04e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55614/0 (socket says 172.21.15.71:55614) 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f7899400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f788c0513f0 con 0x7f788c0ddea0 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7899400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f788c037360 con 0x7f787c004800 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7893e00640 1 -- 172.21.15.71:0/1447147499 learned_addr learned my addr 172.21.15.71:0/1447147499 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f788c051570 con 0x7f788c04fcf0 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7893400640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c0ddea0 0x7f788c04f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7898a00640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f788c04fcf0 0x7f788c050170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7893e00640 1 -- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f788c04fcf0 msgr2=0x7f788c050170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7893e00640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f788c04fcf0 0x7f788c050170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7893e00640 1 -- 172.21.15.71:0/1447147499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c0ddea0 msgr2=0x7f788c04f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7893e00640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c0ddea0 0x7f788c04f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7893e00640 1 -- 172.21.15.71:0/1447147499 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c0de300 con 0x7f787c004800 2024-09-16T07:23:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7893400640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c0ddea0 0x7f788c04f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7f7893e00640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f787c004800 0x7f788c04e220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78940511b0 tx=0x7f78940698c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00741f8f20 con 0x7f00740d09c0 2024-09-16T07:23:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.295+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f00740b9bc0 con 0x7f00740d09c0 2024-09-16T07:23:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.294+0000 7f7b3f200640 1 -- start start 2024-09-16T07:23:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7b3f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b400d0600 0x7f7b400b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7b3f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4015bb60 0x7f7b400b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7b3f200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b4015ebe0 0x7f7b40139f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7b3f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7b400709f0 con 0x7f7b400d0600 2024-09-16T07:23:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7b3f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40070870 con 0x7f7b4015bb60 2024-09-16T07:23:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.296+0000 7f7b3f200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7b40070b70 con 0x7f7b4015ebe0 2024-09-16T07:23:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7b3de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b400d0600 0x7f7b400b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7b3d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4015bb60 0x7f7b400b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7b3d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4015bb60 0x7f7b400b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55626/0 (socket says 172.21.15.71:55626) 2024-09-16T07:23:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7b3d400640 1 -- 172.21.15.71:0/109921750 learned_addr learned my addr 172.21.15.71:0/109921750 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.297+0000 7f7b3e800640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b4015ebe0 0x7f7b40139f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7f7b3e800640 1 -- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4015bb60 msgr2=0x7f7b400b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7f7b3e800640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4015bb60 0x7f7b400b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7f7b3e800640 1 -- 172.21.15.71:0/109921750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b400d0600 msgr2=0x7f7b400b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.298+0000 7f7b3e800640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b400d0600 0x7f7b400b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f00741fd270 con 0x7f00740d09c0 2024-09-16T07:23:00.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7b3e800640 1 -- 172.21.15.71:0/109921750 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4016ba80 con 0x7f7b4015ebe0 2024-09-16T07:23:00.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7b3d400640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b4015bb60 0x7f7b400b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:00.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7b3de00640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b400d0600 0x7f7b400b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:00.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894099070 con 0x7f787c004800 2024-09-16T07:23:00.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78940946e0 con 0x7f787c004800 2024-09-16T07:23:00.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7fb78f400640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb770078a50 0x7fb77007af10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb78000bc00 tx=0x7fb78000b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c0de820 con 0x7f787c004800 2024-09-16T07:23:00.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c04e8c0 con 0x7f787c004800 2024-09-16T07:23:00.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 <== osd.4 v2:172.21.15.177:6802/4258128961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8160032e50 con 0x7f8154080ba0 2024-09-16T07:23:00.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7b3e800640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b4015ebe0 0x7f7b40139f10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009000 tx=0x7f7b30030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.301+0000 7fb78fe00640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb770080c70 0x7fb770083090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.303+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 <== osd.2 v2:172.21.15.71:6824/3801736779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb770083740 con 0x7fb770080c70 2024-09-16T07:23:00.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894094b30 con 0x7f787c004800 2024-09-16T07:23:00.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.302+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7870001030 con 0x7f787c004800 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.303+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f789409c070 con 0x7f787c004800 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.299+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f006000ccb0 con 0x7f00740d09c0 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.301+0000 7f0070c00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0054078920 0x7f005407ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.301+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f006009b200 con 0x7f00740d09c0 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.301+0000 7f0070c00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f0054080b40 0x7f0054082f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.301+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 --> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f006000be50 con 0x7f0054080b40 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.301+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f00600a2050 con 0x7f00740d09c0 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.302+0000 7f0079000640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0054078920 0x7f005407ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.302+0000 7f0079a00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f0054080b40 0x7f0054082f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.303+0000 7f0079a00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f0054080b40 0x7f0054082f60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.303+0000 7f0079000640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0054078920 0x7f005407ade0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f00740b8c80 tx=0x7f0064004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.300+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30030c50 con 0x7f7b4015ebe0 2024-09-16T07:23:00.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.300+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b30003970 con 0x7f7b4015ebe0 2024-09-16T07:23:00.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.300+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30003dc0 con 0x7f7b4015ebe0 2024-09-16T07:23:00.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.301+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4016a9e0 con 0x7f7b4015ebe0 2024-09-16T07:23:00.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.302+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4016afa0 con 0x7f7b4015ebe0 2024-09-16T07:23:00.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.303+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b40071020 con 0x7f7b4015ebe0 2024-09-16T07:23:00.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.305+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 <== osd.6 v2:172.21.15.177:6818/1024130870 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f006000be50 con 0x7f0054080b40 2024-09-16T07:23:00.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.305+0000 7f7887e00640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7874078920 0x7f787407ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.305+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7894116290 con 0x7f787c004800 2024-09-16T07:23:00.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.306+0000 7f7887e00640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f7874080b40 0x7f7874082f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.306+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 --> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78940643f0 con 0x7f7874080b40 2024-09-16T07:23:00.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.306+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f789411d050 con 0x7f787c004800 2024-09-16T07:23:00.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.306+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7b30003f60 con 0x7f7b4015ebe0 2024-09-16T07:23:00.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.306+0000 7f7893400640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7874078920 0x7f787407ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.306+0000 7f7898a00640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f7874080b40 0x7f7874082f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.307+0000 7f7898a00640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f7874080b40 0x7f7874082f60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.308+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 <== osd.5 v2:172.21.15.177:6823/1078185550 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78940643f0 con 0x7f7874080b40 2024-09-16T07:23:00.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.307+0000 7f7b36000640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7b1c0788a0 0x7f7b1c07ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.308+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7b300bbda0 con 0x7f7b4015ebe0 2024-09-16T07:23:00.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.308+0000 7f7b3de00640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7b1c0788a0 0x7f7b1c07ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.308+0000 7f7b36000640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7f7b1c080b20 0x7f7b1c082f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.308+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 --> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b3004ae50 con 0x7f7b1c080b20 2024-09-16T07:23:00.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.308+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7b300be3c0 con 0x7f7b4015ebe0 2024-09-16T07:23:00.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.308+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 --> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb2a4002dd0 con 0x7fb2a8080a70 2024-09-16T07:23:00.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.308+0000 7f7893400640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7874078920 0x7f787407ade0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7888035000 tx=0x7f78880046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.309+0000 7f7b3d400640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7f7b1c080b20 0x7f7b1c082f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.309+0000 7f7b3de00640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7b1c0788a0 0x7f7b1c07ad60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800bfd0 tx=0x7f7b2800b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.309+0000 7fb2bbe00640 1 -- 172.21.15.71:0/2370494681 <== osd.0 v2:172.21.15.71:6800/207466128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb2a4002dd0 con 0x7fb2a8080a70 2024-09-16T07:23:00.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.309+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7fb2a8080a70 msgr2=0x7fb2a8082e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.309+0000 7fb2cee00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7fb2a8080a70 0x7fb2a8082e90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.310+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2a8078850 msgr2=0x7fb2a807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.310+0000 7fb2cee00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2a8078850 0x7fb2a807ad10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4034000 tx=0x7fb2b40046c0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.310+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c0054860 msgr2=0x7fb2c017e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.310+0000 7fb2cee00640 1 --2- 172.21.15.71:0/2370494681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c0054860 0x7fb2c017e100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb2c806b4b0 tx=0x7fb2c806a370 comp rx=0 tx=0).stop 2024-09-16T07:23:00.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.310+0000 7fb2cce00640 1 -- 172.21.15.71:0/2370494681 reap_dead start 2024-09-16T07:23:00.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.310+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 shutdown_connections 2024-09-16T07:23:00.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.310+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 >> 172.21.15.71:0/2370494681 conn(0x7fb2c001e890 msgr2=0x7fb2c00e2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.311+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 shutdown_connections 2024-09-16T07:23:00.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.311+0000 7fb2cee00640 1 -- 172.21.15.71:0/2370494681 wait complete. 2024-09-16T07:23:00.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.311+0000 7f7b3d400640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7f7b1c080b20 0x7f7b1c082f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.312+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 <== osd.7 v2:172.21.15.177:6810/4227809343 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b3004ae50 con 0x7f7b1c080b20 2024-09-16T07:23:00.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.318+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 --> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49fc1554a0 con 0x7f49dc080ac0 2024-09-16T07:23:00.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.320+0000 7f49f9600640 1 -- 172.21.15.71:0/491473354 <== osd.1 v2:172.21.15.71:6816/1940414636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f49fc1554a0 con 0x7f49dc080ac0 2024-09-16T07:23:00.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.321+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f49dc080ac0 msgr2=0x7f49dc082ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.321+0000 7f4a02800640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f49dc080ac0 0x7f49dc082ee0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49dc0788a0 msgr2=0x7f49dc07ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a02800640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49dc0788a0 0x7f49dc07ad60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49e8007fd0 tx=0x7f49e8001370 comp rx=0 tx=0).stop 2024-09-16T07:23:00.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc161440 msgr2=0x7f49fc200b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a02800640 1 --2- 172.21.15.71:0/491473354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc161440 0x7f49fc200b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f49f400eaa0 tx=0x7f49f400ef60 comp rx=0 tx=0).stop 2024-09-16T07:23:00.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a01e00640 1 -- 172.21.15.71:0/491473354 reap_dead start 2024-09-16T07:23:00.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 shutdown_connections 2024-09-16T07:23:00.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 >> 172.21.15.71:0/491473354 conn(0x7f49fc0d3df0 msgr2=0x7f49fc158300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 shutdown_connections 2024-09-16T07:23:00.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.322+0000 7f4a02800640 1 -- 172.21.15.71:0/491473354 wait complete. 2024-09-16T07:23:00.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.324+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 --> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f55740c87d0 con 0x7f5554080a50 2024-09-16T07:23:00.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.324+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 --> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f00741f90f0 con 0x7f0054080b40 2024-09-16T07:23:00.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.325+0000 7f0070c00640 1 -- 172.21.15.71:0/2396636280 <== osd.6 v2:172.21.15.177:6818/1024130870 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f00741f90f0 con 0x7f0054080b40 2024-09-16T07:23:00.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.325+0000 7f556b400640 1 -- 172.21.15.71:0/1726356611 <== osd.3 v2:172.21.15.71:6808/1022096187 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f55740c87d0 con 0x7f5554080a50 2024-09-16T07:23:00.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f0054080b40 msgr2=0x7f0054082f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f007ba00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f0054080b40 0x7f0054082f60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0054078920 msgr2=0x7f005407ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f007ba00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0054078920 0x7f005407ade0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f00740b8c80 tx=0x7f0064004730 comp rx=0 tx=0).stop 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740d09c0 msgr2=0x7f007415c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f007ba00640 1 --2- 172.21.15.71:0/2396636280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740d09c0 0x7f007415c340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f006000e6a0 tx=0x7f006000eb60 comp rx=0 tx=0).stop 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f0079a00640 1 -- 172.21.15.71:0/2396636280 reap_dead start 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 shutdown_connections 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.327+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 >> 172.21.15.71:0/2396636280 conn(0x7f00740d3fe0 msgr2=0x7f00740b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.328+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 shutdown_connections 2024-09-16T07:23:00.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.328+0000 7f007ba00640 1 -- 172.21.15.71:0/2396636280 wait complete. 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.328+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7f5554080a50 msgr2=0x7f5554082e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.328+0000 7f5578e00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7f5554080a50 0x7f5554082e70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.328+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5554078850 msgr2=0x7f555407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.328+0000 7f5578e00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5554078850 0x7f555407ad10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5564000f80 tx=0x7f5564004ab0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.329+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5574161440 msgr2=0x7f55741f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.329+0000 7f5578e00640 1 --2- 172.21.15.71:0/1726356611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5574161440 0x7f55741f3b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c3c0 tx=0x7f556c00c880 comp rx=0 tx=0).stop 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.329+0000 7f5573e00640 1 -- 172.21.15.71:0/1726356611 reap_dead start 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.329+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 shutdown_connections 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.329+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 >> 172.21.15.71:0/1726356611 conn(0x7f55740d3df0 msgr2=0x7f5574158480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.330+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 shutdown_connections 2024-09-16T07:23:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.330+0000 7f5578e00640 1 -- 172.21.15.71:0/1726356611 wait complete. 2024-09-16T07:23:00.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.333+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 --> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8178160760 con 0x7f8154080ba0 2024-09-16T07:23:00.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.333+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 --> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7900d30d0 con 0x7fb770080c70 2024-09-16T07:23:00.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.334+0000 7fb787400640 1 -- 172.21.15.71:0/270888785 <== osd.2 v2:172.21.15.71:6824/3801736779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb7900d30d0 con 0x7fb770080c70 2024-09-16T07:23:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.335+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb770080c70 msgr2=0x7fb770083090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.335+0000 7fb796800640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb770080c70 0x7fb770083090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.336+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb770078a50 msgr2=0x7fb77007af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.336+0000 7fb796800640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb770078a50 0x7fb77007af10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb78000bc00 tx=0x7fb78000b390 comp rx=0 tx=0).stop 2024-09-16T07:23:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.336+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7900d2050 msgr2=0x7fb7900a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.336+0000 7fb796800640 1 --2- 172.21.15.71:0/270888785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7900d2050 0x7fb7900a0b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c3f0 tx=0x7fb77c00c8b0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.336+0000 7fb78fe00640 1 -- 172.21.15.71:0/270888785 reap_dead start 2024-09-16T07:23:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.338+0000 7f816f400640 1 -- 172.21.15.71:0/2794501909 <== osd.4 v2:172.21.15.177:6802/4258128961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8178160760 con 0x7f8154080ba0 2024-09-16T07:23:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.338+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 --> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7870002dd0 con 0x7f7874080b40 2024-09-16T07:23:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.338+0000 7f7887e00640 1 -- 172.21.15.71:0/1447147499 <== osd.5 v2:172.21.15.177:6823/1078185550 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7870002dd0 con 0x7f7874080b40 2024-09-16T07:23:00.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.338+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 shutdown_connections 2024-09-16T07:23:00.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.338+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 >> 172.21.15.71:0/270888785 conn(0x7fb7900d3fe0 msgr2=0x7fb790160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.338+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8154080ba0 msgr2=0x7f8154082fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 shutdown_connections 2024-09-16T07:23:00.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7fb796800640 1 -- 172.21.15.71:0/270888785 wait complete. 2024-09-16T07:23:00.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f817ca00640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8154080ba0 0x7f8154082fc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8154078920 msgr2=0x7f815407ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f7874080b40 msgr2=0x7f7874082f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f7899400640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f7874080b40 0x7f7874082f60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f817ca00640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8154078920 0x7f815407ade0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8168007fd0 tx=0x7f8168007e50 comp rx=0 tx=0).stop 2024-09-16T07:23:00.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81780b34b0 msgr2=0x7f81780d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f817ca00640 1 --2- 172.21.15.71:0/2794501909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81780b34b0 0x7f81780d12e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8160009980 tx=0x7f8160033ba0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f8177e00640 1 -- 172.21.15.71:0/2794501909 reap_dead start 2024-09-16T07:23:00.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 shutdown_connections 2024-09-16T07:23:00.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.339+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 >> 172.21.15.71:0/2794501909 conn(0x7f81780d3df0 msgr2=0x7f81780d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.353 INFO:teuthology.orchestra.run.smithi071.stdout:68719476744 2024-09-16T07:23:00.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.340+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 shutdown_connections 2024-09-16T07:23:00.354 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T07:23:00.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.340+0000 7f817ca00640 1 -- 172.21.15.71:0/2794501909 wait complete. 2024-09-16T07:23:00.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.340+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7874078920 msgr2=0x7f787407ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.340+0000 7f7899400640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7874078920 0x7f787407ade0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7888035000 tx=0x7f78880046c0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.340+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f787c004800 msgr2=0x7f788c04e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.340+0000 7f7899400640 1 --2- 172.21.15.71:0/1447147499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f787c004800 0x7f788c04e220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78940511b0 tx=0x7f78940698c0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.340+0000 7f7898a00640 1 -- 172.21.15.71:0/1447147499 reap_dead start 2024-09-16T07:23:00.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.341+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 shutdown_connections 2024-09-16T07:23:00.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.341+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 >> 172.21.15.71:0/1447147499 conn(0x7f788c056770 msgr2=0x7f788c056b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.341+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 shutdown_connections 2024-09-16T07:23:00.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.341+0000 7f7899400640 1 -- 172.21.15.71:0/1447147499 wait complete. 2024-09-16T07:23:00.356 INFO:teuthology.orchestra.run.smithi071.stdout:68719476745 2024-09-16T07:23:00.357 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T07:23:00.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.349+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 --> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b401682f0 con 0x7f7b1c080b20 2024-09-16T07:23:00.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.350+0000 7f7b36000640 1 -- 172.21.15.71:0/109921750 <== osd.7 v2:172.21.15.177:6810/4227809343 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7b401682f0 con 0x7f7b1c080b20 2024-09-16T07:23:00.358 INFO:teuthology.orchestra.run.smithi071.stdout:64424509448 2024-09-16T07:23:00.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.351+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7f7b1c080b20 msgr2=0x7f7b1c082f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.351+0000 7f7b3f200640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7f7b1c080b20 0x7f7b1c082f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.359 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.351+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7b1c0788a0 msgr2=0x7f7b1c07ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.351+0000 7f7b3f200640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7b1c0788a0 0x7f7b1c07ad60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800bfd0 tx=0x7f7b2800b390 comp rx=0 tx=0).stop 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.351+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b4015ebe0 msgr2=0x7f7b40139f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.351+0000 7f7b3f200640 1 --2- 172.21.15.71:0/109921750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b4015ebe0 0x7f7b40139f10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009000 tx=0x7f7b30030650 comp rx=0 tx=0).stop 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.351+0000 7f7b3e800640 1 -- 172.21.15.71:0/109921750 reap_dead start 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.352+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 shutdown_connections 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.352+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 >> 172.21.15.71:0/109921750 conn(0x7f7b4006dc80 msgr2=0x7f7b4006cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.352+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 shutdown_connections 2024-09-16T07:23:00.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.352+0000 7f7b3f200640 1 -- 172.21.15.71:0/109921750 wait complete. 2024-09-16T07:23:00.361 INFO:teuthology.orchestra.run.smithi071.stdout:68719476745 2024-09-16T07:23:00.361 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T07:23:00.362 INFO:teuthology.orchestra.run.smithi071.stdout:68719476745 2024-09-16T07:23:00.366 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T07:23:00.367 INFO:teuthology.orchestra.run.smithi071.stdout:60129542150 2024-09-16T07:23:00.367 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T07:23:00.370 INFO:teuthology.orchestra.run.smithi071.stdout:64424509448 2024-09-16T07:23:00.370 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T07:23:00.377 INFO:teuthology.orchestra.run.smithi071.stdout:64424509446 2024-09-16T07:23:00.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T07:23:00.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.599+0000 7feec6e00640 1 -- 172.21.15.71:0/906639002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec00ce860 msgr2=0x7feec00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.599+0000 7feec6e00640 1 --2- 172.21.15.71:0/906639002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec00ce860 0x7feec00cecc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feeb0005fd0 tx=0x7feeb002a460 comp rx=0 tx=0).stop 2024-09-16T07:23:00.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.599+0000 7feec6e00640 1 -- 172.21.15.71:0/906639002 shutdown_connections 2024-09-16T07:23:00.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.599+0000 7feec6e00640 1 --2- 172.21.15.71:0/906639002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec00ce860 0x7feec00cecc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.599+0000 7feec6e00640 1 -- 172.21.15.71:0/906639002 >> 172.21.15.71:0/906639002 conn(0x7feec00d3de0 msgr2=0x7feec00d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.600+0000 7feec6e00640 1 -- 172.21.15.71:0/906639002 shutdown_connections 2024-09-16T07:23:00.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.601+0000 7feec6e00640 1 -- 172.21.15.71:0/906639002 wait complete. 2024-09-16T07:23:00.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.603+0000 7feec6e00640 1 Processor -- start 2024-09-16T07:23:00.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.604+0000 7feec6e00640 1 -- start start 2024-09-16T07:23:00.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.605+0000 7feec6e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec00b24d0 0x7feec015c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.605+0000 7feec6e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec00ce860 0x7feec00c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.605+0000 7feec6e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feec015dff0 0x7feec00cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.606+0000 7feebfe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec00b24d0 0x7feec015c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.606+0000 7feec6e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feec00b1080 con 0x7feec00ce860 2024-09-16T07:23:00.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.606+0000 7feec6e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feec00b3070 con 0x7feec00b24d0 2024-09-16T07:23:00.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.606+0000 7feec6e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feec00c95a0 con 0x7feec015dff0 2024-09-16T07:23:00.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.606+0000 7feec4e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feec015dff0 0x7feec00cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.606+0000 7feec4e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feec015dff0 0x7feec00cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:42336/0 (socket says 172.21.15.71:42336) 2024-09-16T07:23:00.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.606+0000 7feec4e00640 1 -- 172.21.15.71:0/3575002260 learned_addr learned my addr 172.21.15.71:0/3575002260 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.606+0000 7feebf400640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec00ce860 0x7feec00c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.607+0000 7feec4e00640 1 -- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec00b24d0 msgr2=0x7feec015c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.607+0000 7feec4e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec00b24d0 0x7feec015c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.607+0000 7feec4e00640 1 -- 172.21.15.71:0/3575002260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec00ce860 msgr2=0x7feec00c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.607+0000 7feec4e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec00ce860 0x7feec00c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.607+0000 7feec4e00640 1 -- 172.21.15.71:0/3575002260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec00c85c0 con 0x7feec015dff0 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.607+0000 7feec4e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feec015dff0 0x7feec00cae20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c730 tx=0x7feeac00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.608+0000 7feeb7e00640 1 -- 172.21.15.71:0/3575002260 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac017070 con 0x7feec015dff0 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.608+0000 7feeb7e00640 1 -- 172.21.15.71:0/3575002260 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeac005380 con 0x7feec015dff0 2024-09-16T07:23:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.608+0000 7feeb7e00640 1 -- 172.21.15.71:0/3575002260 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac00a8a0 con 0x7feec015dff0 2024-09-16T07:23:00.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.609+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec00c8000 con 0x7feec015dff0 2024-09-16T07:23:00.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.609+0000 7efd42e00640 1 -- 172.21.15.71:0/28354112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c13d200 msgr2=0x7efd3c0b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.609+0000 7efd42e00640 1 --2- 172.21.15.71:0/28354112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c13d200 0x7efd3c0b3040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009900 tx=0x7efd2c02a6e0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.609+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feec00c9230 con 0x7feec015dff0 2024-09-16T07:23:00.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.609+0000 7efd42e00640 1 -- 172.21.15.71:0/28354112 shutdown_connections 2024-09-16T07:23:00.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.609+0000 7efd42e00640 1 --2- 172.21.15.71:0/28354112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c13d200 0x7efd3c0b3040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7efd42e00640 1 -- 172.21.15.71:0/28354112 >> 172.21.15.71:0/28354112 conn(0x7efd3c06cd50 msgr2=0x7efd3c06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7efd42e00640 1 -- 172.21.15.71:0/28354112 shutdown_connections 2024-09-16T07:23:00.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7f1ecd800640 1 -- 172.21.15.71:0/2718322742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ec8162110 msgr2=0x7f1eac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7f1ecd800640 1 --2- 172.21.15.71:0/2718322742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ec8162110 0x7f1eac0051c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0002230 tx=0x7f1eac005830 comp rx=0 tx=0).stop 2024-09-16T07:23:00.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7f1ec7400640 1 -- 172.21.15.71:0/2718322742 reap_dead start 2024-09-16T07:23:00.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7efd42e00640 1 -- 172.21.15.71:0/28354112 wait complete. 2024-09-16T07:23:00.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7f1ecd800640 1 -- 172.21.15.71:0/2718322742 shutdown_connections 2024-09-16T07:23:00.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7f1ecd800640 1 -- 172.21.15.71:0/2718322742 >> 172.21.15.71:0/2718322742 conn(0x7f1ec80b58b0 msgr2=0x7f1ec80b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.610+0000 7f1ecd800640 1 -- 172.21.15.71:0/2718322742 shutdown_connections 2024-09-16T07:23:00.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.611+0000 7f1ecd800640 1 -- 172.21.15.71:0/2718322742 wait complete. 2024-09-16T07:23:00.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.613+0000 7efd42e00640 1 Processor -- start 2024-09-16T07:23:00.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.613+0000 7efd42e00640 1 -- start start 2024-09-16T07:23:00.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.613+0000 7f1ecd800640 1 Processor -- start 2024-09-16T07:23:00.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.614+0000 7f1ecd800640 1 -- start start 2024-09-16T07:23:00.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.613+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec00cecc0 con 0x7feec015dff0 2024-09-16T07:23:00.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.613+0000 7feeb7e00640 1 -- 172.21.15.71:0/3575002260 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7feeac00aa40 con 0x7feec015dff0 2024-09-16T07:23:00.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.615+0000 7feeb7e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feea0078880 0x7feea007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.615+0000 7feeb7e00640 1 -- 172.21.15.71:0/3575002260 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7feeac099bd0 con 0x7feec015dff0 2024-09-16T07:23:00.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.615+0000 7feebfe00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feea0078880 0x7feea007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.616+0000 7f1ecd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ec8162110 0x7f1ec80b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd42e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c0706f0 0x7efd3c139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd42e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd3c0cfd50 0x7efd3c1feac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ec8162110 0x7f1ec80b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ec8162110 0x7f1ec80b95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50770/0 (socket says 172.21.15.71:50770) 2024-09-16T07:23:00.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7f1ec6a00640 1 -- 172.21.15.71:0/3839240211 learned_addr learned my addr 172.21.15.71:0/3839240211 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7feebfe00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feea0078880 0x7feea007ad40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feec015d120 tx=0x7feeb800d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd42e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd3c13d200 0x7efd3c13b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd42e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c0d0550 con 0x7efd3c0706f0 2024-09-16T07:23:00.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd42e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c0d03d0 con 0x7efd3c13d200 2024-09-16T07:23:00.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd42e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c0d06d0 con 0x7efd3c0cfd50 2024-09-16T07:23:00.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd40e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd3c13d200 0x7efd3c13b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd40e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd3c13d200 0x7efd3c13b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55650/0 (socket says 172.21.15.71:55650) 2024-09-16T07:23:00.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7f1ecd800640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ec814de40 0x7f1ec80c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd40e00640 1 -- 172.21.15.71:0/2727514562 learned_addr learned my addr 172.21.15.71:0/2727514562 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.617+0000 7efd3be00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c0706f0 0x7efd3c139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7efd3b400640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd3c0cfd50 0x7efd3c1feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7efd40e00640 1 -- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd3c0cfd50 msgr2=0x7efd3c1feac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7efd40e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd3c0cfd50 0x7efd3c1feac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7efd40e00640 1 -- 172.21.15.71:0/2727514562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c0706f0 msgr2=0x7efd3c139a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7f1ec6000640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ec814de40 0x7f1ec80c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7f1ecd800640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ec814e2c0 0x7f1ec80c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7efd40e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c0706f0 0x7efd3c139a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7efd40e00640 1 -- 172.21.15.71:0/2727514562 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c1493b0 con 0x7efd3c13d200 2024-09-16T07:23:00.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.618+0000 7efd40e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd3c13d200 0x7efd3c13b450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efd3400c7a0 tx=0x7efd3400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80d29c0 con 0x7f1ec8162110 2024-09-16T07:23:00.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80ce600 con 0x7f1ec814de40 2024-09-16T07:23:00.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7f1ec7400640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ec814e2c0 0x7f1ec80c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7efd33e00640 1 -- 172.21.15.71:0/2727514562 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34016020 con 0x7efd3c13d200 2024-09-16T07:23:00.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c1ff000 con 0x7efd3c13d200 2024-09-16T07:23:00.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec809cd10 con 0x7f1ec814e2c0 2024-09-16T07:23:00.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7efd33e00640 1 -- 172.21.15.71:0/2727514562 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd34006e40 con 0x7efd3c13d200 2024-09-16T07:23:00.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7efd33e00640 1 -- 172.21.15.71:0/2727514562 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34016020 con 0x7efd3c13d200 2024-09-16T07:23:00.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.619+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c0b0f10 con 0x7efd3c13d200 2024-09-16T07:23:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7f1ec6a00640 1 -- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ec814e2c0 msgr2=0x7f1ec80c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7f1ec6a00640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ec814e2c0 0x7f1ec80c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7f1ec6a00640 1 -- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ec814de40 msgr2=0x7f1ec80c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7f1ec6a00640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ec814de40 0x7f1ec80c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7feeb7e00640 1 -- 172.21.15.71:0/3575002260 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feeac061b50 con 0x7feec015dff0 2024-09-16T07:23:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7f1ec6a00640 1 -- 172.21.15.71:0/3839240211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81f8d90 con 0x7f1ec8162110 2024-09-16T07:23:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7f1ec7400640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ec814e2c0 0x7f1ec80c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7f1ec6000640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ec814de40 0x7f1ec80c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.621+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c0b3040 con 0x7efd3c13d200 2024-09-16T07:23:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.622+0000 7f1ec6a00640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ec8162110 0x7f1ec80b95d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8011ab0 tx=0x7f1eb8011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.622+0000 7f14d4400640 1 -- 172.21.15.71:0/3845028248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14cc0d2530 msgr2=0x7f14cc0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.622+0000 7f14d4400640 1 --2- 172.21.15.71:0/3845028248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14cc0d2530 0x7f14cc0d29c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0cf870 tx=0x7f14c002a650 comp rx=0 tx=0).stop 2024-09-16T07:23:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.622+0000 7f1ebea00640 1 -- 172.21.15.71:0/3839240211 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb801e070 con 0x7f1ec8162110 2024-09-16T07:23:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.622+0000 7f1ebea00640 1 -- 172.21.15.71:0/3839240211 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1eb8004460 con 0x7f1ec8162110 2024-09-16T07:23:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.622+0000 7f1ebea00640 1 -- 172.21.15.71:0/3839240211 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb800fc00 con 0x7f1ec8162110 2024-09-16T07:23:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.623+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81f9050 con 0x7f1ec8162110 2024-09-16T07:23:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.623+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81f9610 con 0x7f1ec8162110 2024-09-16T07:23:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.624+0000 7f14d4400640 1 -- 172.21.15.71:0/3845028248 shutdown_connections 2024-09-16T07:23:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.624+0000 7f14d4400640 1 --2- 172.21.15.71:0/3845028248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14cc0d2530 0x7f14cc0d29c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.624+0000 7f14d4400640 1 -- 172.21.15.71:0/3845028248 >> 172.21.15.71:0/3845028248 conn(0x7f14cc0d3fa0 msgr2=0x7f14cc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.624+0000 7efd33e00640 1 -- 172.21.15.71:0/2727514562 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7efd34003970 con 0x7efd3c13d200 2024-09-16T07:23:00.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.624+0000 7f14d4400640 1 -- 172.21.15.71:0/3845028248 shutdown_connections 2024-09-16T07:23:00.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.624+0000 7f14d4400640 1 -- 172.21.15.71:0/3845028248 wait complete. 2024-09-16T07:23:00.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.627+0000 7efd33e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd1c078a80 0x7efd1c07af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.627+0000 7efd33e00640 1 -- 172.21.15.71:0/2727514562 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efd3409a260 con 0x7efd3c13d200 2024-09-16T07:23:00.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.627+0000 7f14d4400640 1 Processor -- start 2024-09-16T07:23:00.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.628+0000 7f14d4400640 1 -- start start 2024-09-16T07:23:00.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.628+0000 7efd3be00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd1c078a80 0x7efd1c07af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.625+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec80d26e0 con 0x7f1ec8162110 2024-09-16T07:23:00.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.630+0000 7f14d4400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14cc0cfb80 0x7f14cc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7efd3be00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd1c078a80 0x7efd1c07af40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd28001610 tx=0x7efd28009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7f14d4400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14cc0d0990 0x7f14cc0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7f14d4400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14cc0d2530 0x7f14cc0b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7efd33e00640 1 -- 172.21.15.71:0/2727514562 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efd34020020 con 0x7efd3c13d200 2024-09-16T07:23:00.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7f14d2400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14cc0d2530 0x7f14cc0b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7f14d4400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc0b3a50 con 0x7f14cc0d0990 2024-09-16T07:23:00.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7f14d4400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc0b38d0 con 0x7f14cc0cfb80 2024-09-16T07:23:00.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7f14d4400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f14cc0b3bd0 con 0x7f14cc0d2530 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.631+0000 7f14d1a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14cc0cfb80 0x7f14cc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.632+0000 7f14d1a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14cc0cfb80 0x7f14cc0b8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55664/0 (socket says 172.21.15.71:55664) 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.632+0000 7f14d1a00640 1 -- 172.21.15.71:0/2387529932 learned_addr learned my addr 172.21.15.71:0/2387529932 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.632+0000 7f14d1a00640 1 -- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14cc0d2530 msgr2=0x7f14cc0b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.632+0000 7f14d1a00640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14cc0d2530 0x7f14cc0b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.632+0000 7f14d1a00640 1 -- 172.21.15.71:0/2387529932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14cc0d0990 msgr2=0x7f14cc0b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.632+0000 7f14d1a00640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14cc0d0990 0x7f14cc0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.632+0000 7f14d1a00640 1 -- 172.21.15.71:0/2387529932 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14cc15f180 con 0x7f14cc0cfb80 2024-09-16T07:23:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.632+0000 7f1ebea00640 1 -- 172.21.15.71:0/3839240211 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1eb8021070 con 0x7f1ec8162110 2024-09-16T07:23:00.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.633+0000 7f14d1a00640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14cc0cfb80 0x7f14cc0b8af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14b800c450 tx=0x7f14b800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.633+0000 7f14c9600640 1 -- 172.21.15.71:0/2387529932 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8016020 con 0x7f14cc0cfb80 2024-09-16T07:23:00.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.633+0000 7f14c9600640 1 -- 172.21.15.71:0/2387529932 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b80053c0 con 0x7f14cc0cfb80 2024-09-16T07:23:00.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.633+0000 7f14c9600640 1 -- 172.21.15.71:0/2387529932 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8007280 con 0x7f14cc0cfb80 2024-09-16T07:23:00.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.634+0000 7f1ebea00640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ea80788a0 0x7f1ea807ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.634+0000 7f1ebea00640 1 -- 172.21.15.71:0/3839240211 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1eb809b880 con 0x7f1ec8162110 2024-09-16T07:23:00.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.635+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14cc1fc800 con 0x7f14cc0cfb80 2024-09-16T07:23:00.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.635+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14cc0b9030 con 0x7f14cc0cfb80 2024-09-16T07:23:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.635+0000 7f1ec6000640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ea80788a0 0x7f1ea807ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.636+0000 7f1ec6000640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ea80788a0 0x7f1ea807ad60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ec815b390 tx=0x7f1eb4005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.638+0000 7f1ebea00640 1 -- 172.21.15.71:0/3839240211 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1ec80d26e0 con 0x7f1ec8162110 2024-09-16T07:23:00.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.641+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a8005290 con 0x7f14cc0cfb80 2024-09-16T07:23:00.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.646+0000 7f14c9600640 1 -- 172.21.15.71:0/2387529932 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f14b8019070 con 0x7f14cc0cfb80 2024-09-16T07:23:00.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.647+0000 7f14c9600640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f14ac078920 0x7f14ac07ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.648+0000 7f14c9600640 1 -- 172.21.15.71:0/2387529932 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f14b8099b90 con 0x7f14cc0cfb80 2024-09-16T07:23:00.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.648+0000 7f14d1000640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f14ac078920 0x7f14ac07ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.649+0000 7f14c9600640 1 -- 172.21.15.71:0/2387529932 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f14b8062270 con 0x7f14cc0cfb80 2024-09-16T07:23:00.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.653+0000 7f14d1000640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f14ac078920 0x7f14ac07ade0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14cc15a550 tx=0x7f14c4007cf0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.697+0000 7f1939200640 1 -- 172.21.15.71:0/47949342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c0e2a50 msgr2=0x7f192c037930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.697+0000 7f1939200640 1 --2- 172.21.15.71:0/47949342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c0e2a50 0x7f192c037930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1920009830 tx=0x7f192002a530 comp rx=0 tx=0).stop 2024-09-16T07:23:00.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.698+0000 7f1939200640 1 -- 172.21.15.71:0/47949342 shutdown_connections 2024-09-16T07:23:00.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.698+0000 7f1939200640 1 --2- 172.21.15.71:0/47949342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192c0e39d0 0x7f192c03a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.698+0000 7f1939200640 1 --2- 172.21.15.71:0/47949342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c0e2a50 0x7f192c037930 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.698+0000 7f1939200640 1 -- 172.21.15.71:0/47949342 >> 172.21.15.71:0/47949342 conn(0x7f192c055530 msgr2=0x7f192c055910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.699+0000 7f1939200640 1 -- 172.21.15.71:0/47949342 shutdown_connections 2024-09-16T07:23:00.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.699+0000 7f1939200640 1 -- 172.21.15.71:0/47949342 wait complete. 2024-09-16T07:23:00.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.699+0000 7f06b4400640 1 -- 172.21.15.71:0/2467522918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a401d460 msgr2=0x7f06a401d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.699+0000 7f06b4400640 1 --2- 172.21.15.71:0/2467522918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a401d460 0x7f06a401d8a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06ac06afd0 tx=0x7f06ac0a5e40 comp rx=0 tx=0).stop 2024-09-16T07:23:00.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.699+0000 7f06b4400640 1 -- 172.21.15.71:0/2467522918 shutdown_connections 2024-09-16T07:23:00.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.699+0000 7f06b4400640 1 --2- 172.21.15.71:0/2467522918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a401d460 0x7f06a401d8a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.699+0000 7f06b4400640 1 --2- 172.21.15.71:0/2467522918 >> v2:172.21.15.71:3300/0 conn(0x7f06a4053990 0x7f06a4005a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.699+0000 7f06b4400640 1 -- 172.21.15.71:0/2467522918 >> 172.21.15.71:0/2467522918 conn(0x7f06a4054550 msgr2=0x7f06a4054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.700+0000 7f06b4400640 1 -- 172.21.15.71:0/2467522918 shutdown_connections 2024-09-16T07:23:00.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.700+0000 7f06b4400640 1 -- 172.21.15.71:0/2467522918 wait complete. 2024-09-16T07:23:00.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.701+0000 7fcf7ca00640 1 -- 172.21.15.71:0/958927317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780725d0 msgr2=0x7fcf780b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.701+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/958927317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780725d0 0x7fcf780b28f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009830 tx=0x7fcf6802a480 comp rx=0 tx=0).stop 2024-09-16T07:23:00.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.702+0000 7f06b4400640 1 Processor -- start 2024-09-16T07:23:00.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.702+0000 7fcf7ca00640 1 -- 172.21.15.71:0/958927317 shutdown_connections 2024-09-16T07:23:00.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.702+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/958927317 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf780d0540 0x7fcf7806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.702+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/958927317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780725d0 0x7fcf780b28f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.702+0000 7fcf7ca00640 1 -- 172.21.15.71:0/958927317 >> 172.21.15.71:0/958927317 conn(0x7fcf780cfa20 msgr2=0x7fcf780cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.703+0000 7f1939200640 1 Processor -- start 2024-09-16T07:23:00.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.703+0000 7fcf7ca00640 1 -- 172.21.15.71:0/958927317 shutdown_connections 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.702+0000 7f06b4400640 1 -- start start 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.704+0000 7f06b4400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a400ad60 0x7f06a4049040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.704+0000 7f06b4400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a401d460 0x7f06a40db470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.704+0000 7f06b4400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06a4053990 0x7f06a401f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.704+0000 7f06b4400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06a400b370 con 0x7f06a401d460 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.704+0000 7f06b4400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f06a400b1f0 con 0x7f06a4053990 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.704+0000 7f06b4400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f06a400b4f0 con 0x7f06a400ad60 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.704+0000 7f06b1a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a400ad60 0x7f06a4049040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.705+0000 7f06b1a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a400ad60 0x7f06a4049040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:42414/0 (socket says 172.21.15.71:42414) 2024-09-16T07:23:00.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.705+0000 7f06b1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a401d460 0x7f06a40db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.705+0000 7f06b2400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06a4053990 0x7f06a401f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.705+0000 7f06b1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a401d460 0x7f06a40db470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50812/0 (socket says 172.21.15.71:50812) 2024-09-16T07:23:00.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.705+0000 7f06b1000640 1 -- 172.21.15.71:0/455701593 learned_addr learned my addr 172.21.15.71:0/455701593 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.703+0000 7fcf7ca00640 1 -- 172.21.15.71:0/958927317 wait complete. 2024-09-16T07:23:00.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.703+0000 7f1939200640 1 -- start start 2024-09-16T07:23:00.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7f1939200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c03a930 0x7f192c036ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7f06b2400640 1 -- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a400ad60 msgr2=0x7f06a4049040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7f06b2400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a400ad60 0x7f06a4049040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7f06b2400640 1 -- 172.21.15.71:0/455701593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a401d460 msgr2=0x7f06a40db470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7fcf7ca00640 1 Processor -- start 2024-09-16T07:23:00.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7f1939200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192c0e2a50 0x7f192c0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7f1939200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192c0e39d0 0x7f192c056d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7f1931e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c03a930 0x7f192c036ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.706+0000 7fcf7ca00640 1 -- start start 2024-09-16T07:23:00.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.707+0000 7f06b2400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a401d460 0x7f06a40db470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.707+0000 7f1931e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c03a930 0x7f192c036ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50814/0 (socket says 172.21.15.71:50814) 2024-09-16T07:23:00.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.707+0000 7f1931e00640 1 -- 172.21.15.71:0/3076217353 learned_addr learned my addr 172.21.15.71:0/3076217353 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.707+0000 7f1932800640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192c0e39d0 0x7f192c056d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.707+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f192c0e6dd0 con 0x7f192c03a930 2024-09-16T07:23:00.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.707+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f192c0e6c50 con 0x7f192c0e39d0 2024-09-16T07:23:00.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.707+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f192c0e6f50 con 0x7f192c0e2a50 2024-09-16T07:23:00.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.707+0000 7f1931400640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192c0e2a50 0x7f192c0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7fcf7ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780d0540 0x7fcf7815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7f1931400640 1 -- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192c0e39d0 msgr2=0x7f192c056d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7f1931400640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192c0e39d0 0x7f192c056d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7f1931400640 1 -- 172.21.15.71:0/3076217353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c03a930 msgr2=0x7f192c036ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7f06b2400640 1 -- 172.21.15.71:0/455701593 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a417b2e0 con 0x7f06a4053990 2024-09-16T07:23:00.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7f06b1a00640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a400ad60 0x7f06a4049040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:00.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7f06b1000640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a401d460 0x7f06a40db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:00.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7fcf7ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7815bc30 0x7fcf781585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7fcf7ca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf78158b30 0x7fcf7815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7fcf7ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78074660 con 0x7fcf7815bc30 2024-09-16T07:23:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7fcf7ca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7806f860 con 0x7fcf780d0540 2024-09-16T07:23:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7fcf7ca00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7815d970 con 0x7fcf78158b30 2024-09-16T07:23:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7f1931400640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c03a930 0x7f192c036ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.708+0000 7f1931400640 1 -- 172.21.15.71:0/3076217353 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c1798e0 con 0x7f192c0e2a50 2024-09-16T07:23:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780d0540 0x7fcf7815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780d0540 0x7fcf7815e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55692/0 (socket says 172.21.15.71:55692) 2024-09-16T07:23:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77400640 1 -- 172.21.15.71:0/4272040620 learned_addr learned my addr 172.21.15.71:0/4272040620 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77e00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf78158b30 0x7fcf7815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7f1931e00640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c03a930 0x7f192c036ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf76a00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7815bc30 0x7fcf781585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7f1931400640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192c0e2a50 0x7f192c0d2800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f192400d660 tx=0x7f192400db20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77400640 1 -- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf78158b30 msgr2=0x7fcf7815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77400640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf78158b30 0x7fcf7815c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77400640 1 -- 172.21.15.71:0/4272040620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7815bc30 msgr2=0x7fcf781585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77400640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7815bc30 0x7fcf781585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7fcf77400640 1 -- 172.21.15.71:0/4272040620 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7815deb0 con 0x7fcf780d0540 2024-09-16T07:23:00.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7fcf76a00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7815bc30 0x7fcf781585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:00.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.709+0000 7f06b2400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06a4053990 0x7f06a401f780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f069800c410 tx=0x7f069800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7fcf77400640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780d0540 0x7fcf7815e990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000ba50 tx=0x7fcf7000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7f192a000640 1 -- 172.21.15.71:0/3076217353 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924017070 con 0x7f192c0e2a50 2024-09-16T07:23:00.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7fcf6f400640 1 -- 172.21.15.71:0/4272040620 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70019070 con 0x7fcf780d0540 2024-09-16T07:23:00.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7fcf6f400640 1 -- 172.21.15.71:0/4272040620 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf7000cd00 con 0x7fcf780d0540 2024-09-16T07:23:00.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7fcf6f400640 1 -- 172.21.15.71:0/4272040620 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700146a0 con 0x7fcf780d0540 2024-09-16T07:23:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7f06a9600640 1 -- 172.21.15.71:0/455701593 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698011790 con 0x7f06a4053990 2024-09-16T07:23:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7f06a9600640 1 -- 172.21.15.71:0/455701593 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0698017070 con 0x7f06a4053990 2024-09-16T07:23:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7f06a9600640 1 -- 172.21.15.71:0/455701593 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698011cc0 con 0x7f06a4053990 2024-09-16T07:23:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192c179bc0 con 0x7f192c0e2a50 2024-09-16T07:23:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.710+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f192c17a180 con 0x7f192c0e2a50 2024-09-16T07:23:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.711+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7815e190 con 0x7fcf780d0540 2024-09-16T07:23:00.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.711+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf78201fd0 con 0x7fcf780d0540 2024-09-16T07:23:00.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.711+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a4021ca0 con 0x7f06a4053990 2024-09-16T07:23:00.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.711+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a4021950 con 0x7f06a4053990 2024-09-16T07:23:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.712+0000 7f192a000640 1 -- 172.21.15.71:0/3076217353 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1924009a20 con 0x7f192c0e2a50 2024-09-16T07:23:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.713+0000 7f192a000640 1 -- 172.21.15.71:0/3076217353 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924017070 con 0x7f192c0e2a50 2024-09-16T07:23:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.713+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a401d8a0 con 0x7f06a4053990 2024-09-16T07:23:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.714+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7815dcb0 con 0x7fcf780d0540 2024-09-16T07:23:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.715+0000 7fcf6f400640 1 -- 172.21.15.71:0/4272040620 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fcf7001c070 con 0x7fcf780d0540 2024-09-16T07:23:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.715+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c005290 con 0x7f192c0e2a50 2024-09-16T07:23:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.716+0000 7f06a9600640 1 -- 172.21.15.71:0/455701593 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0698007020 con 0x7f06a4053990 2024-09-16T07:23:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.716+0000 7fcf6f400640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf58078850 0x7fcf5807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.717+0000 7fcf6f400640 1 -- 172.21.15.71:0/4272040620 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcf7009a700 con 0x7fcf780d0540 2024-09-16T07:23:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.717+0000 7f192a000640 1 -- 172.21.15.71:0/3076217353 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f192401a070 con 0x7f192c0e2a50 2024-09-16T07:23:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.717+0000 7fcf76a00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf58078850 0x7fcf5807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.718+0000 7f06a9600640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07ccb0 0x7f068c07f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.718+0000 7f06a9600640 1 -- 172.21.15.71:0/455701593 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f069809a6e0 con 0x7f06a4053990 2024-09-16T07:23:00.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.718+0000 7fcf76a00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf58078850 0x7fcf5807ad10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcf68034000 tx=0x7fcf68004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.718+0000 7f06b1a00640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07ccb0 0x7f068c07f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.719+0000 7f06b1a00640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07ccb0 0x7f068c07f170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06a000aef0 tx=0x7f06a000a250 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.719+0000 7f192a000640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1914078c80 0x7f191407b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.720+0000 7f192a000640 1 -- 172.21.15.71:0/3076217353 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1924009820 con 0x7f192c0e2a50 2024-09-16T07:23:00.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.720+0000 7f1931e00640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1914078c80 0x7f191407b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.720+0000 7f06a9600640 1 -- 172.21.15.71:0/455701593 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f06980628a0 con 0x7f06a4053990 2024-09-16T07:23:00.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.721+0000 7f1931e00640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1914078c80 0x7f191407b140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1920035040 tx=0x7f1920004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.725+0000 7fcf6f400640 1 -- 172.21.15.71:0/4272040620 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcf70062d60 con 0x7fcf780d0540 2024-09-16T07:23:00.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.725+0000 7f192a000640 1 -- 172.21.15.71:0/3076217353 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1924021030 con 0x7f192c0e2a50 2024-09-16T07:23:00.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.767+0000 7fe891a00640 1 -- 172.21.15.71:0/1441571380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c1438b0 msgr2=0x7fe88c0cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.767+0000 7fe891a00640 1 --2- 172.21.15.71:0/1441571380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c1438b0 0x7fe88c0cc5f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0b2560 tx=0x7fe88402b8a0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.768+0000 7fe891a00640 1 -- 172.21.15.71:0/1441571380 shutdown_connections 2024-09-16T07:23:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.768+0000 7fe891a00640 1 --2- 172.21.15.71:0/1441571380 >> v2:172.21.15.71:3300/0 conn(0x7fe88c06ff60 0x7fe88c0b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.768+0000 7fe891a00640 1 --2- 172.21.15.71:0/1441571380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c1438b0 0x7fe88c0cc5f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.768+0000 7fe891a00640 1 -- 172.21.15.71:0/1441571380 >> 172.21.15.71:0/1441571380 conn(0x7fe88c0b1950 msgr2=0x7fe88c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:00.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.769+0000 7fe891a00640 1 -- 172.21.15.71:0/1441571380 shutdown_connections 2024-09-16T07:23:00.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.769+0000 7fe891a00640 1 -- 172.21.15.71:0/1441571380 wait complete. 2024-09-16T07:23:00.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.773+0000 7fe891a00640 1 Processor -- start 2024-09-16T07:23:00.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.773+0000 7fe891a00640 1 -- start start 2024-09-16T07:23:00.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.775+0000 7fe891a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c06ff60 0x7fe88c09d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.776+0000 7fe891a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0ccb30 0x7fe88c0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.776+0000 7fe891a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c1438b0 0x7fe88c1f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.776+0000 7fe891a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c070b30 con 0x7fe88c06ff60 2024-09-16T07:23:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.777+0000 7fe88aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c06ff60 0x7fe88c09d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.777+0000 7fe891a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c06efa0 con 0x7fe88c1438b0 2024-09-16T07:23:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.777+0000 7fe891a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c0b4440 con 0x7fe88c0ccb30 2024-09-16T07:23:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.777+0000 7fe88aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c06ff60 0x7fe88c09d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50838/0 (socket says 172.21.15.71:50838) 2024-09-16T07:23:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.777+0000 7fe88aa00640 1 -- 172.21.15.71:0/3737437889 learned_addr learned my addr 172.21.15.71:0/3737437889 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:00.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.778+0000 7fe88aa00640 1 -- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0ccb30 msgr2=0x7fe88c0b1240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:23:00.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.778+0000 7fe88aa00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0ccb30 0x7fe88c0b1240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.778+0000 7fe88aa00640 1 -- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c1438b0 msgr2=0x7fe88c1f0e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T07:23:00.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.779+0000 7fe88aa00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c1438b0 0x7fe88c1f0e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:00.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.779+0000 7fe88aa00640 1 -- 172.21.15.71:0/3737437889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c0b04a0 con 0x7fe88c06ff60 2024-09-16T07:23:00.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.780+0000 7fe88aa00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c06ff60 0x7fe88c09d2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe884009850 tx=0x7fe884003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.780+0000 7fe882a00640 1 -- 172.21.15.71:0/3737437889 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8840319b0 con 0x7fe88c06ff60 2024-09-16T07:23:00.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.780+0000 7fe882a00640 1 -- 172.21.15.71:0/3737437889 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe884031c40 con 0x7fe88c06ff60 2024-09-16T07:23:00.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.780+0000 7fe882a00640 1 -- 172.21.15.71:0/3737437889 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884032430 con 0x7fe88c06ff60 2024-09-16T07:23:00.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.780+0000 7fe891a00640 1 -- 172.21.15.71:0/3737437889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c09d940 con 0x7fe88c06ff60 2024-09-16T07:23:00.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.781+0000 7fe891a00640 1 -- 172.21.15.71:0/3737437889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c0b0a40 con 0x7fe88c06ff60 2024-09-16T07:23:00.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.783+0000 7fe87be00640 1 -- 172.21.15.71:0/3737437889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe85c005290 con 0x7fe88c06ff60 2024-09-16T07:23:00.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.786+0000 7fe882a00640 1 -- 172.21.15.71:0/3737437889 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe884044020 con 0x7fe88c06ff60 2024-09-16T07:23:00.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.788+0000 7fe882a00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe868078850 0x7fe86807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:00.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.788+0000 7fe882a00640 1 -- 172.21.15.71:0/3737437889 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe8840bbf90 con 0x7fe88c06ff60 2024-09-16T07:23:00.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.789+0000 7fe88a000640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe868078850 0x7fe86807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:00.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.790+0000 7fe88a000640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe868078850 0x7fe86807ad10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe88c1edf00 tx=0x7fe8740079b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:00.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:00.796+0000 7fe882a00640 1 -- 172.21.15.71:0/3737437889 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe884084560 con 0x7fe88c06ff60 2024-09-16T07:23:01.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.046+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f14a8005580 con 0x7f14cc0cfb80 2024-09-16T07:23:01.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.047+0000 7f14c9600640 1 -- 172.21.15.71:0/2387529932 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14b8061b70 con 0x7f14cc0cfb80 2024-09-16T07:23:01.051 INFO:teuthology.orchestra.run.smithi071.stdout:68719476744 2024-09-16T07:23:01.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.052+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f14ac078920 msgr2=0x7f14ac07ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.052+0000 7f14d4400640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f14ac078920 0x7f14ac07ade0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f14cc15a550 tx=0x7f14c4007cf0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.052+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14cc0cfb80 msgr2=0x7f14cc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.053+0000 7f14d4400640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14cc0cfb80 0x7f14cc0b8af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14b800c450 tx=0x7f14b800c910 comp rx=0 tx=0).stop 2024-09-16T07:23:01.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.053+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 shutdown_connections 2024-09-16T07:23:01.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.053+0000 7f14d4400640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f14ac078920 0x7f14ac07ade0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.053+0000 7f14d4400640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14cc0d2530 0x7f14cc0b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.053+0000 7f14d4400640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14cc0d0990 0x7f14cc0b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.053+0000 7f14d4400640 1 --2- 172.21.15.71:0/2387529932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14cc0cfb80 0x7f14cc0b8af0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.054+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 >> 172.21.15.71:0/2387529932 conn(0x7f14cc0d3fa0 msgr2=0x7f14cc15d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:01.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.054+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 shutdown_connections 2024-09-16T07:23:01.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.054+0000 7f14d4400640 1 -- 172.21.15.71:0/2387529932 wait complete. 2024-09-16T07:23:01.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.056+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efd3c0cf7c0 con 0x7efd3c13d200 2024-09-16T07:23:01.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.057+0000 7efd33e00640 1 -- 172.21.15.71:0/2727514562 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efd340628c0 con 0x7efd3c13d200 2024-09-16T07:23:01.057 INFO:teuthology.orchestra.run.smithi071.stdout:64424509447 2024-09-16T07:23:01.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd1c078a80 msgr2=0x7efd1c07af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd1c078a80 0x7efd1c07af40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd28001610 tx=0x7efd28009040 comp rx=0 tx=0).stop 2024-09-16T07:23:01.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd3c13d200 msgr2=0x7efd3c13b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd3c13d200 0x7efd3c13b450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efd3400c7a0 tx=0x7efd3400cc60 comp rx=0 tx=0).stop 2024-09-16T07:23:01.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 shutdown_connections 2024-09-16T07:23:01.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd1c078a80 0x7efd1c07af40 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd28001610 tx=0x7efd28009040 comp rx=0 tx=0).stop 2024-09-16T07:23:01.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd3c13d200 0x7efd3c13b450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd3c0cfd50 0x7efd3c1feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 --2- 172.21.15.71:0/2727514562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3c0706f0 0x7efd3c139a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.061+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 >> 172.21.15.71:0/2727514562 conn(0x7efd3c06cd50 msgr2=0x7efd3c0b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:01.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.062+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 shutdown_connections 2024-09-16T07:23:01.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.063+0000 7efd42e00640 1 -- 172.21.15.71:0/2727514562 wait complete. 2024-09-16T07:23:01.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.073+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7feec0158450 con 0x7feec015dff0 2024-09-16T07:23:01.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.074+0000 7feeb7e00640 1 -- 172.21.15.71:0/3575002260 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feeac066580 con 0x7feec015dff0 2024-09-16T07:23:01.075 INFO:teuthology.orchestra.run.smithi071.stdout:68719476743 2024-09-16T07:23:01.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.078+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1ec80b5580 con 0x7f1ec8162110 2024-09-16T07:23:01.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.078+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feea0078880 msgr2=0x7feea007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feea0078880 0x7feea007ad40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feec015d120 tx=0x7feeb800d040 comp rx=0 tx=0).stop 2024-09-16T07:23:01.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feec015dff0 msgr2=0x7feec00cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feec015dff0 0x7feec00cae20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c730 tx=0x7feeac00cbf0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 shutdown_connections 2024-09-16T07:23:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feea0078880 0x7feea007ad40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feec015dff0 0x7feec00cae20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feec00ce860 0x7feec00c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 --2- 172.21.15.71:0/3575002260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec00b24d0 0x7feec015c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 >> 172.21.15.71:0/3575002260 conn(0x7feec00d3de0 msgr2=0x7feec00d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7f1ebea00640 1 -- 172.21.15.71:0/3839240211 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ec80b5580 con 0x7f1ec8162110 2024-09-16T07:23:01.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 shutdown_connections 2024-09-16T07:23:01.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.079+0000 7feec6e00640 1 -- 172.21.15.71:0/3575002260 wait complete. 2024-09-16T07:23:01.081 INFO:teuthology.orchestra.run.smithi071.stdout:68719476744 2024-09-16T07:23:01.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476744 for osd.3 2024-09-16T07:23:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.083+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ea80788a0 msgr2=0x7f1ea807ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.083+0000 7f1ecd800640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ea80788a0 0x7f1ea807ad60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ec815b390 tx=0x7f1eb4005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.083+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ec8162110 msgr2=0x7f1ec80b95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.083+0000 7f1ecd800640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ec8162110 0x7f1ec80b95d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8011ab0 tx=0x7f1eb8011f70 comp rx=0 tx=0).stop 2024-09-16T07:23:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.084+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 shutdown_connections 2024-09-16T07:23:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.084+0000 7f1ecd800640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ea80788a0 0x7f1ea807ad60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.084+0000 7f1ecd800640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ec814e2c0 0x7f1ec80c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.084+0000 7f1ecd800640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ec814de40 0x7f1ec80c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.084+0000 7f1ecd800640 1 --2- 172.21.15.71:0/3839240211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ec8162110 0x7f1ec80b95d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.084+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 >> 172.21.15.71:0/3839240211 conn(0x7f1ec80b58b0 msgr2=0x7f1ec806d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:01.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.084+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 shutdown_connections 2024-09-16T07:23:01.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.085+0000 7f1ecd800640 1 -- 172.21.15.71:0/3839240211 wait complete. 2024-09-16T07:23:01.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.6 2024-09-16T07:23:01.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476744 got 68719476743 for osd.0 2024-09-16T07:23:01.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476744 for osd.1 2024-09-16T07:23:01.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.121+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f190c005850 con 0x7f192c0e2a50 2024-09-16T07:23:01.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.122+0000 7f192a000640 1 -- 172.21.15.71:0/3076217353 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1924061f20 con 0x7f192c0e2a50 2024-09-16T07:23:01.123 INFO:teuthology.orchestra.run.smithi071.stdout:68719476744 2024-09-16T07:23:01.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1914078c80 msgr2=0x7f191407b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1914078c80 0x7f191407b140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1920035040 tx=0x7f1920004730 comp rx=0 tx=0).stop 2024-09-16T07:23:01.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192c0e2a50 msgr2=0x7f192c0d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192c0e2a50 0x7f192c0d2800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f192400d660 tx=0x7f192400db20 comp rx=0 tx=0).stop 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 shutdown_connections 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1914078c80 0x7f191407b140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192c0e39d0 0x7f192c056d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192c0e2a50 0x7f192c0d2800 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 --2- 172.21.15.71:0/3076217353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f192c03a930 0x7f192c036ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 >> 172.21.15.71:0/3076217353 conn(0x7f192c055530 msgr2=0x7f192c03c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 shutdown_connections 2024-09-16T07:23:01.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.126+0000 7f1939200640 1 -- 172.21.15.71:0/3076217353 wait complete. 2024-09-16T07:23:01.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476744 for osd.2 2024-09-16T07:23:01.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.155+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f06a400ab40 con 0x7f06a4053990 2024-09-16T07:23:01.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.155+0000 7f06a9600640 1 -- 172.21.15.71:0/455701593 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06980672d0 con 0x7f06a4053990 2024-09-16T07:23:01.156 INFO:teuthology.orchestra.run.smithi071.stdout:60129542149 2024-09-16T07:23:01.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.156+0000 7fe87be00640 1 -- 172.21.15.71:0/3737437889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe85c005580 con 0x7fe88c06ff60 2024-09-16T07:23:01.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.157+0000 7fe882a00640 1 -- 172.21.15.71:0/3737437889 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe884083e60 con 0x7fe88c06ff60 2024-09-16T07:23:01.157 INFO:teuthology.orchestra.run.smithi071.stdout:64424509445 2024-09-16T07:23:01.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07ccb0 msgr2=0x7f068c07f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07ccb0 0x7f068c07f170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06a000aef0 tx=0x7f06a000a250 comp rx=0 tx=0).stop 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06a4053990 msgr2=0x7f06a401f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06a4053990 0x7f06a401f780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f069800c410 tx=0x7f069800c8d0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 shutdown_connections 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07ccb0 0x7f068c07f170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06a4053990 0x7f06a401f780 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06a401d460 0x7f06a40db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 --2- 172.21.15.71:0/455701593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06a400ad60 0x7f06a4049040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 >> 172.21.15.71:0/455701593 conn(0x7f06a4054550 msgr2=0x7f06a4052970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:01.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 -- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe868078850 msgr2=0x7fe86807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe868078850 0x7fe86807ad10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe88c1edf00 tx=0x7fe8740079b0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 shutdown_connections 2024-09-16T07:23:01.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 -- 172.21.15.71:0/3737437889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c06ff60 msgr2=0x7fe88c09d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c06ff60 0x7fe88c09d2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe884009850 tx=0x7fe884003ed0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 -- 172.21.15.71:0/3737437889 shutdown_connections 2024-09-16T07:23:01.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7f06b4400640 1 -- 172.21.15.71:0/455701593 wait complete. 2024-09-16T07:23:01.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe868078850 0x7fe86807ad10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c1438b0 0x7fe88c1f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0ccb30 0x7fe88c0b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 --2- 172.21.15.71:0/3737437889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c06ff60 0x7fe88c09d2b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 -- 172.21.15.71:0/3737437889 >> 172.21.15.71:0/3737437889 conn(0x7fe88c0b1950 msgr2=0x7fe88c142740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:01.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 -- 172.21.15.71:0/3737437889 shutdown_connections 2024-09-16T07:23:01.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.159+0000 7fe87be00640 1 -- 172.21.15.71:0/3737437889 wait complete. 2024-09-16T07:23:01.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.168+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcf7806fdf0 con 0x7fcf780d0540 2024-09-16T07:23:01.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.169+0000 7fcf6f400640 1 -- 172.21.15.71:0/4272040620 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf70062660 con 0x7fcf780d0540 2024-09-16T07:23:01.170 INFO:teuthology.orchestra.run.smithi071.stdout:64424509447 2024-09-16T07:23:01.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.172+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf58078850 msgr2=0x7fcf5807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.172+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf58078850 0x7fcf5807ad10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcf68034000 tx=0x7fcf68004730 comp rx=0 tx=0).stop 2024-09-16T07:23:01.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780d0540 msgr2=0x7fcf7815e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780d0540 0x7fcf7815e990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000ba50 tx=0x7fcf7000bf10 comp rx=0 tx=0).stop 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 shutdown_connections 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf58078850 0x7fcf5807ad10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf78158b30 0x7fcf7815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf7815bc30 0x7fcf781585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 --2- 172.21.15.71:0/4272040620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf780d0540 0x7fcf7815e990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 >> 172.21.15.71:0/4272040620 conn(0x7fcf780cfa20 msgr2=0x7fcf780ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 shutdown_connections 2024-09-16T07:23:01.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:01.173+0000 7fcf7ca00640 1 -- 172.21.15.71:0/4272040620 wait complete. 2024-09-16T07:23:01.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-09-16T07:23:01.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2024-09-16T07:23:01.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-09-16T07:23:02.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T07:23:02.090 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T07:23:02.103 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T07:23:02.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T07:23:02.142 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T07:23:02.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T07:23:02.181 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T07:23:02.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T07:23:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.308+0000 7f8b1b000640 1 -- 172.21.15.71:0/4189979585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 msgr2=0x7f8b140b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.308+0000 7f8b1b000640 1 --2- 172.21.15.71:0/4189979585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 0x7f8b140b4ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8b08005fd0 tx=0x7f8b0802a5d0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.308+0000 7f8b1b000640 1 -- 172.21.15.71:0/4189979585 shutdown_connections 2024-09-16T07:23:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.308+0000 7f8b1b000640 1 --2- 172.21.15.71:0/4189979585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b140b5410 0x7f8b140b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.308+0000 7f8b1b000640 1 --2- 172.21.15.71:0/4189979585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 0x7f8b140b4ed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.308+0000 7f8b1b000640 1 -- 172.21.15.71:0/4189979585 >> 172.21.15.71:0/4189979585 conn(0x7f8b140d3df0 msgr2=0x7f8b140d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.309+0000 7f8b1b000640 1 -- 172.21.15.71:0/4189979585 shutdown_connections 2024-09-16T07:23:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.309+0000 7f8b1b000640 1 -- 172.21.15.71:0/4189979585 wait complete. 2024-09-16T07:23:02.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.312+0000 7f8b1b000640 1 Processor -- start 2024-09-16T07:23:02.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.312+0000 7f8b1b000640 1 -- start start 2024-09-16T07:23:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.314+0000 7f8b1b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 0x7f8b14154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.315+0000 7f8b1b000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b140b5410 0x7f8b140c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.316+0000 7f8b1b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b140d1fd0 0x7f8b140b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.316+0000 7f8b19200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b140b5410 0x7f8b140c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.316+0000 7f8b19200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b140b5410 0x7f8b140c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:42456/0 (socket says 172.21.15.71:42456) 2024-09-16T07:23:02.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.316+0000 7f8b19200640 1 -- 172.21.15.71:0/102479307 learned_addr learned my addr 172.21.15.71:0/102479307 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:02.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.317+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b140b1520 con 0x7f8b140b4860 2024-09-16T07:23:02.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.317+0000 7f8b1a600640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b140d1fd0 0x7f8b140b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.317+0000 7f8b19c00640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 0x7f8b14154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.317+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b140b13a0 con 0x7f8b140d1fd0 2024-09-16T07:23:02.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.317+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b140b16a0 con 0x7f8b140b5410 2024-09-16T07:23:02.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.318+0000 7f8b1a600640 1 -- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b140b5410 msgr2=0x7f8b140c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.318+0000 7f8b1a600640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b140b5410 0x7f8b140c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.319+0000 7f8b1a600640 1 -- 172.21.15.71:0/102479307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 msgr2=0x7f8b14154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.319+0000 7f8b1a600640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 0x7f8b14154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.319+0000 7f8b1a600640 1 -- 172.21.15.71:0/102479307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b141551e0 con 0x7f8b140d1fd0 2024-09-16T07:23:02.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.319+0000 7f8b19c00640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 0x7f8b14154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:02.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.320+0000 7f8b19200640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b140b5410 0x7f8b140c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:02.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.320+0000 7f8b1a600640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b140d1fd0 0x7f8b140b74c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c970 tx=0x7f8b0000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.321+0000 7f8b11600640 1 -- 172.21.15.71:0/102479307 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00007bb0 con 0x7f8b140d1fd0 2024-09-16T07:23:02.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.321+0000 7f8b11600640 1 -- 172.21.15.71:0/102479307 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b0000f450 con 0x7f8b140d1fd0 2024-09-16T07:23:02.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.321+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b140c7d90 con 0x7f8b140d1fd0 2024-09-16T07:23:02.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.321+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b140c8350 con 0x7f8b140d1fd0 2024-09-16T07:23:02.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.323+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b140cf0d0 con 0x7f8b140d1fd0 2024-09-16T07:23:02.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.323+0000 7f8b11600640 1 -- 172.21.15.71:0/102479307 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00007e60 con 0x7f8b140d1fd0 2024-09-16T07:23:02.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.328+0000 7f8b11600640 1 -- 172.21.15.71:0/102479307 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8b0000f8a0 con 0x7f8b140d1fd0 2024-09-16T07:23:02.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.330+0000 7f8b11600640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8af4078850 0x7f8af407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.331+0000 7f8b11600640 1 -- 172.21.15.71:0/102479307 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8b0009b160 con 0x7f8b140d1fd0 2024-09-16T07:23:02.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.331+0000 7f8b19c00640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8af4078850 0x7f8af407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.332+0000 7f8b11600640 1 -- 172.21.15.71:0/102479307 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b140cf0d0 con 0x7f8b140d1fd0 2024-09-16T07:23:02.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.332+0000 7f8b19c00640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8af4078850 0x7f8af407ad10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0012e0 tx=0x7f8b0c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.385+0000 7efdde400640 1 -- 172.21.15.71:0/2859341841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 msgr2=0x7efdd80ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.385+0000 7efdde400640 1 --2- 172.21.15.71:0/2859341841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 0x7efdd80ce860 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009ca0 tx=0x7efdd002aaf0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.385+0000 7efdde400640 1 -- 172.21.15.71:0/2859341841 shutdown_connections 2024-09-16T07:23:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.385+0000 7efdde400640 1 --2- 172.21.15.71:0/2859341841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 0x7efdd80ce860 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.385+0000 7efdde400640 1 --2- 172.21.15.71:0/2859341841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd815dff0 0x7efdd80b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.385+0000 7efdde400640 1 -- 172.21.15.71:0/2859341841 >> 172.21.15.71:0/2859341841 conn(0x7efdd80d3de0 msgr2=0x7efdd80d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.385+0000 7efdde400640 1 -- 172.21.15.71:0/2859341841 shutdown_connections 2024-09-16T07:23:02.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.386+0000 7efdde400640 1 -- 172.21.15.71:0/2859341841 wait complete. 2024-09-16T07:23:02.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f6332c00640 1 -- 172.21.15.71:0/1827172716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632c161440 msgr2=0x7f632c0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f6332c00640 1 --2- 172.21.15.71:0/1827172716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632c161440 0x7f632c0b5370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f632c0d4ea0 tx=0x7f632002a950 comp rx=0 tx=0).stop 2024-09-16T07:23:02.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f43d5600640 1 -- 172.21.15.71:0/4233399006 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8031c30 con 0x7f43d0009e40 2024-09-16T07:23:02.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f43dd800640 1 -- 172.21.15.71:0/4233399006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0009e40 msgr2=0x7f43d00376c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f6332c00640 1 -- 172.21.15.71:0/1827172716 shutdown_connections 2024-09-16T07:23:02.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f6332c00640 1 --2- 172.21.15.71:0/1827172716 >> v2:172.21.15.177:3300/0 conn(0x7f632c0b34b0 0x7f632c0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f6332c00640 1 --2- 172.21.15.71:0/1827172716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632c161440 0x7f632c0b5370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f6332c00640 1 -- 172.21.15.71:0/1827172716 >> 172.21.15.71:0/1827172716 conn(0x7f632c0d3df0 msgr2=0x7f632c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7efdde400640 1 Processor -- start 2024-09-16T07:23:02.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f43dd800640 1 --2- 172.21.15.71:0/4233399006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0009e40 0x7f43d00376c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43d000d2f0 tx=0x7f43c802a560 comp rx=0 tx=0).stop 2024-09-16T07:23:02.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.389+0000 7efdde400640 1 -- start start 2024-09-16T07:23:02.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.389+0000 7f43dd800640 1 -- 172.21.15.71:0/4233399006 shutdown_connections 2024-09-16T07:23:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.389+0000 7f43dd800640 1 --2- 172.21.15.71:0/4233399006 >> v2:172.21.15.177:3300/0 conn(0x7f43d000a220 0x7f43d0036010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.389+0000 7f43dd800640 1 --2- 172.21.15.71:0/4233399006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d0009e40 0x7f43d00376c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.389+0000 7f43dd800640 1 -- 172.21.15.71:0/4233399006 >> 172.21.15.71:0/4233399006 conn(0x7f43d0054590 msgr2=0x7f43d0054970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.389+0000 7f43dd800640 1 -- 172.21.15.71:0/4233399006 shutdown_connections 2024-09-16T07:23:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.390+0000 7f43dd800640 1 -- 172.21.15.71:0/4233399006 wait complete. 2024-09-16T07:23:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f6332c00640 1 -- 172.21.15.71:0/1827172716 shutdown_connections 2024-09-16T07:23:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.388+0000 7f6332c00640 1 -- 172.21.15.71:0/1827172716 wait complete. 2024-09-16T07:23:02.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.391+0000 7efdde400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 0x7efdd80b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.391+0000 7f6332c00640 1 Processor -- start 2024-09-16T07:23:02.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.391+0000 7f6332c00640 1 -- start start 2024-09-16T07:23:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.391+0000 7efdde400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd80ceda0 0x7efdd80cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.391+0000 7efdde400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd815dff0 0x7efdd80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.392+0000 7efdde400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efdd80d2030 con 0x7efdd80b0eb0 2024-09-16T07:23:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.392+0000 7efdd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 0x7efdd80b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.392+0000 7efdde400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efdd80b6c10 con 0x7efdd80ceda0 2024-09-16T07:23:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.392+0000 7efdde400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efdd80d21b0 con 0x7efdd815dff0 2024-09-16T07:23:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7efdd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 0x7efdd80b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50868/0 (socket says 172.21.15.71:50868) 2024-09-16T07:23:02.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632c0b34b0 0x7f632c201b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632c0b3e50 0x7f632c0ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7efdd7400640 1 -- 172.21.15.71:0/3796168061 learned_addr learned my addr 172.21.15.71:0/3796168061 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:02.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f43dd800640 1 Processor -- start 2024-09-16T07:23:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632c161440 0x7f632c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f632c1619c0 con 0x7f632c161440 2024-09-16T07:23:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f632c0b2990 con 0x7f632c0b3e50 2024-09-16T07:23:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f632c0d2470 con 0x7f632c0b34b0 2024-09-16T07:23:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632c161440 0x7f632c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7efdd7e00640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd815dff0 0x7efdd80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7f43dd800640 1 -- start start 2024-09-16T07:23:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632c161440 0x7f632c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50884/0 (socket says 172.21.15.71:50884) 2024-09-16T07:23:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.393+0000 7f6332200640 1 -- 172.21.15.71:0/1100237433 learned_addr learned my addr 172.21.15.71:0/1100237433 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7f6331800640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632c0b34b0 0x7f632c201b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7f6330e00640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632c0b3e50 0x7f632c0ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7f6332200640 1 -- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632c0b34b0 msgr2=0x7f632c201b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7f6332200640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632c0b34b0 0x7f632c201b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7f6332200640 1 -- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632c0b3e50 msgr2=0x7f632c0ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7f6332200640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632c0b3e50 0x7f632c0ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.394+0000 7f6332200640 1 -- 172.21.15.71:0/1100237433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c202050 con 0x7f632c161440 2024-09-16T07:23:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.395+0000 7efdd7400640 1 -- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd815dff0 msgr2=0x7efdd80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.395+0000 7f6332200640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632c161440 0x7f632c0cda80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f632400c410 tx=0x7f632400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.395+0000 7efdd7400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd815dff0 0x7efdd80cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.395+0000 7efdd7400640 1 -- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd80ceda0 msgr2=0x7efdd80cda00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:23:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.395+0000 7efdd7400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd80ceda0 0x7efdd80cda00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.395+0000 7efdd7400640 1 -- 172.21.15.71:0/3796168061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd80b48b0 con 0x7efdd80b0eb0 2024-09-16T07:23:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.396+0000 7efdd7400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 0x7efdd80b4370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efdc800b3e0 tx=0x7efdc800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.396+0000 7f6329600640 1 -- 172.21.15.71:0/1100237433 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324011790 con 0x7f632c161440 2024-09-16T07:23:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.396+0000 7f6329600640 1 -- 172.21.15.71:0/1100237433 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6324017070 con 0x7f632c161440 2024-09-16T07:23:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.396+0000 7efdcf400640 1 -- 172.21.15.71:0/3796168061 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8015020 con 0x7efdd80b0eb0 2024-09-16T07:23:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.396+0000 7efdcf400640 1 -- 172.21.15.71:0/3796168061 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdc8011940 con 0x7efdd80b0eb0 2024-09-16T07:23:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.396+0000 7efdcf400640 1 -- 172.21.15.71:0/3796168061 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8011d90 con 0x7efdd80b0eb0 2024-09-16T07:23:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.396+0000 7f6329600640 1 -- 172.21.15.71:0/1100237433 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324011cc0 con 0x7f632c161440 2024-09-16T07:23:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.397+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c0ca7d0 con 0x7f632c161440 2024-09-16T07:23:02.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.397+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c0c7da0 con 0x7f632c161440 2024-09-16T07:23:02.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.397+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd81f4dd0 con 0x7efdd80b0eb0 2024-09-16T07:23:02.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.397+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd81f5390 con 0x7efdd80b0eb0 2024-09-16T07:23:02.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.398+0000 7f43dd800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d0009e40 0x7f43d00d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.398+0000 7f43dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d000a220 0x7f43d0176c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.398+0000 7f43dd800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43d0036550 0x7f43d00d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.399+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd80d26a0 con 0x7efdd80b0eb0 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.398+0000 7f43dd800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0006c20 con 0x7f43d000a220 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.398+0000 7f43dd800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0006aa0 con 0x7f43d0036550 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.398+0000 7f43dd800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43d001caa0 con 0x7f43d0009e40 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.398+0000 7f43d6a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d0009e40 0x7f43d00d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.399+0000 7f43d6a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d0009e40 0x7f43d00d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:42504/0 (socket says 172.21.15.71:42504) 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.399+0000 7f43d6a00640 1 -- 172.21.15.71:0/712703832 learned_addr learned my addr 172.21.15.71:0/712703832 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.399+0000 7f43d6000640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d000a220 0x7f43d0176c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.400+0000 7f43d7400640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43d0036550 0x7f43d00d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.401+0000 7f43d6a00640 1 -- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43d0036550 msgr2=0x7f43d00d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.401+0000 7f43d6a00640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43d0036550 0x7f43d00d1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.401+0000 7f43d6a00640 1 -- 172.21.15.71:0/712703832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d000a220 msgr2=0x7f43d0176c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.401+0000 7f43d6a00640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d000a220 0x7f43d0176c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.401+0000 7f43d6a00640 1 -- 172.21.15.71:0/712703832 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d00d0be0 con 0x7f43d0009e40 2024-09-16T07:23:02.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.402+0000 7f43d6a00640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d0009e40 0x7f43d00d06a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009910 tx=0x7f43c8005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.402+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632c0d2990 con 0x7f632c161440 2024-09-16T07:23:02.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.403+0000 7f6329600640 1 -- 172.21.15.71:0/1100237433 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f632401c460 con 0x7f632c161440 2024-09-16T07:23:02.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.403+0000 7f43cea00640 1 -- 172.21.15.71:0/712703832 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8040070 con 0x7f43d0009e40 2024-09-16T07:23:02.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.403+0000 7f43cea00640 1 -- 172.21.15.71:0/712703832 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43c80049a0 con 0x7f43d0009e40 2024-09-16T07:23:02.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.403+0000 7f43cea00640 1 -- 172.21.15.71:0/712703832 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8004dd0 con 0x7f43d0009e40 2024-09-16T07:23:02.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.403+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d0177180 con 0x7f43d0009e40 2024-09-16T07:23:02.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.403+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d00d1140 con 0x7f43d0009e40 2024-09-16T07:23:02.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.403+0000 7efdcf400640 1 -- 172.21.15.71:0/3796168061 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7efdc8018020 con 0x7efdd80b0eb0 2024-09-16T07:23:02.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.405+0000 7f6329600640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630c078920 0x7f630c07ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.405+0000 7f6329600640 1 -- 172.21.15.71:0/1100237433 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f632409a710 con 0x7f632c161440 2024-09-16T07:23:02.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.406+0000 7f6331800640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630c078920 0x7f630c07ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.408+0000 7efdcf400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdb8078850 0x7efdb807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.408+0000 7f6331800640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630c078920 0x7f630c07ade0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6320035000 tx=0x7f6320004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.407+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d001cf50 con 0x7f43d0009e40 2024-09-16T07:23:02.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.408+0000 7efdcf400640 1 -- 172.21.15.71:0/3796168061 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efdc809ad50 con 0x7efdd80b0eb0 2024-09-16T07:23:02.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.409+0000 7efdd6a00640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdb8078850 0x7efdb807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.410+0000 7f43cea00640 1 -- 172.21.15.71:0/712703832 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f43c8039040 con 0x7f43d0009e40 2024-09-16T07:23:02.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.410+0000 7efdd6a00640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdb8078850 0x7efdb807ad10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdd0035000 tx=0x7efdd0004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.411+0000 7efdcf400640 1 -- 172.21.15.71:0/3796168061 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efdc801f020 con 0x7efdd80b0eb0 2024-09-16T07:23:02.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.412+0000 7f43cea00640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f43b8078880 0x7f43b807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.412+0000 7f43cea00640 1 -- 172.21.15.71:0/712703832 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f43c80bbee0 con 0x7f43d0009e40 2024-09-16T07:23:02.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.415+0000 7f43d6000640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f43b8078880 0x7f43b807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.415+0000 7f43cea00640 1 -- 172.21.15.71:0/712703832 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f43d00d0be0 con 0x7f43d0009e40 2024-09-16T07:23:02.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.416+0000 7f43d6000640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f43b8078880 0x7f43b807ad40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f43d80527e0 tx=0x7f43d806f040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.417+0000 7f6329600640 1 -- 172.21.15.71:0/1100237433 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f632c202050 con 0x7f632c161440 2024-09-16T07:23:02.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.465+0000 7f7a84a00640 1 -- 172.21.15.71:0/34292230 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74031600 con 0x7f7a80037990 2024-09-16T07:23:02.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.465+0000 7f7a87200640 1 -- 172.21.15.71:0/34292230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a80037990 msgr2=0x7f7a80052ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.465+0000 7f7a87200640 1 --2- 172.21.15.71:0/34292230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a80037990 0x7f7a80052ea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009830 tx=0x7f7a7402a560 comp rx=0 tx=0).stop 2024-09-16T07:23:02.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.466+0000 7f7a87200640 1 -- 172.21.15.71:0/34292230 shutdown_connections 2024-09-16T07:23:02.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.466+0000 7f7a87200640 1 --2- 172.21.15.71:0/34292230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a80037990 0x7f7a80052ea0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.466+0000 7f7a87200640 1 --2- 172.21.15.71:0/34292230 >> v2:172.21.15.177:3300/0 conn(0x7f7a8003a3d0 0x7f7a8003a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.466+0000 7f7a87200640 1 -- 172.21.15.71:0/34292230 >> 172.21.15.71:0/34292230 conn(0x7f7a80055540 msgr2=0x7f7a80055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.466+0000 7f7a87200640 1 -- 172.21.15.71:0/34292230 shutdown_connections 2024-09-16T07:23:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.466+0000 7f7a87200640 1 -- 172.21.15.71:0/34292230 wait complete. 2024-09-16T07:23:02.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.469+0000 7f7a87200640 1 Processor -- start 2024-09-16T07:23:02.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.469+0000 7f7a87200640 1 -- start start 2024-09-16T07:23:02.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.470+0000 7f2a53e00640 1 -- 172.21.15.71:0/2892191522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a541613a0 msgr2=0x7f2a540d48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.470+0000 7f2a53e00640 1 --2- 172.21.15.71:0/2892191522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a541613a0 0x7f2a540d48e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2a44005fd0 tx=0x7f2a4402a560 comp rx=0 tx=0).stop 2024-09-16T07:23:02.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.470+0000 7f2a53e00640 1 -- 172.21.15.71:0/2892191522 shutdown_connections 2024-09-16T07:23:02.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.470+0000 7f2a53e00640 1 --2- 172.21.15.71:0/2892191522 >> v2:172.21.15.71:3300/0 conn(0x7f2a540b4a50 0x7f2a54161bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.470+0000 7f2a53e00640 1 --2- 172.21.15.71:0/2892191522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a541613a0 0x7f2a540d48e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.470+0000 7f2a53e00640 1 -- 172.21.15.71:0/2892191522 >> 172.21.15.71:0/2892191522 conn(0x7f2a540d3dc0 msgr2=0x7f2a540d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.471+0000 7f7a87200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a80037990 0x7f7a800dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.471+0000 7f7a87200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a8003a3d0 0x7f7a800dece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.471+0000 7f7a87200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a800e3930 0x7f7a8017da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.471+0000 7f7a87200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7a80058610 con 0x7f7a80037990 2024-09-16T07:23:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.471+0000 7f2a53e00640 1 -- 172.21.15.71:0/2892191522 shutdown_connections 2024-09-16T07:23:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7f7a87200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a80058490 con 0x7f7a800e3930 2024-09-16T07:23:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7f7a87200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7a800e2c20 con 0x7f7a8003a3d0 2024-09-16T07:23:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7f7a85e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a80037990 0x7f7a800dcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3035720776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a015e0d0 msgr2=0x7fb7a00d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7f7a85400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a8003a3d0 0x7f7a800dece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7fb7a4e00640 1 --2- 172.21.15.71:0/3035720776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a015e0d0 0x7fb7a00d09c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb7900097e0 tx=0x7fb79002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7f7a85400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a8003a3d0 0x7f7a800dece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:42526/0 (socket says 172.21.15.71:42526) 2024-09-16T07:23:02.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7f7a85400640 1 -- 172.21.15.71:0/2449987359 learned_addr learned my addr 172.21.15.71:0/2449987359 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:02.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3035720776 shutdown_connections 2024-09-16T07:23:02.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7f7a86800640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a800e3930 0x7f7a8017da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7fb7a4e00640 1 --2- 172.21.15.71:0/3035720776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a015e0d0 0x7fb7a00d09c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3035720776 >> 172.21.15.71:0/3035720776 conn(0x7fb7a00d3fe0 msgr2=0x7fb7a00d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.473+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3035720776 shutdown_connections 2024-09-16T07:23:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.473+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3035720776 wait complete. 2024-09-16T07:23:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.473+0000 7f7a85400640 1 -- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a800e3930 msgr2=0x7f7a8017da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.473+0000 7f7a85400640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a800e3930 0x7f7a8017da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.473+0000 7f7a85400640 1 -- 172.21.15.71:0/2449987359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a80037990 msgr2=0x7f7a800dcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.473+0000 7f7a85400640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a80037990 0x7f7a800dcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.472+0000 7f2a53e00640 1 -- 172.21.15.71:0/2892191522 wait complete. 2024-09-16T07:23:02.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.474+0000 7f2a53e00640 1 Processor -- start 2024-09-16T07:23:02.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.475+0000 7f2a53e00640 1 -- start start 2024-09-16T07:23:02.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.475+0000 7fb7a4e00640 1 Processor -- start 2024-09-16T07:23:02.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.475+0000 7fb7a4e00640 1 -- start start 2024-09-16T07:23:02.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7fb7a4e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a00d1610 0x7fb7a01f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7fb7a4e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a00d2050 0x7fb7a0153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7fb7a4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7a015e0d0 0x7fb7a01540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7fb7a4e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a00cf8a0 con 0x7fb7a015e0d0 2024-09-16T07:23:02.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7fb7a4e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a00cf720 con 0x7fb7a00d2050 2024-09-16T07:23:02.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7fb7a4e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a00cfa20 con 0x7fb7a00d1610 2024-09-16T07:23:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.473+0000 7f7a85400640 1 -- 172.21.15.71:0/2449987359 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8017e590 con 0x7f7a8003a3d0 2024-09-16T07:23:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.474+0000 7f7a86800640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a800e3930 0x7f7a8017da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.474+0000 7f7a85400640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a8003a3d0 0x7f7a800dece0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c480 tx=0x7f7a7800c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7f7a7e000640 1 -- 172.21.15.71:0/2449987359 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a780052d0 con 0x7f7a8003a3d0 2024-09-16T07:23:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7f7a7e000640 1 -- 172.21.15.71:0/2449987359 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a78005470 con 0x7f7a8003a3d0 2024-09-16T07:23:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7f7a7e000640 1 -- 172.21.15.71:0/2449987359 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78012870 con 0x7f7a8003a3d0 2024-09-16T07:23:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.476+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8017e870 con 0x7f7a8003a3d0 2024-09-16T07:23:02.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.477+0000 7fb79d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a00d2050 0x7fb7a0153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.477+0000 7fb79de00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a00d1610 0x7fb7a01f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.477+0000 7fb79de00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a00d1610 0x7fb7a01f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:42536/0 (socket says 172.21.15.71:42536) 2024-09-16T07:23:02.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.477+0000 7fb79de00640 1 -- 172.21.15.71:0/3272350545 learned_addr learned my addr 172.21.15.71:0/3272350545 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:02.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a540b4a50 0x7f2a541f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a541613a0 0x7f2a542053a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a54162100 0x7f2a540b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7fb79e800640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7a015e0d0 0x7fb7a01540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7fb79de00640 1 -- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a00d2050 msgr2=0x7fb7a0153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7fb79de00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a00d2050 0x7fb7a0153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7fb79de00640 1 -- 172.21.15.71:0/3272350545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7a015e0d0 msgr2=0x7fb7a01540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7fb79de00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7a015e0d0 0x7fb7a01540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.478+0000 7fb79de00640 1 -- 172.21.15.71:0/3272350545 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a00b3620 con 0x7fb7a00d1610 2024-09-16T07:23:02.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a53e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54161900 con 0x7f2a540b4a50 2024-09-16T07:23:02.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a53e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54161780 con 0x7f2a541613a0 2024-09-16T07:23:02.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a53e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a540b5f50 con 0x7f2a54162100 2024-09-16T07:23:02.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a541613a0 0x7f2a542053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7fb79de00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a00d1610 0x7fb7a01f59e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb79002aac0 tx=0x7fb790003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a541613a0 0x7f2a542053a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55794/0 (socket says 172.21.15.71:55794) 2024-09-16T07:23:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52000640 1 -- 172.21.15.71:0/3358100854 learned_addr learned my addr 172.21.15.71:0/3358100854 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a53400640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a54162100 0x7f2a540b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52a00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a540b4a50 0x7f2a541f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7fb796000640 1 -- 172.21.15.71:0/3272350545 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7900317b0 con 0x7fb7a00d1610 2024-09-16T07:23:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.480+0000 7fb796000640 1 -- 172.21.15.71:0/3272350545 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb790031a40 con 0x7fb7a00d1610 2024-09-16T07:23:02.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.480+0000 7fb796000640 1 -- 172.21.15.71:0/3272350545 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790031e50 con 0x7fb7a00d1610 2024-09-16T07:23:02.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.480+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a801812b0 con 0x7f7a8003a3d0 2024-09-16T07:23:02.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52000640 1 -- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a54162100 msgr2=0x7f2a540b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52000640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a54162100 0x7f2a540b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52000640 1 -- 172.21.15.71:0/3358100854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a540b4a50 msgr2=0x7f2a541f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52000640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a540b4a50 0x7f2a541f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.479+0000 7f2a52000640 1 -- 172.21.15.71:0/3358100854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a54201870 con 0x7f2a541613a0 2024-09-16T07:23:02.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.480+0000 7f2a52000640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a541613a0 0x7f2a542053a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000c9a0 tx=0x7f2a4000ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.480+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a00b4660 con 0x7fb7a00d1610 2024-09-16T07:23:02.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.480+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a00b3b80 con 0x7fb7a00d1610 2024-09-16T07:23:02.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.481+0000 7f2a4aa00640 1 -- 172.21.15.71:0/3358100854 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40007c10 con 0x7f2a541613a0 2024-09-16T07:23:02.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.482+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a800dd330 con 0x7f7a8003a3d0 2024-09-16T07:23:02.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.482+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a00d2680 con 0x7fb7a00d1610 2024-09-16T07:23:02.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.483+0000 7f7a7e000640 1 -- 172.21.15.71:0/2449987359 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7a78012a10 con 0x7f7a8003a3d0 2024-09-16T07:23:02.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.482+0000 7f2a53e00640 1 -- 172.21.15.71:0/3358100854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a54201b50 con 0x7f2a541613a0 2024-09-16T07:23:02.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.482+0000 7f2a53e00640 1 -- 172.21.15.71:0/3358100854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a54202110 con 0x7f2a541613a0 2024-09-16T07:23:02.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.483+0000 7f2a4aa00640 1 -- 172.21.15.71:0/3358100854 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a40018480 con 0x7f2a541613a0 2024-09-16T07:23:02.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.483+0000 7f2a3fe00640 1 -- 172.21.15.71:0/3358100854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2c005290 con 0x7f2a541613a0 2024-09-16T07:23:02.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.484+0000 7f2a4aa00640 1 -- 172.21.15.71:0/3358100854 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400188d0 con 0x7f2a541613a0 2024-09-16T07:23:02.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.484+0000 7fb796000640 1 -- 172.21.15.71:0/3272350545 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb79003b410 con 0x7fb7a00d1610 2024-09-16T07:23:02.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.485+0000 7f7a7e000640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a68078850 0x7f7a6807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.485+0000 7f7a7e000640 1 -- 172.21.15.71:0/2449987359 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7a7809bd10 con 0x7f7a8003a3d0 2024-09-16T07:23:02.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.486+0000 7f7a85e00640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a68078850 0x7f7a6807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.486+0000 7fb796000640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb780078ac0 0x7fb78007af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.487+0000 7f7a85e00640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a68078850 0x7f7a6807ad10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7a88052150 tx=0x7f7a8808d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.486+0000 7fb796000640 1 -- 172.21.15.71:0/3272350545 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb7900bbb60 con 0x7fb7a00d1610 2024-09-16T07:23:02.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.487+0000 7fb79d400640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb780078ac0 0x7fb78007af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.487+0000 7fb79d400640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb780078ac0 0x7fb78007af80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb78c001560 tx=0x7fb78c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.488+0000 7f2a4aa00640 1 -- 172.21.15.71:0/3358100854 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2a40018a70 con 0x7f2a541613a0 2024-09-16T07:23:02.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.489+0000 7f7a7e000640 1 -- 172.21.15.71:0/2449987359 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7a78061b70 con 0x7f7a8003a3d0 2024-09-16T07:23:02.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.490+0000 7f2a4aa00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a34078880 0x7f2a3407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.491+0000 7f2a4aa00640 1 -- 172.21.15.71:0/3358100854 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2a4009b6f0 con 0x7f2a541613a0 2024-09-16T07:23:02.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.491+0000 7f2a52a00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a34078880 0x7f2a3407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.491+0000 7f2a4aa00640 1 -- 172.21.15.71:0/3358100854 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2a40026080 con 0x7f2a541613a0 2024-09-16T07:23:02.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.492+0000 7f2a52a00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a34078880 0x7f2a3407ad40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a44033000 tx=0x7f2a44004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.494+0000 7fb796000640 1 -- 172.21.15.71:0/3272350545 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb7a00d2680 con 0x7fb7a00d1610 2024-09-16T07:23:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.533+0000 7feb98a00640 1 -- 172.21.15.71:0/2673641063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb941529c0 msgr2=0x7feb940b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.533+0000 7feb98a00640 1 --2- 172.21.15.71:0/2673641063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb941529c0 0x7feb940b4850 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feb7c005fd0 tx=0x7feb7c02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.533+0000 7feb98a00640 1 -- 172.21.15.71:0/2673641063 shutdown_connections 2024-09-16T07:23:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.533+0000 7feb98a00640 1 --2- 172.21.15.71:0/2673641063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9415fce0 0x7feb9406f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.533+0000 7feb98a00640 1 --2- 172.21.15.71:0/2673641063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb941529c0 0x7feb940b4850 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.533+0000 7feb98a00640 1 -- 172.21.15.71:0/2673641063 >> 172.21.15.71:0/2673641063 conn(0x7feb9406cb70 msgr2=0x7feb9406cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.533+0000 7feb98a00640 1 -- 172.21.15.71:0/2673641063 shutdown_connections 2024-09-16T07:23:02.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.534+0000 7feb98a00640 1 -- 172.21.15.71:0/2673641063 wait complete. 2024-09-16T07:23:02.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.536+0000 7feb98a00640 1 Processor -- start 2024-09-16T07:23:02.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.536+0000 7feb98a00640 1 -- start start 2024-09-16T07:23:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.538+0000 7feb98a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb940b38c0 0x7feb9414b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.538+0000 7feb98a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb941529c0 0x7feb940b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.538+0000 7feb98a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb9415fce0 0x7feb94149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.538+0000 7feb98a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb9415d670 con 0x7feb9415fce0 2024-09-16T07:23:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.538+0000 7feb98a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feb9415d4f0 con 0x7feb940b38c0 2024-09-16T07:23:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.538+0000 7feb98a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feb9415d7f0 con 0x7feb941529c0 2024-09-16T07:23:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.539+0000 7feb91400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb941529c0 0x7feb940b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.539+0000 7feb91e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb940b38c0 0x7feb9414b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.539+0000 7feb92800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb9415fce0 0x7feb94149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.539+0000 7feb91e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb940b38c0 0x7feb9414b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:55820/0 (socket says 172.21.15.71:55820) 2024-09-16T07:23:02.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.539+0000 7feb91e00640 1 -- 172.21.15.71:0/2713370549 learned_addr learned my addr 172.21.15.71:0/2713370549 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:02.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.540+0000 7feb92800640 1 -- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb941529c0 msgr2=0x7feb940b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.540+0000 7feb92800640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb941529c0 0x7feb940b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.540+0000 7feb92800640 1 -- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb940b38c0 msgr2=0x7feb9414b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.540+0000 7feb92800640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb940b38c0 0x7feb9414b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.540+0000 7feb92800640 1 -- 172.21.15.71:0/2713370549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9414b550 con 0x7feb9415fce0 2024-09-16T07:23:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.540+0000 7feb92800640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb9415fce0 0x7feb94149830 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e9b0 tx=0x7feb8c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.541+0000 7feb8a000640 1 -- 172.21.15.71:0/2713370549 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00cd80 con 0x7feb9415fce0 2024-09-16T07:23:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.541+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb94155bc0 con 0x7feb9415fce0 2024-09-16T07:23:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.541+0000 7feb8a000640 1 -- 172.21.15.71:0/2713370549 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb8c004b20 con 0x7feb9415fce0 2024-09-16T07:23:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.541+0000 7feb8a000640 1 -- 172.21.15.71:0/2713370549 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c005020 con 0x7feb9415fce0 2024-09-16T07:23:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.541+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9414ad60 con 0x7feb9415fce0 2024-09-16T07:23:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.543+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9415dca0 con 0x7feb9415fce0 2024-09-16T07:23:02.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.556+0000 7feb8a000640 1 -- 172.21.15.71:0/2713370549 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7feb8c0051c0 con 0x7feb9415fce0 2024-09-16T07:23:02.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.559+0000 7feb8a000640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb70078850 0x7feb7007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.560+0000 7feb8a000640 1 -- 172.21.15.71:0/2713370549 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7feb8c09bc70 con 0x7feb9415fce0 2024-09-16T07:23:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.560+0000 7feb8a000640 1 -- 172.21.15.71:0/2713370549 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feb8c026020 con 0x7feb9415fce0 2024-09-16T07:23:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.560+0000 7feb91e00640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb70078850 0x7feb7007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.565+0000 7feb91e00640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb70078850 0x7feb7007ad10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006010 tx=0x7feb7c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:02.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.755+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8b140b4ed0 con 0x7f8b140d1fd0 2024-09-16T07:23:02.759 INFO:teuthology.orchestra.run.smithi071.stdout:68719476745 2024-09-16T07:23:02.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.756+0000 7f8b11600640 1 -- 172.21.15.71:0/102479307 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8b00014030 con 0x7f8b140d1fd0 2024-09-16T07:23:02.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8af4078850 msgr2=0x7f8af407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8af4078850 0x7f8af407ad10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0012e0 tx=0x7f8b0c009040 comp rx=0 tx=0).stop 2024-09-16T07:23:02.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b140d1fd0 msgr2=0x7f8b140b74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b140d1fd0 0x7f8b140b74c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c970 tx=0x7f8b0000ce30 comp rx=0 tx=0).stop 2024-09-16T07:23:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 shutdown_connections 2024-09-16T07:23:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8af4078850 0x7f8af407ad10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b140d1fd0 0x7f8b140b74c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b140b5410 0x7f8b140c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 --2- 172.21.15.71:0/102479307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b140b4860 0x7f8b14154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.760+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 >> 172.21.15.71:0/102479307 conn(0x7f8b140d3df0 msgr2=0x7f8b140d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.761+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 shutdown_connections 2024-09-16T07:23:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.761+0000 7f8b1b000640 1 -- 172.21.15.71:0/102479307 wait complete. 2024-09-16T07:23:02.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476745 for osd.3 2024-09-16T07:23:02.788 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:02.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.815+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f43d00e3ca0 con 0x7f43d0009e40 2024-09-16T07:23:02.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.816+0000 7f43cea00640 1 -- 172.21.15.71:0/712703832 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f43c80845c0 con 0x7f43d0009e40 2024-09-16T07:23:02.817 INFO:teuthology.orchestra.run.smithi071.stdout:68719476744 2024-09-16T07:23:02.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f43b8078880 msgr2=0x7f43b807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f43b8078880 0x7f43b807ad40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f43d80527e0 tx=0x7f43d806f040 comp rx=0 tx=0).stop 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d0009e40 msgr2=0x7f43d00d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d0009e40 0x7f43d00d06a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009910 tx=0x7f43c8005250 comp rx=0 tx=0).stop 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 shutdown_connections 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f43b8078880 0x7f43b807ad40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43d0036550 0x7f43d00d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d000a220 0x7f43d0176c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 --2- 172.21.15.71:0/712703832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d0009e40 0x7f43d00d06a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 >> 172.21.15.71:0/712703832 conn(0x7f43d0054590 msgr2=0x7f43d00e20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 shutdown_connections 2024-09-16T07:23:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.819+0000 7f43dd800640 1 -- 172.21.15.71:0/712703832 wait complete. 2024-09-16T07:23:02.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476744 got 68719476744 for osd.0 2024-09-16T07:23:02.837 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.852+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f632c0d45e0 con 0x7f632c161440 2024-09-16T07:23:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.852+0000 7f6329600640 1 -- 172.21.15.71:0/1100237433 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6324062df0 con 0x7f632c161440 2024-09-16T07:23:02.853 INFO:teuthology.orchestra.run.smithi071.stdout:68719476745 2024-09-16T07:23:02.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.855+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efdd80cdf40 con 0x7efdd80b0eb0 2024-09-16T07:23:02.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.855+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630c078920 msgr2=0x7f630c07ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.855+0000 7f6332c00640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630c078920 0x7f630c07ade0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6320035000 tx=0x7f6320004ab0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.855+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632c161440 msgr2=0x7f632c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.855+0000 7f6332c00640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632c161440 0x7f632c0cda80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f632400c410 tx=0x7f632400c8d0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 shutdown_connections 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7f6332c00640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630c078920 0x7f630c07ade0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7f6332c00640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632c161440 0x7f632c0cda80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7f6332c00640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632c0b3e50 0x7f632c0ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7f6332c00640 1 --2- 172.21.15.71:0/1100237433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632c0b34b0 0x7f632c201b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 >> 172.21.15.71:0/1100237433 conn(0x7f632c0d3df0 msgr2=0x7f632c15eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 shutdown_connections 2024-09-16T07:23:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7f6332c00640 1 -- 172.21.15.71:0/1100237433 wait complete. 2024-09-16T07:23:02.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.856+0000 7efdcf400640 1 -- 172.21.15.71:0/3796168061 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efdc80633a0 con 0x7efdd80b0eb0 2024-09-16T07:23:02.858 INFO:teuthology.orchestra.run.smithi071.stdout:64424509448 2024-09-16T07:23:02.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdb8078850 msgr2=0x7efdb807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdb8078850 0x7efdb807ad10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdd0035000 tx=0x7efdd0004ab0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 msgr2=0x7efdd80b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 0x7efdd80b4370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efdc800b3e0 tx=0x7efdc800b8a0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 shutdown_connections 2024-09-16T07:23:02.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdb8078850 0x7efdb807ad10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd815dff0 0x7efdd80cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd80ceda0 0x7efdd80cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 --2- 172.21.15.71:0/3796168061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd80b0eb0 0x7efdd80b4370 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.859+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 >> 172.21.15.71:0/3796168061 conn(0x7efdd80d3de0 msgr2=0x7efdd80b7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.860+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 shutdown_connections 2024-09-16T07:23:02.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.860+0000 7efdde400640 1 -- 172.21.15.71:0/3796168061 wait complete. 2024-09-16T07:23:02.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-09-16T07:23:02.878 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:02.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476745 for osd.1 2024-09-16T07:23:02.880 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:02.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.887+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7a800dde20 con 0x7f7a8003a3d0 2024-09-16T07:23:02.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.888+0000 7f7a7e000640 1 -- 172.21.15.71:0/2449987359 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7a780665a0 con 0x7f7a8003a3d0 2024-09-16T07:23:02.889 INFO:teuthology.orchestra.run.smithi071.stdout:64424509446 2024-09-16T07:23:02.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a68078850 msgr2=0x7f7a6807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a68078850 0x7f7a6807ad10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7a88052150 tx=0x7f7a8808d040 comp rx=0 tx=0).stop 2024-09-16T07:23:02.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a8003a3d0 msgr2=0x7f7a800dece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a8003a3d0 0x7f7a800dece0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c480 tx=0x7f7a7800c940 comp rx=0 tx=0).stop 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 shutdown_connections 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a68078850 0x7f7a6807ad10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a800e3930 0x7f7a8017da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a8003a3d0 0x7f7a800dece0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 --2- 172.21.15.71:0/2449987359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a80037990 0x7f7a800dcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 >> 172.21.15.71:0/2449987359 conn(0x7f7a80055540 msgr2=0x7f7a8000e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 shutdown_connections 2024-09-16T07:23:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.891+0000 7f7a87200640 1 -- 172.21.15.71:0/2449987359 wait complete. 2024-09-16T07:23:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.899+0000 7f2a3fe00640 1 -- 172.21.15.71:0/3358100854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2a2c005850 con 0x7f2a541613a0 2024-09-16T07:23:02.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.900+0000 7f2a4aa00640 1 -- 172.21.15.71:0/3358100854 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2a40063dd0 con 0x7f2a541613a0 2024-09-16T07:23:02.901 INFO:teuthology.orchestra.run.smithi071.stdout:60129542151 2024-09-16T07:23:02.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 -- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a34078880 msgr2=0x7f2a3407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a34078880 0x7f2a3407ad40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a44033000 tx=0x7f2a44004730 comp rx=0 tx=0).stop 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 -- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a541613a0 msgr2=0x7f2a542053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a541613a0 0x7f2a542053a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000c9a0 tx=0x7f2a4000ce60 comp rx=0 tx=0).stop 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 -- 172.21.15.71:0/3358100854 shutdown_connections 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a34078880 0x7f2a3407ad40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a54162100 0x7f2a540b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a541613a0 0x7f2a542053a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 --2- 172.21.15.71:0/3358100854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a540b4a50 0x7f2a541f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.902+0000 7f2a3fe00640 1 -- 172.21.15.71:0/3358100854 >> 172.21.15.71:0/3358100854 conn(0x7f2a540d3dc0 msgr2=0x7f2a540b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.903+0000 7f2a3fe00640 1 -- 172.21.15.71:0/3358100854 shutdown_connections 2024-09-16T07:23:02.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.903+0000 7f2a3fe00640 1 -- 172.21.15.71:0/3358100854 wait complete. 2024-09-16T07:23:02.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-09-16T07:23:02.907 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:02.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.4 2024-09-16T07:23:02.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.918+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb7a00cff60 con 0x7fb7a00d1610 2024-09-16T07:23:02.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.919+0000 7fb796000640 1 -- 172.21.15.71:0/3272350545 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb7a00b3620 con 0x7fb7a00d1610 2024-09-16T07:23:02.920 INFO:teuthology.orchestra.run.smithi071.stdout:68719476745 2024-09-16T07:23:02.920 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb780078ac0 msgr2=0x7fb78007af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb780078ac0 0x7fb78007af80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb78c001560 tx=0x7fb78c009040 comp rx=0 tx=0).stop 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a00d1610 msgr2=0x7fb7a01f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a00d1610 0x7fb7a01f59e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb79002aac0 tx=0x7fb790003b90 comp rx=0 tx=0).stop 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 shutdown_connections 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb780078ac0 0x7fb78007af80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7a015e0d0 0x7fb7a01540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a00d2050 0x7fb7a0153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 --2- 172.21.15.71:0/3272350545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a00d1610 0x7fb7a01f59e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 >> 172.21.15.71:0/3272350545 conn(0x7fb7a00d3fe0 msgr2=0x7fb7a00cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 shutdown_connections 2024-09-16T07:23:02.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.921+0000 7fb7a4e00640 1 -- 172.21.15.71:0/3272350545 wait complete. 2024-09-16T07:23:02.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476745 for osd.2 2024-09-16T07:23:02.938 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:02.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.952+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7feb94073cb0 con 0x7feb9415fce0 2024-09-16T07:23:02.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.953+0000 7feb8a000640 1 -- 172.21.15.71:0/2713370549 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb9414b550 con 0x7feb9415fce0 2024-09-16T07:23:02.953 INFO:teuthology.orchestra.run.smithi071.stdout:64424509448 2024-09-16T07:23:02.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb70078850 msgr2=0x7feb7007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb70078850 0x7feb7007ad10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006010 tx=0x7feb7c004730 comp rx=0 tx=0).stop 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb9415fce0 msgr2=0x7feb94149830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb9415fce0 0x7feb94149830 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e9b0 tx=0x7feb8c00ee70 comp rx=0 tx=0).stop 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 shutdown_connections 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb70078850 0x7feb7007ad10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb9415fce0 0x7feb94149830 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb941529c0 0x7feb940b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 --2- 172.21.15.71:0/2713370549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb940b38c0 0x7feb9414b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 >> 172.21.15.71:0/2713370549 conn(0x7feb9406cb70 msgr2=0x7feb9415ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 shutdown_connections 2024-09-16T07:23:02.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:02.955+0000 7feb98a00640 1 -- 172.21.15.71:0/2713370549 wait complete. 2024-09-16T07:23:02.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.5 2024-09-16T07:23:02.971 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:02.971 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:23:02.971 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:23:03.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.168+0000 7f4633000640 1 -- 172.21.15.71:0/1408016638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4610004000 msgr2=0x7f4610004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.168+0000 7f4633000640 1 --2- 172.21.15.71:0/1408016638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4610004000 0x7f4610004800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4618006fd0 tx=0x7f4610005830 comp rx=0 tx=0).stop 2024-09-16T07:23:03.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.168+0000 7f4631000640 1 -- 172.21.15.71:0/1408016638 reap_dead start 2024-09-16T07:23:03.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.168+0000 7f4633000640 1 -- 172.21.15.71:0/1408016638 shutdown_connections 2024-09-16T07:23:03.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.168+0000 7f4633000640 1 -- 172.21.15.71:0/1408016638 >> 172.21.15.71:0/1408016638 conn(0x7f462c0b1950 msgr2=0x7f462c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:03.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.168+0000 7f4633000640 1 -- 172.21.15.71:0/1408016638 shutdown_connections 2024-09-16T07:23:03.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.168+0000 7f4633000640 1 -- 172.21.15.71:0/1408016638 wait complete. 2024-09-16T07:23:03.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.170+0000 7f4633000640 1 Processor -- start 2024-09-16T07:23:03.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.170+0000 7f4633000640 1 -- start start 2024-09-16T07:23:03.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.170+0000 7f4633000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4610004000 0x7f462c15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:03.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4633000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f462c13df80 0x7f462c1f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:03.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4633000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462c0ae540 0x7f462c0ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:03.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4633000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0730b0 con 0x7f462c0ae540 2024-09-16T07:23:03.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4633000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0b4eb0 con 0x7f462c13df80 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4633000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f462c073230 con 0x7f4610004000 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462c0ae540 0x7f462c0ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462c0ae540 0x7f462c0ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50970/0 (socket says 172.21.15.71:50970) 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 -- 172.21.15.71:0/1886189719 learned_addr learned my addr 172.21.15.71:0/1886189719 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f462be00640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4610004000 0x7f462c15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f462b400640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f462c13df80 0x7f462c1f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 -- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4610004000 msgr2=0x7f462c15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4610004000 0x7f462c15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 -- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f462c13df80 msgr2=0x7f462c1f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f462c13df80 0x7f462c1f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 -- 172.21.15.71:0/1886189719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c1f6840 con 0x7f462c0ae540 2024-09-16T07:23:03.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f462b400640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f462c13df80 0x7f462c1f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:03.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.171+0000 7f4631000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462c0ae540 0x7f462c0ae9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4624005fd0 tx=0x7f462400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:03.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.172+0000 7f4623e00640 1 -- 172.21.15.71:0/1886189719 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624004b20 con 0x7f462c0ae540 2024-09-16T07:23:03.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.172+0000 7f4623e00640 1 -- 172.21.15.71:0/1886189719 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4624004cc0 con 0x7f462c0ae540 2024-09-16T07:23:03.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.172+0000 7f4623e00640 1 -- 172.21.15.71:0/1886189719 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624003bd0 con 0x7f462c0ae540 2024-09-16T07:23:03.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.172+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1f6b20 con 0x7f462c0ae540 2024-09-16T07:23:03.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.172+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c15acc0 con 0x7f462c0ae540 2024-09-16T07:23:03.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.173+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c0737c0 con 0x7f462c0ae540 2024-09-16T07:23:03.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.174+0000 7f4623e00640 1 -- 172.21.15.71:0/1886189719 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4624004030 con 0x7f462c0ae540 2024-09-16T07:23:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.175+0000 7f4623e00640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f460c078850 0x7f460c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.175+0000 7f4623e00640 1 -- 172.21.15.71:0/1886189719 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f462409b220 con 0x7f462c0ae540 2024-09-16T07:23:03.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.175+0000 7f462be00640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f460c078850 0x7f460c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:03.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.176+0000 7f462be00640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f460c078850 0x7f460c07ad10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c006d60 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:03.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.179+0000 7f4623e00640 1 -- 172.21.15.71:0/1886189719 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4624063880 con 0x7f462c0ae540 2024-09-16T07:23:03.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.360+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f462c070a20 con 0x7f460c078850 2024-09-16T07:23:03.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.365+0000 7f4623e00640 1 -- 172.21.15.71:0/1886189719 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f462c070a20 con 0x7f460c078850 2024-09-16T07:23:03.369 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:03.370 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:23:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f460c078850 msgr2=0x7f460c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f460c078850 0x7f460c07ad10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c006d60 comp rx=0 tx=0).stop 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462c0ae540 msgr2=0x7f462c0ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462c0ae540 0x7f462c0ae9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4624005fd0 tx=0x7f462400f3c0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 shutdown_connections 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f460c078850 0x7f460c07ad10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462c0ae540 0x7f462c0ae9f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f462c13df80 0x7f462c1f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 --2- 172.21.15.71:0/1886189719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4610004000 0x7f462c15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 >> 172.21.15.71:0/1886189719 conn(0x7f462c0b1950 msgr2=0x7f462c0d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 shutdown_connections 2024-09-16T07:23:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.370+0000 7f4633000640 1 -- 172.21.15.71:0/1886189719 wait complete. 2024-09-16T07:23:03.387 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-16T07:23:02.296635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223448,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751272,"statfs":{"total":773094113280,"available":772865302528,"internally_reserved":0,"allocated":3088384,"data_stored":1831212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002944"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139414+0000","last_change":"2024-09-16T07:22:49.144943+0000","last_active":"2024-09-16T07:22:50.139414+0000","last_peered":"2024-09-16T07:22:50.139414+0000","last_clean":"2024-09-16T07:22:50.139414+0000","last_became_active":"2024-09-16T07:22:49.144609+0000","last_became_peered":"2024-09-16T07:22:49.144609+0000","last_unstale":"2024-09-16T07:22:50.139414+0000","last_undegraded":"2024-09-16T07:22:50.139414+0000","last_fullsized":"2024-09-16T07:22:50.139414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:21.053710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132260+0000","last_change":"2024-09-16T07:22:49.142860+0000","last_active":"2024-09-16T07:22:50.132260+0000","last_peered":"2024-09-16T07:22:50.132260+0000","last_clean":"2024-09-16T07:22:50.132260+0000","last_became_active":"2024-09-16T07:22:49.142555+0000","last_became_peered":"2024-09-16T07:22:49.142555+0000","last_unstale":"2024-09-16T07:22:50.132260+0000","last_undegraded":"2024-09-16T07:22:50.132260+0000","last_fullsized":"2024-09-16T07:22:50.132260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:20.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130865+0000","last_change":"2024-09-16T07:22:49.141351+0000","last_active":"2024-09-16T07:22:50.130865+0000","last_peered":"2024-09-16T07:22:50.130865+0000","last_clean":"2024-09-16T07:22:50.130865+0000","last_became_active":"2024-09-16T07:22:49.141078+0000","last_became_peered":"2024-09-16T07:22:49.141078+0000","last_unstale":"2024-09-16T07:22:50.130865+0000","last_undegraded":"2024-09-16T07:22:50.130865+0000","last_fullsized":"2024-09-16T07:22:50.130865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:54.707093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132009+0000","last_change":"2024-09-16T07:22:49.130699+0000","last_active":"2024-09-16T07:22:50.132009+0000","last_peered":"2024-09-16T07:22:50.132009+0000","last_clean":"2024-09-16T07:22:50.132009+0000","last_became_active":"2024-09-16T07:22:49.130435+0000","last_became_peered":"2024-09-16T07:22:49.130435+0000","last_unstale":"2024-09-16T07:22:50.132009+0000","last_undegraded":"2024-09-16T07:22:50.132009+0000","last_fullsized":"2024-09-16T07:22:50.132009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:37.684357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139429+0000","last_change":"2024-09-16T07:22:49.139132+0000","last_active":"2024-09-16T07:22:50.139429+0000","last_peered":"2024-09-16T07:22:50.139429+0000","last_clean":"2024-09-16T07:22:50.139429+0000","last_became_active":"2024-09-16T07:22:49.134577+0000","last_became_peered":"2024-09-16T07:22:49.134577+0000","last_unstale":"2024-09-16T07:22:50.139429+0000","last_undegraded":"2024-09-16T07:22:50.139429+0000","last_fullsized":"2024-09-16T07:22:50.139429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:28.720254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131882+0000","last_change":"2024-09-16T07:22:49.130577+0000","last_active":"2024-09-16T07:22:50.131882+0000","last_peered":"2024-09-16T07:22:50.131882+0000","last_clean":"2024-09-16T07:22:50.131882+0000","last_became_active":"2024-09-16T07:22:49.130338+0000","last_became_peered":"2024-09-16T07:22:49.130338+0000","last_unstale":"2024-09-16T07:22:50.131882+0000","last_undegraded":"2024-09-16T07:22:50.131882+0000","last_fullsized":"2024-09-16T07:22:50.131882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139192+0000","last_change":"2024-09-16T07:22:49.142202+0000","last_active":"2024-09-16T07:22:50.139192+0000","last_peered":"2024-09-16T07:22:50.139192+0000","last_clean":"2024-09-16T07:22:50.139192+0000","last_became_active":"2024-09-16T07:22:49.141956+0000","last_became_peered":"2024-09-16T07:22:49.141956+0000","last_unstale":"2024-09-16T07:22:50.139192+0000","last_undegraded":"2024-09-16T07:22:50.139192+0000","last_fullsized":"2024-09-16T07:22:50.139192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:35.827940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130543+0000","last_change":"2024-09-16T07:22:49.130201+0000","last_active":"2024-09-16T07:22:50.130543+0000","last_peered":"2024-09-16T07:22:50.130543+0000","last_clean":"2024-09-16T07:22:50.130543+0000","last_became_active":"2024-09-16T07:22:49.129912+0000","last_became_peered":"2024-09-16T07:22:49.129912+0000","last_unstale":"2024-09-16T07:22:50.130543+0000","last_undegraded":"2024-09-16T07:22:50.130543+0000","last_fullsized":"2024-09-16T07:22:50.130543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:21.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279771+0000","last_change":"2024-09-16T07:22:49.141365+0000","last_active":"2024-09-16T07:22:50.279771+0000","last_peered":"2024-09-16T07:22:50.279771+0000","last_clean":"2024-09-16T07:22:50.279771+0000","last_became_active":"2024-09-16T07:22:49.141041+0000","last_became_peered":"2024-09-16T07:22:49.141041+0000","last_unstale":"2024-09-16T07:22:50.279771+0000","last_undegraded":"2024-09-16T07:22:50.279771+0000","last_fullsized":"2024-09-16T07:22:50.279771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:37.684357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139441+0000","last_change":"2024-09-16T07:22:49.142086+0000","last_active":"2024-09-16T07:22:50.139441+0000","last_peered":"2024-09-16T07:22:50.139441+0000","last_clean":"2024-09-16T07:22:50.139441+0000","last_became_active":"2024-09-16T07:22:49.141832+0000","last_became_peered":"2024-09-16T07:22:49.141832+0000","last_unstale":"2024-09-16T07:22:50.139441+0000","last_undegraded":"2024-09-16T07:22:50.139441+0000","last_fullsized":"2024-09-16T07:22:50.139441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:05.967417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139585+0000","last_change":"2024-09-16T07:22:49.144692+0000","last_active":"2024-09-16T07:22:50.139585+0000","last_peered":"2024-09-16T07:22:50.139585+0000","last_clean":"2024-09-16T07:22:50.139585+0000","last_became_active":"2024-09-16T07:22:49.144396+0000","last_became_peered":"2024-09-16T07:22:49.144396+0000","last_unstale":"2024-09-16T07:22:50.139585+0000","last_undegraded":"2024-09-16T07:22:50.139585+0000","last_fullsized":"2024-09-16T07:22:50.139585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:47.741254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139742+0000","last_change":"2024-09-16T07:22:49.130387+0000","last_active":"2024-09-16T07:22:50.139742+0000","last_peered":"2024-09-16T07:22:50.139742+0000","last_clean":"2024-09-16T07:22:50.139742+0000","last_became_active":"2024-09-16T07:22:49.129994+0000","last_became_peered":"2024-09-16T07:22:49.129994+0000","last_unstale":"2024-09-16T07:22:50.139742+0000","last_undegraded":"2024-09-16T07:22:50.139742+0000","last_fullsized":"2024-09-16T07:22:50.139742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:11.870228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279379+0000","last_change":"2024-09-16T07:22:49.133371+0000","last_active":"2024-09-16T07:22:50.279379+0000","last_peered":"2024-09-16T07:22:50.279379+0000","last_clean":"2024-09-16T07:22:50.279379+0000","last_became_active":"2024-09-16T07:22:49.133088+0000","last_became_peered":"2024-09-16T07:22:49.133088+0000","last_unstale":"2024-09-16T07:22:50.279379+0000","last_undegraded":"2024-09-16T07:22:50.279379+0000","last_fullsized":"2024-09-16T07:22:50.279379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:17.196275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279291+0000","last_change":"2024-09-16T07:22:49.140290+0000","last_active":"2024-09-16T07:22:50.279291+0000","last_peered":"2024-09-16T07:22:50.279291+0000","last_clean":"2024-09-16T07:22:50.279291+0000","last_became_active":"2024-09-16T07:22:49.139999+0000","last_became_peered":"2024-09-16T07:22:49.139999+0000","last_unstale":"2024-09-16T07:22:50.279291+0000","last_undegraded":"2024-09-16T07:22:50.279291+0000","last_fullsized":"2024-09-16T07:22:50.279291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131078+0000","last_change":"2024-09-16T07:22:49.131493+0000","last_active":"2024-09-16T07:22:50.131078+0000","last_peered":"2024-09-16T07:22:50.131078+0000","last_clean":"2024-09-16T07:22:50.131078+0000","last_became_active":"2024-09-16T07:22:49.131241+0000","last_became_peered":"2024-09-16T07:22:49.131241+0000","last_unstale":"2024-09-16T07:22:50.131078+0000","last_undegraded":"2024-09-16T07:22:50.131078+0000","last_fullsized":"2024-09-16T07:22:50.131078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:24.470314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278778+0000","last_change":"2024-09-16T07:22:49.126099+0000","last_active":"2024-09-16T07:22:50.278778+0000","last_peered":"2024-09-16T07:22:50.278778+0000","last_clean":"2024-09-16T07:22:50.278778+0000","last_became_active":"2024-09-16T07:22:49.125826+0000","last_became_peered":"2024-09-16T07:22:49.125826+0000","last_unstale":"2024-09-16T07:22:50.278778+0000","last_undegraded":"2024-09-16T07:22:50.278778+0000","last_fullsized":"2024-09-16T07:22:50.278778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:41.565481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280232+0000","last_change":"2024-09-16T07:22:49.132617+0000","last_active":"2024-09-16T07:22:50.280232+0000","last_peered":"2024-09-16T07:22:50.280232+0000","last_clean":"2024-09-16T07:22:50.280232+0000","last_became_active":"2024-09-16T07:22:49.132278+0000","last_became_peered":"2024-09-16T07:22:49.132278+0000","last_unstale":"2024-09-16T07:22:50.280232+0000","last_undegraded":"2024-09-16T07:22:50.280232+0000","last_fullsized":"2024-09-16T07:22:50.280232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:10.129478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280258+0000","last_change":"2024-09-16T07:22:49.143862+0000","last_active":"2024-09-16T07:22:50.280258+0000","last_peered":"2024-09-16T07:22:50.280258+0000","last_clean":"2024-09-16T07:22:50.280258+0000","last_became_active":"2024-09-16T07:22:49.143457+0000","last_became_peered":"2024-09-16T07:22:49.143457+0000","last_unstale":"2024-09-16T07:22:50.280258+0000","last_undegraded":"2024-09-16T07:22:50.280258+0000","last_fullsized":"2024-09-16T07:22:50.280258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279691+0000","last_change":"2024-09-16T07:22:49.130309+0000","last_active":"2024-09-16T07:22:50.279691+0000","last_peered":"2024-09-16T07:22:50.279691+0000","last_clean":"2024-09-16T07:22:50.279691+0000","last_became_active":"2024-09-16T07:22:49.130069+0000","last_became_peered":"2024-09-16T07:22:49.130069+0000","last_unstale":"2024-09-16T07:22:50.279691+0000","last_undegraded":"2024-09-16T07:22:50.279691+0000","last_fullsized":"2024-09-16T07:22:50.279691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:20.784563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139554+0000","last_change":"2024-09-16T07:22:49.129763+0000","last_active":"2024-09-16T07:22:50.139554+0000","last_peered":"2024-09-16T07:22:50.139554+0000","last_clean":"2024-09-16T07:22:50.139554+0000","last_became_active":"2024-09-16T07:22:49.129067+0000","last_became_peered":"2024-09-16T07:22:49.129067+0000","last_unstale":"2024-09-16T07:22:50.139554+0000","last_undegraded":"2024-09-16T07:22:50.139554+0000","last_fullsized":"2024-09-16T07:22:50.139554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:24.919949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132192+0000","last_change":"2024-09-16T07:22:49.133568+0000","last_active":"2024-09-16T07:22:50.132192+0000","last_peered":"2024-09-16T07:22:50.132192+0000","last_clean":"2024-09-16T07:22:50.132192+0000","last_became_active":"2024-09-16T07:22:49.133323+0000","last_became_peered":"2024-09-16T07:22:49.133323+0000","last_unstale":"2024-09-16T07:22:50.132192+0000","last_undegraded":"2024-09-16T07:22:50.132192+0000","last_fullsized":"2024-09-16T07:22:50.132192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:45.369698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279803+0000","last_change":"2024-09-16T07:22:49.137697+0000","last_active":"2024-09-16T07:22:50.279803+0000","last_peered":"2024-09-16T07:22:50.279803+0000","last_clean":"2024-09-16T07:22:50.279803+0000","last_became_active":"2024-09-16T07:22:49.137443+0000","last_became_peered":"2024-09-16T07:22:49.137443+0000","last_unstale":"2024-09-16T07:22:50.279803+0000","last_undegraded":"2024-09-16T07:22:50.279803+0000","last_fullsized":"2024-09-16T07:22:50.279803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130836+0000","last_change":"2024-09-16T07:22:49.139909+0000","last_active":"2024-09-16T07:22:50.130836+0000","last_peered":"2024-09-16T07:22:50.130836+0000","last_clean":"2024-09-16T07:22:50.130836+0000","last_became_active":"2024-09-16T07:22:49.139628+0000","last_became_peered":"2024-09-16T07:22:49.139628+0000","last_unstale":"2024-09-16T07:22:50.130836+0000","last_undegraded":"2024-09-16T07:22:50.130836+0000","last_fullsized":"2024-09-16T07:22:50.130836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:02.467243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278810+0000","last_change":"2024-09-16T07:22:49.134680+0000","last_active":"2024-09-16T07:22:50.278810+0000","last_peered":"2024-09-16T07:22:50.278810+0000","last_clean":"2024-09-16T07:22:50.278810+0000","last_became_active":"2024-09-16T07:22:49.134374+0000","last_became_peered":"2024-09-16T07:22:49.134374+0000","last_unstale":"2024-09-16T07:22:50.278810+0000","last_undegraded":"2024-09-16T07:22:50.278810+0000","last_fullsized":"2024-09-16T07:22:50.278810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:16.837921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132879+0000","last_change":"2024-09-16T07:22:49.133787+0000","last_active":"2024-09-16T07:22:50.132879+0000","last_peered":"2024-09-16T07:22:50.132879+0000","last_clean":"2024-09-16T07:22:50.132879+0000","last_became_active":"2024-09-16T07:22:49.133552+0000","last_became_peered":"2024-09-16T07:22:49.133552+0000","last_unstale":"2024-09-16T07:22:50.132879+0000","last_undegraded":"2024-09-16T07:22:50.132879+0000","last_fullsized":"2024-09-16T07:22:50.132879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.930728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132323+0000","last_change":"2024-09-16T07:22:49.135809+0000","last_active":"2024-09-16T07:22:50.132323+0000","last_peered":"2024-09-16T07:22:50.132323+0000","last_clean":"2024-09-16T07:22:50.132323+0000","last_became_active":"2024-09-16T07:22:49.135527+0000","last_became_peered":"2024-09-16T07:22:49.135527+0000","last_unstale":"2024-09-16T07:22:50.132323+0000","last_undegraded":"2024-09-16T07:22:50.132323+0000","last_fullsized":"2024-09-16T07:22:50.132323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279729+0000","last_change":"2024-09-16T07:22:49.140506+0000","last_active":"2024-09-16T07:22:50.279729+0000","last_peered":"2024-09-16T07:22:50.279729+0000","last_clean":"2024-09-16T07:22:50.279729+0000","last_became_active":"2024-09-16T07:22:49.140006+0000","last_became_peered":"2024-09-16T07:22:49.140006+0000","last_unstale":"2024-09-16T07:22:50.279729+0000","last_undegraded":"2024-09-16T07:22:50.279729+0000","last_fullsized":"2024-09-16T07:22:50.279729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:29.077071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279953+0000","last_change":"2024-09-16T07:22:49.136459+0000","last_active":"2024-09-16T07:22:50.279953+0000","last_peered":"2024-09-16T07:22:50.279953+0000","last_clean":"2024-09-16T07:22:50.279953+0000","last_became_active":"2024-09-16T07:22:49.136183+0000","last_became_peered":"2024-09-16T07:22:49.136183+0000","last_unstale":"2024-09-16T07:22:50.279953+0000","last_undegraded":"2024-09-16T07:22:50.279953+0000","last_fullsized":"2024-09-16T07:22:50.279953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:33.154370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130793+0000","last_change":"2024-09-16T07:22:49.136171+0000","last_active":"2024-09-16T07:22:50.130793+0000","last_peered":"2024-09-16T07:22:50.130793+0000","last_clean":"2024-09-16T07:22:50.130793+0000","last_became_active":"2024-09-16T07:22:49.135900+0000","last_became_peered":"2024-09-16T07:22:49.135900+0000","last_unstale":"2024-09-16T07:22:50.130793+0000","last_undegraded":"2024-09-16T07:22:50.130793+0000","last_fullsized":"2024-09-16T07:22:50.130793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:01.273769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279131+0000","last_change":"2024-09-16T07:22:49.133629+0000","last_active":"2024-09-16T07:22:50.279131+0000","last_peered":"2024-09-16T07:22:50.279131+0000","last_clean":"2024-09-16T07:22:50.279131+0000","last_became_active":"2024-09-16T07:22:49.133394+0000","last_became_peered":"2024-09-16T07:22:49.133394+0000","last_unstale":"2024-09-16T07:22:50.279131+0000","last_undegraded":"2024-09-16T07:22:50.279131+0000","last_fullsized":"2024-09-16T07:22:50.279131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:41.976217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279399+0000","last_change":"2024-09-16T07:22:49.144945+0000","last_active":"2024-09-16T07:22:50.279399+0000","last_peered":"2024-09-16T07:22:50.279399+0000","last_clean":"2024-09-16T07:22:50.279399+0000","last_became_active":"2024-09-16T07:22:49.144649+0000","last_became_peered":"2024-09-16T07:22:49.144649+0000","last_unstale":"2024-09-16T07:22:50.279399+0000","last_undegraded":"2024-09-16T07:22:50.279399+0000","last_fullsized":"2024-09-16T07:22:50.279399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:59.994632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131706+0000","last_change":"2024-09-16T07:22:49.136574+0000","last_active":"2024-09-16T07:22:50.131706+0000","last_peered":"2024-09-16T07:22:50.131706+0000","last_clean":"2024-09-16T07:22:50.131706+0000","last_became_active":"2024-09-16T07:22:49.136412+0000","last_became_peered":"2024-09-16T07:22:49.136412+0000","last_unstale":"2024-09-16T07:22:50.131706+0000","last_undegraded":"2024-09-16T07:22:50.131706+0000","last_fullsized":"2024-09-16T07:22:50.131706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280014+0000","last_change":"2024-09-16T07:22:49.141959+0000","last_active":"2024-09-16T07:22:50.280014+0000","last_peered":"2024-09-16T07:22:50.280014+0000","last_clean":"2024-09-16T07:22:50.280014+0000","last_became_active":"2024-09-16T07:22:49.140661+0000","last_became_peered":"2024-09-16T07:22:49.140661+0000","last_unstale":"2024-09-16T07:22:50.280014+0000","last_undegraded":"2024-09-16T07:22:50.280014+0000","last_fullsized":"2024-09-16T07:22:50.280014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130863+0000","last_change":"2024-09-16T07:22:48.137796+0000","last_active":"2024-09-16T07:22:50.130863+0000","last_peered":"2024-09-16T07:22:50.130863+0000","last_clean":"2024-09-16T07:22:50.130863+0000","last_became_active":"2024-09-16T07:22:48.136473+0000","last_became_peered":"2024-09-16T07:22:48.136473+0000","last_unstale":"2024-09-16T07:22:50.130863+0000","last_undegraded":"2024-09-16T07:22:50.130863+0000","last_fullsized":"2024-09-16T07:22:50.130863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:16.721715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132215+0000","last_change":"2024-09-16T07:22:49.132936+0000","last_active":"2024-09-16T07:22:50.132215+0000","last_peered":"2024-09-16T07:22:50.132215+0000","last_clean":"2024-09-16T07:22:50.132215+0000","last_became_active":"2024-09-16T07:22:49.132677+0000","last_became_peered":"2024-09-16T07:22:49.132677+0000","last_unstale":"2024-09-16T07:22:50.132215+0000","last_undegraded":"2024-09-16T07:22:50.132215+0000","last_fullsized":"2024-09-16T07:22:50.132215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:16.641320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279272+0000","last_change":"2024-09-16T07:22:48.140563+0000","last_active":"2024-09-16T07:22:50.279272+0000","last_peered":"2024-09-16T07:22:50.279272+0000","last_clean":"2024-09-16T07:22:50.279272+0000","last_became_active":"2024-09-16T07:22:48.139864+0000","last_became_peered":"2024-09-16T07:22:48.139864+0000","last_unstale":"2024-09-16T07:22:50.279272+0000","last_undegraded":"2024-09-16T07:22:50.279272+0000","last_fullsized":"2024-09-16T07:22:50.279272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:36.678707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279310+0000","last_change":"2024-09-16T07:22:48.135463+0000","last_active":"2024-09-16T07:22:50.279310+0000","last_peered":"2024-09-16T07:22:50.279310+0000","last_clean":"2024-09-16T07:22:50.279310+0000","last_became_active":"2024-09-16T07:22:48.135133+0000","last_became_peered":"2024-09-16T07:22:48.135133+0000","last_unstale":"2024-09-16T07:22:50.279310+0000","last_undegraded":"2024-09-16T07:22:50.279310+0000","last_fullsized":"2024-09-16T07:22:50.279310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130763+0000","last_change":"2024-09-16T07:22:49.138488+0000","last_active":"2024-09-16T07:22:50.130763+0000","last_peered":"2024-09-16T07:22:50.130763+0000","last_clean":"2024-09-16T07:22:50.130763+0000","last_became_active":"2024-09-16T07:22:49.138042+0000","last_became_peered":"2024-09-16T07:22:49.138042+0000","last_unstale":"2024-09-16T07:22:50.130763+0000","last_undegraded":"2024-09-16T07:22:50.130763+0000","last_fullsized":"2024-09-16T07:22:50.130763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:20.696268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279074+0000","last_change":"2024-09-16T07:22:48.135129+0000","last_active":"2024-09-16T07:22:50.279074+0000","last_peered":"2024-09-16T07:22:50.279074+0000","last_clean":"2024-09-16T07:22:50.279074+0000","last_became_active":"2024-09-16T07:22:48.134665+0000","last_became_peered":"2024-09-16T07:22:48.134665+0000","last_unstale":"2024-09-16T07:22:50.279074+0000","last_undegraded":"2024-09-16T07:22:50.279074+0000","last_fullsized":"2024-09-16T07:22:50.279074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:58.988982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130571+0000","last_change":"2024-09-16T07:22:49.129262+0000","last_active":"2024-09-16T07:22:50.130571+0000","last_peered":"2024-09-16T07:22:50.130571+0000","last_clean":"2024-09-16T07:22:50.130571+0000","last_became_active":"2024-09-16T07:22:49.128955+0000","last_became_peered":"2024-09-16T07:22:49.128955+0000","last_unstale":"2024-09-16T07:22:50.130571+0000","last_undegraded":"2024-09-16T07:22:50.130571+0000","last_fullsized":"2024-09-16T07:22:50.130571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:35.169775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279936+0000","last_change":"2024-09-16T07:22:48.139495+0000","last_active":"2024-09-16T07:22:50.279936+0000","last_peered":"2024-09-16T07:22:50.279936+0000","last_clean":"2024-09-16T07:22:50.279936+0000","last_became_active":"2024-09-16T07:22:48.139004+0000","last_became_peered":"2024-09-16T07:22:48.139004+0000","last_unstale":"2024-09-16T07:22:50.279936+0000","last_undegraded":"2024-09-16T07:22:50.279936+0000","last_fullsized":"2024-09-16T07:22:50.279936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:22.689639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132362+0000","last_change":"2024-09-16T07:22:49.134719+0000","last_active":"2024-09-16T07:22:50.132362+0000","last_peered":"2024-09-16T07:22:50.132362+0000","last_clean":"2024-09-16T07:22:50.132362+0000","last_became_active":"2024-09-16T07:22:49.134471+0000","last_became_peered":"2024-09-16T07:22:49.134471+0000","last_unstale":"2024-09-16T07:22:50.132362+0000","last_undegraded":"2024-09-16T07:22:50.132362+0000","last_fullsized":"2024-09-16T07:22:50.132362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:15.125396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139537+0000","last_change":"2024-09-16T07:22:48.130409+0000","last_active":"2024-09-16T07:22:50.139537+0000","last_peered":"2024-09-16T07:22:50.139537+0000","last_clean":"2024-09-16T07:22:50.139537+0000","last_became_active":"2024-09-16T07:22:48.130159+0000","last_became_peered":"2024-09-16T07:22:48.130159+0000","last_unstale":"2024-09-16T07:22:50.139537+0000","last_undegraded":"2024-09-16T07:22:50.139537+0000","last_fullsized":"2024-09-16T07:22:50.139537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:08.500032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.133118+0000","last_change":"2024-09-16T07:22:49.140179+0000","last_active":"2024-09-16T07:22:50.133118+0000","last_peered":"2024-09-16T07:22:50.133118+0000","last_clean":"2024-09-16T07:22:50.133118+0000","last_became_active":"2024-09-16T07:22:49.137012+0000","last_became_peered":"2024-09-16T07:22:49.137012+0000","last_unstale":"2024-09-16T07:22:50.133118+0000","last_undegraded":"2024-09-16T07:22:50.133118+0000","last_fullsized":"2024-09-16T07:22:50.133118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:48.929979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280281+0000","last_change":"2024-09-16T07:22:48.140081+0000","last_active":"2024-09-16T07:22:50.280281+0000","last_peered":"2024-09-16T07:22:50.280281+0000","last_clean":"2024-09-16T07:22:50.280281+0000","last_became_active":"2024-09-16T07:22:48.139786+0000","last_became_peered":"2024-09-16T07:22:48.139786+0000","last_unstale":"2024-09-16T07:22:50.280281+0000","last_undegraded":"2024-09-16T07:22:50.280281+0000","last_fullsized":"2024-09-16T07:22:50.280281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280002+0000","last_change":"2024-09-16T07:22:49.135775+0000","last_active":"2024-09-16T07:22:50.280002+0000","last_peered":"2024-09-16T07:22:50.280002+0000","last_clean":"2024-09-16T07:22:50.280002+0000","last_became_active":"2024-09-16T07:22:49.135501+0000","last_became_peered":"2024-09-16T07:22:49.135501+0000","last_unstale":"2024-09-16T07:22:50.280002+0000","last_undegraded":"2024-09-16T07:22:50.280002+0000","last_fullsized":"2024-09-16T07:22:50.280002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:03:48.213796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132027+0000","last_change":"2024-09-16T07:22:48.130827+0000","last_active":"2024-09-16T07:22:50.132027+0000","last_peered":"2024-09-16T07:22:50.132027+0000","last_clean":"2024-09-16T07:22:50.132027+0000","last_became_active":"2024-09-16T07:22:48.129479+0000","last_became_peered":"2024-09-16T07:22:48.129479+0000","last_unstale":"2024-09-16T07:22:50.132027+0000","last_undegraded":"2024-09-16T07:22:50.132027+0000","last_fullsized":"2024-09-16T07:22:50.132027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:34.669437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139797+0000","last_change":"2024-09-16T07:22:49.141394+0000","last_active":"2024-09-16T07:22:50.139797+0000","last_peered":"2024-09-16T07:22:50.139797+0000","last_clean":"2024-09-16T07:22:50.139797+0000","last_became_active":"2024-09-16T07:22:49.141098+0000","last_became_peered":"2024-09-16T07:22:49.141098+0000","last_unstale":"2024-09-16T07:22:50.139797+0000","last_undegraded":"2024-09-16T07:22:50.139797+0000","last_fullsized":"2024-09-16T07:22:50.139797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:52.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279971+0000","last_change":"2024-09-16T07:22:48.137959+0000","last_active":"2024-09-16T07:22:50.279971+0000","last_peered":"2024-09-16T07:22:50.279971+0000","last_clean":"2024-09-16T07:22:50.279971+0000","last_became_active":"2024-09-16T07:22:48.137703+0000","last_became_peered":"2024-09-16T07:22:48.137703+0000","last_unstale":"2024-09-16T07:22:50.279971+0000","last_undegraded":"2024-09-16T07:22:50.279971+0000","last_fullsized":"2024-09-16T07:22:50.279971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279516+0000","last_change":"2024-09-16T07:22:49.133885+0000","last_active":"2024-09-16T07:22:50.279516+0000","last_peered":"2024-09-16T07:22:50.279516+0000","last_clean":"2024-09-16T07:22:50.279516+0000","last_became_active":"2024-09-16T07:22:49.133562+0000","last_became_peered":"2024-09-16T07:22:49.133562+0000","last_unstale":"2024-09-16T07:22:50.279516+0000","last_undegraded":"2024-09-16T07:22:50.279516+0000","last_fullsized":"2024-09-16T07:22:50.279516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:43.685739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279361+0000","last_change":"2024-09-16T07:22:48.137679+0000","last_active":"2024-09-16T07:22:50.279361+0000","last_peered":"2024-09-16T07:22:50.279361+0000","last_clean":"2024-09-16T07:22:50.279361+0000","last_became_active":"2024-09-16T07:22:48.134910+0000","last_became_peered":"2024-09-16T07:22:48.134910+0000","last_unstale":"2024-09-16T07:22:50.279361+0000","last_undegraded":"2024-09-16T07:22:50.279361+0000","last_fullsized":"2024-09-16T07:22:50.279361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:06.837305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279526+0000","last_change":"2024-09-16T07:22:49.131231+0000","last_active":"2024-09-16T07:22:50.279526+0000","last_peered":"2024-09-16T07:22:50.279526+0000","last_clean":"2024-09-16T07:22:50.279526+0000","last_became_active":"2024-09-16T07:22:49.130991+0000","last_became_peered":"2024-09-16T07:22:49.130991+0000","last_unstale":"2024-09-16T07:22:50.279526+0000","last_undegraded":"2024-09-16T07:22:50.279526+0000","last_fullsized":"2024-09-16T07:22:50.279526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:14:00.862069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139154+0000","last_change":"2024-09-16T07:22:48.139812+0000","last_active":"2024-09-16T07:22:50.139154+0000","last_peered":"2024-09-16T07:22:50.139154+0000","last_clean":"2024-09-16T07:22:50.139154+0000","last_became_active":"2024-09-16T07:22:48.139509+0000","last_became_peered":"2024-09-16T07:22:48.139509+0000","last_unstale":"2024-09-16T07:22:50.139154+0000","last_undegraded":"2024-09-16T07:22:50.139154+0000","last_fullsized":"2024-09-16T07:22:50.139154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:46.703518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278833+0000","last_change":"2024-09-16T07:22:49.143207+0000","last_active":"2024-09-16T07:22:50.278833+0000","last_peered":"2024-09-16T07:22:50.278833+0000","last_clean":"2024-09-16T07:22:50.278833+0000","last_became_active":"2024-09-16T07:22:49.139696+0000","last_became_peered":"2024-09-16T07:22:49.139696+0000","last_unstale":"2024-09-16T07:22:50.278833+0000","last_undegraded":"2024-09-16T07:22:50.278833+0000","last_fullsized":"2024-09-16T07:22:50.278833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130773+0000","last_change":"2024-09-16T07:22:48.134633+0000","last_active":"2024-09-16T07:22:50.130773+0000","last_peered":"2024-09-16T07:22:50.130773+0000","last_clean":"2024-09-16T07:22:50.130773+0000","last_became_active":"2024-09-16T07:22:48.134291+0000","last_became_peered":"2024-09-16T07:22:48.134291+0000","last_unstale":"2024-09-16T07:22:50.130773+0000","last_undegraded":"2024-09-16T07:22:50.130773+0000","last_fullsized":"2024-09-16T07:22:50.130773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:35.346741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278836+0000","last_change":"2024-09-16T07:22:49.127175+0000","last_active":"2024-09-16T07:22:50.278836+0000","last_peered":"2024-09-16T07:22:50.278836+0000","last_clean":"2024-09-16T07:22:50.278836+0000","last_became_active":"2024-09-16T07:22:49.126846+0000","last_became_peered":"2024-09-16T07:22:49.126846+0000","last_unstale":"2024-09-16T07:22:50.278836+0000","last_undegraded":"2024-09-16T07:22:50.278836+0000","last_fullsized":"2024-09-16T07:22:50.278836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:51.576491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139738+0000","last_change":"2024-09-16T07:22:48.138039+0000","last_active":"2024-09-16T07:22:50.139738+0000","last_peered":"2024-09-16T07:22:50.139738+0000","last_clean":"2024-09-16T07:22:50.139738+0000","last_became_active":"2024-09-16T07:22:48.137746+0000","last_became_peered":"2024-09-16T07:22:48.137746+0000","last_unstale":"2024-09-16T07:22:50.139738+0000","last_undegraded":"2024-09-16T07:22:50.139738+0000","last_fullsized":"2024-09-16T07:22:50.139738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:09.846796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132500+0000","last_change":"2024-09-16T07:22:49.132362+0000","last_active":"2024-09-16T07:22:50.132500+0000","last_peered":"2024-09-16T07:22:50.132500+0000","last_clean":"2024-09-16T07:22:50.132500+0000","last_became_active":"2024-09-16T07:22:49.131756+0000","last_became_peered":"2024-09-16T07:22:49.131756+0000","last_unstale":"2024-09-16T07:22:50.132500+0000","last_undegraded":"2024-09-16T07:22:50.132500+0000","last_fullsized":"2024-09-16T07:22:50.132500+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:43.685739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132541+0000","last_change":"2024-09-16T07:22:48.137778+0000","last_active":"2024-09-16T07:22:50.132541+0000","last_peered":"2024-09-16T07:22:50.132541+0000","last_clean":"2024-09-16T07:22:50.132541+0000","last_became_active":"2024-09-16T07:22:48.137159+0000","last_became_peered":"2024-09-16T07:22:48.137159+0000","last_unstale":"2024-09-16T07:22:50.132541+0000","last_undegraded":"2024-09-16T07:22:50.132541+0000","last_fullsized":"2024-09-16T07:22:50.132541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:37.703838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139626+0000","last_change":"2024-09-16T07:22:49.130481+0000","last_active":"2024-09-16T07:22:50.139626+0000","last_peered":"2024-09-16T07:22:50.139626+0000","last_clean":"2024-09-16T07:22:50.139626+0000","last_became_active":"2024-09-16T07:22:49.130181+0000","last_became_peered":"2024-09-16T07:22:49.130181+0000","last_unstale":"2024-09-16T07:22:50.139626+0000","last_undegraded":"2024-09-16T07:22:50.139626+0000","last_fullsized":"2024-09-16T07:22:50.139626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:41.565481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130923+0000","last_change":"2024-09-16T07:22:48.135490+0000","last_active":"2024-09-16T07:22:50.130923+0000","last_peered":"2024-09-16T07:22:50.130923+0000","last_clean":"2024-09-16T07:22:50.130923+0000","last_became_active":"2024-09-16T07:22:48.134343+0000","last_became_peered":"2024-09-16T07:22:48.134343+0000","last_unstale":"2024-09-16T07:22:50.130923+0000","last_undegraded":"2024-09-16T07:22:50.130923+0000","last_fullsized":"2024-09-16T07:22:50.130923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:45.020349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139776+0000","last_change":"2024-09-16T07:22:49.144669+0000","last_active":"2024-09-16T07:22:50.139776+0000","last_peered":"2024-09-16T07:22:50.139776+0000","last_clean":"2024-09-16T07:22:50.139776+0000","last_became_active":"2024-09-16T07:22:49.144415+0000","last_became_peered":"2024-09-16T07:22:49.144415+0000","last_unstale":"2024-09-16T07:22:50.139776+0000","last_undegraded":"2024-09-16T07:22:50.139776+0000","last_fullsized":"2024-09-16T07:22:50.139776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:51.576491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280181+0000","last_change":"2024-09-16T07:22:48.135506+0000","last_active":"2024-09-16T07:22:50.280181+0000","last_peered":"2024-09-16T07:22:50.280181+0000","last_clean":"2024-09-16T07:22:50.280181+0000","last_became_active":"2024-09-16T07:22:48.134852+0000","last_became_peered":"2024-09-16T07:22:48.134852+0000","last_unstale":"2024-09-16T07:22:50.280181+0000","last_undegraded":"2024-09-16T07:22:50.280181+0000","last_fullsized":"2024-09-16T07:22:50.280181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279526+0000","last_change":"2024-09-16T07:22:49.134414+0000","last_active":"2024-09-16T07:22:50.279526+0000","last_peered":"2024-09-16T07:22:50.279526+0000","last_clean":"2024-09-16T07:22:50.279526+0000","last_became_active":"2024-09-16T07:22:49.130571+0000","last_became_peered":"2024-09-16T07:22:49.130571+0000","last_unstale":"2024-09-16T07:22:50.279526+0000","last_undegraded":"2024-09-16T07:22:50.279526+0000","last_fullsized":"2024-09-16T07:22:50.279526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:02.699542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279112+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:22:50.279112+0000","last_peered":"2024-09-16T07:22:50.279112+0000","last_clean":"2024-09-16T07:22:50.279112+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:22:50.279112+0000","last_undegraded":"2024-09-16T07:22:50.279112+0000","last_fullsized":"2024-09-16T07:22:50.279112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279775+0000","last_change":"2024-09-16T07:22:48.139675+0000","last_active":"2024-09-16T07:22:50.279775+0000","last_peered":"2024-09-16T07:22:50.279775+0000","last_clean":"2024-09-16T07:22:50.279775+0000","last_became_active":"2024-09-16T07:22:48.137989+0000","last_became_peered":"2024-09-16T07:22:48.137989+0000","last_unstale":"2024-09-16T07:22:50.279775+0000","last_undegraded":"2024-09-16T07:22:50.279775+0000","last_fullsized":"2024-09-16T07:22:50.279775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:16.190625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132139+0000","last_change":"2024-09-16T07:22:49.137102+0000","last_active":"2024-09-16T07:22:50.132139+0000","last_peered":"2024-09-16T07:22:50.132139+0000","last_clean":"2024-09-16T07:22:50.132139+0000","last_became_active":"2024-09-16T07:22:49.136755+0000","last_became_peered":"2024-09-16T07:22:49.136755+0000","last_unstale":"2024-09-16T07:22:50.132139+0000","last_undegraded":"2024-09-16T07:22:50.132139+0000","last_fullsized":"2024-09-16T07:22:50.132139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:26.935160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278962+0000","last_change":"2024-09-16T07:22:48.133273+0000","last_active":"2024-09-16T07:22:50.278962+0000","last_peered":"2024-09-16T07:22:50.278962+0000","last_clean":"2024-09-16T07:22:50.278962+0000","last_became_active":"2024-09-16T07:22:48.132984+0000","last_became_peered":"2024-09-16T07:22:48.132984+0000","last_unstale":"2024-09-16T07:22:50.278962+0000","last_undegraded":"2024-09-16T07:22:50.278962+0000","last_fullsized":"2024-09-16T07:22:50.278962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.657246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279239+0000","last_change":"2024-09-16T07:22:49.134936+0000","last_active":"2024-09-16T07:22:50.279239+0000","last_peered":"2024-09-16T07:22:50.279239+0000","last_clean":"2024-09-16T07:22:50.279239+0000","last_became_active":"2024-09-16T07:22:49.134575+0000","last_became_peered":"2024-09-16T07:22:49.134575+0000","last_unstale":"2024-09-16T07:22:50.279239+0000","last_undegraded":"2024-09-16T07:22:50.279239+0000","last_fullsized":"2024-09-16T07:22:50.279239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279457+0000","last_change":"2024-09-16T07:22:48.140454+0000","last_active":"2024-09-16T07:22:50.279457+0000","last_peered":"2024-09-16T07:22:50.279457+0000","last_clean":"2024-09-16T07:22:50.279457+0000","last_became_active":"2024-09-16T07:22:48.140025+0000","last_became_peered":"2024-09-16T07:22:48.140025+0000","last_unstale":"2024-09-16T07:22:50.279457+0000","last_undegraded":"2024-09-16T07:22:50.279457+0000","last_fullsized":"2024-09-16T07:22:50.279457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130545+0000","last_change":"2024-09-16T07:22:49.138294+0000","last_active":"2024-09-16T07:22:50.130545+0000","last_peered":"2024-09-16T07:22:50.130545+0000","last_clean":"2024-09-16T07:22:50.130545+0000","last_became_active":"2024-09-16T07:22:49.137685+0000","last_became_peered":"2024-09-16T07:22:49.137685+0000","last_unstale":"2024-09-16T07:22:50.130545+0000","last_undegraded":"2024-09-16T07:22:50.130545+0000","last_fullsized":"2024-09-16T07:22:50.130545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:51.068492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279367+0000","last_change":"2024-09-16T07:22:48.139380+0000","last_active":"2024-09-16T07:22:50.279367+0000","last_peered":"2024-09-16T07:22:50.279367+0000","last_clean":"2024-09-16T07:22:50.279367+0000","last_became_active":"2024-09-16T07:22:48.138052+0000","last_became_peered":"2024-09-16T07:22:48.138052+0000","last_unstale":"2024-09-16T07:22:50.279367+0000","last_undegraded":"2024-09-16T07:22:50.279367+0000","last_fullsized":"2024-09-16T07:22:50.279367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:28.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139289+0000","last_change":"2024-09-16T07:22:49.133663+0000","last_active":"2024-09-16T07:22:50.139289+0000","last_peered":"2024-09-16T07:22:50.139289+0000","last_clean":"2024-09-16T07:22:50.139289+0000","last_became_active":"2024-09-16T07:22:49.133342+0000","last_became_peered":"2024-09-16T07:22:49.133342+0000","last_unstale":"2024-09-16T07:22:50.139289+0000","last_undegraded":"2024-09-16T07:22:50.139289+0000","last_fullsized":"2024-09-16T07:22:50.139289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:04.490402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130957+0000","last_change":"2024-09-16T07:22:48.141538+0000","last_active":"2024-09-16T07:22:50.130957+0000","last_peered":"2024-09-16T07:22:50.130957+0000","last_clean":"2024-09-16T07:22:50.130957+0000","last_became_active":"2024-09-16T07:22:48.138174+0000","last_became_peered":"2024-09-16T07:22:48.138174+0000","last_unstale":"2024-09-16T07:22:50.130957+0000","last_undegraded":"2024-09-16T07:22:50.130957+0000","last_fullsized":"2024-09-16T07:22:50.130957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:29.651356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132217+0000","last_change":"2024-09-16T07:22:49.136727+0000","last_active":"2024-09-16T07:22:50.132217+0000","last_peered":"2024-09-16T07:22:50.132217+0000","last_clean":"2024-09-16T07:22:50.132217+0000","last_became_active":"2024-09-16T07:22:49.136064+0000","last_became_peered":"2024-09-16T07:22:49.136064+0000","last_unstale":"2024-09-16T07:22:50.132217+0000","last_undegraded":"2024-09-16T07:22:50.132217+0000","last_fullsized":"2024-09-16T07:22:50.132217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279830+0000","last_change":"2024-09-16T07:22:48.135598+0000","last_active":"2024-09-16T07:22:50.279830+0000","last_peered":"2024-09-16T07:22:50.279830+0000","last_clean":"2024-09-16T07:22:50.279830+0000","last_became_active":"2024-09-16T07:22:48.135179+0000","last_became_peered":"2024-09-16T07:22:48.135179+0000","last_unstale":"2024-09-16T07:22:50.279830+0000","last_undegraded":"2024-09-16T07:22:50.279830+0000","last_fullsized":"2024-09-16T07:22:50.279830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:09.846796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280111+0000","last_change":"2024-09-16T07:22:49.137714+0000","last_active":"2024-09-16T07:22:50.280111+0000","last_peered":"2024-09-16T07:22:50.280111+0000","last_clean":"2024-09-16T07:22:50.280111+0000","last_became_active":"2024-09-16T07:22:49.134740+0000","last_became_peered":"2024-09-16T07:22:49.134740+0000","last_unstale":"2024-09-16T07:22:50.280111+0000","last_undegraded":"2024-09-16T07:22:50.280111+0000","last_fullsized":"2024-09-16T07:22:50.280111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:44.194572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138704+0000","last_change":"2024-09-16T07:22:48.137476+0000","last_active":"2024-09-16T07:22:50.138704+0000","last_peered":"2024-09-16T07:22:50.138704+0000","last_clean":"2024-09-16T07:22:50.138704+0000","last_became_active":"2024-09-16T07:22:48.136772+0000","last_became_peered":"2024-09-16T07:22:48.136772+0000","last_unstale":"2024-09-16T07:22:50.138704+0000","last_undegraded":"2024-09-16T07:22:50.138704+0000","last_fullsized":"2024-09-16T07:22:50.138704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.906034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132443+0000","last_change":"2024-09-16T07:22:49.144331+0000","last_active":"2024-09-16T07:22:50.132443+0000","last_peered":"2024-09-16T07:22:50.132443+0000","last_clean":"2024-09-16T07:22:50.132443+0000","last_became_active":"2024-09-16T07:22:49.144071+0000","last_became_peered":"2024-09-16T07:22:49.144071+0000","last_unstale":"2024-09-16T07:22:50.132443+0000","last_undegraded":"2024-09-16T07:22:50.132443+0000","last_fullsized":"2024-09-16T07:22:50.132443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.753793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280211+0000","last_change":"2024-09-16T07:22:48.132847+0000","last_active":"2024-09-16T07:22:50.280211+0000","last_peered":"2024-09-16T07:22:50.280211+0000","last_clean":"2024-09-16T07:22:50.280211+0000","last_became_active":"2024-09-16T07:22:48.132264+0000","last_became_peered":"2024-09-16T07:22:48.132264+0000","last_unstale":"2024-09-16T07:22:50.280211+0000","last_undegraded":"2024-09-16T07:22:50.280211+0000","last_fullsized":"2024-09-16T07:22:50.280211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:42.680089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:49.138949+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:49.138580+0000","last_became_peered":"2024-09-16T07:22:49.138580+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:30.446387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138995+0000","last_change":"2024-09-16T07:22:48.129412+0000","last_active":"2024-09-16T07:22:50.138995+0000","last_peered":"2024-09-16T07:22:50.138995+0000","last_clean":"2024-09-16T07:22:50.138995+0000","last_became_active":"2024-09-16T07:22:48.129137+0000","last_became_peered":"2024-09-16T07:22:48.129137+0000","last_unstale":"2024-09-16T07:22:50.138995+0000","last_undegraded":"2024-09-16T07:22:50.138995+0000","last_fullsized":"2024-09-16T07:22:50.138995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:07.509813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279051+0000","last_change":"2024-09-16T07:22:49.129827+0000","last_active":"2024-09-16T07:22:50.279051+0000","last_peered":"2024-09-16T07:22:50.279051+0000","last_clean":"2024-09-16T07:22:50.279051+0000","last_became_active":"2024-09-16T07:22:49.129474+0000","last_became_peered":"2024-09-16T07:22:49.129474+0000","last_unstale":"2024-09-16T07:22:50.279051+0000","last_undegraded":"2024-09-16T07:22:50.279051+0000","last_fullsized":"2024-09-16T07:22:50.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:57.650915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130669+0000","last_change":"2024-09-16T07:22:48.134437+0000","last_active":"2024-09-16T07:22:50.130669+0000","last_peered":"2024-09-16T07:22:50.130669+0000","last_clean":"2024-09-16T07:22:50.130669+0000","last_became_active":"2024-09-16T07:22:48.134144+0000","last_became_peered":"2024-09-16T07:22:48.134144+0000","last_unstale":"2024-09-16T07:22:50.130669+0000","last_undegraded":"2024-09-16T07:22:50.130669+0000","last_fullsized":"2024-09-16T07:22:50.130669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:04.051560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130660+0000","last_change":"2024-09-16T07:22:49.141869+0000","last_active":"2024-09-16T07:22:50.130660+0000","last_peered":"2024-09-16T07:22:50.130660+0000","last_clean":"2024-09-16T07:22:50.130660+0000","last_became_active":"2024-09-16T07:22:49.141602+0000","last_became_peered":"2024-09-16T07:22:49.141602+0000","last_unstale":"2024-09-16T07:22:50.130660+0000","last_undegraded":"2024-09-16T07:22:50.130660+0000","last_fullsized":"2024-09-16T07:22:50.130660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:44.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279669+0000","last_change":"2024-09-16T07:22:48.137499+0000","last_active":"2024-09-16T07:22:50.279669+0000","last_peered":"2024-09-16T07:22:50.279669+0000","last_clean":"2024-09-16T07:22:50.279669+0000","last_became_active":"2024-09-16T07:22:48.137177+0000","last_became_peered":"2024-09-16T07:22:48.137177+0000","last_unstale":"2024-09-16T07:22:50.279669+0000","last_undegraded":"2024-09-16T07:22:50.279669+0000","last_fullsized":"2024-09-16T07:22:50.279669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280080+0000","last_change":"2024-09-16T07:22:49.143083+0000","last_active":"2024-09-16T07:22:50.280080+0000","last_peered":"2024-09-16T07:22:50.280080+0000","last_clean":"2024-09-16T07:22:50.280080+0000","last_became_active":"2024-09-16T07:22:49.142817+0000","last_became_peered":"2024-09-16T07:22:49.142817+0000","last_unstale":"2024-09-16T07:22:50.280080+0000","last_undegraded":"2024-09-16T07:22:50.280080+0000","last_fullsized":"2024-09-16T07:22:50.280080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279682+0000","last_change":"2024-09-16T07:22:48.131412+0000","last_active":"2024-09-16T07:22:50.279682+0000","last_peered":"2024-09-16T07:22:50.279682+0000","last_clean":"2024-09-16T07:22:50.279682+0000","last_became_active":"2024-09-16T07:22:48.131167+0000","last_became_peered":"2024-09-16T07:22:48.131167+0000","last_unstale":"2024-09-16T07:22:50.279682+0000","last_undegraded":"2024-09-16T07:22:50.279682+0000","last_fullsized":"2024-09-16T07:22:50.279682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:40.642232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279172+0000","last_change":"2024-09-16T07:22:49.129812+0000","last_active":"2024-09-16T07:22:50.279172+0000","last_peered":"2024-09-16T07:22:50.279172+0000","last_clean":"2024-09-16T07:22:50.279172+0000","last_became_active":"2024-09-16T07:22:49.129511+0000","last_became_peered":"2024-09-16T07:22:49.129511+0000","last_unstale":"2024-09-16T07:22:50.279172+0000","last_undegraded":"2024-09-16T07:22:50.279172+0000","last_fullsized":"2024-09-16T07:22:50.279172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:12.678514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279340+0000","last_change":"2024-09-16T07:22:49.137458+0000","last_active":"2024-09-16T07:22:50.279340+0000","last_peered":"2024-09-16T07:22:50.279340+0000","last_clean":"2024-09-16T07:22:50.279340+0000","last_became_active":"2024-09-16T07:22:49.137122+0000","last_became_peered":"2024-09-16T07:22:49.137122+0000","last_unstale":"2024-09-16T07:22:50.279340+0000","last_undegraded":"2024-09-16T07:22:50.279340+0000","last_fullsized":"2024-09-16T07:22:50.279340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:52.744207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139907+0000","last_change":"2024-09-16T07:22:48.130394+0000","last_active":"2024-09-16T07:22:50.139907+0000","last_peered":"2024-09-16T07:22:50.139907+0000","last_clean":"2024-09-16T07:22:50.139907+0000","last_became_active":"2024-09-16T07:22:48.130013+0000","last_became_peered":"2024-09-16T07:22:48.130013+0000","last_unstale":"2024-09-16T07:22:50.139907+0000","last_undegraded":"2024-09-16T07:22:50.139907+0000","last_fullsized":"2024-09-16T07:22:50.139907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.657246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279527+0000","last_change":"2024-09-16T07:22:49.136594+0000","last_active":"2024-09-16T07:22:50.279527+0000","last_peered":"2024-09-16T07:22:50.279527+0000","last_clean":"2024-09-16T07:22:50.279527+0000","last_became_active":"2024-09-16T07:22:49.136266+0000","last_became_peered":"2024-09-16T07:22:49.136266+0000","last_unstale":"2024-09-16T07:22:50.279527+0000","last_undegraded":"2024-09-16T07:22:50.279527+0000","last_fullsized":"2024-09-16T07:22:50.279527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139950+0000","last_change":"2024-09-16T07:22:48.135403+0000","last_active":"2024-09-16T07:22:50.139950+0000","last_peered":"2024-09-16T07:22:50.139950+0000","last_clean":"2024-09-16T07:22:50.139950+0000","last_became_active":"2024-09-16T07:22:48.135142+0000","last_became_peered":"2024-09-16T07:22:48.135142+0000","last_unstale":"2024-09-16T07:22:50.139950+0000","last_undegraded":"2024-09-16T07:22:50.139950+0000","last_fullsized":"2024-09-16T07:22:50.139950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:58.988982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.133007+0000","last_change":"2024-09-16T07:22:49.129289+0000","last_active":"2024-09-16T07:22:50.133007+0000","last_peered":"2024-09-16T07:22:50.133007+0000","last_clean":"2024-09-16T07:22:50.133007+0000","last_became_active":"2024-09-16T07:22:49.129000+0000","last_became_peered":"2024-09-16T07:22:49.129000+0000","last_unstale":"2024-09-16T07:22:50.133007+0000","last_undegraded":"2024-09-16T07:22:50.133007+0000","last_fullsized":"2024-09-16T07:22:50.133007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:31.607464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:48.132860+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:48.132544+0000","last_became_peered":"2024-09-16T07:22:48.132544+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:52:05.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132482+0000","last_change":"2024-09-16T07:22:49.142356+0000","last_active":"2024-09-16T07:22:50.132482+0000","last_peered":"2024-09-16T07:22:50.132482+0000","last_clean":"2024-09-16T07:22:50.132482+0000","last_became_active":"2024-09-16T07:22:49.141639+0000","last_became_peered":"2024-09-16T07:22:49.141639+0000","last_unstale":"2024-09-16T07:22:50.132482+0000","last_undegraded":"2024-09-16T07:22:50.132482+0000","last_fullsized":"2024-09-16T07:22:50.132482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:57.949375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132622+0000","last_change":"2024-09-16T07:22:48.141242+0000","last_active":"2024-09-16T07:22:50.132622+0000","last_peered":"2024-09-16T07:22:50.132622+0000","last_clean":"2024-09-16T07:22:50.132622+0000","last_became_active":"2024-09-16T07:22:48.139871+0000","last_became_peered":"2024-09-16T07:22:48.139871+0000","last_unstale":"2024-09-16T07:22:50.132622+0000","last_undegraded":"2024-09-16T07:22:50.132622+0000","last_fullsized":"2024-09-16T07:22:50.132622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:32.916879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476745,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":81289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476745,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476745,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":671145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476744,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":80763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":80831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":80707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":831488,"data_stored":671167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:23:03.391 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:23:03.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.584+0000 7ff2fc000640 1 -- 172.21.15.71:0/2328348569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2f415e1b0 msgr2=0x7ff2f40d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.584+0000 7ff2fc000640 1 --2- 172.21.15.71:0/2328348569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2f415e1b0 0x7ff2f40d2020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40d0050 tx=0x7ff2e802a500 comp rx=0 tx=0).stop 2024-09-16T07:23:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.584+0000 7ff2fc000640 1 -- 172.21.15.71:0/2328348569 shutdown_connections 2024-09-16T07:23:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.584+0000 7ff2fc000640 1 --2- 172.21.15.71:0/2328348569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2f40d1b70 0x7ff2f40d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.584+0000 7ff2fc000640 1 --2- 172.21.15.71:0/2328348569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2f415e1b0 0x7ff2f40d2020 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.584+0000 7ff2fc000640 1 -- 172.21.15.71:0/2328348569 >> 172.21.15.71:0/2328348569 conn(0x7ff2f40d3fa0 msgr2=0x7ff2f40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.584+0000 7ff2fc000640 1 -- 172.21.15.71:0/2328348569 shutdown_connections 2024-09-16T07:23:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.585+0000 7ff2fc000640 1 -- 172.21.15.71:0/2328348569 wait complete. 2024-09-16T07:23:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.586+0000 7ff2fc000640 1 Processor -- start 2024-09-16T07:23:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.586+0000 7ff2fc000640 1 -- start start 2024-09-16T07:23:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2fc000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f40d1b70 0x7ff2f40cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2fc000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2f40d2aa0 0x7ff2f41f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2fc000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2f415e1b0 0x7ff2f41f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2fc000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f40d0e60 con 0x7ff2f40d1b70 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2fc000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f40b8940 con 0x7ff2f40d2aa0 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2fc000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f40d0fe0 con 0x7ff2f415e1b0 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2f9600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f40d1b70 0x7ff2f40cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2f9600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f40d1b70 0x7ff2f40cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50982/0 (socket says 172.21.15.71:50982) 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.587+0000 7ff2f9600640 1 -- 172.21.15.71:0/751471130 learned_addr learned my addr 172.21.15.71:0/751471130 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.588+0000 7ff2fa000640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2f415e1b0 0x7ff2f41f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.588+0000 7ff2f9600640 1 -- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2f415e1b0 msgr2=0x7ff2f41f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.588+0000 7ff2f9600640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2f415e1b0 0x7ff2f41f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.588+0000 7ff2f9600640 1 -- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2f40d2aa0 msgr2=0x7ff2f41f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.588+0000 7ff2f9600640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2f40d2aa0 0x7ff2f41f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.588+0000 7ff2f9600640 1 -- 172.21.15.71:0/751471130 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f41f5530 con 0x7ff2f40d1b70 2024-09-16T07:23:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.588+0000 7ff2f9600640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f40d1b70 0x7ff2f40cd4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff2e802aaf0 tx=0x7ff2e8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.588+0000 7ff2f1600640 1 -- 172.21.15.71:0/751471130 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8031d60 con 0x7ff2f40d1b70 2024-09-16T07:23:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.589+0000 7ff2f1600640 1 -- 172.21.15.71:0/751471130 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2e8033e10 con 0x7ff2f40d1b70 2024-09-16T07:23:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.589+0000 7ff2f1600640 1 -- 172.21.15.71:0/751471130 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8005260 con 0x7ff2f40d1b70 2024-09-16T07:23:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.589+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f40c78f0 con 0x7ff2f40d1b70 2024-09-16T07:23:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.589+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f40c8190 con 0x7ff2f40d1b70 2024-09-16T07:23:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.590+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f40d1490 con 0x7ff2f40d1b70 2024-09-16T07:23:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.591+0000 7ff2f1600640 1 -- 172.21.15.71:0/751471130 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff2e8005400 con 0x7ff2f40d1b70 2024-09-16T07:23:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.592+0000 7ff2f1600640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2d4078850 0x7ff2d407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.592+0000 7ff2f1600640 1 -- 172.21.15.71:0/751471130 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff2e80bb8d0 con 0x7ff2f40d1b70 2024-09-16T07:23:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.593+0000 7ff2f8c00640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2d4078850 0x7ff2d407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:03.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.595+0000 7ff2f8c00640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2d4078850 0x7ff2d407ad10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00058c0 tx=0x7ff2e0005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:03.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.596+0000 7ff2f1600640 1 -- 172.21.15.71:0/751471130 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff2e803f030 con 0x7ff2f40d1b70 2024-09-16T07:23:03.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.778+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2f40b9d20 con 0x7ff2d4078850 2024-09-16T07:23:03.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.783+0000 7ff2f1600640 1 -- 172.21.15.71:0/751471130 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7ff2f40b9d20 con 0x7ff2d4078850 2024-09-16T07:23:03.786 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:03.787 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:23:03.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.786+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2d4078850 msgr2=0x7ff2d407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.786+0000 7ff2fc000640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2d4078850 0x7ff2d407ad10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00058c0 tx=0x7ff2e0005850 comp rx=0 tx=0).stop 2024-09-16T07:23:03.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.786+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f40d1b70 msgr2=0x7ff2f40cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:03.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.786+0000 7ff2fc000640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f40d1b70 0x7ff2f40cd4b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff2e802aaf0 tx=0x7ff2e8033c20 comp rx=0 tx=0).stop 2024-09-16T07:23:03.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.787+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 shutdown_connections 2024-09-16T07:23:03.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.787+0000 7ff2fc000640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2d4078850 0x7ff2d407ad10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.787+0000 7ff2fc000640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2f415e1b0 0x7ff2f41f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.787+0000 7ff2fc000640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2f40d2aa0 0x7ff2f41f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.787+0000 7ff2fc000640 1 --2- 172.21.15.71:0/751471130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f40d1b70 0x7ff2f40cd4b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.787+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 >> 172.21.15.71:0/751471130 conn(0x7ff2f40d3fa0 msgr2=0x7ff2f40b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.787+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 shutdown_connections 2024-09-16T07:23:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.787+0000 7ff2fc000640 1 -- 172.21.15.71:0/751471130 wait complete. 2024-09-16T07:23:03.803 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-16T07:23:02.296635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223448,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751272,"statfs":{"total":773094113280,"available":772865302528,"internally_reserved":0,"allocated":3088384,"data_stored":1831212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002944"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139414+0000","last_change":"2024-09-16T07:22:49.144943+0000","last_active":"2024-09-16T07:22:50.139414+0000","last_peered":"2024-09-16T07:22:50.139414+0000","last_clean":"2024-09-16T07:22:50.139414+0000","last_became_active":"2024-09-16T07:22:49.144609+0000","last_became_peered":"2024-09-16T07:22:49.144609+0000","last_unstale":"2024-09-16T07:22:50.139414+0000","last_undegraded":"2024-09-16T07:22:50.139414+0000","last_fullsized":"2024-09-16T07:22:50.139414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:21.053710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132260+0000","last_change":"2024-09-16T07:22:49.142860+0000","last_active":"2024-09-16T07:22:50.132260+0000","last_peered":"2024-09-16T07:22:50.132260+0000","last_clean":"2024-09-16T07:22:50.132260+0000","last_became_active":"2024-09-16T07:22:49.142555+0000","last_became_peered":"2024-09-16T07:22:49.142555+0000","last_unstale":"2024-09-16T07:22:50.132260+0000","last_undegraded":"2024-09-16T07:22:50.132260+0000","last_fullsized":"2024-09-16T07:22:50.132260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:20.138653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130865+0000","last_change":"2024-09-16T07:22:49.141351+0000","last_active":"2024-09-16T07:22:50.130865+0000","last_peered":"2024-09-16T07:22:50.130865+0000","last_clean":"2024-09-16T07:22:50.130865+0000","last_became_active":"2024-09-16T07:22:49.141078+0000","last_became_peered":"2024-09-16T07:22:49.141078+0000","last_unstale":"2024-09-16T07:22:50.130865+0000","last_undegraded":"2024-09-16T07:22:50.130865+0000","last_fullsized":"2024-09-16T07:22:50.130865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:54.707093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132009+0000","last_change":"2024-09-16T07:22:49.130699+0000","last_active":"2024-09-16T07:22:50.132009+0000","last_peered":"2024-09-16T07:22:50.132009+0000","last_clean":"2024-09-16T07:22:50.132009+0000","last_became_active":"2024-09-16T07:22:49.130435+0000","last_became_peered":"2024-09-16T07:22:49.130435+0000","last_unstale":"2024-09-16T07:22:50.132009+0000","last_undegraded":"2024-09-16T07:22:50.132009+0000","last_fullsized":"2024-09-16T07:22:50.132009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:37.684357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139429+0000","last_change":"2024-09-16T07:22:49.139132+0000","last_active":"2024-09-16T07:22:50.139429+0000","last_peered":"2024-09-16T07:22:50.139429+0000","last_clean":"2024-09-16T07:22:50.139429+0000","last_became_active":"2024-09-16T07:22:49.134577+0000","last_became_peered":"2024-09-16T07:22:49.134577+0000","last_unstale":"2024-09-16T07:22:50.139429+0000","last_undegraded":"2024-09-16T07:22:50.139429+0000","last_fullsized":"2024-09-16T07:22:50.139429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:28.720254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131882+0000","last_change":"2024-09-16T07:22:49.130577+0000","last_active":"2024-09-16T07:22:50.131882+0000","last_peered":"2024-09-16T07:22:50.131882+0000","last_clean":"2024-09-16T07:22:50.131882+0000","last_became_active":"2024-09-16T07:22:49.130338+0000","last_became_peered":"2024-09-16T07:22:49.130338+0000","last_unstale":"2024-09-16T07:22:50.131882+0000","last_undegraded":"2024-09-16T07:22:50.131882+0000","last_fullsized":"2024-09-16T07:22:50.131882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139192+0000","last_change":"2024-09-16T07:22:49.142202+0000","last_active":"2024-09-16T07:22:50.139192+0000","last_peered":"2024-09-16T07:22:50.139192+0000","last_clean":"2024-09-16T07:22:50.139192+0000","last_became_active":"2024-09-16T07:22:49.141956+0000","last_became_peered":"2024-09-16T07:22:49.141956+0000","last_unstale":"2024-09-16T07:22:50.139192+0000","last_undegraded":"2024-09-16T07:22:50.139192+0000","last_fullsized":"2024-09-16T07:22:50.139192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:35.827940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130543+0000","last_change":"2024-09-16T07:22:49.130201+0000","last_active":"2024-09-16T07:22:50.130543+0000","last_peered":"2024-09-16T07:22:50.130543+0000","last_clean":"2024-09-16T07:22:50.130543+0000","last_became_active":"2024-09-16T07:22:49.129912+0000","last_became_peered":"2024-09-16T07:22:49.129912+0000","last_unstale":"2024-09-16T07:22:50.130543+0000","last_undegraded":"2024-09-16T07:22:50.130543+0000","last_fullsized":"2024-09-16T07:22:50.130543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:21.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279771+0000","last_change":"2024-09-16T07:22:49.141365+0000","last_active":"2024-09-16T07:22:50.279771+0000","last_peered":"2024-09-16T07:22:50.279771+0000","last_clean":"2024-09-16T07:22:50.279771+0000","last_became_active":"2024-09-16T07:22:49.141041+0000","last_became_peered":"2024-09-16T07:22:49.141041+0000","last_unstale":"2024-09-16T07:22:50.279771+0000","last_undegraded":"2024-09-16T07:22:50.279771+0000","last_fullsized":"2024-09-16T07:22:50.279771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:37.684357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139441+0000","last_change":"2024-09-16T07:22:49.142086+0000","last_active":"2024-09-16T07:22:50.139441+0000","last_peered":"2024-09-16T07:22:50.139441+0000","last_clean":"2024-09-16T07:22:50.139441+0000","last_became_active":"2024-09-16T07:22:49.141832+0000","last_became_peered":"2024-09-16T07:22:49.141832+0000","last_unstale":"2024-09-16T07:22:50.139441+0000","last_undegraded":"2024-09-16T07:22:50.139441+0000","last_fullsized":"2024-09-16T07:22:50.139441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:05.967417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139585+0000","last_change":"2024-09-16T07:22:49.144692+0000","last_active":"2024-09-16T07:22:50.139585+0000","last_peered":"2024-09-16T07:22:50.139585+0000","last_clean":"2024-09-16T07:22:50.139585+0000","last_became_active":"2024-09-16T07:22:49.144396+0000","last_became_peered":"2024-09-16T07:22:49.144396+0000","last_unstale":"2024-09-16T07:22:50.139585+0000","last_undegraded":"2024-09-16T07:22:50.139585+0000","last_fullsized":"2024-09-16T07:22:50.139585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:47.741254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139742+0000","last_change":"2024-09-16T07:22:49.130387+0000","last_active":"2024-09-16T07:22:50.139742+0000","last_peered":"2024-09-16T07:22:50.139742+0000","last_clean":"2024-09-16T07:22:50.139742+0000","last_became_active":"2024-09-16T07:22:49.129994+0000","last_became_peered":"2024-09-16T07:22:49.129994+0000","last_unstale":"2024-09-16T07:22:50.139742+0000","last_undegraded":"2024-09-16T07:22:50.139742+0000","last_fullsized":"2024-09-16T07:22:50.139742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:11.870228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279379+0000","last_change":"2024-09-16T07:22:49.133371+0000","last_active":"2024-09-16T07:22:50.279379+0000","last_peered":"2024-09-16T07:22:50.279379+0000","last_clean":"2024-09-16T07:22:50.279379+0000","last_became_active":"2024-09-16T07:22:49.133088+0000","last_became_peered":"2024-09-16T07:22:49.133088+0000","last_unstale":"2024-09-16T07:22:50.279379+0000","last_undegraded":"2024-09-16T07:22:50.279379+0000","last_fullsized":"2024-09-16T07:22:50.279379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:17.196275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279291+0000","last_change":"2024-09-16T07:22:49.140290+0000","last_active":"2024-09-16T07:22:50.279291+0000","last_peered":"2024-09-16T07:22:50.279291+0000","last_clean":"2024-09-16T07:22:50.279291+0000","last_became_active":"2024-09-16T07:22:49.139999+0000","last_became_peered":"2024-09-16T07:22:49.139999+0000","last_unstale":"2024-09-16T07:22:50.279291+0000","last_undegraded":"2024-09-16T07:22:50.279291+0000","last_fullsized":"2024-09-16T07:22:50.279291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131078+0000","last_change":"2024-09-16T07:22:49.131493+0000","last_active":"2024-09-16T07:22:50.131078+0000","last_peered":"2024-09-16T07:22:50.131078+0000","last_clean":"2024-09-16T07:22:50.131078+0000","last_became_active":"2024-09-16T07:22:49.131241+0000","last_became_peered":"2024-09-16T07:22:49.131241+0000","last_unstale":"2024-09-16T07:22:50.131078+0000","last_undegraded":"2024-09-16T07:22:50.131078+0000","last_fullsized":"2024-09-16T07:22:50.131078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:24.470314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278778+0000","last_change":"2024-09-16T07:22:49.126099+0000","last_active":"2024-09-16T07:22:50.278778+0000","last_peered":"2024-09-16T07:22:50.278778+0000","last_clean":"2024-09-16T07:22:50.278778+0000","last_became_active":"2024-09-16T07:22:49.125826+0000","last_became_peered":"2024-09-16T07:22:49.125826+0000","last_unstale":"2024-09-16T07:22:50.278778+0000","last_undegraded":"2024-09-16T07:22:50.278778+0000","last_fullsized":"2024-09-16T07:22:50.278778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:41.565481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280232+0000","last_change":"2024-09-16T07:22:49.132617+0000","last_active":"2024-09-16T07:22:50.280232+0000","last_peered":"2024-09-16T07:22:50.280232+0000","last_clean":"2024-09-16T07:22:50.280232+0000","last_became_active":"2024-09-16T07:22:49.132278+0000","last_became_peered":"2024-09-16T07:22:49.132278+0000","last_unstale":"2024-09-16T07:22:50.280232+0000","last_undegraded":"2024-09-16T07:22:50.280232+0000","last_fullsized":"2024-09-16T07:22:50.280232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:52:10.129478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280258+0000","last_change":"2024-09-16T07:22:49.143862+0000","last_active":"2024-09-16T07:22:50.280258+0000","last_peered":"2024-09-16T07:22:50.280258+0000","last_clean":"2024-09-16T07:22:50.280258+0000","last_became_active":"2024-09-16T07:22:49.143457+0000","last_became_peered":"2024-09-16T07:22:49.143457+0000","last_unstale":"2024-09-16T07:22:50.280258+0000","last_undegraded":"2024-09-16T07:22:50.280258+0000","last_fullsized":"2024-09-16T07:22:50.280258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279691+0000","last_change":"2024-09-16T07:22:49.130309+0000","last_active":"2024-09-16T07:22:50.279691+0000","last_peered":"2024-09-16T07:22:50.279691+0000","last_clean":"2024-09-16T07:22:50.279691+0000","last_became_active":"2024-09-16T07:22:49.130069+0000","last_became_peered":"2024-09-16T07:22:49.130069+0000","last_unstale":"2024-09-16T07:22:50.279691+0000","last_undegraded":"2024-09-16T07:22:50.279691+0000","last_fullsized":"2024-09-16T07:22:50.279691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:20.784563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139554+0000","last_change":"2024-09-16T07:22:49.129763+0000","last_active":"2024-09-16T07:22:50.139554+0000","last_peered":"2024-09-16T07:22:50.139554+0000","last_clean":"2024-09-16T07:22:50.139554+0000","last_became_active":"2024-09-16T07:22:49.129067+0000","last_became_peered":"2024-09-16T07:22:49.129067+0000","last_unstale":"2024-09-16T07:22:50.139554+0000","last_undegraded":"2024-09-16T07:22:50.139554+0000","last_fullsized":"2024-09-16T07:22:50.139554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:24.919949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132192+0000","last_change":"2024-09-16T07:22:49.133568+0000","last_active":"2024-09-16T07:22:50.132192+0000","last_peered":"2024-09-16T07:22:50.132192+0000","last_clean":"2024-09-16T07:22:50.132192+0000","last_became_active":"2024-09-16T07:22:49.133323+0000","last_became_peered":"2024-09-16T07:22:49.133323+0000","last_unstale":"2024-09-16T07:22:50.132192+0000","last_undegraded":"2024-09-16T07:22:50.132192+0000","last_fullsized":"2024-09-16T07:22:50.132192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:45.369698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279803+0000","last_change":"2024-09-16T07:22:49.137697+0000","last_active":"2024-09-16T07:22:50.279803+0000","last_peered":"2024-09-16T07:22:50.279803+0000","last_clean":"2024-09-16T07:22:50.279803+0000","last_became_active":"2024-09-16T07:22:49.137443+0000","last_became_peered":"2024-09-16T07:22:49.137443+0000","last_unstale":"2024-09-16T07:22:50.279803+0000","last_undegraded":"2024-09-16T07:22:50.279803+0000","last_fullsized":"2024-09-16T07:22:50.279803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130836+0000","last_change":"2024-09-16T07:22:49.139909+0000","last_active":"2024-09-16T07:22:50.130836+0000","last_peered":"2024-09-16T07:22:50.130836+0000","last_clean":"2024-09-16T07:22:50.130836+0000","last_became_active":"2024-09-16T07:22:49.139628+0000","last_became_peered":"2024-09-16T07:22:49.139628+0000","last_unstale":"2024-09-16T07:22:50.130836+0000","last_undegraded":"2024-09-16T07:22:50.130836+0000","last_fullsized":"2024-09-16T07:22:50.130836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:02.467243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278810+0000","last_change":"2024-09-16T07:22:49.134680+0000","last_active":"2024-09-16T07:22:50.278810+0000","last_peered":"2024-09-16T07:22:50.278810+0000","last_clean":"2024-09-16T07:22:50.278810+0000","last_became_active":"2024-09-16T07:22:49.134374+0000","last_became_peered":"2024-09-16T07:22:49.134374+0000","last_unstale":"2024-09-16T07:22:50.278810+0000","last_undegraded":"2024-09-16T07:22:50.278810+0000","last_fullsized":"2024-09-16T07:22:50.278810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:16.837921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132879+0000","last_change":"2024-09-16T07:22:49.133787+0000","last_active":"2024-09-16T07:22:50.132879+0000","last_peered":"2024-09-16T07:22:50.132879+0000","last_clean":"2024-09-16T07:22:50.132879+0000","last_became_active":"2024-09-16T07:22:49.133552+0000","last_became_peered":"2024-09-16T07:22:49.133552+0000","last_unstale":"2024-09-16T07:22:50.132879+0000","last_undegraded":"2024-09-16T07:22:50.132879+0000","last_fullsized":"2024-09-16T07:22:50.132879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.930728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132323+0000","last_change":"2024-09-16T07:22:49.135809+0000","last_active":"2024-09-16T07:22:50.132323+0000","last_peered":"2024-09-16T07:22:50.132323+0000","last_clean":"2024-09-16T07:22:50.132323+0000","last_became_active":"2024-09-16T07:22:49.135527+0000","last_became_peered":"2024-09-16T07:22:49.135527+0000","last_unstale":"2024-09-16T07:22:50.132323+0000","last_undegraded":"2024-09-16T07:22:50.132323+0000","last_fullsized":"2024-09-16T07:22:50.132323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279729+0000","last_change":"2024-09-16T07:22:49.140506+0000","last_active":"2024-09-16T07:22:50.279729+0000","last_peered":"2024-09-16T07:22:50.279729+0000","last_clean":"2024-09-16T07:22:50.279729+0000","last_became_active":"2024-09-16T07:22:49.140006+0000","last_became_peered":"2024-09-16T07:22:49.140006+0000","last_unstale":"2024-09-16T07:22:50.279729+0000","last_undegraded":"2024-09-16T07:22:50.279729+0000","last_fullsized":"2024-09-16T07:22:50.279729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:29.077071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279953+0000","last_change":"2024-09-16T07:22:49.136459+0000","last_active":"2024-09-16T07:22:50.279953+0000","last_peered":"2024-09-16T07:22:50.279953+0000","last_clean":"2024-09-16T07:22:50.279953+0000","last_became_active":"2024-09-16T07:22:49.136183+0000","last_became_peered":"2024-09-16T07:22:49.136183+0000","last_unstale":"2024-09-16T07:22:50.279953+0000","last_undegraded":"2024-09-16T07:22:50.279953+0000","last_fullsized":"2024-09-16T07:22:50.279953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:33.154370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130793+0000","last_change":"2024-09-16T07:22:49.136171+0000","last_active":"2024-09-16T07:22:50.130793+0000","last_peered":"2024-09-16T07:22:50.130793+0000","last_clean":"2024-09-16T07:22:50.130793+0000","last_became_active":"2024-09-16T07:22:49.135900+0000","last_became_peered":"2024-09-16T07:22:49.135900+0000","last_unstale":"2024-09-16T07:22:50.130793+0000","last_undegraded":"2024-09-16T07:22:50.130793+0000","last_fullsized":"2024-09-16T07:22:50.130793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:01.273769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279131+0000","last_change":"2024-09-16T07:22:49.133629+0000","last_active":"2024-09-16T07:22:50.279131+0000","last_peered":"2024-09-16T07:22:50.279131+0000","last_clean":"2024-09-16T07:22:50.279131+0000","last_became_active":"2024-09-16T07:22:49.133394+0000","last_became_peered":"2024-09-16T07:22:49.133394+0000","last_unstale":"2024-09-16T07:22:50.279131+0000","last_undegraded":"2024-09-16T07:22:50.279131+0000","last_fullsized":"2024-09-16T07:22:50.279131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:59:41.976217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279399+0000","last_change":"2024-09-16T07:22:49.144945+0000","last_active":"2024-09-16T07:22:50.279399+0000","last_peered":"2024-09-16T07:22:50.279399+0000","last_clean":"2024-09-16T07:22:50.279399+0000","last_became_active":"2024-09-16T07:22:49.144649+0000","last_became_peered":"2024-09-16T07:22:49.144649+0000","last_unstale":"2024-09-16T07:22:50.279399+0000","last_undegraded":"2024-09-16T07:22:50.279399+0000","last_fullsized":"2024-09-16T07:22:50.279399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:59.994632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.131706+0000","last_change":"2024-09-16T07:22:49.136574+0000","last_active":"2024-09-16T07:22:50.131706+0000","last_peered":"2024-09-16T07:22:50.131706+0000","last_clean":"2024-09-16T07:22:50.131706+0000","last_became_active":"2024-09-16T07:22:49.136412+0000","last_became_peered":"2024-09-16T07:22:49.136412+0000","last_unstale":"2024-09-16T07:22:50.131706+0000","last_undegraded":"2024-09-16T07:22:50.131706+0000","last_fullsized":"2024-09-16T07:22:50.131706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280014+0000","last_change":"2024-09-16T07:22:49.141959+0000","last_active":"2024-09-16T07:22:50.280014+0000","last_peered":"2024-09-16T07:22:50.280014+0000","last_clean":"2024-09-16T07:22:50.280014+0000","last_became_active":"2024-09-16T07:22:49.140661+0000","last_became_peered":"2024-09-16T07:22:49.140661+0000","last_unstale":"2024-09-16T07:22:50.280014+0000","last_undegraded":"2024-09-16T07:22:50.280014+0000","last_fullsized":"2024-09-16T07:22:50.280014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:41.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130863+0000","last_change":"2024-09-16T07:22:48.137796+0000","last_active":"2024-09-16T07:22:50.130863+0000","last_peered":"2024-09-16T07:22:50.130863+0000","last_clean":"2024-09-16T07:22:50.130863+0000","last_became_active":"2024-09-16T07:22:48.136473+0000","last_became_peered":"2024-09-16T07:22:48.136473+0000","last_unstale":"2024-09-16T07:22:50.130863+0000","last_undegraded":"2024-09-16T07:22:50.130863+0000","last_fullsized":"2024-09-16T07:22:50.130863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:16.721715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132215+0000","last_change":"2024-09-16T07:22:49.132936+0000","last_active":"2024-09-16T07:22:50.132215+0000","last_peered":"2024-09-16T07:22:50.132215+0000","last_clean":"2024-09-16T07:22:50.132215+0000","last_became_active":"2024-09-16T07:22:49.132677+0000","last_became_peered":"2024-09-16T07:22:49.132677+0000","last_unstale":"2024-09-16T07:22:50.132215+0000","last_undegraded":"2024-09-16T07:22:50.132215+0000","last_fullsized":"2024-09-16T07:22:50.132215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:16.641320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279272+0000","last_change":"2024-09-16T07:22:48.140563+0000","last_active":"2024-09-16T07:22:50.279272+0000","last_peered":"2024-09-16T07:22:50.279272+0000","last_clean":"2024-09-16T07:22:50.279272+0000","last_became_active":"2024-09-16T07:22:48.139864+0000","last_became_peered":"2024-09-16T07:22:48.139864+0000","last_unstale":"2024-09-16T07:22:50.279272+0000","last_undegraded":"2024-09-16T07:22:50.279272+0000","last_fullsized":"2024-09-16T07:22:50.279272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:36.678707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279310+0000","last_change":"2024-09-16T07:22:48.135463+0000","last_active":"2024-09-16T07:22:50.279310+0000","last_peered":"2024-09-16T07:22:50.279310+0000","last_clean":"2024-09-16T07:22:50.279310+0000","last_became_active":"2024-09-16T07:22:48.135133+0000","last_became_peered":"2024-09-16T07:22:48.135133+0000","last_unstale":"2024-09-16T07:22:50.279310+0000","last_undegraded":"2024-09-16T07:22:50.279310+0000","last_fullsized":"2024-09-16T07:22:50.279310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130763+0000","last_change":"2024-09-16T07:22:49.138488+0000","last_active":"2024-09-16T07:22:50.130763+0000","last_peered":"2024-09-16T07:22:50.130763+0000","last_clean":"2024-09-16T07:22:50.130763+0000","last_became_active":"2024-09-16T07:22:49.138042+0000","last_became_peered":"2024-09-16T07:22:49.138042+0000","last_unstale":"2024-09-16T07:22:50.130763+0000","last_undegraded":"2024-09-16T07:22:50.130763+0000","last_fullsized":"2024-09-16T07:22:50.130763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:20.696268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279074+0000","last_change":"2024-09-16T07:22:48.135129+0000","last_active":"2024-09-16T07:22:50.279074+0000","last_peered":"2024-09-16T07:22:50.279074+0000","last_clean":"2024-09-16T07:22:50.279074+0000","last_became_active":"2024-09-16T07:22:48.134665+0000","last_became_peered":"2024-09-16T07:22:48.134665+0000","last_unstale":"2024-09-16T07:22:50.279074+0000","last_undegraded":"2024-09-16T07:22:50.279074+0000","last_fullsized":"2024-09-16T07:22:50.279074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:58.988982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130571+0000","last_change":"2024-09-16T07:22:49.129262+0000","last_active":"2024-09-16T07:22:50.130571+0000","last_peered":"2024-09-16T07:22:50.130571+0000","last_clean":"2024-09-16T07:22:50.130571+0000","last_became_active":"2024-09-16T07:22:49.128955+0000","last_became_peered":"2024-09-16T07:22:49.128955+0000","last_unstale":"2024-09-16T07:22:50.130571+0000","last_undegraded":"2024-09-16T07:22:50.130571+0000","last_fullsized":"2024-09-16T07:22:50.130571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:35.169775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279936+0000","last_change":"2024-09-16T07:22:48.139495+0000","last_active":"2024-09-16T07:22:50.279936+0000","last_peered":"2024-09-16T07:22:50.279936+0000","last_clean":"2024-09-16T07:22:50.279936+0000","last_became_active":"2024-09-16T07:22:48.139004+0000","last_became_peered":"2024-09-16T07:22:48.139004+0000","last_unstale":"2024-09-16T07:22:50.279936+0000","last_undegraded":"2024-09-16T07:22:50.279936+0000","last_fullsized":"2024-09-16T07:22:50.279936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:22.689639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132362+0000","last_change":"2024-09-16T07:22:49.134719+0000","last_active":"2024-09-16T07:22:50.132362+0000","last_peered":"2024-09-16T07:22:50.132362+0000","last_clean":"2024-09-16T07:22:50.132362+0000","last_became_active":"2024-09-16T07:22:49.134471+0000","last_became_peered":"2024-09-16T07:22:49.134471+0000","last_unstale":"2024-09-16T07:22:50.132362+0000","last_undegraded":"2024-09-16T07:22:50.132362+0000","last_fullsized":"2024-09-16T07:22:50.132362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:15.125396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139537+0000","last_change":"2024-09-16T07:22:48.130409+0000","last_active":"2024-09-16T07:22:50.139537+0000","last_peered":"2024-09-16T07:22:50.139537+0000","last_clean":"2024-09-16T07:22:50.139537+0000","last_became_active":"2024-09-16T07:22:48.130159+0000","last_became_peered":"2024-09-16T07:22:48.130159+0000","last_unstale":"2024-09-16T07:22:50.139537+0000","last_undegraded":"2024-09-16T07:22:50.139537+0000","last_fullsized":"2024-09-16T07:22:50.139537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:08.500032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.133118+0000","last_change":"2024-09-16T07:22:49.140179+0000","last_active":"2024-09-16T07:22:50.133118+0000","last_peered":"2024-09-16T07:22:50.133118+0000","last_clean":"2024-09-16T07:22:50.133118+0000","last_became_active":"2024-09-16T07:22:49.137012+0000","last_became_peered":"2024-09-16T07:22:49.137012+0000","last_unstale":"2024-09-16T07:22:50.133118+0000","last_undegraded":"2024-09-16T07:22:50.133118+0000","last_fullsized":"2024-09-16T07:22:50.133118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:48.929979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280281+0000","last_change":"2024-09-16T07:22:48.140081+0000","last_active":"2024-09-16T07:22:50.280281+0000","last_peered":"2024-09-16T07:22:50.280281+0000","last_clean":"2024-09-16T07:22:50.280281+0000","last_became_active":"2024-09-16T07:22:48.139786+0000","last_became_peered":"2024-09-16T07:22:48.139786+0000","last_unstale":"2024-09-16T07:22:50.280281+0000","last_undegraded":"2024-09-16T07:22:50.280281+0000","last_fullsized":"2024-09-16T07:22:50.280281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280002+0000","last_change":"2024-09-16T07:22:49.135775+0000","last_active":"2024-09-16T07:22:50.280002+0000","last_peered":"2024-09-16T07:22:50.280002+0000","last_clean":"2024-09-16T07:22:50.280002+0000","last_became_active":"2024-09-16T07:22:49.135501+0000","last_became_peered":"2024-09-16T07:22:49.135501+0000","last_unstale":"2024-09-16T07:22:50.280002+0000","last_undegraded":"2024-09-16T07:22:50.280002+0000","last_fullsized":"2024-09-16T07:22:50.280002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:03:48.213796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132027+0000","last_change":"2024-09-16T07:22:48.130827+0000","last_active":"2024-09-16T07:22:50.132027+0000","last_peered":"2024-09-16T07:22:50.132027+0000","last_clean":"2024-09-16T07:22:50.132027+0000","last_became_active":"2024-09-16T07:22:48.129479+0000","last_became_peered":"2024-09-16T07:22:48.129479+0000","last_unstale":"2024-09-16T07:22:50.132027+0000","last_undegraded":"2024-09-16T07:22:50.132027+0000","last_fullsized":"2024-09-16T07:22:50.132027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:34.669437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139797+0000","last_change":"2024-09-16T07:22:49.141394+0000","last_active":"2024-09-16T07:22:50.139797+0000","last_peered":"2024-09-16T07:22:50.139797+0000","last_clean":"2024-09-16T07:22:50.139797+0000","last_became_active":"2024-09-16T07:22:49.141098+0000","last_became_peered":"2024-09-16T07:22:49.141098+0000","last_unstale":"2024-09-16T07:22:50.139797+0000","last_undegraded":"2024-09-16T07:22:50.139797+0000","last_fullsized":"2024-09-16T07:22:50.139797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:52.847731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279971+0000","last_change":"2024-09-16T07:22:48.137959+0000","last_active":"2024-09-16T07:22:50.279971+0000","last_peered":"2024-09-16T07:22:50.279971+0000","last_clean":"2024-09-16T07:22:50.279971+0000","last_became_active":"2024-09-16T07:22:48.137703+0000","last_became_peered":"2024-09-16T07:22:48.137703+0000","last_unstale":"2024-09-16T07:22:50.279971+0000","last_undegraded":"2024-09-16T07:22:50.279971+0000","last_fullsized":"2024-09-16T07:22:50.279971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279516+0000","last_change":"2024-09-16T07:22:49.133885+0000","last_active":"2024-09-16T07:22:50.279516+0000","last_peered":"2024-09-16T07:22:50.279516+0000","last_clean":"2024-09-16T07:22:50.279516+0000","last_became_active":"2024-09-16T07:22:49.133562+0000","last_became_peered":"2024-09-16T07:22:49.133562+0000","last_unstale":"2024-09-16T07:22:50.279516+0000","last_undegraded":"2024-09-16T07:22:50.279516+0000","last_fullsized":"2024-09-16T07:22:50.279516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:43.685739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279361+0000","last_change":"2024-09-16T07:22:48.137679+0000","last_active":"2024-09-16T07:22:50.279361+0000","last_peered":"2024-09-16T07:22:50.279361+0000","last_clean":"2024-09-16T07:22:50.279361+0000","last_became_active":"2024-09-16T07:22:48.134910+0000","last_became_peered":"2024-09-16T07:22:48.134910+0000","last_unstale":"2024-09-16T07:22:50.279361+0000","last_undegraded":"2024-09-16T07:22:50.279361+0000","last_fullsized":"2024-09-16T07:22:50.279361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:06.837305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279526+0000","last_change":"2024-09-16T07:22:49.131231+0000","last_active":"2024-09-16T07:22:50.279526+0000","last_peered":"2024-09-16T07:22:50.279526+0000","last_clean":"2024-09-16T07:22:50.279526+0000","last_became_active":"2024-09-16T07:22:49.130991+0000","last_became_peered":"2024-09-16T07:22:49.130991+0000","last_unstale":"2024-09-16T07:22:50.279526+0000","last_undegraded":"2024-09-16T07:22:50.279526+0000","last_fullsized":"2024-09-16T07:22:50.279526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:14:00.862069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139154+0000","last_change":"2024-09-16T07:22:48.139812+0000","last_active":"2024-09-16T07:22:50.139154+0000","last_peered":"2024-09-16T07:22:50.139154+0000","last_clean":"2024-09-16T07:22:50.139154+0000","last_became_active":"2024-09-16T07:22:48.139509+0000","last_became_peered":"2024-09-16T07:22:48.139509+0000","last_unstale":"2024-09-16T07:22:50.139154+0000","last_undegraded":"2024-09-16T07:22:50.139154+0000","last_fullsized":"2024-09-16T07:22:50.139154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:46.703518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278833+0000","last_change":"2024-09-16T07:22:49.143207+0000","last_active":"2024-09-16T07:22:50.278833+0000","last_peered":"2024-09-16T07:22:50.278833+0000","last_clean":"2024-09-16T07:22:50.278833+0000","last_became_active":"2024-09-16T07:22:49.139696+0000","last_became_peered":"2024-09-16T07:22:49.139696+0000","last_unstale":"2024-09-16T07:22:50.278833+0000","last_undegraded":"2024-09-16T07:22:50.278833+0000","last_fullsized":"2024-09-16T07:22:50.278833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130773+0000","last_change":"2024-09-16T07:22:48.134633+0000","last_active":"2024-09-16T07:22:50.130773+0000","last_peered":"2024-09-16T07:22:50.130773+0000","last_clean":"2024-09-16T07:22:50.130773+0000","last_became_active":"2024-09-16T07:22:48.134291+0000","last_became_peered":"2024-09-16T07:22:48.134291+0000","last_unstale":"2024-09-16T07:22:50.130773+0000","last_undegraded":"2024-09-16T07:22:50.130773+0000","last_fullsized":"2024-09-16T07:22:50.130773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:35.346741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278836+0000","last_change":"2024-09-16T07:22:49.127175+0000","last_active":"2024-09-16T07:22:50.278836+0000","last_peered":"2024-09-16T07:22:50.278836+0000","last_clean":"2024-09-16T07:22:50.278836+0000","last_became_active":"2024-09-16T07:22:49.126846+0000","last_became_peered":"2024-09-16T07:22:49.126846+0000","last_unstale":"2024-09-16T07:22:50.278836+0000","last_undegraded":"2024-09-16T07:22:50.278836+0000","last_fullsized":"2024-09-16T07:22:50.278836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:51.576491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139738+0000","last_change":"2024-09-16T07:22:48.138039+0000","last_active":"2024-09-16T07:22:50.139738+0000","last_peered":"2024-09-16T07:22:50.139738+0000","last_clean":"2024-09-16T07:22:50.139738+0000","last_became_active":"2024-09-16T07:22:48.137746+0000","last_became_peered":"2024-09-16T07:22:48.137746+0000","last_unstale":"2024-09-16T07:22:50.139738+0000","last_undegraded":"2024-09-16T07:22:50.139738+0000","last_fullsized":"2024-09-16T07:22:50.139738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:09.846796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132500+0000","last_change":"2024-09-16T07:22:49.132362+0000","last_active":"2024-09-16T07:22:50.132500+0000","last_peered":"2024-09-16T07:22:50.132500+0000","last_clean":"2024-09-16T07:22:50.132500+0000","last_became_active":"2024-09-16T07:22:49.131756+0000","last_became_peered":"2024-09-16T07:22:49.131756+0000","last_unstale":"2024-09-16T07:22:50.132500+0000","last_undegraded":"2024-09-16T07:22:50.132500+0000","last_fullsized":"2024-09-16T07:22:50.132500+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:43.685739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132541+0000","last_change":"2024-09-16T07:22:48.137778+0000","last_active":"2024-09-16T07:22:50.132541+0000","last_peered":"2024-09-16T07:22:50.132541+0000","last_clean":"2024-09-16T07:22:50.132541+0000","last_became_active":"2024-09-16T07:22:48.137159+0000","last_became_peered":"2024-09-16T07:22:48.137159+0000","last_unstale":"2024-09-16T07:22:50.132541+0000","last_undegraded":"2024-09-16T07:22:50.132541+0000","last_fullsized":"2024-09-16T07:22:50.132541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:37.703838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139626+0000","last_change":"2024-09-16T07:22:49.130481+0000","last_active":"2024-09-16T07:22:50.139626+0000","last_peered":"2024-09-16T07:22:50.139626+0000","last_clean":"2024-09-16T07:22:50.139626+0000","last_became_active":"2024-09-16T07:22:49.130181+0000","last_became_peered":"2024-09-16T07:22:49.130181+0000","last_unstale":"2024-09-16T07:22:50.139626+0000","last_undegraded":"2024-09-16T07:22:50.139626+0000","last_fullsized":"2024-09-16T07:22:50.139626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:41.565481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130923+0000","last_change":"2024-09-16T07:22:48.135490+0000","last_active":"2024-09-16T07:22:50.130923+0000","last_peered":"2024-09-16T07:22:50.130923+0000","last_clean":"2024-09-16T07:22:50.130923+0000","last_became_active":"2024-09-16T07:22:48.134343+0000","last_became_peered":"2024-09-16T07:22:48.134343+0000","last_unstale":"2024-09-16T07:22:50.130923+0000","last_undegraded":"2024-09-16T07:22:50.130923+0000","last_fullsized":"2024-09-16T07:22:50.130923+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:45.020349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139776+0000","last_change":"2024-09-16T07:22:49.144669+0000","last_active":"2024-09-16T07:22:50.139776+0000","last_peered":"2024-09-16T07:22:50.139776+0000","last_clean":"2024-09-16T07:22:50.139776+0000","last_became_active":"2024-09-16T07:22:49.144415+0000","last_became_peered":"2024-09-16T07:22:49.144415+0000","last_unstale":"2024-09-16T07:22:50.139776+0000","last_undegraded":"2024-09-16T07:22:50.139776+0000","last_fullsized":"2024-09-16T07:22:50.139776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:51.576491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280181+0000","last_change":"2024-09-16T07:22:48.135506+0000","last_active":"2024-09-16T07:22:50.280181+0000","last_peered":"2024-09-16T07:22:50.280181+0000","last_clean":"2024-09-16T07:22:50.280181+0000","last_became_active":"2024-09-16T07:22:48.134852+0000","last_became_peered":"2024-09-16T07:22:48.134852+0000","last_unstale":"2024-09-16T07:22:50.280181+0000","last_undegraded":"2024-09-16T07:22:50.280181+0000","last_fullsized":"2024-09-16T07:22:50.280181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:55.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279526+0000","last_change":"2024-09-16T07:22:49.134414+0000","last_active":"2024-09-16T07:22:50.279526+0000","last_peered":"2024-09-16T07:22:50.279526+0000","last_clean":"2024-09-16T07:22:50.279526+0000","last_became_active":"2024-09-16T07:22:49.130571+0000","last_became_peered":"2024-09-16T07:22:49.130571+0000","last_unstale":"2024-09-16T07:22:50.279526+0000","last_undegraded":"2024-09-16T07:22:50.279526+0000","last_fullsized":"2024-09-16T07:22:50.279526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:02.699542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279112+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:22:50.279112+0000","last_peered":"2024-09-16T07:22:50.279112+0000","last_clean":"2024-09-16T07:22:50.279112+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:22:50.279112+0000","last_undegraded":"2024-09-16T07:22:50.279112+0000","last_fullsized":"2024-09-16T07:22:50.279112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279775+0000","last_change":"2024-09-16T07:22:48.139675+0000","last_active":"2024-09-16T07:22:50.279775+0000","last_peered":"2024-09-16T07:22:50.279775+0000","last_clean":"2024-09-16T07:22:50.279775+0000","last_became_active":"2024-09-16T07:22:48.137989+0000","last_became_peered":"2024-09-16T07:22:48.137989+0000","last_unstale":"2024-09-16T07:22:50.279775+0000","last_undegraded":"2024-09-16T07:22:50.279775+0000","last_fullsized":"2024-09-16T07:22:50.279775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:17:16.190625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132139+0000","last_change":"2024-09-16T07:22:49.137102+0000","last_active":"2024-09-16T07:22:50.132139+0000","last_peered":"2024-09-16T07:22:50.132139+0000","last_clean":"2024-09-16T07:22:50.132139+0000","last_became_active":"2024-09-16T07:22:49.136755+0000","last_became_peered":"2024-09-16T07:22:49.136755+0000","last_unstale":"2024-09-16T07:22:50.132139+0000","last_undegraded":"2024-09-16T07:22:50.132139+0000","last_fullsized":"2024-09-16T07:22:50.132139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:26.935160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.278962+0000","last_change":"2024-09-16T07:22:48.133273+0000","last_active":"2024-09-16T07:22:50.278962+0000","last_peered":"2024-09-16T07:22:50.278962+0000","last_clean":"2024-09-16T07:22:50.278962+0000","last_became_active":"2024-09-16T07:22:48.132984+0000","last_became_peered":"2024-09-16T07:22:48.132984+0000","last_unstale":"2024-09-16T07:22:50.278962+0000","last_undegraded":"2024-09-16T07:22:50.278962+0000","last_fullsized":"2024-09-16T07:22:50.278962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.657246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279239+0000","last_change":"2024-09-16T07:22:49.134936+0000","last_active":"2024-09-16T07:22:50.279239+0000","last_peered":"2024-09-16T07:22:50.279239+0000","last_clean":"2024-09-16T07:22:50.279239+0000","last_became_active":"2024-09-16T07:22:49.134575+0000","last_became_peered":"2024-09-16T07:22:49.134575+0000","last_unstale":"2024-09-16T07:22:50.279239+0000","last_undegraded":"2024-09-16T07:22:50.279239+0000","last_fullsized":"2024-09-16T07:22:50.279239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279457+0000","last_change":"2024-09-16T07:22:48.140454+0000","last_active":"2024-09-16T07:22:50.279457+0000","last_peered":"2024-09-16T07:22:50.279457+0000","last_clean":"2024-09-16T07:22:50.279457+0000","last_became_active":"2024-09-16T07:22:48.140025+0000","last_became_peered":"2024-09-16T07:22:48.140025+0000","last_unstale":"2024-09-16T07:22:50.279457+0000","last_undegraded":"2024-09-16T07:22:50.279457+0000","last_fullsized":"2024-09-16T07:22:50.279457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:38.086419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130545+0000","last_change":"2024-09-16T07:22:49.138294+0000","last_active":"2024-09-16T07:22:50.130545+0000","last_peered":"2024-09-16T07:22:50.130545+0000","last_clean":"2024-09-16T07:22:50.130545+0000","last_became_active":"2024-09-16T07:22:49.137685+0000","last_became_peered":"2024-09-16T07:22:49.137685+0000","last_unstale":"2024-09-16T07:22:50.130545+0000","last_undegraded":"2024-09-16T07:22:50.130545+0000","last_fullsized":"2024-09-16T07:22:50.130545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:51.068492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279367+0000","last_change":"2024-09-16T07:22:48.139380+0000","last_active":"2024-09-16T07:22:50.279367+0000","last_peered":"2024-09-16T07:22:50.279367+0000","last_clean":"2024-09-16T07:22:50.279367+0000","last_became_active":"2024-09-16T07:22:48.138052+0000","last_became_peered":"2024-09-16T07:22:48.138052+0000","last_unstale":"2024-09-16T07:22:50.279367+0000","last_undegraded":"2024-09-16T07:22:50.279367+0000","last_fullsized":"2024-09-16T07:22:50.279367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:28.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139289+0000","last_change":"2024-09-16T07:22:49.133663+0000","last_active":"2024-09-16T07:22:50.139289+0000","last_peered":"2024-09-16T07:22:50.139289+0000","last_clean":"2024-09-16T07:22:50.139289+0000","last_became_active":"2024-09-16T07:22:49.133342+0000","last_became_peered":"2024-09-16T07:22:49.133342+0000","last_unstale":"2024-09-16T07:22:50.139289+0000","last_undegraded":"2024-09-16T07:22:50.139289+0000","last_fullsized":"2024-09-16T07:22:50.139289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:04.490402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130957+0000","last_change":"2024-09-16T07:22:48.141538+0000","last_active":"2024-09-16T07:22:50.130957+0000","last_peered":"2024-09-16T07:22:50.130957+0000","last_clean":"2024-09-16T07:22:50.130957+0000","last_became_active":"2024-09-16T07:22:48.138174+0000","last_became_peered":"2024-09-16T07:22:48.138174+0000","last_unstale":"2024-09-16T07:22:50.130957+0000","last_undegraded":"2024-09-16T07:22:50.130957+0000","last_fullsized":"2024-09-16T07:22:50.130957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:29.651356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132217+0000","last_change":"2024-09-16T07:22:49.136727+0000","last_active":"2024-09-16T07:22:50.132217+0000","last_peered":"2024-09-16T07:22:50.132217+0000","last_clean":"2024-09-16T07:22:50.132217+0000","last_became_active":"2024-09-16T07:22:49.136064+0000","last_became_peered":"2024-09-16T07:22:49.136064+0000","last_unstale":"2024-09-16T07:22:50.132217+0000","last_undegraded":"2024-09-16T07:22:50.132217+0000","last_fullsized":"2024-09-16T07:22:50.132217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:07.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279830+0000","last_change":"2024-09-16T07:22:48.135598+0000","last_active":"2024-09-16T07:22:50.279830+0000","last_peered":"2024-09-16T07:22:50.279830+0000","last_clean":"2024-09-16T07:22:50.279830+0000","last_became_active":"2024-09-16T07:22:48.135179+0000","last_became_peered":"2024-09-16T07:22:48.135179+0000","last_unstale":"2024-09-16T07:22:50.279830+0000","last_undegraded":"2024-09-16T07:22:50.279830+0000","last_fullsized":"2024-09-16T07:22:50.279830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:09.846796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280111+0000","last_change":"2024-09-16T07:22:49.137714+0000","last_active":"2024-09-16T07:22:50.280111+0000","last_peered":"2024-09-16T07:22:50.280111+0000","last_clean":"2024-09-16T07:22:50.280111+0000","last_became_active":"2024-09-16T07:22:49.134740+0000","last_became_peered":"2024-09-16T07:22:49.134740+0000","last_unstale":"2024-09-16T07:22:50.280111+0000","last_undegraded":"2024-09-16T07:22:50.280111+0000","last_fullsized":"2024-09-16T07:22:50.280111+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:44.194572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138704+0000","last_change":"2024-09-16T07:22:48.137476+0000","last_active":"2024-09-16T07:22:50.138704+0000","last_peered":"2024-09-16T07:22:50.138704+0000","last_clean":"2024-09-16T07:22:50.138704+0000","last_became_active":"2024-09-16T07:22:48.136772+0000","last_became_peered":"2024-09-16T07:22:48.136772+0000","last_unstale":"2024-09-16T07:22:50.138704+0000","last_undegraded":"2024-09-16T07:22:50.138704+0000","last_fullsized":"2024-09-16T07:22:50.138704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.906034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132443+0000","last_change":"2024-09-16T07:22:49.144331+0000","last_active":"2024-09-16T07:22:50.132443+0000","last_peered":"2024-09-16T07:22:50.132443+0000","last_clean":"2024-09-16T07:22:50.132443+0000","last_became_active":"2024-09-16T07:22:49.144071+0000","last_became_peered":"2024-09-16T07:22:49.144071+0000","last_unstale":"2024-09-16T07:22:50.132443+0000","last_undegraded":"2024-09-16T07:22:50.132443+0000","last_fullsized":"2024-09-16T07:22:50.132443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:57.753793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280211+0000","last_change":"2024-09-16T07:22:48.132847+0000","last_active":"2024-09-16T07:22:50.280211+0000","last_peered":"2024-09-16T07:22:50.280211+0000","last_clean":"2024-09-16T07:22:50.280211+0000","last_became_active":"2024-09-16T07:22:48.132264+0000","last_became_peered":"2024-09-16T07:22:48.132264+0000","last_unstale":"2024-09-16T07:22:50.280211+0000","last_undegraded":"2024-09-16T07:22:50.280211+0000","last_fullsized":"2024-09-16T07:22:50.280211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:42.680089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:49.138949+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:49.138580+0000","last_became_peered":"2024-09-16T07:22:49.138580+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:30.446387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.138995+0000","last_change":"2024-09-16T07:22:48.129412+0000","last_active":"2024-09-16T07:22:50.138995+0000","last_peered":"2024-09-16T07:22:50.138995+0000","last_clean":"2024-09-16T07:22:50.138995+0000","last_became_active":"2024-09-16T07:22:48.129137+0000","last_became_peered":"2024-09-16T07:22:48.129137+0000","last_unstale":"2024-09-16T07:22:50.138995+0000","last_undegraded":"2024-09-16T07:22:50.138995+0000","last_fullsized":"2024-09-16T07:22:50.138995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:07.509813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279051+0000","last_change":"2024-09-16T07:22:49.129827+0000","last_active":"2024-09-16T07:22:50.279051+0000","last_peered":"2024-09-16T07:22:50.279051+0000","last_clean":"2024-09-16T07:22:50.279051+0000","last_became_active":"2024-09-16T07:22:49.129474+0000","last_became_peered":"2024-09-16T07:22:49.129474+0000","last_unstale":"2024-09-16T07:22:50.279051+0000","last_undegraded":"2024-09-16T07:22:50.279051+0000","last_fullsized":"2024-09-16T07:22:50.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:57.650915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130669+0000","last_change":"2024-09-16T07:22:48.134437+0000","last_active":"2024-09-16T07:22:50.130669+0000","last_peered":"2024-09-16T07:22:50.130669+0000","last_clean":"2024-09-16T07:22:50.130669+0000","last_became_active":"2024-09-16T07:22:48.134144+0000","last_became_peered":"2024-09-16T07:22:48.134144+0000","last_unstale":"2024-09-16T07:22:50.130669+0000","last_undegraded":"2024-09-16T07:22:50.130669+0000","last_fullsized":"2024-09-16T07:22:50.130669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:04.051560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.130660+0000","last_change":"2024-09-16T07:22:49.141869+0000","last_active":"2024-09-16T07:22:50.130660+0000","last_peered":"2024-09-16T07:22:50.130660+0000","last_clean":"2024-09-16T07:22:50.130660+0000","last_became_active":"2024-09-16T07:22:49.141602+0000","last_became_peered":"2024-09-16T07:22:49.141602+0000","last_unstale":"2024-09-16T07:22:50.130660+0000","last_undegraded":"2024-09-16T07:22:50.130660+0000","last_fullsized":"2024-09-16T07:22:50.130660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:44.213115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279669+0000","last_change":"2024-09-16T07:22:48.137499+0000","last_active":"2024-09-16T07:22:50.279669+0000","last_peered":"2024-09-16T07:22:50.279669+0000","last_clean":"2024-09-16T07:22:50.279669+0000","last_became_active":"2024-09-16T07:22:48.137177+0000","last_became_peered":"2024-09-16T07:22:48.137177+0000","last_unstale":"2024-09-16T07:22:50.279669+0000","last_undegraded":"2024-09-16T07:22:50.279669+0000","last_fullsized":"2024-09-16T07:22:50.279669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:49.997304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.280080+0000","last_change":"2024-09-16T07:22:49.143083+0000","last_active":"2024-09-16T07:22:50.280080+0000","last_peered":"2024-09-16T07:22:50.280080+0000","last_clean":"2024-09-16T07:22:50.280080+0000","last_became_active":"2024-09-16T07:22:49.142817+0000","last_became_peered":"2024-09-16T07:22:49.142817+0000","last_unstale":"2024-09-16T07:22:50.280080+0000","last_undegraded":"2024-09-16T07:22:50.280080+0000","last_fullsized":"2024-09-16T07:22:50.280080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:04.662896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279682+0000","last_change":"2024-09-16T07:22:48.131412+0000","last_active":"2024-09-16T07:22:50.279682+0000","last_peered":"2024-09-16T07:22:50.279682+0000","last_clean":"2024-09-16T07:22:50.279682+0000","last_became_active":"2024-09-16T07:22:48.131167+0000","last_became_peered":"2024-09-16T07:22:48.131167+0000","last_unstale":"2024-09-16T07:22:50.279682+0000","last_undegraded":"2024-09-16T07:22:50.279682+0000","last_fullsized":"2024-09-16T07:22:50.279682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:40.642232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279172+0000","last_change":"2024-09-16T07:22:49.129812+0000","last_active":"2024-09-16T07:22:50.279172+0000","last_peered":"2024-09-16T07:22:50.279172+0000","last_clean":"2024-09-16T07:22:50.279172+0000","last_became_active":"2024-09-16T07:22:49.129511+0000","last_became_peered":"2024-09-16T07:22:49.129511+0000","last_unstale":"2024-09-16T07:22:50.279172+0000","last_undegraded":"2024-09-16T07:22:50.279172+0000","last_fullsized":"2024-09-16T07:22:50.279172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:12.678514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279340+0000","last_change":"2024-09-16T07:22:49.137458+0000","last_active":"2024-09-16T07:22:50.279340+0000","last_peered":"2024-09-16T07:22:50.279340+0000","last_clean":"2024-09-16T07:22:50.279340+0000","last_became_active":"2024-09-16T07:22:49.137122+0000","last_became_peered":"2024-09-16T07:22:49.137122+0000","last_unstale":"2024-09-16T07:22:50.279340+0000","last_undegraded":"2024-09-16T07:22:50.279340+0000","last_fullsized":"2024-09-16T07:22:50.279340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:52.744207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139907+0000","last_change":"2024-09-16T07:22:48.130394+0000","last_active":"2024-09-16T07:22:50.139907+0000","last_peered":"2024-09-16T07:22:50.139907+0000","last_clean":"2024-09-16T07:22:50.139907+0000","last_became_active":"2024-09-16T07:22:48.130013+0000","last_became_peered":"2024-09-16T07:22:48.130013+0000","last_unstale":"2024-09-16T07:22:50.139907+0000","last_undegraded":"2024-09-16T07:22:50.139907+0000","last_fullsized":"2024-09-16T07:22:50.139907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:03.657246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.279527+0000","last_change":"2024-09-16T07:22:49.136594+0000","last_active":"2024-09-16T07:22:50.279527+0000","last_peered":"2024-09-16T07:22:50.279527+0000","last_clean":"2024-09-16T07:22:50.279527+0000","last_became_active":"2024-09-16T07:22:49.136266+0000","last_became_peered":"2024-09-16T07:22:49.136266+0000","last_unstale":"2024-09-16T07:22:50.279527+0000","last_undegraded":"2024-09-16T07:22:50.279527+0000","last_fullsized":"2024-09-16T07:22:50.279527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:10.852446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.139950+0000","last_change":"2024-09-16T07:22:48.135403+0000","last_active":"2024-09-16T07:22:50.139950+0000","last_peered":"2024-09-16T07:22:50.139950+0000","last_clean":"2024-09-16T07:22:50.139950+0000","last_became_active":"2024-09-16T07:22:48.135142+0000","last_became_peered":"2024-09-16T07:22:48.135142+0000","last_unstale":"2024-09-16T07:22:50.139950+0000","last_undegraded":"2024-09-16T07:22:50.139950+0000","last_fullsized":"2024-09-16T07:22:50.139950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:58.988982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.133007+0000","last_change":"2024-09-16T07:22:49.129289+0000","last_active":"2024-09-16T07:22:50.133007+0000","last_peered":"2024-09-16T07:22:50.133007+0000","last_clean":"2024-09-16T07:22:50.133007+0000","last_became_active":"2024-09-16T07:22:49.129000+0000","last_became_peered":"2024-09-16T07:22:49.129000+0000","last_unstale":"2024-09-16T07:22:50.133007+0000","last_undegraded":"2024-09-16T07:22:50.133007+0000","last_fullsized":"2024-09-16T07:22:50.133007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:31.607464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132559+0000","last_change":"2024-09-16T07:22:48.132860+0000","last_active":"2024-09-16T07:22:50.132559+0000","last_peered":"2024-09-16T07:22:50.132559+0000","last_clean":"2024-09-16T07:22:50.132559+0000","last_became_active":"2024-09-16T07:22:48.132544+0000","last_became_peered":"2024-09-16T07:22:48.132544+0000","last_unstale":"2024-09-16T07:22:50.132559+0000","last_undegraded":"2024-09-16T07:22:50.132559+0000","last_fullsized":"2024-09-16T07:22:50.132559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:52:05.258261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132482+0000","last_change":"2024-09-16T07:22:49.142356+0000","last_active":"2024-09-16T07:22:50.132482+0000","last_peered":"2024-09-16T07:22:50.132482+0000","last_clean":"2024-09-16T07:22:50.132482+0000","last_became_active":"2024-09-16T07:22:49.141639+0000","last_became_peered":"2024-09-16T07:22:49.141639+0000","last_unstale":"2024-09-16T07:22:50.132482+0000","last_undegraded":"2024-09-16T07:22:50.132482+0000","last_fullsized":"2024-09-16T07:22:50.132482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:48.099589+0000","last_clean_scrub_stamp":"2024-09-16T07:22:48.099589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:57.949375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T07:22:50.132622+0000","last_change":"2024-09-16T07:22:48.141242+0000","last_active":"2024-09-16T07:22:50.132622+0000","last_peered":"2024-09-16T07:22:50.132622+0000","last_clean":"2024-09-16T07:22:50.132622+0000","last_became_active":"2024-09-16T07:22:48.139871+0000","last_became_peered":"2024-09-16T07:22:48.139871+0000","last_unstale":"2024-09-16T07:22:50.132622+0000","last_undegraded":"2024-09-16T07:22:50.132622+0000","last_fullsized":"2024-09-16T07:22:50.132622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:47.093938+0000","last_clean_scrub_stamp":"2024-09-16T07:22:47.093938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:32.916879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476745,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":81289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476745,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476745,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":671145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476744,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":80763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":80831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":80707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":831488,"data_stored":671167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:23:03.806 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:23:03.806 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T07:23:03.807 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T07:23:03.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T07:23:04.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.998+0000 7fb2d4e00640 1 -- 172.21.15.71:0/3080637287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d0151cd0 msgr2=0x7fb2d015e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.998+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/3080637287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d0151cd0 0x7fb2d015e6f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005fd0 tx=0x7fb2bc02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.999+0000 7fb2d4e00640 1 -- 172.21.15.71:0/3080637287 shutdown_connections 2024-09-16T07:23:04.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.999+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/3080637287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d0151cd0 0x7fb2d015e6f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.999+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/3080637287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d01518f0 0x7fb2d00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.999+0000 7fb2d4e00640 1 -- 172.21.15.71:0/3080637287 >> 172.21.15.71:0/3080637287 conn(0x7fb2d0069560 msgr2=0x7fb2d006cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:04.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.999+0000 7fb2d4e00640 1 -- 172.21.15.71:0/3080637287 shutdown_connections 2024-09-16T07:23:04.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:03.999+0000 7fb2d4e00640 1 -- 172.21.15.71:0/3080637287 wait complete. 2024-09-16T07:23:04.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.000+0000 7fb2d4e00640 1 Processor -- start 2024-09-16T07:23:04.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.000+0000 7fb2d4e00640 1 -- start start 2024-09-16T07:23:04.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.001+0000 7fb2d4e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d01518f0 0x7fb2d014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.001+0000 7fb2d4e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d0151cd0 0x7fb2d00cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.001+0000 7fb2d4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d015ec30 0x7fb2d00cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.001+0000 7fb2d4e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d015bbc0 con 0x7fb2d015ec30 2024-09-16T07:23:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.001+0000 7fb2d4e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d015ba40 con 0x7fb2d01518f0 2024-09-16T07:23:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.001+0000 7fb2d4e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d015bd40 con 0x7fb2d0151cd0 2024-09-16T07:23:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d015ec30 0x7fb2d00cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d015ec30 0x7fb2d00cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51012/0 (socket says 172.21.15.71:51012) 2024-09-16T07:23:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 -- 172.21.15.71:0/640932392 learned_addr learned my addr 172.21.15.71:0/640932392 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2cde00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d01518f0 0x7fb2d014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2cd400640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d0151cd0 0x7fb2d00cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 -- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d0151cd0 msgr2=0x7fb2d00cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d0151cd0 0x7fb2d00cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 -- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d01518f0 msgr2=0x7fb2d014af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d01518f0 0x7fb2d014af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 -- 172.21.15.71:0/640932392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d014b4d0 con 0x7fb2d015ec30 2024-09-16T07:23:04.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2cd400640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d0151cd0 0x7fb2d00cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:04.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2ce800640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d015ec30 0x7fb2d00cdf10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800c6b0 tx=0x7fb2c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:04.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2c6000640 1 -- 172.21.15.71:0/640932392 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8003b70 con 0x7fb2d015ec30 2024-09-16T07:23:04.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.002+0000 7fb2c6000640 1 -- 172.21.15.71:0/640932392 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2c8003d30 con 0x7fb2d015ec30 2024-09-16T07:23:04.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.003+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d01f5bb0 con 0x7fb2d015ec30 2024-09-16T07:23:04.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.003+0000 7fb2c6000640 1 -- 172.21.15.71:0/640932392 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8005330 con 0x7fb2d015ec30 2024-09-16T07:23:04.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.003+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d01f4df0 con 0x7fb2d015ec30 2024-09-16T07:23:04.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.004+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d00b3960 con 0x7fb2d015ec30 2024-09-16T07:23:04.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.005+0000 7fb2c6000640 1 -- 172.21.15.71:0/640932392 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb2c80054d0 con 0x7fb2d015ec30 2024-09-16T07:23:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.006+0000 7fb2c6000640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2b0078850 0x7fb2b007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.006+0000 7fb2c6000640 1 -- 172.21.15.71:0/640932392 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb2c809a650 con 0x7fb2d015ec30 2024-09-16T07:23:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.006+0000 7fb2cde00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2b0078850 0x7fb2b007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.007+0000 7fb2cde00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2b0078850 0x7fb2b007ad10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009bd0 tx=0x7fb2c0005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:04.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.010+0000 7fb2c6000640 1 -- 172.21.15.71:0/640932392 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb2d014b4d0 con 0x7fb2d015ec30 2024-09-16T07:23:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.234+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb2d00d1210 con 0x7fb2d015ec30 2024-09-16T07:23:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.235+0000 7fb2c6000640 1 -- 172.21.15.71:0/640932392 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb2c8062cc0 con 0x7fb2d015ec30 2024-09-16T07:23:04.235 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:04.235 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2b0078850 msgr2=0x7fb2b007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2b0078850 0x7fb2b007ad10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009bd0 tx=0x7fb2c0005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d015ec30 msgr2=0x7fb2d00cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d015ec30 0x7fb2d00cdf10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800c6b0 tx=0x7fb2c800cb70 comp rx=0 tx=0).stop 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 shutdown_connections 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2b0078850 0x7fb2b007ad10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2d015ec30 0x7fb2d00cdf10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d0151cd0 0x7fb2d00cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 --2- 172.21.15.71:0/640932392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d01518f0 0x7fb2d014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 >> 172.21.15.71:0/640932392 conn(0x7fb2d0069560 msgr2=0x7fb2d015d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:04.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 shutdown_connections 2024-09-16T07:23:04.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.237+0000 7fb2d4e00640 1 -- 172.21.15.71:0/640932392 wait complete. 2024-09-16T07:23:04.253 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T07:23:04.254 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:04.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.411+0000 7f3b49400640 1 -- 172.21.15.71:0/1519091717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 msgr2=0x7f3b2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.411+0000 7f3b49400640 1 --2- 172.21.15.71:0/1519091717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 0x7f3b2800b9c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c001d90 tx=0x7f3b28005830 comp rx=0 tx=0).stop 2024-09-16T07:23:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.412+0000 7f3b49400640 1 -- 172.21.15.71:0/1519091717 shutdown_connections 2024-09-16T07:23:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.412+0000 7f3b49400640 1 --2- 172.21.15.71:0/1519091717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 0x7f3b2800b9c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.412+0000 7f3b49400640 1 -- 172.21.15.71:0/1519091717 >> 172.21.15.71:0/1519091717 conn(0x7f3b440b1950 msgr2=0x7f3b440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.412+0000 7f3b49400640 1 -- 172.21.15.71:0/1519091717 shutdown_connections 2024-09-16T07:23:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.412+0000 7f3b49400640 1 -- 172.21.15.71:0/1519091717 wait complete. 2024-09-16T07:23:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.413+0000 7f3b49400640 1 Processor -- start 2024-09-16T07:23:04.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.413+0000 7f3b49400640 1 -- start start 2024-09-16T07:23:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b49400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 0x7f3b441f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b49400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b44144200 0x7f3b44152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b49400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440ae150 0x7f3b440ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b49400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44074420 con 0x7f3b280052e0 2024-09-16T07:23:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b49400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3b440b2700 con 0x7f3b440ae150 2024-09-16T07:23:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b49400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3b440745a0 con 0x7f3b44144200 2024-09-16T07:23:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b42800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 0x7f3b441f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b42800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 0x7f3b441f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51026/0 (socket says 172.21.15.71:51026) 2024-09-16T07:23:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.414+0000 7f3b42800640 1 -- 172.21.15.71:0/1479866904 learned_addr learned my addr 172.21.15.71:0/1479866904 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b3be00640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b44144200 0x7f3b44152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b43200640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440ae150 0x7f3b440ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b42800640 1 -- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b44144200 msgr2=0x7f3b44152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b42800640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b44144200 0x7f3b44152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b42800640 1 -- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440ae150 msgr2=0x7f3b440ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b42800640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440ae150 0x7f3b440ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b42800640 1 -- 172.21.15.71:0/1479866904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b44150a70 con 0x7f3b280052e0 2024-09-16T07:23:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b42800640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 0x7f3b441f5d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b340066e0 tx=0x7f3b340039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b3aa00640 1 -- 172.21.15.71:0/1479866904 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400f760 con 0x7f3b280052e0 2024-09-16T07:23:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b3aa00640 1 -- 172.21.15.71:0/1479866904 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b34016020 con 0x7f3b280052e0 2024-09-16T07:23:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b3aa00640 1 -- 172.21.15.71:0/1479866904 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400fba0 con 0x7f3b280052e0 2024-09-16T07:23:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b441f6360 con 0x7f3b280052e0 2024-09-16T07:23:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.415+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441f6920 con 0x7f3b280052e0 2024-09-16T07:23:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.417+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b440acd30 con 0x7f3b280052e0 2024-09-16T07:23:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.418+0000 7f3b3aa00640 1 -- 172.21.15.71:0/1479866904 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f3b3400fd40 con 0x7f3b280052e0 2024-09-16T07:23:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.418+0000 7f3b3aa00640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b20078880 0x7f3b2007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.419+0000 7f3b3aa00640 1 -- 172.21.15.71:0/1479866904 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3b3409b3a0 con 0x7f3b280052e0 2024-09-16T07:23:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.419+0000 7f3b3be00640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b20078880 0x7f3b2007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.419+0000 7f3b3be00640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b20078880 0x7f3b2007ad40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b44150040 tx=0x7f3b2c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:04.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.423+0000 7f3b3aa00640 1 -- 172.21.15.71:0/1479866904 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b34020020 con 0x7f3b280052e0 2024-09-16T07:23:04.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.643+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b44154bb0 con 0x7f3b280052e0 2024-09-16T07:23:04.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.644+0000 7f3b3aa00640 1 -- 172.21.15.71:0/1479866904 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f3b34063aa0 con 0x7f3b280052e0 2024-09-16T07:23:04.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:04.645 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:22:48.541520+0000","modified":"2024-09-16T07:22:51.551572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4404":{"gid":4404,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4404,"qdb_cluster":[4404]},"id":1}]} 2024-09-16T07:23:04.645 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-09-16T07:23:04.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b20078880 msgr2=0x7f3b2007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b20078880 0x7f3b2007ad40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b44150040 tx=0x7f3b2c005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:04.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 msgr2=0x7f3b441f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 0x7f3b441f5d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b340066e0 tx=0x7f3b340039c0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 shutdown_connections 2024-09-16T07:23:04.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b20078880 0x7f3b2007ad40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440ae150 0x7f3b440ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b44144200 0x7f3b44152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 --2- 172.21.15.71:0/1479866904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b280052e0 0x7f3b441f5d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.646+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 >> 172.21.15.71:0/1479866904 conn(0x7f3b440b1950 msgr2=0x7f3b4409ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:04.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.647+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 shutdown_connections 2024-09-16T07:23:04.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.647+0000 7f3b49400640 1 -- 172.21.15.71:0/1479866904 wait complete. 2024-09-16T07:23:04.663 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:04.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.817+0000 7fcb81600640 1 -- 172.21.15.71:0/4202940292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0725d0 msgr2=0x7fcb7c073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.818+0000 7fcb81600640 1 --2- 172.21.15.71:0/4202940292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0725d0 0x7fcb7c073d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0cd570 tx=0x7fcb6802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.818+0000 7fcb81600640 1 -- 172.21.15.71:0/4202940292 shutdown_connections 2024-09-16T07:23:04.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.818+0000 7fcb81600640 1 --2- 172.21.15.71:0/4202940292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0725d0 0x7fcb7c073d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.818+0000 7fcb81600640 1 --2- 172.21.15.71:0/4202940292 >> v2:172.21.15.177:3300/0 conn(0x7fcb7c15bc30 0x7fcb7c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.818+0000 7fcb81600640 1 -- 172.21.15.71:0/4202940292 >> 172.21.15.71:0/4202940292 conn(0x7fcb7c0cfa20 msgr2=0x7fcb7c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:04.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.818+0000 7fcb81600640 1 -- 172.21.15.71:0/4202940292 shutdown_connections 2024-09-16T07:23:04.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.818+0000 7fcb81600640 1 -- 172.21.15.71:0/4202940292 wait complete. 2024-09-16T07:23:04.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.819+0000 7fcb81600640 1 Processor -- start 2024-09-16T07:23:04.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.819+0000 7fcb81600640 1 -- start start 2024-09-16T07:23:04.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.820+0000 7fcb81600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0d0540 0x7fcb7c1f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.820+0000 7fcb81600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb7c15bc30 0x7fcb7c151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.820+0000 7fcb81600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb7c155050 0x7fcb7c1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.820+0000 7fcb81600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c074610 con 0x7fcb7c0d0540 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.820+0000 7fcb81600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c074490 con 0x7fcb7c155050 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.820+0000 7fcb81600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcb7c074790 con 0x7fcb7c15bc30 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0d0540 0x7fcb7c1f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0d0540 0x7fcb7c1f8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36540/0 (socket says 172.21.15.71:36540) 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 -- 172.21.15.71:0/2655000762 learned_addr learned my addr 172.21.15.71:0/2655000762 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb80c00640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb7c155050 0x7fcb7c1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7b400640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb7c15bc30 0x7fcb7c151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 -- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb7c15bc30 msgr2=0x7fcb7c151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb7c15bc30 0x7fcb7c151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 -- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb7c155050 msgr2=0x7fcb7c1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb7c155050 0x7fcb7c1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 -- 172.21.15.71:0/2655000762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c1f8e90 con 0x7fcb7c0d0540 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7b400640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb7c15bc30 0x7fcb7c151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb7be00640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0d0540 0x7fcb7c1f8950 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00e7c0 tx=0x7fcb6c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb73e00640 1 -- 172.21.15.71:0/2655000762 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00ccc0 con 0x7fcb7c0d0540 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.821+0000 7fcb73e00640 1 -- 172.21.15.71:0/2655000762 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb6c00ce80 con 0x7fcb7c0d0540 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.822+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c0b0b10 con 0x7fcb7c0d0540 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.822+0000 7fcb73e00640 1 -- 172.21.15.71:0/2655000762 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c005280 con 0x7fcb7c0d0540 2024-09-16T07:23:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.822+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c153e60 con 0x7fcb7c0d0540 2024-09-16T07:23:04.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.823+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c154750 con 0x7fcb7c0d0540 2024-09-16T07:23:04.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.824+0000 7fcb73e00640 1 -- 172.21.15.71:0/2655000762 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fcb6c005420 con 0x7fcb7c0d0540 2024-09-16T07:23:04.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.825+0000 7fcb73e00640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcb5c078850 0x7fcb5c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:04.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.825+0000 7fcb73e00640 1 -- 172.21.15.71:0/2655000762 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcb6c09b250 con 0x7fcb7c0d0540 2024-09-16T07:23:04.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.825+0000 7fcb7b400640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcb5c078850 0x7fcb5c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:04.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.826+0000 7fcb7b400640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcb5c078850 0x7fcb5c07ad10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb68033000 tx=0x7fcb68004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:04.829+0000 7fcb73e00640 1 -- 172.21.15.71:0/2655000762 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcb7c1f8e90 con 0x7fcb7c0d0540 2024-09-16T07:23:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.051+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb7c06fd20 con 0x7fcb7c0d0540 2024-09-16T07:23:05.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.052+0000 7fcb73e00640 1 -- 172.21.15.71:0/2655000762 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7fcb6c063820 con 0x7fcb7c0d0540 2024-09-16T07:23:05.052 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:05.052 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:22:48.541520+0000","modified":"2024-09-16T07:22:51.551572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4404":{"gid":4404,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4404,"qdb_cluster":[4404]},"id":1}]} 2024-09-16T07:23:05.053 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-09-16T07:23:05.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcb5c078850 msgr2=0x7fcb5c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcb5c078850 0x7fcb5c07ad10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb68033000 tx=0x7fcb68004730 comp rx=0 tx=0).stop 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0d0540 msgr2=0x7fcb7c1f8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0d0540 0x7fcb7c1f8950 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00e7c0 tx=0x7fcb6c00ec80 comp rx=0 tx=0).stop 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 shutdown_connections 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcb5c078850 0x7fcb5c07ad10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb7c155050 0x7fcb7c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb7c15bc30 0x7fcb7c151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 --2- 172.21.15.71:0/2655000762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb7c0d0540 0x7fcb7c1f8950 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 >> 172.21.15.71:0/2655000762 conn(0x7fcb7c0cfa20 msgr2=0x7fcb7c157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 shutdown_connections 2024-09-16T07:23:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.054+0000 7fcb81600640 1 -- 172.21.15.71:0/2655000762 wait complete. 2024-09-16T07:23:05.070 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:23:05.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.228+0000 7f95f1800640 1 -- 172.21.15.71:0/2838410641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec1612d0 msgr2=0x7f95ec0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.228+0000 7f95f1800640 1 --2- 172.21.15.71:0/2838410641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec1612d0 0x7f95ec0b4e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0d1870 tx=0x7f95dc02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.228+0000 7f95f1800640 1 -- 172.21.15.71:0/2838410641 shutdown_connections 2024-09-16T07:23:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.228+0000 7f95f1800640 1 --2- 172.21.15.71:0/2838410641 >> v2:172.21.15.177:3300/0 conn(0x7f95ec1616b0 0x7f95ec0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.228+0000 7f95f1800640 1 --2- 172.21.15.71:0/2838410641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec1612d0 0x7f95ec0b4e00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.228+0000 7f95f1800640 1 -- 172.21.15.71:0/2838410641 >> 172.21.15.71:0/2838410641 conn(0x7f95ec0d3df0 msgr2=0x7f95ec0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.228+0000 7f95f1800640 1 -- 172.21.15.71:0/2838410641 shutdown_connections 2024-09-16T07:23:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.228+0000 7f95f1800640 1 -- 172.21.15.71:0/2838410641 wait complete. 2024-09-16T07:23:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.230+0000 7f95f1800640 1 Processor -- start 2024-09-16T07:23:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.230+0000 7f95f1800640 1 -- start start 2024-09-16T07:23:05.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95f1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec0d4980 0x7f95ec156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:05.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95f1800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ec1612d0 0x7f95ec15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:05.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95f1800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ec1616b0 0x7f95ec15e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:05.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95f1800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec0b1b60 con 0x7f95ec0d4980 2024-09-16T07:23:05.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95f1800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec0b19e0 con 0x7f95ec1616b0 2024-09-16T07:23:05.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95f1800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f95ec0b1ce0 con 0x7f95ec1612d0 2024-09-16T07:23:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95ebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec0d4980 0x7f95ec156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95ebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec0d4980 0x7f95ec156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36554/0 (socket says 172.21.15.71:36554) 2024-09-16T07:23:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.231+0000 7f95ebe00640 1 -- 172.21.15.71:0/3538623176 learned_addr learned my addr 172.21.15.71:0/3538623176 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95e3e00640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ec1612d0 0x7f95ec15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95f0e00640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ec1616b0 0x7f95ec15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95ebe00640 1 -- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ec1612d0 msgr2=0x7f95ec15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95ebe00640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ec1612d0 0x7f95ec15e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95ebe00640 1 -- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ec1616b0 msgr2=0x7f95ec15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95ebe00640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ec1616b0 0x7f95ec15e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95ebe00640 1 -- 172.21.15.71:0/3538623176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ec1f4fc0 con 0x7f95ec0d4980 2024-09-16T07:23:05.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95ebe00640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec0d4980 0x7f95ec156c00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009850 tx=0x7f95dc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:05.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95e8c00640 1 -- 172.21.15.71:0/3538623176 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc03f070 con 0x7f95ec0d4980 2024-09-16T07:23:05.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95e8c00640 1 -- 172.21.15.71:0/3538623176 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95dc004890 con 0x7f95ec0d4980 2024-09-16T07:23:05.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95e8c00640 1 -- 172.21.15.71:0/3538623176 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc035030 con 0x7f95ec0d4980 2024-09-16T07:23:05.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.232+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ec1f5260 con 0x7f95ec0d4980 2024-09-16T07:23:05.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.233+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ec1f5820 con 0x7f95ec0d4980 2024-09-16T07:23:05.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.234+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95ec0b4e00 con 0x7f95ec0d4980 2024-09-16T07:23:05.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.235+0000 7f95e8c00640 1 -- 172.21.15.71:0/3538623176 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f95dc042070 con 0x7f95ec0d4980 2024-09-16T07:23:05.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.236+0000 7f95e8c00640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95c8078850 0x7f95c807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:05.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.236+0000 7f95e8c00640 1 -- 172.21.15.71:0/3538623176 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f95dc0bb8f0 con 0x7f95ec0d4980 2024-09-16T07:23:05.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.236+0000 7f95e3e00640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95c8078850 0x7f95c807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:05.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.237+0000 7f95e3e00640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95c8078850 0x7f95c807ad10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95ec158170 tx=0x7f95d4007e30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:05.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.240+0000 7f95e8c00640 1 -- 172.21.15.71:0/3538623176 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f95ec0d2cc0 con 0x7f95ec0d4980 2024-09-16T07:23:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.427+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95ec15fb20 con 0x7f95ec0d4980 2024-09-16T07:23:05.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.428+0000 7f95e8c00640 1 -- 172.21.15.71:0/3538623176 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7f95dc083880 con 0x7f95ec0d4980 2024-09-16T07:23:05.429 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:05.429 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":23,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:50.122046+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:23:05.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95c8078850 msgr2=0x7f95c807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95c8078850 0x7f95c807ad10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95ec158170 tx=0x7f95d4007e30 comp rx=0 tx=0).stop 2024-09-16T07:23:05.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec0d4980 msgr2=0x7f95ec156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec0d4980 0x7f95ec156c00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009850 tx=0x7f95dc0039e0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 shutdown_connections 2024-09-16T07:23:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95c8078850 0x7f95c807ad10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ec1616b0 0x7f95ec15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ec1612d0 0x7f95ec15e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.430+0000 7f95f1800640 1 --2- 172.21.15.71:0/3538623176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95ec0d4980 0x7f95ec156c00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.431+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 >> 172.21.15.71:0/3538623176 conn(0x7f95ec0d3df0 msgr2=0x7f95ec06dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.431+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 shutdown_connections 2024-09-16T07:23:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.431+0000 7f95f1800640 1 -- 172.21.15.71:0/3538623176 wait complete. 2024-09-16T07:23:05.448 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:05.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.605+0000 7f2484e00640 1 -- 172.21.15.71:0/1737250248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2464003a30 msgr2=0x7f2464003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.605+0000 7f2484e00640 1 --2- 172.21.15.71:0/1737250248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2464003a30 0x7f2464003e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2470006fd0 tx=0x7f2464005830 comp rx=0 tx=0).stop 2024-09-16T07:23:05.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.605+0000 7f247fe00640 1 -- 172.21.15.71:0/1737250248 reap_dead start 2024-09-16T07:23:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.605+0000 7f2484e00640 1 -- 172.21.15.71:0/1737250248 shutdown_connections 2024-09-16T07:23:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.605+0000 7f2484e00640 1 -- 172.21.15.71:0/1737250248 >> 172.21.15.71:0/1737250248 conn(0x7f24800d3df0 msgr2=0x7f24800d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.606+0000 7f2484e00640 1 -- 172.21.15.71:0/1737250248 shutdown_connections 2024-09-16T07:23:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.606+0000 7f2484e00640 1 -- 172.21.15.71:0/1737250248 wait complete. 2024-09-16T07:23:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.607+0000 7f2484e00640 1 Processor -- start 2024-09-16T07:23:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.607+0000 7f2484e00640 1 -- start start 2024-09-16T07:23:05.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f2484e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2464003a30 0x7f2480155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:05.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f2484e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248015c200 0x7f24801f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:05.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f2484e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24800d0780 0x7f24800b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:05.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f2484e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2480166da0 con 0x7f24800d0780 2024-09-16T07:23:05.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f2484e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2480166c20 con 0x7f248015c200 2024-09-16T07:23:05.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f2484e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2480166f20 con 0x7f2464003a30 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f247fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24800d0780 0x7f24800b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f247fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24800d0780 0x7f24800b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36570/0 (socket says 172.21.15.71:36570) 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f247fe00640 1 -- 172.21.15.71:0/2280161284 learned_addr learned my addr 172.21.15.71:0/2280161284 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f247f400640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2464003a30 0x7f2480155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.608+0000 7f247ea00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248015c200 0x7f24801f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f247fe00640 1 -- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2464003a30 msgr2=0x7f2480155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f247fe00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2464003a30 0x7f2480155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f247fe00640 1 -- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248015c200 msgr2=0x7f24801f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f247fe00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248015c200 0x7f24801f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f247fe00640 1 -- 172.21.15.71:0/2280161284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24801f5900 con 0x7f24800d0780 2024-09-16T07:23:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f247fe00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24800d0780 0x7f24800b36c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f24800b6990 tx=0x7f2478009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:05.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f2477400640 1 -- 172.21.15.71:0/2280161284 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478011910 con 0x7f24800d0780 2024-09-16T07:23:05.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f2477400640 1 -- 172.21.15.71:0/2280161284 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2478019070 con 0x7f24800d0780 2024-09-16T07:23:05.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248015c7c0 con 0x7f24800d0780 2024-09-16T07:23:05.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f2477400640 1 -- 172.21.15.71:0/2280161284 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478011e60 con 0x7f24800d0780 2024-09-16T07:23:05.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.609+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24801f4bc0 con 0x7f24800d0780 2024-09-16T07:23:05.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.611+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24800b34b0 con 0x7f24800d0780 2024-09-16T07:23:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.612+0000 7f2477400640 1 -- 172.21.15.71:0/2280161284 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f247801e460 con 0x7f24800d0780 2024-09-16T07:23:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.612+0000 7f2477400640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2460078880 0x7f246007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.613+0000 7f2477400640 1 -- 172.21.15.71:0/2280161284 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f247809af10 con 0x7f24800d0780 2024-09-16T07:23:05.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.613+0000 7f247f400640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2460078880 0x7f246007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:05.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.614+0000 7f247f400640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2460078880 0x7f246007ad40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2470033000 tx=0x7f2470038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:05.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.617+0000 7f2477400640 1 -- 172.21.15.71:0/2280161284 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2478062f30 con 0x7f24800d0780 2024-09-16T07:23:05.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.837+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24800ce230 con 0x7f24800d0780 2024-09-16T07:23:05.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.838+0000 7f2477400640 1 -- 172.21.15.71:0/2280161284 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f2478067960 con 0x7f24800d0780 2024-09-16T07:23:05.839 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:05.839 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:22:48.541520+0000","modified":"2024-09-16T07:22:51.551572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4404":{"gid":4404,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4404,"qdb_cluster":[4404]},"id":1}]} 2024-09-16T07:23:05.839 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-09-16T07:23:05.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2460078880 msgr2=0x7f246007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2460078880 0x7f246007ad40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2470033000 tx=0x7f2470038040 comp rx=0 tx=0).stop 2024-09-16T07:23:05.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24800d0780 msgr2=0x7f24800b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24800d0780 0x7f24800b36c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f24800b6990 tx=0x7f2478009b00 comp rx=0 tx=0).stop 2024-09-16T07:23:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 shutdown_connections 2024-09-16T07:23:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2460078880 0x7f246007ad40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24800d0780 0x7f24800b36c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248015c200 0x7f24801f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 --2- 172.21.15.71:0/2280161284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2464003a30 0x7f2480155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 >> 172.21.15.71:0/2280161284 conn(0x7f24800d3df0 msgr2=0x7f248015dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 shutdown_connections 2024-09-16T07:23:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:05.840+0000 7f2484e00640 1 -- 172.21.15.71:0/2280161284 wait complete. 2024-09-16T07:23:06.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:22:48.541520+0000', 'modified': '2024-09-16T07:22:51.551572+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4404}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4404': {'gid': 4404, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6837/1387172624', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 1387172624}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 1387172624}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4404, 'qdb_cluster': [4404]} 2024-09-16T07:23:06.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:23:06.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-16T07:23:06.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.419+0000 7f2ec7c00640 1 -- 172.21.15.71:0/1878644095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec015f5a0 msgr2=0x7f2ec00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.419+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/1878644095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec015f5a0 0x7f2ec00b2440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4005fd0 tx=0x7f2eb402a540 comp rx=0 tx=0).stop 2024-09-16T07:23:06.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.419+0000 7f2ec7c00640 1 -- 172.21.15.71:0/1878644095 shutdown_connections 2024-09-16T07:23:06.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.419+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/1878644095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec0073fb0 0x7f2ec0074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.419+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/1878644095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec015f5a0 0x7f2ec00b2440 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.419+0000 7f2ec7c00640 1 -- 172.21.15.71:0/1878644095 >> 172.21.15.71:0/1878644095 conn(0x7f2ec00b1950 msgr2=0x7f2ec00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:06.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.420+0000 7f2ec7c00640 1 -- 172.21.15.71:0/1878644095 shutdown_connections 2024-09-16T07:23:06.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.420+0000 7f2ec7c00640 1 -- 172.21.15.71:0/1878644095 wait complete. 2024-09-16T07:23:06.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.421+0000 7f2ec7c00640 1 Processor -- start 2024-09-16T07:23:06.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.421+0000 7f2ec7c00640 1 -- start start 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.422+0000 7f2ec7c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec0073fb0 0x7f2ec014e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.422+0000 7f2ec7c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec00ccf00 0x7f2ec00cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.422+0000 7f2ec7c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec015f5a0 0x7f2ec00cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.422+0000 7f2ec7c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec006f2e0 con 0x7f2ec015f5a0 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.422+0000 7f2ec7c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec006f160 con 0x7f2ec00ccf00 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.422+0000 7f2ec7c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec006f460 con 0x7f2ec0073fb0 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.422+0000 7f2ec5c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec015f5a0 0x7f2ec00cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec015f5a0 0x7f2ec00cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36586/0 (socket says 172.21.15.71:36586) 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5c00640 1 -- 172.21.15.71:0/340676127 learned_addr learned my addr 172.21.15.71:0/340676127 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:06.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5200640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec0073fb0 0x7f2ec014e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ebfe00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec00ccf00 0x7f2ec00cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5c00640 1 -- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec0073fb0 msgr2=0x7f2ec014e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec0073fb0 0x7f2ec014e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5c00640 1 -- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec00ccf00 msgr2=0x7f2ec00cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec00ccf00 0x7f2ec00cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5c00640 1 -- 172.21.15.71:0/340676127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec009ac10 con 0x7f2ec015f5a0 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec5c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec015f5a0 0x7f2ec00cc5d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800eb40 tx=0x7f2eb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ebcc00640 1 -- 172.21.15.71:0/340676127 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8004b20 con 0x7f2ec015f5a0 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ebcc00640 1 -- 172.21.15.71:0/340676127 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb8004ce0 con 0x7f2ec015f5a0 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ebcc00640 1 -- 172.21.15.71:0/340676127 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8005280 con 0x7f2ec015f5a0 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.423+0000 7f2ec7c00640 1 -- 172.21.15.71:0/340676127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec009aef0 con 0x7f2ec015f5a0 2024-09-16T07:23:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.424+0000 7f2ec7c00640 1 -- 172.21.15.71:0/340676127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec014e9f0 con 0x7f2ec015f5a0 2024-09-16T07:23:06.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.426+0000 7f2ebcc00640 1 -- 172.21.15.71:0/340676127 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2eb8005420 con 0x7f2ec015f5a0 2024-09-16T07:23:06.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.427+0000 7f2ebcc00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ea0078850 0x7f2ea007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.427+0000 7f2ebcc00640 1 -- 172.21.15.71:0/340676127 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2eb809b570 con 0x7f2ec015f5a0 2024-09-16T07:23:06.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.427+0000 7f2eca9f6740 1 Processor -- start 2024-09-16T07:23:06.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.427+0000 7f2eca9f6740 1 -- start start 2024-09-16T07:23:06.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.427+0000 7f2ec5200640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ea0078850 0x7f2ea007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.428+0000 7f2ec5200640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ea0078850 0x7f2ea007ad10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4033000 tx=0x7f2eb4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.428+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x55712108ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.428+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557121049a20 0x55712108d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.428+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x55712108d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.428+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557120fcc430 con 0x55712108e8b0 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.428+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557120fbc5d0 con 0x557121049fa0 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.428+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x557120faba00 con 0x557121049a20 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x55712108ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x55712108ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36598/0 (socket says 172.21.15.71:36598) 2024-09-16T07:23:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 -- 172.21.15.71:0/1563085370 learned_addr learned my addr 172.21.15.71:0/1563085370 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ebfe00640 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557121049a20 0x55712108d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5c00640 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x55712108d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 -- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557121049a20 msgr2=0x55712108d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557121049a20 0x55712108d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 -- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 msgr2=0x55712108d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x55712108d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 -- 172.21.15.71:0/1563085370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557120f74520 con 0x55712108e8b0 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5c00640 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x55712108d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2ec5200640 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x55712108ec90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4006010 tx=0x7f2eb403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167b180cfce3f378 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.429+0000 7f2eb1600640 1 -- 172.21.15.71:0/1563085370 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4035040 con 0x55712108e8b0 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eb1600640 1 -- 172.21.15.71:0/1563085370 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb404d070 con 0x55712108e8b0 2024-09-16T07:23:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eb1600640 1 -- 172.21.15.71:0/1563085370 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb403c670 con 0x55712108e8b0 2024-09-16T07:23:06.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1563085370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 msgr2=0x55712108ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x55712108ec90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4006010 tx=0x7f2eb403c000 comp rx=0 tx=0).stop 2024-09-16T07:23:06.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1563085370 shutdown_connections 2024-09-16T07:23:06.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x55712108d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557121049a20 0x55712108d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/1563085370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x55712108ec90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1563085370 >> 172.21.15.71:0/1563085370 conn(0x557121044ff0 msgr2=0x557121048a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1563085370 shutdown_connections 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.430+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1563085370 wait complete. 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.431+0000 7f2eca9f6740 1 -- start start 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.431+0000 7f2eca9f6740 1 Processor -- start 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x5571210a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x5571210a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x55712109fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557120f52330 con 0x55712108e8b0 2024-09-16T07:23:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557120faf810 con 0x557121049fa0 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x557120d1c620 con 0x55712109f8d0 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2eca9f6740 10 client.0 _ll_register_callbacks cb 0x557120f57420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2ebfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x5571210a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2ebfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x5571210a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36608/0 (socket says 172.21.15.71:36608) 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2ebfe00640 1 -- 172.21.15.71:0/3997728506 learned_addr learned my addr 172.21.15.71:0/3997728506 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.432+0000 7f2ec5200640 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x5571210a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ec5c00640 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x55712109fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ebfe00640 1 -- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 msgr2=0x55712109fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ebfe00640 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x55712109fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ebfe00640 1 -- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 msgr2=0x5571210a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ebfe00640 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x5571210a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ebfe00640 1 -- 172.21.15.71:0/3997728506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557120d0d850 con 0x55712108e8b0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ebfe00640 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x5571210a1230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2eac003ce0 tx=0x7f2eac001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44398e1a2c5bae18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ea6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ea6000640 1 -- 172.21.15.71:0/3997728506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac013540 con 0x55712108e8b0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ea6000640 1 -- 172.21.15.71:0/3997728506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2eac010040 con 0x55712108e8b0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2eca9f6740 1 -- 172.21.15.71:0/3997728506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb404d070 con 0x55712108e8b0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2ea6000640 1 -- 172.21.15.71:0/3997728506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f2eac0136e0 con 0x55712108e8b0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2eca9f6740 10 client.4719 fetch_fsmap learned FSMap version 8 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.433+0000 7f2eca9f6740 1 -- 172.21.15.71:0/3997728506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x557120d846b0 con 0x55712108e8b0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2ea6000640 1 -- 172.21.15.71:0/3997728506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb404d070 con 0x55712108e8b0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2ea6000640 1 -- 172.21.15.71:0/3997728506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f2eac013970 con 0x55712108e8b0 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2eca9f6740 10 client.4719 fetch_fsmap finished waiting for FSMap version 8 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2eca9f6740 10 client.4719 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:23:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2eca9f6740 20 client.4719 populate_metadata read hostname 'smithi071' 2024-09-16T07:23:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] conn(0x55712109dd00 0x5571210ab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2eca9f6740 4 client.4719 mds_command: new command op to 4404 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:23:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2eca9f6740 1 -- 172.21.15.71:0/3997728506 --> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557120fbe520 con 0x55712109dd00 2024-09-16T07:23:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.434+0000 7f2ec5200640 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] conn(0x55712109dd00 0x5571210ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.435+0000 7f2ec5200640 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] conn(0x55712109dd00 0x5571210ab580 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e36afb4b7e62dc server_cookie=97c37cc6e75e4dbd in_seq=0 out_seq=0 2024-09-16T07:23:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.435+0000 7f2ea6000640 10 client.4719 ms_handle_connect on v2:172.21.15.177:6836/1387172624 2024-09-16T07:23:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.436+0000 7f2ea6000640 1 -- 172.21.15.71:0/3997728506 <== mds.0 v2:172.21.15.177:6836/1387172624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557120fbe520 con 0x55712109dd00 2024-09-16T07:23:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.436+0000 7f2ea6000640 10 client.4719 handle_command_reply: tid=0 2024-09-16T07:23:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.436+0000 7f2eca9f6740 1 client.4719 shutdown 2024-09-16T07:23:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.436+0000 7f2eca9f6740 1 -- 172.21.15.71:0/3997728506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 msgr2=0x5571210a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.436+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x5571210a1230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2eac003ce0 tx=0x7f2eac001630 comp rx=0 tx=0).stop 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 1 -- 172.21.15.71:0/3997728506 shutdown_connections 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] conn(0x55712109dd00 0x5571210ab580 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x55712109fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x5571210a1230 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2ea6000640 0 client.4719 ms_handle_reset on v2:172.21.15.177:6836/1387172624 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/3997728506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x5571210a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 1 -- 172.21.15.71:0/3997728506 >> 172.21.15.71:0/3997728506 conn(0x557121049a20 msgr2=0x557121043bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 1 -- 172.21.15.71:0/3997728506 shutdown_connections 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 1 -- 172.21.15.71:0/3997728506 wait complete. 2024-09-16T07:23:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.437+0000 7f2eca9f6740 20 client.4719 trim_cache size 0 max 16384 2024-09-16T07:23:06.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.455+0000 7f2eca9f6740 1 Processor -- start 2024-09-16T07:23:06.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.456+0000 7f2eca9f6740 1 -- start start 2024-09-16T07:23:06.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.456+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557121049fa0 0x557121100c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.456+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55712108e8b0 0x55712106eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.456+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x557121099fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.456+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec009cbf0 con 0x557121049fa0 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.456+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01f8ec0 con 0x55712108e8b0 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.456+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec009c5a0 con 0x55712109f8d0 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557121049fa0 0x557121100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557121049fa0 0x557121100c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36614/0 (socket says 172.21.15.71:36614) 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 -- 172.21.15.71:0/1197400074 learned_addr learned my addr 172.21.15.71:0/1197400074 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 -- 172.21.15.71:0/1197400074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 msgr2=0x557121099fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 --2- 172.21.15.71:0/1197400074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x557121099fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 -- 172.21.15.71:0/1197400074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55712108e8b0 msgr2=0x55712106eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 --2- 172.21.15.71:0/1197400074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55712108e8b0 0x55712106eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 -- 172.21.15.71:0/1197400074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55712108fa80 con 0x557121049fa0 2024-09-16T07:23:06.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.457+0000 7f2ec5200640 1 --2- 172.21.15.71:0/1197400074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557121049fa0 0x557121100c20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40099f0 tx=0x7f2eb403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbe5428d416000c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eb0c00640 1 -- 172.21.15.71:0/1197400074 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb403c670 con 0x557121049fa0 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eb0c00640 1 -- 172.21.15.71:0/1197400074 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb405d030 con 0x557121049fa0 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eb0c00640 1 -- 172.21.15.71:0/1197400074 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb403cb00 con 0x557121049fa0 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1197400074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557121049fa0 msgr2=0x557121100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/1197400074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557121049fa0 0x557121100c20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40099f0 tx=0x7f2eb403e040 comp rx=0 tx=0).stop 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1197400074 shutdown_connections 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/1197400074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x557121099fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/1197400074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55712108e8b0 0x55712106eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/1197400074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557121049fa0 0x557121100c20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1197400074 >> 172.21.15.71:0/1197400074 conn(0x557121049a20 msgr2=0x55712106dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:06.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1197400074 shutdown_connections 2024-09-16T07:23:06.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.458+0000 7f2eca9f6740 1 -- 172.21.15.71:0/1197400074 wait complete. 2024-09-16T07:23:06.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.459+0000 7f2eca9f6740 1 -- start start 2024-09-16T07:23:06.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.459+0000 7f2eca9f6740 1 Processor -- start 2024-09-16T07:23:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x5571210f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x557121047e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2eca9f6740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x5571210483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55712109db80 con 0x55712108e8b0 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55712109f3b0 con 0x557121049fa0 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2eca9f6740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55712109e1a0 con 0x55712109f8d0 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2eca9f6740 10 client.0 _ll_register_callbacks cb 0x7f2eac005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2ebfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x557121047e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2ebfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x557121047e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36622/0 (socket says 172.21.15.71:36622) 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2ebfe00640 1 -- 172.21.15.71:0/2528650618 learned_addr learned my addr 172.21.15.71:0/2528650618 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2ec5c00640 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x5571210483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.460+0000 7f2ec5200640 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x5571210f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ebfe00640 1 -- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 msgr2=0x5571210483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ebfe00640 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x5571210483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ebfe00640 1 -- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 msgr2=0x5571210f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ebfe00640 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x5571210f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ebfe00640 1 -- 172.21.15.71:0/2528650618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55712110e9e0 con 0x55712108e8b0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ebfe00640 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x557121047e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2eac015730 tx=0x7f2eac015760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43931c434ffdc047 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ea6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ea6000640 1 -- 172.21.15.71:0/2528650618 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac013540 con 0x55712108e8b0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ea6000640 1 -- 172.21.15.71:0/2528650618 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2eac001b30 con 0x55712108e8b0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2eca9f6740 1 -- 172.21.15.71:0/2528650618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb405d030 con 0x55712108e8b0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2ea6000640 1 -- 172.21.15.71:0/2528650618 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f2eac031050 con 0x55712108e8b0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2eca9f6740 10 client.4725 fetch_fsmap learned FSMap version 8 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.461+0000 7f2eca9f6740 1 -- 172.21.15.71:0/2528650618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55712110f280 con 0x55712108e8b0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2ea6000640 1 -- 172.21.15.71:0/2528650618 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eb405d030 con 0x55712108e8b0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2ea6000640 1 -- 172.21.15.71:0/2528650618 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f2eac034020 con 0x55712108e8b0 2024-09-16T07:23:06.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2eca9f6740 10 client.4725 fetch_fsmap finished waiting for FSMap version 8 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2eca9f6740 10 client.4725 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2eca9f6740 20 client.4725 populate_metadata read hostname 'smithi071' 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] conn(0x55712109aaf0 0x557121041ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2eca9f6740 4 client.4725 mds_command: new command op to 4404 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2eca9f6740 1 -- 172.21.15.71:0/2528650618 --> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] -- command(tid 0: {"prefix": "status"}) -- 0x557120fb9fd0 con 0x55712109aaf0 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.462+0000 7f2ec5200640 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] conn(0x55712109aaf0 0x557121041ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2ec5200640 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] conn(0x55712109aaf0 0x557121041ed0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b141073b861518 server_cookie=cd176d044591f204 in_seq=0 out_seq=0 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2ea6000640 10 client.4725 ms_handle_connect on v2:172.21.15.177:6836/1387172624 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2ea6000640 1 -- 172.21.15.71:0/2528650618 <== mds.0 v2:172.21.15.177:6836/1387172624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x557120fb9fd0 con 0x55712109aaf0 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2ea6000640 10 client.4725 handle_command_reply: tid=0 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 client.4725 shutdown 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 -- 172.21.15.71:0/2528650618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 msgr2=0x557121047e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x557121047e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2eac015730 tx=0x7f2eac015760 comp rx=0 tx=0).stop 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 -- 172.21.15.71:0/2528650618 shutdown_connections 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:6836/1387172624,v1:172.21.15.177:6837/1387172624] conn(0x55712109aaf0 0x557121041ed0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55712109f8d0 0x5571210483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55712108e8b0 0x557121047e60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 --2- 172.21.15.71:0/2528650618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557121049fa0 0x5571210f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2ea6000640 0 client.4725 ms_handle_reset on v2:172.21.15.177:6836/1387172624 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 -- 172.21.15.71:0/2528650618 >> 172.21.15.71:0/2528650618 conn(0x557121049a20 msgr2=0x5571210ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 -- 172.21.15.71:0/2528650618 shutdown_connections 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 1 -- 172.21.15.71:0/2528650618 wait complete. 2024-09-16T07:23:06.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.463+0000 7f2eca9f6740 20 client.4725 trim_cache size 0 max 16384 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4404, 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 17.340020676000002, 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 8, 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 23, 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 23, 2024-09-16T07:23:06.469 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 23.110674541000002 2024-09-16T07:23:06.470 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:23:06.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 -- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ea0078850 msgr2=0x7f2ea007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ea0078850 0x7f2ea007ad10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4033000 tx=0x7f2eb4004730 comp rx=0 tx=0).stop 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 -- 172.21.15.71:0/340676127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec015f5a0 msgr2=0x7f2ec00cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec015f5a0 0x7f2ec00cc5d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800eb40 tx=0x7f2eb800c2d0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 -- 172.21.15.71:0/340676127 shutdown_connections 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ea0078850 0x7f2ea007ad10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec015f5a0 0x7f2ec00cc5d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec00ccf00 0x7f2ec00cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 --2- 172.21.15.71:0/340676127 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec0073fb0 0x7f2ec014e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 -- 172.21.15.71:0/340676127 >> 172.21.15.71:0/340676127 conn(0x7f2ec00b1950 msgr2=0x7f2ec00b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 -- 172.21.15.71:0/340676127 shutdown_connections 2024-09-16T07:23:06.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:06.464+0000 7f2ec7c00640 1 -- 172.21.15.71:0/340676127 wait complete. 2024-09-16T07:23:06.479 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T07:23:06.505 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T07:23:06.505 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-16T07:23:06.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:06.505 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:23:06.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:06.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-09-16T07:23:06.639 INFO:teuthology.orchestra.run.smithi071.stderr:Cannot find device "ceph-brx" 2024-09-16T07:23:06.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:23:06.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:06.642 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:23:06.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:06.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-09-16T07:23:06.775 INFO:teuthology.orchestra.run.smithi071.stderr:Cannot find device "ceph-brx" 2024-09-16T07:23:06.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:23:06.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:06.777 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-09-16T07:23:06.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:06.818 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2024-09-16T07:23:06.920 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2024-09-16T07:23:06.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:23:06.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:06.924 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-09-16T07:23:06.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:06.961 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2024-09-16T07:23:07.058 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2024-09-16T07:23:07.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:23:07.061 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-16T07:23:07.061 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T07:23:07.062 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:07.062 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:23:07.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.218+0000 7ff8da800640 1 -- 172.21.15.71:0/2035335672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d4073fb0 msgr2=0x7ff8d4074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:07.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.218+0000 7ff8da800640 1 --2- 172.21.15.71:0/2035335672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d4073fb0 0x7ff8d4074440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc005fd0 tx=0x7ff8cc02aa10 comp rx=0 tx=0).stop 2024-09-16T07:23:07.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.218+0000 7ff8da800640 1 -- 172.21.15.71:0/2035335672 shutdown_connections 2024-09-16T07:23:07.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.218+0000 7ff8da800640 1 --2- 172.21.15.71:0/2035335672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8d40ccf00 0x7ff8d40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.218+0000 7ff8da800640 1 --2- 172.21.15.71:0/2035335672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d4073fb0 0x7ff8d4074440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.219+0000 7ff8da800640 1 -- 172.21.15.71:0/2035335672 >> 172.21.15.71:0/2035335672 conn(0x7ff8d40b1950 msgr2=0x7ff8d40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:07.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.219+0000 7ff8da800640 1 -- 172.21.15.71:0/2035335672 shutdown_connections 2024-09-16T07:23:07.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.219+0000 7ff8da800640 1 -- 172.21.15.71:0/2035335672 wait complete. 2024-09-16T07:23:07.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.220+0000 7ff8da800640 1 Processor -- start 2024-09-16T07:23:07.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.220+0000 7ff8da800640 1 -- start start 2024-09-16T07:23:07.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.221+0000 7ff8da800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8d4073fb0 0x7ff8d4148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:07.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.221+0000 7ff8da800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d40ccf00 0x7ff8d4156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:07.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.221+0000 7ff8da800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8d415f5a0 0x7ff8d415da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:07.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.221+0000 7ff8da800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d406f2e0 con 0x7ff8d40ccf00 2024-09-16T07:23:07.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.221+0000 7ff8da800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d406f160 con 0x7ff8d415f5a0 2024-09-16T07:23:07.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.221+0000 7ff8da800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d406f460 con 0x7ff8d4073fb0 2024-09-16T07:23:07.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d40ccf00 0x7ff8d4156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:07.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d40ccf00 0x7ff8d4156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36652/0 (socket says 172.21.15.71:36652) 2024-09-16T07:23:07.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 -- 172.21.15.71:0/4138989583 learned_addr learned my addr 172.21.15.71:0/4138989583 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d3400640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8d4073fb0 0x7ff8d4148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d3e00640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8d415f5a0 0x7ff8d415da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 -- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8d4073fb0 msgr2=0x7ff8d4148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8d4073fb0 0x7ff8d4148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 -- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8d415f5a0 msgr2=0x7ff8d415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8d415f5a0 0x7ff8d415da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 -- 172.21.15.71:0/4138989583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d40cea40 con 0x7ff8d40ccf00 2024-09-16T07:23:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.222+0000 7ff8d2a00640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d40ccf00 0x7ff8d4156090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000ea20 tx=0x7ff8c000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:07.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.223+0000 7ff8cb400640 1 -- 172.21.15.71:0/4138989583 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000cd80 con 0x7ff8d40ccf00 2024-09-16T07:23:07.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.223+0000 7ff8cb400640 1 -- 172.21.15.71:0/4138989583 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8c0010430 con 0x7ff8d40ccf00 2024-09-16T07:23:07.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.223+0000 7ff8cb400640 1 -- 172.21.15.71:0/4138989583 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0010840 con 0x7ff8d40ccf00 2024-09-16T07:23:07.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.223+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d4148bc0 con 0x7ff8d40ccf00 2024-09-16T07:23:07.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.223+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d40cdf60 con 0x7ff8d40ccf00 2024-09-16T07:23:07.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.224+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d406f9f0 con 0x7ff8d40ccf00 2024-09-16T07:23:07.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.225+0000 7ff8cb400640 1 -- 172.21.15.71:0/4138989583 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff8c00109e0 con 0x7ff8d40ccf00 2024-09-16T07:23:07.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.226+0000 7ff8cb400640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8b4078850 0x7ff8b407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:07.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.226+0000 7ff8cb400640 1 -- 172.21.15.71:0/4138989583 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff8c009b4f0 con 0x7ff8d40ccf00 2024-09-16T07:23:07.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.226+0000 7ff8d3400640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8b4078850 0x7ff8b407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.227+0000 7ff8d3400640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8b4078850 0x7ff8b407ad10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4001720 tx=0x7ff8c4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.230+0000 7ff8cb400640 1 -- 172.21.15.71:0/4138989583 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff8c0063470 con 0x7ff8d40ccf00 2024-09-16T07:23:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.451+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff8d40b0650 con 0x7ff8d40ccf00 2024-09-16T07:23:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.451+0000 7ff8cb400640 1 -- 172.21.15.71:0/4138989583 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7ff8c0067ea0 con 0x7ff8d40ccf00 2024-09-16T07:23:07.452 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:23:07.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8b4078850 msgr2=0x7ff8b407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:07.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8b4078850 0x7ff8b407ad10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4001720 tx=0x7ff8c4009040 comp rx=0 tx=0).stop 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d40ccf00 msgr2=0x7ff8d4156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d40ccf00 0x7ff8d4156090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000ea20 tx=0x7ff8c000eee0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 shutdown_connections 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8b4078850 0x7ff8b407ad10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8d415f5a0 0x7ff8d415da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8d40ccf00 0x7ff8d4156090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.453+0000 7ff8da800640 1 --2- 172.21.15.71:0/4138989583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8d4073fb0 0x7ff8d4148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.454+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 >> 172.21.15.71:0/4138989583 conn(0x7ff8d40b1950 msgr2=0x7ff8d40af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.454+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 shutdown_connections 2024-09-16T07:23:07.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07.454+0000 7ff8da800640 1 -- 172.21.15.71:0/4138989583 wait complete. 2024-09-16T07:23:07.470 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T07:23:07.470 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:23:07.470 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:23:07.470 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:23:07.471 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:07.471 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T07:23:07.471 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T07:23:07.471 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:23:07.471 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:23:07.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:07.471 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:23:07.507 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:07.507 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:07.507 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:07.507 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:23:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:07.509 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:23:07.509 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42689sec preferred_lft 42689sec 2024-09-16T07:23:07.509 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:23:07.509 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:07.509 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T07:23:07.509 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:07.510 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:07.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2024-09-16T07:23:07.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2024-09-16T07:23:07.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2024-09-16T07:23:07.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T07:23:07.510 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:07.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:07.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:07.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:07.926 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:23:07.982 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:23:07.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:07.986 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:23:08.052 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:23:08.052 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:23:08.052 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:23:08.053 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:23:08.053 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:23:08.053 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:08.053 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:08.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:23:08.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:23:08.054 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:23:08.054 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:08.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:08.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:08.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:08.645 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:23:08.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:08.681 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-09-16T07:23:08.756 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:08.757 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:08.757 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:23:08.757 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T07:23:08.757 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:08.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:08.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:08.932 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T07:23:08.933 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:08.933 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:08.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T07:23:08.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T07:23:08.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T07:23:08.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T07:23:08.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T07:23:08.933 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:09.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:09.174 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:09.174 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:09.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:23:09.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T07:23:09.174 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:09.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:09.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:09.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:09.339 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:09.374 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:23:09.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:09.374 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:09.449 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:23:09.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:09.450 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:09.594 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:09.594 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:10.259 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:10.259 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:10.260 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:23:10.260 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:23:10.260 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:23:10.261 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:23:10.261 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:23:10.261 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:23:10.261 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:23:10.262 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:23:10.264 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:23:10.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:10.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:10.323 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:10.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:10.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:10.422 INFO:teuthology.orchestra.run.smithi071.stdout:client4678 2024-09-16T07:23:10.422 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:10.422 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:10.458 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:10.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:10.597 INFO:teuthology.orchestra.run.smithi071.stdout:client4678 2024-09-16T07:23:10.597 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:10.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:10.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:10.697 INFO:teuthology.orchestra.run.smithi071.stdout:client4678 2024-09-16T07:23:10.697 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5821e025-6a94-4c05-a5f1-d6094eac5c68.client4678/status') 2024-09-16T07:23:10.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:10.806 INFO:teuthology.orchestra.run.smithi071.stdout:instance: client.4678 (3)172.21.15.71:0/4068490833 2024-09-16T07:23:10.806 INFO:teuthology.orchestra.run.smithi071.stdout:blocklisted: false 2024-09-16T07:23:10.806 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records in 2024-09-16T07:23:10.806 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records out 2024-09-16T07:23:10.806 INFO:teuthology.orchestra.run.smithi071.stderr:70 bytes copied, 5.1642e-05 s, 1.4 MB/s 2024-09-16T07:23:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:10.817 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-16T07:23:10.817 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T07:23:10.817 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-16T07:23:10.817 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:23:10.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.974+0000 7fa95be00640 1 -- 172.21.15.71:0/4026038149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 msgr2=0x7fa95c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.974+0000 7fa95be00640 1 --2- 172.21.15.71:0/4026038149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 0x7fa95c0b28f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005fd0 tx=0x7fa94c02a670 comp rx=0 tx=0).stop 2024-09-16T07:23:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.974+0000 7fa95be00640 1 -- 172.21.15.71:0/4026038149 shutdown_connections 2024-09-16T07:23:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.974+0000 7fa95be00640 1 --2- 172.21.15.71:0/4026038149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 0x7fa95c0b28f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.974+0000 7fa95be00640 1 -- 172.21.15.71:0/4026038149 >> 172.21.15.71:0/4026038149 conn(0x7fa95c0cfa20 msgr2=0x7fa95c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.974+0000 7fa95be00640 1 -- 172.21.15.71:0/4026038149 shutdown_connections 2024-09-16T07:23:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.974+0000 7fa95be00640 1 -- 172.21.15.71:0/4026038149 wait complete. 2024-09-16T07:23:10.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.975+0000 7fa95be00640 1 Processor -- start 2024-09-16T07:23:10.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.975+0000 7fa95be00640 1 -- start start 2024-09-16T07:23:10.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.976+0000 7fa95be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa95c0d0540 0x7fa95c14c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:10.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.976+0000 7fa95be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 0x7fa95c14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:10.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.976+0000 7fa95be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa95c14e4e0 0x7fa95c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:10.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.976+0000 7fa95be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c074660 con 0x7fa95c15bc30 2024-09-16T07:23:10.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.976+0000 7fa95be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c06fa30 con 0x7fa95c0d0540 2024-09-16T07:23:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.976+0000 7fa95be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa95c15cce0 con 0x7fa95c14e4e0 2024-09-16T07:23:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 0x7fa95c14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 0x7fa95c14ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36666/0 (socket says 172.21.15.71:36666) 2024-09-16T07:23:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 -- 172.21.15.71:0/2600620437 learned_addr learned my addr 172.21.15.71:0/2600620437 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95b400640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa95c14e4e0 0x7fa95c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95aa00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa95c0d0540 0x7fa95c14c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 -- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa95c14e4e0 msgr2=0x7fa95c14d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa95c14e4e0 0x7fa95c14d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 -- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa95c0d0540 msgr2=0x7fa95c14c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa95c0d0540 0x7fa95c14c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 -- 172.21.15.71:0/2600620437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c14cd10 con 0x7fa95c15bc30 2024-09-16T07:23:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.977+0000 7fa95a000640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 0x7fa95c14ea10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa94800c4d0 tx=0x7fa94800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.978+0000 7fa952a00640 1 -- 172.21.15.71:0/2600620437 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480052d0 con 0x7fa95c15bc30 2024-09-16T07:23:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.978+0000 7fa952a00640 1 -- 172.21.15.71:0/2600620437 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa948005470 con 0x7fa95c15bc30 2024-09-16T07:23:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.978+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95c14e1b0 con 0x7fa95c15bc30 2024-09-16T07:23:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.978+0000 7fa952a00640 1 -- 172.21.15.71:0/2600620437 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948012870 con 0x7fa95c15bc30 2024-09-16T07:23:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.978+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95c15c690 con 0x7fa95c15bc30 2024-09-16T07:23:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.979+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95c0b28f0 con 0x7fa95c15bc30 2024-09-16T07:23:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.980+0000 7fa952a00640 1 -- 172.21.15.71:0/2600620437 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa9480035f0 con 0x7fa95c15bc30 2024-09-16T07:23:10.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.981+0000 7fa952a00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa93c078880 0x7fa93c07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:10.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.981+0000 7fa952a00640 1 -- 172.21.15.71:0/2600620437 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa94809a310 con 0x7fa95c15bc30 2024-09-16T07:23:10.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.981+0000 7fa95aa00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa93c078880 0x7fa93c07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:10.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.982+0000 7fa95aa00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa93c078880 0x7fa93c07ad40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa94c033000 tx=0x7fa94c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:10.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:10.985+0000 7fa952a00640 1 -- 172.21.15.71:0/2600620437 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa95c14cd10 con 0x7fa95c15bc30 2024-09-16T07:23:11.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.209+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa95c15ce60 con 0x7fa95c15bc30 2024-09-16T07:23:11.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.210+0000 7fa952a00640 1 -- 172.21.15.71:0/2600620437 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa948062320 con 0x7fa95c15bc30 2024-09-16T07:23:11.209 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:23:11.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa93c078880 msgr2=0x7fa93c07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa93c078880 0x7fa93c07ad40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa94c033000 tx=0x7fa94c004730 comp rx=0 tx=0).stop 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 msgr2=0x7fa95c14ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 0x7fa95c14ea10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa94800c4d0 tx=0x7fa94800c990 comp rx=0 tx=0).stop 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 shutdown_connections 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa93c078880 0x7fa93c07ad40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa95c14e4e0 0x7fa95c14d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95c15bc30 0x7fa95c14ea10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 --2- 172.21.15.71:0/2600620437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa95c0d0540 0x7fa95c14c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 >> 172.21.15.71:0/2600620437 conn(0x7fa95c0cfa20 msgr2=0x7fa95c159100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 shutdown_connections 2024-09-16T07:23:11.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11.212+0000 7fa95be00640 1 -- 172.21.15.71:0/2600620437 wait complete. 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:23:11.229 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-16T07:23:11.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:11.230 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:23:11.265 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:11.266 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42686sec preferred_lft 42686sec 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.267 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:23:11.268 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.268 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:23:11.268 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:23:11.268 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:23:11.268 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:11.268 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:23:11.360 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:23:11.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:11.363 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:23:11.399 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:23:11.399 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:23:11.399 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:23:11.399 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:23:11.399 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:23:11.399 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:11.399 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:11.399 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:23:11.400 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:23:11.400 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:23:11.400 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:11.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:11.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:11.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:11.594 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:23:11.629 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:23:11.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:11.629 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-09-16T07:23:11.704 INFO:teuthology.orchestra.run.smithi071.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-16T07:23:11.705 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:11.705 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:11.705 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:23:11.705 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-16T07:23:11.705 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:11.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:11.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:11.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:11.880 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-16T07:23:11.936 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:11.936 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:11.936 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:11.936 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.936 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:11.937 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.937 INFO:teuthology.orchestra.run.smithi071.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:23:11.937 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 2e:18:d3:e2:c8:d4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T07:23:11.937 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T07:23:11.937 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.937 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::2c18:d3ff:fee2:c8d4/64 scope link 2024-09-16T07:23:11.937 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:11.939 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 2e:18:d3:e2:c8:d4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T07:23:11.994 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.995 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::2c18:d3ff:fee2:c8d4/64 scope link 2024-09-16T07:23:11.995 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:11.997 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-16T07:23:11.997 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:11.997 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:11.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-16T07:23:11.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-16T07:23:11.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-16T07:23:11.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-16T07:23:11.998 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-16T07:23:11.998 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:12.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:12.213 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:12.214 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:12.214 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.1 up 2024-09-16T07:23:12.214 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.1 master ceph-brx 2024-09-16T07:23:12.214 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:12.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:12.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:12.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:12.380 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-16T07:23:12.415 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-16T07:23:12.415 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:12.415 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-16T07:23:12.489 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-16T07:23:12.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:12.490 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:12.617 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:12.617 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:12.646 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:12.646 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:12.647 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:23:12.647 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 2@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:23:12.647 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:23:12.648 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-16T07:23:12.648 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:23:12.648 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:23:12.648 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:23:12.648 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:23:12.651 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:23:12.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:12.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-16T07:23:12.706 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:12.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:12.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:12.795 INFO:teuthology.orchestra.run.smithi071.stdout:client4743 2024-09-16T07:23:12.796 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:12.796 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:12.831 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:12.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:12.959 INFO:teuthology.orchestra.run.smithi071.stdout:client4743 2024-09-16T07:23:12.959 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:13.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:13.047 INFO:teuthology.orchestra.run.smithi071.stdout:client4743 2024-09-16T07:23:13.047 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5821e025-6a94-4c05-a5f1-d6094eac5c68.client4743/status') 2024-09-16T07:23:13.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:13.150 INFO:teuthology.orchestra.run.smithi071.stdout:instance: client.4743 (3)192.168.0.2:0/792562903 2024-09-16T07:23:13.150 INFO:teuthology.orchestra.run.smithi071.stdout:blocklisted: false 2024-09-16T07:23:13.151 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records in 2024-09-16T07:23:13.151 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records out 2024-09-16T07:23:13.151 INFO:teuthology.orchestra.run.smithi071.stderr:68 bytes copied, 5.6239e-05 s, 1.2 MB/s 2024-09-16T07:23:13.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:13.161 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-16T07:23:13.161 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T07:23:13.161 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T07:23:13.162 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:23:13.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.322+0000 7efc60e00640 1 -- 172.21.15.71:0/1496114627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c161440 msgr2=0x7efc5c0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:13.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.322+0000 7efc60e00640 1 --2- 172.21.15.71:0/1496114627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c161440 0x7efc5c0b5370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009860 tx=0x7efc4c02a510 comp rx=0 tx=0).stop 2024-09-16T07:23:13.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.322+0000 7efc60e00640 1 -- 172.21.15.71:0/1496114627 shutdown_connections 2024-09-16T07:23:13.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.322+0000 7efc60e00640 1 --2- 172.21.15.71:0/1496114627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c161440 0x7efc5c0b5370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:13.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.322+0000 7efc60e00640 1 -- 172.21.15.71:0/1496114627 >> 172.21.15.71:0/1496114627 conn(0x7efc5c0d3df0 msgr2=0x7efc5c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:13.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.323+0000 7efc60e00640 1 -- 172.21.15.71:0/1496114627 shutdown_connections 2024-09-16T07:23:13.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.323+0000 7efc60e00640 1 -- 172.21.15.71:0/1496114627 wait complete. 2024-09-16T07:23:13.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.324+0000 7efc60e00640 1 Processor -- start 2024-09-16T07:23:13.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.324+0000 7efc60e00640 1 -- start start 2024-09-16T07:23:13.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.325+0000 7efc60e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c0b34b0 0x7efc5c0ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:13.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.325+0000 7efc60e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc5c0b4020 0x7efc5c0cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:13.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.325+0000 7efc60e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c161440 0x7efc5c0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:13.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.325+0000 7efc60e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c0b2210 con 0x7efc5c161440 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.325+0000 7efc60e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c0b2090 con 0x7efc5c0b4020 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.325+0000 7efc60e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efc5c0b2390 con 0x7efc5c0b34b0 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.325+0000 7efc5be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c161440 0x7efc5c0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c161440 0x7efc5c0d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36672/0 (socket says 172.21.15.71:36672) 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5be00640 1 -- 172.21.15.71:0/995097573 learned_addr learned my addr 172.21.15.71:0/995097573 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc53e00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc5c0b4020 0x7efc5c0cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5b400640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c0b34b0 0x7efc5c0ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5be00640 1 -- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c0b34b0 msgr2=0x7efc5c0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5be00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c0b34b0 0x7efc5c0ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5be00640 1 -- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc5c0b4020 msgr2=0x7efc5c0cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5be00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc5c0b4020 0x7efc5c0cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5be00640 1 -- 172.21.15.71:0/995097573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c0cc880 con 0x7efc5c161440 2024-09-16T07:23:13.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5b400640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c0b34b0 0x7efc5c0ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:13.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.326+0000 7efc5be00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c161440 0x7efc5c0d19a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc5400cbd0 tx=0x7efc5400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:13.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.327+0000 7efc53400640 1 -- 172.21.15.71:0/995097573 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54016020 con 0x7efc5c161440 2024-09-16T07:23:13.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.327+0000 7efc53400640 1 -- 172.21.15.71:0/995097573 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc5400ae50 con 0x7efc5c161440 2024-09-16T07:23:13.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.327+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c0ccbc0 con 0x7efc5c161440 2024-09-16T07:23:13.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.327+0000 7efc53400640 1 -- 172.21.15.71:0/995097573 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54012680 con 0x7efc5c161440 2024-09-16T07:23:13.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.327+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c0ca620 con 0x7efc5c161440 2024-09-16T07:23:13.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.328+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5c0b27b0 con 0x7efc5c161440 2024-09-16T07:23:13.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.329+0000 7efc53400640 1 -- 172.21.15.71:0/995097573 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7efc54019020 con 0x7efc5c161440 2024-09-16T07:23:13.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.330+0000 7efc53400640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc38078850 0x7efc3807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.330+0000 7efc53400640 1 -- 172.21.15.71:0/995097573 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efc5409b280 con 0x7efc5c161440 2024-09-16T07:23:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.331+0000 7efc5b400640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc38078850 0x7efc3807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:13.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.332+0000 7efc5b400640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc38078850 0x7efc3807ad10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc4c034000 tx=0x7efc4c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:13.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.334+0000 7efc53400640 1 -- 172.21.15.71:0/995097573 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc54020020 con 0x7efc5c161440 2024-09-16T07:23:13.368 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:23:13.372 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:23:13.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.567+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efc5c0b3a90 con 0x7efc5c161440 2024-09-16T07:23:13.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.567+0000 7efc53400640 1 -- 172.21.15.71:0/995097573 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7efc5c0ccbc0 con 0x7efc5c161440 2024-09-16T07:23:13.567 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:23:13.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc38078850 msgr2=0x7efc3807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:13.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc38078850 0x7efc3807ad10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc4c034000 tx=0x7efc4c004730 comp rx=0 tx=0).stop 2024-09-16T07:23:13.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c161440 msgr2=0x7efc5c0d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c161440 0x7efc5c0d19a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc5400cbd0 tx=0x7efc5400a430 comp rx=0 tx=0).stop 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 shutdown_connections 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc38078850 0x7efc3807ad10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c161440 0x7efc5c0d19a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc5c0b4020 0x7efc5c0cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 --2- 172.21.15.71:0/995097573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc5c0b34b0 0x7efc5c0ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 >> 172.21.15.71:0/995097573 conn(0x7efc5c0d3df0 msgr2=0x7efc5c0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 shutdown_connections 2024-09-16T07:23:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:13.570+0000 7efc60e00640 1 -- 172.21.15.71:0/995097573 wait complete. 2024-09-16T07:23:13.584 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T07:23:13.584 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:23:13.585 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2024-09-16T07:23:13.585 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T07:23:13.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T07:23:13.585 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T07:23:13.585 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T07:23:13.585 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:23:13.585 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T07:23:13.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:13.585 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2024-09-16T07:23:13.622 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:13.622 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:13.623 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:23:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42668sec preferred_lft 42668sec 2024-09-16T07:23:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link noprefixroute 2024-09-16T07:23:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:13.625 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T07:23:13.625 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:13.625 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:13.625 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add name ceph-brx type bridge 2024-09-16T07:23:13.625 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr flush dev ceph-brx 2024-09-16T07:23:13.625 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set ceph-brx up 2024-09-16T07:23:13.625 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T07:23:13.625 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:13.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:14.037 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:14.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:14.045 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:23:14.101 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-09-16T07:23:14.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:14.105 DEBUG:teuthology.orchestra.run.smithi177:> route 2024-09-16T07:23:14.155 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2024-09-16T07:23:14.155 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:23:14.155 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:23:14.156 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:23:14.156 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:23:14.156 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:14.156 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:14.156 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:23:14.156 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:23:14.156 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:23:14.156 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:14.234 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:14.710 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:14.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:14.718 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-09-16T07:23:14.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:14.754 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2024-09-16T07:23:14.829 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:14.830 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:14.830 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T07:23:14.830 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-16T07:23:14.830 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:14.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:14.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:15.007 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-16T07:23:15.007 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:15.007 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:15.007 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-16T07:23:15.007 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T07:23:15.007 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-16T07:23:15.007 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-16T07:23:15.007 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-16T07:23:15.008 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:15.084 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:15.229 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:15.237 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:15.237 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:15.237 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.0 up 2024-09-16T07:23:15.237 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T07:23:15.237 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:15.314 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:15.395 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:15.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:15.403 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T07:23:15.438 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T07:23:15.438 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:15.438 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T07:23:15.513 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T07:23:15.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:15.513 DEBUG:teuthology.orchestra.run.smithi177:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:15.654 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:15.655 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:16.238 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:16.238 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:16.238 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:23:16.238 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: trying mount with new device syntax: 1@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:23:16.238 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:23:16.240 INFO:teuthology.orchestra.run.smithi177.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T07:23:16.240 INFO:teuthology.orchestra.run.smithi177.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:23:16.240 INFO:teuthology.orchestra.run.smithi177.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:23:16.240 INFO:teuthology.orchestra.run.smithi177.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:23:16.241 INFO:teuthology.orchestra.run.smithi177.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:23:16.243 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:23:16.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:16.243 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T07:23:16.300 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:16.382 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:16.386 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:16.394 INFO:teuthology.orchestra.run.smithi177.stdout:client4752 2024-09-16T07:23:16.394 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:23:16.394 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:16.430 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:16.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:16.550 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:16.557 INFO:teuthology.orchestra.run.smithi177.stdout:client4752 2024-09-16T07:23:16.558 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:16.635 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:16.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:16.647 INFO:teuthology.orchestra.run.smithi177.stdout:client4752 2024-09-16T07:23:16.647 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5821e025-6a94-4c05-a5f1-d6094eac5c68.client4752/status') 2024-09-16T07:23:16.724 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:16.745 INFO:teuthology.orchestra.run.smithi177.stdout:instance: client.4752 (3)172.21.15.177:0/2745013274 2024-09-16T07:23:16.745 INFO:teuthology.orchestra.run.smithi177.stdout:blocklisted: false 2024-09-16T07:23:16.745 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records in 2024-09-16T07:23:16.745 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records out 2024-09-16T07:23:16.746 INFO:teuthology.orchestra.run.smithi177.stderr:71 bytes copied, 5.575e-05 s, 1.3 MB/s 2024-09-16T07:23:16.747 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:16.756 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-16T07:23:16.756 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T07:23:16.756 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-16T07:23:16.756 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:23:16.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.926+0000 7f5d9b200640 1 -- 172.21.15.71:0/3784598059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ccf00 msgr2=0x7f5d940acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:16.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.926+0000 7f5d9b200640 1 --2- 172.21.15.71:0/3784598059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ccf00 0x7f5d940acd30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5d80005fd0 tx=0x7f5d8002a540 comp rx=0 tx=0).stop 2024-09-16T07:23:16.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.926+0000 7f5d9b200640 1 -- 172.21.15.71:0/3784598059 shutdown_connections 2024-09-16T07:23:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.926+0000 7f5d9b200640 1 --2- 172.21.15.71:0/3784598059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ccf00 0x7f5d940acd30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.926+0000 7f5d9b200640 1 --2- 172.21.15.71:0/3784598059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d9415f5a0 0x7f5d940b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.926+0000 7f5d9b200640 1 -- 172.21.15.71:0/3784598059 >> 172.21.15.71:0/3784598059 conn(0x7f5d940b1950 msgr2=0x7f5d940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.926+0000 7f5d9b200640 1 -- 172.21.15.71:0/3784598059 shutdown_connections 2024-09-16T07:23:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.926+0000 7f5d9b200640 1 -- 172.21.15.71:0/3784598059 wait complete. 2024-09-16T07:23:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.928+0000 7f5d9b200640 1 Processor -- start 2024-09-16T07:23:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.928+0000 7f5d9b200640 1 -- start start 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d9b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ad270 0x7f5d9415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d9b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d940ccf00 0x7f5d9414c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d9b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9415f5a0 0x7f5d941e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d9b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9406f2e0 con 0x7f5d9415f5a0 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d9b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9406f160 con 0x7f5d940ad270 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d9b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9406f460 con 0x7f5d940ccf00 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d99200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9415f5a0 0x7f5d941e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d99200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9415f5a0 0x7f5d941e8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56468/0 (socket says 172.21.15.71:56468) 2024-09-16T07:23:16.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.929+0000 7f5d99200640 1 -- 172.21.15.71:0/1036324951 learned_addr learned my addr 172.21.15.71:0/1036324951 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d93e00640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ad270 0x7f5d9415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d93400640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d940ccf00 0x7f5d9414c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d99200640 1 -- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d940ccf00 msgr2=0x7f5d9414c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d99200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d940ccf00 0x7f5d9414c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d99200640 1 -- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ad270 msgr2=0x7f5d9415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d99200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ad270 0x7f5d9415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d99200640 1 -- 172.21.15.71:0/1036324951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d940b4960 con 0x7f5d9415f5a0 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d93e00640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ad270 0x7f5d9415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d99200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9415f5a0 0x7f5d941e8b40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c400 tx=0x7f5d8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d8be00640 1 -- 172.21.15.71:0/1036324951 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c017070 con 0x7f5d9415f5a0 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d8be00640 1 -- 172.21.15.71:0/1036324951 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d8c0053c0 con 0x7f5d9415f5a0 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.930+0000 7f5d8be00640 1 -- 172.21.15.71:0/1036324951 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c003850 con 0x7f5d9415f5a0 2024-09-16T07:23:16.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.931+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9415a600 con 0x7f5d9415f5a0 2024-09-16T07:23:16.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.931+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d940b2e30 con 0x7f5d9415f5a0 2024-09-16T07:23:16.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.932+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9406f980 con 0x7f5d9415f5a0 2024-09-16T07:23:16.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.933+0000 7f5d8be00640 1 -- 172.21.15.71:0/1036324951 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5d8c01a070 con 0x7f5d9415f5a0 2024-09-16T07:23:16.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.934+0000 7f5d8be00640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d74078880 0x7f5d7407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:16.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.934+0000 7f5d8be00640 1 -- 172.21.15.71:0/1036324951 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5d8c099d20 con 0x7f5d9415f5a0 2024-09-16T07:23:16.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.934+0000 7f5d93e00640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d74078880 0x7f5d7407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:16.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.935+0000 7f5d93e00640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d74078880 0x7f5d7407ad40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d940d0790 tx=0x7f5d84009500 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:16.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:16.938+0000 7f5d8be00640 1 -- 172.21.15.71:0/1036324951 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5d8c062440 con 0x7f5d9415f5a0 2024-09-16T07:23:17.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.158+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5d940741a0 con 0x7f5d9415f5a0 2024-09-16T07:23:17.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.158+0000 7f5d8be00640 1 -- 172.21.15.71:0/1036324951 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f5d8c061d40 con 0x7f5d9415f5a0 2024-09-16T07:23:17.158 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:23:17.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.160+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d74078880 msgr2=0x7f5d7407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:17.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d74078880 0x7f5d7407ad40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d940d0790 tx=0x7f5d84009500 comp rx=0 tx=0).stop 2024-09-16T07:23:17.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9415f5a0 msgr2=0x7f5d941e8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9415f5a0 0x7f5d941e8b40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c400 tx=0x7f5d8c00c8c0 comp rx=0 tx=0).stop 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 shutdown_connections 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d74078880 0x7f5d7407ad40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9415f5a0 0x7f5d941e8b40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d940ccf00 0x7f5d9414c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 --2- 172.21.15.71:0/1036324951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d940ad270 0x7f5d9415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 >> 172.21.15.71:0/1036324951 conn(0x7f5d940b1950 msgr2=0x7f5d9409d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 shutdown_connections 2024-09-16T07:23:17.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:17.161+0000 7f5d9b200640 1 -- 172.21.15.71:0/1036324951 wait complete. 2024-09-16T07:23:17.176 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T07:23:17.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:23:17.177 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2024-09-16T07:23:17.177 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-16T07:23:17.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-16T07:23:17.177 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T07:23:17.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T07:23:17.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:23:17.177 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-16T07:23:17.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:17.177 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2024-09-16T07:23:17.214 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:17.215 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42665sec preferred_lft 42665sec 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link noprefixroute 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 02:eb:ae:4e:50:5f brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ccf3:88ff:fe2c:ff9e/64 scope link 2024-09-16T07:23:17.216 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.217 INFO:teuthology.orchestra.run.smithi177.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:23:17.217 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 02:eb:ae:4e:50:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T07:23:17.217 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::eb:aeff:fe4e:505f/64 scope link tentative 2024-09-16T07:23:17.217 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:17.217 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:23:17.309 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-09-16T07:23:17.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:17.312 DEBUG:teuthology.orchestra.run.smithi177:> route 2024-09-16T07:23:17.348 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2024-09-16T07:23:17.349 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:23:17.349 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:23:17.349 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:23:17.349 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:23:17.349 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:17.349 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:17.349 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:23:17.350 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:23:17.350 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:23:17.350 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:17.465 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:17.535 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:17.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:17.544 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-09-16T07:23:17.581 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T07:23:17.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:17.582 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2024-09-16T07:23:17.656 INFO:teuthology.orchestra.run.smithi177.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-16T07:23:17.657 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:17.657 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:17.657 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T07:23:17.657 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-16T07:23:17.657 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:17.774 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:17.826 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:17.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:17.834 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0a:d4:5c:85:c0:be brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.890 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::8d4:5cff:fe85:c0be/64 scope link 2024-09-16T07:23:17.891 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:17.893 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-16T07:23:17.948 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.948 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.949 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:23:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0a:d4:5c:85:c0:be brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T07:23:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T07:23:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::8d4:5cff:fe85:c0be/64 scope link 2024-09-16T07:23:17.949 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:17.951 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-16T07:23:17.951 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:17.951 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:17.951 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-16T07:23:17.952 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-16T07:23:17.952 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-16T07:23:17.952 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-16T07:23:17.952 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-16T07:23:17.952 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:18.029 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:18.159 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:18.167 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:18.167 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-09-16T07:23:18.167 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.1 up 2024-09-16T07:23:18.167 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.1 master ceph-brx 2024-09-16T07:23:18.167 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-09-16T07:23:18.245 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:18.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:18.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:18.333 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-16T07:23:18.370 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-16T07:23:18.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:18.370 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-16T07:23:18.444 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-16T07:23:18.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:18.444 DEBUG:teuthology.orchestra.run.smithi177:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:18.573 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:18.573 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:18.601 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:18.601 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:18.601 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:23:18.601 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: trying mount with new device syntax: 3@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:23:18.601 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:23:18.603 INFO:teuthology.orchestra.run.smithi177.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-16T07:23:18.603 INFO:teuthology.orchestra.run.smithi177.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:23:18.603 INFO:teuthology.orchestra.run.smithi177.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:23:18.603 INFO:teuthology.orchestra.run.smithi177.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:23:18.603 INFO:teuthology.orchestra.run.smithi177.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:23:18.606 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:23:18.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:18.606 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-16T07:23:18.662 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:18.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:18.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:18.752 INFO:teuthology.orchestra.run.smithi177.stdout:client4758 2024-09-16T07:23:18.753 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:23:18.753 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:18.789 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:18.905 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:18.908 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:18.915 INFO:teuthology.orchestra.run.smithi177.stdout:client4758 2024-09-16T07:23:18.915 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:18.993 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:18.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:19.004 INFO:teuthology.orchestra.run.smithi177.stdout:client4758 2024-09-16T07:23:19.004 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5821e025-6a94-4c05-a5f1-d6094eac5c68.client4758/status') 2024-09-16T07:23:19.082 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:19.102 INFO:teuthology.orchestra.run.smithi177.stdout:instance: client.4758 (3)172.21.15.177:0/2596312301 2024-09-16T07:23:19.103 INFO:teuthology.orchestra.run.smithi177.stdout:blocklisted: false 2024-09-16T07:23:19.103 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records in 2024-09-16T07:23:19.103 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records out 2024-09-16T07:23:19.103 INFO:teuthology.orchestra.run.smithi177.stderr:71 bytes copied, 7.4031e-05 s, 959 kB/s 2024-09-16T07:23:19.105 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-16T07:23:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:19.113 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-16T07:23:19.204 INFO:tasks.cephfs_test_runner:Starting test: test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Initiate cloning where clone subvolume and source subvolume are located 2024-09-16T07:23:19.204 INFO:tasks.cephfs_test_runner:test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:23:19.204 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:23:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.373+0000 7fb68b600640 1 -- 172.21.15.71:0/2153165334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 msgr2=0x7fb6840cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.373+0000 7fb68b600640 1 --2- 172.21.15.71:0/2153165334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 0x7fb6840cf130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009ca0 tx=0x7fb67c02aad0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.373+0000 7fb68b600640 1 -- 172.21.15.71:0/2153165334 shutdown_connections 2024-09-16T07:23:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.373+0000 7fb68b600640 1 --2- 172.21.15.71:0/2153165334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 0x7fb6840cf130 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.373+0000 7fb68b600640 1 --2- 172.21.15.71:0/2153165334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6841445d0 0x7fb6840cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.373+0000 7fb68b600640 1 -- 172.21.15.71:0/2153165334 >> 172.21.15.71:0/2153165334 conn(0x7fb6840b1950 msgr2=0x7fb6840b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:19.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.373+0000 7fb68b600640 1 -- 172.21.15.71:0/2153165334 shutdown_connections 2024-09-16T07:23:19.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.373+0000 7fb68b600640 1 -- 172.21.15.71:0/2153165334 wait complete. 2024-09-16T07:23:19.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.375+0000 7fb68b600640 1 Processor -- start 2024-09-16T07:23:19.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.375+0000 7fb68b600640 1 -- start start 2024-09-16T07:23:19.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb68b600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 0x7fb68413dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:19.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb68b600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6840ccfd0 0x7fb68409ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:19.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb68b600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6841445d0 0x7fb68409cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:19.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb68b600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb68415f080 con 0x7fb6841445d0 2024-09-16T07:23:19.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb68b600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb684073bc0 con 0x7fb6840b4ca0 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb68b600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb68415f200 con 0x7fb6840ccfd0 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb689600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6841445d0 0x7fb68409cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb689600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6841445d0 0x7fb68409cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56470/0 (socket says 172.21.15.71:56470) 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb689600640 1 -- 172.21.15.71:0/3488775714 learned_addr learned my addr 172.21.15.71:0/3488775714 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb688c00640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 0x7fb68413dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.376+0000 7fb683e00640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6840ccfd0 0x7fb68409ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb689600640 1 -- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6840ccfd0 msgr2=0x7fb68409ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb689600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6840ccfd0 0x7fb68409ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb689600640 1 -- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 msgr2=0x7fb68413dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb689600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 0x7fb68413dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb689600640 1 -- 172.21.15.71:0/3488775714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb684070570 con 0x7fb6841445d0 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb688c00640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 0x7fb68413dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb689600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6841445d0 0x7fb68409cf50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009ca0 tx=0x7fb67c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:19.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb680c00640 1 -- 172.21.15.71:0/3488775714 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c046070 con 0x7fb6841445d0 2024-09-16T07:23:19.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb680c00640 1 -- 172.21.15.71:0/3488775714 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb67c004040 con 0x7fb6841445d0 2024-09-16T07:23:19.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6841f8280 con 0x7fb6841445d0 2024-09-16T07:23:19.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb680c00640 1 -- 172.21.15.71:0/3488775714 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c004430 con 0x7fb6841445d0 2024-09-16T07:23:19.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.377+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6841f8840 con 0x7fb6841445d0 2024-09-16T07:23:19.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.379+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68415f6b0 con 0x7fb6841445d0 2024-09-16T07:23:19.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.380+0000 7fb680c00640 1 -- 172.21.15.71:0/3488775714 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb67c049070 con 0x7fb6841445d0 2024-09-16T07:23:19.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.380+0000 7fb680c00640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb6640788b0 0x7fb66407ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:19.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.381+0000 7fb680c00640 1 -- 172.21.15.71:0/3488775714 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb67c0bb600 con 0x7fb6841445d0 2024-09-16T07:23:19.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.381+0000 7fb688c00640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb6640788b0 0x7fb66407ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:19.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.382+0000 7fb688c00640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb6640788b0 0x7fb66407ad70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb678009f20 tx=0x7fb678005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:19.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.385+0000 7fb680c00640 1 -- 172.21.15.71:0/3488775714 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb67c032030 con 0x7fb6841445d0 2024-09-16T07:23:19.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.568+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb684142f40 con 0x7fb6841445d0 2024-09-16T07:23:19.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.570+0000 7fb680c00640 1 -- 172.21.15.71:0/3488775714 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7fb67c083580 con 0x7fb6841445d0 2024-09-16T07:23:19.569 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:19.569 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":23,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:50.122046+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:23:19.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb6640788b0 msgr2=0x7fb66407ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb6640788b0 0x7fb66407ad70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb678009f20 tx=0x7fb678005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:19.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6841445d0 msgr2=0x7fb68409cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6841445d0 0x7fb68409cf50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009ca0 tx=0x7fb67c003ce0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 shutdown_connections 2024-09-16T07:23:19.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb6640788b0 0x7fb66407ad70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6841445d0 0x7fb68409cf50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6840ccfd0 0x7fb68409ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 --2- 172.21.15.71:0/3488775714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6840b4ca0 0x7fb68413dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 >> 172.21.15.71:0/3488775714 conn(0x7fb6840b1950 msgr2=0x7fb6840b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:19.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 shutdown_connections 2024-09-16T07:23:19.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.572+0000 7fb68b600640 1 -- 172.21.15.71:0/3488775714 wait complete. 2024-09-16T07:23:19.588 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T07:22:46.797343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T07:22:47.495216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:23:19.588 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:23:19.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.752+0000 7f0793e00640 1 -- 172.21.15.71:0/1323150893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0cd490 msgr2=0x7f078c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.752+0000 7f0793e00640 1 --2- 172.21.15.71:0/1323150893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0cd490 0x7f078c0cdad0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f078c072fb0 tx=0x7f077802a5a0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.752+0000 7f0793e00640 1 -- 172.21.15.71:0/1323150893 shutdown_connections 2024-09-16T07:23:19.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.752+0000 7f0793e00640 1 --2- 172.21.15.71:0/1323150893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0cd490 0x7f078c0cdad0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.752+0000 7f0793e00640 1 -- 172.21.15.71:0/1323150893 >> 172.21.15.71:0/1323150893 conn(0x7f078c0b1840 msgr2=0x7f078c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:19.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.752+0000 7f0793e00640 1 -- 172.21.15.71:0/1323150893 shutdown_connections 2024-09-16T07:23:19.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.752+0000 7f0793e00640 1 -- 172.21.15.71:0/1323150893 wait complete. 2024-09-16T07:23:19.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.754+0000 7f0793e00640 1 Processor -- start 2024-09-16T07:23:19.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.754+0000 7f0793e00640 1 -- start start 2024-09-16T07:23:19.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0793e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0739d0 0x7f078c14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0793e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f078c0cd490 0x7f078c0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0793e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078c1517a0 0x7f078c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0793e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f078c0cc7d0 con 0x7f078c0739d0 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0793e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f078c0cc650 con 0x7f078c1517a0 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0793e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f078c0cc950 con 0x7f078c0cd490 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0739d0 0x7f078c14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0739d0 0x7f078c14e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56492/0 (socket says 172.21.15.71:56492) 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791400640 1 -- 172.21.15.71:0/2709818806 learned_addr learned my addr 172.21.15.71:0/2709818806 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0790a00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f078c0cd490 0x7f078c0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791e00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078c1517a0 0x7f078c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791400640 1 -- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f078c0cd490 msgr2=0x7f078c0b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791400640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f078c0cd490 0x7f078c0b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791400640 1 -- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078c1517a0 msgr2=0x7f078c15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791400640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078c1517a0 0x7f078c15c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.755+0000 7f0791400640 1 -- 172.21.15.71:0/2709818806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078c14eb90 con 0x7f078c0739d0 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.756+0000 7f0791400640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0739d0 0x7f078c14e650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f078000c3a0 tx=0x7f078000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.756+0000 7f0789600640 1 -- 172.21.15.71:0/2709818806 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780017070 con 0x7f078c0739d0 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.756+0000 7f0789600640 1 -- 172.21.15.71:0/2709818806 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07800053c0 con 0x7f078c0739d0 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.756+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078c0b0b30 con 0x7f078c0739d0 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.756+0000 7f0789600640 1 -- 172.21.15.71:0/2709818806 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780003850 con 0x7f078c0739d0 2024-09-16T07:23:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.756+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f078c151d50 con 0x7f078c0739d0 2024-09-16T07:23:19.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.757+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078c0cdad0 con 0x7f078c0739d0 2024-09-16T07:23:19.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.758+0000 7f0789600640 1 -- 172.21.15.71:0/2709818806 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f078001a070 con 0x7f078c0739d0 2024-09-16T07:23:19.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.759+0000 7f0789600640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f076c078850 0x7f076c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.759+0000 7f0789600640 1 -- 172.21.15.71:0/2709818806 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0780099bd0 con 0x7f078c0739d0 2024-09-16T07:23:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.760+0000 7f0790a00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f076c078850 0x7f076c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.760+0000 7f0790a00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f076c078850 0x7f076c07ad10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0778034000 tx=0x7f0778004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:19.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.763+0000 7f0789600640 1 -- 172.21.15.71:0/2709818806 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0780061c80 con 0x7f078c0739d0 2024-09-16T07:23:19.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.948+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f078c0b2430 con 0x7f078c0739d0 2024-09-16T07:23:19.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.948+0000 7f0789600640 1 -- 172.21.15.71:0/2709818806 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f07800666b0 con 0x7f078c0739d0 2024-09-16T07:23:19.948 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:23:19.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.950+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f076c078850 msgr2=0x7f076c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.950+0000 7f0793e00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f076c078850 0x7f076c07ad10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0778034000 tx=0x7f0778004730 comp rx=0 tx=0).stop 2024-09-16T07:23:19.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.950+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0739d0 msgr2=0x7f078c14e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:19.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.950+0000 7f0793e00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0739d0 0x7f078c14e650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f078000c3a0 tx=0x7f078000c860 comp rx=0 tx=0).stop 2024-09-16T07:23:19.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.950+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 shutdown_connections 2024-09-16T07:23:19.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.950+0000 7f0793e00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f076c078850 0x7f076c07ad10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.950+0000 7f0793e00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078c1517a0 0x7f078c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.950+0000 7f0793e00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f078c0cd490 0x7f078c0b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.951+0000 7f0793e00640 1 --2- 172.21.15.71:0/2709818806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c0739d0 0x7f078c14e650 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:19.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.951+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 >> 172.21.15.71:0/2709818806 conn(0x7f078c0b1840 msgr2=0x7f078c15a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:19.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.951+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 shutdown_connections 2024-09-16T07:23:19.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:19.951+0000 7f0793e00640 1 -- 172.21.15.71:0/2709818806 wait complete. 2024-09-16T07:23:19.965 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T07:23:20.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.125+0000 7fbb95000640 1 -- 172.21.15.71:0/2078675501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb901517a0 msgr2=0x7fbb900cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.125+0000 7fbb95000640 1 --2- 172.21.15.71:0/2078675501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb901517a0 0x7fbb900cdad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005fd0 tx=0x7fbb8002a5d0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.125+0000 7fbb95000640 1 -- 172.21.15.71:0/2078675501 shutdown_connections 2024-09-16T07:23:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.125+0000 7fbb95000640 1 --2- 172.21.15.71:0/2078675501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb900cd490 0x7fbb900732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.125+0000 7fbb95000640 1 --2- 172.21.15.71:0/2078675501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb901517a0 0x7fbb900cdad0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.125+0000 7fbb95000640 1 -- 172.21.15.71:0/2078675501 >> 172.21.15.71:0/2078675501 conn(0x7fbb900b1840 msgr2=0x7fbb900b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.126+0000 7fbb95000640 1 -- 172.21.15.71:0/2078675501 shutdown_connections 2024-09-16T07:23:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.126+0000 7fbb95000640 1 -- 172.21.15.71:0/2078675501 wait complete. 2024-09-16T07:23:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.127+0000 7fbb95000640 1 Processor -- start 2024-09-16T07:23:20.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.127+0000 7fbb95000640 1 -- start start 2024-09-16T07:23:20.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb95000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb90073800 0x7fbb9015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb95000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb900cd490 0x7fbb9015c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb95000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb901517a0 0x7fbb901599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb95000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbb900cc650 con 0x7fbb901517a0 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb95000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb90073fc0 con 0x7fbb90073800 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb95000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb900cc7d0 con 0x7fbb900cd490 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb8e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb901517a0 0x7fbb901599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb8e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb901517a0 0x7fbb901599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56504/0 (socket says 172.21.15.71:56504) 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.128+0000 7fbb8e800640 1 -- 172.21.15.71:0/3017667974 learned_addr learned my addr 172.21.15.71:0/3017667974 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.129+0000 7fbb8d400640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb900cd490 0x7fbb9015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.129+0000 7fbb8de00640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb90073800 0x7fbb9015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.129+0000 7fbb8e800640 1 -- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb900cd490 msgr2=0x7fbb9015c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.129+0000 7fbb8e800640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb900cd490 0x7fbb9015c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.129+0000 7fbb8e800640 1 -- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb90073800 msgr2=0x7fbb9015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.129+0000 7fbb8e800640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb90073800 0x7fbb9015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.129+0000 7fbb8e800640 1 -- 172.21.15.71:0/3017667974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9015b2f0 con 0x7fbb901517a0 2024-09-16T07:23:20.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.129+0000 7fbb8e800640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb901517a0 0x7fbb901599a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c7a0 tx=0x7fbb8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:20.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.130+0000 7fbb86000640 1 -- 172.21.15.71:0/3017667974 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88003b70 con 0x7fbb901517a0 2024-09-16T07:23:20.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.130+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb901f61f0 con 0x7fbb901517a0 2024-09-16T07:23:20.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.130+0000 7fbb86000640 1 -- 172.21.15.71:0/3017667974 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb88003d10 con 0x7fbb901517a0 2024-09-16T07:23:20.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.130+0000 7fbb86000640 1 -- 172.21.15.71:0/3017667974 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800aa50 con 0x7fbb901517a0 2024-09-16T07:23:20.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.130+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb901f67b0 con 0x7fbb901517a0 2024-09-16T07:23:20.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.131+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb900ccc80 con 0x7fbb901517a0 2024-09-16T07:23:20.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.132+0000 7fbb86000640 1 -- 172.21.15.71:0/3017667974 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fbb8800abf0 con 0x7fbb901517a0 2024-09-16T07:23:20.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.133+0000 7fbb86000640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb70078850 0x7fbb7007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.133+0000 7fbb86000640 1 -- 172.21.15.71:0/3017667974 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbb8809a5f0 con 0x7fbb901517a0 2024-09-16T07:23:20.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.133+0000 7fbb8de00640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb70078850 0x7fbb7007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.134+0000 7fbb8de00640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb70078850 0x7fbb7007ad10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbb80033000 tx=0x7fbb80004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:20.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.137+0000 7fbb86000640 1 -- 172.21.15.71:0/3017667974 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb88062600 con 0x7fbb901517a0 2024-09-16T07:23:20.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.322+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbb9014ed70 con 0x7fbb901517a0 2024-09-16T07:23:20.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.323+0000 7fbb86000640 1 -- 172.21.15.71:0/3017667974 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fbb88067030 con 0x7fbb901517a0 2024-09-16T07:23:20.322 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-09-16T07:23:20.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb70078850 msgr2=0x7fbb7007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb70078850 0x7fbb7007ad10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbb80033000 tx=0x7fbb80004730 comp rx=0 tx=0).stop 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb901517a0 msgr2=0x7fbb901599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb901517a0 0x7fbb901599a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c7a0 tx=0x7fbb8800cc60 comp rx=0 tx=0).stop 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 shutdown_connections 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb70078850 0x7fbb7007ad10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb901517a0 0x7fbb901599a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb900cd490 0x7fbb9015c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 --2- 172.21.15.71:0/3017667974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb90073800 0x7fbb9015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 >> 172.21.15.71:0/3017667974 conn(0x7fbb900b1840 msgr2=0x7fbb900cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 shutdown_connections 2024-09-16T07:23:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.325+0000 7fbb95000640 1 -- 172.21.15.71:0/3017667974 wait complete. 2024-09-16T07:23:20.340 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T07:23:20.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.501+0000 7f18eee00640 1 -- 172.21.15.71:0/1688918615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cd5c0 msgr2=0x7f18e80739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.501+0000 7f18eee00640 1 --2- 172.21.15.71:0/1688918615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cd5c0 0x7f18e80739c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f18e0005fd0 tx=0x7f18e002a940 comp rx=0 tx=0).stop 2024-09-16T07:23:20.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.501+0000 7f18eee00640 1 -- 172.21.15.71:0/1688918615 shutdown_connections 2024-09-16T07:23:20.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.501+0000 7f18eee00640 1 --2- 172.21.15.71:0/1688918615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cd5c0 0x7f18e80739c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.501+0000 7f18eee00640 1 --2- 172.21.15.71:0/1688918615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e80ca960 0x7f18e806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.501+0000 7f18eee00640 1 -- 172.21.15.71:0/1688918615 >> 172.21.15.71:0/1688918615 conn(0x7f18e80cf9f0 msgr2=0x7f18e80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:20.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.501+0000 7f18eee00640 1 -- 172.21.15.71:0/1688918615 shutdown_connections 2024-09-16T07:23:20.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.501+0000 7f18eee00640 1 -- 172.21.15.71:0/1688918615 wait complete. 2024-09-16T07:23:20.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.502+0000 7f18eee00640 1 Processor -- start 2024-09-16T07:23:20.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.502+0000 7f18eee00640 1 -- start start 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.503+0000 7f18eee00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e80ca960 0x7f18e815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.503+0000 7f18eee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80cd5c0 0x7f18e80c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.503+0000 7f18eee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cf5a0 0x7f18e81436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.503+0000 7f18eee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18e806e720 con 0x7f18e80cf5a0 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.503+0000 7f18eee00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18e806e5a0 con 0x7f18e80cd5c0 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.503+0000 7f18eee00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f18e806e8a0 con 0x7f18e80ca960 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ee400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cf5a0 0x7f18e81436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ee400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cf5a0 0x7f18e81436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56516/0 (socket says 172.21.15.71:56516) 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ee400640 1 -- 172.21.15.71:0/1670546342 learned_addr learned my addr 172.21.15.71:0/1670546342 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:20.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18eda00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e80ca960 0x7f18e815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ed000640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80cd5c0 0x7f18e80c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ee400640 1 -- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e80ca960 msgr2=0x7f18e815ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ee400640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e80ca960 0x7f18e815ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ee400640 1 -- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80cd5c0 msgr2=0x7f18e80c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ee400640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80cd5c0 0x7f18e80c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ee400640 1 -- 172.21.15.71:0/1670546342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e815b280 con 0x7f18e80cf5a0 2024-09-16T07:23:20.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.504+0000 7f18ed000640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80cd5c0 0x7f18e80c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:20.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.505+0000 7f18ee400640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cf5a0 0x7f18e81436f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009cd0 tx=0x7f18e0031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:20.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.505+0000 7f18e5600640 1 -- 172.21.15.71:0/1670546342 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0030430 con 0x7f18e80cf5a0 2024-09-16T07:23:20.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.505+0000 7f18e5600640 1 -- 172.21.15.71:0/1670546342 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18e0004240 con 0x7f18e80cf5a0 2024-09-16T07:23:20.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.505+0000 7f18e5600640 1 -- 172.21.15.71:0/1670546342 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0004650 con 0x7f18e80cf5a0 2024-09-16T07:23:20.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.505+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e81f5a90 con 0x7f18e80cf5a0 2024-09-16T07:23:20.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.505+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e81f6050 con 0x7f18e80cf5a0 2024-09-16T07:23:20.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.506+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e806f840 con 0x7f18e80cf5a0 2024-09-16T07:23:20.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.507+0000 7f18e5600640 1 -- 172.21.15.71:0/1670546342 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f18e0039440 con 0x7f18e80cf5a0 2024-09-16T07:23:20.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.508+0000 7f18e5600640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c8078850 0x7f18c807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.508+0000 7f18e5600640 1 -- 172.21.15.71:0/1670546342 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f18e00bc300 con 0x7f18e80cf5a0 2024-09-16T07:23:20.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.508+0000 7f18eda00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c8078850 0x7f18c807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.509+0000 7f18eda00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c8078850 0x7f18c807ad10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000f60 tx=0x7f18dc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:20.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.512+0000 7f18e5600640 1 -- 172.21.15.71:0/1670546342 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f18e00848d0 con 0x7f18e80cf5a0 2024-09-16T07:23:20.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.699+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18e80d0710 con 0x7f18e80cf5a0 2024-09-16T07:23:20.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.699+0000 7f18e5600640 1 -- 172.21.15.71:0/1670546342 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f18e00841d0 con 0x7f18e80cf5a0 2024-09-16T07:23:20.699 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-09-16T07:23:20.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c8078850 msgr2=0x7f18c807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c8078850 0x7f18c807ad10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000f60 tx=0x7f18dc009040 comp rx=0 tx=0).stop 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cf5a0 msgr2=0x7f18e81436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cf5a0 0x7f18e81436f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009cd0 tx=0x7f18e0031ea0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 shutdown_connections 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c8078850 0x7f18c807ad10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e80cf5a0 0x7f18e81436f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80cd5c0 0x7f18e80c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 --2- 172.21.15.71:0/1670546342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e80ca960 0x7f18e815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 >> 172.21.15.71:0/1670546342 conn(0x7f18e80cf9f0 msgr2=0x7f18e80b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 shutdown_connections 2024-09-16T07:23:20.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.702+0000 7f18eee00640 1 -- 172.21.15.71:0/1670546342 wait complete. 2024-09-16T07:23:20.717 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-16T07:23:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.875+0000 7f73cec00640 1 -- 172.21.15.71:0/1644893465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73ac004d90 msgr2=0x7f73ac005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.875+0000 7f73cec00640 1 --2- 172.21.15.71:0/1644893465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73ac004d90 0x7f73ac005210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73c0001ed0 tx=0x7f73ac005830 comp rx=0 tx=0).stop 2024-09-16T07:23:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.875+0000 7f73ccc00640 1 -- 172.21.15.71:0/1644893465 reap_dead start 2024-09-16T07:23:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.875+0000 7f73cec00640 1 -- 172.21.15.71:0/1644893465 shutdown_connections 2024-09-16T07:23:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.875+0000 7f73cec00640 1 -- 172.21.15.71:0/1644893465 >> 172.21.15.71:0/1644893465 conn(0x7f73c80b1950 msgr2=0x7f73c80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:20.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.875+0000 7f73cec00640 1 -- 172.21.15.71:0/1644893465 shutdown_connections 2024-09-16T07:23:20.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.875+0000 7f73cec00640 1 -- 172.21.15.71:0/1644893465 wait complete. 2024-09-16T07:23:20.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.877+0000 7f73cec00640 1 Processor -- start 2024-09-16T07:23:20.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.877+0000 7f73cec00640 1 -- start start 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73cec00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac004d90 0x7f73c81474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73cec00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c80c4bc0 0x7f73c8147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73cec00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73c80c5040 0x7f73c815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73cec00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73c806f9f0 con 0x7f73c80c4bc0 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73cec00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f73c806f870 con 0x7f73c80c5040 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73cec00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f73c80b2440 con 0x7f73ac004d90 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73c7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c80c4bc0 0x7f73c8147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73c7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c80c4bc0 0x7f73c8147d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56530/0 (socket says 172.21.15.71:56530) 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73c7400640 1 -- 172.21.15.71:0/89714125 learned_addr learned my addr 172.21.15.71:0/89714125 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73ccc00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73c80c5040 0x7f73c815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.878+0000 7f73c7e00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac004d90 0x7f73c81474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73c7400640 1 -- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac004d90 msgr2=0x7f73c81474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73c7400640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac004d90 0x7f73c81474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73c7400640 1 -- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73c80c5040 msgr2=0x7f73c815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73c7400640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73c80c5040 0x7f73c815b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73c7400640 1 -- 172.21.15.71:0/89714125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c815a4a0 con 0x7f73c80c4bc0 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73ccc00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73c80c5040 0x7f73c815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73c7400640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c80c4bc0 0x7f73c8147d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73b4011ae0 tx=0x7f73b4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73bfe00640 1 -- 172.21.15.71:0/89714125 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b401e070 con 0x7f73c80c4bc0 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73bfe00640 1 -- 172.21.15.71:0/89714125 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73b400fe70 con 0x7f73c80c4bc0 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73bfe00640 1 -- 172.21.15.71:0/89714125 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b4004290 con 0x7f73c80c4bc0 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c815a760 con 0x7f73c80c4bc0 2024-09-16T07:23:20.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.879+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c815ad20 con 0x7f73c80c4bc0 2024-09-16T07:23:20.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.881+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c8074440 con 0x7f73c80c4bc0 2024-09-16T07:23:20.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.882+0000 7f73bfe00640 1 -- 172.21.15.71:0/89714125 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f73b4021070 con 0x7f73c80c4bc0 2024-09-16T07:23:20.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.883+0000 7f73bfe00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f73a8078850 0x7f73a807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:20.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.883+0000 7f73bfe00640 1 -- 172.21.15.71:0/89714125 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f73b409b2f0 con 0x7f73c80c4bc0 2024-09-16T07:23:20.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.883+0000 7f73c7e00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f73a8078850 0x7f73a807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:20.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.884+0000 7f73c7e00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f73a8078850 0x7f73a807ad10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73b8001eb0 tx=0x7f73b8001040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:20.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:20.887+0000 7f73bfe00640 1 -- 172.21.15.71:0/89714125 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f73b40633d0 con 0x7f73c80c4bc0 2024-09-16T07:23:21.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.141+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f73c80ae8a0 con 0x7f73c80c4bc0 2024-09-16T07:23:21.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.629+0000 7f73bfe00640 1 -- 172.21.15.71:0/89714125 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v57) ==== 187+0+0 (secure 0 0 0) 0x7f73b4067e00 con 0x7f73c80c4bc0 2024-09-16T07:23:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.631+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f73a8078850 msgr2=0x7f73a807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.631+0000 7f73cec00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f73a8078850 0x7f73a807ad10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73b8001eb0 tx=0x7f73b8001040 comp rx=0 tx=0).stop 2024-09-16T07:23:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.631+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c80c4bc0 msgr2=0x7f73c8147d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.631+0000 7f73cec00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c80c4bc0 0x7f73c8147d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73b4011ae0 tx=0x7f73b4009e90 comp rx=0 tx=0).stop 2024-09-16T07:23:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.632+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 shutdown_connections 2024-09-16T07:23:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.632+0000 7f73cec00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f73a8078850 0x7f73a807ad10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.632+0000 7f73cec00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73c80c5040 0x7f73c815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.632+0000 7f73cec00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73c80c4bc0 0x7f73c8147d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.632+0000 7f73cec00640 1 --2- 172.21.15.71:0/89714125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac004d90 0x7f73c81474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.632+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 >> 172.21.15.71:0/89714125 conn(0x7f73c80b1950 msgr2=0x7f73c8075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.632+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 shutdown_connections 2024-09-16T07:23:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.632+0000 7f73cec00640 1 -- 172.21.15.71:0/89714125 wait complete. 2024-09-16T07:23:21.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:23:21.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.808+0000 7f7254200640 1 -- 172.21.15.71:0/2331278097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c15f5a0 msgr2=0x7f724c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:21.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.808+0000 7f7254200640 1 --2- 172.21.15.71:0/2331278097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c15f5a0 0x7f724c0b2440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7240005fd0 tx=0x7f724002a970 comp rx=0 tx=0).stop 2024-09-16T07:23:21.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.808+0000 7f7254200640 1 -- 172.21.15.71:0/2331278097 shutdown_connections 2024-09-16T07:23:21.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.808+0000 7f7254200640 1 --2- 172.21.15.71:0/2331278097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724c0ccf00 0x7f724c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:21.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.808+0000 7f7254200640 1 --2- 172.21.15.71:0/2331278097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c15f5a0 0x7f724c0b2440 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:21.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.808+0000 7f7254200640 1 -- 172.21.15.71:0/2331278097 >> 172.21.15.71:0/2331278097 conn(0x7f724c0b1950 msgr2=0x7f724c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:21.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.808+0000 7f7254200640 1 -- 172.21.15.71:0/2331278097 shutdown_connections 2024-09-16T07:23:21.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.808+0000 7f7254200640 1 -- 172.21.15.71:0/2331278097 wait complete. 2024-09-16T07:23:21.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.810+0000 7f7254200640 1 Processor -- start 2024-09-16T07:23:21.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.810+0000 7f7254200640 1 -- start start 2024-09-16T07:23:21.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7254200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724c0ad270 0x7f724c14d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7254200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c0ccf00 0x7f724c14f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7254200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f724c15f5a0 0x7f724c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7254200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f724c06f2e0 con 0x7f724c0ccf00 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7254200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f724c06f160 con 0x7f724c15f5a0 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7254200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f724c06f460 con 0x7f724c0ad270 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7250e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c0ccf00 0x7f724c14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7250e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c0ccf00 0x7f724c14f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56544/0 (socket says 172.21.15.71:56544) 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7250e00640 1 -- 172.21.15.71:0/163803607 learned_addr learned my addr 172.21.15.71:0/163803607 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7251800640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724c0ad270 0x7f724c14d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7252200640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f724c15f5a0 0x7f724c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7250e00640 1 -- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724c0ad270 msgr2=0x7f724c14d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7250e00640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724c0ad270 0x7f724c14d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7250e00640 1 -- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f724c15f5a0 msgr2=0x7f724c15e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7250e00640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f724c15f5a0 0x7f724c15e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.811+0000 7f7250e00640 1 -- 172.21.15.71:0/163803607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c14f680 con 0x7f724c0ccf00 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.812+0000 7f7250e00640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c0ccf00 0x7f724c14f140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f723800d770 tx=0x7f723800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.812+0000 7f7249600640 1 -- 172.21.15.71:0/163803607 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238017070 con 0x7f724c0ccf00 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.812+0000 7f7249600640 1 -- 172.21.15.71:0/163803607 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7238009d20 con 0x7f724c0ccf00 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.812+0000 7f7249600640 1 -- 172.21.15.71:0/163803607 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238005280 con 0x7f724c0ccf00 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.812+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c1f68c0 con 0x7f724c0ccf00 2024-09-16T07:23:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.812+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c1f6e80 con 0x7f724c0ccf00 2024-09-16T07:23:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.813+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c06f9f0 con 0x7f724c0ccf00 2024-09-16T07:23:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.814+0000 7f7249600640 1 -- 172.21.15.71:0/163803607 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7238005420 con 0x7f724c0ccf00 2024-09-16T07:23:21.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.815+0000 7f7249600640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f722c078850 0x7f722c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:21.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.815+0000 7f7249600640 1 -- 172.21.15.71:0/163803607 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f723809aa90 con 0x7f724c0ccf00 2024-09-16T07:23:21.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.816+0000 7f7251800640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f722c078850 0x7f722c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:21.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.817+0000 7f7251800640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f722c078850 0x7f722c07ad10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7240035000 tx=0x7f7240004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:21.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:21.819+0000 7f7249600640 1 -- 172.21.15.71:0/163803607 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7238062a10 con 0x7f724c0ccf00 2024-09-16T07:23:22.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.006+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f724c09aa00 con 0x7f724c0ccf00 2024-09-16T07:23:22.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.007+0000 7f7249600640 1 -- 172.21.15.71:0/163803607 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f7238067440 con 0x7f724c0ccf00 2024-09-16T07:23:22.006 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:23:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:23:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:23:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:23:22.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.009+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f722c078850 msgr2=0x7f722c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.009+0000 7f7254200640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f722c078850 0x7f722c07ad10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7240035000 tx=0x7f7240004ab0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.009+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c0ccf00 msgr2=0x7f724c14f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.009+0000 7f7254200640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c0ccf00 0x7f724c14f140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f723800d770 tx=0x7f723800dc30 comp rx=0 tx=0).stop 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.010+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 shutdown_connections 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.010+0000 7f7254200640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f722c078850 0x7f722c07ad10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.010+0000 7f7254200640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f724c15f5a0 0x7f724c15e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.010+0000 7f7254200640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f724c0ccf00 0x7f724c14f140 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.010+0000 7f7254200640 1 --2- 172.21.15.71:0/163803607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724c0ad270 0x7f724c14d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.010+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 >> 172.21.15.71:0/163803607 conn(0x7f724c0b1950 msgr2=0x7f724c0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.010+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 shutdown_connections 2024-09-16T07:23:22.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.010+0000 7f7254200640 1 -- 172.21.15.71:0/163803607 wait complete. 2024-09-16T07:23:22.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:23:22.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.185+0000 7fa191c00640 1 -- 172.21.15.71:0/1309683682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa18c0cccd0 msgr2=0x7fa18c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.185+0000 7fa191c00640 1 --2- 172.21.15.71:0/1309683682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa18c0cccd0 0x7fa18c073440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa178009830 tx=0x7fa17802a480 comp rx=0 tx=0).stop 2024-09-16T07:23:22.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.185+0000 7fa191c00640 1 -- 172.21.15.71:0/1309683682 shutdown_connections 2024-09-16T07:23:22.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.185+0000 7fa191c00640 1 --2- 172.21.15.71:0/1309683682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa18c06efa0 0x7fa18c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.185+0000 7fa191c00640 1 --2- 172.21.15.71:0/1309683682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa18c0cccd0 0x7fa18c073440 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.185+0000 7fa191c00640 1 -- 172.21.15.71:0/1309683682 >> 172.21.15.71:0/1309683682 conn(0x7fa18c0b1840 msgr2=0x7fa18c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:22.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.185+0000 7fa191c00640 1 -- 172.21.15.71:0/1309683682 shutdown_connections 2024-09-16T07:23:22.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.185+0000 7fa191c00640 1 -- 172.21.15.71:0/1309683682 wait complete. 2024-09-16T07:23:22.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.187+0000 7fa191c00640 1 Processor -- start 2024-09-16T07:23:22.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.187+0000 7fa191c00640 1 -- start start 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa191c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa18c06efa0 0x7fa18c1f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa191c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa18c0cccd0 0x7fa18c141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa191c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa18c145750 0x7fa18c0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa191c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c0ce480 con 0x7fa18c145750 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa191c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c0ce300 con 0x7fa18c06efa0 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa191c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa18c0ce600 con 0x7fa18c0cccd0 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa18b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa18c145750 0x7fa18c0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa18b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa18c145750 0x7fa18c0c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56546/0 (socket says 172.21.15.71:56546) 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa18b400640 1 -- 172.21.15.71:0/2672267395 learned_addr learned my addr 172.21.15.71:0/2672267395 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.188+0000 7fa18a000640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa18c0cccd0 0x7fa18c141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa18aa00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa18c06efa0 0x7fa18c1f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa18b400640 1 -- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa18c0cccd0 msgr2=0x7fa18c141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa18b400640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa18c0cccd0 0x7fa18c141570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa18b400640 1 -- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa18c06efa0 msgr2=0x7fa18c1f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa18b400640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa18c06efa0 0x7fa18c1f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa18b400640 1 -- 172.21.15.71:0/2672267395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c141310 con 0x7fa18c145750 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa18b400640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa18c145750 0x7fa18c0c6e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa18400eb10 tx=0x7fa18400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa182a00640 1 -- 172.21.15.71:0/2672267395 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184004b20 con 0x7fa18c145750 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa182a00640 1 -- 172.21.15.71:0/2672267395 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa184004ce0 con 0x7fa18c145750 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18c13fca0 con 0x7fa18c145750 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa182a00640 1 -- 172.21.15.71:0/2672267395 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184005280 con 0x7fa18c145750 2024-09-16T07:23:22.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.189+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18c140260 con 0x7fa18c145750 2024-09-16T07:23:22.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.190+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18c0c4b70 con 0x7fa18c145750 2024-09-16T07:23:22.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.192+0000 7fa182a00640 1 -- 172.21.15.71:0/2672267395 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa184005420 con 0x7fa18c145750 2024-09-16T07:23:22.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.192+0000 7fa182a00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa16c078850 0x7fa16c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:22.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.193+0000 7fa182a00640 1 -- 172.21.15.71:0/2672267395 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa18409b590 con 0x7fa18c145750 2024-09-16T07:23:22.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.193+0000 7fa18aa00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa16c078850 0x7fa16c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:22.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.194+0000 7fa18aa00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa16c078850 0x7fa16c07ad10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa17c001610 tx=0x7fa17c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:22.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.196+0000 7fa182a00640 1 -- 172.21.15.71:0/2672267395 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa18c0c4b70 con 0x7fa18c145750 2024-09-16T07:23:22.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.382+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa18c0cdbb0 con 0x7fa18c145750 2024-09-16T07:23:22.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.385+0000 7fa182a00640 1 -- 172.21.15.71:0/2672267395 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fa18c141310 con 0x7fa18c145750 2024-09-16T07:23:22.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa16c078850 msgr2=0x7fa16c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa16c078850 0x7fa16c07ad10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa17c001610 tx=0x7fa17c009040 comp rx=0 tx=0).stop 2024-09-16T07:23:22.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa18c145750 msgr2=0x7fa18c0c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa18c145750 0x7fa18c0c6e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa18400eb10 tx=0x7fa18400c2d0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 shutdown_connections 2024-09-16T07:23:22.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa16c078850 0x7fa16c07ad10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa18c145750 0x7fa18c0c6e40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa18c0cccd0 0x7fa18c141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 --2- 172.21.15.71:0/2672267395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa18c06efa0 0x7fa18c1f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 >> 172.21.15.71:0/2672267395 conn(0x7fa18c0b1840 msgr2=0x7fa18c0aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:22.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 shutdown_connections 2024-09-16T07:23:22.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.388+0000 7fa191c00640 1 -- 172.21.15.71:0/2672267395 wait complete. 2024-09-16T07:23:22.403 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:23:22.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:23:22.493 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:23:22.583 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:23:22.583 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:22.583 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:22.618 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:22.619 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:22.696 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:22.696 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:22.771 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:23:22.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.968+0000 7f7287000640 1 -- 172.21.15.71:0/1807950524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7280073fb0 msgr2=0x7f7280074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.968+0000 7f7287000640 1 --2- 172.21.15.71:0/1807950524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7280073fb0 0x7f7280074420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7280075130 tx=0x7f727802aa30 comp rx=0 tx=0).stop 2024-09-16T07:23:22.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.968+0000 7f7287000640 1 -- 172.21.15.71:0/1807950524 shutdown_connections 2024-09-16T07:23:22.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.968+0000 7f7287000640 1 --2- 172.21.15.71:0/1807950524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7280073fb0 0x7f7280074420 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.968+0000 7f7287000640 1 --2- 172.21.15.71:0/1807950524 >> v2:172.21.15.177:3300/0 conn(0x7f728015f5a0 0x7f72800b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.968+0000 7f7287000640 1 -- 172.21.15.71:0/1807950524 >> 172.21.15.71:0/1807950524 conn(0x7f72800b1950 msgr2=0x7f72800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:22.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.968+0000 7f7287000640 1 -- 172.21.15.71:0/1807950524 shutdown_connections 2024-09-16T07:23:22.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.968+0000 7f7287000640 1 -- 172.21.15.71:0/1807950524 wait complete. 2024-09-16T07:23:22.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.970+0000 7f7287000640 1 Processor -- start 2024-09-16T07:23:22.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.970+0000 7f7287000640 1 -- start start 2024-09-16T07:23:22.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.970+0000 7f7287000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7280073fb0 0x7f7280148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:22.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.970+0000 7f7287000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800ccf00 0x7f7280148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:22.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7287000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f728015f5a0 0x7f72801f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7287000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f728006f2e0 con 0x7f728015f5a0 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7287000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f728006f160 con 0x7f72800ccf00 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7287000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f728006f460 con 0x7f7280073fb0 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7285000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f728015f5a0 0x7f72801f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7285000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f728015f5a0 0x7f72801f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56564/0 (socket says 172.21.15.71:56564) 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7285000640 1 -- 172.21.15.71:0/2003351391 learned_addr learned my addr 172.21.15.71:0/2003351391 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f727fe00640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7280073fb0 0x7f7280148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f727f400640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800ccf00 0x7f7280148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7285000640 1 -- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7280073fb0 msgr2=0x7f7280148830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7285000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7280073fb0 0x7f7280148830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7285000640 1 -- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800ccf00 msgr2=0x7f7280148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7285000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800ccf00 0x7f7280148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:22.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.971+0000 7f7285000640 1 -- 172.21.15.71:0/2003351391 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7280155d90 con 0x7f728015f5a0 2024-09-16T07:23:22.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.972+0000 7f7285000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f728015f5a0 0x7f72801f8320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f727802b020 tx=0x7f7278003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:22.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.972+0000 7f7277e00640 1 -- 172.21.15.71:0/2003351391 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278046070 con 0x7f728015f5a0 2024-09-16T07:23:22.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.972+0000 7f7277e00640 1 -- 172.21.15.71:0/2003351391 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7278004250 con 0x7f728015f5a0 2024-09-16T07:23:22.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.972+0000 7f7277e00640 1 -- 172.21.15.71:0/2003351391 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278033050 con 0x7f728015f5a0 2024-09-16T07:23:22.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.972+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72800cb360 con 0x7f728015f5a0 2024-09-16T07:23:22.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.972+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72800cb920 con 0x7f728015f5a0 2024-09-16T07:23:22.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.973+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728006f9f0 con 0x7f728015f5a0 2024-09-16T07:23:22.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.974+0000 7f7277e00640 1 -- 172.21.15.71:0/2003351391 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7278049070 con 0x7f728015f5a0 2024-09-16T07:23:22.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.975+0000 7f7277e00640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7260078850 0x7f726007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:22.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.975+0000 7f7277e00640 1 -- 172.21.15.71:0/2003351391 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f72780bb880 con 0x7f728015f5a0 2024-09-16T07:23:22.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.975+0000 7f727fe00640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7260078850 0x7f726007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:22.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.976+0000 7f727fe00640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7260078850 0x7f726007ad10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7270009ac0 tx=0x7f7270005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:22.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:22.979+0000 7f7277e00640 1 -- 172.21.15.71:0/2003351391 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7278083ee0 con 0x7f728015f5a0 2024-09-16T07:23:23.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.173+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f72800b09c0 con 0x7f728015f5a0 2024-09-16T07:23:23.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.173+0000 7f7277e00640 1 -- 172.21.15.71:0/2003351391 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f72780837e0 con 0x7f728015f5a0 2024-09-16T07:23:23.174 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:23:23.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.175+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7260078850 msgr2=0x7f726007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:23.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.175+0000 7f7287000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7260078850 0x7f726007ad10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7270009ac0 tx=0x7f7270005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:23.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.175+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f728015f5a0 msgr2=0x7f72801f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:23.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.175+0000 7f7287000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f728015f5a0 0x7f72801f8320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f727802b020 tx=0x7f7278003f10 comp rx=0 tx=0).stop 2024-09-16T07:23:23.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.176+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 shutdown_connections 2024-09-16T07:23:23.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.176+0000 7f7287000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7260078850 0x7f726007ad10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.176+0000 7f7287000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f728015f5a0 0x7f72801f8320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.176+0000 7f7287000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800ccf00 0x7f7280148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.176+0000 7f7287000640 1 --2- 172.21.15.71:0/2003351391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7280073fb0 0x7f7280148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.176+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 >> 172.21.15.71:0/2003351391 conn(0x7f72800b1950 msgr2=0x7f72800b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:23.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.176+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 shutdown_connections 2024-09-16T07:23:23.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.176+0000 7f7287000640 1 -- 172.21.15.71:0/2003351391 wait complete. 2024-09-16T07:23:23.191 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:23:23.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:23.192 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-16T07:23:23.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:23:23.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:23.254 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:23:23.316 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:23:23.316 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:23.357 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:23:23.357 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:23.435 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:23:23.435 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:23.512 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:23:23.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.673+0000 7f6453000640 1 -- 172.21.15.71:0/539396423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 msgr2=0x7f644c074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:23.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.673+0000 7f6453000640 1 --2- 172.21.15.71:0/539396423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 0x7f644c074460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6444005fd0 tx=0x7f644402aa10 comp rx=0 tx=0).stop 2024-09-16T07:23:23.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.673+0000 7f6453000640 1 -- 172.21.15.71:0/539396423 shutdown_connections 2024-09-16T07:23:23.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.673+0000 7f6453000640 1 --2- 172.21.15.71:0/539396423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 0x7f644c074460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.673+0000 7f6453000640 1 --2- 172.21.15.71:0/539396423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644c15f5a0 0x7f644c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.673+0000 7f6453000640 1 -- 172.21.15.71:0/539396423 >> 172.21.15.71:0/539396423 conn(0x7f644c0b1950 msgr2=0x7f644c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:23.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.673+0000 7f6453000640 1 -- 172.21.15.71:0/539396423 shutdown_connections 2024-09-16T07:23:23.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.673+0000 7f6453000640 1 -- 172.21.15.71:0/539396423 wait complete. 2024-09-16T07:23:23.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.675+0000 7f6453000640 1 Processor -- start 2024-09-16T07:23:23.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.675+0000 7f6453000640 1 -- start start 2024-09-16T07:23:23.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f6453000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 0x7f644c1f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:23.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f6453000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644c0ccf00 0x7f644c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:23.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f6453000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644c15f5a0 0x7f644c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:23.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f6453000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f644c06f2e0 con 0x7f644c0ccf00 2024-09-16T07:23:23.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f6453000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f644c06f160 con 0x7f644c073fb0 2024-09-16T07:23:23.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f6453000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f644c06f460 con 0x7f644c15f5a0 2024-09-16T07:23:23.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f644b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644c0ccf00 0x7f644c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:23.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f6451000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644c15f5a0 0x7f644c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:23.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f644b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644c0ccf00 0x7f644c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56568/0 (socket says 172.21.15.71:56568) 2024-09-16T07:23:23.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f644be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 0x7f644c1f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:23.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.676+0000 7f644b400640 1 -- 172.21.15.71:0/887531201 learned_addr learned my addr 172.21.15.71:0/887531201 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f644b400640 1 -- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644c15f5a0 msgr2=0x7f644c1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f644b400640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644c15f5a0 0x7f644c1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f644b400640 1 -- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 msgr2=0x7f644c1f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f644b400640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 0x7f644c1f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f644b400640 1 -- 172.21.15.71:0/887531201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c145730 con 0x7f644c0ccf00 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f644be00640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 0x7f644c1f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f644b400640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644c0ccf00 0x7f644c0cff20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f643800c420 tx=0x7f643800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f6443e00640 1 -- 172.21.15.71:0/887531201 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438017070 con 0x7f644c0ccf00 2024-09-16T07:23:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f6443e00640 1 -- 172.21.15.71:0/887531201 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64380053c0 con 0x7f644c0ccf00 2024-09-16T07:23:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f6443e00640 1 -- 172.21.15.71:0/887531201 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438003850 con 0x7f644c0ccf00 2024-09-16T07:23:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c146290 con 0x7f644c0ccf00 2024-09-16T07:23:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.677+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c145cd0 con 0x7f644c0ccf00 2024-09-16T07:23:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.678+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c06f9f0 con 0x7f644c0ccf00 2024-09-16T07:23:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.679+0000 7f6443e00640 1 -- 172.21.15.71:0/887531201 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f643801a070 con 0x7f644c0ccf00 2024-09-16T07:23:23.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.681+0000 7f6443e00640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f642c078850 0x7f642c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:23.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.681+0000 7f6443e00640 1 -- 172.21.15.71:0/887531201 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6438099c20 con 0x7f644c0ccf00 2024-09-16T07:23:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.681+0000 7f644be00640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f642c078850 0x7f642c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.682+0000 7f644be00640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f642c078850 0x7f642c07ad10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f643c009f20 tx=0x7f643c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:23.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.686+0000 7f6443e00640 1 -- 172.21.15.71:0/887531201 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6438061cd0 con 0x7f644c0ccf00 2024-09-16T07:23:23.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.883+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f644c0aceb0 con 0x7f644c0ccf00 2024-09-16T07:23:23.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.883+0000 7f6443e00640 1 -- 172.21.15.71:0/887531201 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f6438066700 con 0x7f644c0ccf00 2024-09-16T07:23:23.883 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:23:23.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.885+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f642c078850 msgr2=0x7f642c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:23.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.885+0000 7f6453000640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f642c078850 0x7f642c07ad10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f643c009f20 tx=0x7f643c005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:23.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.885+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644c0ccf00 msgr2=0x7f644c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:23.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.885+0000 7f6453000640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644c0ccf00 0x7f644c0cff20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f643800c420 tx=0x7f643800c8e0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.886+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 shutdown_connections 2024-09-16T07:23:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.886+0000 7f6453000640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f642c078850 0x7f642c07ad10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.886+0000 7f6453000640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644c15f5a0 0x7f644c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.886+0000 7f6453000640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644c0ccf00 0x7f644c0cff20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.886+0000 7f6453000640 1 --2- 172.21.15.71:0/887531201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644c073fb0 0x7f644c1f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.886+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 >> 172.21.15.71:0/887531201 conn(0x7f644c0b1950 msgr2=0x7f644c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.886+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 shutdown_connections 2024-09-16T07:23:23.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:23.886+0000 7f6453000640 1 -- 172.21.15.71:0/887531201 wait complete. 2024-09-16T07:23:23.901 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T07:23:23.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:23.901 DEBUG:teuthology.orchestra.run.smithi177:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-16T07:23:23.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:23:23.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:23.966 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T07:23:24.019 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:24.020 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:24.055 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:24.055 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:24.132 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:24.132 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:24.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:23:24.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 -- 172.21.15.71:0/4179964787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c0ccf00 msgr2=0x7f300c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:24.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 --2- 172.21.15.71:0/4179964787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c0ccf00 0x7f300c0acd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f300c075130 tx=0x7f2ff802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 -- 172.21.15.71:0/4179964787 shutdown_connections 2024-09-16T07:23:24.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 --2- 172.21.15.71:0/4179964787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300c0ad270 0x7f300c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 --2- 172.21.15.71:0/4179964787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c0ccf00 0x7f300c0acd30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 --2- 172.21.15.71:0/4179964787 >> v2:172.21.15.177:3300/0 conn(0x7f300c15f5a0 0x7f300c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 -- 172.21.15.71:0/4179964787 >> 172.21.15.71:0/4179964787 conn(0x7f300c0b1950 msgr2=0x7f300c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:24.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 -- 172.21.15.71:0/4179964787 shutdown_connections 2024-09-16T07:23:24.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.405+0000 7f3013a00640 1 -- 172.21.15.71:0/4179964787 wait complete. 2024-09-16T07:23:24.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.407+0000 7f3013a00640 1 Processor -- start 2024-09-16T07:23:24.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.407+0000 7f3013a00640 1 -- start start 2024-09-16T07:23:24.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3013a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f300c0ad270 0x7f300c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3013a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300c0ccf00 0x7f300c15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3013a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c15f5a0 0x7f300c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3013a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f300c074880 con 0x7f300c15f5a0 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3013a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f300c074700 con 0x7f300c0ad270 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3013a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f300c06f160 con 0x7f300c0ccf00 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3011a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c15f5a0 0x7f300c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3011000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f300c0ad270 0x7f300c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f300be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300c0ccf00 0x7f300c15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3011000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f300c0ad270 0x7f300c1f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:53122/0 (socket says 172.21.15.71:53122) 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f300be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300c0ccf00 0x7f300c15b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:54228/0 (socket says 172.21.15.71:54228) 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.408+0000 7f3011000640 1 -- 172.21.15.71:0/2815240694 learned_addr learned my addr 172.21.15.71:0/2815240694 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3011a00640 1 -- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300c0ccf00 msgr2=0x7f300c15b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:24.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3011a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300c0ccf00 0x7f300c15b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3011a00640 1 -- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f300c0ad270 msgr2=0x7f300c1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3011a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f300c0ad270 0x7f300c1f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3011a00640 1 -- 172.21.15.71:0/2815240694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c15bd10 con 0x7f300c15f5a0 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3011000640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f300c0ad270 0x7f300c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f300be00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300c0ccf00 0x7f300c15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3011a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c15f5a0 0x7f300c1598e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f300400e960 tx=0x7f300400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3008c00640 1 -- 172.21.15.71:0/2815240694 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004004d20 con 0x7f300c15f5a0 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3008c00640 1 -- 172.21.15.71:0/2815240694 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f300400cb20 con 0x7f300c15f5a0 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3008c00640 1 -- 172.21.15.71:0/2815240694 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004005020 con 0x7f300c15f5a0 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c09d010 con 0x7f300c15f5a0 2024-09-16T07:23:24.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.409+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c15afd0 con 0x7f300c15f5a0 2024-09-16T07:23:24.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.410+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c073fb0 con 0x7f300c15f5a0 2024-09-16T07:23:24.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.411+0000 7f3008c00640 1 -- 172.21.15.71:0/2815240694 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f30040051c0 con 0x7f300c15f5a0 2024-09-16T07:23:24.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.412+0000 7f3008c00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fec078850 0x7f2fec07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:24.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.412+0000 7f3008c00640 1 -- 172.21.15.71:0/2815240694 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f300409b220 con 0x7f300c15f5a0 2024-09-16T07:23:24.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.413+0000 7f3011000640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fec078850 0x7f2fec07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:24.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.414+0000 7f3011000640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fec078850 0x7f2fec07ad10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0058c0 tx=0x7f2ffc00b410 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:24.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.417+0000 7f3008c00640 1 -- 172.21.15.71:0/2815240694 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3004016030 con 0x7f300c15f5a0 2024-09-16T07:23:24.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.611+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f300c0ae900 con 0x7f300c15f5a0 2024-09-16T07:23:24.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.611+0000 7f3008c00640 1 -- 172.21.15.71:0/2815240694 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f3004063120 con 0x7f300c15f5a0 2024-09-16T07:23:24.611 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fec078850 msgr2=0x7f2fec07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fec078850 0x7f2fec07ad10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0058c0 tx=0x7f2ffc00b410 comp rx=0 tx=0).stop 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c15f5a0 msgr2=0x7f300c1598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c15f5a0 0x7f300c1598e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f300400e960 tx=0x7f300400ee20 comp rx=0 tx=0).stop 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 shutdown_connections 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fec078850 0x7f2fec07ad10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300c15f5a0 0x7f300c1598e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300c0ccf00 0x7f300c15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 --2- 172.21.15.71:0/2815240694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f300c0ad270 0x7f300c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 >> 172.21.15.71:0/2815240694 conn(0x7f300c0b1950 msgr2=0x7f300c0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 shutdown_connections 2024-09-16T07:23:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:24.614+0000 7f3013a00640 1 -- 172.21.15.71:0/2815240694 wait complete. 2024-09-16T07:23:24.629 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-16T07:23:24.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:24.629 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-16T07:23:24.692 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:23:24.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:24.693 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-16T07:23:24.728 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:23:24.729 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:24.765 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:23:24.766 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:24.841 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:23:24.841 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:24.917 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:23:25.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.076+0000 7fec32800640 1 -- 172.21.15.71:0/1467077901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c161440 msgr2=0x7fec2c0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.076+0000 7fec32800640 1 --2- 172.21.15.71:0/1467077901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c161440 0x7fec2c0b5370 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0b3440 tx=0x7fec2002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.076+0000 7fec32800640 1 -- 172.21.15.71:0/1467077901 shutdown_connections 2024-09-16T07:23:25.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.076+0000 7fec32800640 1 --2- 172.21.15.71:0/1467077901 >> v2:172.21.15.177:3300/0 conn(0x7fec2c0b3e50 0x7fec2c0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.076+0000 7fec32800640 1 --2- 172.21.15.71:0/1467077901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec2c0b34b0 0x7fec2c0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.076+0000 7fec32800640 1 --2- 172.21.15.71:0/1467077901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c161440 0x7fec2c0b5370 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.077+0000 7fec32800640 1 -- 172.21.15.71:0/1467077901 >> 172.21.15.71:0/1467077901 conn(0x7fec2c0d3df0 msgr2=0x7fec2c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:25.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.077+0000 7fec32800640 1 -- 172.21.15.71:0/1467077901 shutdown_connections 2024-09-16T07:23:25.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.077+0000 7fec32800640 1 -- 172.21.15.71:0/1467077901 wait complete. 2024-09-16T07:23:25.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.078+0000 7fec32800640 1 Processor -- start 2024-09-16T07:23:25.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.078+0000 7fec32800640 1 -- start start 2024-09-16T07:23:25.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec32800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c0b34b0 0x7fec2c149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec32800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec2c0b3e50 0x7fec2c0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec32800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c161440 0x7fec2c0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec32800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c0d2420 con 0x7fec2c0b34b0 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec32800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c0d22a0 con 0x7fec2c161440 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec32800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c0d25a0 con 0x7fec2c0b3e50 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec31400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c0b34b0 0x7fec2c149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec31400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c0b34b0 0x7fec2c149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49558/0 (socket says 172.21.15.71:49558) 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.079+0000 7fec31400640 1 -- 172.21.15.71:0/3776692557 learned_addr learned my addr 172.21.15.71:0/3776692557 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec30a00640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec2c0b3e50 0x7fec2c0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec31e00640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c161440 0x7fec2c0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec31400640 1 -- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec2c0b3e50 msgr2=0x7fec2c0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec31400640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec2c0b3e50 0x7fec2c0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec31400640 1 -- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c161440 msgr2=0x7fec2c0cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec31400640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c161440 0x7fec2c0cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec31400640 1 -- 172.21.15.71:0/3776692557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c1f83b0 con 0x7fec2c0b34b0 2024-09-16T07:23:25.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec31e00640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c161440 0x7fec2c0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:25.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec31400640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c0b34b0 0x7fec2c149b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec20000c00 tx=0x7fec20003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:25.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec29600640 1 -- 172.21.15.71:0/3776692557 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2003e030 con 0x7fec2c0b34b0 2024-09-16T07:23:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec29600640 1 -- 172.21.15.71:0/3776692557 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec20003e00 con 0x7fec2c0b34b0 2024-09-16T07:23:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c1f8690 con 0x7fec2c0b34b0 2024-09-16T07:23:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.080+0000 7fec29600640 1 -- 172.21.15.71:0/3776692557 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20035040 con 0x7fec2c0b34b0 2024-09-16T07:23:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.081+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c1f8c50 con 0x7fec2c0b34b0 2024-09-16T07:23:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.082+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2c160ca0 con 0x7fec2c0b34b0 2024-09-16T07:23:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.083+0000 7fec29600640 1 -- 172.21.15.71:0/3776692557 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fec20041030 con 0x7fec2c0b34b0 2024-09-16T07:23:25.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.084+0000 7fec29600640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec0c078880 0x7fec0c07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.084+0000 7fec29600640 1 -- 172.21.15.71:0/3776692557 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fec200baf30 con 0x7fec2c0b34b0 2024-09-16T07:23:25.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.084+0000 7fec30a00640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec0c078880 0x7fec0c07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.085+0000 7fec30a00640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec0c078880 0x7fec0c07ad40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec2c158d20 tx=0x7fec1800b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:25.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.088+0000 7fec29600640 1 -- 172.21.15.71:0/3776692557 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fec2c160ca0 con 0x7fec2c0b34b0 2024-09-16T07:23:25.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.282+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fec2c0d2b30 con 0x7fec2c0b34b0 2024-09-16T07:23:25.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.283+0000 7fec29600640 1 -- 172.21.15.71:0/3776692557 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fec20082f40 con 0x7fec2c0b34b0 2024-09-16T07:23:25.282 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:23:25.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec0c078880 msgr2=0x7fec0c07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec0c078880 0x7fec0c07ad40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec2c158d20 tx=0x7fec1800b2e0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c0b34b0 msgr2=0x7fec2c149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c0b34b0 0x7fec2c149b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec20000c00 tx=0x7fec20003910 comp rx=0 tx=0).stop 2024-09-16T07:23:25.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 shutdown_connections 2024-09-16T07:23:25.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec0c078880 0x7fec0c07ad40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec2c161440 0x7fec2c0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec2c0b3e50 0x7fec2c0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 --2- 172.21.15.71:0/3776692557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec2c0b34b0 0x7fec2c149b70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 >> 172.21.15.71:0/3776692557 conn(0x7fec2c0d3df0 msgr2=0x7fec2c0b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:25.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 shutdown_connections 2024-09-16T07:23:25.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.285+0000 7fec32800640 1 -- 172.21.15.71:0/3776692557 wait complete. 2024-09-16T07:23:25.300 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-16T07:23:25.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:25.301 DEBUG:teuthology.orchestra.run.smithi177:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-16T07:23:25.362 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:23:25.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:25.362 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-16T07:23:25.398 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:23:25.398 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:25.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.555+0000 7f1cc6e00640 1 -- 172.21.15.71:0/549809586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b5370 msgr2=0x7f1cc00b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.555+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/549809586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b5370 0x7f1cc00b3bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1cac005fd0 tx=0x7f1cac02a540 comp rx=0 tx=0).stop 2024-09-16T07:23:25.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.555+0000 7f1cc6e00640 1 -- 172.21.15.71:0/549809586 shutdown_connections 2024-09-16T07:23:25.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.555+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/549809586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cc00b4100 0x7f1cc00b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.555+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/549809586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b5370 0x7f1cc00b3bc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.555+0000 7f1cc6e00640 1 -- 172.21.15.71:0/549809586 >> 172.21.15.71:0/549809586 conn(0x7f1cc00d3df0 msgr2=0x7f1cc00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:25.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.555+0000 7f1cc6e00640 1 -- 172.21.15.71:0/549809586 shutdown_connections 2024-09-16T07:23:25.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.555+0000 7f1cc6e00640 1 -- 172.21.15.71:0/549809586 wait complete. 2024-09-16T07:23:25.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.557+0000 7f1cc6e00640 1 Processor -- start 2024-09-16T07:23:25.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.557+0000 7f1cc6e00640 1 -- start start 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.557+0000 7f1cc6e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b4100 0x7f1cc00c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.557+0000 7f1cc6e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc00b5370 0x7f1cc0157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.557+0000 7f1cc6e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cc0161440 0x7f1cc00b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc6e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00b2040 con 0x7f1cc00b4100 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc6e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00b1ec0 con 0x7f1cc00b5370 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc6e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc00b21c0 con 0x7f1cc0161440 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b4100 0x7f1cc00c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b4100 0x7f1cc00c9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49568/0 (socket says 172.21.15.71:49568) 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5a00640 1 -- 172.21.15.71:0/84492848 learned_addr learned my addr 172.21.15.71:0/84492848 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5000640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc00b5370 0x7f1cc0157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc6400640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cc0161440 0x7f1cc00b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5a00640 1 -- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cc0161440 msgr2=0x7f1cc00b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5a00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cc0161440 0x7f1cc00b9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5a00640 1 -- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc00b5370 msgr2=0x7f1cc0157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5a00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc00b5370 0x7f1cc0157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.558+0000 7f1cc5a00640 1 -- 172.21.15.71:0/84492848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc00cc5b0 con 0x7f1cc00b4100 2024-09-16T07:23:25.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.559+0000 7f1cc5a00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b4100 0x7f1cc00c9940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c370 tx=0x7f1cb400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:25.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.559+0000 7f1cbd600640 1 -- 172.21.15.71:0/84492848 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40052d0 con 0x7f1cc00b4100 2024-09-16T07:23:25.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.559+0000 7f1cbd600640 1 -- 172.21.15.71:0/84492848 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb4007020 con 0x7f1cc00b4100 2024-09-16T07:23:25.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.559+0000 7f1cbd600640 1 -- 172.21.15.71:0/84492848 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4007470 con 0x7f1cc00b4100 2024-09-16T07:23:25.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.559+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc00cc890 con 0x7f1cc00b4100 2024-09-16T07:23:25.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.559+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc00cce50 con 0x7f1cc00b4100 2024-09-16T07:23:25.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.560+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc00b1770 con 0x7f1cc00b4100 2024-09-16T07:23:25.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.561+0000 7f1cbd600640 1 -- 172.21.15.71:0/84492848 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1cb40067d0 con 0x7f1cc00b4100 2024-09-16T07:23:25.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.562+0000 7f1cbd600640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ca0078850 0x7f1ca007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.562+0000 7f1cbd600640 1 -- 172.21.15.71:0/84492848 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1cb4099d90 con 0x7f1cc00b4100 2024-09-16T07:23:25.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.562+0000 7f1cc5000640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ca0078850 0x7f1ca007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.563+0000 7f1cc5000640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ca0078850 0x7f1ca007ad10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1cac033000 tx=0x7f1cac004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:25.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.566+0000 7f1cbd600640 1 -- 172.21.15.71:0/84492848 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1cb4061d20 con 0x7f1cc00b4100 2024-09-16T07:23:25.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.788+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cc015fd20 con 0x7f1cc00b4100 2024-09-16T07:23:25.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.789+0000 7f1cbd600640 1 -- 172.21.15.71:0/84492848 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f1cb4066750 con 0x7f1cc00b4100 2024-09-16T07:23:25.789 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:25.789 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:22:48.541520+0000","modified":"2024-09-16T07:22:51.551572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4404":{"gid":4404,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4404,"qdb_cluster":[4404]},"id":1}]} 2024-09-16T07:23:25.789 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-09-16T07:23:25.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ca0078850 msgr2=0x7f1ca007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ca0078850 0x7f1ca007ad10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1cac033000 tx=0x7f1cac004730 comp rx=0 tx=0).stop 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b4100 msgr2=0x7f1cc00c9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b4100 0x7f1cc00c9940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c370 tx=0x7f1cb400c830 comp rx=0 tx=0).stop 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 shutdown_connections 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ca0078850 0x7f1ca007ad10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cc0161440 0x7f1cc00b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cc00b5370 0x7f1cc0157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 --2- 172.21.15.71:0/84492848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc00b4100 0x7f1cc00c9940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 >> 172.21.15.71:0/84492848 conn(0x7f1cc00d3df0 msgr2=0x7f1cc00d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 shutdown_connections 2024-09-16T07:23:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.792+0000 7f1cc6e00640 1 -- 172.21.15.71:0/84492848 wait complete. 2024-09-16T07:23:25.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:25.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.963+0000 7f30b2c00640 1 -- 172.21.15.71:0/2911440765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ccf00 msgr2=0x7f30ac0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.963+0000 7f30b2c00640 1 --2- 172.21.15.71:0/2911440765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ccf00 0x7f30ac0acd30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3098005fd0 tx=0x7f309802a540 comp rx=0 tx=0).stop 2024-09-16T07:23:25.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.963+0000 7f30b2c00640 1 -- 172.21.15.71:0/2911440765 shutdown_connections 2024-09-16T07:23:25.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.963+0000 7f30b2c00640 1 --2- 172.21.15.71:0/2911440765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ccf00 0x7f30ac0acd30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.963+0000 7f30b2c00640 1 --2- 172.21.15.71:0/2911440765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ac15f5a0 0x7f30ac0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.963+0000 7f30b2c00640 1 -- 172.21.15.71:0/2911440765 >> 172.21.15.71:0/2911440765 conn(0x7f30ac0b1950 msgr2=0x7f30ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:25.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.964+0000 7f30b2c00640 1 -- 172.21.15.71:0/2911440765 shutdown_connections 2024-09-16T07:23:25.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.964+0000 7f30b2c00640 1 -- 172.21.15.71:0/2911440765 wait complete. 2024-09-16T07:23:25.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.965+0000 7f30b2c00640 1 Processor -- start 2024-09-16T07:23:25.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.965+0000 7f30b2c00640 1 -- start start 2024-09-16T07:23:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30b2c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ad270 0x7f30ac1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30b2c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ac0ccf00 0x7f30ac148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30b2c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ac15f5a0 0x7f30ac09cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30b2c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac06f2e0 con 0x7f30ac0ad270 2024-09-16T07:23:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30b2c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac06f160 con 0x7f30ac0ccf00 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30b2c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30ac06f460 con 0x7f30ac15f5a0 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ad270 0x7f30ac1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ad270 0x7f30ac1f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49582/0 (socket says 172.21.15.71:49582) 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.966+0000 7f30abe00640 1 -- 172.21.15.71:0/3722718314 learned_addr learned my addr 172.21.15.71:0/3722718314 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30b0c00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ac15f5a0 0x7f30ac09cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30ab400640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ac0ccf00 0x7f30ac148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30abe00640 1 -- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ac15f5a0 msgr2=0x7f30ac09cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30abe00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ac15f5a0 0x7f30ac09cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30abe00640 1 -- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ac0ccf00 msgr2=0x7f30ac148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30abe00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ac0ccf00 0x7f30ac148450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30abe00640 1 -- 172.21.15.71:0/3722718314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac1563c0 con 0x7f30ac0ad270 2024-09-16T07:23:25.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30abe00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ad270 0x7f30ac1f4c60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f309c00e9f0 tx=0x7f309c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30a3e00640 1 -- 172.21.15.71:0/3722718314 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c00cd80 con 0x7f30ac0ad270 2024-09-16T07:23:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30a3e00640 1 -- 172.21.15.71:0/3722718314 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f309c004c10 con 0x7f30ac0ad270 2024-09-16T07:23:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac0cf360 con 0x7f30ac0ad270 2024-09-16T07:23:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.967+0000 7f30a3e00640 1 -- 172.21.15.71:0/3722718314 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c005020 con 0x7f30ac0ad270 2024-09-16T07:23:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.968+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac156090 con 0x7f30ac0ad270 2024-09-16T07:23:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.968+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ac06f980 con 0x7f30ac0ad270 2024-09-16T07:23:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.969+0000 7f30a3e00640 1 -- 172.21.15.71:0/3722718314 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f309c0051c0 con 0x7f30ac0ad270 2024-09-16T07:23:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.970+0000 7f30a3e00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f308c078850 0x7f308c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.970+0000 7f30a3e00640 1 -- 172.21.15.71:0/3722718314 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f309c09b2e0 con 0x7f30ac0ad270 2024-09-16T07:23:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.971+0000 7f30ab400640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f308c078850 0x7f308c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.971+0000 7f30ab400640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f308c078850 0x7f308c07ad10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3098033000 tx=0x7f3098004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:25.974+0000 7f30a3e00640 1 -- 172.21.15.71:0/3722718314 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f309c016030 con 0x7f30ac0ad270 2024-09-16T07:23:26.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.193+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30ac0b2d50 con 0x7f30ac0ad270 2024-09-16T07:23:26.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.194+0000 7f30a3e00640 1 -- 172.21.15.71:0/3722718314 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f309c063940 con 0x7f30ac0ad270 2024-09-16T07:23:26.193 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:26.193 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:22:48.541520+0000","modified":"2024-09-16T07:22:51.551572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4404":{"gid":4404,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4404,"qdb_cluster":[4404]},"id":1}]} 2024-09-16T07:23:26.194 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-09-16T07:23:26.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f308c078850 msgr2=0x7f308c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f308c078850 0x7f308c07ad10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3098033000 tx=0x7f3098004730 comp rx=0 tx=0).stop 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ad270 msgr2=0x7f30ac1f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ad270 0x7f30ac1f4c60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f309c00e9f0 tx=0x7f309c00eeb0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 shutdown_connections 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f308c078850 0x7f308c07ad10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ac15f5a0 0x7f30ac09cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ac0ccf00 0x7f30ac148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 --2- 172.21.15.71:0/3722718314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30ac0ad270 0x7f30ac1f4c60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 >> 172.21.15.71:0/3722718314 conn(0x7f30ac0b1950 msgr2=0x7f30ac0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 shutdown_connections 2024-09-16T07:23:26.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.196+0000 7f30b2c00640 1 -- 172.21.15.71:0/3722718314 wait complete. 2024-09-16T07:23:26.211 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:23:26.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.367+0000 7f0d24c00640 1 -- 172.21.15.71:0/2783724020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 msgr2=0x7f0d200b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.367+0000 7f0d24c00640 1 --2- 172.21.15.71:0/2783724020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 0x7f0d200b3bc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0d200d18c0 tx=0x7f0d0c02a560 comp rx=0 tx=0).stop 2024-09-16T07:23:26.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.368+0000 7f0d24c00640 1 -- 172.21.15.71:0/2783724020 shutdown_connections 2024-09-16T07:23:26.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.368+0000 7f0d24c00640 1 --2- 172.21.15.71:0/2783724020 >> v2:172.21.15.177:3300/0 conn(0x7f0d200b4100 0x7f0d200b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.368+0000 7f0d24c00640 1 --2- 172.21.15.71:0/2783724020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 0x7f0d200b3bc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.368+0000 7f0d24c00640 1 -- 172.21.15.71:0/2783724020 >> 172.21.15.71:0/2783724020 conn(0x7f0d200d3df0 msgr2=0x7f0d200d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:26.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.368+0000 7f0d24c00640 1 -- 172.21.15.71:0/2783724020 shutdown_connections 2024-09-16T07:23:26.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.368+0000 7f0d24c00640 1 -- 172.21.15.71:0/2783724020 wait complete. 2024-09-16T07:23:26.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.369+0000 7f0d24c00640 1 Processor -- start 2024-09-16T07:23:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.369+0000 7f0d24c00640 1 -- start start 2024-09-16T07:23:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.370+0000 7f0d24c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d200b4100 0x7f0d201f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.370+0000 7f0d24c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 0x7f0d200cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.370+0000 7f0d24c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d20161440 0x7f0d200c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.370+0000 7f0d24c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200b2040 con 0x7f0d200b5370 2024-09-16T07:23:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.370+0000 7f0d24c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200b1ec0 con 0x7f0d200b4100 2024-09-16T07:23:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.370+0000 7f0d24c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d200b21c0 con 0x7f0d20161440 2024-09-16T07:23:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 0x7f0d200cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 0x7f0d200cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49596/0 (socket says 172.21.15.71:49596) 2024-09-16T07:23:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 -- 172.21.15.71:0/4175713166 learned_addr learned my addr 172.21.15.71:0/4175713166 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1fe00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d20161440 0x7f0d200c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1f400640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d200b4100 0x7f0d201f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 -- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d20161440 msgr2=0x7f0d200c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d20161440 0x7f0d200c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 -- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d200b4100 msgr2=0x7f0d201f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d200b4100 0x7f0d201f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 -- 172.21.15.71:0/4175713166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d200ca500 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.371+0000 7f0d1ea00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 0x7f0d200cfed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c02ab50 tx=0x7f0d0c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.372+0000 7f0d17400640 1 -- 172.21.15.71:0/4175713166 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c03f070 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.372+0000 7f0d17400640 1 -- 172.21.15.71:0/4175713166 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d0c0047a0 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.372+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d200ca7e0 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.372+0000 7f0d17400640 1 -- 172.21.15.71:0/4175713166 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035030 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.372+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d200ca1f0 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.373+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d200b1770 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.374+0000 7f0d17400640 1 -- 172.21.15.71:0/4175713166 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0d0c042070 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.375+0000 7f0d17400640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d00078850 0x7f0d0007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.375+0000 7f0d17400640 1 -- 172.21.15.71:0/4175713166 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0d0c0bb650 con 0x7f0d200b5370 2024-09-16T07:23:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.375+0000 7f0d1f400640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d00078850 0x7f0d0007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.376+0000 7f0d1f400640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d00078850 0x7f0d0007ad10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d100015e0 tx=0x7f0d10009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.379+0000 7f0d17400640 1 -- 172.21.15.71:0/4175713166 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0d0c083660 con 0x7f0d200b5370 2024-09-16T07:23:26.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.564+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d2006f0c0 con 0x7f0d200b5370 2024-09-16T07:23:26.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.566+0000 7f0d17400640 1 -- 172.21.15.71:0/4175713166 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7f0d0c088090 con 0x7f0d200b5370 2024-09-16T07:23:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:26.566 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":23,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:50.122046+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:23:26.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d00078850 msgr2=0x7f0d0007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d00078850 0x7f0d0007ad10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d100015e0 tx=0x7f0d10009040 comp rx=0 tx=0).stop 2024-09-16T07:23:26.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 msgr2=0x7f0d200cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 0x7f0d200cfed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c02ab50 tx=0x7f0d0c0038a0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 shutdown_connections 2024-09-16T07:23:26.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d00078850 0x7f0d0007ad10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d20161440 0x7f0d200c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d200b5370 0x7f0d200cfed0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 --2- 172.21.15.71:0/4175713166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d200b4100 0x7f0d201f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 >> 172.21.15.71:0/4175713166 conn(0x7f0d200d3df0 msgr2=0x7f0d2006e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:26.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 shutdown_connections 2024-09-16T07:23:26.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.568+0000 7f0d24c00640 1 -- 172.21.15.71:0/4175713166 wait complete. 2024-09-16T07:23:26.585 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:23:26.585 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.742+0000 7fc13aa00640 1 -- 172.21.15.71:0/640225513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340b2440 msgr2=0x7fc13406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.742+0000 7fc13aa00640 1 --2- 172.21.15.71:0/640225513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340b2440 0x7fc13406f160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009860 tx=0x7fc12c02a4a0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.743+0000 7fc13aa00640 1 -- 172.21.15.71:0/640225513 shutdown_connections 2024-09-16T07:23:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.743+0000 7fc13aa00640 1 --2- 172.21.15.71:0/640225513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340b2440 0x7fc13406f160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.743+0000 7fc13aa00640 1 -- 172.21.15.71:0/640225513 >> 172.21.15.71:0/640225513 conn(0x7fc1340b1950 msgr2=0x7fc1340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.743+0000 7fc13aa00640 1 -- 172.21.15.71:0/640225513 shutdown_connections 2024-09-16T07:23:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.743+0000 7fc13aa00640 1 -- 172.21.15.71:0/640225513 wait complete. 2024-09-16T07:23:26.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.744+0000 7fc13aa00640 1 Processor -- start 2024-09-16T07:23:26.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.744+0000 7fc13aa00640 1 -- start start 2024-09-16T07:23:26.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.745+0000 7fc13aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b2440 0x7fc1340cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:26.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.745+0000 7fc13aa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340ccf00 0x7fc1340cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:26.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.745+0000 7fc13aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc13415f5a0 0x7fc13415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.745+0000 7fc13aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc134074840 con 0x7fc1340b2440 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.745+0000 7fc13aa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1340746c0 con 0x7fc13415f5a0 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.745+0000 7fc13aa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc13406f6a0 con 0x7fc1340ccf00 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b2440 0x7fc1340cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b2440 0x7fc1340cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49604/0 (socket says 172.21.15.71:49604) 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 -- 172.21.15.71:0/3208799070 learned_addr learned my addr 172.21.15.71:0/3208799070 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc138a00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc13415f5a0 0x7fc13415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133400640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340ccf00 0x7fc1340cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 -- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340ccf00 msgr2=0x7fc1340cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340ccf00 0x7fc1340cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 -- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc13415f5a0 msgr2=0x7fc13415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc13415f5a0 0x7fc13415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 -- 172.21.15.71:0/3208799070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc134146530 con 0x7fc1340b2440 2024-09-16T07:23:26.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133400640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340ccf00 0x7fc1340cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:26.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc133e00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b2440 0x7fc1340cc0c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c730 tx=0x7fc12400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:26.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.746+0000 7fc12be00640 1 -- 172.21.15.71:0/3208799070 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124003b70 con 0x7fc1340b2440 2024-09-16T07:23:26.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.747+0000 7fc12be00640 1 -- 172.21.15.71:0/3208799070 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc124003d30 con 0x7fc1340b2440 2024-09-16T07:23:26.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.747+0000 7fc12be00640 1 -- 172.21.15.71:0/3208799070 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12400aab0 con 0x7fc1340b2440 2024-09-16T07:23:26.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.747+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc134156f80 con 0x7fc1340b2440 2024-09-16T07:23:26.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.747+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc134157820 con 0x7fc1340b2440 2024-09-16T07:23:26.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.748+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc134157370 con 0x7fc1340b2440 2024-09-16T07:23:26.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.749+0000 7fc12be00640 1 -- 172.21.15.71:0/3208799070 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc12400ac50 con 0x7fc1340b2440 2024-09-16T07:23:26.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.750+0000 7fc12be00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc114078850 0x7fc11407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:26.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.750+0000 7fc12be00640 1 -- 172.21.15.71:0/3208799070 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc12409a7d0 con 0x7fc1340b2440 2024-09-16T07:23:26.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.750+0000 7fc133400640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc114078850 0x7fc11407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:26.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.751+0000 7fc133400640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc114078850 0x7fc11407ad10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc120001860 tx=0x7fc120009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:26.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.754+0000 7fc12be00640 1 -- 172.21.15.71:0/3208799070 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc124062da0 con 0x7fc1340b2440 2024-09-16T07:23:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.977+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1340b09e0 con 0x7fc1340b2440 2024-09-16T07:23:26.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.977+0000 7fc12be00640 1 -- 172.21.15.71:0/3208799070 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7fc1240626a0 con 0x7fc1340b2440 2024-09-16T07:23:26.977 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:26.977 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:22:48.541520+0000","modified":"2024-09-16T07:22:51.551572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4404":{"gid":4404,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4404,"qdb_cluster":[4404]},"id":1}]} 2024-09-16T07:23:26.977 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-09-16T07:23:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.979+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc114078850 msgr2=0x7fc11407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.979+0000 7fc13aa00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc114078850 0x7fc11407ad10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc120001860 tx=0x7fc120009040 comp rx=0 tx=0).stop 2024-09-16T07:23:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.979+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b2440 msgr2=0x7fc1340cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.979+0000 7fc13aa00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b2440 0x7fc1340cc0c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c730 tx=0x7fc12400cbf0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.980+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 shutdown_connections 2024-09-16T07:23:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.980+0000 7fc13aa00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc114078850 0x7fc11407ad10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.980+0000 7fc13aa00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc13415f5a0 0x7fc13415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.980+0000 7fc13aa00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1340ccf00 0x7fc1340cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.980+0000 7fc13aa00640 1 --2- 172.21.15.71:0/3208799070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b2440 0x7fc1340cc0c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.980+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 >> 172.21.15.71:0/3208799070 conn(0x7fc1340b1950 msgr2=0x7fc1340b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.980+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 shutdown_connections 2024-09-16T07:23:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:26.980+0000 7fc13aa00640 1 -- 172.21.15.71:0/3208799070 wait complete. 2024-09-16T07:23:26.995 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:27.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.152+0000 7f4e51a00640 1 -- 172.21.15.71:0/2259954822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c15f5a0 msgr2=0x7f4e4c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.152+0000 7f4e51a00640 1 --2- 172.21.15.71:0/2259954822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c15f5a0 0x7f4e4c0b2440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009830 tx=0x7f4e3c02a480 comp rx=0 tx=0).stop 2024-09-16T07:23:27.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.153+0000 7f4e51a00640 1 -- 172.21.15.71:0/2259954822 shutdown_connections 2024-09-16T07:23:27.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.153+0000 7f4e51a00640 1 --2- 172.21.15.71:0/2259954822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c0ccf00 0x7f4e4c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.153+0000 7f4e51a00640 1 --2- 172.21.15.71:0/2259954822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c15f5a0 0x7f4e4c0b2440 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.153+0000 7f4e51a00640 1 -- 172.21.15.71:0/2259954822 >> 172.21.15.71:0/2259954822 conn(0x7f4e4c0b1950 msgr2=0x7f4e4c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:27.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.153+0000 7f4e51a00640 1 -- 172.21.15.71:0/2259954822 shutdown_connections 2024-09-16T07:23:27.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.153+0000 7f4e51a00640 1 -- 172.21.15.71:0/2259954822 wait complete. 2024-09-16T07:23:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.154+0000 7f4e51a00640 1 Processor -- start 2024-09-16T07:23:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.154+0000 7f4e51a00640 1 -- start start 2024-09-16T07:23:27.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e51a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c0ad270 0x7f4e4c14c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e51a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c0ccf00 0x7f4e4c14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e51a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c15f5a0 0x7f4e4c14f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e51a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c06f2e0 con 0x7f4e4c0ad270 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e51a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c06f160 con 0x7f4e4c15f5a0 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e51a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4c06f460 con 0x7f4e4c0ccf00 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e4aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c0ad270 0x7f4e4c14c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e4aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c0ad270 0x7f4e4c14c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49624/0 (socket says 172.21.15.71:49624) 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.155+0000 7f4e4aa00640 1 -- 172.21.15.71:0/1862889982 learned_addr learned my addr 172.21.15.71:0/1862889982 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4a000640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c0ccf00 0x7f4e4c14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4b400640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c15f5a0 0x7f4e4c14f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4aa00640 1 -- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c0ccf00 msgr2=0x7f4e4c14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4aa00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c0ccf00 0x7f4e4c14ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4aa00640 1 -- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c15f5a0 msgr2=0x7f4e4c14f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4aa00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c15f5a0 0x7f4e4c14f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4aa00640 1 -- 172.21.15.71:0/1862889982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c14e820 con 0x7f4e4c0ad270 2024-09-16T07:23:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4a000640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c0ccf00 0x7f4e4c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e4aa00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c0ad270 0x7f4e4c14c500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009950 tx=0x7f4e3c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e42a00640 1 -- 172.21.15.71:0/1862889982 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c031d60 con 0x7f4e4c0ad270 2024-09-16T07:23:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e42a00640 1 -- 172.21.15.71:0/1862889982 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e3c033e10 con 0x7f4e4c0ad270 2024-09-16T07:23:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.156+0000 7f4e42a00640 1 -- 172.21.15.71:0/1862889982 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c005260 con 0x7f4e4c0ad270 2024-09-16T07:23:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.157+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c14eb00 con 0x7f4e4c0ad270 2024-09-16T07:23:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.157+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c150cc0 con 0x7f4e4c0ad270 2024-09-16T07:23:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.158+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c06f9f0 con 0x7f4e4c0ad270 2024-09-16T07:23:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.159+0000 7f4e42a00640 1 -- 172.21.15.71:0/1862889982 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4e3c005400 con 0x7f4e4c0ad270 2024-09-16T07:23:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.160+0000 7f4e42a00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e2c078850 0x7f4e2c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.160+0000 7f4e42a00640 1 -- 172.21.15.71:0/1862889982 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4e3c0bbab0 con 0x7f4e4c0ad270 2024-09-16T07:23:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.160+0000 7f4e4a000640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e2c078850 0x7f4e2c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.161+0000 7f4e4a000640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e2c078850 0x7f4e2c07ad10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e380058c0 tx=0x7f4e38005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:27.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.164+0000 7f4e42a00640 1 -- 172.21.15.71:0/1862889982 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e3c03f030 con 0x7f4e4c0ad270 2024-09-16T07:23:27.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.383+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e4c06fce0 con 0x7f4e4c0ad270 2024-09-16T07:23:27.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.384+0000 7f4e42a00640 1 -- 172.21.15.71:0/1862889982 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f4e3c0839a0 con 0x7f4e4c0ad270 2024-09-16T07:23:27.384 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:27.385 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"btime":"2024-09-16T07:22:52:133942+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/878084774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":878084774},{"type":"v1","addr":"172.21.15.71:6833","nonce":878084774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3716611879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3716611879},{"type":"v1","addr":"172.21.15.71:6835","nonce":3716611879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1455004028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1455004028},{"type":"v1","addr":"172.21.15.177:6835","nonce":1455004028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:22:48.541520+0000","modified":"2024-09-16T07:22:51.551572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4404":{"gid":4404,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1387172624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1387172624},{"type":"v1","addr":"172.21.15.177:6837","nonce":1387172624}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4404,"qdb_cluster":[4404]},"id":1}]} 2024-09-16T07:23:27.385 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-09-16T07:23:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.386+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e2c078850 msgr2=0x7f4e2c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.386+0000 7f4e51a00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e2c078850 0x7f4e2c07ad10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e380058c0 tx=0x7f4e38005850 comp rx=0 tx=0).stop 2024-09-16T07:23:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.386+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c0ad270 msgr2=0x7f4e4c14c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.386+0000 7f4e51a00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c0ad270 0x7f4e4c14c500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009950 tx=0x7f4e3c033c20 comp rx=0 tx=0).stop 2024-09-16T07:23:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.387+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 shutdown_connections 2024-09-16T07:23:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.387+0000 7f4e51a00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e2c078850 0x7f4e2c07ad10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.387+0000 7f4e51a00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c15f5a0 0x7f4e4c14f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.387+0000 7f4e51a00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c0ccf00 0x7f4e4c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.387+0000 7f4e51a00640 1 --2- 172.21.15.71:0/1862889982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e4c0ad270 0x7f4e4c14c500 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.387+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 >> 172.21.15.71:0/1862889982 conn(0x7f4e4c0b1950 msgr2=0x7f4e4c15e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.387+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 shutdown_connections 2024-09-16T07:23:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.387+0000 7f4e51a00640 1 -- 172.21.15.71:0/1862889982 wait complete. 2024-09-16T07:23:27.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:23:27.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.560+0000 7f64b6000640 1 -- 172.21.15.71:0/687407304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b5370 msgr2=0x7f64b00b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.560+0000 7f64b6000640 1 --2- 172.21.15.71:0/687407304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b5370 0x7f64b00b3bc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f64b00d18c0 tx=0x7f649802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.561+0000 7f64b6000640 1 -- 172.21.15.71:0/687407304 shutdown_connections 2024-09-16T07:23:27.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.561+0000 7f64b6000640 1 --2- 172.21.15.71:0/687407304 >> v2:172.21.15.177:3300/0 conn(0x7f64b00b4100 0x7f64b00b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.561+0000 7f64b6000640 1 --2- 172.21.15.71:0/687407304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b5370 0x7f64b00b3bc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.561+0000 7f64b6000640 1 -- 172.21.15.71:0/687407304 >> 172.21.15.71:0/687407304 conn(0x7f64b00d3df0 msgr2=0x7f64b00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:27.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.561+0000 7f64b6000640 1 -- 172.21.15.71:0/687407304 shutdown_connections 2024-09-16T07:23:27.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.561+0000 7f64b6000640 1 -- 172.21.15.71:0/687407304 wait complete. 2024-09-16T07:23:27.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.562+0000 7f64b6000640 1 Processor -- start 2024-09-16T07:23:27.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.562+0000 7f64b6000640 1 -- start start 2024-09-16T07:23:27.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.563+0000 7f64b6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b4100 0x7f64b01f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.563+0000 7f64b6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64b00b5370 0x7f64b006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.563+0000 7f64b6000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64b0161440 0x7f64b00b9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.563+0000 7f64b6000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f64b00b2040 con 0x7f64b00b4100 2024-09-16T07:23:27.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.563+0000 7f64b6000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64b00b1ec0 con 0x7f64b00b5370 2024-09-16T07:23:27.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.563+0000 7f64b6000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64b00b21c0 con 0x7f64b0161440 2024-09-16T07:23:27.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.563+0000 7f64b4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b4100 0x7f64b01f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b4100 0x7f64b01f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49652/0 (socket says 172.21.15.71:49652) 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b4c00640 1 -- 172.21.15.71:0/1374631227 learned_addr learned my addr 172.21.15.71:0/1374631227 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64a7e00640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64b00b5370 0x7f64b006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b5600640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64b0161440 0x7f64b00b9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b4c00640 1 -- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64b0161440 msgr2=0x7f64b00b9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b4c00640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64b0161440 0x7f64b00b9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b4c00640 1 -- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64b00b5370 msgr2=0x7f64b006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b4c00640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64b00b5370 0x7f64b006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b4c00640 1 -- 172.21.15.71:0/1374631227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b01f8930 con 0x7f64b00b4100 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64a7e00640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64b00b5370 0x7f64b006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b4c00640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b4100 0x7f64b01f53e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64a000b3f0 tx=0x7f64a000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64ad600640 1 -- 172.21.15.71:0/1374631227 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0015020 con 0x7f64b00b4100 2024-09-16T07:23:27.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64ad600640 1 -- 172.21.15.71:0/1374631227 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64a0011920 con 0x7f64b00b4100 2024-09-16T07:23:27.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64ad600640 1 -- 172.21.15.71:0/1374631227 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0011d70 con 0x7f64b00b4100 2024-09-16T07:23:27.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.564+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b01f4900 con 0x7f64b00b4100 2024-09-16T07:23:27.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.565+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b01f4ec0 con 0x7f64b00b4100 2024-09-16T07:23:27.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.566+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b00b1770 con 0x7f64b00b4100 2024-09-16T07:23:27.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.567+0000 7f64ad600640 1 -- 172.21.15.71:0/1374631227 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f64a0018020 con 0x7f64b00b4100 2024-09-16T07:23:27.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.568+0000 7f64ad600640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f648c078850 0x7f648c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.568+0000 7f64ad600640 1 -- 172.21.15.71:0/1374631227 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f64a009a600 con 0x7f64b00b4100 2024-09-16T07:23:27.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.568+0000 7f64a7e00640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f648c078850 0x7f648c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.569+0000 7f64a7e00640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f648c078850 0x7f648c07ad10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6498033000 tx=0x7f6498004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:27.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.572+0000 7f64ad600640 1 -- 172.21.15.71:0/1374631227 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f64a001f030 con 0x7f64b00b4100 2024-09-16T07:23:27.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.757+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64b00cdf80 con 0x7f64b00b4100 2024-09-16T07:23:27.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.758+0000 7f64ad600640 1 -- 172.21.15.71:0/1374631227 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7f64a0062c60 con 0x7f64b00b4100 2024-09-16T07:23:27.758 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:27.758 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":23,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:22:50.122046+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:22:46.797343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T07:22:47.495216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:23:27.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f648c078850 msgr2=0x7f648c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f648c078850 0x7f648c07ad10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6498033000 tx=0x7f6498004730 comp rx=0 tx=0).stop 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b4100 msgr2=0x7f64b01f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b4100 0x7f64b01f53e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64a000b3f0 tx=0x7f64a000b8b0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 shutdown_connections 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f648c078850 0x7f648c07ad10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64b0161440 0x7f64b00b9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64b00b5370 0x7f64b006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 --2- 172.21.15.71:0/1374631227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b00b4100 0x7f64b01f53e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 >> 172.21.15.71:0/1374631227 conn(0x7f64b00d3df0 msgr2=0x7f64b00b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 shutdown_connections 2024-09-16T07:23:27.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.761+0000 7f64b6000640 1 -- 172.21.15.71:0/1374631227 wait complete. 2024-09-16T07:23:27.776 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:23:27.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 -- 172.21.15.71:0/592078342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ad270 msgr2=0x7fe6a0073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/592078342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ad270 0x7fe6a0073fb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00723f0 tx=0x7fe69802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 -- 172.21.15.71:0/592078342 shutdown_connections 2024-09-16T07:23:27.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/592078342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ad270 0x7fe6a0073fb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/592078342 >> v2:172.21.15.177:3300/0 conn(0x7fe6a00ccf00 0x7fe6a00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/592078342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6a015f5a0 0x7fe6a00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 -- 172.21.15.71:0/592078342 >> 172.21.15.71:0/592078342 conn(0x7fe6a00b1950 msgr2=0x7fe6a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:27.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 -- 172.21.15.71:0/592078342 shutdown_connections 2024-09-16T07:23:27.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.933+0000 7fe6a4c00640 1 -- 172.21.15.71:0/592078342 wait complete. 2024-09-16T07:23:27.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.935+0000 7fe6a4c00640 1 Processor -- start 2024-09-16T07:23:27.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.935+0000 7fe6a4c00640 1 -- start start 2024-09-16T07:23:27.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe6a4c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6a00ad270 0x7fe6a00ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe6a4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ccf00 0x7fe6a01f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe6a4c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a015f5a0 0x7fe6a01f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe6a4c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a006f160 con 0x7fe6a00ccf00 2024-09-16T07:23:27.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe6a4c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0074780 con 0x7fe6a015f5a0 2024-09-16T07:23:27.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe6a4c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a006f2e0 con 0x7fe6a00ad270 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe69d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ccf00 0x7fe6a01f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe69d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ccf00 0x7fe6a01f8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49680/0 (socket says 172.21.15.71:49680) 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe69d400640 1 -- 172.21.15.71:0/1226563586 learned_addr learned my addr 172.21.15.71:0/1226563586 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe69de00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6a00ad270 0x7fe6a00ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe69e800640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a015f5a0 0x7fe6a01f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.936+0000 7fe69d400640 1 -- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6a00ad270 msgr2=0x7fe6a00ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe69d400640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6a00ad270 0x7fe6a00ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe69d400640 1 -- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a015f5a0 msgr2=0x7fe6a01f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe69d400640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a015f5a0 0x7fe6a01f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe69d400640 1 -- 172.21.15.71:0/1226563586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a00cf670 con 0x7fe6a00ccf00 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe69e800640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a015f5a0 0x7fe6a01f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:27.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe69de00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6a00ad270 0x7fe6a00ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:27.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe69d400640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ccf00 0x7fe6a01f8020 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00b480 tx=0x7fe68c00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:27.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe696000640 1 -- 172.21.15.71:0/1226563586 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c014070 con 0x7fe6a00ccf00 2024-09-16T07:23:27.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe696000640 1 -- 172.21.15.71:0/1226563586 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe68c010960 con 0x7fe6a00ccf00 2024-09-16T07:23:27.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe696000640 1 -- 172.21.15.71:0/1226563586 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c010db0 con 0x7fe6a00ccf00 2024-09-16T07:23:27.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.937+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a00cf950 con 0x7fe6a00ccf00 2024-09-16T07:23:27.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.938+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a00cff10 con 0x7fe6a00ccf00 2024-09-16T07:23:27.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.939+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a006f870 con 0x7fe6a00ccf00 2024-09-16T07:23:27.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.940+0000 7fe696000640 1 -- 172.21.15.71:0/1226563586 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe68c017070 con 0x7fe6a00ccf00 2024-09-16T07:23:27.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.941+0000 7fe696000640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe680078850 0x7fe68007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:27.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.941+0000 7fe696000640 1 -- 172.21.15.71:0/1226563586 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe68c09a750 con 0x7fe6a00ccf00 2024-09-16T07:23:27.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.942+0000 7fe69de00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe680078850 0x7fe68007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:27.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.942+0000 7fe69de00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe680078850 0x7fe68007ad10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe690007eb0 tx=0x7fe6900073c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:27.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:27.946+0000 7fe696000640 1 -- 172.21.15.71:0/1226563586 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe68c062db0 con 0x7fe6a00ccf00 2024-09-16T07:23:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.168+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe6a015da90 con 0x7fe6a00ccf00 2024-09-16T07:23:28.168 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:23:28.168+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:23:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.170+0000 7fe696000640 1 -- 172.21.15.71:0/1226563586 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fe68c0626b0 con 0x7fe6a00ccf00 2024-09-16T07:23:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:23:28.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.173+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe680078850 msgr2=0x7fe68007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.173+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe680078850 0x7fe68007ad10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe690007eb0 tx=0x7fe6900073c0 comp rx=0 tx=0).stop 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ccf00 msgr2=0x7fe6a01f8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ccf00 0x7fe6a01f8020 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00b480 tx=0x7fe68c00b940 comp rx=0 tx=0).stop 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 shutdown_connections 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe680078850 0x7fe68007ad10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a015f5a0 0x7fe6a01f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a00ccf00 0x7fe6a01f8020 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 --2- 172.21.15.71:0/1226563586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6a00ad270 0x7fe6a00ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 >> 172.21.15.71:0/1226563586 conn(0x7fe6a00b1950 msgr2=0x7fe6a009d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 shutdown_connections 2024-09-16T07:23:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.174+0000 7fe6a4c00640 1 -- 172.21.15.71:0/1226563586 wait complete. 2024-09-16T07:23:28.185 INFO:tasks.ceph.mds.d.smithi177.stderr: -7208> 2024-09-16T07:22:54.358+0000 7f6fe6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:23:28.189 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:23:28.238 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:23:28.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.353+0000 7f4238600640 1 -- 172.21.15.71:0/3814547679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 msgr2=0x7f4218003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:28.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.353+0000 7f4238600640 1 --2- 172.21.15.71:0/3814547679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 0x7f4218003e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4224009ed0 tx=0x7f4218005830 comp rx=0 tx=0).stop 2024-09-16T07:23:28.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.353+0000 7f4236600640 1 -- 172.21.15.71:0/3814547679 reap_dead start 2024-09-16T07:23:28.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.353+0000 7f4238600640 1 -- 172.21.15.71:0/3814547679 shutdown_connections 2024-09-16T07:23:28.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.353+0000 7f4238600640 1 -- 172.21.15.71:0/3814547679 >> 172.21.15.71:0/3814547679 conn(0x7f42300b1840 msgr2=0x7f42300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:28.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.353+0000 7f4238600640 1 -- 172.21.15.71:0/3814547679 shutdown_connections 2024-09-16T07:23:28.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.353+0000 7f4238600640 1 -- 172.21.15.71:0/3814547679 wait complete. 2024-09-16T07:23:28.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.355+0000 7f4238600640 1 Processor -- start 2024-09-16T07:23:28.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.355+0000 7f4238600640 1 -- start start 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4238600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 0x7f423013e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4238600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42301e8600 0x7f42301e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4238600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423013ec40 0x7f423013f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4238600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f42300b2680 con 0x7f4218003a00 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4238600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f42300cedd0 con 0x7f42301e8600 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4238600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4230073440 con 0x7f423013ec40 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 0x7f423013e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 0x7f423013e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49694/0 (socket says 172.21.15.71:49694) 2024-09-16T07:23:28.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235c00640 1 -- 172.21.15.71:0/1519242981 learned_addr learned my addr 172.21.15.71:0/1519242981 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4236600640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423013ec40 0x7f423013f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235200640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42301e8600 0x7f42301e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235c00640 1 -- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423013ec40 msgr2=0x7f423013f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235c00640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423013ec40 0x7f423013f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235c00640 1 -- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42301e8600 msgr2=0x7f42301e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235c00640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42301e8600 0x7f42301e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.356+0000 7f4235c00640 1 -- 172.21.15.71:0/1519242981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423013fde0 con 0x7f4218003a00 2024-09-16T07:23:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.357+0000 7f4235c00640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 0x7f423013e190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4224009ed0 tx=0x7f4224033bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:28.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.357+0000 7f422d600640 1 -- 172.21.15.71:0/1519242981 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224043070 con 0x7f4218003a00 2024-09-16T07:23:28.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.357+0000 7f422d600640 1 -- 172.21.15.71:0/1519242981 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4224033db0 con 0x7f4218003a00 2024-09-16T07:23:28.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.357+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423015eeb0 con 0x7f4218003a00 2024-09-16T07:23:28.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.357+0000 7f422d600640 1 -- 172.21.15.71:0/1519242981 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224038040 con 0x7f4218003a00 2024-09-16T07:23:28.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.357+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423013f820 con 0x7f4218003a00 2024-09-16T07:23:28.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.358+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42300cc5c0 con 0x7f4218003a00 2024-09-16T07:23:28.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.359+0000 7f422d600640 1 -- 172.21.15.71:0/1519242981 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4224046070 con 0x7f4218003a00 2024-09-16T07:23:28.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.360+0000 7f422d600640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4210078850 0x7f421007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:28.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.360+0000 7f422d600640 1 -- 172.21.15.71:0/1519242981 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5795+0+0 (secure 0 0 0) 0x7f42240bb2a0 con 0x7f4218003a00 2024-09-16T07:23:28.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.360+0000 7f4235200640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4210078850 0x7f421007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:28.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.361+0000 7f4235200640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4210078850 0x7f421007ad10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f42300b42c0 tx=0x7f421c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:28.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.364+0000 7f422d600640 1 -- 172.21.15.71:0/1519242981 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f423015eeb0 con 0x7f4218003a00 2024-09-16T07:23:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:28.586+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f42301436f0 con 0x7f4218003a00 2024-09-16T07:23:29.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.176+0000 7f422d600640 1 -- 172.21.15.71:0/1519242981 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f4224083130 con 0x7f4218003a00 2024-09-16T07:23:29.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4210078850 msgr2=0x7f421007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:29.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4210078850 0x7f421007ad10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f42300b42c0 tx=0x7f421c005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:29.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 msgr2=0x7f423013e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:29.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 0x7f423013e190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4224009ed0 tx=0x7f4224033bc0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 shutdown_connections 2024-09-16T07:23:29.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4210078850 0x7f421007ad10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423013ec40 0x7f423013f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42301e8600 0x7f42301e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 --2- 172.21.15.71:0/1519242981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4218003a00 0x7f423013e190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.179+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 >> 172.21.15.71:0/1519242981 conn(0x7f42300b1840 msgr2=0x7f423006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:29.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.180+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 shutdown_connections 2024-09-16T07:23:29.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.180+0000 7f4238600640 1 -- 172.21.15.71:0/1519242981 wait complete. 2024-09-16T07:23:29.197 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:23:29.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.362+0000 7fbdfe200640 1 -- 172.21.15.71:0/2822825517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf806e590 msgr2=0x7fbdf806fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.362+0000 7fbdfe200640 1 --2- 172.21.15.71:0/2822825517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf806e590 0x7fbdf806fe40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8074a70 tx=0x7fbde402a5f0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.362+0000 7fbdfe200640 1 -- 172.21.15.71:0/2822825517 shutdown_connections 2024-09-16T07:23:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.362+0000 7fbdfe200640 1 --2- 172.21.15.71:0/2822825517 >> v2:172.21.15.177:3300/0 conn(0x7fbdf8070380 0x7fbdf80cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.362+0000 7fbdfe200640 1 --2- 172.21.15.71:0/2822825517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf806e590 0x7fbdf806fe40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.362+0000 7fbdfe200640 1 -- 172.21.15.71:0/2822825517 >> 172.21.15.71:0/2822825517 conn(0x7fbdf80cfa70 msgr2=0x7fbdf80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.362+0000 7fbdfe200640 1 -- 172.21.15.71:0/2822825517 shutdown_connections 2024-09-16T07:23:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.362+0000 7fbdfe200640 1 -- 172.21.15.71:0/2822825517 wait complete. 2024-09-16T07:23:29.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.363+0000 7fbdfe200640 1 Processor -- start 2024-09-16T07:23:29.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.364+0000 7fbdfe200640 1 -- start start 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.364+0000 7fbdfe200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf806e590 0x7fbdf815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.364+0000 7fbdfe200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf8070380 0x7fbdf80b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.364+0000 7fbdfe200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf815bda0 0x7fbdf815e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.364+0000 7fbdfe200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80cba90 con 0x7fbdf8070380 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.364+0000 7fbdfe200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80cb910 con 0x7fbdf815bda0 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.364+0000 7fbdfe200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf80cbc10 con 0x7fbdf806e590 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf8070380 0x7fbdf80b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf8070380 0x7fbdf80b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49702/0 (socket says 172.21.15.71:49702) 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 -- 172.21.15.71:0/1491796197 learned_addr learned my addr 172.21.15.71:0/1491796197 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdfce00640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf806e590 0x7fbdf815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdfd800640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf815bda0 0x7fbdf815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:29.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 -- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf806e590 msgr2=0x7fbdf815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:29.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf806e590 0x7fbdf815c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 -- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf815bda0 msgr2=0x7fbdf815e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:29.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf815bda0 0x7fbdf815e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:29.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 -- 172.21.15.71:0/1491796197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf815c840 con 0x7fbdf8070380 2024-09-16T07:23:29.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.365+0000 7fbdf7e00640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf8070380 0x7fbdf80b0690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbde4000c00 tx=0x7fbde4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:29.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.366+0000 7fbdf4c00640 1 -- 172.21.15.71:0/1491796197 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde403f070 con 0x7fbdf8070380 2024-09-16T07:23:29.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.366+0000 7fbdf4c00640 1 -- 172.21.15.71:0/1491796197 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbde40047a0 con 0x7fbdf8070380 2024-09-16T07:23:29.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.366+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf80b0bd0 con 0x7fbdf8070380 2024-09-16T07:23:29.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.366+0000 7fbdf4c00640 1 -- 172.21.15.71:0/1491796197 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4035030 con 0x7fbdf8070380 2024-09-16T07:23:29.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.366+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf80c7fa0 con 0x7fbdf8070380 2024-09-16T07:23:29.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.367+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf80cc960 con 0x7fbdf8070380 2024-09-16T07:23:29.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.368+0000 7fbdf4c00640 1 -- 172.21.15.71:0/1491796197 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fbde4042070 con 0x7fbdf8070380 2024-09-16T07:23:29.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.369+0000 7fbdf4c00640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdd8078850 0x7fbdd807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:29.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.369+0000 7fbdf4c00640 1 -- 172.21.15.71:0/1491796197 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7fbde40bb7d0 con 0x7fbdf8070380 2024-09-16T07:23:29.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.370+0000 7fbdfce00640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdd8078850 0x7fbdd807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:29.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.370+0000 7fbdfce00640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdd8078850 0x7fbdd807ad10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbdf815f050 tx=0x7fbdec009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:29.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.373+0000 7fbdf4c00640 1 -- 172.21.15.71:0/1491796197 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbde4083790 con 0x7fbdf8070380 2024-09-16T07:23:29.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:29.557+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbdf80c8710 con 0x7fbdf8070380 2024-09-16T07:23:30.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.178+0000 7fbdf4c00640 1 -- 172.21.15.71:0/1491796197 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7fbde40881c0 con 0x7fbdf8070380 2024-09-16T07:23:30.177 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:23:30.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.180+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdd8078850 msgr2=0x7fbdd807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:30.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdd8078850 0x7fbdd807ad10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbdf815f050 tx=0x7fbdec009040 comp rx=0 tx=0).stop 2024-09-16T07:23:30.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf8070380 msgr2=0x7fbdf80b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf8070380 0x7fbdf80b0690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbde4000c00 tx=0x7fbde4003910 comp rx=0 tx=0).stop 2024-09-16T07:23:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 shutdown_connections 2024-09-16T07:23:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdd8078850 0x7fbdd807ad10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf815bda0 0x7fbdf815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf8070380 0x7fbdf80b0690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 --2- 172.21.15.71:0/1491796197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf806e590 0x7fbdf815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.181+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 >> 172.21.15.71:0/1491796197 conn(0x7fbdf80cfa70 msgr2=0x7fbdf80ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.182+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 shutdown_connections 2024-09-16T07:23:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.182+0000 7fbdfe200640 1 -- 172.21.15.71:0/1491796197 wait complete. 2024-09-16T07:23:30.201 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:23:30.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.363+0000 7fe451200640 1 -- 172.21.15.71:0/3620740610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c15f910 msgr2=0x7fe44c15fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:30.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.363+0000 7fe451200640 1 --2- 172.21.15.71:0/3620740610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c15f910 0x7fe44c15fcf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe44c070820 tx=0x7fe43c02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.363+0000 7fe451200640 1 -- 172.21.15.71:0/3620740610 shutdown_connections 2024-09-16T07:23:30.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.363+0000 7fe451200640 1 --2- 172.21.15.71:0/3620740610 >> v2:172.21.15.177:3300/0 conn(0x7fe44c06e6f0 0x7fe44c06eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.363+0000 7fe451200640 1 --2- 172.21.15.71:0/3620740610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c15f910 0x7fe44c15fcf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.363+0000 7fe451200640 1 -- 172.21.15.71:0/3620740610 >> 172.21.15.71:0/3620740610 conn(0x7fe44c0b18e0 msgr2=0x7fe44c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:30.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.363+0000 7fe451200640 1 -- 172.21.15.71:0/3620740610 shutdown_connections 2024-09-16T07:23:30.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.363+0000 7fe451200640 1 -- 172.21.15.71:0/3620740610 wait complete. 2024-09-16T07:23:30.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.365+0000 7fe451200640 1 Processor -- start 2024-09-16T07:23:30.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.365+0000 7fe451200640 1 -- start start 2024-09-16T07:23:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe451200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c06e6f0 0x7fe44c0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe451200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c15f910 0x7fe44c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe451200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c153560 0x7fe44c0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe451200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c06f260 con 0x7fe44c06e6f0 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe451200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c06f0e0 con 0x7fe44c153560 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe451200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe44c06f3e0 con 0x7fe44c15f910 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe449e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c06e6f0 0x7fe44c0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe449e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c06e6f0 0x7fe44c0c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49718/0 (socket says 172.21.15.71:49718) 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe449e00640 1 -- 172.21.15.71:0/2488608365 learned_addr learned my addr 172.21.15.71:0/2488608365 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe449e00640 1 -- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c15f910 msgr2=0x7fe44c0b47d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe449e00640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c15f910 0x7fe44c0b47d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe449e00640 1 -- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c153560 msgr2=0x7fe44c0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe449e00640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c153560 0x7fe44c0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.366+0000 7fe449e00640 1 -- 172.21.15.71:0/2488608365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c0b4d10 con 0x7fe44c06e6f0 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.367+0000 7fe449e00640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c06e6f0 0x7fe44c0c8f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe43c02abc0 tx=0x7fe43c0339f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.367+0000 7fe442000640 1 -- 172.21.15.71:0/2488608365 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c031950 con 0x7fe44c06e6f0 2024-09-16T07:23:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.367+0000 7fe442000640 1 -- 172.21.15.71:0/2488608365 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe43c031be0 con 0x7fe44c06e6f0 2024-09-16T07:23:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.367+0000 7fe442000640 1 -- 172.21.15.71:0/2488608365 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c005020 con 0x7fe44c06e6f0 2024-09-16T07:23:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.367+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c0b3bd0 con 0x7fe44c06e6f0 2024-09-16T07:23:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.367+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c0b33d0 con 0x7fe44c06e6f0 2024-09-16T07:23:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.368+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c069360 con 0x7fe44c06e6f0 2024-09-16T07:23:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.369+0000 7fe442000640 1 -- 172.21.15.71:0/2488608365 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe43c0051e0 con 0x7fe44c06e6f0 2024-09-16T07:23:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.370+0000 7fe442000640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe42c078850 0x7fe42c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.370+0000 7fe442000640 1 -- 172.21.15.71:0/2488608365 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5386+0+0 (secure 0 0 0) 0x7fe43c0bb770 con 0x7fe44c06e6f0 2024-09-16T07:23:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.371+0000 7fe449400640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe42c078850 0x7fe42c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:30.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.372+0000 7fe449400640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe42c078850 0x7fe42c07ad10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe438000f60 tx=0x7fe4380078c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:30.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.374+0000 7fe442000640 1 -- 172.21.15.71:0/2488608365 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe44c0b4d10 con 0x7fe44c06e6f0 2024-09-16T07:23:30.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:30.559+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe44c159360 con 0x7fe44c06e6f0 2024-09-16T07:23:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.185+0000 7fe442000640 1 -- 172.21.15.71:0/2488608365 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7fe44c159360 con 0x7fe44c06e6f0 2024-09-16T07:23:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:23:31.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe42c078850 msgr2=0x7fe42c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:31.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe42c078850 0x7fe42c07ad10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe438000f60 tx=0x7fe4380078c0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c06e6f0 msgr2=0x7fe44c0c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:31.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c06e6f0 0x7fe44c0c8f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe43c02abc0 tx=0x7fe43c0339f0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 shutdown_connections 2024-09-16T07:23:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe42c078850 0x7fe42c07ad10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c153560 0x7fe44c0b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c15f910 0x7fe44c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 --2- 172.21.15.71:0/2488608365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe44c06e6f0 0x7fe44c0c8f60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 >> 172.21.15.71:0/2488608365 conn(0x7fe44c0b18e0 msgr2=0x7fe44c0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 shutdown_connections 2024-09-16T07:23:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.188+0000 7fe451200640 1 -- 172.21.15.71:0/2488608365 wait complete. 2024-09-16T07:23:31.208 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:23:31.209 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:23:31.209 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:23:31.209 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:23:31.209 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:23:31.209 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:23:31.209 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:23:31.210 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:23:31.210 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:23:31.210 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:23:31.210 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:23:31.210 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:23:31.211 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:23:31.211 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:23:31.211 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:23:31.211 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:23:31.224 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:23:31.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:23:31.225 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:23:31.225 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:23:31.227 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:23:31.227 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:23:31.228 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:23:31.228 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:23:31.269 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:23:31.269 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:31.271 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:23:31.271 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:31.273 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:23:31.273 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:31.275 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:23:31.275 DEBUG:teuthology.parallel:result is None 2024-09-16T07:23:31.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:23:31.402 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:23:31.402 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:23:31.405 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:23:31.405 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:23:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.445+0000 7fab85400640 1 -- 172.21.15.71:0/3106164223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 msgr2=0x7fab800b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.445+0000 7fab85400640 1 --2- 172.21.15.71:0/3106164223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 0x7fab800b4390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab800b2a10 tx=0x7fab7802a560 comp rx=0 tx=0).stop 2024-09-16T07:23:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.445+0000 7fab85400640 1 -- 172.21.15.71:0/3106164223 shutdown_connections 2024-09-16T07:23:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.445+0000 7fab85400640 1 --2- 172.21.15.71:0/3106164223 >> v2:172.21.15.177:3300/0 conn(0x7fab801438b0 0x7fab8006ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.445+0000 7fab85400640 1 --2- 172.21.15.71:0/3106164223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 0x7fab800b4390 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.445+0000 7fab85400640 1 -- 172.21.15.71:0/3106164223 >> 172.21.15.71:0/3106164223 conn(0x7fab800b1950 msgr2=0x7fab800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.445+0000 7fab85400640 1 -- 172.21.15.71:0/3106164223 shutdown_connections 2024-09-16T07:23:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.445+0000 7fab85400640 1 -- 172.21.15.71:0/3106164223 wait complete. 2024-09-16T07:23:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.447+0000 7fab85400640 1 Processor -- start 2024-09-16T07:23:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.447+0000 7fab85400640 1 -- start start 2024-09-16T07:23:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab85400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab80070920 0x7fab8015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab85400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 0x7fab800af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab85400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab801438b0 0x7fab80138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab85400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab800b4d40 con 0x7fab800b3f00 2024-09-16T07:23:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab85400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fab800b4bc0 con 0x7fab801438b0 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab85400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fab800b4ec0 con 0x7fab80070920 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab7e800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab80070920 0x7fab8015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab7de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 0x7fab800af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab7e800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab80070920 0x7fab8015a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:39264/0 (socket says 172.21.15.71:39264) 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab7e800640 1 -- 172.21.15.71:0/2098023036 learned_addr learned my addr 172.21.15.71:0/2098023036 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.448+0000 7fab7f200640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab801438b0 0x7fab80138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab7e800640 1 -- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab801438b0 msgr2=0x7fab80138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab7e800640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab801438b0 0x7fab80138fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab7e800640 1 -- 172.21.15.71:0/2098023036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 msgr2=0x7fab800af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab7e800640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 0x7fab800af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab7e800640 1 -- 172.21.15.71:0/2098023036 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab80159900 con 0x7fab80070920 2024-09-16T07:23:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab7de00640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 0x7fab800af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:31.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab7e800640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab80070920 0x7fab8015a6c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab7000c7c0 tx=0x7fab7000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:31.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab76a00640 1 -- 172.21.15.71:0/2098023036 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70016020 con 0x7fab80070920 2024-09-16T07:23:31.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab76a00640 1 -- 172.21.15.71:0/2098023036 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab7000aae0 con 0x7fab80070920 2024-09-16T07:23:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab76a00640 1 -- 172.21.15.71:0/2098023036 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70012420 con 0x7fab80070920 2024-09-16T07:23:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.449+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab80159be0 con 0x7fab80070920 2024-09-16T07:23:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.450+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8015afa0 con 0x7fab80070920 2024-09-16T07:23:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.451+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab800b4390 con 0x7fab80070920 2024-09-16T07:23:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.452+0000 7fab76a00640 1 -- 172.21.15.71:0/2098023036 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fab70019020 con 0x7fab80070920 2024-09-16T07:23:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.453+0000 7fab76a00640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab60078850 0x7fab6007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.453+0000 7fab76a00640 1 -- 172.21.15.71:0/2098023036 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7fab7009a570 con 0x7fab80070920 2024-09-16T07:23:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.453+0000 7fab7de00640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab60078850 0x7fab6007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:31.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.453+0000 7fab7de00640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab60078850 0x7fab6007ad10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab8013e600 tx=0x7fab6c00a250 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:31.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.457+0000 7fab76a00640 1 -- 172.21.15.71:0/2098023036 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fab70020030 con 0x7fab80070920 2024-09-16T07:23:31.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:31.656+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fab800cd430 con 0x7fab80070920 2024-09-16T07:23:32.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.207+0000 7fab76a00640 1 -- 172.21.15.71:0/2098023036 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7fab70062880 con 0x7fab80070920 2024-09-16T07:23:32.207 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:23:32.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab60078850 msgr2=0x7fab6007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab60078850 0x7fab6007ad10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab8013e600 tx=0x7fab6c00a250 comp rx=0 tx=0).stop 2024-09-16T07:23:32.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab80070920 msgr2=0x7fab8015a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab80070920 0x7fab8015a6c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab7000c7c0 tx=0x7fab7000cc80 comp rx=0 tx=0).stop 2024-09-16T07:23:32.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 shutdown_connections 2024-09-16T07:23:32.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab60078850 0x7fab6007ad10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab801438b0 0x7fab80138fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab800b3f00 0x7fab800af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 --2- 172.21.15.71:0/2098023036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab80070920 0x7fab8015a6c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 >> 172.21.15.71:0/2098023036 conn(0x7fab800b1950 msgr2=0x7fab800cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:32.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.210+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 shutdown_connections 2024-09-16T07:23:32.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.211+0000 7fab85400640 1 -- 172.21.15.71:0/2098023036 wait complete. 2024-09-16T07:23:32.226 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:23:32.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.385+0000 7f0376e00640 1 -- 172.21.15.71:0/1244623009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0370073fb0 msgr2=0x7f0370074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.385+0000 7f0376e00640 1 --2- 172.21.15.71:0/1244623009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0370073fb0 0x7f0370074420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f03700cfbb0 tx=0x7f036802a4a0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.385+0000 7f0376e00640 1 -- 172.21.15.71:0/1244623009 shutdown_connections 2024-09-16T07:23:32.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.385+0000 7f0376e00640 1 --2- 172.21.15.71:0/1244623009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0370073fb0 0x7f0370074420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.385+0000 7f0376e00640 1 --2- 172.21.15.71:0/1244623009 >> v2:172.21.15.177:3300/0 conn(0x7f03700ccf00 0x7f03700b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.385+0000 7f0376e00640 1 -- 172.21.15.71:0/1244623009 >> 172.21.15.71:0/1244623009 conn(0x7f03700b1950 msgr2=0x7f03700b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:32.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.386+0000 7f0376e00640 1 -- 172.21.15.71:0/1244623009 shutdown_connections 2024-09-16T07:23:32.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.386+0000 7f0376e00640 1 -- 172.21.15.71:0/1244623009 wait complete. 2024-09-16T07:23:32.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.387+0000 7f0376e00640 1 Processor -- start 2024-09-16T07:23:32.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.387+0000 7f0376e00640 1 -- start start 2024-09-16T07:23:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.388+0000 7f0376e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0370073fb0 0x7f03701f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.388+0000 7f0376e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03700ccf00 0x7f0370153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.388+0000 7f0376e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f037015f5a0 0x7f03700cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.388+0000 7f0376e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f037006f2e0 con 0x7f037015f5a0 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.388+0000 7f0376e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f037006f160 con 0x7f0370073fb0 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.388+0000 7f0376e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f037006f460 con 0x7f03700ccf00 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.388+0000 7f0374e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f037015f5a0 0x7f03700cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.388+0000 7f0374e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f037015f5a0 0x7f03700cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49792/0 (socket says 172.21.15.71:49792) 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0374e00640 1 -- 172.21.15.71:0/318519563 learned_addr learned my addr 172.21.15.71:0/318519563 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f036f400640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03700ccf00 0x7f0370153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f036fe00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0370073fb0 0x7f03701f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0374e00640 1 -- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03700ccf00 msgr2=0x7f0370153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0374e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03700ccf00 0x7f0370153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0374e00640 1 -- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0370073fb0 msgr2=0x7f03701f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0374e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0370073fb0 0x7f03701f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0374e00640 1 -- 172.21.15.71:0/318519563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03701f55c0 con 0x7f037015f5a0 2024-09-16T07:23:32.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f036fe00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0370073fb0 0x7f03701f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:32.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0374e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f037015f5a0 0x7f03700cec60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0368009920 tx=0x7f0368032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:32.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0367e00640 1 -- 172.21.15.71:0/318519563 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368040070 con 0x7f037015f5a0 2024-09-16T07:23:32.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0367e00640 1 -- 172.21.15.71:0/318519563 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0368005330 con 0x7f037015f5a0 2024-09-16T07:23:32.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.389+0000 7f0367e00640 1 -- 172.21.15.71:0/318519563 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03680047a0 con 0x7f037015f5a0 2024-09-16T07:23:32.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.390+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0370153e90 con 0x7f037015f5a0 2024-09-16T07:23:32.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.390+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03700cba00 con 0x7f037015f5a0 2024-09-16T07:23:32.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.391+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03700cbc00 con 0x7f037015f5a0 2024-09-16T07:23:32.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.392+0000 7f0367e00640 1 -- 172.21.15.71:0/318519563 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0368039040 con 0x7f037015f5a0 2024-09-16T07:23:32.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.393+0000 7f0367e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0350078850 0x7f035007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:32.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.393+0000 7f0367e00640 1 -- 172.21.15.71:0/318519563 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f03680bb640 con 0x7f037015f5a0 2024-09-16T07:23:32.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.394+0000 7f036fe00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0350078850 0x7f035007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:32.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.394+0000 7f036fe00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0350078850 0x7f035007ad10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03700b4900 tx=0x7f0360009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.397+0000 7f0367e00640 1 -- 172.21.15.71:0/318519563 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f03700cbc00 con 0x7f037015f5a0 2024-09-16T07:23:32.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.649+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f03700b0970 con 0x7f037015f5a0 2024-09-16T07:23:32.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.650+0000 7f0367e00640 1 -- 172.21.15.71:0/318519563 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f0368084000 con 0x7f037015f5a0 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:23:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:23:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:23:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:23:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:23:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-mds", 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCv3Odm74m+BhAAlR1X9nm6BiaYxL+3X17qqA==", 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-mds" 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-mgr", 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCv3Odm5Zi+BhAALL9Cu4OFOVPvp0jDt0tmVA==", 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-osd", 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCv3OdmZqi+BhAAMBKpili1KjMawzmCKNl92g==", 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-osd" 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rbd", 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCv3Odm2rS+BhAAMdzfbofBecj+/CMdDl5jKQ==", 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCv3Odmxr2+BhAAwjStK4HvNremTgQqoSUipw==", 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rgw", 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCv3OdmTcm+BhAAGkV511IOASDtsztrEcmLfA==", 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:23:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:23:32.668 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:23:32.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0350078850 msgr2=0x7f035007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0350078850 0x7f035007ad10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03700b4900 tx=0x7f0360009040 comp rx=0 tx=0).stop 2024-09-16T07:23:32.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f037015f5a0 msgr2=0x7f03700cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f037015f5a0 0x7f03700cec60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0368009920 tx=0x7f0368032b90 comp rx=0 tx=0).stop 2024-09-16T07:23:32.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 shutdown_connections 2024-09-16T07:23:32.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0350078850 0x7f035007ad10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f037015f5a0 0x7f03700cec60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03700ccf00 0x7f0370153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 --2- 172.21.15.71:0/318519563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0370073fb0 0x7f03701f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.652+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 >> 172.21.15.71:0/318519563 conn(0x7f03700b1950 msgr2=0x7f037015d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:32.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.653+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 shutdown_connections 2024-09-16T07:23:32.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.653+0000 7f0376e00640 1 -- 172.21.15.71:0/318519563 wait complete. 2024-09-16T07:23:32.671 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-16T07:23:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.829+0000 7f6f27200640 1 -- 172.21.15.71:0/2621409571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280d0670 msgr2=0x7f6f280d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.829+0000 7f6f27200640 1 --2- 172.21.15.71:0/2621409571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280d0670 0x7f6f280d2760 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6f280cc220 tx=0x7f6f1402a530 comp rx=0 tx=0).stop 2024-09-16T07:23:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.830+0000 7f6f27200640 1 -- 172.21.15.71:0/2621409571 shutdown_connections 2024-09-16T07:23:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.830+0000 7f6f27200640 1 --2- 172.21.15.71:0/2621409571 >> v2:172.21.15.177:3300/0 conn(0x7f6f280b1680 0x7f6f280b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.830+0000 7f6f27200640 1 --2- 172.21.15.71:0/2621409571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280d0670 0x7f6f280d2760 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.830+0000 7f6f27200640 1 -- 172.21.15.71:0/2621409571 >> 172.21.15.71:0/2621409571 conn(0x7f6f280d1c40 msgr2=0x7f6f280d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.830+0000 7f6f27200640 1 -- 172.21.15.71:0/2621409571 shutdown_connections 2024-09-16T07:23:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.830+0000 7f6f27200640 1 -- 172.21.15.71:0/2621409571 wait complete. 2024-09-16T07:23:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.831+0000 7f6f27200640 1 Processor -- start 2024-09-16T07:23:32.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.831+0000 7f6f27200640 1 -- start start 2024-09-16T07:23:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.832+0000 7f6f27200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280b1680 0x7f6f281f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.832+0000 7f6f27200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f280d0670 0x7f6f281f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.832+0000 7f6f27200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f2815bd30 0x7f6f28159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.832+0000 7f6f27200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6f280b21f0 con 0x7f6f280b1680 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.832+0000 7f6f27200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6f280b2070 con 0x7f6f2815bd30 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.832+0000 7f6f27200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6f280b2370 con 0x7f6f280d0670 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.832+0000 7f6f25e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280b1680 0x7f6f281f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280b1680 0x7f6f281f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49818/0 (socket says 172.21.15.71:49818) 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25e00640 1 -- 172.21.15.71:0/2649528539 learned_addr learned my addr 172.21.15.71:0/2649528539 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25400640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f280d0670 0x7f6f281f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f26800640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f2815bd30 0x7f6f28159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25e00640 1 -- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f280d0670 msgr2=0x7f6f281f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25e00640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f280d0670 0x7f6f281f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25e00640 1 -- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f2815bd30 msgr2=0x7f6f28159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25e00640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f2815bd30 0x7f6f28159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25e00640 1 -- 172.21.15.71:0/2649528539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f28159ec0 con 0x7f6f280b1680 2024-09-16T07:23:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f25e00640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280b1680 0x7f6f281f8ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800cb70 tx=0x7f6f1800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f1e000640 1 -- 172.21.15.71:0/2649528539 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18016020 con 0x7f6f280b1680 2024-09-16T07:23:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f1e000640 1 -- 172.21.15.71:0/2649528539 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f1800ae50 con 0x7f6f280b1680 2024-09-16T07:23:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.833+0000 7f6f1e000640 1 -- 172.21.15.71:0/2649528539 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18012680 con 0x7f6f280b1680 2024-09-16T07:23:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.834+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f281f4a10 con 0x7f6f280b1680 2024-09-16T07:23:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.834+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f281f4fd0 con 0x7f6f280b1680 2024-09-16T07:23:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.835+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f280d16b0 con 0x7f6f280b1680 2024-09-16T07:23:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.836+0000 7f6f1e000640 1 -- 172.21.15.71:0/2649528539 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6f18019020 con 0x7f6f280b1680 2024-09-16T07:23:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.836+0000 7f6f1e000640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f08078850 0x7f6f0807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.837+0000 7f6f1e000640 1 -- 172.21.15.71:0/2649528539 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f6f1809a9a0 con 0x7f6f280b1680 2024-09-16T07:23:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.837+0000 7f6f25400640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f08078850 0x7f6f0807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:32.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.838+0000 7f6f25400640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f08078850 0x7f6f0807ad10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f14034000 tx=0x7f6f14004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:32.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:32.841+0000 7f6f1e000640 1 -- 172.21.15.71:0/2649528539 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6f280d16b0 con 0x7f6f280b1680 2024-09-16T07:23:33.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.089+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f6f2815a200 con 0x7f6f280b1680 2024-09-16T07:23:33.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.092+0000 7f6f1e000640 1 -- 172.21.15.71:0/2649528539 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f6f18020030 con 0x7f6f280b1680 2024-09-16T07:23:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f08078850 msgr2=0x7f6f0807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f08078850 0x7f6f0807ad10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f14034000 tx=0x7f6f14004730 comp rx=0 tx=0).stop 2024-09-16T07:23:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280b1680 msgr2=0x7f6f281f8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280b1680 0x7f6f281f8ac0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800cb70 tx=0x7f6f1800a430 comp rx=0 tx=0).stop 2024-09-16T07:23:33.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 shutdown_connections 2024-09-16T07:23:33.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f08078850 0x7f6f0807ad10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f2815bd30 0x7f6f28159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f280d0670 0x7f6f281f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 --2- 172.21.15.71:0/2649528539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f280b1680 0x7f6f281f8ac0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.094+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 >> 172.21.15.71:0/2649528539 conn(0x7f6f280d1c40 msgr2=0x7f6f2806de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:33.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.095+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 shutdown_connections 2024-09-16T07:23:33.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.095+0000 7f6f27200640 1 -- 172.21.15.71:0/2649528539 wait complete. 2024-09-16T07:23:33.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-16T07:23:33.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.270+0000 7f7637e00640 1 -- 172.21.15.71:0/1700481599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b34b0 msgr2=0x7f76380b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.270+0000 7f7637e00640 1 --2- 172.21.15.71:0/1700481599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b34b0 0x7f76380b3940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f76380b6930 tx=0x7f762402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.270+0000 7f7637e00640 1 -- 172.21.15.71:0/1700481599 shutdown_connections 2024-09-16T07:23:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.270+0000 7f7637e00640 1 --2- 172.21.15.71:0/1700481599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b34b0 0x7f76380b3940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.270+0000 7f7637e00640 1 --2- 172.21.15.71:0/1700481599 >> v2:172.21.15.177:3300/0 conn(0x7f7638161440 0x7f76380b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.270+0000 7f7637e00640 1 -- 172.21.15.71:0/1700481599 >> 172.21.15.71:0/1700481599 conn(0x7f76380d3df0 msgr2=0x7f76380d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.270+0000 7f7637e00640 1 -- 172.21.15.71:0/1700481599 shutdown_connections 2024-09-16T07:23:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.270+0000 7f7637e00640 1 -- 172.21.15.71:0/1700481599 wait complete. 2024-09-16T07:23:33.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.271+0000 7f7637e00640 1 Processor -- start 2024-09-16T07:23:33.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.272+0000 7f7637e00640 1 -- start start 2024-09-16T07:23:33.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.272+0000 7f7637e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76380b34b0 0x7f76380b9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:33.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.272+0000 7f7637e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b3e80 0x7f763815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7637e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7638161440 0x7f76381f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7637e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f76380b2900 con 0x7f76380b3e80 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7637e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76380b2780 con 0x7f76380b34b0 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7637e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f76380b2a80 con 0x7f7638161440 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b3e80 0x7f763815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b3e80 0x7f763815b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49824/0 (socket says 172.21.15.71:49824) 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636000640 1 -- 172.21.15.71:0/2598204039 learned_addr learned my addr 172.21.15.71:0/2598204039 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636a00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76380b34b0 0x7f76380b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7637400640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7638161440 0x7f76381f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636000640 1 -- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7638161440 msgr2=0x7f76381f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636000640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7638161440 0x7f76381f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636000640 1 -- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76380b34b0 msgr2=0x7f76380b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636000640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76380b34b0 0x7f76380b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.273+0000 7f7636000640 1 -- 172.21.15.71:0/2598204039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7638159190 con 0x7f76380b3e80 2024-09-16T07:23:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.274+0000 7f7636000640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b3e80 0x7f763815b950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f76240099c0 tx=0x7f76240039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.274+0000 7f762ea00640 1 -- 172.21.15.71:0/2598204039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762403f070 con 0x7f76380b3e80 2024-09-16T07:23:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.274+0000 7f762ea00640 1 -- 172.21.15.71:0/2598204039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7624004890 con 0x7f76380b3e80 2024-09-16T07:23:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.274+0000 7f762ea00640 1 -- 172.21.15.71:0/2598204039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624035030 con 0x7f76380b3e80 2024-09-16T07:23:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.274+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7638159470 con 0x7f76380b3e80 2024-09-16T07:23:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.274+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7638159a30 con 0x7f76380b3e80 2024-09-16T07:23:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.275+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76380b3940 con 0x7f76380b3e80 2024-09-16T07:23:33.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.276+0000 7f762ea00640 1 -- 172.21.15.71:0/2598204039 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7624042070 con 0x7f76380b3e80 2024-09-16T07:23:33.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.277+0000 7f762ea00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7618078880 0x7f761807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:33.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.277+0000 7f762ea00640 1 -- 172.21.15.71:0/2598204039 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f76240bb3b0 con 0x7f76380b3e80 2024-09-16T07:23:33.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.278+0000 7f7636a00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7618078880 0x7f761807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:33.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.279+0000 7f7636a00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7618078880 0x7f761807ad40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7628004630 tx=0x7f762800b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:33.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.281+0000 7f762ea00640 1 -- 172.21.15.71:0/2598204039 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7624083710 con 0x7f76380b3e80 2024-09-16T07:23:33.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.535+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f7638160420 con 0x7f76380b3e80 2024-09-16T07:23:33.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.537+0000 7f762ea00640 1 -- 172.21.15.71:0/2598204039 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f7624088140 con 0x7f76380b3e80 2024-09-16T07:23:33.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7618078880 msgr2=0x7f761807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7618078880 0x7f761807ad40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7628004630 tx=0x7f762800b2e0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b3e80 msgr2=0x7f763815b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b3e80 0x7f763815b950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f76240099c0 tx=0x7f76240039e0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 shutdown_connections 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7618078880 0x7f761807ad40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7638161440 0x7f76381f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76380b3e80 0x7f763815b950 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 --2- 172.21.15.71:0/2598204039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76380b34b0 0x7f76380b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 >> 172.21.15.71:0/2598204039 conn(0x7f76380d3df0 msgr2=0x7f7638158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 shutdown_connections 2024-09-16T07:23:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.540+0000 7f7637e00640 1 -- 172.21.15.71:0/2598204039 wait complete. 2024-09-16T07:23:33.555 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-16T07:23:33.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.713+0000 7fe07ea00640 1 -- 172.21.15.71:0/493036598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0780ccf00 msgr2=0x7fe0780acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.713+0000 7fe07ea00640 1 --2- 172.21.15.71:0/493036598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0780ccf00 0x7fe0780acd30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe078075130 tx=0x7fe06402a560 comp rx=0 tx=0).stop 2024-09-16T07:23:33.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.713+0000 7fe07ea00640 1 -- 172.21.15.71:0/493036598 shutdown_connections 2024-09-16T07:23:33.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.713+0000 7fe07ea00640 1 --2- 172.21.15.71:0/493036598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0780ccf00 0x7fe0780acd30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.713+0000 7fe07ea00640 1 --2- 172.21.15.71:0/493036598 >> v2:172.21.15.177:3300/0 conn(0x7fe07815f5a0 0x7fe0780b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.713+0000 7fe07ea00640 1 -- 172.21.15.71:0/493036598 >> 172.21.15.71:0/493036598 conn(0x7fe0780b1950 msgr2=0x7fe0780b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:33.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.713+0000 7fe07ea00640 1 -- 172.21.15.71:0/493036598 shutdown_connections 2024-09-16T07:23:33.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.713+0000 7fe07ea00640 1 -- 172.21.15.71:0/493036598 wait complete. 2024-09-16T07:23:33.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.714+0000 7fe07ea00640 1 Processor -- start 2024-09-16T07:23:33.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.715+0000 7fe07ea00640 1 -- start start 2024-09-16T07:23:33.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.715+0000 7fe07ea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0780ad270 0x7fe07815be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:33.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.715+0000 7fe07ea00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0780ccf00 0x7fe07815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:33.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.715+0000 7fe07ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe07815f5a0 0x7fe078158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:33.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.715+0000 7fe07ea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe07806f160 con 0x7fe07815f5a0 2024-09-16T07:23:33.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.715+0000 7fe07ea00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe07806f2e0 con 0x7fe0780ad270 2024-09-16T07:23:33.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.715+0000 7fe07ea00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe07806f460 con 0x7fe0780ccf00 2024-09-16T07:23:33.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe07815f5a0 0x7fe078158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe07815f5a0 0x7fe078158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49832/0 (socket says 172.21.15.71:49832) 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 -- 172.21.15.71:0/3715421060 learned_addr learned my addr 172.21.15.71:0/3715421060 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe077e00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0780ad270 0x7fe07815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe077400640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0780ccf00 0x7fe07815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 -- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0780ccf00 msgr2=0x7fe07815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0780ccf00 0x7fe07815a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 -- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0780ad270 msgr2=0x7fe07815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0780ad270 0x7fe07815be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 -- 172.21.15.71:0/3715421060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07815ad60 con 0x7fe07815f5a0 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.716+0000 7fe07ca00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe07815f5a0 0x7fe078158e20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe07000c7a0 tx=0x7fe07000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.717+0000 7fe06fe00640 1 -- 172.21.15.71:0/3715421060 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070003b70 con 0x7fe07815f5a0 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.717+0000 7fe06fe00640 1 -- 172.21.15.71:0/3715421060 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe070003d30 con 0x7fe07815f5a0 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.717+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07815b040 con 0x7fe07815f5a0 2024-09-16T07:23:33.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.717+0000 7fe06fe00640 1 -- 172.21.15.71:0/3715421060 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07000aab0 con 0x7fe07815f5a0 2024-09-16T07:23:33.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.717+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07815b600 con 0x7fe07815f5a0 2024-09-16T07:23:33.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.718+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07806f9f0 con 0x7fe07815f5a0 2024-09-16T07:23:33.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.719+0000 7fe06fe00640 1 -- 172.21.15.71:0/3715421060 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe07000ac50 con 0x7fe07815f5a0 2024-09-16T07:23:33.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.720+0000 7fe06fe00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe058078850 0x7fe05807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:33.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.720+0000 7fe06fe00640 1 -- 172.21.15.71:0/3715421060 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fe07009a430 con 0x7fe07815f5a0 2024-09-16T07:23:33.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.720+0000 7fe077e00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe058078850 0x7fe05807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:33.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.721+0000 7fe077e00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe058078850 0x7fe05807ad10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe06800b810 tx=0x7fe068008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:33.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.724+0000 7fe06fe00640 1 -- 172.21.15.71:0/3715421060 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe070062de0 con 0x7fe07815f5a0 2024-09-16T07:23:33.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.975+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fe0780cc3d0 con 0x7fe07815f5a0 2024-09-16T07:23:33.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.979+0000 7fe06fe00640 1 -- 172.21.15.71:0/3715421060 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fe0700626e0 con 0x7fe07815f5a0 2024-09-16T07:23:33.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe058078850 msgr2=0x7fe05807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe058078850 0x7fe05807ad10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe06800b810 tx=0x7fe068008040 comp rx=0 tx=0).stop 2024-09-16T07:23:33.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe07815f5a0 msgr2=0x7fe078158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:33.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe07815f5a0 0x7fe078158e20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe07000c7a0 tx=0x7fe07000cc60 comp rx=0 tx=0).stop 2024-09-16T07:23:33.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 shutdown_connections 2024-09-16T07:23:33.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe058078850 0x7fe05807ad10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe07815f5a0 0x7fe078158e20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0780ccf00 0x7fe07815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 --2- 172.21.15.71:0/3715421060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0780ad270 0x7fe07815be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:33.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.981+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 >> 172.21.15.71:0/3715421060 conn(0x7fe0780b1950 msgr2=0x7fe07815e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:33.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.982+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 shutdown_connections 2024-09-16T07:23:33.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:33.982+0000 7fe07ea00640 1 -- 172.21.15.71:0/3715421060 wait complete. 2024-09-16T07:23:33.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-16T07:23:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.156+0000 7febb1e00640 1 -- 172.21.15.71:0/3279434968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac073fb0 msgr2=0x7febac074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.156+0000 7febb1e00640 1 --2- 172.21.15.71:0/3279434968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac073fb0 0x7febac074420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7febac075130 tx=0x7feba402a820 comp rx=0 tx=0).stop 2024-09-16T07:23:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.156+0000 7febb1e00640 1 -- 172.21.15.71:0/3279434968 shutdown_connections 2024-09-16T07:23:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.156+0000 7febb1e00640 1 --2- 172.21.15.71:0/3279434968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac073fb0 0x7febac074420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.156+0000 7febb1e00640 1 --2- 172.21.15.71:0/3279434968 >> v2:172.21.15.177:3300/0 conn(0x7febac15f5a0 0x7febac0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.156+0000 7febb1e00640 1 -- 172.21.15.71:0/3279434968 >> 172.21.15.71:0/3279434968 conn(0x7febac0b1950 msgr2=0x7febac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.156+0000 7febb1e00640 1 -- 172.21.15.71:0/3279434968 shutdown_connections 2024-09-16T07:23:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.156+0000 7febb1e00640 1 -- 172.21.15.71:0/3279434968 wait complete. 2024-09-16T07:23:34.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.158+0000 7febb1e00640 1 Processor -- start 2024-09-16T07:23:34.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.158+0000 7febb1e00640 1 -- start start 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febb1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febac073fb0 0x7febac158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febb1e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febac0ccf00 0x7febac1f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febb1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac15f5a0 0x7febac156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febb1e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7febac06f2e0 con 0x7febac15f5a0 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febb1e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7febac06f160 con 0x7febac073fb0 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febb1e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7febac06f460 con 0x7febac0ccf00 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac15f5a0 0x7febac156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac15f5a0 0x7febac156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49854/0 (socket says 172.21.15.71:49854) 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 -- 172.21.15.71:0/2843399838 learned_addr learned my addr 172.21.15.71:0/2843399838 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febaa000640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febac0ccf00 0x7febac1f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febaaa00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febac073fb0 0x7febac158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 -- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febac0ccf00 msgr2=0x7febac1f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febac0ccf00 0x7febac1f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 -- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febac073fb0 msgr2=0x7febac158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febac073fb0 0x7febac158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 -- 172.21.15.71:0/2843399838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac157ef0 con 0x7febac15f5a0 2024-09-16T07:23:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.159+0000 7febab400640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac15f5a0 0x7febac156100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feba402ae10 tx=0x7feba4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:34.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.160+0000 7feba3400640 1 -- 172.21.15.71:0/2843399838 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4037040 con 0x7febac15f5a0 2024-09-16T07:23:34.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.160+0000 7feba3400640 1 -- 172.21.15.71:0/2843399838 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feba4042070 con 0x7febac15f5a0 2024-09-16T07:23:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.160+0000 7feba3400640 1 -- 172.21.15.71:0/2843399838 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba40042a0 con 0x7febac15f5a0 2024-09-16T07:23:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.160+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac1581d0 con 0x7febac15f5a0 2024-09-16T07:23:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.160+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac158790 con 0x7febac15f5a0 2024-09-16T07:23:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.161+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febac144f60 con 0x7febac15f5a0 2024-09-16T07:23:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.162+0000 7feba3400640 1 -- 172.21.15.71:0/2843399838 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7feba4004440 con 0x7febac15f5a0 2024-09-16T07:23:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.163+0000 7feba3400640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8c078850 0x7feb8c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.163+0000 7feba3400640 1 -- 172.21.15.71:0/2843399838 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7feba40bb380 con 0x7febac15f5a0 2024-09-16T07:23:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.163+0000 7febaaa00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8c078850 0x7feb8c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.164+0000 7febaaa00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8c078850 0x7feb8c07ad10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005fd0 tx=0x7feb9c005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:34.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.167+0000 7feba3400640 1 -- 172.21.15.71:0/2843399838 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7febac144f60 con 0x7febac15f5a0 2024-09-16T07:23:34.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.421+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7febac06f9f0 con 0x7febac15f5a0 2024-09-16T07:23:34.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.425+0000 7feba3400640 1 -- 172.21.15.71:0/2843399838 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7feba4083d30 con 0x7febac15f5a0 2024-09-16T07:23:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8c078850 msgr2=0x7feb8c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8c078850 0x7feb8c07ad10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005fd0 tx=0x7feb9c005850 comp rx=0 tx=0).stop 2024-09-16T07:23:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac15f5a0 msgr2=0x7febac156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac15f5a0 0x7febac156100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feba402ae10 tx=0x7feba4032d10 comp rx=0 tx=0).stop 2024-09-16T07:23:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 shutdown_connections 2024-09-16T07:23:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8c078850 0x7feb8c07ad10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac15f5a0 0x7febac156100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febac0ccf00 0x7febac1f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 --2- 172.21.15.71:0/2843399838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febac073fb0 0x7febac158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 >> 172.21.15.71:0/2843399838 conn(0x7febac0b1950 msgr2=0x7febac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.427+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 shutdown_connections 2024-09-16T07:23:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.428+0000 7febb1e00640 1 -- 172.21.15.71:0/2843399838 wait complete. 2024-09-16T07:23:34.442 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-16T07:23:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.600+0000 7ff348a00640 1 -- 172.21.15.71:0/1301383051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 msgr2=0x7ff328003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.600+0000 7ff348a00640 1 --2- 172.21.15.71:0/1301383051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 0x7ff328003e70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c2e0 tx=0x7ff328005830 comp rx=0 tx=0).stop 2024-09-16T07:23:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.600+0000 7ff348a00640 1 -- 172.21.15.71:0/1301383051 shutdown_connections 2024-09-16T07:23:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.600+0000 7ff348a00640 1 --2- 172.21.15.71:0/1301383051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 0x7ff328003e70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.600+0000 7ff348a00640 1 -- 172.21.15.71:0/1301383051 >> 172.21.15.71:0/1301383051 conn(0x7ff3440b1950 msgr2=0x7ff3440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:34.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.600+0000 7ff348a00640 1 -- 172.21.15.71:0/1301383051 shutdown_connections 2024-09-16T07:23:34.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.600+0000 7ff348a00640 1 -- 172.21.15.71:0/1301383051 wait complete. 2024-09-16T07:23:34.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.602+0000 7ff348a00640 1 Processor -- start 2024-09-16T07:23:34.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.602+0000 7ff348a00640 1 -- start start 2024-09-16T07:23:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff348a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 0x7ff3440ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff348a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff344147f30 0x7ff344159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff348a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff344159d50 0x7ff34409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff348a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff34406f2e0 con 0x7ff328003a50 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff348a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff34406f160 con 0x7ff344159d50 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff348a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff34406f460 con 0x7ff344147f30 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff341e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 0x7ff3440ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff341e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 0x7ff3440ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49868/0 (socket says 172.21.15.71:49868) 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff341e00640 1 -- 172.21.15.71:0/2611190934 learned_addr learned my addr 172.21.15.71:0/2611190934 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff341400640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff344147f30 0x7ff344159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff342800640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff344159d50 0x7ff34409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff341e00640 1 -- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff344147f30 msgr2=0x7ff344159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff341e00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff344147f30 0x7ff344159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.603+0000 7ff341e00640 1 -- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff344159d50 msgr2=0x7ff34409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff341e00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff344159d50 0x7ff34409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff341e00640 1 -- 172.21.15.71:0/2611190934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3441f9430 con 0x7ff328003a50 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff342800640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff344159d50 0x7ff34409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff341e00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 0x7ff3440ce140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c400 tx=0x7ff334004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff33a000640 1 -- 172.21.15.71:0/2611190934 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334044070 con 0x7ff328003a50 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff33a000640 1 -- 172.21.15.71:0/2611190934 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff334006420 con 0x7ff328003a50 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff33a000640 1 -- 172.21.15.71:0/2611190934 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334031040 con 0x7ff328003a50 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3441f8690 con 0x7ff328003a50 2024-09-16T07:23:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.604+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3441f8c50 con 0x7ff328003a50 2024-09-16T07:23:34.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.605+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34406f9c0 con 0x7ff328003a50 2024-09-16T07:23:34.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.606+0000 7ff33a000640 1 -- 172.21.15.71:0/2611190934 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff334034070 con 0x7ff328003a50 2024-09-16T07:23:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.607+0000 7ff33a000640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff324078880 0x7ff32407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.607+0000 7ff33a000640 1 -- 172.21.15.71:0/2611190934 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7ff3340b9e40 con 0x7ff328003a50 2024-09-16T07:23:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.608+0000 7ff341400640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff324078880 0x7ff32407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.608+0000 7ff341400640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff324078880 0x7ff32407ad40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff344064710 tx=0x7ff330005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:34.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.611+0000 7ff33a000640 1 -- 172.21.15.71:0/2611190934 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff3340821a0 con 0x7ff328003a50 2024-09-16T07:23:34.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.865+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7ff344073330 con 0x7ff328003a50 2024-09-16T07:23:34.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.870+0000 7ff33a000640 1 -- 172.21.15.71:0/2611190934 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7ff3340890f0 con 0x7ff328003a50 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff324078880 msgr2=0x7ff32407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff324078880 0x7ff32407ad40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff344064710 tx=0x7ff330005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 msgr2=0x7ff3440ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 0x7ff3440ce140 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c400 tx=0x7ff334004be0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 shutdown_connections 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff324078880 0x7ff32407ad40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff344159d50 0x7ff34409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff344147f30 0x7ff344159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 --2- 172.21.15.71:0/2611190934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff328003a50 0x7ff3440ce140 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:34.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 >> 172.21.15.71:0/2611190934 conn(0x7ff3440b1950 msgr2=0x7ff344073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:34.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 shutdown_connections 2024-09-16T07:23:34.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:34.872+0000 7ff348a00640 1 -- 172.21.15.71:0/2611190934 wait complete. 2024-09-16T07:23:34.888 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-16T07:23:35.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.044+0000 7fcd16a00640 1 -- 172.21.15.71:0/1163616952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b4100 msgr2=0x7fcd100b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.044+0000 7fcd16a00640 1 --2- 172.21.15.71:0/1163616952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b4100 0x7fcd100b1770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcd100d4ea0 tx=0x7fcd0802a470 comp rx=0 tx=0).stop 2024-09-16T07:23:35.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.044+0000 7fcd16a00640 1 -- 172.21.15.71:0/1163616952 shutdown_connections 2024-09-16T07:23:35.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.044+0000 7fcd16a00640 1 --2- 172.21.15.71:0/1163616952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b4100 0x7fcd100b1770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.044+0000 7fcd16a00640 1 --2- 172.21.15.71:0/1163616952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd100b5370 0x7fcd100b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.045+0000 7fcd16a00640 1 -- 172.21.15.71:0/1163616952 >> 172.21.15.71:0/1163616952 conn(0x7fcd100d3df0 msgr2=0x7fcd100d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:35.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.045+0000 7fcd16a00640 1 -- 172.21.15.71:0/1163616952 shutdown_connections 2024-09-16T07:23:35.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.045+0000 7fcd16a00640 1 -- 172.21.15.71:0/1163616952 wait complete. 2024-09-16T07:23:35.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.046+0000 7fcd16a00640 1 Processor -- start 2024-09-16T07:23:35.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.046+0000 7fcd16a00640 1 -- start start 2024-09-16T07:23:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.047+0000 7fcd16a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd100b4100 0x7fcd100cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.047+0000 7fcd16a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b5370 0x7fcd1006d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.047+0000 7fcd16a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd10161440 0x7fcd1006de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.047+0000 7fcd16a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd100b2040 con 0x7fcd100b5370 2024-09-16T07:23:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.047+0000 7fcd16a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd100b1ec0 con 0x7fcd10161440 2024-09-16T07:23:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.047+0000 7fcd16a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcd100b21c0 con 0x7fcd100b4100 2024-09-16T07:23:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.047+0000 7fcd14c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b5370 0x7fcd1006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd14c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b5370 0x7fcd1006d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44742/0 (socket says 172.21.15.71:44742) 2024-09-16T07:23:35.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd14c00640 1 -- 172.21.15.71:0/749660786 learned_addr learned my addr 172.21.15.71:0/749660786 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd16000640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd10161440 0x7fcd1006de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd15600640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd100b4100 0x7fcd100cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd14c00640 1 -- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd100b4100 msgr2=0x7fcd100cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd14c00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd100b4100 0x7fcd100cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd14c00640 1 -- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd10161440 msgr2=0x7fcd1006de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd14c00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd10161440 0x7fcd1006de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd14c00640 1 -- 172.21.15.71:0/749660786 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd101f7c40 con 0x7fcd100b5370 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd16000640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd10161440 0x7fcd1006de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd15600640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd100b4100 0x7fcd100cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:35.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd14c00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b5370 0x7fcd1006d950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00b4a0 tx=0x7fccfc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:35.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.048+0000 7fcd0d600640 1 -- 172.21.15.71:0/749660786 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc014070 con 0x7fcd100b5370 2024-09-16T07:23:35.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.049+0000 7fcd0d600640 1 -- 172.21.15.71:0/749660786 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccfc010700 con 0x7fcd100b5370 2024-09-16T07:23:35.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.049+0000 7fcd0d600640 1 -- 172.21.15.71:0/749660786 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc010b50 con 0x7fcd100b5370 2024-09-16T07:23:35.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.049+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd101f7f20 con 0x7fcd100b5370 2024-09-16T07:23:35.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.049+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd101f84e0 con 0x7fcd100b5370 2024-09-16T07:23:35.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.050+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1015e060 con 0x7fcd100b5370 2024-09-16T07:23:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.051+0000 7fcd0d600640 1 -- 172.21.15.71:0/749660786 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fccfc017070 con 0x7fcd100b5370 2024-09-16T07:23:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.052+0000 7fcd0d600640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fccf0078850 0x7fccf007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.052+0000 7fcd0d600640 1 -- 172.21.15.71:0/749660786 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fccfc099fa0 con 0x7fcd100b5370 2024-09-16T07:23:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.052+0000 7fcd15600640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fccf0078850 0x7fccf007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.053+0000 7fcd15600640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fccf0078850 0x7fccf007ad10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcd040020f0 tx=0x7fcd04009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:35.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.056+0000 7fcd0d600640 1 -- 172.21.15.71:0/749660786 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcd1015e060 con 0x7fcd100b5370 2024-09-16T07:23:35.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.313+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fcd1006e4c0 con 0x7fcd100b5370 2024-09-16T07:23:35.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.318+0000 7fcd0d600640 1 -- 172.21.15.71:0/749660786 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fcd1006e4c0 con 0x7fcd100b5370 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fccf0078850 msgr2=0x7fccf007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fccf0078850 0x7fccf007ad10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcd040020f0 tx=0x7fcd04009040 comp rx=0 tx=0).stop 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b5370 msgr2=0x7fcd1006d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b5370 0x7fcd1006d950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00b4a0 tx=0x7fccfc00b960 comp rx=0 tx=0).stop 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 shutdown_connections 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fccf0078850 0x7fccf007ad10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd10161440 0x7fcd1006de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd100b5370 0x7fcd1006d950 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 --2- 172.21.15.71:0/749660786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd100b4100 0x7fcd100cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 >> 172.21.15.71:0/749660786 conn(0x7fcd100d3df0 msgr2=0x7fcd1015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:35.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 shutdown_connections 2024-09-16T07:23:35.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.320+0000 7fcd16a00640 1 -- 172.21.15.71:0/749660786 wait complete. 2024-09-16T07:23:35.335 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:23:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.492+0000 7f88a6400640 1 -- 172.21.15.71:0/1432490611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a015f5a0 msgr2=0x7f88a00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.492+0000 7f88a6400640 1 --2- 172.21.15.71:0/1432490611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a015f5a0 0x7f88a00b2440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8890005fd0 tx=0x7f889002a540 comp rx=0 tx=0).stop 2024-09-16T07:23:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.492+0000 7f88a6400640 1 -- 172.21.15.71:0/1432490611 shutdown_connections 2024-09-16T07:23:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.492+0000 7f88a6400640 1 --2- 172.21.15.71:0/1432490611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a00ccf00 0x7f88a00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.492+0000 7f88a6400640 1 --2- 172.21.15.71:0/1432490611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a015f5a0 0x7f88a00b2440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.492+0000 7f88a6400640 1 -- 172.21.15.71:0/1432490611 >> 172.21.15.71:0/1432490611 conn(0x7f88a00b1950 msgr2=0x7f88a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.492+0000 7f88a6400640 1 -- 172.21.15.71:0/1432490611 shutdown_connections 2024-09-16T07:23:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.492+0000 7f88a6400640 1 -- 172.21.15.71:0/1432490611 wait complete. 2024-09-16T07:23:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.494+0000 7f88a6400640 1 Processor -- start 2024-09-16T07:23:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.494+0000 7f88a6400640 1 -- start start 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f88a6400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a00ad270 0x7f88a0154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f88a6400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a00ccf00 0x7f88a01f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f88a6400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a015f5a0 0x7f88a01f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f88a6400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88a006f2e0 con 0x7f88a00ccf00 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f88a6400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88a006f160 con 0x7f88a00ad270 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f88a6400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f88a006f460 con 0x7f88a015f5a0 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f889ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a00ccf00 0x7f88a01f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f889ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a00ccf00 0x7f88a01f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44760/0 (socket says 172.21.15.71:44760) 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.495+0000 7f889ea00640 1 -- 172.21.15.71:0/3004757074 learned_addr learned my addr 172.21.15.71:0/3004757074 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889fe00640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a015f5a0 0x7f88a01f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889f400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a00ad270 0x7f88a0154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889ea00640 1 -- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a015f5a0 msgr2=0x7f88a01f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889ea00640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a015f5a0 0x7f88a01f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889ea00640 1 -- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a00ad270 msgr2=0x7f88a0154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889ea00640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a00ad270 0x7f88a0154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889ea00640 1 -- 172.21.15.71:0/3004757074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a0156cd0 con 0x7f88a00ccf00 2024-09-16T07:23:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889fe00640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a015f5a0 0x7f88a01f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:35.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889f400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a00ad270 0x7f88a0154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:35.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f889ea00640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a00ccf00 0x7f88a01f86d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f888c00c920 tx=0x7f888c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:35.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f8897400640 1 -- 172.21.15.71:0/3004757074 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c007c70 con 0x7f88a00ccf00 2024-09-16T07:23:35.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f8897400640 1 -- 172.21.15.71:0/3004757074 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f888c00f450 con 0x7f88a00ccf00 2024-09-16T07:23:35.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f8897400640 1 -- 172.21.15.71:0/3004757074 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c00f8a0 con 0x7f88a00ccf00 2024-09-16T07:23:35.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a0156fb0 con 0x7f88a00ccf00 2024-09-16T07:23:35.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.496+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a0156270 con 0x7f88a00ccf00 2024-09-16T07:23:35.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.497+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a01f4720 con 0x7f88a00ccf00 2024-09-16T07:23:35.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.498+0000 7f8897400640 1 -- 172.21.15.71:0/3004757074 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f888c00fa40 con 0x7f88a00ccf00 2024-09-16T07:23:35.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.499+0000 7f8897400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8880078850 0x7f888007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.499+0000 7f8897400640 1 -- 172.21.15.71:0/3004757074 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f888c09ae00 con 0x7f88a00ccf00 2024-09-16T07:23:35.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.500+0000 7f889f400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8880078850 0x7f888007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.501+0000 7f889f400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8880078850 0x7f888007ad10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8890033000 tx=0x7f8890004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.503+0000 7f8897400640 1 -- 172.21.15.71:0/3004757074 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f888c014030 con 0x7f88a00ccf00 2024-09-16T07:23:35.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.688+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f88a006f9f0 con 0x7f88a00ccf00 2024-09-16T07:23:35.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.688+0000 7f8897400640 1 -- 172.21.15.71:0/3004757074 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7f888c0637b0 con 0x7f88a00ccf00 2024-09-16T07:23:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:35.688 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:23:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:23:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:23:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:35.689 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:23:35.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.690+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8880078850 msgr2=0x7f888007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8880078850 0x7f888007ad10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8890033000 tx=0x7f8890004730 comp rx=0 tx=0).stop 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a00ccf00 msgr2=0x7f88a01f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a00ccf00 0x7f88a01f86d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f888c00c920 tx=0x7f888c00cde0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 shutdown_connections 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8880078850 0x7f888007ad10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a015f5a0 0x7f88a01f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88a00ccf00 0x7f88a01f86d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 --2- 172.21.15.71:0/3004757074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a00ad270 0x7f88a0154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 >> 172.21.15.71:0/3004757074 conn(0x7f88a00b1950 msgr2=0x7f88a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 shutdown_connections 2024-09-16T07:23:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.691+0000 7f88a6400640 1 -- 172.21.15.71:0/3004757074 wait complete. 2024-09-16T07:23:35.705 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:23:35.705 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:23:35.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.865+0000 7f3505400640 1 -- 172.21.15.71:0/354418285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f350006f220 msgr2=0x7f350006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.865+0000 7f3505400640 1 --2- 172.21.15.71:0/354418285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f350006f220 0x7f350006f6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009860 tx=0x7f34f802a540 comp rx=0 tx=0).stop 2024-09-16T07:23:35.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.866+0000 7f3505400640 1 -- 172.21.15.71:0/354418285 shutdown_connections 2024-09-16T07:23:35.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.866+0000 7f3505400640 1 --2- 172.21.15.71:0/354418285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f350006f220 0x7f350006f6d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.866+0000 7f3505400640 1 --2- 172.21.15.71:0/354418285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3500159aa0 0x7f35000741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.866+0000 7f3505400640 1 -- 172.21.15.71:0/354418285 >> 172.21.15.71:0/354418285 conn(0x7f35000b18e0 msgr2=0x7f35000b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:35.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.866+0000 7f3505400640 1 -- 172.21.15.71:0/354418285 shutdown_connections 2024-09-16T07:23:35.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.866+0000 7f3505400640 1 -- 172.21.15.71:0/354418285 wait complete. 2024-09-16T07:23:35.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.867+0000 7f3505400640 1 Processor -- start 2024-09-16T07:23:35.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.867+0000 7f3505400640 1 -- start start 2024-09-16T07:23:35.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.868+0000 7f3505400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f350006f220 0x7f35000cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.868+0000 7f3505400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3500159aa0 0x7f3500154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.868+0000 7f3505400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f350015cff0 0x7f3500155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.868+0000 7f3505400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f35000cc520 con 0x7f3500159aa0 2024-09-16T07:23:35.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.868+0000 7f3505400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f35000cc3a0 con 0x7f350015cff0 2024-09-16T07:23:35.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.868+0000 7f3505400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f35000cc6a0 con 0x7f350006f220 2024-09-16T07:23:35.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3500159aa0 0x7f3500154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3500159aa0 0x7f3500154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44780/0 (socket says 172.21.15.71:44780) 2024-09-16T07:23:35.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 -- 172.21.15.71:0/3974480280 learned_addr learned my addr 172.21.15.71:0/3974480280 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:35.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34ff200640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f350015cff0 0x7f3500155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fe800640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f350006f220 0x7f35000cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 -- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f350006f220 msgr2=0x7f35000cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f350006f220 0x7f35000cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 -- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f350015cff0 msgr2=0x7f3500155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f350015cff0 0x7f3500155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 -- 172.21.15.71:0/3974480280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35000b3bf0 con 0x7f3500159aa0 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.869+0000 7f34fde00640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3500159aa0 0x7f3500154e50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c390 tx=0x7f34e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.870+0000 7f34f6a00640 1 -- 172.21.15.71:0/3974480280 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8017070 con 0x7f3500159aa0 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.870+0000 7f34f6a00640 1 -- 172.21.15.71:0/3974480280 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34e80035f0 con 0x7f3500159aa0 2024-09-16T07:23:35.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.870+0000 7f34f6a00640 1 -- 172.21.15.71:0/3974480280 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8003a20 con 0x7f3500159aa0 2024-09-16T07:23:35.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.870+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35000b3ed0 con 0x7f3500159aa0 2024-09-16T07:23:35.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.870+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35000b4490 con 0x7f3500159aa0 2024-09-16T07:23:35.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.871+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35000ccbc0 con 0x7f3500159aa0 2024-09-16T07:23:35.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.872+0000 7f34f6a00640 1 -- 172.21.15.71:0/3974480280 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f34e801a070 con 0x7f3500159aa0 2024-09-16T07:23:35.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.873+0000 7f34f6a00640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34dc078850 0x7f34dc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:35.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.873+0000 7f34f6a00640 1 -- 172.21.15.71:0/3974480280 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f34e8099730 con 0x7f3500159aa0 2024-09-16T07:23:35.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.873+0000 7f34fe800640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34dc078850 0x7f34dc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:35.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.874+0000 7f34fe800640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34dc078850 0x7f34dc07ad10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f350015d8c0 tx=0x7f34f000d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:35.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:35.877+0000 7f34f6a00640 1 -- 172.21.15.71:0/3974480280 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f34e8061b30 con 0x7f3500159aa0 2024-09-16T07:23:36.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.061+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f350015e4e0 con 0x7f3500159aa0 2024-09-16T07:23:36.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.321+0000 7f34f6a00640 1 -- 172.21.15.71:0/3974480280 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7f34e8066560 con 0x7f3500159aa0 2024-09-16T07:23:36.321 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34dc078850 msgr2=0x7f34dc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34dc078850 0x7f34dc07ad10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f350015d8c0 tx=0x7f34f000d040 comp rx=0 tx=0).stop 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3500159aa0 msgr2=0x7f3500154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3500159aa0 0x7f3500154e50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c390 tx=0x7f34e800c850 comp rx=0 tx=0).stop 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 shutdown_connections 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34dc078850 0x7f34dc07ad10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f350015cff0 0x7f3500155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3500159aa0 0x7f3500154e50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 --2- 172.21.15.71:0/3974480280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f350006f220 0x7f35000cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 >> 172.21.15.71:0/3974480280 conn(0x7f35000b18e0 msgr2=0x7f350015ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:36.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 shutdown_connections 2024-09-16T07:23:36.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.324+0000 7f3505400640 1 -- 172.21.15.71:0/3974480280 wait complete. 2024-09-16T07:23:36.342 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:23:36.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.507+0000 7feb5b000640 1 -- 172.21.15.71:0/1779863884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 msgr2=0x7feb5415f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:36.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.507+0000 7feb5b000640 1 --2- 172.21.15.71:0/1779863884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 0x7feb5415f9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feb540d0cd0 tx=0x7feb4c02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.507+0000 7feb5b000640 1 -- 172.21.15.71:0/1779863884 shutdown_connections 2024-09-16T07:23:36.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.507+0000 7feb5b000640 1 --2- 172.21.15.71:0/1779863884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 0x7feb5415f9f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.507+0000 7feb5b000640 1 --2- 172.21.15.71:0/1779863884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb5406efa0 0x7feb540cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.507+0000 7feb5b000640 1 --2- 172.21.15.71:0/1779863884 >> v2:172.21.15.177:3300/0 conn(0x7feb54145750 0x7feb54073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.507+0000 7feb5b000640 1 -- 172.21.15.71:0/1779863884 >> 172.21.15.71:0/1779863884 conn(0x7feb540b1840 msgr2=0x7feb540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:36.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.507+0000 7feb5b000640 1 -- 172.21.15.71:0/1779863884 shutdown_connections 2024-09-16T07:23:36.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.508+0000 7feb5b000640 1 -- 172.21.15.71:0/1779863884 wait complete. 2024-09-16T07:23:36.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.509+0000 7feb5b000640 1 Processor -- start 2024-09-16T07:23:36.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.509+0000 7feb5b000640 1 -- start start 2024-09-16T07:23:36.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb5b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb5406efa0 0x7feb541413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:36.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb5b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 0x7feb540cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb5b000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb54145750 0x7feb540cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb5b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb541600b0 con 0x7feb540ccb00 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb5b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feb5415ff30 con 0x7feb5406efa0 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb5b000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feb54160230 con 0x7feb54145750 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb53400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 0x7feb540cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb53400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 0x7feb540cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44794/0 (socket says 172.21.15.71:44794) 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.510+0000 7feb53400640 1 -- 172.21.15.71:0/2671224837 learned_addr learned my addr 172.21.15.71:0/2671224837 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb59000640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb54145750 0x7feb540cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb53e00640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb5406efa0 0x7feb541413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb53400640 1 -- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb54145750 msgr2=0x7feb540cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb53400640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb54145750 0x7feb540cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb53400640 1 -- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb5406efa0 msgr2=0x7feb541413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:36.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb53400640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb5406efa0 0x7feb541413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:36.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb53400640 1 -- 172.21.15.71:0/2671224837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb540b2de0 con 0x7feb540ccb00 2024-09-16T07:23:36.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb53400640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 0x7feb540cb000 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feb4000d780 tx=0x7feb4000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:36.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.511+0000 7feb4be00640 1 -- 172.21.15.71:0/2671224837 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40017070 con 0x7feb540ccb00 2024-09-16T07:23:36.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.512+0000 7feb4be00640 1 -- 172.21.15.71:0/2671224837 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb4000de90 con 0x7feb540ccb00 2024-09-16T07:23:36.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.512+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb540cbb30 con 0x7feb540ccb00 2024-09-16T07:23:36.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.512+0000 7feb4be00640 1 -- 172.21.15.71:0/2671224837 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40005020 con 0x7feb540ccb00 2024-09-16T07:23:36.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.512+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb540cc0f0 con 0x7feb540ccb00 2024-09-16T07:23:36.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.513+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb54160710 con 0x7feb540ccb00 2024-09-16T07:23:36.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.514+0000 7feb4be00640 1 -- 172.21.15.71:0/2671224837 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7feb400051c0 con 0x7feb540ccb00 2024-09-16T07:23:36.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.515+0000 7feb4be00640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb34078850 0x7feb3407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:36.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.515+0000 7feb4be00640 1 -- 172.21.15.71:0/2671224837 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7feb4009a830 con 0x7feb540ccb00 2024-09-16T07:23:36.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.515+0000 7feb53e00640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb34078850 0x7feb3407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:36.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.516+0000 7feb53e00640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb34078850 0x7feb3407ad10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb44005fd0 tx=0x7feb440095e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:36.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.520+0000 7feb4be00640 1 -- 172.21.15.71:0/2671224837 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feb40062990 con 0x7feb540ccb00 2024-09-16T07:23:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:36.704+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7feb540b2fb0 con 0x7feb540ccb00 2024-09-16T07:23:37.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.324+0000 7feb4be00640 1 -- 172.21.15.71:0/2671224837 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7feb400673c0 con 0x7feb540ccb00 2024-09-16T07:23:37.324 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb34078850 msgr2=0x7feb3407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb34078850 0x7feb3407ad10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb44005fd0 tx=0x7feb440095e0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 msgr2=0x7feb540cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 0x7feb540cb000 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feb4000d780 tx=0x7feb4000dc40 comp rx=0 tx=0).stop 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 shutdown_connections 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb34078850 0x7feb3407ad10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb54145750 0x7feb540cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540ccb00 0x7feb540cb000 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feb4000d780 tx=0x7feb4000dc40 comp rx=0 tx=0).stop 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.327+0000 7feb5b000640 1 --2- 172.21.15.71:0/2671224837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb5406efa0 0x7feb541413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.328+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 >> 172.21.15.71:0/2671224837 conn(0x7feb540b1840 msgr2=0x7feb540af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:37.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.328+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 shutdown_connections 2024-09-16T07:23:37.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.328+0000 7feb5b000640 1 -- 172.21.15.71:0/2671224837 wait complete. 2024-09-16T07:23:37.356 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:23:37.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 -- 172.21.15.71:0/993597773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c06efa0 msgr2=0x7f975c0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:37.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 --2- 172.21.15.71:0/993597773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c06efa0 0x7f975c0cc5c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f975c0cc550 tx=0x7f974802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 -- 172.21.15.71:0/993597773 shutdown_connections 2024-09-16T07:23:37.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 --2- 172.21.15.71:0/993597773 >> v2:172.21.15.177:3300/0 conn(0x7f975c0ccb00 0x7f975c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 --2- 172.21.15.71:0/993597773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c06efa0 0x7f975c0cc5c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 --2- 172.21.15.71:0/993597773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f975c145750 0x7f975c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 -- 172.21.15.71:0/993597773 >> 172.21.15.71:0/993597773 conn(0x7f975c0b1840 msgr2=0x7f975c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:37.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 -- 172.21.15.71:0/993597773 shutdown_connections 2024-09-16T07:23:37.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.517+0000 7f9763a00640 1 -- 172.21.15.71:0/993597773 wait complete. 2024-09-16T07:23:37.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.519+0000 7f9763a00640 1 Processor -- start 2024-09-16T07:23:37.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.519+0000 7f9763a00640 1 -- start start 2024-09-16T07:23:37.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f9763a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f975c06efa0 0x7f975c0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:37.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f9763a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c0ccb00 0x7f975c145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:37.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f9763a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f975c145750 0x7f975c146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:37.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f9763a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1600b0 con 0x7f975c0ccb00 2024-09-16T07:23:37.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f9763a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f975c15ff30 con 0x7f975c06efa0 2024-09-16T07:23:37.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f9763a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f975c160230 con 0x7f975c145750 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f975be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c0ccb00 0x7f975c145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f975be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c0ccb00 0x7f975c145d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44820/0 (socket says 172.21.15.71:44820) 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f975be00640 1 -- 172.21.15.71:0/1812216930 learned_addr learned my addr 172.21.15.71:0/1812216930 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f9761000640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f975c06efa0 0x7f975c0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.520+0000 7f9761a00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f975c145750 0x7f975c146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f975be00640 1 -- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f975c145750 msgr2=0x7f975c146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f975be00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f975c145750 0x7f975c146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f975be00640 1 -- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f975c06efa0 msgr2=0x7f975c0cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f975be00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f975c06efa0 0x7f975c0cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f975be00640 1 -- 172.21.15.71:0/1812216930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c0b37d0 con 0x7f975c0ccb00 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f9761a00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f975c145750 0x7f975c146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:37.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f975be00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c0ccb00 0x7f975c145d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9748000c00 tx=0x7f97480039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:37.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f9758c00640 1 -- 172.21.15.71:0/1812216930 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974803f070 con 0x7f975c0ccb00 2024-09-16T07:23:37.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f9758c00640 1 -- 172.21.15.71:0/1812216930 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9748004890 con 0x7f975c0ccb00 2024-09-16T07:23:37.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f9758c00640 1 -- 172.21.15.71:0/1812216930 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748035030 con 0x7f975c0ccb00 2024-09-16T07:23:37.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c0b3ab0 con 0x7f975c0ccb00 2024-09-16T07:23:37.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.521+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c0b4070 con 0x7f975c0ccb00 2024-09-16T07:23:37.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.523+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c160750 con 0x7f975c0ccb00 2024-09-16T07:23:37.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.524+0000 7f9758c00640 1 -- 172.21.15.71:0/1812216930 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9748042070 con 0x7f975c0ccb00 2024-09-16T07:23:37.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.525+0000 7f9758c00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f973c078850 0x7f973c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:37.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.526+0000 7f9758c00640 1 -- 172.21.15.71:0/1812216930 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7f97480bb7b0 con 0x7f975c0ccb00 2024-09-16T07:23:37.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.526+0000 7f9761000640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f973c078850 0x7f973c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:37.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.527+0000 7f9761000640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f973c078850 0x7f973c07ad10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f974c0098e0 tx=0x7f974c009060 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.530+0000 7f9758c00640 1 -- 172.21.15.71:0/1812216930 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9748083820 con 0x7f975c0ccb00 2024-09-16T07:23:37.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:37.750+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f975c0a9d00 con 0x7f975c0ccb00 2024-09-16T07:23:38.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.326+0000 7f9758c00640 1 -- 172.21.15.71:0/1812216930 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f9748088250 con 0x7f975c0ccb00 2024-09-16T07:23:38.326 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 4 and data pool 5 2024-09-16T07:23:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.330+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f973c078850 msgr2=0x7f973c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.330+0000 7f9763a00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f973c078850 0x7f973c07ad10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f974c0098e0 tx=0x7f974c009060 comp rx=0 tx=0).stop 2024-09-16T07:23:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.330+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c0ccb00 msgr2=0x7f975c145d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.330+0000 7f9763a00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c0ccb00 0x7f975c145d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9748000c00 tx=0x7f97480039e0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.331+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 shutdown_connections 2024-09-16T07:23:38.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.331+0000 7f9763a00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f973c078850 0x7f973c07ad10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.331+0000 7f9763a00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f975c145750 0x7f975c146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.331+0000 7f9763a00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f975c0ccb00 0x7f975c145d30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.331+0000 7f9763a00640 1 --2- 172.21.15.71:0/1812216930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f975c06efa0 0x7f975c0cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.331+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 >> 172.21.15.71:0/1812216930 conn(0x7f975c0b1840 msgr2=0x7f975c06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:38.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.331+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 shutdown_connections 2024-09-16T07:23:38.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.331+0000 7f9763a00640 1 -- 172.21.15.71:0/1812216930 wait complete. 2024-09-16T07:23:38.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:23:38.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.524+0000 7fb0bc600640 1 -- 172.21.15.71:0/2574770196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 msgr2=0x7fb0b406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.524+0000 7fb0bc600640 1 --2- 172.21.15.71:0/2574770196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 0x7fb0b406f160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac005fd0 tx=0x7fb0ac02aa10 comp rx=0 tx=0).stop 2024-09-16T07:23:38.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.524+0000 7fb0bc600640 1 -- 172.21.15.71:0/2574770196 shutdown_connections 2024-09-16T07:23:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.524+0000 7fb0bc600640 1 --2- 172.21.15.71:0/2574770196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 0x7fb0b406f160 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.524+0000 7fb0bc600640 1 -- 172.21.15.71:0/2574770196 >> 172.21.15.71:0/2574770196 conn(0x7fb0b40b1950 msgr2=0x7fb0b40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.524+0000 7fb0bc600640 1 -- 172.21.15.71:0/2574770196 shutdown_connections 2024-09-16T07:23:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.524+0000 7fb0bc600640 1 -- 172.21.15.71:0/2574770196 wait complete. 2024-09-16T07:23:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.526+0000 7fb0bc600640 1 Processor -- start 2024-09-16T07:23:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.526+0000 7fb0bc600640 1 -- start start 2024-09-16T07:23:38.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0bc600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 0x7fb0b415e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0bc600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b40ccf00 0x7fb0b40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0bc600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b415f5a0 0x7fb0b40b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0bc600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4074840 con 0x7fb0b40b2440 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0bc600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b40746c0 con 0x7fb0b415f5a0 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0bc600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b406f6a0 con 0x7fb0b40ccf00 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 0x7fb0b415e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 0x7fb0b415e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44830/0 (socket says 172.21.15.71:44830) 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9c00640 1 -- 172.21.15.71:0/2705685279 learned_addr learned my addr 172.21.15.71:0/2705685279 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9200640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b40ccf00 0x7fb0b40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0ba600640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b415f5a0 0x7fb0b40b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9c00640 1 -- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b40ccf00 msgr2=0x7fb0b40b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9c00640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b40ccf00 0x7fb0b40b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9c00640 1 -- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b415f5a0 msgr2=0x7fb0b40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9c00640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b415f5a0 0x7fb0b40b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.527+0000 7fb0b9c00640 1 -- 172.21.15.71:0/2705685279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b41f5d10 con 0x7fb0b40b2440 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.528+0000 7fb0b9c00640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 0x7fb0b415e380 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c3a0 tx=0x7fb0a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.528+0000 7fb0b1600640 1 -- 172.21.15.71:0/2705685279 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8017070 con 0x7fb0b40b2440 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.528+0000 7fb0b1600640 1 -- 172.21.15.71:0/2705685279 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0a80053c0 con 0x7fb0b40b2440 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.528+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b40b38c0 con 0x7fb0b40b2440 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.528+0000 7fb0b1600640 1 -- 172.21.15.71:0/2705685279 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8003850 con 0x7fb0b40b2440 2024-09-16T07:23:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.528+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b40b3e80 con 0x7fb0b40b2440 2024-09-16T07:23:38.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.529+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b406f160 con 0x7fb0b40b2440 2024-09-16T07:23:38.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.530+0000 7fb0b1600640 1 -- 172.21.15.71:0/2705685279 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb0a801a070 con 0x7fb0b40b2440 2024-09-16T07:23:38.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.531+0000 7fb0b1600640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb094078850 0x7fb09407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:38.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.531+0000 7fb0b1600640 1 -- 172.21.15.71:0/2705685279 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb0a8099d40 con 0x7fb0b40b2440 2024-09-16T07:23:38.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.532+0000 7fb0b9200640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb094078850 0x7fb09407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:38.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.532+0000 7fb0b9200640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb094078850 0x7fb09407ad10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb0b41f8000 tx=0x7fb0a0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:38.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.536+0000 7fb0b1600640 1 -- 172.21.15.71:0/2705685279 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb0a8062310 con 0x7fb0b40b2440 2024-09-16T07:23:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.722+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0b40ad0b0 con 0x7fb0b40b2440 2024-09-16T07:23:38.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.723+0000 7fb0b1600640 1 -- 172.21.15.71:0/2705685279 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16694 (secure 0 0 0) 0x7fb0a8061c10 con 0x7fb0b40b2440 2024-09-16T07:23:38.723 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:38.724 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":34,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:23:37.752066+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:23:36.070239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T07:23:36.713734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:23:38.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb094078850 msgr2=0x7fb09407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb094078850 0x7fb09407ad10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb0b41f8000 tx=0x7fb0a0009040 comp rx=0 tx=0).stop 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 msgr2=0x7fb0b415e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 0x7fb0b415e380 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c3a0 tx=0x7fb0a800c860 comp rx=0 tx=0).stop 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 shutdown_connections 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb094078850 0x7fb09407ad10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b415f5a0 0x7fb0b40b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b40ccf00 0x7fb0b40b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 --2- 172.21.15.71:0/2705685279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b40b2440 0x7fb0b415e380 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 >> 172.21.15.71:0/2705685279 conn(0x7fb0b40b1950 msgr2=0x7fb0b40ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:38.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 shutdown_connections 2024-09-16T07:23:38.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.726+0000 7fb0bc600640 1 -- 172.21.15.71:0/2705685279 wait complete. 2024-09-16T07:23:38.742 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:23:38.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 -- 172.21.15.71:0/3042811634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e80ad270 msgr2=0x7f03e8073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 --2- 172.21.15.71:0/3042811634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e80ad270 0x7f03e8073fb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f03e80723f0 tx=0x7f03e002a980 comp rx=0 tx=0).stop 2024-09-16T07:23:38.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 -- 172.21.15.71:0/3042811634 shutdown_connections 2024-09-16T07:23:38.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 --2- 172.21.15.71:0/3042811634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e80ad270 0x7f03e8073fb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 --2- 172.21.15.71:0/3042811634 >> v2:172.21.15.177:3300/0 conn(0x7f03e80ccf00 0x7f03e80acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 --2- 172.21.15.71:0/3042811634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03e815f5a0 0x7f03e80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 -- 172.21.15.71:0/3042811634 >> 172.21.15.71:0/3042811634 conn(0x7f03e80b1950 msgr2=0x7f03e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 -- 172.21.15.71:0/3042811634 shutdown_connections 2024-09-16T07:23:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.898+0000 7f03ee400640 1 -- 172.21.15.71:0/3042811634 wait complete. 2024-09-16T07:23:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.900+0000 7f03ee400640 1 Processor -- start 2024-09-16T07:23:38.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.900+0000 7f03ee400640 1 -- start start 2024-09-16T07:23:38.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03ee400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03e80ad270 0x7f03e81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:38.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03ee400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03e80ccf00 0x7f03e81440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:38.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03ee400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e815f5a0 0x7f03e8144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:38.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03ee400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f03e806f160 con 0x7f03e815f5a0 2024-09-16T07:23:38.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03ee400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8074780 con 0x7f03e80ad270 2024-09-16T07:23:38.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03ee400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f03e806f2e0 con 0x7f03e80ccf00 2024-09-16T07:23:38.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e815f5a0 0x7f03e8144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:38.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e815f5a0 0x7f03e8144630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44846/0 (socket says 172.21.15.71:44846) 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e7e00640 1 -- 172.21.15.71:0/4289985929 learned_addr learned my addr 172.21.15.71:0/4289985929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e7400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03e80ad270 0x7f03e81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e6a00640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03e80ccf00 0x7f03e81440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e7e00640 1 -- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03e80ccf00 msgr2=0x7f03e81440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e7e00640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03e80ccf00 0x7f03e81440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e7e00640 1 -- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03e80ad270 msgr2=0x7f03e81527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.901+0000 7f03e7e00640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03e80ad270 0x7f03e81527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03e7e00640 1 -- 172.21.15.71:0/4289985929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e8144b70 con 0x7f03e815f5a0 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03e7400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03e80ad270 0x7f03e81527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:38.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03e6a00640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03e80ccf00 0x7f03e81440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:38.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03e7e00640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e815f5a0 0x7f03e8144630 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009d60 tx=0x7f03e0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:38.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03df400640 1 -- 172.21.15.71:0/4289985929 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0034050 con 0x7f03e815f5a0 2024-09-16T07:23:38.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03df400640 1 -- 172.21.15.71:0/4289985929 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03e0004450 con 0x7f03e815f5a0 2024-09-16T07:23:38.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03df400640 1 -- 172.21.15.71:0/4289985929 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0043440 con 0x7f03e815f5a0 2024-09-16T07:23:38.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e8150ce0 con 0x7f03e815f5a0 2024-09-16T07:23:38.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.902+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e81512a0 con 0x7f03e815f5a0 2024-09-16T07:23:38.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.903+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e806f870 con 0x7f03e815f5a0 2024-09-16T07:23:38.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.904+0000 7f03df400640 1 -- 172.21.15.71:0/4289985929 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f03e0037020 con 0x7f03e815f5a0 2024-09-16T07:23:38.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.905+0000 7f03df400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f03c8078850 0x7f03c807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:38.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.905+0000 7f03df400640 1 -- 172.21.15.71:0/4289985929 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f03e00bbd20 con 0x7f03e815f5a0 2024-09-16T07:23:38.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.906+0000 7f03e7400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f03c8078850 0x7f03c807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:38.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.907+0000 7f03e7400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f03c8078850 0x7f03c807ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f03e8152330 tx=0x7f03d8005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:38.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:38.909+0000 7f03df400640 1 -- 172.21.15.71:0/4289985929 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f03e0084340 con 0x7f03e815f5a0 2024-09-16T07:23:39.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.093+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03e80af9d0 con 0x7f03e815f5a0 2024-09-16T07:23:39.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.094+0000 7f03df400640 1 -- 172.21.15.71:0/4289985929 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16694 (secure 0 0 0) 0x7f03e0083c40 con 0x7f03e815f5a0 2024-09-16T07:23:39.094 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:39.095 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":34,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:23:37.752066+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:23:36.070239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T07:23:36.713734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f03c8078850 msgr2=0x7f03c807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f03c8078850 0x7f03c807ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f03e8152330 tx=0x7f03d8005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e815f5a0 msgr2=0x7f03e8144630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e815f5a0 0x7f03e8144630 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009d60 tx=0x7f03e0003f10 comp rx=0 tx=0).stop 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 shutdown_connections 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f03c8078850 0x7f03c807ad10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03e815f5a0 0x7f03e8144630 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03e80ccf00 0x7f03e81440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 --2- 172.21.15.71:0/4289985929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03e80ad270 0x7f03e81527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 >> 172.21.15.71:0/4289985929 conn(0x7f03e80b1950 msgr2=0x7f03e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:39.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 shutdown_connections 2024-09-16T07:23:39.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.097+0000 7f03ee400640 1 -- 172.21.15.71:0/4289985929 wait complete. 2024-09-16T07:23:39.112 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:23:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.272+0000 7fab3b200640 1 -- 172.21.15.71:0/618577749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 msgr2=0x7fab340cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.272+0000 7fab3b200640 1 --2- 172.21.15.71:0/618577749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 0x7fab340cebe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab24009830 tx=0x7fab2402a510 comp rx=0 tx=0).stop 2024-09-16T07:23:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.272+0000 7fab3b200640 1 -- 172.21.15.71:0/618577749 shutdown_connections 2024-09-16T07:23:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.272+0000 7fab3b200640 1 --2- 172.21.15.71:0/618577749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab3415f090 0x7fab340cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.272+0000 7fab3b200640 1 --2- 172.21.15.71:0/618577749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 0x7fab340cebe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.272+0000 7fab3b200640 1 -- 172.21.15.71:0/618577749 >> 172.21.15.71:0/618577749 conn(0x7fab340b1840 msgr2=0x7fab340b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.272+0000 7fab3b200640 1 -- 172.21.15.71:0/618577749 shutdown_connections 2024-09-16T07:23:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.272+0000 7fab3b200640 1 -- 172.21.15.71:0/618577749 wait complete. 2024-09-16T07:23:39.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.273+0000 7fab3b200640 1 Processor -- start 2024-09-16T07:23:39.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.273+0000 7fab3b200640 1 -- start start 2024-09-16T07:23:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.274+0000 7fab3b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 0x7fab3415db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.274+0000 7fab3b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab3415c080 0x7fab3415f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.274+0000 7fab3b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab3415f090 0x7fab3415ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.274+0000 7fab3b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab34164e50 con 0x7fab3415f090 2024-09-16T07:23:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.274+0000 7fab3b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fab34164cd0 con 0x7fab340b3370 2024-09-16T07:23:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.274+0000 7fab3b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fab34164fd0 con 0x7fab3415c080 2024-09-16T07:23:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab3415f090 0x7fab3415ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab3415f090 0x7fab3415ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44858/0 (socket says 172.21.15.71:44858) 2024-09-16T07:23:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 -- 172.21.15.71:0/2295675981 learned_addr learned my addr 172.21.15.71:0/2295675981 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab33e00640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 0x7fab3415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 -- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab3415c080 msgr2=0x7fab3415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab3415c080 0x7fab3415f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 -- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 msgr2=0x7fab3415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 0x7fab3415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 -- 172.21.15.71:0/2295675981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3415caf0 con 0x7fab3415f090 2024-09-16T07:23:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab33e00640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 0x7fab3415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.275+0000 7fab39200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab3415f090 0x7fab3415ff10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00ec20 tx=0x7fab2c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.276+0000 7fab2be00640 1 -- 172.21.15.71:0/2295675981 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c003b70 con 0x7fab3415f090 2024-09-16T07:23:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.276+0000 7fab2be00640 1 -- 172.21.15.71:0/2295675981 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab2c003e00 con 0x7fab3415f090 2024-09-16T07:23:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.276+0000 7fab2be00640 1 -- 172.21.15.71:0/2295675981 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c004280 con 0x7fab3415f090 2024-09-16T07:23:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.276+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3415cdd0 con 0x7fab3415f090 2024-09-16T07:23:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.276+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3415d390 con 0x7fab3415f090 2024-09-16T07:23:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.277+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3415e980 con 0x7fab3415f090 2024-09-16T07:23:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.278+0000 7fab2be00640 1 -- 172.21.15.71:0/2295675981 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fab2c004420 con 0x7fab3415f090 2024-09-16T07:23:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.279+0000 7fab2be00640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab14078850 0x7fab1407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.279+0000 7fab2be00640 1 -- 172.21.15.71:0/2295675981 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fab2c09b800 con 0x7fab3415f090 2024-09-16T07:23:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.279+0000 7fab33e00640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab14078850 0x7fab1407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:39.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.280+0000 7fab33e00640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab14078850 0x7fab1407ad10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fab24034000 tx=0x7fab24004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:39.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.283+0000 7fab2be00640 1 -- 172.21.15.71:0/2295675981 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fab2c063e60 con 0x7fab3415f090 2024-09-16T07:23:39.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:39.508+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fab3413c3a0 con 0x7fab3415f090 2024-09-16T07:23:40.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.353+0000 7fab2be00640 1 -- 172.21.15.71:0/2295675981 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fab2c063760 con 0x7fab3415f090 2024-09-16T07:23:40.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab14078850 msgr2=0x7fab1407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:40.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab14078850 0x7fab1407ad10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fab24034000 tx=0x7fab24004730 comp rx=0 tx=0).stop 2024-09-16T07:23:40.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab3415f090 msgr2=0x7fab3415ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:40.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab3415f090 0x7fab3415ff10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00ec20 tx=0x7fab2c00c6a0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 shutdown_connections 2024-09-16T07:23:40.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab14078850 0x7fab1407ad10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab3415f090 0x7fab3415ff10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab3415c080 0x7fab3415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 --2- 172.21.15.71:0/2295675981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab340b3370 0x7fab3415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.356+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 >> 172.21.15.71:0/2295675981 conn(0x7fab340b1840 msgr2=0x7fab340b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:40.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.357+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 shutdown_connections 2024-09-16T07:23:40.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.357+0000 7fab3b200640 1 -- 172.21.15.71:0/2295675981 wait complete. 2024-09-16T07:23:40.375 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:23:40.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:23:40.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.538+0000 7fdf75e00640 1 -- 172.21.15.71:0/1408262911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf700b43c0 msgr2=0x7fdf700d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:40.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.538+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1408262911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf700b43c0 0x7fdf700d2830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf68005fd0 tx=0x7fdf6802a820 comp rx=0 tx=0).stop 2024-09-16T07:23:40.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.538+0000 7fdf75e00640 1 -- 172.21.15.71:0/1408262911 shutdown_connections 2024-09-16T07:23:40.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.538+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1408262911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf700b43c0 0x7fdf700d2830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.538+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1408262911 >> v2:172.21.15.177:3300/0 conn(0x7fdf70161460 0x7fdf700d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.538+0000 7fdf75e00640 1 -- 172.21.15.71:0/1408262911 >> 172.21.15.71:0/1408262911 conn(0x7fdf7009d0b0 msgr2=0x7fdf700d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:40.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.538+0000 7fdf75e00640 1 -- 172.21.15.71:0/1408262911 shutdown_connections 2024-09-16T07:23:40.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.538+0000 7fdf75e00640 1 -- 172.21.15.71:0/1408262911 wait complete. 2024-09-16T07:23:40.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.540+0000 7fdf75e00640 1 Processor -- start 2024-09-16T07:23:40.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.540+0000 7fdf75e00640 1 -- start start 2024-09-16T07:23:40.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf700b43c0 0x7fdf700ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:40.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf700b5480 0x7fdf701f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:40.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf70161460 0x7fdf700b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:40.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf700cef30 con 0x7fdf70161460 2024-09-16T07:23:40.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdf700b5200 con 0x7fdf700b43c0 2024-09-16T07:23:40.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdf700cf0b0 con 0x7fdf700b5480 2024-09-16T07:23:40.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf70161460 0x7fdf700b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:40.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf70161460 0x7fdf700b91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44862/0 (socket says 172.21.15.71:44862) 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75400640 1 -- 172.21.15.71:0/1768332082 learned_addr learned my addr 172.21.15.71:0/1768332082 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf74a00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf700b43c0 0x7fdf700ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf6fe00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf700b5480 0x7fdf701f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75400640 1 -- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf700b5480 msgr2=0x7fdf701f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75400640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf700b5480 0x7fdf701f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75400640 1 -- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf700b43c0 msgr2=0x7fdf700ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75400640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf700b43c0 0x7fdf700ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.541+0000 7fdf75400640 1 -- 172.21.15.71:0/1768332082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf700b6de0 con 0x7fdf70161460 2024-09-16T07:23:40.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.542+0000 7fdf75400640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf70161460 0x7fdf700b91e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf68035040 tx=0x7fdf68032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:40.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.542+0000 7fdf6cc00640 1 -- 172.21.15.71:0/1768332082 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68037040 con 0x7fdf70161460 2024-09-16T07:23:40.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.542+0000 7fdf6cc00640 1 -- 172.21.15.71:0/1768332082 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf68042070 con 0x7fdf70161460 2024-09-16T07:23:40.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.542+0000 7fdf6cc00640 1 -- 172.21.15.71:0/1768332082 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68003e60 con 0x7fdf70161460 2024-09-16T07:23:40.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.542+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf700b70c0 con 0x7fdf70161460 2024-09-16T07:23:40.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.542+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf700b7680 con 0x7fdf70161460 2024-09-16T07:23:40.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.543+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf700b4b30 con 0x7fdf70161460 2024-09-16T07:23:40.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.544+0000 7fdf6cc00640 1 -- 172.21.15.71:0/1768332082 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fdf68004260 con 0x7fdf70161460 2024-09-16T07:23:40.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.545+0000 7fdf6cc00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf50078850 0x7fdf5007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:40.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.545+0000 7fdf6cc00640 1 -- 172.21.15.71:0/1768332082 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdf680bb450 con 0x7fdf70161460 2024-09-16T07:23:40.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.546+0000 7fdf74a00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf50078850 0x7fdf5007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:40.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.547+0000 7fdf74a00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf50078850 0x7fdf5007ad10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009b30 tx=0x7fdf64005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:40.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.550+0000 7fdf6cc00640 1 -- 172.21.15.71:0/1768332082 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdf700b4b30 con 0x7fdf70161460 2024-09-16T07:23:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:40.773+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdf700ce820 con 0x7fdf70161460 2024-09-16T07:23:41.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.362+0000 7fdf6cc00640 1 -- 172.21.15.71:0/1768332082 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fdf680833d0 con 0x7fdf70161460 2024-09-16T07:23:41.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.364+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf50078850 msgr2=0x7fdf5007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.364+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf50078850 0x7fdf5007ad10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009b30 tx=0x7fdf64005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:41.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.364+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf70161460 msgr2=0x7fdf700b91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.364+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf70161460 0x7fdf700b91e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf68035040 tx=0x7fdf68032c60 comp rx=0 tx=0).stop 2024-09-16T07:23:41.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.365+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 shutdown_connections 2024-09-16T07:23:41.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.365+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf50078850 0x7fdf5007ad10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.365+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf70161460 0x7fdf700b91e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.365+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf700b5480 0x7fdf701f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.365+0000 7fdf75e00640 1 --2- 172.21.15.71:0/1768332082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf700b43c0 0x7fdf700ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.365+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 >> 172.21.15.71:0/1768332082 conn(0x7fdf7009d0b0 msgr2=0x7fdf700b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:41.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.365+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 shutdown_connections 2024-09-16T07:23:41.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.365+0000 7fdf75e00640 1 -- 172.21.15.71:0/1768332082 wait complete. 2024-09-16T07:23:41.381 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:23:41.382 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.538+0000 7f4976800640 1 -- 172.21.15.71:0/2557637419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 msgr2=0x7f497006f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.538+0000 7f4976800640 1 --2- 172.21.15.71:0/2557637419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 0x7f497006f320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4970073b00 tx=0x7f496802a540 comp rx=0 tx=0).stop 2024-09-16T07:23:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.539+0000 7f4976800640 1 -- 172.21.15.71:0/2557637419 shutdown_connections 2024-09-16T07:23:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.539+0000 7f4976800640 1 --2- 172.21.15.71:0/2557637419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 0x7f497006f320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.539+0000 7f4976800640 1 --2- 172.21.15.71:0/2557637419 >> v2:172.21.15.177:3300/0 conn(0x7f49700725d0 0x7f49700b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.539+0000 7f4976800640 1 -- 172.21.15.71:0/2557637419 >> 172.21.15.71:0/2557637419 conn(0x7f49700cfa20 msgr2=0x7f49700cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.539+0000 7f4976800640 1 -- 172.21.15.71:0/2557637419 shutdown_connections 2024-09-16T07:23:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.539+0000 7f4976800640 1 -- 172.21.15.71:0/2557637419 wait complete. 2024-09-16T07:23:41.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.540+0000 7f4976800640 1 Processor -- start 2024-09-16T07:23:41.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.540+0000 7f4976800640 1 -- start start 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.541+0000 7f4976800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 0x7f4970155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.541+0000 7f4976800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497015bc30 0x7f4970148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.541+0000 7f4976800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970154780 0x7f4970156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.541+0000 7f4976800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4970074660 con 0x7f49700d0540 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.541+0000 7f4976800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f497006f860 con 0x7f4970154780 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.541+0000 7f4976800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4970154c40 con 0x7f497015bc30 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.541+0000 7f4975400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 0x7f4970155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 0x7f4970155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44884/0 (socket says 172.21.15.71:44884) 2024-09-16T07:23:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975400640 1 -- 172.21.15.71:0/994486126 learned_addr learned my addr 172.21.15.71:0/994486126 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975e00640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970154780 0x7f4970156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4974a00640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497015bc30 0x7f4970148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975400640 1 -- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497015bc30 msgr2=0x7f4970148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975400640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497015bc30 0x7f4970148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975400640 1 -- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970154780 msgr2=0x7f4970156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975400640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970154780 0x7f4970156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975400640 1 -- 172.21.15.71:0/994486126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4970154f20 con 0x7f49700d0540 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f4975400640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 0x7f4970155ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f496400cb40 tx=0x7f496400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f496d600640 1 -- 172.21.15.71:0/994486126 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964016020 con 0x7f49700d0540 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.542+0000 7f496d600640 1 -- 172.21.15.71:0/994486126 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f496400adb0 con 0x7f49700d0540 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.543+0000 7f496d600640 1 -- 172.21.15.71:0/994486126 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964012680 con 0x7f49700d0540 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.543+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4970155200 con 0x7f49700d0540 2024-09-16T07:23:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.543+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49701557c0 con 0x7f49700d0540 2024-09-16T07:23:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.544+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49701559c0 con 0x7f49700d0540 2024-09-16T07:23:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.545+0000 7f496d600640 1 -- 172.21.15.71:0/994486126 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4964019020 con 0x7f49700d0540 2024-09-16T07:23:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.546+0000 7f496d600640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4950078850 0x7f495007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.546+0000 7f496d600640 1 -- 172.21.15.71:0/994486126 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f496409aea0 con 0x7f49700d0540 2024-09-16T07:23:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.546+0000 7f4974a00640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4950078850 0x7f495007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.547+0000 7f4974a00640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4950078850 0x7f495007ad10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4970152b90 tx=0x7f495c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:41.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.550+0000 7f496d600640 1 -- 172.21.15.71:0/994486126 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4964020020 con 0x7f49700d0540 2024-09-16T07:23:41.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.770+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f497006fdf0 con 0x7f49700d0540 2024-09-16T07:23:41.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.771+0000 7f496d600640 1 -- 172.21.15.71:0/994486126 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f4964062e20 con 0x7f49700d0540 2024-09-16T07:23:41.771 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:41.771 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"btime":"2024-09-16T07:23:41:354934+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4834,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3085563418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3085563418},{"type":"v1","addr":"172.21.15.71:6834","nonce":3085563418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3025435692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3025435692},{"type":"v1","addr":"172.21.15.71:6835","nonce":3025435692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3253921551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":3253921551},{"type":"v1","addr":"172.21.15.177:6836","nonce":3253921551}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:23:37.752030+0000","modified":"2024-09-16T07:23:40.774583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1652086843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1652086843},{"type":"v1","addr":"172.21.15.177:6837","nonce":1652086843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4917,"qdb_cluster":[4917]},"id":2}]} 2024-09-16T07:23:41.771 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 17 2024-09-16T07:23:41.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4950078850 msgr2=0x7f495007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4950078850 0x7f495007ad10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4970152b90 tx=0x7f495c005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:41.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 msgr2=0x7f4970155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 0x7f4970155ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f496400cb40 tx=0x7f496400a060 comp rx=0 tx=0).stop 2024-09-16T07:23:41.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 shutdown_connections 2024-09-16T07:23:41.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4950078850 0x7f495007ad10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970154780 0x7f4970156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497015bc30 0x7f4970148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 --2- 172.21.15.71:0/994486126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49700d0540 0x7f4970155ce0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.773+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 >> 172.21.15.71:0/994486126 conn(0x7f49700cfa20 msgr2=0x7f49700b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:41.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.774+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 shutdown_connections 2024-09-16T07:23:41.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.774+0000 7f4976800640 1 -- 172.21.15.71:0/994486126 wait complete. 2024-09-16T07:23:41.789 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:23:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.948+0000 7f34fe800640 1 -- 172.21.15.71:0/3700543618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b1a00 msgr2=0x7f34f80b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.948+0000 7f34fe800640 1 --2- 172.21.15.71:0/3700543618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b1a00 0x7f34f80b1e90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f34f0005fd0 tx=0x7f34f002aa90 comp rx=0 tx=0).stop 2024-09-16T07:23:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.948+0000 7f34fe800640 1 -- 172.21.15.71:0/3700543618 shutdown_connections 2024-09-16T07:23:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.948+0000 7f34fe800640 1 --2- 172.21.15.71:0/3700543618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b1a00 0x7f34f80b1e90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.948+0000 7f34fe800640 1 --2- 172.21.15.71:0/3700543618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34f80d28c0 0x7f34f80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.948+0000 7f34fe800640 1 -- 172.21.15.71:0/3700543618 >> 172.21.15.71:0/3700543618 conn(0x7f34f80d1da0 msgr2=0x7f34f80d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.948+0000 7f34fe800640 1 -- 172.21.15.71:0/3700543618 shutdown_connections 2024-09-16T07:23:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.948+0000 7f34fe800640 1 -- 172.21.15.71:0/3700543618 wait complete. 2024-09-16T07:23:41.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.950+0000 7f34fe800640 1 Processor -- start 2024-09-16T07:23:41.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.950+0000 7f34fe800640 1 -- start start 2024-09-16T07:23:41.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34fe800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34f80b1a00 0x7f34f8159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:41.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34fe800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80d28c0 0x7f34f815ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:41.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34fe800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34f815c040 0x7f34f80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34fe800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34f80b2550 con 0x7f34f80d28c0 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34fe800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34f80b23d0 con 0x7f34f815c040 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34fe800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f34f80b26d0 con 0x7f34f80b1a00 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80d28c0 0x7f34f815ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80d28c0 0x7f34f815ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44900/0 (socket says 172.21.15.71:44900) 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f6a00640 1 -- 172.21.15.71:0/471610483 learned_addr learned my addr 172.21.15.71:0/471610483 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f7400640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34f80b1a00 0x7f34f8159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f7e00640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34f815c040 0x7f34f80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f6a00640 1 -- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34f80b1a00 msgr2=0x7f34f8159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f6a00640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34f80b1a00 0x7f34f8159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f6a00640 1 -- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34f815c040 msgr2=0x7f34f80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:41.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f6a00640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34f815c040 0x7f34f80c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:41.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f6a00640 1 -- 172.21.15.71:0/471610483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f815a180 con 0x7f34f80d28c0 2024-09-16T07:23:41.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.951+0000 7f34f7400640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34f80b1a00 0x7f34f8159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:41.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.952+0000 7f34f6a00640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80d28c0 0x7f34f815ef90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f34e400d950 tx=0x7f34e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:41.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.952+0000 7f34ef400640 1 -- 172.21.15.71:0/471610483 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4017070 con 0x7f34f80d28c0 2024-09-16T07:23:41.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.952+0000 7f34ef400640 1 -- 172.21.15.71:0/471610483 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34e4009d20 con 0x7f34f80d28c0 2024-09-16T07:23:41.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.952+0000 7f34ef400640 1 -- 172.21.15.71:0/471610483 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e40106c0 con 0x7f34f80d28c0 2024-09-16T07:23:41.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.952+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f81f4b30 con 0x7f34f80d28c0 2024-09-16T07:23:41.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.952+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81f4dd0 con 0x7f34f80d28c0 2024-09-16T07:23:41.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.953+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f80b2c60 con 0x7f34f80d28c0 2024-09-16T07:23:41.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.954+0000 7f34ef400640 1 -- 172.21.15.71:0/471610483 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f34e4010860 con 0x7f34f80d28c0 2024-09-16T07:23:41.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.956+0000 7f34ef400640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34d8078880 0x7f34d807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:41.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.956+0000 7f34ef400640 1 -- 172.21.15.71:0/471610483 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f34e409ad80 con 0x7f34f80d28c0 2024-09-16T07:23:41.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.956+0000 7f34f7400640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34d8078880 0x7f34d807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:41.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.957+0000 7f34f7400640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34d8078880 0x7f34d807ad40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34e8000f60 tx=0x7f34e8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:41.960+0000 7f34ef400640 1 -- 172.21.15.71:0/471610483 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f34e40633e0 con 0x7f34f80d28c0 2024-09-16T07:23:42.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.148+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34f815a9c0 con 0x7f34f80d28c0 2024-09-16T07:23:42.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.149+0000 7f34ef400640 1 -- 172.21.15.71:0/471610483 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7f34e4062ce0 con 0x7f34f80d28c0 2024-09-16T07:23:42.149 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:42.149 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":35,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:23:39.337992+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:23:36.070239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T07:23:36.713734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:23:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34d8078880 msgr2=0x7f34d807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34d8078880 0x7f34d807ad40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34e8000f60 tx=0x7f34e8009040 comp rx=0 tx=0).stop 2024-09-16T07:23:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80d28c0 msgr2=0x7f34f815ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80d28c0 0x7f34f815ef90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f34e400d950 tx=0x7f34e400de10 comp rx=0 tx=0).stop 2024-09-16T07:23:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 shutdown_connections 2024-09-16T07:23:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34d8078880 0x7f34d807ad40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34f815c040 0x7f34f80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80d28c0 0x7f34f815ef90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 --2- 172.21.15.71:0/471610483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34f80b1a00 0x7f34f8159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 >> 172.21.15.71:0/471610483 conn(0x7f34f80d1da0 msgr2=0x7f34f80d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 shutdown_connections 2024-09-16T07:23:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.152+0000 7f34fe800640 1 -- 172.21.15.71:0/471610483 wait complete. 2024-09-16T07:23:42.169 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:23:42.169 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:23:42.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.328+0000 7ff94d400640 1 -- 172.21.15.71:0/2350456069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480b2440 msgr2=0x7ff94806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.328+0000 7ff94d400640 1 --2- 172.21.15.71:0/2350456069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480b2440 0x7ff94806f160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff940005fd0 tx=0x7ff94002a8b0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.328+0000 7ff94d400640 1 -- 172.21.15.71:0/2350456069 shutdown_connections 2024-09-16T07:23:42.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.328+0000 7ff94d400640 1 --2- 172.21.15.71:0/2350456069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480b2440 0x7ff94806f160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.328+0000 7ff94d400640 1 -- 172.21.15.71:0/2350456069 >> 172.21.15.71:0/2350456069 conn(0x7ff9480b1950 msgr2=0x7ff9480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:42.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.328+0000 7ff94d400640 1 -- 172.21.15.71:0/2350456069 shutdown_connections 2024-09-16T07:23:42.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.328+0000 7ff94d400640 1 -- 172.21.15.71:0/2350456069 wait complete. 2024-09-16T07:23:42.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.330+0000 7ff94d400640 1 Processor -- start 2024-09-16T07:23:42.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.330+0000 7ff94d400640 1 -- start start 2024-09-16T07:23:42.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff94d400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9480b2440 0x7ff94815bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:42.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff94d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480ccf00 0x7ff9480b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:42.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff94d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff94815f5a0 0x7ff94814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:42.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff94d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff948074840 con 0x7ff9480ccf00 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff94d400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff9480746c0 con 0x7ff94815f5a0 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff94d400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff94806f6a0 con 0x7ff9480b2440 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff945e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480ccf00 0x7ff9480b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff945e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480ccf00 0x7ff9480b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44922/0 (socket says 172.21.15.71:44922) 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.331+0000 7ff945e00640 1 -- 172.21.15.71:0/1740377307 learned_addr learned my addr 172.21.15.71:0/1740377307 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff946800640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9480b2440 0x7ff94815bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff945e00640 1 -- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9480b2440 msgr2=0x7ff94815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff947200640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff94815f5a0 0x7ff94814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff945e00640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9480b2440 0x7ff94815bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff945e00640 1 -- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff94815f5a0 msgr2=0x7ff94814f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff945e00640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff94815f5a0 0x7ff94814f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff945e00640 1 -- 172.21.15.71:0/1740377307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94814f7e0 con 0x7ff9480ccf00 2024-09-16T07:23:42.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff947200640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff94815f5a0 0x7ff94814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:42.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff946800640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9480b2440 0x7ff94815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:42.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff945e00640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480ccf00 0x7ff9480b3700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c500 tx=0x7ff93400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:42.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff93ea00640 1 -- 172.21.15.71:0/1740377307 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9340052d0 con 0x7ff9480ccf00 2024-09-16T07:23:42.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff93ea00640 1 -- 172.21.15.71:0/1740377307 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff934005470 con 0x7ff9480ccf00 2024-09-16T07:23:42.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff93ea00640 1 -- 172.21.15.71:0/1740377307 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934012870 con 0x7ff9480ccf00 2024-09-16T07:23:42.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9481f4830 con 0x7ff9480ccf00 2024-09-16T07:23:42.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.332+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9481f4df0 con 0x7ff9480ccf00 2024-09-16T07:23:42.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.333+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94806f160 con 0x7ff9480ccf00 2024-09-16T07:23:42.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.335+0000 7ff93ea00640 1 -- 172.21.15.71:0/1740377307 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff9340035f0 con 0x7ff9480ccf00 2024-09-16T07:23:42.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.336+0000 7ff93ea00640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff928078850 0x7ff92807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:42.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.336+0000 7ff93ea00640 1 -- 172.21.15.71:0/1740377307 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7ff93409a4d0 con 0x7ff9480ccf00 2024-09-16T07:23:42.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.336+0000 7ff946800640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff928078850 0x7ff92807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:42.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.337+0000 7ff946800640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff928078850 0x7ff92807ad10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9380046b0 tx=0x7ff938009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:42.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.340+0000 7ff93ea00640 1 -- 172.21.15.71:0/1740377307 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff934062ab0 con 0x7ff9480ccf00 2024-09-16T07:23:42.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.526+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff948153ec0 con 0x7ff928078850 2024-09-16T07:23:42.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.532+0000 7ff93ea00640 1 -- 172.21.15.71:0/1740377307 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250440 (secure 0 0 0) 0x7ff948153ec0 con 0x7ff928078850 2024-09-16T07:23:42.536 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:42.538 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:23:42.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff928078850 msgr2=0x7ff92807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff928078850 0x7ff92807ad10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9380046b0 tx=0x7ff938009040 comp rx=0 tx=0).stop 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480ccf00 msgr2=0x7ff9480b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480ccf00 0x7ff9480b3700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c500 tx=0x7ff93400c9c0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 shutdown_connections 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff928078850 0x7ff92807ad10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff94815f5a0 0x7ff94814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9480ccf00 0x7ff9480b3700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 --2- 172.21.15.71:0/1740377307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9480b2440 0x7ff94815bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 >> 172.21.15.71:0/1740377307 conn(0x7ff9480b1950 msgr2=0x7ff94809cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 shutdown_connections 2024-09-16T07:23:42.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.536+0000 7ff94d400640 1 -- 172.21.15.71:0/1740377307 wait complete. 2024-09-16T07:23:42.551 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-16T07:23:42.305320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":119,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224400,"kb_used_data":3992,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750320,"statfs":{"total":773094113280,"available":772864327680,"internally_reserved":0,"allocated":4087808,"data_stored":2278407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.979386"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350368+0000","last_change":"2024-09-16T07:23:38.347812+0000","last_active":"2024-09-16T07:23:38.350368+0000","last_peered":"2024-09-16T07:23:38.350368+0000","last_clean":"2024-09-16T07:23:38.350368+0000","last_became_active":"2024-09-16T07:23:38.347516+0000","last_became_peered":"2024-09-16T07:23:38.347516+0000","last_unstale":"2024-09-16T07:23:38.350368+0000","last_undegraded":"2024-09-16T07:23:38.350368+0000","last_fullsized":"2024-09-16T07:23:38.350368+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:06.049071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334422+0000","last_change":"2024-09-16T07:23:37.334422+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334422+0000","last_undegraded":"2024-09-16T07:23:37.334422+0000","last_fullsized":"2024-09-16T07:23:37.334422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.360416+0000","last_change":"2024-09-16T07:23:38.352895+0000","last_active":"2024-09-16T07:23:38.360416+0000","last_peered":"2024-09-16T07:23:38.360416+0000","last_clean":"2024-09-16T07:23:38.360416+0000","last_became_active":"2024-09-16T07:23:38.352621+0000","last_became_peered":"2024-09-16T07:23:38.352621+0000","last_unstale":"2024-09-16T07:23:38.360416+0000","last_undegraded":"2024-09-16T07:23:38.360416+0000","last_fullsized":"2024-09-16T07:23:38.360416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:14.142804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615292+0000","last_change":"2024-09-16T07:23:38.350692+0000","last_active":"2024-09-16T07:23:40.615292+0000","last_peered":"2024-09-16T07:23:40.615292+0000","last_clean":"2024-09-16T07:23:40.615292+0000","last_became_active":"2024-09-16T07:23:38.347475+0000","last_became_peered":"2024-09-16T07:23:38.347475+0000","last_unstale":"2024-09-16T07:23:40.615292+0000","last_undegraded":"2024-09-16T07:23:40.615292+0000","last_fullsized":"2024-09-16T07:23:40.615292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:01.740131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.331162+0000","last_change":"2024-09-16T07:23:37.331162+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.331162+0000","last_undegraded":"2024-09-16T07:23:37.331162+0000","last_fullsized":"2024-09-16T07:23:37.331162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334158+0000","last_change":"2024-09-16T07:23:37.334158+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334158+0000","last_undegraded":"2024-09-16T07:23:37.334158+0000","last_fullsized":"2024-09-16T07:23:37.334158+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.368246+0000","last_change":"2024-09-16T07:23:38.367300+0000","last_active":"2024-09-16T07:23:38.368246+0000","last_peered":"2024-09-16T07:23:38.368246+0000","last_clean":"2024-09-16T07:23:38.368246+0000","last_became_active":"2024-09-16T07:23:38.365509+0000","last_became_peered":"2024-09-16T07:23:38.365509+0000","last_unstale":"2024-09-16T07:23:38.368246+0000","last_undegraded":"2024-09-16T07:23:38.368246+0000","last_fullsized":"2024-09-16T07:23:38.368246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:19.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.358930+0000","last_change":"2024-09-16T07:23:38.357105+0000","last_active":"2024-09-16T07:23:38.358930+0000","last_peered":"2024-09-16T07:23:38.358930+0000","last_clean":"2024-09-16T07:23:38.358930+0000","last_became_active":"2024-09-16T07:23:38.355543+0000","last_became_peered":"2024-09-16T07:23:38.355543+0000","last_unstale":"2024-09-16T07:23:38.358930+0000","last_undegraded":"2024-09-16T07:23:38.358930+0000","last_fullsized":"2024-09-16T07:23:38.358930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:21.370546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359316+0000","last_change":"2024-09-16T07:23:38.356452+0000","last_active":"2024-09-16T07:23:38.359316+0000","last_peered":"2024-09-16T07:23:38.359316+0000","last_clean":"2024-09-16T07:23:38.359316+0000","last_became_active":"2024-09-16T07:23:38.355663+0000","last_became_peered":"2024-09-16T07:23:38.355663+0000","last_unstale":"2024-09-16T07:23:38.359316+0000","last_undegraded":"2024-09-16T07:23:38.359316+0000","last_fullsized":"2024-09-16T07:23:38.359316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.788335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615484+0000","last_change":"2024-09-16T07:23:38.356446+0000","last_active":"2024-09-16T07:23:40.615484+0000","last_peered":"2024-09-16T07:23:40.615484+0000","last_clean":"2024-09-16T07:23:40.615484+0000","last_became_active":"2024-09-16T07:23:38.354502+0000","last_became_peered":"2024-09-16T07:23:38.354502+0000","last_unstale":"2024-09-16T07:23:40.615484+0000","last_undegraded":"2024-09-16T07:23:40.615484+0000","last_fullsized":"2024-09-16T07:23:40.615484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:27.342707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352817+0000","last_change":"2024-09-16T07:23:38.351223+0000","last_active":"2024-09-16T07:23:38.352817+0000","last_peered":"2024-09-16T07:23:38.352817+0000","last_clean":"2024-09-16T07:23:38.352817+0000","last_became_active":"2024-09-16T07:23:38.349810+0000","last_became_peered":"2024-09-16T07:23:38.349810+0000","last_unstale":"2024-09-16T07:23:38.352817+0000","last_undegraded":"2024-09-16T07:23:38.352817+0000","last_fullsized":"2024-09-16T07:23:38.352817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:37.812192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.367826+0000","last_change":"2024-09-16T07:23:38.365322+0000","last_active":"2024-09-16T07:23:38.367826+0000","last_peered":"2024-09-16T07:23:38.367826+0000","last_clean":"2024-09-16T07:23:38.367826+0000","last_became_active":"2024-09-16T07:23:38.364995+0000","last_became_peered":"2024-09-16T07:23:38.364995+0000","last_unstale":"2024-09-16T07:23:38.367826+0000","last_undegraded":"2024-09-16T07:23:38.367826+0000","last_fullsized":"2024-09-16T07:23:38.367826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:40.799346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.335382+0000","last_change":"2024-09-16T07:23:37.335382+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.335382+0000","last_undegraded":"2024-09-16T07:23:37.335382+0000","last_fullsized":"2024-09-16T07:23:37.335382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614877+0000","last_change":"2024-09-16T07:23:38.347109+0000","last_active":"2024-09-16T07:23:40.614877+0000","last_peered":"2024-09-16T07:23:40.614877+0000","last_clean":"2024-09-16T07:23:40.614877+0000","last_became_active":"2024-09-16T07:23:38.346821+0000","last_became_peered":"2024-09-16T07:23:38.346821+0000","last_unstale":"2024-09-16T07:23:40.614877+0000","last_undegraded":"2024-09-16T07:23:40.614877+0000","last_fullsized":"2024-09-16T07:23:40.614877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:59.555576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.345227+0000","last_change":"2024-09-16T07:23:38.341149+0000","last_active":"2024-09-16T07:23:38.345227+0000","last_peered":"2024-09-16T07:23:38.345227+0000","last_clean":"2024-09-16T07:23:38.345227+0000","last_became_active":"2024-09-16T07:23:38.340794+0000","last_became_peered":"2024-09-16T07:23:38.340794+0000","last_unstale":"2024-09-16T07:23:38.345227+0000","last_undegraded":"2024-09-16T07:23:38.345227+0000","last_fullsized":"2024-09-16T07:23:38.345227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:17.507704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362540+0000","last_change":"2024-09-16T07:23:38.360203+0000","last_active":"2024-09-16T07:23:38.362540+0000","last_peered":"2024-09-16T07:23:38.362540+0000","last_clean":"2024-09-16T07:23:38.362540+0000","last_became_active":"2024-09-16T07:23:38.359933+0000","last_became_peered":"2024-09-16T07:23:38.359933+0000","last_unstale":"2024-09-16T07:23:38.362540+0000","last_undegraded":"2024-09-16T07:23:38.362540+0000","last_fullsized":"2024-09-16T07:23:38.362540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:55.869414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352904+0000","last_change":"2024-09-16T07:23:38.350866+0000","last_active":"2024-09-16T07:23:38.352904+0000","last_peered":"2024-09-16T07:23:38.352904+0000","last_clean":"2024-09-16T07:23:38.352904+0000","last_became_active":"2024-09-16T07:23:38.350617+0000","last_became_peered":"2024-09-16T07:23:38.350617+0000","last_unstale":"2024-09-16T07:23:38.352904+0000","last_undegraded":"2024-09-16T07:23:38.352904+0000","last_fullsized":"2024-09-16T07:23:38.352904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:14:27.853553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.338226+0000","last_change":"2024-09-16T07:23:37.338226+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.338226+0000","last_undegraded":"2024-09-16T07:23:37.338226+0000","last_fullsized":"2024-09-16T07:23:37.338226+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334511+0000","last_change":"2024-09-16T07:23:37.334511+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334511+0000","last_undegraded":"2024-09-16T07:23:37.334511+0000","last_fullsized":"2024-09-16T07:23:37.334511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.337607+0000","last_change":"2024-09-16T07:23:37.337607+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.337607+0000","last_undegraded":"2024-09-16T07:23:37.337607+0000","last_fullsized":"2024-09-16T07:23:37.337607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.357360+0000","last_change":"2024-09-16T07:23:38.355303+0000","last_active":"2024-09-16T07:23:38.357360+0000","last_peered":"2024-09-16T07:23:38.357360+0000","last_clean":"2024-09-16T07:23:38.357360+0000","last_became_active":"2024-09-16T07:23:38.355033+0000","last_became_peered":"2024-09-16T07:23:38.355033+0000","last_unstale":"2024-09-16T07:23:38.357360+0000","last_undegraded":"2024-09-16T07:23:38.357360+0000","last_fullsized":"2024-09-16T07:23:38.357360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:19.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350313+0000","last_change":"2024-09-16T07:23:38.345255+0000","last_active":"2024-09-16T07:23:38.350313+0000","last_peered":"2024-09-16T07:23:38.350313+0000","last_clean":"2024-09-16T07:23:38.350313+0000","last_became_active":"2024-09-16T07:23:38.344965+0000","last_became_peered":"2024-09-16T07:23:38.344965+0000","last_unstale":"2024-09-16T07:23:38.350313+0000","last_undegraded":"2024-09-16T07:23:38.350313+0000","last_fullsized":"2024-09-16T07:23:38.350313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:09.790205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334034+0000","last_change":"2024-09-16T07:23:37.334034+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334034+0000","last_undegraded":"2024-09-16T07:23:37.334034+0000","last_fullsized":"2024-09-16T07:23:37.334034+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352659+0000","last_change":"2024-09-16T07:23:38.348792+0000","last_active":"2024-09-16T07:23:38.352659+0000","last_peered":"2024-09-16T07:23:38.352659+0000","last_clean":"2024-09-16T07:23:38.352659+0000","last_became_active":"2024-09-16T07:23:38.345353+0000","last_became_peered":"2024-09-16T07:23:38.345353+0000","last_unstale":"2024-09-16T07:23:38.352659+0000","last_undegraded":"2024-09-16T07:23:38.352659+0000","last_fullsized":"2024-09-16T07:23:38.352659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:32.489176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615553+0000","last_change":"2024-09-16T07:23:38.351284+0000","last_active":"2024-09-16T07:23:40.615553+0000","last_peered":"2024-09-16T07:23:40.615553+0000","last_clean":"2024-09-16T07:23:40.615553+0000","last_became_active":"2024-09-16T07:23:38.350972+0000","last_became_peered":"2024-09-16T07:23:38.350972+0000","last_unstale":"2024-09-16T07:23:40.615553+0000","last_undegraded":"2024-09-16T07:23:40.615553+0000","last_fullsized":"2024-09-16T07:23:40.615553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:25.493457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615534+0000","last_change":"2024-09-16T07:23:38.356598+0000","last_active":"2024-09-16T07:23:40.615534+0000","last_peered":"2024-09-16T07:23:40.615534+0000","last_clean":"2024-09-16T07:23:40.615534+0000","last_became_active":"2024-09-16T07:23:38.356177+0000","last_became_peered":"2024-09-16T07:23:38.356177+0000","last_unstale":"2024-09-16T07:23:40.615534+0000","last_undegraded":"2024-09-16T07:23:40.615534+0000","last_fullsized":"2024-09-16T07:23:40.615534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:42.647875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.330534+0000","last_change":"2024-09-16T07:23:37.330534+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.330534+0000","last_undegraded":"2024-09-16T07:23:37.330534+0000","last_fullsized":"2024-09-16T07:23:37.330534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.364143+0000","last_change":"2024-09-16T07:23:38.362005+0000","last_active":"2024-09-16T07:23:38.364143+0000","last_peered":"2024-09-16T07:23:38.364143+0000","last_clean":"2024-09-16T07:23:38.364143+0000","last_became_active":"2024-09-16T07:23:38.361748+0000","last_became_peered":"2024-09-16T07:23:38.361748+0000","last_unstale":"2024-09-16T07:23:38.364143+0000","last_undegraded":"2024-09-16T07:23:38.364143+0000","last_fullsized":"2024-09-16T07:23:38.364143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:17.507704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.343486+0000","last_change":"2024-09-16T07:23:38.337747+0000","last_active":"2024-09-16T07:23:38.343486+0000","last_peered":"2024-09-16T07:23:38.343486+0000","last_clean":"2024-09-16T07:23:38.343486+0000","last_became_active":"2024-09-16T07:23:38.337455+0000","last_became_peered":"2024-09-16T07:23:38.337455+0000","last_unstale":"2024-09-16T07:23:38.343486+0000","last_undegraded":"2024-09-16T07:23:38.343486+0000","last_fullsized":"2024-09-16T07:23:38.343486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:21.370546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.351724+0000","last_change":"2024-09-16T07:23:38.348476+0000","last_active":"2024-09-16T07:23:38.351724+0000","last_peered":"2024-09-16T07:23:38.351724+0000","last_clean":"2024-09-16T07:23:38.351724+0000","last_became_active":"2024-09-16T07:23:38.348263+0000","last_became_peered":"2024-09-16T07:23:38.348263+0000","last_unstale":"2024-09-16T07:23:38.351724+0000","last_undegraded":"2024-09-16T07:23:38.351724+0000","last_fullsized":"2024-09-16T07:23:38.351724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:24.558417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.358843+0000","last_change":"2024-09-16T07:23:38.356216+0000","last_active":"2024-09-16T07:23:38.358843+0000","last_peered":"2024-09-16T07:23:38.358843+0000","last_clean":"2024-09-16T07:23:38.358843+0000","last_became_active":"2024-09-16T07:23:38.355971+0000","last_became_peered":"2024-09-16T07:23:38.355971+0000","last_unstale":"2024-09-16T07:23:38.358843+0000","last_undegraded":"2024-09-16T07:23:38.358843+0000","last_fullsized":"2024-09-16T07:23:38.358843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:25.955048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.346389+0000","last_change":"2024-09-16T07:23:38.342538+0000","last_active":"2024-09-16T07:23:38.346389+0000","last_peered":"2024-09-16T07:23:38.346389+0000","last_clean":"2024-09-16T07:23:38.346389+0000","last_became_active":"2024-09-16T07:23:38.342085+0000","last_became_peered":"2024-09-16T07:23:38.342085+0000","last_unstale":"2024-09-16T07:23:38.346389+0000","last_undegraded":"2024-09-16T07:23:38.346389+0000","last_fullsized":"2024-09-16T07:23:38.346389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:56.231147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333820+0000","last_change":"2024-09-16T07:23:37.333820+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333820+0000","last_undegraded":"2024-09-16T07:23:37.333820+0000","last_fullsized":"2024-09-16T07:23:37.333820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.375763+0000","last_change":"2024-09-16T07:23:37.346853+0000","last_active":"2024-09-16T07:23:38.375763+0000","last_peered":"2024-09-16T07:23:38.375763+0000","last_clean":"2024-09-16T07:23:38.375763+0000","last_became_active":"2024-09-16T07:23:37.344215+0000","last_became_peered":"2024-09-16T07:23:37.344215+0000","last_unstale":"2024-09-16T07:23:38.375763+0000","last_undegraded":"2024-09-16T07:23:38.375763+0000","last_fullsized":"2024-09-16T07:23:38.375763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:55.470787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350348+0000","last_change":"2024-09-16T07:23:38.347363+0000","last_active":"2024-09-16T07:23:38.350348+0000","last_peered":"2024-09-16T07:23:38.350348+0000","last_clean":"2024-09-16T07:23:38.350348+0000","last_became_active":"2024-09-16T07:23:38.347091+0000","last_became_peered":"2024-09-16T07:23:38.347091+0000","last_unstale":"2024-09-16T07:23:38.350348+0000","last_undegraded":"2024-09-16T07:23:38.350348+0000","last_fullsized":"2024-09-16T07:23:38.350348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:16.751332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615579+0000","last_change":"2024-09-16T07:23:37.349796+0000","last_active":"2024-09-16T07:23:40.615579+0000","last_peered":"2024-09-16T07:23:40.615579+0000","last_clean":"2024-09-16T07:23:40.615579+0000","last_became_active":"2024-09-16T07:23:37.349505+0000","last_became_peered":"2024-09-16T07:23:37.349505+0000","last_unstale":"2024-09-16T07:23:40.615579+0000","last_undegraded":"2024-09-16T07:23:40.615579+0000","last_fullsized":"2024-09-16T07:23:40.615579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:04.156099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.349933+0000","last_change":"2024-09-16T07:23:38.344847+0000","last_active":"2024-09-16T07:23:38.349933+0000","last_peered":"2024-09-16T07:23:38.349933+0000","last_clean":"2024-09-16T07:23:38.349933+0000","last_became_active":"2024-09-16T07:23:38.344502+0000","last_became_peered":"2024-09-16T07:23:38.344502+0000","last_unstale":"2024-09-16T07:23:38.349933+0000","last_undegraded":"2024-09-16T07:23:38.349933+0000","last_fullsized":"2024-09-16T07:23:38.349933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:26.398231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350064+0000","last_change":"2024-09-16T07:23:37.347456+0000","last_active":"2024-09-16T07:23:37.350064+0000","last_peered":"2024-09-16T07:23:37.350064+0000","last_clean":"2024-09-16T07:23:37.350064+0000","last_became_active":"2024-09-16T07:23:37.347152+0000","last_became_peered":"2024-09-16T07:23:37.347152+0000","last_unstale":"2024-09-16T07:23:37.350064+0000","last_undegraded":"2024-09-16T07:23:37.350064+0000","last_fullsized":"2024-09-16T07:23:37.350064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:51:05.553785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615486+0000","last_change":"2024-09-16T07:23:38.353195+0000","last_active":"2024-09-16T07:23:40.615486+0000","last_peered":"2024-09-16T07:23:40.615486+0000","last_clean":"2024-09-16T07:23:40.615486+0000","last_became_active":"2024-09-16T07:23:38.350558+0000","last_became_peered":"2024-09-16T07:23:38.350558+0000","last_unstale":"2024-09-16T07:23:40.615486+0000","last_undegraded":"2024-09-16T07:23:40.615486+0000","last_fullsized":"2024-09-16T07:23:40.615486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:23.647474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615384+0000","last_change":"2024-09-16T07:23:37.348375+0000","last_active":"2024-09-16T07:23:40.615384+0000","last_peered":"2024-09-16T07:23:40.615384+0000","last_clean":"2024-09-16T07:23:40.615384+0000","last_became_active":"2024-09-16T07:23:37.348096+0000","last_became_peered":"2024-09-16T07:23:37.348096+0000","last_unstale":"2024-09-16T07:23:40.615384+0000","last_undegraded":"2024-09-16T07:23:40.615384+0000","last_fullsized":"2024-09-16T07:23:40.615384+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:37.348737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359143+0000","last_change":"2024-09-16T07:23:38.351112+0000","last_active":"2024-09-16T07:23:38.359143+0000","last_peered":"2024-09-16T07:23:38.359143+0000","last_clean":"2024-09-16T07:23:38.359143+0000","last_became_active":"2024-09-16T07:23:38.350832+0000","last_became_peered":"2024-09-16T07:23:38.350832+0000","last_unstale":"2024-09-16T07:23:38.359143+0000","last_undegraded":"2024-09-16T07:23:38.359143+0000","last_fullsized":"2024-09-16T07:23:38.359143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:44.019182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359294+0000","last_change":"2024-09-16T07:23:37.347090+0000","last_active":"2024-09-16T07:23:38.359294+0000","last_peered":"2024-09-16T07:23:38.359294+0000","last_clean":"2024-09-16T07:23:38.359294+0000","last_became_active":"2024-09-16T07:23:37.346832+0000","last_became_peered":"2024-09-16T07:23:37.346832+0000","last_unstale":"2024-09-16T07:23:38.359294+0000","last_undegraded":"2024-09-16T07:23:38.359294+0000","last_fullsized":"2024-09-16T07:23:38.359294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:41.066939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.364807+0000","last_change":"2024-09-16T07:23:38.363179+0000","last_active":"2024-09-16T07:23:38.364807+0000","last_peered":"2024-09-16T07:23:38.364807+0000","last_clean":"2024-09-16T07:23:38.364807+0000","last_became_active":"2024-09-16T07:23:38.360647+0000","last_became_peered":"2024-09-16T07:23:38.360647+0000","last_unstale":"2024-09-16T07:23:38.364807+0000","last_undegraded":"2024-09-16T07:23:38.364807+0000","last_fullsized":"2024-09-16T07:23:38.364807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:24.558417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.363776+0000","last_change":"2024-09-16T07:23:37.344503+0000","last_active":"2024-09-16T07:23:38.363776+0000","last_peered":"2024-09-16T07:23:38.363776+0000","last_clean":"2024-09-16T07:23:38.363776+0000","last_became_active":"2024-09-16T07:23:37.344139+0000","last_became_peered":"2024-09-16T07:23:37.344139+0000","last_unstale":"2024-09-16T07:23:38.363776+0000","last_undegraded":"2024-09-16T07:23:38.363776+0000","last_fullsized":"2024-09-16T07:23:38.363776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:59.430414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352723+0000","last_change":"2024-09-16T07:23:38.348789+0000","last_active":"2024-09-16T07:23:38.352723+0000","last_peered":"2024-09-16T07:23:38.352723+0000","last_clean":"2024-09-16T07:23:38.352723+0000","last_became_active":"2024-09-16T07:23:38.348529+0000","last_became_peered":"2024-09-16T07:23:38.348529+0000","last_unstale":"2024-09-16T07:23:38.352723+0000","last_undegraded":"2024-09-16T07:23:38.352723+0000","last_fullsized":"2024-09-16T07:23:38.352723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:16.637145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350090+0000","last_change":"2024-09-16T07:23:37.348311+0000","last_active":"2024-09-16T07:23:37.350090+0000","last_peered":"2024-09-16T07:23:37.350090+0000","last_clean":"2024-09-16T07:23:37.350090+0000","last_became_active":"2024-09-16T07:23:37.348006+0000","last_became_peered":"2024-09-16T07:23:37.348006+0000","last_unstale":"2024-09-16T07:23:37.350090+0000","last_undegraded":"2024-09-16T07:23:37.350090+0000","last_fullsized":"2024-09-16T07:23:37.350090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:29.668774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614873+0000","last_change":"2024-09-16T07:23:38.352705+0000","last_active":"2024-09-16T07:23:40.614873+0000","last_peered":"2024-09-16T07:23:40.614873+0000","last_clean":"2024-09-16T07:23:40.614873+0000","last_became_active":"2024-09-16T07:23:38.352415+0000","last_became_peered":"2024-09-16T07:23:38.352415+0000","last_unstale":"2024-09-16T07:23:40.614873+0000","last_undegraded":"2024-09-16T07:23:40.614873+0000","last_fullsized":"2024-09-16T07:23:40.614873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:10.916617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.335903+0000","last_change":"2024-09-16T07:23:37.353028+0000","last_active":"2024-09-16T07:23:38.335903+0000","last_peered":"2024-09-16T07:23:38.335903+0000","last_clean":"2024-09-16T07:23:38.335903+0000","last_became_active":"2024-09-16T07:23:37.352718+0000","last_became_peered":"2024-09-16T07:23:37.352718+0000","last_unstale":"2024-09-16T07:23:38.335903+0000","last_undegraded":"2024-09-16T07:23:38.335903+0000","last_fullsized":"2024-09-16T07:23:38.335903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:41.965086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.360894+0000","last_change":"2024-09-16T07:23:38.353299+0000","last_active":"2024-09-16T07:23:38.360894+0000","last_peered":"2024-09-16T07:23:38.360894+0000","last_clean":"2024-09-16T07:23:38.360894+0000","last_became_active":"2024-09-16T07:23:38.352818+0000","last_became_peered":"2024-09-16T07:23:38.352818+0000","last_unstale":"2024-09-16T07:23:38.360894+0000","last_undegraded":"2024-09-16T07:23:38.360894+0000","last_fullsized":"2024-09-16T07:23:38.360894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:02.666636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.349313+0000","last_change":"2024-09-16T07:23:37.346845+0000","last_active":"2024-09-16T07:23:37.349313+0000","last_peered":"2024-09-16T07:23:37.349313+0000","last_clean":"2024-09-16T07:23:37.349313+0000","last_became_active":"2024-09-16T07:23:37.346574+0000","last_became_peered":"2024-09-16T07:23:37.346574+0000","last_unstale":"2024-09-16T07:23:37.349313+0000","last_undegraded":"2024-09-16T07:23:37.349313+0000","last_fullsized":"2024-09-16T07:23:37.349313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:30.460837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615689+0000","last_change":"2024-09-16T07:23:38.358332+0000","last_active":"2024-09-16T07:23:40.615689+0000","last_peered":"2024-09-16T07:23:40.615689+0000","last_clean":"2024-09-16T07:23:40.615689+0000","last_became_active":"2024-09-16T07:23:38.355145+0000","last_became_peered":"2024-09-16T07:23:38.355145+0000","last_unstale":"2024-09-16T07:23:40.615689+0000","last_undegraded":"2024-09-16T07:23:40.615689+0000","last_fullsized":"2024-09-16T07:23:40.615689+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.765410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346902+0000","last_change":"2024-09-16T07:23:37.345039+0000","last_active":"2024-09-16T07:23:37.346902+0000","last_peered":"2024-09-16T07:23:37.346902+0000","last_clean":"2024-09-16T07:23:37.346902+0000","last_became_active":"2024-09-16T07:23:37.344529+0000","last_became_peered":"2024-09-16T07:23:37.344529+0000","last_unstale":"2024-09-16T07:23:37.346902+0000","last_undegraded":"2024-09-16T07:23:37.346902+0000","last_fullsized":"2024-09-16T07:23:37.346902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:18.045656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615409+0000","last_change":"2024-09-16T07:23:38.344916+0000","last_active":"2024-09-16T07:23:40.615409+0000","last_peered":"2024-09-16T07:23:40.615409+0000","last_clean":"2024-09-16T07:23:40.615409+0000","last_became_active":"2024-09-16T07:23:38.344552+0000","last_became_peered":"2024-09-16T07:23:38.344552+0000","last_unstale":"2024-09-16T07:23:40.615409+0000","last_undegraded":"2024-09-16T07:23:40.615409+0000","last_fullsized":"2024-09-16T07:23:40.615409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:07:30.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.374362+0000","last_change":"2024-09-16T07:23:37.337958+0000","last_active":"2024-09-16T07:23:38.374362+0000","last_peered":"2024-09-16T07:23:38.374362+0000","last_clean":"2024-09-16T07:23:38.374362+0000","last_became_active":"2024-09-16T07:23:37.337681+0000","last_became_peered":"2024-09-16T07:23:37.337681+0000","last_unstale":"2024-09-16T07:23:38.374362+0000","last_undegraded":"2024-09-16T07:23:38.374362+0000","last_fullsized":"2024-09-16T07:23:38.374362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:31.334181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352729+0000","last_change":"2024-09-16T07:23:38.350404+0000","last_active":"2024-09-16T07:23:38.352729+0000","last_peered":"2024-09-16T07:23:38.352729+0000","last_clean":"2024-09-16T07:23:38.352729+0000","last_became_active":"2024-09-16T07:23:38.350111+0000","last_became_peered":"2024-09-16T07:23:38.350111+0000","last_unstale":"2024-09-16T07:23:38.352729+0000","last_undegraded":"2024-09-16T07:23:38.352729+0000","last_fullsized":"2024-09-16T07:23:38.352729+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:22.548419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.354301+0000","last_change":"2024-09-16T07:23:37.353132+0000","last_active":"2024-09-16T07:23:37.354301+0000","last_peered":"2024-09-16T07:23:37.354301+0000","last_clean":"2024-09-16T07:23:37.354301+0000","last_became_active":"2024-09-16T07:23:37.352306+0000","last_became_peered":"2024-09-16T07:23:37.352306+0000","last_unstale":"2024-09-16T07:23:37.354301+0000","last_undegraded":"2024-09-16T07:23:37.354301+0000","last_fullsized":"2024-09-16T07:23:37.354301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:30.195425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333905+0000","last_change":"2024-09-16T07:23:37.333905+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333905+0000","last_undegraded":"2024-09-16T07:23:37.333905+0000","last_fullsized":"2024-09-16T07:23:37.333905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.365888+0000","last_change":"2024-09-16T07:23:37.347964+0000","last_active":"2024-09-16T07:23:38.365888+0000","last_peered":"2024-09-16T07:23:38.365888+0000","last_clean":"2024-09-16T07:23:38.365888+0000","last_became_active":"2024-09-16T07:23:37.347687+0000","last_became_peered":"2024-09-16T07:23:37.347687+0000","last_unstale":"2024-09-16T07:23:38.365888+0000","last_undegraded":"2024-09-16T07:23:38.365888+0000","last_fullsized":"2024-09-16T07:23:38.365888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:24.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615132+0000","last_change":"2024-09-16T07:23:38.352259+0000","last_active":"2024-09-16T07:23:40.615132+0000","last_peered":"2024-09-16T07:23:40.615132+0000","last_clean":"2024-09-16T07:23:40.615132+0000","last_became_active":"2024-09-16T07:23:38.351935+0000","last_became_peered":"2024-09-16T07:23:38.351935+0000","last_unstale":"2024-09-16T07:23:40.615132+0000","last_undegraded":"2024-09-16T07:23:40.615132+0000","last_fullsized":"2024-09-16T07:23:40.615132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:09:10.158498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614756+0000","last_change":"2024-09-16T07:23:37.350953+0000","last_active":"2024-09-16T07:23:40.614756+0000","last_peered":"2024-09-16T07:23:40.614756+0000","last_clean":"2024-09-16T07:23:40.614756+0000","last_became_active":"2024-09-16T07:23:37.350621+0000","last_became_peered":"2024-09-16T07:23:37.350621+0000","last_unstale":"2024-09-16T07:23:40.614756+0000","last_undegraded":"2024-09-16T07:23:40.614756+0000","last_fullsized":"2024-09-16T07:23:40.614756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:34.314424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352682+0000","last_change":"2024-09-16T07:23:38.348443+0000","last_active":"2024-09-16T07:23:38.352682+0000","last_peered":"2024-09-16T07:23:38.352682+0000","last_clean":"2024-09-16T07:23:38.352682+0000","last_became_active":"2024-09-16T07:23:38.347974+0000","last_became_peered":"2024-09-16T07:23:38.347974+0000","last_unstale":"2024-09-16T07:23:38.352682+0000","last_undegraded":"2024-09-16T07:23:38.352682+0000","last_fullsized":"2024-09-16T07:23:38.352682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:25:27.231251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614598+0000","last_change":"2024-09-16T07:23:37.352837+0000","last_active":"2024-09-16T07:23:40.614598+0000","last_peered":"2024-09-16T07:23:40.614598+0000","last_clean":"2024-09-16T07:23:40.614598+0000","last_became_active":"2024-09-16T07:23:37.352566+0000","last_became_peered":"2024-09-16T07:23:37.352566+0000","last_unstale":"2024-09-16T07:23:40.614598+0000","last_undegraded":"2024-09-16T07:23:40.614598+0000","last_fullsized":"2024-09-16T07:23:40.614598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:20.188668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.347236+0000","last_change":"2024-09-16T07:23:38.344224+0000","last_active":"2024-09-16T07:23:38.347236+0000","last_peered":"2024-09-16T07:23:38.347236+0000","last_clean":"2024-09-16T07:23:38.347236+0000","last_became_active":"2024-09-16T07:23:38.343918+0000","last_became_peered":"2024-09-16T07:23:38.343918+0000","last_unstale":"2024-09-16T07:23:38.347236+0000","last_undegraded":"2024-09-16T07:23:38.347236+0000","last_fullsized":"2024-09-16T07:23:38.347236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:55.009205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350824+0000","last_change":"2024-09-16T07:23:37.349547+0000","last_active":"2024-09-16T07:23:37.350824+0000","last_peered":"2024-09-16T07:23:37.350824+0000","last_clean":"2024-09-16T07:23:37.350824+0000","last_became_active":"2024-09-16T07:23:37.349283+0000","last_became_peered":"2024-09-16T07:23:37.349283+0000","last_unstale":"2024-09-16T07:23:37.350824+0000","last_undegraded":"2024-09-16T07:23:37.350824+0000","last_fullsized":"2024-09-16T07:23:37.350824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:41.491685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.353115+0000","last_change":"2024-09-16T07:23:38.349714+0000","last_active":"2024-09-16T07:23:38.353115+0000","last_peered":"2024-09-16T07:23:38.353115+0000","last_clean":"2024-09-16T07:23:38.353115+0000","last_became_active":"2024-09-16T07:23:38.349431+0000","last_became_peered":"2024-09-16T07:23:38.349431+0000","last_unstale":"2024-09-16T07:23:38.353115+0000","last_undegraded":"2024-09-16T07:23:38.353115+0000","last_fullsized":"2024-09-16T07:23:38.353115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:28.910006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.353383+0000","last_change":"2024-09-16T07:23:37.352287+0000","last_active":"2024-09-16T07:23:37.353383+0000","last_peered":"2024-09-16T07:23:37.353383+0000","last_clean":"2024-09-16T07:23:37.353383+0000","last_became_active":"2024-09-16T07:23:37.351844+0000","last_became_peered":"2024-09-16T07:23:37.351844+0000","last_unstale":"2024-09-16T07:23:37.353383+0000","last_undegraded":"2024-09-16T07:23:37.353383+0000","last_fullsized":"2024-09-16T07:23:37.353383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:24.047148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359294+0000","last_change":"2024-09-16T07:23:38.356161+0000","last_active":"2024-09-16T07:23:38.359294+0000","last_peered":"2024-09-16T07:23:38.359294+0000","last_clean":"2024-09-16T07:23:38.359294+0000","last_became_active":"2024-09-16T07:23:38.354062+0000","last_became_peered":"2024-09-16T07:23:38.354062+0000","last_unstale":"2024-09-16T07:23:38.359294+0000","last_undegraded":"2024-09-16T07:23:38.359294+0000","last_fullsized":"2024-09-16T07:23:38.359294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:36.964109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.366712+0000","last_change":"2024-09-16T07:23:37.349714+0000","last_active":"2024-09-16T07:23:38.366712+0000","last_peered":"2024-09-16T07:23:38.366712+0000","last_clean":"2024-09-16T07:23:38.366712+0000","last_became_active":"2024-09-16T07:23:37.349169+0000","last_became_peered":"2024-09-16T07:23:37.349169+0000","last_unstale":"2024-09-16T07:23:38.366712+0000","last_undegraded":"2024-09-16T07:23:38.366712+0000","last_fullsized":"2024-09-16T07:23:38.366712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:36.809629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.331286+0000","last_change":"2024-09-16T07:23:37.331286+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.331286+0000","last_undegraded":"2024-09-16T07:23:37.331286+0000","last_fullsized":"2024-09-16T07:23:37.331286+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.374468+0000","last_change":"2024-09-16T07:23:37.345215+0000","last_active":"2024-09-16T07:23:38.374468+0000","last_peered":"2024-09-16T07:23:38.374468+0000","last_clean":"2024-09-16T07:23:38.374468+0000","last_became_active":"2024-09-16T07:23:37.344006+0000","last_became_peered":"2024-09-16T07:23:37.344006+0000","last_unstale":"2024-09-16T07:23:38.374468+0000","last_undegraded":"2024-09-16T07:23:38.374468+0000","last_fullsized":"2024-09-16T07:23:38.374468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:43.819327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334029+0000","last_change":"2024-09-16T07:23:37.334029+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334029+0000","last_undegraded":"2024-09-16T07:23:37.334029+0000","last_fullsized":"2024-09-16T07:23:37.334029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.330726+0000","last_change":"2024-09-16T07:23:37.353056+0000","last_active":"2024-09-16T07:23:38.330726+0000","last_peered":"2024-09-16T07:23:38.330726+0000","last_clean":"2024-09-16T07:23:38.330726+0000","last_became_active":"2024-09-16T07:23:37.352723+0000","last_became_peered":"2024-09-16T07:23:37.352723+0000","last_unstale":"2024-09-16T07:23:38.330726+0000","last_undegraded":"2024-09-16T07:23:38.330726+0000","last_fullsized":"2024-09-16T07:23:38.330726+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:24.951401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334406+0000","last_change":"2024-09-16T07:23:37.334406+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334406+0000","last_undegraded":"2024-09-16T07:23:37.334406+0000","last_fullsized":"2024-09-16T07:23:37.334406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350390+0000","last_change":"2024-09-16T07:23:37.347166+0000","last_active":"2024-09-16T07:23:37.350390+0000","last_peered":"2024-09-16T07:23:37.350390+0000","last_clean":"2024-09-16T07:23:37.350390+0000","last_became_active":"2024-09-16T07:23:37.346890+0000","last_became_peered":"2024-09-16T07:23:37.346890+0000","last_unstale":"2024-09-16T07:23:37.350390+0000","last_undegraded":"2024-09-16T07:23:37.350390+0000","last_fullsized":"2024-09-16T07:23:37.350390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:09.516212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.330484+0000","last_change":"2024-09-16T07:23:37.330484+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.330484+0000","last_undegraded":"2024-09-16T07:23:37.330484+0000","last_fullsized":"2024-09-16T07:23:37.330484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.336973+0000","last_change":"2024-09-16T07:23:37.349623+0000","last_active":"2024-09-16T07:23:38.336973+0000","last_peered":"2024-09-16T07:23:38.336973+0000","last_clean":"2024-09-16T07:23:38.336973+0000","last_became_active":"2024-09-16T07:23:37.349374+0000","last_became_peered":"2024-09-16T07:23:37.349374+0000","last_unstale":"2024-09-16T07:23:38.336973+0000","last_undegraded":"2024-09-16T07:23:38.336973+0000","last_fullsized":"2024-09-16T07:23:38.336973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:46.136303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.336554+0000","last_change":"2024-09-16T07:23:37.336554+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.336554+0000","last_undegraded":"2024-09-16T07:23:37.336554+0000","last_fullsized":"2024-09-16T07:23:37.336554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.343402+0000","last_change":"2024-09-16T07:23:37.347390+0000","last_active":"2024-09-16T07:23:38.343402+0000","last_peered":"2024-09-16T07:23:38.343402+0000","last_clean":"2024-09-16T07:23:38.343402+0000","last_became_active":"2024-09-16T07:23:37.346226+0000","last_became_peered":"2024-09-16T07:23:37.346226+0000","last_unstale":"2024-09-16T07:23:38.343402+0000","last_undegraded":"2024-09-16T07:23:38.343402+0000","last_fullsized":"2024-09-16T07:23:38.343402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:54.186625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359489+0000","last_change":"2024-09-16T07:23:38.353841+0000","last_active":"2024-09-16T07:23:38.359489+0000","last_peered":"2024-09-16T07:23:38.359489+0000","last_clean":"2024-09-16T07:23:38.359489+0000","last_became_active":"2024-09-16T07:23:38.352266+0000","last_became_peered":"2024-09-16T07:23:38.352266+0000","last_unstale":"2024-09-16T07:23:38.359489+0000","last_undegraded":"2024-09-16T07:23:38.359489+0000","last_fullsized":"2024-09-16T07:23:38.359489+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:01.093082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.334377+0000","last_change":"2024-09-16T07:23:37.339731+0000","last_active":"2024-09-16T07:23:38.334377+0000","last_peered":"2024-09-16T07:23:38.334377+0000","last_clean":"2024-09-16T07:23:38.334377+0000","last_became_active":"2024-09-16T07:23:37.339480+0000","last_became_peered":"2024-09-16T07:23:37.339480+0000","last_unstale":"2024-09-16T07:23:38.334377+0000","last_undegraded":"2024-09-16T07:23:38.334377+0000","last_fullsized":"2024-09-16T07:23:38.334377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:58.263118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.337719+0000","last_change":"2024-09-16T07:23:37.337719+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.337719+0000","last_undegraded":"2024-09-16T07:23:37.337719+0000","last_fullsized":"2024-09-16T07:23:37.337719+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346812+0000","last_change":"2024-09-16T07:23:37.344100+0000","last_active":"2024-09-16T07:23:37.346812+0000","last_peered":"2024-09-16T07:23:37.346812+0000","last_clean":"2024-09-16T07:23:37.346812+0000","last_became_active":"2024-09-16T07:23:37.343269+0000","last_became_peered":"2024-09-16T07:23:37.343269+0000","last_unstale":"2024-09-16T07:23:37.346812+0000","last_undegraded":"2024-09-16T07:23:37.346812+0000","last_fullsized":"2024-09-16T07:23:37.346812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:39.795699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362195+0000","last_change":"2024-09-16T07:23:38.354226+0000","last_active":"2024-09-16T07:23:38.362195+0000","last_peered":"2024-09-16T07:23:38.362195+0000","last_clean":"2024-09-16T07:23:38.362195+0000","last_became_active":"2024-09-16T07:23:38.354001+0000","last_became_peered":"2024-09-16T07:23:38.354001+0000","last_unstale":"2024-09-16T07:23:38.362195+0000","last_undegraded":"2024-09-16T07:23:38.362195+0000","last_fullsized":"2024-09-16T07:23:38.362195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:16.310897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.345689+0000","last_change":"2024-09-16T07:23:37.343487+0000","last_active":"2024-09-16T07:23:37.345689+0000","last_peered":"2024-09-16T07:23:37.345689+0000","last_clean":"2024-09-16T07:23:37.345689+0000","last_became_active":"2024-09-16T07:23:37.343170+0000","last_became_peered":"2024-09-16T07:23:37.343170+0000","last_unstale":"2024-09-16T07:23:37.345689+0000","last_undegraded":"2024-09-16T07:23:37.345689+0000","last_fullsized":"2024-09-16T07:23:37.345689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:10.785300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333534+0000","last_change":"2024-09-16T07:23:37.333534+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333534+0000","last_undegraded":"2024-09-16T07:23:37.333534+0000","last_fullsized":"2024-09-16T07:23:37.333534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.348747+0000","last_change":"2024-09-16T07:23:37.346738+0000","last_active":"2024-09-16T07:23:37.348747+0000","last_peered":"2024-09-16T07:23:37.348747+0000","last_clean":"2024-09-16T07:23:37.348747+0000","last_became_active":"2024-09-16T07:23:37.345610+0000","last_became_peered":"2024-09-16T07:23:37.345610+0000","last_unstale":"2024-09-16T07:23:37.348747+0000","last_undegraded":"2024-09-16T07:23:37.348747+0000","last_fullsized":"2024-09-16T07:23:37.348747+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:45.870123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362706+0000","last_change":"2024-09-16T07:23:38.358556+0000","last_active":"2024-09-16T07:23:38.362706+0000","last_peered":"2024-09-16T07:23:38.362706+0000","last_clean":"2024-09-16T07:23:38.362706+0000","last_became_active":"2024-09-16T07:23:38.358288+0000","last_became_peered":"2024-09-16T07:23:38.358288+0000","last_unstale":"2024-09-16T07:23:38.362706+0000","last_undegraded":"2024-09-16T07:23:38.362706+0000","last_fullsized":"2024-09-16T07:23:38.362706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:34.849304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346559+0000","last_change":"2024-09-16T07:23:37.343946+0000","last_active":"2024-09-16T07:23:37.346559+0000","last_peered":"2024-09-16T07:23:37.346559+0000","last_clean":"2024-09-16T07:23:37.346559+0000","last_became_active":"2024-09-16T07:23:37.343628+0000","last_became_peered":"2024-09-16T07:23:37.343628+0000","last_unstale":"2024-09-16T07:23:37.346559+0000","last_undegraded":"2024-09-16T07:23:37.346559+0000","last_fullsized":"2024-09-16T07:23:37.346559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:00.089436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615243+0000","last_change":"2024-09-16T07:23:38.346303+0000","last_active":"2024-09-16T07:23:40.615243+0000","last_peered":"2024-09-16T07:23:40.615243+0000","last_clean":"2024-09-16T07:23:40.615243+0000","last_became_active":"2024-09-16T07:23:38.346039+0000","last_became_peered":"2024-09-16T07:23:38.346039+0000","last_unstale":"2024-09-16T07:23:40.615243+0000","last_undegraded":"2024-09-16T07:23:40.615243+0000","last_fullsized":"2024-09-16T07:23:40.615243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:47.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.342710+0000","last_change":"2024-09-16T07:23:37.340258+0000","last_active":"2024-09-16T07:23:37.342710+0000","last_peered":"2024-09-16T07:23:37.342710+0000","last_clean":"2024-09-16T07:23:37.342710+0000","last_became_active":"2024-09-16T07:23:37.339993+0000","last_became_peered":"2024-09-16T07:23:37.339993+0000","last_unstale":"2024-09-16T07:23:37.342710+0000","last_undegraded":"2024-09-16T07:23:37.342710+0000","last_fullsized":"2024-09-16T07:23:37.342710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:29.784689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614910+0000","last_change":"2024-09-16T07:23:38.342446+0000","last_active":"2024-09-16T07:23:40.614910+0000","last_peered":"2024-09-16T07:23:40.614910+0000","last_clean":"2024-09-16T07:23:40.614910+0000","last_became_active":"2024-09-16T07:23:38.341488+0000","last_became_peered":"2024-09-16T07:23:38.341488+0000","last_unstale":"2024-09-16T07:23:40.614910+0000","last_undegraded":"2024-09-16T07:23:40.614910+0000","last_fullsized":"2024-09-16T07:23:40.614910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:51.750252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.336765+0000","last_change":"2024-09-16T07:23:37.348185+0000","last_active":"2024-09-16T07:23:38.336765+0000","last_peered":"2024-09-16T07:23:38.336765+0000","last_clean":"2024-09-16T07:23:38.336765+0000","last_became_active":"2024-09-16T07:23:37.347959+0000","last_became_peered":"2024-09-16T07:23:37.347959+0000","last_unstale":"2024-09-16T07:23:38.336765+0000","last_undegraded":"2024-09-16T07:23:38.336765+0000","last_fullsized":"2024-09-16T07:23:38.336765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:21:54.247700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362963+0000","last_change":"2024-09-16T07:23:38.355905+0000","last_active":"2024-09-16T07:23:38.362963+0000","last_peered":"2024-09-16T07:23:38.362963+0000","last_clean":"2024-09-16T07:23:38.362963+0000","last_became_active":"2024-09-16T07:23:38.355546+0000","last_became_peered":"2024-09-16T07:23:38.355546+0000","last_unstale":"2024-09-16T07:23:38.362963+0000","last_undegraded":"2024-09-16T07:23:38.362963+0000","last_fullsized":"2024-09-16T07:23:38.362963+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:42.332816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614759+0000","last_change":"2024-09-16T07:23:37.345122+0000","last_active":"2024-09-16T07:23:40.614759+0000","last_peered":"2024-09-16T07:23:40.614759+0000","last_clean":"2024-09-16T07:23:40.614759+0000","last_became_active":"2024-09-16T07:23:37.344793+0000","last_became_peered":"2024-09-16T07:23:37.344793+0000","last_unstale":"2024-09-16T07:23:40.614759+0000","last_undegraded":"2024-09-16T07:23:40.614759+0000","last_fullsized":"2024-09-16T07:23:40.614759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:56.809362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362575+0000","last_change":"2024-09-16T07:23:38.358386+0000","last_active":"2024-09-16T07:23:38.362575+0000","last_peered":"2024-09-16T07:23:38.362575+0000","last_clean":"2024-09-16T07:23:38.362575+0000","last_became_active":"2024-09-16T07:23:38.358092+0000","last_became_peered":"2024-09-16T07:23:38.358092+0000","last_unstale":"2024-09-16T07:23:38.362575+0000","last_undegraded":"2024-09-16T07:23:38.362575+0000","last_fullsized":"2024-09-16T07:23:38.362575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:58:30.618313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.338916+0000","last_change":"2024-09-16T07:23:37.345296+0000","last_active":"2024-09-16T07:23:38.338916+0000","last_peered":"2024-09-16T07:23:38.338916+0000","last_clean":"2024-09-16T07:23:38.338916+0000","last_became_active":"2024-09-16T07:23:37.343905+0000","last_became_peered":"2024-09-16T07:23:37.343905+0000","last_unstale":"2024-09-16T07:23:38.338916+0000","last_undegraded":"2024-09-16T07:23:38.338916+0000","last_fullsized":"2024-09-16T07:23:38.338916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:43.228036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":130,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.338930+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:23:38.338930+0000","last_peered":"2024-09-16T07:23:38.338930+0000","last_clean":"2024-09-16T07:23:38.338930+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:23:38.338930+0000","last_undegraded":"2024-09-16T07:23:38.338930+0000","last_fullsized":"2024-09-16T07:23:38.338930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476753,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476753,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":727307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":130664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":721032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":151648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:23:42.556 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:23:42.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.749+0000 7fd69f000640 1 -- 172.21.15.71:0/2808091259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6980ccf00 msgr2=0x7fd6980b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.749+0000 7fd69f000640 1 --2- 172.21.15.71:0/2808091259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6980ccf00 0x7fd6980b2440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd680009860 tx=0x7fd68002a4a0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.749+0000 7fd69f000640 1 -- 172.21.15.71:0/2808091259 shutdown_connections 2024-09-16T07:23:42.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.749+0000 7fd69f000640 1 --2- 172.21.15.71:0/2808091259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6980ccf00 0x7fd6980b2440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.749+0000 7fd69f000640 1 -- 172.21.15.71:0/2808091259 >> 172.21.15.71:0/2808091259 conn(0x7fd6980b1950 msgr2=0x7fd6980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:42.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.749+0000 7fd69f000640 1 -- 172.21.15.71:0/2808091259 shutdown_connections 2024-09-16T07:23:42.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.749+0000 7fd69f000640 1 -- 172.21.15.71:0/2808091259 wait complete. 2024-09-16T07:23:42.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.751+0000 7fd69f000640 1 Processor -- start 2024-09-16T07:23:42.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.751+0000 7fd69f000640 1 -- start start 2024-09-16T07:23:42.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd69f000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd698073fb0 0x7fd698152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:42.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd69f000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6980ccf00 0x7fd69814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:42.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd69f000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd69815f5a0 0x7fd69815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:42.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd69f000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd69806ef30 con 0x7fd698073fb0 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd69f000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd69806fa80 con 0x7fd6980ccf00 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd69f000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd698074440 con 0x7fd69815f5a0 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd697e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd698073fb0 0x7fd698152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd697e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd698073fb0 0x7fd698152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44934/0 (socket says 172.21.15.71:44934) 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd697e00640 1 -- 172.21.15.71:0/2710310660 learned_addr learned my addr 172.21.15.71:0/2710310660 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd68fe00640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6980ccf00 0x7fd69814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.752+0000 7fd69d000640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd69815f5a0 0x7fd69815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd697e00640 1 -- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd69815f5a0 msgr2=0x7fd69815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd697e00640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd69815f5a0 0x7fd69815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd697e00640 1 -- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6980ccf00 msgr2=0x7fd69814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd697e00640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6980ccf00 0x7fd69814c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd697e00640 1 -- 172.21.15.71:0/2710310660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6980cdf50 con 0x7fd698073fb0 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd697e00640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd698073fb0 0x7fd698152d10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c390 tx=0x7fd68800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:42.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd694c00640 1 -- 172.21.15.71:0/2710310660 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688017070 con 0x7fd698073fb0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd694c00640 1 -- 172.21.15.71:0/2710310660 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6880053c0 con 0x7fd698073fb0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6980ce230 con 0x7fd698073fb0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd694c00640 1 -- 172.21.15.71:0/2710310660 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688003850 con 0x7fd698073fb0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.753+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6980ce7f0 con 0x7fd698073fb0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.754+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6980b2440 con 0x7fd698073fb0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.755+0000 7fd694c00640 1 -- 172.21.15.71:0/2710310660 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd68801a070 con 0x7fd698073fb0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.756+0000 7fd694c00640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd674078850 0x7fd67407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.756+0000 7fd694c00640 1 -- 172.21.15.71:0/2710310660 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd688099d60 con 0x7fd698073fb0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.757+0000 7fd68fe00640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd674078850 0x7fd67407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:42.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.757+0000 7fd68fe00640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd674078850 0x7fd67407ad10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd680034000 tx=0x7fd680004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:42.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.760+0000 7fd694c00640 1 -- 172.21.15.71:0/2710310660 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd6880623c0 con 0x7fd698073fb0 2024-09-16T07:23:42.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.940+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd69809ba50 con 0x7fd674078850 2024-09-16T07:23:42.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.945+0000 7fd694c00640 1 -- 172.21.15.71:0/2710310660 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250440 (secure 0 0 0) 0x7fd6800041c0 con 0x7fd674078850 2024-09-16T07:23:42.947 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:42.949 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd674078850 msgr2=0x7fd67407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd674078850 0x7fd67407ad10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd680034000 tx=0x7fd680004730 comp rx=0 tx=0).stop 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd698073fb0 msgr2=0x7fd698152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd698073fb0 0x7fd698152d10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c390 tx=0x7fd68800c850 comp rx=0 tx=0).stop 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 shutdown_connections 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd674078850 0x7fd67407ad10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd69815f5a0 0x7fd69815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6980ccf00 0x7fd69814c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 --2- 172.21.15.71:0/2710310660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd698073fb0 0x7fd698152d10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:42.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 >> 172.21.15.71:0/2710310660 conn(0x7fd6980b1950 msgr2=0x7fd6980aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:42.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.949+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 shutdown_connections 2024-09-16T07:23:42.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:42.950+0000 7fd69f000640 1 -- 172.21.15.71:0/2710310660 wait complete. 2024-09-16T07:23:42.965 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-16T07:23:42.305320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":119,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224400,"kb_used_data":3992,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750320,"statfs":{"total":773094113280,"available":772864327680,"internally_reserved":0,"allocated":4087808,"data_stored":2278407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.979386"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350368+0000","last_change":"2024-09-16T07:23:38.347812+0000","last_active":"2024-09-16T07:23:38.350368+0000","last_peered":"2024-09-16T07:23:38.350368+0000","last_clean":"2024-09-16T07:23:38.350368+0000","last_became_active":"2024-09-16T07:23:38.347516+0000","last_became_peered":"2024-09-16T07:23:38.347516+0000","last_unstale":"2024-09-16T07:23:38.350368+0000","last_undegraded":"2024-09-16T07:23:38.350368+0000","last_fullsized":"2024-09-16T07:23:38.350368+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:06.049071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334422+0000","last_change":"2024-09-16T07:23:37.334422+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334422+0000","last_undegraded":"2024-09-16T07:23:37.334422+0000","last_fullsized":"2024-09-16T07:23:37.334422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.360416+0000","last_change":"2024-09-16T07:23:38.352895+0000","last_active":"2024-09-16T07:23:38.360416+0000","last_peered":"2024-09-16T07:23:38.360416+0000","last_clean":"2024-09-16T07:23:38.360416+0000","last_became_active":"2024-09-16T07:23:38.352621+0000","last_became_peered":"2024-09-16T07:23:38.352621+0000","last_unstale":"2024-09-16T07:23:38.360416+0000","last_undegraded":"2024-09-16T07:23:38.360416+0000","last_fullsized":"2024-09-16T07:23:38.360416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:14.142804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615292+0000","last_change":"2024-09-16T07:23:38.350692+0000","last_active":"2024-09-16T07:23:40.615292+0000","last_peered":"2024-09-16T07:23:40.615292+0000","last_clean":"2024-09-16T07:23:40.615292+0000","last_became_active":"2024-09-16T07:23:38.347475+0000","last_became_peered":"2024-09-16T07:23:38.347475+0000","last_unstale":"2024-09-16T07:23:40.615292+0000","last_undegraded":"2024-09-16T07:23:40.615292+0000","last_fullsized":"2024-09-16T07:23:40.615292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:01.740131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.331162+0000","last_change":"2024-09-16T07:23:37.331162+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.331162+0000","last_undegraded":"2024-09-16T07:23:37.331162+0000","last_fullsized":"2024-09-16T07:23:37.331162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334158+0000","last_change":"2024-09-16T07:23:37.334158+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334158+0000","last_undegraded":"2024-09-16T07:23:37.334158+0000","last_fullsized":"2024-09-16T07:23:37.334158+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.368246+0000","last_change":"2024-09-16T07:23:38.367300+0000","last_active":"2024-09-16T07:23:38.368246+0000","last_peered":"2024-09-16T07:23:38.368246+0000","last_clean":"2024-09-16T07:23:38.368246+0000","last_became_active":"2024-09-16T07:23:38.365509+0000","last_became_peered":"2024-09-16T07:23:38.365509+0000","last_unstale":"2024-09-16T07:23:38.368246+0000","last_undegraded":"2024-09-16T07:23:38.368246+0000","last_fullsized":"2024-09-16T07:23:38.368246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:19.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.358930+0000","last_change":"2024-09-16T07:23:38.357105+0000","last_active":"2024-09-16T07:23:38.358930+0000","last_peered":"2024-09-16T07:23:38.358930+0000","last_clean":"2024-09-16T07:23:38.358930+0000","last_became_active":"2024-09-16T07:23:38.355543+0000","last_became_peered":"2024-09-16T07:23:38.355543+0000","last_unstale":"2024-09-16T07:23:38.358930+0000","last_undegraded":"2024-09-16T07:23:38.358930+0000","last_fullsized":"2024-09-16T07:23:38.358930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:21.370546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359316+0000","last_change":"2024-09-16T07:23:38.356452+0000","last_active":"2024-09-16T07:23:38.359316+0000","last_peered":"2024-09-16T07:23:38.359316+0000","last_clean":"2024-09-16T07:23:38.359316+0000","last_became_active":"2024-09-16T07:23:38.355663+0000","last_became_peered":"2024-09-16T07:23:38.355663+0000","last_unstale":"2024-09-16T07:23:38.359316+0000","last_undegraded":"2024-09-16T07:23:38.359316+0000","last_fullsized":"2024-09-16T07:23:38.359316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.788335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615484+0000","last_change":"2024-09-16T07:23:38.356446+0000","last_active":"2024-09-16T07:23:40.615484+0000","last_peered":"2024-09-16T07:23:40.615484+0000","last_clean":"2024-09-16T07:23:40.615484+0000","last_became_active":"2024-09-16T07:23:38.354502+0000","last_became_peered":"2024-09-16T07:23:38.354502+0000","last_unstale":"2024-09-16T07:23:40.615484+0000","last_undegraded":"2024-09-16T07:23:40.615484+0000","last_fullsized":"2024-09-16T07:23:40.615484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:27.342707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352817+0000","last_change":"2024-09-16T07:23:38.351223+0000","last_active":"2024-09-16T07:23:38.352817+0000","last_peered":"2024-09-16T07:23:38.352817+0000","last_clean":"2024-09-16T07:23:38.352817+0000","last_became_active":"2024-09-16T07:23:38.349810+0000","last_became_peered":"2024-09-16T07:23:38.349810+0000","last_unstale":"2024-09-16T07:23:38.352817+0000","last_undegraded":"2024-09-16T07:23:38.352817+0000","last_fullsized":"2024-09-16T07:23:38.352817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:37.812192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.367826+0000","last_change":"2024-09-16T07:23:38.365322+0000","last_active":"2024-09-16T07:23:38.367826+0000","last_peered":"2024-09-16T07:23:38.367826+0000","last_clean":"2024-09-16T07:23:38.367826+0000","last_became_active":"2024-09-16T07:23:38.364995+0000","last_became_peered":"2024-09-16T07:23:38.364995+0000","last_unstale":"2024-09-16T07:23:38.367826+0000","last_undegraded":"2024-09-16T07:23:38.367826+0000","last_fullsized":"2024-09-16T07:23:38.367826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:40.799346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.335382+0000","last_change":"2024-09-16T07:23:37.335382+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.335382+0000","last_undegraded":"2024-09-16T07:23:37.335382+0000","last_fullsized":"2024-09-16T07:23:37.335382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614877+0000","last_change":"2024-09-16T07:23:38.347109+0000","last_active":"2024-09-16T07:23:40.614877+0000","last_peered":"2024-09-16T07:23:40.614877+0000","last_clean":"2024-09-16T07:23:40.614877+0000","last_became_active":"2024-09-16T07:23:38.346821+0000","last_became_peered":"2024-09-16T07:23:38.346821+0000","last_unstale":"2024-09-16T07:23:40.614877+0000","last_undegraded":"2024-09-16T07:23:40.614877+0000","last_fullsized":"2024-09-16T07:23:40.614877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:59.555576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.345227+0000","last_change":"2024-09-16T07:23:38.341149+0000","last_active":"2024-09-16T07:23:38.345227+0000","last_peered":"2024-09-16T07:23:38.345227+0000","last_clean":"2024-09-16T07:23:38.345227+0000","last_became_active":"2024-09-16T07:23:38.340794+0000","last_became_peered":"2024-09-16T07:23:38.340794+0000","last_unstale":"2024-09-16T07:23:38.345227+0000","last_undegraded":"2024-09-16T07:23:38.345227+0000","last_fullsized":"2024-09-16T07:23:38.345227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:17.507704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362540+0000","last_change":"2024-09-16T07:23:38.360203+0000","last_active":"2024-09-16T07:23:38.362540+0000","last_peered":"2024-09-16T07:23:38.362540+0000","last_clean":"2024-09-16T07:23:38.362540+0000","last_became_active":"2024-09-16T07:23:38.359933+0000","last_became_peered":"2024-09-16T07:23:38.359933+0000","last_unstale":"2024-09-16T07:23:38.362540+0000","last_undegraded":"2024-09-16T07:23:38.362540+0000","last_fullsized":"2024-09-16T07:23:38.362540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:55.869414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352904+0000","last_change":"2024-09-16T07:23:38.350866+0000","last_active":"2024-09-16T07:23:38.352904+0000","last_peered":"2024-09-16T07:23:38.352904+0000","last_clean":"2024-09-16T07:23:38.352904+0000","last_became_active":"2024-09-16T07:23:38.350617+0000","last_became_peered":"2024-09-16T07:23:38.350617+0000","last_unstale":"2024-09-16T07:23:38.352904+0000","last_undegraded":"2024-09-16T07:23:38.352904+0000","last_fullsized":"2024-09-16T07:23:38.352904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:14:27.853553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.338226+0000","last_change":"2024-09-16T07:23:37.338226+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.338226+0000","last_undegraded":"2024-09-16T07:23:37.338226+0000","last_fullsized":"2024-09-16T07:23:37.338226+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334511+0000","last_change":"2024-09-16T07:23:37.334511+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334511+0000","last_undegraded":"2024-09-16T07:23:37.334511+0000","last_fullsized":"2024-09-16T07:23:37.334511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.337607+0000","last_change":"2024-09-16T07:23:37.337607+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.337607+0000","last_undegraded":"2024-09-16T07:23:37.337607+0000","last_fullsized":"2024-09-16T07:23:37.337607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.357360+0000","last_change":"2024-09-16T07:23:38.355303+0000","last_active":"2024-09-16T07:23:38.357360+0000","last_peered":"2024-09-16T07:23:38.357360+0000","last_clean":"2024-09-16T07:23:38.357360+0000","last_became_active":"2024-09-16T07:23:38.355033+0000","last_became_peered":"2024-09-16T07:23:38.355033+0000","last_unstale":"2024-09-16T07:23:38.357360+0000","last_undegraded":"2024-09-16T07:23:38.357360+0000","last_fullsized":"2024-09-16T07:23:38.357360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:19.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350313+0000","last_change":"2024-09-16T07:23:38.345255+0000","last_active":"2024-09-16T07:23:38.350313+0000","last_peered":"2024-09-16T07:23:38.350313+0000","last_clean":"2024-09-16T07:23:38.350313+0000","last_became_active":"2024-09-16T07:23:38.344965+0000","last_became_peered":"2024-09-16T07:23:38.344965+0000","last_unstale":"2024-09-16T07:23:38.350313+0000","last_undegraded":"2024-09-16T07:23:38.350313+0000","last_fullsized":"2024-09-16T07:23:38.350313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:09.790205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334034+0000","last_change":"2024-09-16T07:23:37.334034+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334034+0000","last_undegraded":"2024-09-16T07:23:37.334034+0000","last_fullsized":"2024-09-16T07:23:37.334034+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352659+0000","last_change":"2024-09-16T07:23:38.348792+0000","last_active":"2024-09-16T07:23:38.352659+0000","last_peered":"2024-09-16T07:23:38.352659+0000","last_clean":"2024-09-16T07:23:38.352659+0000","last_became_active":"2024-09-16T07:23:38.345353+0000","last_became_peered":"2024-09-16T07:23:38.345353+0000","last_unstale":"2024-09-16T07:23:38.352659+0000","last_undegraded":"2024-09-16T07:23:38.352659+0000","last_fullsized":"2024-09-16T07:23:38.352659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:32.489176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615553+0000","last_change":"2024-09-16T07:23:38.351284+0000","last_active":"2024-09-16T07:23:40.615553+0000","last_peered":"2024-09-16T07:23:40.615553+0000","last_clean":"2024-09-16T07:23:40.615553+0000","last_became_active":"2024-09-16T07:23:38.350972+0000","last_became_peered":"2024-09-16T07:23:38.350972+0000","last_unstale":"2024-09-16T07:23:40.615553+0000","last_undegraded":"2024-09-16T07:23:40.615553+0000","last_fullsized":"2024-09-16T07:23:40.615553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:25.493457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615534+0000","last_change":"2024-09-16T07:23:38.356598+0000","last_active":"2024-09-16T07:23:40.615534+0000","last_peered":"2024-09-16T07:23:40.615534+0000","last_clean":"2024-09-16T07:23:40.615534+0000","last_became_active":"2024-09-16T07:23:38.356177+0000","last_became_peered":"2024-09-16T07:23:38.356177+0000","last_unstale":"2024-09-16T07:23:40.615534+0000","last_undegraded":"2024-09-16T07:23:40.615534+0000","last_fullsized":"2024-09-16T07:23:40.615534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:42.647875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.330534+0000","last_change":"2024-09-16T07:23:37.330534+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.330534+0000","last_undegraded":"2024-09-16T07:23:37.330534+0000","last_fullsized":"2024-09-16T07:23:37.330534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.364143+0000","last_change":"2024-09-16T07:23:38.362005+0000","last_active":"2024-09-16T07:23:38.364143+0000","last_peered":"2024-09-16T07:23:38.364143+0000","last_clean":"2024-09-16T07:23:38.364143+0000","last_became_active":"2024-09-16T07:23:38.361748+0000","last_became_peered":"2024-09-16T07:23:38.361748+0000","last_unstale":"2024-09-16T07:23:38.364143+0000","last_undegraded":"2024-09-16T07:23:38.364143+0000","last_fullsized":"2024-09-16T07:23:38.364143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:17.507704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.343486+0000","last_change":"2024-09-16T07:23:38.337747+0000","last_active":"2024-09-16T07:23:38.343486+0000","last_peered":"2024-09-16T07:23:38.343486+0000","last_clean":"2024-09-16T07:23:38.343486+0000","last_became_active":"2024-09-16T07:23:38.337455+0000","last_became_peered":"2024-09-16T07:23:38.337455+0000","last_unstale":"2024-09-16T07:23:38.343486+0000","last_undegraded":"2024-09-16T07:23:38.343486+0000","last_fullsized":"2024-09-16T07:23:38.343486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:21.370546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.351724+0000","last_change":"2024-09-16T07:23:38.348476+0000","last_active":"2024-09-16T07:23:38.351724+0000","last_peered":"2024-09-16T07:23:38.351724+0000","last_clean":"2024-09-16T07:23:38.351724+0000","last_became_active":"2024-09-16T07:23:38.348263+0000","last_became_peered":"2024-09-16T07:23:38.348263+0000","last_unstale":"2024-09-16T07:23:38.351724+0000","last_undegraded":"2024-09-16T07:23:38.351724+0000","last_fullsized":"2024-09-16T07:23:38.351724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:24.558417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.358843+0000","last_change":"2024-09-16T07:23:38.356216+0000","last_active":"2024-09-16T07:23:38.358843+0000","last_peered":"2024-09-16T07:23:38.358843+0000","last_clean":"2024-09-16T07:23:38.358843+0000","last_became_active":"2024-09-16T07:23:38.355971+0000","last_became_peered":"2024-09-16T07:23:38.355971+0000","last_unstale":"2024-09-16T07:23:38.358843+0000","last_undegraded":"2024-09-16T07:23:38.358843+0000","last_fullsized":"2024-09-16T07:23:38.358843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:25.955048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.346389+0000","last_change":"2024-09-16T07:23:38.342538+0000","last_active":"2024-09-16T07:23:38.346389+0000","last_peered":"2024-09-16T07:23:38.346389+0000","last_clean":"2024-09-16T07:23:38.346389+0000","last_became_active":"2024-09-16T07:23:38.342085+0000","last_became_peered":"2024-09-16T07:23:38.342085+0000","last_unstale":"2024-09-16T07:23:38.346389+0000","last_undegraded":"2024-09-16T07:23:38.346389+0000","last_fullsized":"2024-09-16T07:23:38.346389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:56.231147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333820+0000","last_change":"2024-09-16T07:23:37.333820+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333820+0000","last_undegraded":"2024-09-16T07:23:37.333820+0000","last_fullsized":"2024-09-16T07:23:37.333820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.375763+0000","last_change":"2024-09-16T07:23:37.346853+0000","last_active":"2024-09-16T07:23:38.375763+0000","last_peered":"2024-09-16T07:23:38.375763+0000","last_clean":"2024-09-16T07:23:38.375763+0000","last_became_active":"2024-09-16T07:23:37.344215+0000","last_became_peered":"2024-09-16T07:23:37.344215+0000","last_unstale":"2024-09-16T07:23:38.375763+0000","last_undegraded":"2024-09-16T07:23:38.375763+0000","last_fullsized":"2024-09-16T07:23:38.375763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:55.470787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350348+0000","last_change":"2024-09-16T07:23:38.347363+0000","last_active":"2024-09-16T07:23:38.350348+0000","last_peered":"2024-09-16T07:23:38.350348+0000","last_clean":"2024-09-16T07:23:38.350348+0000","last_became_active":"2024-09-16T07:23:38.347091+0000","last_became_peered":"2024-09-16T07:23:38.347091+0000","last_unstale":"2024-09-16T07:23:38.350348+0000","last_undegraded":"2024-09-16T07:23:38.350348+0000","last_fullsized":"2024-09-16T07:23:38.350348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:16.751332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615579+0000","last_change":"2024-09-16T07:23:37.349796+0000","last_active":"2024-09-16T07:23:40.615579+0000","last_peered":"2024-09-16T07:23:40.615579+0000","last_clean":"2024-09-16T07:23:40.615579+0000","last_became_active":"2024-09-16T07:23:37.349505+0000","last_became_peered":"2024-09-16T07:23:37.349505+0000","last_unstale":"2024-09-16T07:23:40.615579+0000","last_undegraded":"2024-09-16T07:23:40.615579+0000","last_fullsized":"2024-09-16T07:23:40.615579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:04.156099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.349933+0000","last_change":"2024-09-16T07:23:38.344847+0000","last_active":"2024-09-16T07:23:38.349933+0000","last_peered":"2024-09-16T07:23:38.349933+0000","last_clean":"2024-09-16T07:23:38.349933+0000","last_became_active":"2024-09-16T07:23:38.344502+0000","last_became_peered":"2024-09-16T07:23:38.344502+0000","last_unstale":"2024-09-16T07:23:38.349933+0000","last_undegraded":"2024-09-16T07:23:38.349933+0000","last_fullsized":"2024-09-16T07:23:38.349933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:26.398231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350064+0000","last_change":"2024-09-16T07:23:37.347456+0000","last_active":"2024-09-16T07:23:37.350064+0000","last_peered":"2024-09-16T07:23:37.350064+0000","last_clean":"2024-09-16T07:23:37.350064+0000","last_became_active":"2024-09-16T07:23:37.347152+0000","last_became_peered":"2024-09-16T07:23:37.347152+0000","last_unstale":"2024-09-16T07:23:37.350064+0000","last_undegraded":"2024-09-16T07:23:37.350064+0000","last_fullsized":"2024-09-16T07:23:37.350064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:51:05.553785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615486+0000","last_change":"2024-09-16T07:23:38.353195+0000","last_active":"2024-09-16T07:23:40.615486+0000","last_peered":"2024-09-16T07:23:40.615486+0000","last_clean":"2024-09-16T07:23:40.615486+0000","last_became_active":"2024-09-16T07:23:38.350558+0000","last_became_peered":"2024-09-16T07:23:38.350558+0000","last_unstale":"2024-09-16T07:23:40.615486+0000","last_undegraded":"2024-09-16T07:23:40.615486+0000","last_fullsized":"2024-09-16T07:23:40.615486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:23.647474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615384+0000","last_change":"2024-09-16T07:23:37.348375+0000","last_active":"2024-09-16T07:23:40.615384+0000","last_peered":"2024-09-16T07:23:40.615384+0000","last_clean":"2024-09-16T07:23:40.615384+0000","last_became_active":"2024-09-16T07:23:37.348096+0000","last_became_peered":"2024-09-16T07:23:37.348096+0000","last_unstale":"2024-09-16T07:23:40.615384+0000","last_undegraded":"2024-09-16T07:23:40.615384+0000","last_fullsized":"2024-09-16T07:23:40.615384+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:37.348737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359143+0000","last_change":"2024-09-16T07:23:38.351112+0000","last_active":"2024-09-16T07:23:38.359143+0000","last_peered":"2024-09-16T07:23:38.359143+0000","last_clean":"2024-09-16T07:23:38.359143+0000","last_became_active":"2024-09-16T07:23:38.350832+0000","last_became_peered":"2024-09-16T07:23:38.350832+0000","last_unstale":"2024-09-16T07:23:38.359143+0000","last_undegraded":"2024-09-16T07:23:38.359143+0000","last_fullsized":"2024-09-16T07:23:38.359143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:44.019182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359294+0000","last_change":"2024-09-16T07:23:37.347090+0000","last_active":"2024-09-16T07:23:38.359294+0000","last_peered":"2024-09-16T07:23:38.359294+0000","last_clean":"2024-09-16T07:23:38.359294+0000","last_became_active":"2024-09-16T07:23:37.346832+0000","last_became_peered":"2024-09-16T07:23:37.346832+0000","last_unstale":"2024-09-16T07:23:38.359294+0000","last_undegraded":"2024-09-16T07:23:38.359294+0000","last_fullsized":"2024-09-16T07:23:38.359294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:41.066939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.364807+0000","last_change":"2024-09-16T07:23:38.363179+0000","last_active":"2024-09-16T07:23:38.364807+0000","last_peered":"2024-09-16T07:23:38.364807+0000","last_clean":"2024-09-16T07:23:38.364807+0000","last_became_active":"2024-09-16T07:23:38.360647+0000","last_became_peered":"2024-09-16T07:23:38.360647+0000","last_unstale":"2024-09-16T07:23:38.364807+0000","last_undegraded":"2024-09-16T07:23:38.364807+0000","last_fullsized":"2024-09-16T07:23:38.364807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:24.558417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.363776+0000","last_change":"2024-09-16T07:23:37.344503+0000","last_active":"2024-09-16T07:23:38.363776+0000","last_peered":"2024-09-16T07:23:38.363776+0000","last_clean":"2024-09-16T07:23:38.363776+0000","last_became_active":"2024-09-16T07:23:37.344139+0000","last_became_peered":"2024-09-16T07:23:37.344139+0000","last_unstale":"2024-09-16T07:23:38.363776+0000","last_undegraded":"2024-09-16T07:23:38.363776+0000","last_fullsized":"2024-09-16T07:23:38.363776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:59.430414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352723+0000","last_change":"2024-09-16T07:23:38.348789+0000","last_active":"2024-09-16T07:23:38.352723+0000","last_peered":"2024-09-16T07:23:38.352723+0000","last_clean":"2024-09-16T07:23:38.352723+0000","last_became_active":"2024-09-16T07:23:38.348529+0000","last_became_peered":"2024-09-16T07:23:38.348529+0000","last_unstale":"2024-09-16T07:23:38.352723+0000","last_undegraded":"2024-09-16T07:23:38.352723+0000","last_fullsized":"2024-09-16T07:23:38.352723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:16.637145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350090+0000","last_change":"2024-09-16T07:23:37.348311+0000","last_active":"2024-09-16T07:23:37.350090+0000","last_peered":"2024-09-16T07:23:37.350090+0000","last_clean":"2024-09-16T07:23:37.350090+0000","last_became_active":"2024-09-16T07:23:37.348006+0000","last_became_peered":"2024-09-16T07:23:37.348006+0000","last_unstale":"2024-09-16T07:23:37.350090+0000","last_undegraded":"2024-09-16T07:23:37.350090+0000","last_fullsized":"2024-09-16T07:23:37.350090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:29.668774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614873+0000","last_change":"2024-09-16T07:23:38.352705+0000","last_active":"2024-09-16T07:23:40.614873+0000","last_peered":"2024-09-16T07:23:40.614873+0000","last_clean":"2024-09-16T07:23:40.614873+0000","last_became_active":"2024-09-16T07:23:38.352415+0000","last_became_peered":"2024-09-16T07:23:38.352415+0000","last_unstale":"2024-09-16T07:23:40.614873+0000","last_undegraded":"2024-09-16T07:23:40.614873+0000","last_fullsized":"2024-09-16T07:23:40.614873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:10.916617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.335903+0000","last_change":"2024-09-16T07:23:37.353028+0000","last_active":"2024-09-16T07:23:38.335903+0000","last_peered":"2024-09-16T07:23:38.335903+0000","last_clean":"2024-09-16T07:23:38.335903+0000","last_became_active":"2024-09-16T07:23:37.352718+0000","last_became_peered":"2024-09-16T07:23:37.352718+0000","last_unstale":"2024-09-16T07:23:38.335903+0000","last_undegraded":"2024-09-16T07:23:38.335903+0000","last_fullsized":"2024-09-16T07:23:38.335903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:41.965086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.360894+0000","last_change":"2024-09-16T07:23:38.353299+0000","last_active":"2024-09-16T07:23:38.360894+0000","last_peered":"2024-09-16T07:23:38.360894+0000","last_clean":"2024-09-16T07:23:38.360894+0000","last_became_active":"2024-09-16T07:23:38.352818+0000","last_became_peered":"2024-09-16T07:23:38.352818+0000","last_unstale":"2024-09-16T07:23:38.360894+0000","last_undegraded":"2024-09-16T07:23:38.360894+0000","last_fullsized":"2024-09-16T07:23:38.360894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:02.666636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.349313+0000","last_change":"2024-09-16T07:23:37.346845+0000","last_active":"2024-09-16T07:23:37.349313+0000","last_peered":"2024-09-16T07:23:37.349313+0000","last_clean":"2024-09-16T07:23:37.349313+0000","last_became_active":"2024-09-16T07:23:37.346574+0000","last_became_peered":"2024-09-16T07:23:37.346574+0000","last_unstale":"2024-09-16T07:23:37.349313+0000","last_undegraded":"2024-09-16T07:23:37.349313+0000","last_fullsized":"2024-09-16T07:23:37.349313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:30.460837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615689+0000","last_change":"2024-09-16T07:23:38.358332+0000","last_active":"2024-09-16T07:23:40.615689+0000","last_peered":"2024-09-16T07:23:40.615689+0000","last_clean":"2024-09-16T07:23:40.615689+0000","last_became_active":"2024-09-16T07:23:38.355145+0000","last_became_peered":"2024-09-16T07:23:38.355145+0000","last_unstale":"2024-09-16T07:23:40.615689+0000","last_undegraded":"2024-09-16T07:23:40.615689+0000","last_fullsized":"2024-09-16T07:23:40.615689+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.765410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346902+0000","last_change":"2024-09-16T07:23:37.345039+0000","last_active":"2024-09-16T07:23:37.346902+0000","last_peered":"2024-09-16T07:23:37.346902+0000","last_clean":"2024-09-16T07:23:37.346902+0000","last_became_active":"2024-09-16T07:23:37.344529+0000","last_became_peered":"2024-09-16T07:23:37.344529+0000","last_unstale":"2024-09-16T07:23:37.346902+0000","last_undegraded":"2024-09-16T07:23:37.346902+0000","last_fullsized":"2024-09-16T07:23:37.346902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:18.045656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615409+0000","last_change":"2024-09-16T07:23:38.344916+0000","last_active":"2024-09-16T07:23:40.615409+0000","last_peered":"2024-09-16T07:23:40.615409+0000","last_clean":"2024-09-16T07:23:40.615409+0000","last_became_active":"2024-09-16T07:23:38.344552+0000","last_became_peered":"2024-09-16T07:23:38.344552+0000","last_unstale":"2024-09-16T07:23:40.615409+0000","last_undegraded":"2024-09-16T07:23:40.615409+0000","last_fullsized":"2024-09-16T07:23:40.615409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:07:30.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.374362+0000","last_change":"2024-09-16T07:23:37.337958+0000","last_active":"2024-09-16T07:23:38.374362+0000","last_peered":"2024-09-16T07:23:38.374362+0000","last_clean":"2024-09-16T07:23:38.374362+0000","last_became_active":"2024-09-16T07:23:37.337681+0000","last_became_peered":"2024-09-16T07:23:37.337681+0000","last_unstale":"2024-09-16T07:23:38.374362+0000","last_undegraded":"2024-09-16T07:23:38.374362+0000","last_fullsized":"2024-09-16T07:23:38.374362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:31.334181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352729+0000","last_change":"2024-09-16T07:23:38.350404+0000","last_active":"2024-09-16T07:23:38.352729+0000","last_peered":"2024-09-16T07:23:38.352729+0000","last_clean":"2024-09-16T07:23:38.352729+0000","last_became_active":"2024-09-16T07:23:38.350111+0000","last_became_peered":"2024-09-16T07:23:38.350111+0000","last_unstale":"2024-09-16T07:23:38.352729+0000","last_undegraded":"2024-09-16T07:23:38.352729+0000","last_fullsized":"2024-09-16T07:23:38.352729+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:22.548419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.354301+0000","last_change":"2024-09-16T07:23:37.353132+0000","last_active":"2024-09-16T07:23:37.354301+0000","last_peered":"2024-09-16T07:23:37.354301+0000","last_clean":"2024-09-16T07:23:37.354301+0000","last_became_active":"2024-09-16T07:23:37.352306+0000","last_became_peered":"2024-09-16T07:23:37.352306+0000","last_unstale":"2024-09-16T07:23:37.354301+0000","last_undegraded":"2024-09-16T07:23:37.354301+0000","last_fullsized":"2024-09-16T07:23:37.354301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:30.195425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333905+0000","last_change":"2024-09-16T07:23:37.333905+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333905+0000","last_undegraded":"2024-09-16T07:23:37.333905+0000","last_fullsized":"2024-09-16T07:23:37.333905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.365888+0000","last_change":"2024-09-16T07:23:37.347964+0000","last_active":"2024-09-16T07:23:38.365888+0000","last_peered":"2024-09-16T07:23:38.365888+0000","last_clean":"2024-09-16T07:23:38.365888+0000","last_became_active":"2024-09-16T07:23:37.347687+0000","last_became_peered":"2024-09-16T07:23:37.347687+0000","last_unstale":"2024-09-16T07:23:38.365888+0000","last_undegraded":"2024-09-16T07:23:38.365888+0000","last_fullsized":"2024-09-16T07:23:38.365888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:24.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615132+0000","last_change":"2024-09-16T07:23:38.352259+0000","last_active":"2024-09-16T07:23:40.615132+0000","last_peered":"2024-09-16T07:23:40.615132+0000","last_clean":"2024-09-16T07:23:40.615132+0000","last_became_active":"2024-09-16T07:23:38.351935+0000","last_became_peered":"2024-09-16T07:23:38.351935+0000","last_unstale":"2024-09-16T07:23:40.615132+0000","last_undegraded":"2024-09-16T07:23:40.615132+0000","last_fullsized":"2024-09-16T07:23:40.615132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:09:10.158498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614756+0000","last_change":"2024-09-16T07:23:37.350953+0000","last_active":"2024-09-16T07:23:40.614756+0000","last_peered":"2024-09-16T07:23:40.614756+0000","last_clean":"2024-09-16T07:23:40.614756+0000","last_became_active":"2024-09-16T07:23:37.350621+0000","last_became_peered":"2024-09-16T07:23:37.350621+0000","last_unstale":"2024-09-16T07:23:40.614756+0000","last_undegraded":"2024-09-16T07:23:40.614756+0000","last_fullsized":"2024-09-16T07:23:40.614756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:34.314424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352682+0000","last_change":"2024-09-16T07:23:38.348443+0000","last_active":"2024-09-16T07:23:38.352682+0000","last_peered":"2024-09-16T07:23:38.352682+0000","last_clean":"2024-09-16T07:23:38.352682+0000","last_became_active":"2024-09-16T07:23:38.347974+0000","last_became_peered":"2024-09-16T07:23:38.347974+0000","last_unstale":"2024-09-16T07:23:38.352682+0000","last_undegraded":"2024-09-16T07:23:38.352682+0000","last_fullsized":"2024-09-16T07:23:38.352682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:25:27.231251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614598+0000","last_change":"2024-09-16T07:23:37.352837+0000","last_active":"2024-09-16T07:23:40.614598+0000","last_peered":"2024-09-16T07:23:40.614598+0000","last_clean":"2024-09-16T07:23:40.614598+0000","last_became_active":"2024-09-16T07:23:37.352566+0000","last_became_peered":"2024-09-16T07:23:37.352566+0000","last_unstale":"2024-09-16T07:23:40.614598+0000","last_undegraded":"2024-09-16T07:23:40.614598+0000","last_fullsized":"2024-09-16T07:23:40.614598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:20.188668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.347236+0000","last_change":"2024-09-16T07:23:38.344224+0000","last_active":"2024-09-16T07:23:38.347236+0000","last_peered":"2024-09-16T07:23:38.347236+0000","last_clean":"2024-09-16T07:23:38.347236+0000","last_became_active":"2024-09-16T07:23:38.343918+0000","last_became_peered":"2024-09-16T07:23:38.343918+0000","last_unstale":"2024-09-16T07:23:38.347236+0000","last_undegraded":"2024-09-16T07:23:38.347236+0000","last_fullsized":"2024-09-16T07:23:38.347236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:55.009205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350824+0000","last_change":"2024-09-16T07:23:37.349547+0000","last_active":"2024-09-16T07:23:37.350824+0000","last_peered":"2024-09-16T07:23:37.350824+0000","last_clean":"2024-09-16T07:23:37.350824+0000","last_became_active":"2024-09-16T07:23:37.349283+0000","last_became_peered":"2024-09-16T07:23:37.349283+0000","last_unstale":"2024-09-16T07:23:37.350824+0000","last_undegraded":"2024-09-16T07:23:37.350824+0000","last_fullsized":"2024-09-16T07:23:37.350824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:41.491685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.353115+0000","last_change":"2024-09-16T07:23:38.349714+0000","last_active":"2024-09-16T07:23:38.353115+0000","last_peered":"2024-09-16T07:23:38.353115+0000","last_clean":"2024-09-16T07:23:38.353115+0000","last_became_active":"2024-09-16T07:23:38.349431+0000","last_became_peered":"2024-09-16T07:23:38.349431+0000","last_unstale":"2024-09-16T07:23:38.353115+0000","last_undegraded":"2024-09-16T07:23:38.353115+0000","last_fullsized":"2024-09-16T07:23:38.353115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:28.910006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.353383+0000","last_change":"2024-09-16T07:23:37.352287+0000","last_active":"2024-09-16T07:23:37.353383+0000","last_peered":"2024-09-16T07:23:37.353383+0000","last_clean":"2024-09-16T07:23:37.353383+0000","last_became_active":"2024-09-16T07:23:37.351844+0000","last_became_peered":"2024-09-16T07:23:37.351844+0000","last_unstale":"2024-09-16T07:23:37.353383+0000","last_undegraded":"2024-09-16T07:23:37.353383+0000","last_fullsized":"2024-09-16T07:23:37.353383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:24.047148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359294+0000","last_change":"2024-09-16T07:23:38.356161+0000","last_active":"2024-09-16T07:23:38.359294+0000","last_peered":"2024-09-16T07:23:38.359294+0000","last_clean":"2024-09-16T07:23:38.359294+0000","last_became_active":"2024-09-16T07:23:38.354062+0000","last_became_peered":"2024-09-16T07:23:38.354062+0000","last_unstale":"2024-09-16T07:23:38.359294+0000","last_undegraded":"2024-09-16T07:23:38.359294+0000","last_fullsized":"2024-09-16T07:23:38.359294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:36.964109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.366712+0000","last_change":"2024-09-16T07:23:37.349714+0000","last_active":"2024-09-16T07:23:38.366712+0000","last_peered":"2024-09-16T07:23:38.366712+0000","last_clean":"2024-09-16T07:23:38.366712+0000","last_became_active":"2024-09-16T07:23:37.349169+0000","last_became_peered":"2024-09-16T07:23:37.349169+0000","last_unstale":"2024-09-16T07:23:38.366712+0000","last_undegraded":"2024-09-16T07:23:38.366712+0000","last_fullsized":"2024-09-16T07:23:38.366712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:36.809629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.331286+0000","last_change":"2024-09-16T07:23:37.331286+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.331286+0000","last_undegraded":"2024-09-16T07:23:37.331286+0000","last_fullsized":"2024-09-16T07:23:37.331286+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.374468+0000","last_change":"2024-09-16T07:23:37.345215+0000","last_active":"2024-09-16T07:23:38.374468+0000","last_peered":"2024-09-16T07:23:38.374468+0000","last_clean":"2024-09-16T07:23:38.374468+0000","last_became_active":"2024-09-16T07:23:37.344006+0000","last_became_peered":"2024-09-16T07:23:37.344006+0000","last_unstale":"2024-09-16T07:23:38.374468+0000","last_undegraded":"2024-09-16T07:23:38.374468+0000","last_fullsized":"2024-09-16T07:23:38.374468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:43.819327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334029+0000","last_change":"2024-09-16T07:23:37.334029+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334029+0000","last_undegraded":"2024-09-16T07:23:37.334029+0000","last_fullsized":"2024-09-16T07:23:37.334029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.330726+0000","last_change":"2024-09-16T07:23:37.353056+0000","last_active":"2024-09-16T07:23:38.330726+0000","last_peered":"2024-09-16T07:23:38.330726+0000","last_clean":"2024-09-16T07:23:38.330726+0000","last_became_active":"2024-09-16T07:23:37.352723+0000","last_became_peered":"2024-09-16T07:23:37.352723+0000","last_unstale":"2024-09-16T07:23:38.330726+0000","last_undegraded":"2024-09-16T07:23:38.330726+0000","last_fullsized":"2024-09-16T07:23:38.330726+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:24.951401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334406+0000","last_change":"2024-09-16T07:23:37.334406+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334406+0000","last_undegraded":"2024-09-16T07:23:37.334406+0000","last_fullsized":"2024-09-16T07:23:37.334406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350390+0000","last_change":"2024-09-16T07:23:37.347166+0000","last_active":"2024-09-16T07:23:37.350390+0000","last_peered":"2024-09-16T07:23:37.350390+0000","last_clean":"2024-09-16T07:23:37.350390+0000","last_became_active":"2024-09-16T07:23:37.346890+0000","last_became_peered":"2024-09-16T07:23:37.346890+0000","last_unstale":"2024-09-16T07:23:37.350390+0000","last_undegraded":"2024-09-16T07:23:37.350390+0000","last_fullsized":"2024-09-16T07:23:37.350390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:09.516212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.330484+0000","last_change":"2024-09-16T07:23:37.330484+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.330484+0000","last_undegraded":"2024-09-16T07:23:37.330484+0000","last_fullsized":"2024-09-16T07:23:37.330484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.336973+0000","last_change":"2024-09-16T07:23:37.349623+0000","last_active":"2024-09-16T07:23:38.336973+0000","last_peered":"2024-09-16T07:23:38.336973+0000","last_clean":"2024-09-16T07:23:38.336973+0000","last_became_active":"2024-09-16T07:23:37.349374+0000","last_became_peered":"2024-09-16T07:23:37.349374+0000","last_unstale":"2024-09-16T07:23:38.336973+0000","last_undegraded":"2024-09-16T07:23:38.336973+0000","last_fullsized":"2024-09-16T07:23:38.336973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:46.136303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.336554+0000","last_change":"2024-09-16T07:23:37.336554+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.336554+0000","last_undegraded":"2024-09-16T07:23:37.336554+0000","last_fullsized":"2024-09-16T07:23:37.336554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.343402+0000","last_change":"2024-09-16T07:23:37.347390+0000","last_active":"2024-09-16T07:23:38.343402+0000","last_peered":"2024-09-16T07:23:38.343402+0000","last_clean":"2024-09-16T07:23:38.343402+0000","last_became_active":"2024-09-16T07:23:37.346226+0000","last_became_peered":"2024-09-16T07:23:37.346226+0000","last_unstale":"2024-09-16T07:23:38.343402+0000","last_undegraded":"2024-09-16T07:23:38.343402+0000","last_fullsized":"2024-09-16T07:23:38.343402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:54.186625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359489+0000","last_change":"2024-09-16T07:23:38.353841+0000","last_active":"2024-09-16T07:23:38.359489+0000","last_peered":"2024-09-16T07:23:38.359489+0000","last_clean":"2024-09-16T07:23:38.359489+0000","last_became_active":"2024-09-16T07:23:38.352266+0000","last_became_peered":"2024-09-16T07:23:38.352266+0000","last_unstale":"2024-09-16T07:23:38.359489+0000","last_undegraded":"2024-09-16T07:23:38.359489+0000","last_fullsized":"2024-09-16T07:23:38.359489+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:01.093082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.334377+0000","last_change":"2024-09-16T07:23:37.339731+0000","last_active":"2024-09-16T07:23:38.334377+0000","last_peered":"2024-09-16T07:23:38.334377+0000","last_clean":"2024-09-16T07:23:38.334377+0000","last_became_active":"2024-09-16T07:23:37.339480+0000","last_became_peered":"2024-09-16T07:23:37.339480+0000","last_unstale":"2024-09-16T07:23:38.334377+0000","last_undegraded":"2024-09-16T07:23:38.334377+0000","last_fullsized":"2024-09-16T07:23:38.334377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:58.263118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.337719+0000","last_change":"2024-09-16T07:23:37.337719+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.337719+0000","last_undegraded":"2024-09-16T07:23:37.337719+0000","last_fullsized":"2024-09-16T07:23:37.337719+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346812+0000","last_change":"2024-09-16T07:23:37.344100+0000","last_active":"2024-09-16T07:23:37.346812+0000","last_peered":"2024-09-16T07:23:37.346812+0000","last_clean":"2024-09-16T07:23:37.346812+0000","last_became_active":"2024-09-16T07:23:37.343269+0000","last_became_peered":"2024-09-16T07:23:37.343269+0000","last_unstale":"2024-09-16T07:23:37.346812+0000","last_undegraded":"2024-09-16T07:23:37.346812+0000","last_fullsized":"2024-09-16T07:23:37.346812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:39.795699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362195+0000","last_change":"2024-09-16T07:23:38.354226+0000","last_active":"2024-09-16T07:23:38.362195+0000","last_peered":"2024-09-16T07:23:38.362195+0000","last_clean":"2024-09-16T07:23:38.362195+0000","last_became_active":"2024-09-16T07:23:38.354001+0000","last_became_peered":"2024-09-16T07:23:38.354001+0000","last_unstale":"2024-09-16T07:23:38.362195+0000","last_undegraded":"2024-09-16T07:23:38.362195+0000","last_fullsized":"2024-09-16T07:23:38.362195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:16.310897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.345689+0000","last_change":"2024-09-16T07:23:37.343487+0000","last_active":"2024-09-16T07:23:37.345689+0000","last_peered":"2024-09-16T07:23:37.345689+0000","last_clean":"2024-09-16T07:23:37.345689+0000","last_became_active":"2024-09-16T07:23:37.343170+0000","last_became_peered":"2024-09-16T07:23:37.343170+0000","last_unstale":"2024-09-16T07:23:37.345689+0000","last_undegraded":"2024-09-16T07:23:37.345689+0000","last_fullsized":"2024-09-16T07:23:37.345689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:10.785300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333534+0000","last_change":"2024-09-16T07:23:37.333534+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333534+0000","last_undegraded":"2024-09-16T07:23:37.333534+0000","last_fullsized":"2024-09-16T07:23:37.333534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.348747+0000","last_change":"2024-09-16T07:23:37.346738+0000","last_active":"2024-09-16T07:23:37.348747+0000","last_peered":"2024-09-16T07:23:37.348747+0000","last_clean":"2024-09-16T07:23:37.348747+0000","last_became_active":"2024-09-16T07:23:37.345610+0000","last_became_peered":"2024-09-16T07:23:37.345610+0000","last_unstale":"2024-09-16T07:23:37.348747+0000","last_undegraded":"2024-09-16T07:23:37.348747+0000","last_fullsized":"2024-09-16T07:23:37.348747+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:45.870123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362706+0000","last_change":"2024-09-16T07:23:38.358556+0000","last_active":"2024-09-16T07:23:38.362706+0000","last_peered":"2024-09-16T07:23:38.362706+0000","last_clean":"2024-09-16T07:23:38.362706+0000","last_became_active":"2024-09-16T07:23:38.358288+0000","last_became_peered":"2024-09-16T07:23:38.358288+0000","last_unstale":"2024-09-16T07:23:38.362706+0000","last_undegraded":"2024-09-16T07:23:38.362706+0000","last_fullsized":"2024-09-16T07:23:38.362706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:34.849304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346559+0000","last_change":"2024-09-16T07:23:37.343946+0000","last_active":"2024-09-16T07:23:37.346559+0000","last_peered":"2024-09-16T07:23:37.346559+0000","last_clean":"2024-09-16T07:23:37.346559+0000","last_became_active":"2024-09-16T07:23:37.343628+0000","last_became_peered":"2024-09-16T07:23:37.343628+0000","last_unstale":"2024-09-16T07:23:37.346559+0000","last_undegraded":"2024-09-16T07:23:37.346559+0000","last_fullsized":"2024-09-16T07:23:37.346559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:00.089436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615243+0000","last_change":"2024-09-16T07:23:38.346303+0000","last_active":"2024-09-16T07:23:40.615243+0000","last_peered":"2024-09-16T07:23:40.615243+0000","last_clean":"2024-09-16T07:23:40.615243+0000","last_became_active":"2024-09-16T07:23:38.346039+0000","last_became_peered":"2024-09-16T07:23:38.346039+0000","last_unstale":"2024-09-16T07:23:40.615243+0000","last_undegraded":"2024-09-16T07:23:40.615243+0000","last_fullsized":"2024-09-16T07:23:40.615243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:47.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.342710+0000","last_change":"2024-09-16T07:23:37.340258+0000","last_active":"2024-09-16T07:23:37.342710+0000","last_peered":"2024-09-16T07:23:37.342710+0000","last_clean":"2024-09-16T07:23:37.342710+0000","last_became_active":"2024-09-16T07:23:37.339993+0000","last_became_peered":"2024-09-16T07:23:37.339993+0000","last_unstale":"2024-09-16T07:23:37.342710+0000","last_undegraded":"2024-09-16T07:23:37.342710+0000","last_fullsized":"2024-09-16T07:23:37.342710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:29.784689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614910+0000","last_change":"2024-09-16T07:23:38.342446+0000","last_active":"2024-09-16T07:23:40.614910+0000","last_peered":"2024-09-16T07:23:40.614910+0000","last_clean":"2024-09-16T07:23:40.614910+0000","last_became_active":"2024-09-16T07:23:38.341488+0000","last_became_peered":"2024-09-16T07:23:38.341488+0000","last_unstale":"2024-09-16T07:23:40.614910+0000","last_undegraded":"2024-09-16T07:23:40.614910+0000","last_fullsized":"2024-09-16T07:23:40.614910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:51.750252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.336765+0000","last_change":"2024-09-16T07:23:37.348185+0000","last_active":"2024-09-16T07:23:38.336765+0000","last_peered":"2024-09-16T07:23:38.336765+0000","last_clean":"2024-09-16T07:23:38.336765+0000","last_became_active":"2024-09-16T07:23:37.347959+0000","last_became_peered":"2024-09-16T07:23:37.347959+0000","last_unstale":"2024-09-16T07:23:38.336765+0000","last_undegraded":"2024-09-16T07:23:38.336765+0000","last_fullsized":"2024-09-16T07:23:38.336765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:21:54.247700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362963+0000","last_change":"2024-09-16T07:23:38.355905+0000","last_active":"2024-09-16T07:23:38.362963+0000","last_peered":"2024-09-16T07:23:38.362963+0000","last_clean":"2024-09-16T07:23:38.362963+0000","last_became_active":"2024-09-16T07:23:38.355546+0000","last_became_peered":"2024-09-16T07:23:38.355546+0000","last_unstale":"2024-09-16T07:23:38.362963+0000","last_undegraded":"2024-09-16T07:23:38.362963+0000","last_fullsized":"2024-09-16T07:23:38.362963+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:42.332816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614759+0000","last_change":"2024-09-16T07:23:37.345122+0000","last_active":"2024-09-16T07:23:40.614759+0000","last_peered":"2024-09-16T07:23:40.614759+0000","last_clean":"2024-09-16T07:23:40.614759+0000","last_became_active":"2024-09-16T07:23:37.344793+0000","last_became_peered":"2024-09-16T07:23:37.344793+0000","last_unstale":"2024-09-16T07:23:40.614759+0000","last_undegraded":"2024-09-16T07:23:40.614759+0000","last_fullsized":"2024-09-16T07:23:40.614759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:56.809362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362575+0000","last_change":"2024-09-16T07:23:38.358386+0000","last_active":"2024-09-16T07:23:38.362575+0000","last_peered":"2024-09-16T07:23:38.362575+0000","last_clean":"2024-09-16T07:23:38.362575+0000","last_became_active":"2024-09-16T07:23:38.358092+0000","last_became_peered":"2024-09-16T07:23:38.358092+0000","last_unstale":"2024-09-16T07:23:38.362575+0000","last_undegraded":"2024-09-16T07:23:38.362575+0000","last_fullsized":"2024-09-16T07:23:38.362575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:58:30.618313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.338916+0000","last_change":"2024-09-16T07:23:37.345296+0000","last_active":"2024-09-16T07:23:38.338916+0000","last_peered":"2024-09-16T07:23:38.338916+0000","last_clean":"2024-09-16T07:23:38.338916+0000","last_became_active":"2024-09-16T07:23:37.343905+0000","last_became_peered":"2024-09-16T07:23:37.343905+0000","last_unstale":"2024-09-16T07:23:38.338916+0000","last_undegraded":"2024-09-16T07:23:38.338916+0000","last_fullsized":"2024-09-16T07:23:38.338916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:43.228036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":130,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.338930+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:23:38.338930+0000","last_peered":"2024-09-16T07:23:38.338930+0000","last_clean":"2024-09-16T07:23:38.338930+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:23:38.338930+0000","last_undegraded":"2024-09-16T07:23:38.338930+0000","last_fullsized":"2024-09-16T07:23:38.338930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476753,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476753,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":727307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":130664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":721032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":151648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:23:42.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:23:43.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 -- 172.21.15.71:0/2227374011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 msgr2=0x7f0f9015e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 --2- 172.21.15.71:0/2227374011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 0x7f0f9015e970 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f88005fd0 tx=0x7f0f8802aa50 comp rx=0 tx=0).stop 2024-09-16T07:23:43.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 -- 172.21.15.71:0/2227374011 shutdown_connections 2024-09-16T07:23:43.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 --2- 172.21.15.71:0/2227374011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 0x7f0f9015e970 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 --2- 172.21.15.71:0/2227374011 >> v2:172.21.15.177:3300/0 conn(0x7f0f900ccfd0 0x7f0f900cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 --2- 172.21.15.71:0/2227374011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f901445d0 0x7f0f900cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 -- 172.21.15.71:0/2227374011 >> 172.21.15.71:0/2227374011 conn(0x7f0f900b1950 msgr2=0x7f0f900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:43.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 -- 172.21.15.71:0/2227374011 shutdown_connections 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.163+0000 7f0f96200640 1 -- 172.21.15.71:0/2227374011 wait complete. 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.165+0000 7f0f96200640 1 Processor -- start 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.165+0000 7f0f96200640 1 -- start start 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f96200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 0x7f0f901434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f96200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f900ccfd0 0x7f0f901428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f96200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f901445d0 0x7f0f901f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f96200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9015f240 con 0x7f0f900b4ca0 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f96200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9015f0c0 con 0x7f0f901445d0 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f96200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9015f3c0 con 0x7f0f900ccfd0 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 0x7f0f901434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:43.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 0x7f0f901434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44962/0 (socket says 172.21.15.71:44962) 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8f400640 1 -- 172.21.15.71:0/3291285100 learned_addr learned my addr 172.21.15.71:0/3291285100 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8fe00640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f901445d0 0x7f0f901f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8ea00640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f900ccfd0 0x7f0f901428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8f400640 1 -- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f900ccfd0 msgr2=0x7f0f901428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8f400640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f900ccfd0 0x7f0f901428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8f400640 1 -- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f901445d0 msgr2=0x7f0f901f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8f400640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f901445d0 0x7f0f901f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8f400640 1 -- 172.21.15.71:0/3291285100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f90142e10 con 0x7f0f900b4ca0 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.166+0000 7f0f8ea00640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f900ccfd0 0x7f0f901428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.167+0000 7f0f8f400640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 0x7f0f901434e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000d6d0 tx=0x7f0f8000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.167+0000 7f0f87400640 1 -- 172.21.15.71:0/3291285100 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80017070 con 0x7f0f900b4ca0 2024-09-16T07:23:43.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.167+0000 7f0f87400640 1 -- 172.21.15.71:0/3291285100 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f80005330 con 0x7f0f900b4ca0 2024-09-16T07:23:43.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.167+0000 7f0f87400640 1 -- 172.21.15.71:0/3291285100 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800097c0 con 0x7f0f900b4ca0 2024-09-16T07:23:43.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.167+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f90143200 con 0x7f0f900b4ca0 2024-09-16T07:23:43.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.167+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f90167c30 con 0x7f0f900b4ca0 2024-09-16T07:23:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.168+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f90167740 con 0x7f0f900b4ca0 2024-09-16T07:23:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.169+0000 7f0f87400640 1 -- 172.21.15.71:0/3291285100 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0f80009b70 con 0x7f0f900b4ca0 2024-09-16T07:23:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.170+0000 7f0f87400640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f70078850 0x7f0f7007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.170+0000 7f0f87400640 1 -- 172.21.15.71:0/3291285100 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0f8009abb0 con 0x7f0f900b4ca0 2024-09-16T07:23:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.171+0000 7f0f8ea00640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f70078850 0x7f0f7007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.171+0000 7f0f8ea00640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f70078850 0x7f0f7007ad10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f900ae620 tx=0x7f0f7c005f10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:43.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.174+0000 7f0f87400640 1 -- 172.21.15.71:0/3291285100 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0f80062b30 con 0x7f0f900b4ca0 2024-09-16T07:23:43.214 INFO:tasks.ceph.mds.b.smithi177.stderr:2024-09-16T07:23:43.215+0000 7f5a40800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:23:43.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.355+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f90070a20 con 0x7f0f70078850 2024-09-16T07:23:43.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.359+0000 7f0f87400640 1 -- 172.21.15.71:0/3291285100 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250440 (secure 0 0 0) 0x7f0f90070a20 con 0x7f0f70078850 2024-09-16T07:23:43.362 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:43.363 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:23:43.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f70078850 msgr2=0x7f0f7007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f70078850 0x7f0f7007ad10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f900ae620 tx=0x7f0f7c005f10 comp rx=0 tx=0).stop 2024-09-16T07:23:43.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 msgr2=0x7f0f901434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 0x7f0f901434e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000d6d0 tx=0x7f0f8000db90 comp rx=0 tx=0).stop 2024-09-16T07:23:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 shutdown_connections 2024-09-16T07:23:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f70078850 0x7f0f7007ad10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f901445d0 0x7f0f901f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f900ccfd0 0x7f0f901428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 --2- 172.21.15.71:0/3291285100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f900b4ca0 0x7f0f901434e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.363+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 >> 172.21.15.71:0/3291285100 conn(0x7f0f900b1950 msgr2=0x7f0f900b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.364+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 shutdown_connections 2024-09-16T07:23:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.364+0000 7f0f96200640 1 -- 172.21.15.71:0/3291285100 wait complete. 2024-09-16T07:23:43.379 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-16T07:23:42.305320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":119,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224400,"kb_used_data":3992,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750320,"statfs":{"total":773094113280,"available":772864327680,"internally_reserved":0,"allocated":4087808,"data_stored":2278407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.979386"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350368+0000","last_change":"2024-09-16T07:23:38.347812+0000","last_active":"2024-09-16T07:23:38.350368+0000","last_peered":"2024-09-16T07:23:38.350368+0000","last_clean":"2024-09-16T07:23:38.350368+0000","last_became_active":"2024-09-16T07:23:38.347516+0000","last_became_peered":"2024-09-16T07:23:38.347516+0000","last_unstale":"2024-09-16T07:23:38.350368+0000","last_undegraded":"2024-09-16T07:23:38.350368+0000","last_fullsized":"2024-09-16T07:23:38.350368+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:06.049071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334422+0000","last_change":"2024-09-16T07:23:37.334422+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334422+0000","last_undegraded":"2024-09-16T07:23:37.334422+0000","last_fullsized":"2024-09-16T07:23:37.334422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.360416+0000","last_change":"2024-09-16T07:23:38.352895+0000","last_active":"2024-09-16T07:23:38.360416+0000","last_peered":"2024-09-16T07:23:38.360416+0000","last_clean":"2024-09-16T07:23:38.360416+0000","last_became_active":"2024-09-16T07:23:38.352621+0000","last_became_peered":"2024-09-16T07:23:38.352621+0000","last_unstale":"2024-09-16T07:23:38.360416+0000","last_undegraded":"2024-09-16T07:23:38.360416+0000","last_fullsized":"2024-09-16T07:23:38.360416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:14.142804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615292+0000","last_change":"2024-09-16T07:23:38.350692+0000","last_active":"2024-09-16T07:23:40.615292+0000","last_peered":"2024-09-16T07:23:40.615292+0000","last_clean":"2024-09-16T07:23:40.615292+0000","last_became_active":"2024-09-16T07:23:38.347475+0000","last_became_peered":"2024-09-16T07:23:38.347475+0000","last_unstale":"2024-09-16T07:23:40.615292+0000","last_undegraded":"2024-09-16T07:23:40.615292+0000","last_fullsized":"2024-09-16T07:23:40.615292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:01.740131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.331162+0000","last_change":"2024-09-16T07:23:37.331162+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.331162+0000","last_undegraded":"2024-09-16T07:23:37.331162+0000","last_fullsized":"2024-09-16T07:23:37.331162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334158+0000","last_change":"2024-09-16T07:23:37.334158+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334158+0000","last_undegraded":"2024-09-16T07:23:37.334158+0000","last_fullsized":"2024-09-16T07:23:37.334158+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.368246+0000","last_change":"2024-09-16T07:23:38.367300+0000","last_active":"2024-09-16T07:23:38.368246+0000","last_peered":"2024-09-16T07:23:38.368246+0000","last_clean":"2024-09-16T07:23:38.368246+0000","last_became_active":"2024-09-16T07:23:38.365509+0000","last_became_peered":"2024-09-16T07:23:38.365509+0000","last_unstale":"2024-09-16T07:23:38.368246+0000","last_undegraded":"2024-09-16T07:23:38.368246+0000","last_fullsized":"2024-09-16T07:23:38.368246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:19.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.358930+0000","last_change":"2024-09-16T07:23:38.357105+0000","last_active":"2024-09-16T07:23:38.358930+0000","last_peered":"2024-09-16T07:23:38.358930+0000","last_clean":"2024-09-16T07:23:38.358930+0000","last_became_active":"2024-09-16T07:23:38.355543+0000","last_became_peered":"2024-09-16T07:23:38.355543+0000","last_unstale":"2024-09-16T07:23:38.358930+0000","last_undegraded":"2024-09-16T07:23:38.358930+0000","last_fullsized":"2024-09-16T07:23:38.358930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:21.370546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359316+0000","last_change":"2024-09-16T07:23:38.356452+0000","last_active":"2024-09-16T07:23:38.359316+0000","last_peered":"2024-09-16T07:23:38.359316+0000","last_clean":"2024-09-16T07:23:38.359316+0000","last_became_active":"2024-09-16T07:23:38.355663+0000","last_became_peered":"2024-09-16T07:23:38.355663+0000","last_unstale":"2024-09-16T07:23:38.359316+0000","last_undegraded":"2024-09-16T07:23:38.359316+0000","last_fullsized":"2024-09-16T07:23:38.359316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.788335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615484+0000","last_change":"2024-09-16T07:23:38.356446+0000","last_active":"2024-09-16T07:23:40.615484+0000","last_peered":"2024-09-16T07:23:40.615484+0000","last_clean":"2024-09-16T07:23:40.615484+0000","last_became_active":"2024-09-16T07:23:38.354502+0000","last_became_peered":"2024-09-16T07:23:38.354502+0000","last_unstale":"2024-09-16T07:23:40.615484+0000","last_undegraded":"2024-09-16T07:23:40.615484+0000","last_fullsized":"2024-09-16T07:23:40.615484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:27.342707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352817+0000","last_change":"2024-09-16T07:23:38.351223+0000","last_active":"2024-09-16T07:23:38.352817+0000","last_peered":"2024-09-16T07:23:38.352817+0000","last_clean":"2024-09-16T07:23:38.352817+0000","last_became_active":"2024-09-16T07:23:38.349810+0000","last_became_peered":"2024-09-16T07:23:38.349810+0000","last_unstale":"2024-09-16T07:23:38.352817+0000","last_undegraded":"2024-09-16T07:23:38.352817+0000","last_fullsized":"2024-09-16T07:23:38.352817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:37.812192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.367826+0000","last_change":"2024-09-16T07:23:38.365322+0000","last_active":"2024-09-16T07:23:38.367826+0000","last_peered":"2024-09-16T07:23:38.367826+0000","last_clean":"2024-09-16T07:23:38.367826+0000","last_became_active":"2024-09-16T07:23:38.364995+0000","last_became_peered":"2024-09-16T07:23:38.364995+0000","last_unstale":"2024-09-16T07:23:38.367826+0000","last_undegraded":"2024-09-16T07:23:38.367826+0000","last_fullsized":"2024-09-16T07:23:38.367826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:40.799346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.335382+0000","last_change":"2024-09-16T07:23:37.335382+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.335382+0000","last_undegraded":"2024-09-16T07:23:37.335382+0000","last_fullsized":"2024-09-16T07:23:37.335382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614877+0000","last_change":"2024-09-16T07:23:38.347109+0000","last_active":"2024-09-16T07:23:40.614877+0000","last_peered":"2024-09-16T07:23:40.614877+0000","last_clean":"2024-09-16T07:23:40.614877+0000","last_became_active":"2024-09-16T07:23:38.346821+0000","last_became_peered":"2024-09-16T07:23:38.346821+0000","last_unstale":"2024-09-16T07:23:40.614877+0000","last_undegraded":"2024-09-16T07:23:40.614877+0000","last_fullsized":"2024-09-16T07:23:40.614877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:59.555576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.345227+0000","last_change":"2024-09-16T07:23:38.341149+0000","last_active":"2024-09-16T07:23:38.345227+0000","last_peered":"2024-09-16T07:23:38.345227+0000","last_clean":"2024-09-16T07:23:38.345227+0000","last_became_active":"2024-09-16T07:23:38.340794+0000","last_became_peered":"2024-09-16T07:23:38.340794+0000","last_unstale":"2024-09-16T07:23:38.345227+0000","last_undegraded":"2024-09-16T07:23:38.345227+0000","last_fullsized":"2024-09-16T07:23:38.345227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:17.507704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362540+0000","last_change":"2024-09-16T07:23:38.360203+0000","last_active":"2024-09-16T07:23:38.362540+0000","last_peered":"2024-09-16T07:23:38.362540+0000","last_clean":"2024-09-16T07:23:38.362540+0000","last_became_active":"2024-09-16T07:23:38.359933+0000","last_became_peered":"2024-09-16T07:23:38.359933+0000","last_unstale":"2024-09-16T07:23:38.362540+0000","last_undegraded":"2024-09-16T07:23:38.362540+0000","last_fullsized":"2024-09-16T07:23:38.362540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:55.869414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352904+0000","last_change":"2024-09-16T07:23:38.350866+0000","last_active":"2024-09-16T07:23:38.352904+0000","last_peered":"2024-09-16T07:23:38.352904+0000","last_clean":"2024-09-16T07:23:38.352904+0000","last_became_active":"2024-09-16T07:23:38.350617+0000","last_became_peered":"2024-09-16T07:23:38.350617+0000","last_unstale":"2024-09-16T07:23:38.352904+0000","last_undegraded":"2024-09-16T07:23:38.352904+0000","last_fullsized":"2024-09-16T07:23:38.352904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:14:27.853553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.338226+0000","last_change":"2024-09-16T07:23:37.338226+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.338226+0000","last_undegraded":"2024-09-16T07:23:37.338226+0000","last_fullsized":"2024-09-16T07:23:37.338226+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334511+0000","last_change":"2024-09-16T07:23:37.334511+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334511+0000","last_undegraded":"2024-09-16T07:23:37.334511+0000","last_fullsized":"2024-09-16T07:23:37.334511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.337607+0000","last_change":"2024-09-16T07:23:37.337607+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.337607+0000","last_undegraded":"2024-09-16T07:23:37.337607+0000","last_fullsized":"2024-09-16T07:23:37.337607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.357360+0000","last_change":"2024-09-16T07:23:38.355303+0000","last_active":"2024-09-16T07:23:38.357360+0000","last_peered":"2024-09-16T07:23:38.357360+0000","last_clean":"2024-09-16T07:23:38.357360+0000","last_became_active":"2024-09-16T07:23:38.355033+0000","last_became_peered":"2024-09-16T07:23:38.355033+0000","last_unstale":"2024-09-16T07:23:38.357360+0000","last_undegraded":"2024-09-16T07:23:38.357360+0000","last_fullsized":"2024-09-16T07:23:38.357360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:19.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350313+0000","last_change":"2024-09-16T07:23:38.345255+0000","last_active":"2024-09-16T07:23:38.350313+0000","last_peered":"2024-09-16T07:23:38.350313+0000","last_clean":"2024-09-16T07:23:38.350313+0000","last_became_active":"2024-09-16T07:23:38.344965+0000","last_became_peered":"2024-09-16T07:23:38.344965+0000","last_unstale":"2024-09-16T07:23:38.350313+0000","last_undegraded":"2024-09-16T07:23:38.350313+0000","last_fullsized":"2024-09-16T07:23:38.350313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:09.790205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334034+0000","last_change":"2024-09-16T07:23:37.334034+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334034+0000","last_undegraded":"2024-09-16T07:23:37.334034+0000","last_fullsized":"2024-09-16T07:23:37.334034+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352659+0000","last_change":"2024-09-16T07:23:38.348792+0000","last_active":"2024-09-16T07:23:38.352659+0000","last_peered":"2024-09-16T07:23:38.352659+0000","last_clean":"2024-09-16T07:23:38.352659+0000","last_became_active":"2024-09-16T07:23:38.345353+0000","last_became_peered":"2024-09-16T07:23:38.345353+0000","last_unstale":"2024-09-16T07:23:38.352659+0000","last_undegraded":"2024-09-16T07:23:38.352659+0000","last_fullsized":"2024-09-16T07:23:38.352659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:32.489176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615553+0000","last_change":"2024-09-16T07:23:38.351284+0000","last_active":"2024-09-16T07:23:40.615553+0000","last_peered":"2024-09-16T07:23:40.615553+0000","last_clean":"2024-09-16T07:23:40.615553+0000","last_became_active":"2024-09-16T07:23:38.350972+0000","last_became_peered":"2024-09-16T07:23:38.350972+0000","last_unstale":"2024-09-16T07:23:40.615553+0000","last_undegraded":"2024-09-16T07:23:40.615553+0000","last_fullsized":"2024-09-16T07:23:40.615553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:25.493457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615534+0000","last_change":"2024-09-16T07:23:38.356598+0000","last_active":"2024-09-16T07:23:40.615534+0000","last_peered":"2024-09-16T07:23:40.615534+0000","last_clean":"2024-09-16T07:23:40.615534+0000","last_became_active":"2024-09-16T07:23:38.356177+0000","last_became_peered":"2024-09-16T07:23:38.356177+0000","last_unstale":"2024-09-16T07:23:40.615534+0000","last_undegraded":"2024-09-16T07:23:40.615534+0000","last_fullsized":"2024-09-16T07:23:40.615534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:42.647875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.330534+0000","last_change":"2024-09-16T07:23:37.330534+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.330534+0000","last_undegraded":"2024-09-16T07:23:37.330534+0000","last_fullsized":"2024-09-16T07:23:37.330534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.364143+0000","last_change":"2024-09-16T07:23:38.362005+0000","last_active":"2024-09-16T07:23:38.364143+0000","last_peered":"2024-09-16T07:23:38.364143+0000","last_clean":"2024-09-16T07:23:38.364143+0000","last_became_active":"2024-09-16T07:23:38.361748+0000","last_became_peered":"2024-09-16T07:23:38.361748+0000","last_unstale":"2024-09-16T07:23:38.364143+0000","last_undegraded":"2024-09-16T07:23:38.364143+0000","last_fullsized":"2024-09-16T07:23:38.364143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:17.507704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.343486+0000","last_change":"2024-09-16T07:23:38.337747+0000","last_active":"2024-09-16T07:23:38.343486+0000","last_peered":"2024-09-16T07:23:38.343486+0000","last_clean":"2024-09-16T07:23:38.343486+0000","last_became_active":"2024-09-16T07:23:38.337455+0000","last_became_peered":"2024-09-16T07:23:38.337455+0000","last_unstale":"2024-09-16T07:23:38.343486+0000","last_undegraded":"2024-09-16T07:23:38.343486+0000","last_fullsized":"2024-09-16T07:23:38.343486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:21.370546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.351724+0000","last_change":"2024-09-16T07:23:38.348476+0000","last_active":"2024-09-16T07:23:38.351724+0000","last_peered":"2024-09-16T07:23:38.351724+0000","last_clean":"2024-09-16T07:23:38.351724+0000","last_became_active":"2024-09-16T07:23:38.348263+0000","last_became_peered":"2024-09-16T07:23:38.348263+0000","last_unstale":"2024-09-16T07:23:38.351724+0000","last_undegraded":"2024-09-16T07:23:38.351724+0000","last_fullsized":"2024-09-16T07:23:38.351724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:24.558417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.358843+0000","last_change":"2024-09-16T07:23:38.356216+0000","last_active":"2024-09-16T07:23:38.358843+0000","last_peered":"2024-09-16T07:23:38.358843+0000","last_clean":"2024-09-16T07:23:38.358843+0000","last_became_active":"2024-09-16T07:23:38.355971+0000","last_became_peered":"2024-09-16T07:23:38.355971+0000","last_unstale":"2024-09-16T07:23:38.358843+0000","last_undegraded":"2024-09-16T07:23:38.358843+0000","last_fullsized":"2024-09-16T07:23:38.358843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:25.955048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.346389+0000","last_change":"2024-09-16T07:23:38.342538+0000","last_active":"2024-09-16T07:23:38.346389+0000","last_peered":"2024-09-16T07:23:38.346389+0000","last_clean":"2024-09-16T07:23:38.346389+0000","last_became_active":"2024-09-16T07:23:38.342085+0000","last_became_peered":"2024-09-16T07:23:38.342085+0000","last_unstale":"2024-09-16T07:23:38.346389+0000","last_undegraded":"2024-09-16T07:23:38.346389+0000","last_fullsized":"2024-09-16T07:23:38.346389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:56.231147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333820+0000","last_change":"2024-09-16T07:23:37.333820+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333820+0000","last_undegraded":"2024-09-16T07:23:37.333820+0000","last_fullsized":"2024-09-16T07:23:37.333820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.375763+0000","last_change":"2024-09-16T07:23:37.346853+0000","last_active":"2024-09-16T07:23:38.375763+0000","last_peered":"2024-09-16T07:23:38.375763+0000","last_clean":"2024-09-16T07:23:38.375763+0000","last_became_active":"2024-09-16T07:23:37.344215+0000","last_became_peered":"2024-09-16T07:23:37.344215+0000","last_unstale":"2024-09-16T07:23:38.375763+0000","last_undegraded":"2024-09-16T07:23:38.375763+0000","last_fullsized":"2024-09-16T07:23:38.375763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:55.470787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350348+0000","last_change":"2024-09-16T07:23:38.347363+0000","last_active":"2024-09-16T07:23:38.350348+0000","last_peered":"2024-09-16T07:23:38.350348+0000","last_clean":"2024-09-16T07:23:38.350348+0000","last_became_active":"2024-09-16T07:23:38.347091+0000","last_became_peered":"2024-09-16T07:23:38.347091+0000","last_unstale":"2024-09-16T07:23:38.350348+0000","last_undegraded":"2024-09-16T07:23:38.350348+0000","last_fullsized":"2024-09-16T07:23:38.350348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:16.751332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615579+0000","last_change":"2024-09-16T07:23:37.349796+0000","last_active":"2024-09-16T07:23:40.615579+0000","last_peered":"2024-09-16T07:23:40.615579+0000","last_clean":"2024-09-16T07:23:40.615579+0000","last_became_active":"2024-09-16T07:23:37.349505+0000","last_became_peered":"2024-09-16T07:23:37.349505+0000","last_unstale":"2024-09-16T07:23:40.615579+0000","last_undegraded":"2024-09-16T07:23:40.615579+0000","last_fullsized":"2024-09-16T07:23:40.615579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:04.156099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.349933+0000","last_change":"2024-09-16T07:23:38.344847+0000","last_active":"2024-09-16T07:23:38.349933+0000","last_peered":"2024-09-16T07:23:38.349933+0000","last_clean":"2024-09-16T07:23:38.349933+0000","last_became_active":"2024-09-16T07:23:38.344502+0000","last_became_peered":"2024-09-16T07:23:38.344502+0000","last_unstale":"2024-09-16T07:23:38.349933+0000","last_undegraded":"2024-09-16T07:23:38.349933+0000","last_fullsized":"2024-09-16T07:23:38.349933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:26.398231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350064+0000","last_change":"2024-09-16T07:23:37.347456+0000","last_active":"2024-09-16T07:23:37.350064+0000","last_peered":"2024-09-16T07:23:37.350064+0000","last_clean":"2024-09-16T07:23:37.350064+0000","last_became_active":"2024-09-16T07:23:37.347152+0000","last_became_peered":"2024-09-16T07:23:37.347152+0000","last_unstale":"2024-09-16T07:23:37.350064+0000","last_undegraded":"2024-09-16T07:23:37.350064+0000","last_fullsized":"2024-09-16T07:23:37.350064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:51:05.553785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615486+0000","last_change":"2024-09-16T07:23:38.353195+0000","last_active":"2024-09-16T07:23:40.615486+0000","last_peered":"2024-09-16T07:23:40.615486+0000","last_clean":"2024-09-16T07:23:40.615486+0000","last_became_active":"2024-09-16T07:23:38.350558+0000","last_became_peered":"2024-09-16T07:23:38.350558+0000","last_unstale":"2024-09-16T07:23:40.615486+0000","last_undegraded":"2024-09-16T07:23:40.615486+0000","last_fullsized":"2024-09-16T07:23:40.615486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:23.647474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615384+0000","last_change":"2024-09-16T07:23:37.348375+0000","last_active":"2024-09-16T07:23:40.615384+0000","last_peered":"2024-09-16T07:23:40.615384+0000","last_clean":"2024-09-16T07:23:40.615384+0000","last_became_active":"2024-09-16T07:23:37.348096+0000","last_became_peered":"2024-09-16T07:23:37.348096+0000","last_unstale":"2024-09-16T07:23:40.615384+0000","last_undegraded":"2024-09-16T07:23:40.615384+0000","last_fullsized":"2024-09-16T07:23:40.615384+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:37.348737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359143+0000","last_change":"2024-09-16T07:23:38.351112+0000","last_active":"2024-09-16T07:23:38.359143+0000","last_peered":"2024-09-16T07:23:38.359143+0000","last_clean":"2024-09-16T07:23:38.359143+0000","last_became_active":"2024-09-16T07:23:38.350832+0000","last_became_peered":"2024-09-16T07:23:38.350832+0000","last_unstale":"2024-09-16T07:23:38.359143+0000","last_undegraded":"2024-09-16T07:23:38.359143+0000","last_fullsized":"2024-09-16T07:23:38.359143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:44.019182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359294+0000","last_change":"2024-09-16T07:23:37.347090+0000","last_active":"2024-09-16T07:23:38.359294+0000","last_peered":"2024-09-16T07:23:38.359294+0000","last_clean":"2024-09-16T07:23:38.359294+0000","last_became_active":"2024-09-16T07:23:37.346832+0000","last_became_peered":"2024-09-16T07:23:37.346832+0000","last_unstale":"2024-09-16T07:23:38.359294+0000","last_undegraded":"2024-09-16T07:23:38.359294+0000","last_fullsized":"2024-09-16T07:23:38.359294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:41.066939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.364807+0000","last_change":"2024-09-16T07:23:38.363179+0000","last_active":"2024-09-16T07:23:38.364807+0000","last_peered":"2024-09-16T07:23:38.364807+0000","last_clean":"2024-09-16T07:23:38.364807+0000","last_became_active":"2024-09-16T07:23:38.360647+0000","last_became_peered":"2024-09-16T07:23:38.360647+0000","last_unstale":"2024-09-16T07:23:38.364807+0000","last_undegraded":"2024-09-16T07:23:38.364807+0000","last_fullsized":"2024-09-16T07:23:38.364807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:24.558417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.363776+0000","last_change":"2024-09-16T07:23:37.344503+0000","last_active":"2024-09-16T07:23:38.363776+0000","last_peered":"2024-09-16T07:23:38.363776+0000","last_clean":"2024-09-16T07:23:38.363776+0000","last_became_active":"2024-09-16T07:23:37.344139+0000","last_became_peered":"2024-09-16T07:23:37.344139+0000","last_unstale":"2024-09-16T07:23:38.363776+0000","last_undegraded":"2024-09-16T07:23:38.363776+0000","last_fullsized":"2024-09-16T07:23:38.363776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:59.430414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352723+0000","last_change":"2024-09-16T07:23:38.348789+0000","last_active":"2024-09-16T07:23:38.352723+0000","last_peered":"2024-09-16T07:23:38.352723+0000","last_clean":"2024-09-16T07:23:38.352723+0000","last_became_active":"2024-09-16T07:23:38.348529+0000","last_became_peered":"2024-09-16T07:23:38.348529+0000","last_unstale":"2024-09-16T07:23:38.352723+0000","last_undegraded":"2024-09-16T07:23:38.352723+0000","last_fullsized":"2024-09-16T07:23:38.352723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:16.637145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350090+0000","last_change":"2024-09-16T07:23:37.348311+0000","last_active":"2024-09-16T07:23:37.350090+0000","last_peered":"2024-09-16T07:23:37.350090+0000","last_clean":"2024-09-16T07:23:37.350090+0000","last_became_active":"2024-09-16T07:23:37.348006+0000","last_became_peered":"2024-09-16T07:23:37.348006+0000","last_unstale":"2024-09-16T07:23:37.350090+0000","last_undegraded":"2024-09-16T07:23:37.350090+0000","last_fullsized":"2024-09-16T07:23:37.350090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:29.668774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614873+0000","last_change":"2024-09-16T07:23:38.352705+0000","last_active":"2024-09-16T07:23:40.614873+0000","last_peered":"2024-09-16T07:23:40.614873+0000","last_clean":"2024-09-16T07:23:40.614873+0000","last_became_active":"2024-09-16T07:23:38.352415+0000","last_became_peered":"2024-09-16T07:23:38.352415+0000","last_unstale":"2024-09-16T07:23:40.614873+0000","last_undegraded":"2024-09-16T07:23:40.614873+0000","last_fullsized":"2024-09-16T07:23:40.614873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:10.916617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.335903+0000","last_change":"2024-09-16T07:23:37.353028+0000","last_active":"2024-09-16T07:23:38.335903+0000","last_peered":"2024-09-16T07:23:38.335903+0000","last_clean":"2024-09-16T07:23:38.335903+0000","last_became_active":"2024-09-16T07:23:37.352718+0000","last_became_peered":"2024-09-16T07:23:37.352718+0000","last_unstale":"2024-09-16T07:23:38.335903+0000","last_undegraded":"2024-09-16T07:23:38.335903+0000","last_fullsized":"2024-09-16T07:23:38.335903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:41.965086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.360894+0000","last_change":"2024-09-16T07:23:38.353299+0000","last_active":"2024-09-16T07:23:38.360894+0000","last_peered":"2024-09-16T07:23:38.360894+0000","last_clean":"2024-09-16T07:23:38.360894+0000","last_became_active":"2024-09-16T07:23:38.352818+0000","last_became_peered":"2024-09-16T07:23:38.352818+0000","last_unstale":"2024-09-16T07:23:38.360894+0000","last_undegraded":"2024-09-16T07:23:38.360894+0000","last_fullsized":"2024-09-16T07:23:38.360894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:02.666636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.349313+0000","last_change":"2024-09-16T07:23:37.346845+0000","last_active":"2024-09-16T07:23:37.349313+0000","last_peered":"2024-09-16T07:23:37.349313+0000","last_clean":"2024-09-16T07:23:37.349313+0000","last_became_active":"2024-09-16T07:23:37.346574+0000","last_became_peered":"2024-09-16T07:23:37.346574+0000","last_unstale":"2024-09-16T07:23:37.349313+0000","last_undegraded":"2024-09-16T07:23:37.349313+0000","last_fullsized":"2024-09-16T07:23:37.349313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:30.460837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615689+0000","last_change":"2024-09-16T07:23:38.358332+0000","last_active":"2024-09-16T07:23:40.615689+0000","last_peered":"2024-09-16T07:23:40.615689+0000","last_clean":"2024-09-16T07:23:40.615689+0000","last_became_active":"2024-09-16T07:23:38.355145+0000","last_became_peered":"2024-09-16T07:23:38.355145+0000","last_unstale":"2024-09-16T07:23:40.615689+0000","last_undegraded":"2024-09-16T07:23:40.615689+0000","last_fullsized":"2024-09-16T07:23:40.615689+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.765410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346902+0000","last_change":"2024-09-16T07:23:37.345039+0000","last_active":"2024-09-16T07:23:37.346902+0000","last_peered":"2024-09-16T07:23:37.346902+0000","last_clean":"2024-09-16T07:23:37.346902+0000","last_became_active":"2024-09-16T07:23:37.344529+0000","last_became_peered":"2024-09-16T07:23:37.344529+0000","last_unstale":"2024-09-16T07:23:37.346902+0000","last_undegraded":"2024-09-16T07:23:37.346902+0000","last_fullsized":"2024-09-16T07:23:37.346902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:18.045656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615409+0000","last_change":"2024-09-16T07:23:38.344916+0000","last_active":"2024-09-16T07:23:40.615409+0000","last_peered":"2024-09-16T07:23:40.615409+0000","last_clean":"2024-09-16T07:23:40.615409+0000","last_became_active":"2024-09-16T07:23:38.344552+0000","last_became_peered":"2024-09-16T07:23:38.344552+0000","last_unstale":"2024-09-16T07:23:40.615409+0000","last_undegraded":"2024-09-16T07:23:40.615409+0000","last_fullsized":"2024-09-16T07:23:40.615409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:07:30.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.374362+0000","last_change":"2024-09-16T07:23:37.337958+0000","last_active":"2024-09-16T07:23:38.374362+0000","last_peered":"2024-09-16T07:23:38.374362+0000","last_clean":"2024-09-16T07:23:38.374362+0000","last_became_active":"2024-09-16T07:23:37.337681+0000","last_became_peered":"2024-09-16T07:23:37.337681+0000","last_unstale":"2024-09-16T07:23:38.374362+0000","last_undegraded":"2024-09-16T07:23:38.374362+0000","last_fullsized":"2024-09-16T07:23:38.374362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:31.334181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352729+0000","last_change":"2024-09-16T07:23:38.350404+0000","last_active":"2024-09-16T07:23:38.352729+0000","last_peered":"2024-09-16T07:23:38.352729+0000","last_clean":"2024-09-16T07:23:38.352729+0000","last_became_active":"2024-09-16T07:23:38.350111+0000","last_became_peered":"2024-09-16T07:23:38.350111+0000","last_unstale":"2024-09-16T07:23:38.352729+0000","last_undegraded":"2024-09-16T07:23:38.352729+0000","last_fullsized":"2024-09-16T07:23:38.352729+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:22.548419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.354301+0000","last_change":"2024-09-16T07:23:37.353132+0000","last_active":"2024-09-16T07:23:37.354301+0000","last_peered":"2024-09-16T07:23:37.354301+0000","last_clean":"2024-09-16T07:23:37.354301+0000","last_became_active":"2024-09-16T07:23:37.352306+0000","last_became_peered":"2024-09-16T07:23:37.352306+0000","last_unstale":"2024-09-16T07:23:37.354301+0000","last_undegraded":"2024-09-16T07:23:37.354301+0000","last_fullsized":"2024-09-16T07:23:37.354301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:30.195425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333905+0000","last_change":"2024-09-16T07:23:37.333905+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333905+0000","last_undegraded":"2024-09-16T07:23:37.333905+0000","last_fullsized":"2024-09-16T07:23:37.333905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.365888+0000","last_change":"2024-09-16T07:23:37.347964+0000","last_active":"2024-09-16T07:23:38.365888+0000","last_peered":"2024-09-16T07:23:38.365888+0000","last_clean":"2024-09-16T07:23:38.365888+0000","last_became_active":"2024-09-16T07:23:37.347687+0000","last_became_peered":"2024-09-16T07:23:37.347687+0000","last_unstale":"2024-09-16T07:23:38.365888+0000","last_undegraded":"2024-09-16T07:23:38.365888+0000","last_fullsized":"2024-09-16T07:23:38.365888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:24.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615132+0000","last_change":"2024-09-16T07:23:38.352259+0000","last_active":"2024-09-16T07:23:40.615132+0000","last_peered":"2024-09-16T07:23:40.615132+0000","last_clean":"2024-09-16T07:23:40.615132+0000","last_became_active":"2024-09-16T07:23:38.351935+0000","last_became_peered":"2024-09-16T07:23:38.351935+0000","last_unstale":"2024-09-16T07:23:40.615132+0000","last_undegraded":"2024-09-16T07:23:40.615132+0000","last_fullsized":"2024-09-16T07:23:40.615132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:09:10.158498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614756+0000","last_change":"2024-09-16T07:23:37.350953+0000","last_active":"2024-09-16T07:23:40.614756+0000","last_peered":"2024-09-16T07:23:40.614756+0000","last_clean":"2024-09-16T07:23:40.614756+0000","last_became_active":"2024-09-16T07:23:37.350621+0000","last_became_peered":"2024-09-16T07:23:37.350621+0000","last_unstale":"2024-09-16T07:23:40.614756+0000","last_undegraded":"2024-09-16T07:23:40.614756+0000","last_fullsized":"2024-09-16T07:23:40.614756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:34.314424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352682+0000","last_change":"2024-09-16T07:23:38.348443+0000","last_active":"2024-09-16T07:23:38.352682+0000","last_peered":"2024-09-16T07:23:38.352682+0000","last_clean":"2024-09-16T07:23:38.352682+0000","last_became_active":"2024-09-16T07:23:38.347974+0000","last_became_peered":"2024-09-16T07:23:38.347974+0000","last_unstale":"2024-09-16T07:23:38.352682+0000","last_undegraded":"2024-09-16T07:23:38.352682+0000","last_fullsized":"2024-09-16T07:23:38.352682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:25:27.231251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614598+0000","last_change":"2024-09-16T07:23:37.352837+0000","last_active":"2024-09-16T07:23:40.614598+0000","last_peered":"2024-09-16T07:23:40.614598+0000","last_clean":"2024-09-16T07:23:40.614598+0000","last_became_active":"2024-09-16T07:23:37.352566+0000","last_became_peered":"2024-09-16T07:23:37.352566+0000","last_unstale":"2024-09-16T07:23:40.614598+0000","last_undegraded":"2024-09-16T07:23:40.614598+0000","last_fullsized":"2024-09-16T07:23:40.614598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:20.188668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.347236+0000","last_change":"2024-09-16T07:23:38.344224+0000","last_active":"2024-09-16T07:23:38.347236+0000","last_peered":"2024-09-16T07:23:38.347236+0000","last_clean":"2024-09-16T07:23:38.347236+0000","last_became_active":"2024-09-16T07:23:38.343918+0000","last_became_peered":"2024-09-16T07:23:38.343918+0000","last_unstale":"2024-09-16T07:23:38.347236+0000","last_undegraded":"2024-09-16T07:23:38.347236+0000","last_fullsized":"2024-09-16T07:23:38.347236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:55.009205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350824+0000","last_change":"2024-09-16T07:23:37.349547+0000","last_active":"2024-09-16T07:23:37.350824+0000","last_peered":"2024-09-16T07:23:37.350824+0000","last_clean":"2024-09-16T07:23:37.350824+0000","last_became_active":"2024-09-16T07:23:37.349283+0000","last_became_peered":"2024-09-16T07:23:37.349283+0000","last_unstale":"2024-09-16T07:23:37.350824+0000","last_undegraded":"2024-09-16T07:23:37.350824+0000","last_fullsized":"2024-09-16T07:23:37.350824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:41.491685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.353115+0000","last_change":"2024-09-16T07:23:38.349714+0000","last_active":"2024-09-16T07:23:38.353115+0000","last_peered":"2024-09-16T07:23:38.353115+0000","last_clean":"2024-09-16T07:23:38.353115+0000","last_became_active":"2024-09-16T07:23:38.349431+0000","last_became_peered":"2024-09-16T07:23:38.349431+0000","last_unstale":"2024-09-16T07:23:38.353115+0000","last_undegraded":"2024-09-16T07:23:38.353115+0000","last_fullsized":"2024-09-16T07:23:38.353115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:28.910006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.353383+0000","last_change":"2024-09-16T07:23:37.352287+0000","last_active":"2024-09-16T07:23:37.353383+0000","last_peered":"2024-09-16T07:23:37.353383+0000","last_clean":"2024-09-16T07:23:37.353383+0000","last_became_active":"2024-09-16T07:23:37.351844+0000","last_became_peered":"2024-09-16T07:23:37.351844+0000","last_unstale":"2024-09-16T07:23:37.353383+0000","last_undegraded":"2024-09-16T07:23:37.353383+0000","last_fullsized":"2024-09-16T07:23:37.353383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:24.047148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359294+0000","last_change":"2024-09-16T07:23:38.356161+0000","last_active":"2024-09-16T07:23:38.359294+0000","last_peered":"2024-09-16T07:23:38.359294+0000","last_clean":"2024-09-16T07:23:38.359294+0000","last_became_active":"2024-09-16T07:23:38.354062+0000","last_became_peered":"2024-09-16T07:23:38.354062+0000","last_unstale":"2024-09-16T07:23:38.359294+0000","last_undegraded":"2024-09-16T07:23:38.359294+0000","last_fullsized":"2024-09-16T07:23:38.359294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:36.964109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.366712+0000","last_change":"2024-09-16T07:23:37.349714+0000","last_active":"2024-09-16T07:23:38.366712+0000","last_peered":"2024-09-16T07:23:38.366712+0000","last_clean":"2024-09-16T07:23:38.366712+0000","last_became_active":"2024-09-16T07:23:37.349169+0000","last_became_peered":"2024-09-16T07:23:37.349169+0000","last_unstale":"2024-09-16T07:23:38.366712+0000","last_undegraded":"2024-09-16T07:23:38.366712+0000","last_fullsized":"2024-09-16T07:23:38.366712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:36.809629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.331286+0000","last_change":"2024-09-16T07:23:37.331286+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.331286+0000","last_undegraded":"2024-09-16T07:23:37.331286+0000","last_fullsized":"2024-09-16T07:23:37.331286+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.374468+0000","last_change":"2024-09-16T07:23:37.345215+0000","last_active":"2024-09-16T07:23:38.374468+0000","last_peered":"2024-09-16T07:23:38.374468+0000","last_clean":"2024-09-16T07:23:38.374468+0000","last_became_active":"2024-09-16T07:23:37.344006+0000","last_became_peered":"2024-09-16T07:23:37.344006+0000","last_unstale":"2024-09-16T07:23:38.374468+0000","last_undegraded":"2024-09-16T07:23:38.374468+0000","last_fullsized":"2024-09-16T07:23:38.374468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:43.819327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334029+0000","last_change":"2024-09-16T07:23:37.334029+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334029+0000","last_undegraded":"2024-09-16T07:23:37.334029+0000","last_fullsized":"2024-09-16T07:23:37.334029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.330726+0000","last_change":"2024-09-16T07:23:37.353056+0000","last_active":"2024-09-16T07:23:38.330726+0000","last_peered":"2024-09-16T07:23:38.330726+0000","last_clean":"2024-09-16T07:23:38.330726+0000","last_became_active":"2024-09-16T07:23:37.352723+0000","last_became_peered":"2024-09-16T07:23:37.352723+0000","last_unstale":"2024-09-16T07:23:38.330726+0000","last_undegraded":"2024-09-16T07:23:38.330726+0000","last_fullsized":"2024-09-16T07:23:38.330726+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:24.951401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334406+0000","last_change":"2024-09-16T07:23:37.334406+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334406+0000","last_undegraded":"2024-09-16T07:23:37.334406+0000","last_fullsized":"2024-09-16T07:23:37.334406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350390+0000","last_change":"2024-09-16T07:23:37.347166+0000","last_active":"2024-09-16T07:23:37.350390+0000","last_peered":"2024-09-16T07:23:37.350390+0000","last_clean":"2024-09-16T07:23:37.350390+0000","last_became_active":"2024-09-16T07:23:37.346890+0000","last_became_peered":"2024-09-16T07:23:37.346890+0000","last_unstale":"2024-09-16T07:23:37.350390+0000","last_undegraded":"2024-09-16T07:23:37.350390+0000","last_fullsized":"2024-09-16T07:23:37.350390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:09.516212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.330484+0000","last_change":"2024-09-16T07:23:37.330484+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.330484+0000","last_undegraded":"2024-09-16T07:23:37.330484+0000","last_fullsized":"2024-09-16T07:23:37.330484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.336973+0000","last_change":"2024-09-16T07:23:37.349623+0000","last_active":"2024-09-16T07:23:38.336973+0000","last_peered":"2024-09-16T07:23:38.336973+0000","last_clean":"2024-09-16T07:23:38.336973+0000","last_became_active":"2024-09-16T07:23:37.349374+0000","last_became_peered":"2024-09-16T07:23:37.349374+0000","last_unstale":"2024-09-16T07:23:38.336973+0000","last_undegraded":"2024-09-16T07:23:38.336973+0000","last_fullsized":"2024-09-16T07:23:38.336973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:46.136303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.336554+0000","last_change":"2024-09-16T07:23:37.336554+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.336554+0000","last_undegraded":"2024-09-16T07:23:37.336554+0000","last_fullsized":"2024-09-16T07:23:37.336554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.343402+0000","last_change":"2024-09-16T07:23:37.347390+0000","last_active":"2024-09-16T07:23:38.343402+0000","last_peered":"2024-09-16T07:23:38.343402+0000","last_clean":"2024-09-16T07:23:38.343402+0000","last_became_active":"2024-09-16T07:23:37.346226+0000","last_became_peered":"2024-09-16T07:23:37.346226+0000","last_unstale":"2024-09-16T07:23:38.343402+0000","last_undegraded":"2024-09-16T07:23:38.343402+0000","last_fullsized":"2024-09-16T07:23:38.343402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:54.186625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359489+0000","last_change":"2024-09-16T07:23:38.353841+0000","last_active":"2024-09-16T07:23:38.359489+0000","last_peered":"2024-09-16T07:23:38.359489+0000","last_clean":"2024-09-16T07:23:38.359489+0000","last_became_active":"2024-09-16T07:23:38.352266+0000","last_became_peered":"2024-09-16T07:23:38.352266+0000","last_unstale":"2024-09-16T07:23:38.359489+0000","last_undegraded":"2024-09-16T07:23:38.359489+0000","last_fullsized":"2024-09-16T07:23:38.359489+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:01.093082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.334377+0000","last_change":"2024-09-16T07:23:37.339731+0000","last_active":"2024-09-16T07:23:38.334377+0000","last_peered":"2024-09-16T07:23:38.334377+0000","last_clean":"2024-09-16T07:23:38.334377+0000","last_became_active":"2024-09-16T07:23:37.339480+0000","last_became_peered":"2024-09-16T07:23:37.339480+0000","last_unstale":"2024-09-16T07:23:38.334377+0000","last_undegraded":"2024-09-16T07:23:38.334377+0000","last_fullsized":"2024-09-16T07:23:38.334377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:58.263118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.337719+0000","last_change":"2024-09-16T07:23:37.337719+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.337719+0000","last_undegraded":"2024-09-16T07:23:37.337719+0000","last_fullsized":"2024-09-16T07:23:37.337719+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346812+0000","last_change":"2024-09-16T07:23:37.344100+0000","last_active":"2024-09-16T07:23:37.346812+0000","last_peered":"2024-09-16T07:23:37.346812+0000","last_clean":"2024-09-16T07:23:37.346812+0000","last_became_active":"2024-09-16T07:23:37.343269+0000","last_became_peered":"2024-09-16T07:23:37.343269+0000","last_unstale":"2024-09-16T07:23:37.346812+0000","last_undegraded":"2024-09-16T07:23:37.346812+0000","last_fullsized":"2024-09-16T07:23:37.346812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:39.795699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362195+0000","last_change":"2024-09-16T07:23:38.354226+0000","last_active":"2024-09-16T07:23:38.362195+0000","last_peered":"2024-09-16T07:23:38.362195+0000","last_clean":"2024-09-16T07:23:38.362195+0000","last_became_active":"2024-09-16T07:23:38.354001+0000","last_became_peered":"2024-09-16T07:23:38.354001+0000","last_unstale":"2024-09-16T07:23:38.362195+0000","last_undegraded":"2024-09-16T07:23:38.362195+0000","last_fullsized":"2024-09-16T07:23:38.362195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:16.310897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.345689+0000","last_change":"2024-09-16T07:23:37.343487+0000","last_active":"2024-09-16T07:23:37.345689+0000","last_peered":"2024-09-16T07:23:37.345689+0000","last_clean":"2024-09-16T07:23:37.345689+0000","last_became_active":"2024-09-16T07:23:37.343170+0000","last_became_peered":"2024-09-16T07:23:37.343170+0000","last_unstale":"2024-09-16T07:23:37.345689+0000","last_undegraded":"2024-09-16T07:23:37.345689+0000","last_fullsized":"2024-09-16T07:23:37.345689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:10.785300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333534+0000","last_change":"2024-09-16T07:23:37.333534+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333534+0000","last_undegraded":"2024-09-16T07:23:37.333534+0000","last_fullsized":"2024-09-16T07:23:37.333534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.348747+0000","last_change":"2024-09-16T07:23:37.346738+0000","last_active":"2024-09-16T07:23:37.348747+0000","last_peered":"2024-09-16T07:23:37.348747+0000","last_clean":"2024-09-16T07:23:37.348747+0000","last_became_active":"2024-09-16T07:23:37.345610+0000","last_became_peered":"2024-09-16T07:23:37.345610+0000","last_unstale":"2024-09-16T07:23:37.348747+0000","last_undegraded":"2024-09-16T07:23:37.348747+0000","last_fullsized":"2024-09-16T07:23:37.348747+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:45.870123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362706+0000","last_change":"2024-09-16T07:23:38.358556+0000","last_active":"2024-09-16T07:23:38.362706+0000","last_peered":"2024-09-16T07:23:38.362706+0000","last_clean":"2024-09-16T07:23:38.362706+0000","last_became_active":"2024-09-16T07:23:38.358288+0000","last_became_peered":"2024-09-16T07:23:38.358288+0000","last_unstale":"2024-09-16T07:23:38.362706+0000","last_undegraded":"2024-09-16T07:23:38.362706+0000","last_fullsized":"2024-09-16T07:23:38.362706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:34.849304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346559+0000","last_change":"2024-09-16T07:23:37.343946+0000","last_active":"2024-09-16T07:23:37.346559+0000","last_peered":"2024-09-16T07:23:37.346559+0000","last_clean":"2024-09-16T07:23:37.346559+0000","last_became_active":"2024-09-16T07:23:37.343628+0000","last_became_peered":"2024-09-16T07:23:37.343628+0000","last_unstale":"2024-09-16T07:23:37.346559+0000","last_undegraded":"2024-09-16T07:23:37.346559+0000","last_fullsized":"2024-09-16T07:23:37.346559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:00.089436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615243+0000","last_change":"2024-09-16T07:23:38.346303+0000","last_active":"2024-09-16T07:23:40.615243+0000","last_peered":"2024-09-16T07:23:40.615243+0000","last_clean":"2024-09-16T07:23:40.615243+0000","last_became_active":"2024-09-16T07:23:38.346039+0000","last_became_peered":"2024-09-16T07:23:38.346039+0000","last_unstale":"2024-09-16T07:23:40.615243+0000","last_undegraded":"2024-09-16T07:23:40.615243+0000","last_fullsized":"2024-09-16T07:23:40.615243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:47.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.342710+0000","last_change":"2024-09-16T07:23:37.340258+0000","last_active":"2024-09-16T07:23:37.342710+0000","last_peered":"2024-09-16T07:23:37.342710+0000","last_clean":"2024-09-16T07:23:37.342710+0000","last_became_active":"2024-09-16T07:23:37.339993+0000","last_became_peered":"2024-09-16T07:23:37.339993+0000","last_unstale":"2024-09-16T07:23:37.342710+0000","last_undegraded":"2024-09-16T07:23:37.342710+0000","last_fullsized":"2024-09-16T07:23:37.342710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:29.784689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614910+0000","last_change":"2024-09-16T07:23:38.342446+0000","last_active":"2024-09-16T07:23:40.614910+0000","last_peered":"2024-09-16T07:23:40.614910+0000","last_clean":"2024-09-16T07:23:40.614910+0000","last_became_active":"2024-09-16T07:23:38.341488+0000","last_became_peered":"2024-09-16T07:23:38.341488+0000","last_unstale":"2024-09-16T07:23:40.614910+0000","last_undegraded":"2024-09-16T07:23:40.614910+0000","last_fullsized":"2024-09-16T07:23:40.614910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:51.750252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.336765+0000","last_change":"2024-09-16T07:23:37.348185+0000","last_active":"2024-09-16T07:23:38.336765+0000","last_peered":"2024-09-16T07:23:38.336765+0000","last_clean":"2024-09-16T07:23:38.336765+0000","last_became_active":"2024-09-16T07:23:37.347959+0000","last_became_peered":"2024-09-16T07:23:37.347959+0000","last_unstale":"2024-09-16T07:23:38.336765+0000","last_undegraded":"2024-09-16T07:23:38.336765+0000","last_fullsized":"2024-09-16T07:23:38.336765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:21:54.247700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362963+0000","last_change":"2024-09-16T07:23:38.355905+0000","last_active":"2024-09-16T07:23:38.362963+0000","last_peered":"2024-09-16T07:23:38.362963+0000","last_clean":"2024-09-16T07:23:38.362963+0000","last_became_active":"2024-09-16T07:23:38.355546+0000","last_became_peered":"2024-09-16T07:23:38.355546+0000","last_unstale":"2024-09-16T07:23:38.362963+0000","last_undegraded":"2024-09-16T07:23:38.362963+0000","last_fullsized":"2024-09-16T07:23:38.362963+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:42.332816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614759+0000","last_change":"2024-09-16T07:23:37.345122+0000","last_active":"2024-09-16T07:23:40.614759+0000","last_peered":"2024-09-16T07:23:40.614759+0000","last_clean":"2024-09-16T07:23:40.614759+0000","last_became_active":"2024-09-16T07:23:37.344793+0000","last_became_peered":"2024-09-16T07:23:37.344793+0000","last_unstale":"2024-09-16T07:23:40.614759+0000","last_undegraded":"2024-09-16T07:23:40.614759+0000","last_fullsized":"2024-09-16T07:23:40.614759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:56.809362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362575+0000","last_change":"2024-09-16T07:23:38.358386+0000","last_active":"2024-09-16T07:23:38.362575+0000","last_peered":"2024-09-16T07:23:38.362575+0000","last_clean":"2024-09-16T07:23:38.362575+0000","last_became_active":"2024-09-16T07:23:38.358092+0000","last_became_peered":"2024-09-16T07:23:38.358092+0000","last_unstale":"2024-09-16T07:23:38.362575+0000","last_undegraded":"2024-09-16T07:23:38.362575+0000","last_fullsized":"2024-09-16T07:23:38.362575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:58:30.618313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.338916+0000","last_change":"2024-09-16T07:23:37.345296+0000","last_active":"2024-09-16T07:23:38.338916+0000","last_peered":"2024-09-16T07:23:38.338916+0000","last_clean":"2024-09-16T07:23:38.338916+0000","last_became_active":"2024-09-16T07:23:37.343905+0000","last_became_peered":"2024-09-16T07:23:37.343905+0000","last_unstale":"2024-09-16T07:23:38.338916+0000","last_undegraded":"2024-09-16T07:23:38.338916+0000","last_fullsized":"2024-09-16T07:23:38.338916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:43.228036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":130,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.338930+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:23:38.338930+0000","last_peered":"2024-09-16T07:23:38.338930+0000","last_clean":"2024-09-16T07:23:38.338930+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:23:38.338930+0000","last_undegraded":"2024-09-16T07:23:38.338930+0000","last_fullsized":"2024-09-16T07:23:38.338930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476753,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476753,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":727307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":130664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":721032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":151648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:23:43.447 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.334422+0000', 'last_change': '2024-09-16T07:23:37.334422+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.334422+0000', 'last_undegraded': '2024-09-16T07:23:37.334422+0000', 'last_fullsized': '2024-09-16T07:23:37.334422+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.331162+0000', 'last_change': '2024-09-16T07:23:37.331162+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.331162+0000', 'last_undegraded': '2024-09-16T07:23:37.331162+0000', 'last_fullsized': '2024-09-16T07:23:37.331162+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3a is not active+clean 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.334158+0000', 'last_change': '2024-09-16T07:23:37.334158+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.334158+0000', 'last_undegraded': '2024-09-16T07:23:37.334158+0000', 'last_fullsized': '2024-09-16T07:23:37.334158+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:PG 5.33 is not active+clean 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.335382+0000', 'last_change': '2024-09-16T07:23:37.335382+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.335382+0000', 'last_undegraded': '2024-09-16T07:23:37.335382+0000', 'last_fullsized': '2024-09-16T07:23:37.335382+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2e is not active+clean 2024-09-16T07:23:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.338226+0000', 'last_change': '2024-09-16T07:23:37.338226+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.338226+0000', 'last_undegraded': '2024-09-16T07:23:37.338226+0000', 'last_fullsized': '2024-09-16T07:23:37.338226+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2d is not active+clean 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.334511+0000', 'last_change': '2024-09-16T07:23:37.334511+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.334511+0000', 'last_undegraded': '2024-09-16T07:23:37.334511+0000', 'last_fullsized': '2024-09-16T07:23:37.334511+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2c is not active+clean 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.337607+0000', 'last_change': '2024-09-16T07:23:37.337607+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.337607+0000', 'last_undegraded': '2024-09-16T07:23:37.337607+0000', 'last_fullsized': '2024-09-16T07:23:37.337607+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 5.29 is not active+clean 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.334034+0000', 'last_change': '2024-09-16T07:23:37.334034+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.334034+0000', 'last_undegraded': '2024-09-16T07:23:37.334034+0000', 'last_fullsized': '2024-09-16T07:23:37.334034+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 5.25 is not active+clean 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.330534+0000', 'last_change': '2024-09-16T07:23:37.330534+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.330534+0000', 'last_undegraded': '2024-09-16T07:23:37.330534+0000', 'last_fullsized': '2024-09-16T07:23:37.330534+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-16T07:23:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.333820+0000', 'last_change': '2024-09-16T07:23:37.333820+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.333820+0000', 'last_undegraded': '2024-09-16T07:23:37.333820+0000', 'last_fullsized': '2024-09-16T07:23:37.333820+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 5.12 is not active+clean 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.333905+0000', 'last_change': '2024-09-16T07:23:37.333905+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.333905+0000', 'last_undegraded': '2024-09-16T07:23:37.333905+0000', 'last_fullsized': '2024-09-16T07:23:37.333905+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.331286+0000', 'last_change': '2024-09-16T07:23:37.331286+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.331286+0000', 'last_undegraded': '2024-09-16T07:23:37.331286+0000', 'last_fullsized': '2024-09-16T07:23:37.331286+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.334029+0000', 'last_change': '2024-09-16T07:23:37.334029+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.334029+0000', 'last_undegraded': '2024-09-16T07:23:37.334029+0000', 'last_fullsized': '2024-09-16T07:23:37.334029+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.334406+0000', 'last_change': '2024-09-16T07:23:37.334406+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.334406+0000', 'last_undegraded': '2024-09-16T07:23:37.334406+0000', 'last_fullsized': '2024-09-16T07:23:37.334406+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.330484+0000', 'last_change': '2024-09-16T07:23:37.330484+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.330484+0000', 'last_undegraded': '2024-09-16T07:23:37.330484+0000', 'last_fullsized': '2024-09-16T07:23:37.330484+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:23:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-09-16T07:23:43.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.336554+0000', 'last_change': '2024-09-16T07:23:37.336554+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.336554+0000', 'last_undegraded': '2024-09-16T07:23:37.336554+0000', 'last_fullsized': '2024-09-16T07:23:37.336554+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:23:43.451 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-09-16T07:23:43.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.337719+0000', 'last_change': '2024-09-16T07:23:37.337719+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.337719+0000', 'last_undegraded': '2024-09-16T07:23:37.337719+0000', 'last_fullsized': '2024-09-16T07:23:37.337719+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:23:43.451 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-16T07:23:43.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:23:37.333534+0000', 'last_change': '2024-09-16T07:23:37.333534+0000', 'last_active': '2024-09-16T07:23:37.322443+0000', 'last_peered': '2024-09-16T07:23:37.322443+0000', 'last_clean': '2024-09-16T07:23:37.322443+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:23:37.333534+0000', 'last_undegraded': '2024-09-16T07:23:37.333534+0000', 'last_fullsized': '2024-09-16T07:23:37.333534+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'last_clean_scrub_stamp': '2024-09-16T07:23:37.322443+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:23:43.452 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T07:23:43.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:23:43.501 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:23:43.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.614+0000 7f9b3e000640 1 -- 172.21.15.71:0/739654301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b380ccba0 msgr2=0x7f9b38073280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.614+0000 7f9b3e000640 1 --2- 172.21.15.71:0/739654301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b380ccba0 0x7f9b38073280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9b38070580 tx=0x7f9b3002aac0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.614+0000 7f9b3e000640 1 -- 172.21.15.71:0/739654301 shutdown_connections 2024-09-16T07:23:43.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.614+0000 7f9b3e000640 1 --2- 172.21.15.71:0/739654301 >> v2:172.21.15.177:3300/0 conn(0x7f9b3806e9b0 0x7f9b380cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.614+0000 7f9b3e000640 1 --2- 172.21.15.71:0/739654301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b380ccba0 0x7f9b38073280 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.614+0000 7f9b3e000640 1 -- 172.21.15.71:0/739654301 >> 172.21.15.71:0/739654301 conn(0x7f9b380b18e0 msgr2=0x7f9b380b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:43.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.614+0000 7f9b3e000640 1 -- 172.21.15.71:0/739654301 shutdown_connections 2024-09-16T07:23:43.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.614+0000 7f9b3e000640 1 -- 172.21.15.71:0/739654301 wait complete. 2024-09-16T07:23:43.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.616+0000 7f9b3e000640 1 Processor -- start 2024-09-16T07:23:43.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.616+0000 7f9b3e000640 1 -- start start 2024-09-16T07:23:43.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b3806e9b0 0x7f9b3814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:43.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3e000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b380ccba0 0x7f9b380d0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:43.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3e000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b381563a0 0x7f9b381e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3e000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38073bd0 con 0x7f9b3806e9b0 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3e000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38073a50 con 0x7f9b381563a0 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3e000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9b38073d50 con 0x7f9b380ccba0 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b3806e9b0 0x7f9b3814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b3806e9b0 0x7f9b3814fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44974/0 (socket says 172.21.15.71:44974) 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3cc00640 1 -- 172.21.15.71:0/1282429773 learned_addr learned my addr 172.21.15.71:0/1282429773 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3d600640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b381563a0 0x7f9b381e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b37e00640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b380ccba0 0x7f9b380d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3cc00640 1 -- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b380ccba0 msgr2=0x7f9b380d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3cc00640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b380ccba0 0x7f9b380d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3cc00640 1 -- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b381563a0 msgr2=0x7f9b381e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3cc00640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b381563a0 0x7f9b381e8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3cc00640 1 -- 172.21.15.71:0/1282429773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b380ced60 con 0x7f9b3806e9b0 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b3d600640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b381563a0 0x7f9b381e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.617+0000 7f9b37e00640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b380ccba0 0x7f9b380d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.618+0000 7f9b3cc00640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b3806e9b0 0x7f9b3814fef0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800c3d0 tx=0x7f9b2800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.618+0000 7f9b34c00640 1 -- 172.21.15.71:0/1282429773 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28017070 con 0x7f9b3806e9b0 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.618+0000 7f9b34c00640 1 -- 172.21.15.71:0/1282429773 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b280053c0 con 0x7f9b3806e9b0 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.618+0000 7f9b34c00640 1 -- 172.21.15.71:0/1282429773 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28003850 con 0x7f9b3806e9b0 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.618+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b380cf040 con 0x7f9b3806e9b0 2024-09-16T07:23:43.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.618+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b380cf600 con 0x7f9b3806e9b0 2024-09-16T07:23:43.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.619+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b380cc660 con 0x7f9b3806e9b0 2024-09-16T07:23:43.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.620+0000 7f9b34c00640 1 -- 172.21.15.71:0/1282429773 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9b2801a070 con 0x7f9b3806e9b0 2024-09-16T07:23:43.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.621+0000 7f9b34c00640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b18078850 0x7f9b1807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:43.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.621+0000 7f9b34c00640 1 -- 172.21.15.71:0/1282429773 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9b28099d90 con 0x7f9b3806e9b0 2024-09-16T07:23:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.621+0000 7f9b37e00640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b18078850 0x7f9b1807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.622+0000 7f9b37e00640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b18078850 0x7f9b1807ad10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b381f7d50 tx=0x7f9b24005e50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:43.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.625+0000 7f9b34c00640 1 -- 172.21.15.71:0/1282429773 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9b28062360 con 0x7f9b3806e9b0 2024-09-16T07:23:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.883+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9b38070ae0 con 0x7f9b3806e9b0 2024-09-16T07:23:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.884+0000 7f9b34c00640 1 -- 172.21.15.71:0/1282429773 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1688 (secure 0 0 0) 0x7f9b28061c60 con 0x7f9b3806e9b0 2024-09-16T07:23:43.884 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:43.884 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":80,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1726471357,"num_in_osds":8,"osd_in_since":1726471347,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":79},{"state_name":"creating+peering","count":18}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":591144,"bytes_used":229785600,"bytes_avail":772864327680,"bytes_total":773094113280,"inactive_pgs_ratio":0.18556700646877289,"write_bytes_sec":1712,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":17,"btime":"2024-09-16T07:23:41:354934+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":4917}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-16T07:23:40.304739+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T07:23:43.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b18078850 msgr2=0x7f9b1807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b18078850 0x7f9b1807ad10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b381f7d50 tx=0x7f9b24005e50 comp rx=0 tx=0).stop 2024-09-16T07:23:43.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b3806e9b0 msgr2=0x7f9b3814fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:43.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b3806e9b0 0x7f9b3814fef0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800c3d0 tx=0x7f9b2800c890 comp rx=0 tx=0).stop 2024-09-16T07:23:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 shutdown_connections 2024-09-16T07:23:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b18078850 0x7f9b1807ad10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b381563a0 0x7f9b381e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b380ccba0 0x7f9b380d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 --2- 172.21.15.71:0/1282429773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b3806e9b0 0x7f9b3814fef0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 >> 172.21.15.71:0/1282429773 conn(0x7f9b380b18e0 msgr2=0x7f9b3815cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 shutdown_connections 2024-09-16T07:23:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:43.886+0000 7f9b3e000640 1 -- 172.21.15.71:0/1282429773 wait complete. 2024-09-16T07:23:43.901 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T07:23:43.901 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:23:44.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.099+0000 7f6073a00640 1 -- 172.21.15.71:0/248254738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6050004d40 msgr2=0x7f60500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:44.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.099+0000 7f6073a00640 1 --2- 172.21.15.71:0/248254738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6050004d40 0x7f60500051c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6064009b50 tx=0x7f6050005830 comp rx=0 tx=0).stop 2024-09-16T07:23:44.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.099+0000 7f6071a00640 1 -- 172.21.15.71:0/248254738 reap_dead start 2024-09-16T07:23:44.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.099+0000 7f6073a00640 1 -- 172.21.15.71:0/248254738 shutdown_connections 2024-09-16T07:23:44.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.099+0000 7f6073a00640 1 -- 172.21.15.71:0/248254738 >> 172.21.15.71:0/248254738 conn(0x7f606c0b1950 msgr2=0x7f606c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:44.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.099+0000 7f6073a00640 1 -- 172.21.15.71:0/248254738 shutdown_connections 2024-09-16T07:23:44.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.099+0000 7f6073a00640 1 -- 172.21.15.71:0/248254738 wait complete. 2024-09-16T07:23:44.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.101+0000 7f6073a00640 1 Processor -- start 2024-09-16T07:23:44.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.101+0000 7f6073a00640 1 -- start start 2024-09-16T07:23:44.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f6073a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6050004d40 0x7f606c0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f6073a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c09b730 0x7f606c15a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f6073a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c13f3d0 0x7f606c0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f6073a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f606c072ee0 con 0x7f606c09b730 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f6073a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f606c0b4eb0 con 0x7f606c13f3d0 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f6073a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f606c073060 con 0x7f6050004d40 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f606be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c09b730 0x7f606c15a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f606be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c09b730 0x7f606c15a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44996/0 (socket says 172.21.15.71:44996) 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f606be00640 1 -- 172.21.15.71:0/3178276968 learned_addr learned my addr 172.21.15.71:0/3178276968 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f6071a00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c13f3d0 0x7f606c0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:44.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.102+0000 7f6071000640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6050004d40 0x7f606c0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f606be00640 1 -- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6050004d40 msgr2=0x7f606c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f606be00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6050004d40 0x7f606c0cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f606be00640 1 -- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c13f3d0 msgr2=0x7f606c0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f606be00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c13f3d0 0x7f606c0c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f606be00640 1 -- 172.21.15.71:0/3178276968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c13fb40 con 0x7f606c09b730 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f606be00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c09b730 0x7f606c15a7f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6058011ab0 tx=0x7f6058011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f6068c00640 1 -- 172.21.15.71:0/3178276968 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605801e070 con 0x7f606c09b730 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f6068c00640 1 -- 172.21.15.71:0/3178276968 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f605800fb50 con 0x7f606c09b730 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c0cc4b0 con 0x7f606c09b730 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f6068c00640 1 -- 172.21.15.71:0/3178276968 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058019410 con 0x7f606c09b730 2024-09-16T07:23:44.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.103+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c09af30 con 0x7f606c09b730 2024-09-16T07:23:44.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.104+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c0729a0 con 0x7f606c09b730 2024-09-16T07:23:44.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.106+0000 7f6068c00640 1 -- 172.21.15.71:0/3178276968 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6058021070 con 0x7f606c09b730 2024-09-16T07:23:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.106+0000 7f6068c00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f604c078880 0x7f604c07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.107+0000 7f6068c00640 1 -- 172.21.15.71:0/3178276968 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f605809b040 con 0x7f606c09b730 2024-09-16T07:23:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.107+0000 7f6071000640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f604c078880 0x7f604c07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:44.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.108+0000 7f6071000640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f604c078880 0x7f604c07ad40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f605c001be0 tx=0x7f605c006040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:44.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.111+0000 7f6068c00640 1 -- 172.21.15.71:0/3178276968 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6058062fd0 con 0x7f606c09b730 2024-09-16T07:23:44.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.291+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f606c070a20 con 0x7f604c078880 2024-09-16T07:23:44.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.295+0000 7f6068c00640 1 -- 172.21.15.71:0/3178276968 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250440 (secure 0 0 0) 0x7f606c070a20 con 0x7f604c078880 2024-09-16T07:23:44.298 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:44.299 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:23:44.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f604c078880 msgr2=0x7f604c07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f604c078880 0x7f604c07ad40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f605c001be0 tx=0x7f605c006040 comp rx=0 tx=0).stop 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c09b730 msgr2=0x7f606c15a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c09b730 0x7f606c15a7f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6058011ab0 tx=0x7f6058011eb0 comp rx=0 tx=0).stop 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 shutdown_connections 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f604c078880 0x7f604c07ad40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c13f3d0 0x7f606c0c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c09b730 0x7f606c15a7f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 --2- 172.21.15.71:0/3178276968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6050004d40 0x7f606c0cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.299+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 >> 172.21.15.71:0/3178276968 conn(0x7f606c0b1950 msgr2=0x7f606c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.300+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 shutdown_connections 2024-09-16T07:23:44.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:44.300+0000 7f6073a00640 1 -- 172.21.15.71:0/3178276968 wait complete. 2024-09-16T07:23:44.315 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-16T07:23:42.305320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":119,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224400,"kb_used_data":3992,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750320,"statfs":{"total":773094113280,"available":772864327680,"internally_reserved":0,"allocated":4087808,"data_stored":2278407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.979386"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350368+0000","last_change":"2024-09-16T07:23:38.347812+0000","last_active":"2024-09-16T07:23:38.350368+0000","last_peered":"2024-09-16T07:23:38.350368+0000","last_clean":"2024-09-16T07:23:38.350368+0000","last_became_active":"2024-09-16T07:23:38.347516+0000","last_became_peered":"2024-09-16T07:23:38.347516+0000","last_unstale":"2024-09-16T07:23:38.350368+0000","last_undegraded":"2024-09-16T07:23:38.350368+0000","last_fullsized":"2024-09-16T07:23:38.350368+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:06.049071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334422+0000","last_change":"2024-09-16T07:23:37.334422+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334422+0000","last_undegraded":"2024-09-16T07:23:37.334422+0000","last_fullsized":"2024-09-16T07:23:37.334422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.360416+0000","last_change":"2024-09-16T07:23:38.352895+0000","last_active":"2024-09-16T07:23:38.360416+0000","last_peered":"2024-09-16T07:23:38.360416+0000","last_clean":"2024-09-16T07:23:38.360416+0000","last_became_active":"2024-09-16T07:23:38.352621+0000","last_became_peered":"2024-09-16T07:23:38.352621+0000","last_unstale":"2024-09-16T07:23:38.360416+0000","last_undegraded":"2024-09-16T07:23:38.360416+0000","last_fullsized":"2024-09-16T07:23:38.360416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:14.142804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615292+0000","last_change":"2024-09-16T07:23:38.350692+0000","last_active":"2024-09-16T07:23:40.615292+0000","last_peered":"2024-09-16T07:23:40.615292+0000","last_clean":"2024-09-16T07:23:40.615292+0000","last_became_active":"2024-09-16T07:23:38.347475+0000","last_became_peered":"2024-09-16T07:23:38.347475+0000","last_unstale":"2024-09-16T07:23:40.615292+0000","last_undegraded":"2024-09-16T07:23:40.615292+0000","last_fullsized":"2024-09-16T07:23:40.615292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:01.740131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.331162+0000","last_change":"2024-09-16T07:23:37.331162+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.331162+0000","last_undegraded":"2024-09-16T07:23:37.331162+0000","last_fullsized":"2024-09-16T07:23:37.331162+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334158+0000","last_change":"2024-09-16T07:23:37.334158+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334158+0000","last_undegraded":"2024-09-16T07:23:37.334158+0000","last_fullsized":"2024-09-16T07:23:37.334158+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.368246+0000","last_change":"2024-09-16T07:23:38.367300+0000","last_active":"2024-09-16T07:23:38.368246+0000","last_peered":"2024-09-16T07:23:38.368246+0000","last_clean":"2024-09-16T07:23:38.368246+0000","last_became_active":"2024-09-16T07:23:38.365509+0000","last_became_peered":"2024-09-16T07:23:38.365509+0000","last_unstale":"2024-09-16T07:23:38.368246+0000","last_undegraded":"2024-09-16T07:23:38.368246+0000","last_fullsized":"2024-09-16T07:23:38.368246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:19.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.358930+0000","last_change":"2024-09-16T07:23:38.357105+0000","last_active":"2024-09-16T07:23:38.358930+0000","last_peered":"2024-09-16T07:23:38.358930+0000","last_clean":"2024-09-16T07:23:38.358930+0000","last_became_active":"2024-09-16T07:23:38.355543+0000","last_became_peered":"2024-09-16T07:23:38.355543+0000","last_unstale":"2024-09-16T07:23:38.358930+0000","last_undegraded":"2024-09-16T07:23:38.358930+0000","last_fullsized":"2024-09-16T07:23:38.358930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:21.370546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359316+0000","last_change":"2024-09-16T07:23:38.356452+0000","last_active":"2024-09-16T07:23:38.359316+0000","last_peered":"2024-09-16T07:23:38.359316+0000","last_clean":"2024-09-16T07:23:38.359316+0000","last_became_active":"2024-09-16T07:23:38.355663+0000","last_became_peered":"2024-09-16T07:23:38.355663+0000","last_unstale":"2024-09-16T07:23:38.359316+0000","last_undegraded":"2024-09-16T07:23:38.359316+0000","last_fullsized":"2024-09-16T07:23:38.359316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:30.788335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615484+0000","last_change":"2024-09-16T07:23:38.356446+0000","last_active":"2024-09-16T07:23:40.615484+0000","last_peered":"2024-09-16T07:23:40.615484+0000","last_clean":"2024-09-16T07:23:40.615484+0000","last_became_active":"2024-09-16T07:23:38.354502+0000","last_became_peered":"2024-09-16T07:23:38.354502+0000","last_unstale":"2024-09-16T07:23:40.615484+0000","last_undegraded":"2024-09-16T07:23:40.615484+0000","last_fullsized":"2024-09-16T07:23:40.615484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:27.342707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352817+0000","last_change":"2024-09-16T07:23:38.351223+0000","last_active":"2024-09-16T07:23:38.352817+0000","last_peered":"2024-09-16T07:23:38.352817+0000","last_clean":"2024-09-16T07:23:38.352817+0000","last_became_active":"2024-09-16T07:23:38.349810+0000","last_became_peered":"2024-09-16T07:23:38.349810+0000","last_unstale":"2024-09-16T07:23:38.352817+0000","last_undegraded":"2024-09-16T07:23:38.352817+0000","last_fullsized":"2024-09-16T07:23:38.352817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:08:37.812192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.367826+0000","last_change":"2024-09-16T07:23:38.365322+0000","last_active":"2024-09-16T07:23:38.367826+0000","last_peered":"2024-09-16T07:23:38.367826+0000","last_clean":"2024-09-16T07:23:38.367826+0000","last_became_active":"2024-09-16T07:23:38.364995+0000","last_became_peered":"2024-09-16T07:23:38.364995+0000","last_unstale":"2024-09-16T07:23:38.367826+0000","last_undegraded":"2024-09-16T07:23:38.367826+0000","last_fullsized":"2024-09-16T07:23:38.367826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:40.799346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.335382+0000","last_change":"2024-09-16T07:23:37.335382+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.335382+0000","last_undegraded":"2024-09-16T07:23:37.335382+0000","last_fullsized":"2024-09-16T07:23:37.335382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614877+0000","last_change":"2024-09-16T07:23:38.347109+0000","last_active":"2024-09-16T07:23:40.614877+0000","last_peered":"2024-09-16T07:23:40.614877+0000","last_clean":"2024-09-16T07:23:40.614877+0000","last_became_active":"2024-09-16T07:23:38.346821+0000","last_became_peered":"2024-09-16T07:23:38.346821+0000","last_unstale":"2024-09-16T07:23:40.614877+0000","last_undegraded":"2024-09-16T07:23:40.614877+0000","last_fullsized":"2024-09-16T07:23:40.614877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:59.555576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.345227+0000","last_change":"2024-09-16T07:23:38.341149+0000","last_active":"2024-09-16T07:23:38.345227+0000","last_peered":"2024-09-16T07:23:38.345227+0000","last_clean":"2024-09-16T07:23:38.345227+0000","last_became_active":"2024-09-16T07:23:38.340794+0000","last_became_peered":"2024-09-16T07:23:38.340794+0000","last_unstale":"2024-09-16T07:23:38.345227+0000","last_undegraded":"2024-09-16T07:23:38.345227+0000","last_fullsized":"2024-09-16T07:23:38.345227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:17.507704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362540+0000","last_change":"2024-09-16T07:23:38.360203+0000","last_active":"2024-09-16T07:23:38.362540+0000","last_peered":"2024-09-16T07:23:38.362540+0000","last_clean":"2024-09-16T07:23:38.362540+0000","last_became_active":"2024-09-16T07:23:38.359933+0000","last_became_peered":"2024-09-16T07:23:38.359933+0000","last_unstale":"2024-09-16T07:23:38.362540+0000","last_undegraded":"2024-09-16T07:23:38.362540+0000","last_fullsized":"2024-09-16T07:23:38.362540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:55.869414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352904+0000","last_change":"2024-09-16T07:23:38.350866+0000","last_active":"2024-09-16T07:23:38.352904+0000","last_peered":"2024-09-16T07:23:38.352904+0000","last_clean":"2024-09-16T07:23:38.352904+0000","last_became_active":"2024-09-16T07:23:38.350617+0000","last_became_peered":"2024-09-16T07:23:38.350617+0000","last_unstale":"2024-09-16T07:23:38.352904+0000","last_undegraded":"2024-09-16T07:23:38.352904+0000","last_fullsized":"2024-09-16T07:23:38.352904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:14:27.853553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.338226+0000","last_change":"2024-09-16T07:23:37.338226+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.338226+0000","last_undegraded":"2024-09-16T07:23:37.338226+0000","last_fullsized":"2024-09-16T07:23:37.338226+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334511+0000","last_change":"2024-09-16T07:23:37.334511+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334511+0000","last_undegraded":"2024-09-16T07:23:37.334511+0000","last_fullsized":"2024-09-16T07:23:37.334511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.337607+0000","last_change":"2024-09-16T07:23:37.337607+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.337607+0000","last_undegraded":"2024-09-16T07:23:37.337607+0000","last_fullsized":"2024-09-16T07:23:37.337607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.357360+0000","last_change":"2024-09-16T07:23:38.355303+0000","last_active":"2024-09-16T07:23:38.357360+0000","last_peered":"2024-09-16T07:23:38.357360+0000","last_clean":"2024-09-16T07:23:38.357360+0000","last_became_active":"2024-09-16T07:23:38.355033+0000","last_became_peered":"2024-09-16T07:23:38.355033+0000","last_unstale":"2024-09-16T07:23:38.357360+0000","last_undegraded":"2024-09-16T07:23:38.357360+0000","last_fullsized":"2024-09-16T07:23:38.357360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:19.995317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350313+0000","last_change":"2024-09-16T07:23:38.345255+0000","last_active":"2024-09-16T07:23:38.350313+0000","last_peered":"2024-09-16T07:23:38.350313+0000","last_clean":"2024-09-16T07:23:38.350313+0000","last_became_active":"2024-09-16T07:23:38.344965+0000","last_became_peered":"2024-09-16T07:23:38.344965+0000","last_unstale":"2024-09-16T07:23:38.350313+0000","last_undegraded":"2024-09-16T07:23:38.350313+0000","last_fullsized":"2024-09-16T07:23:38.350313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:09.790205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334034+0000","last_change":"2024-09-16T07:23:37.334034+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334034+0000","last_undegraded":"2024-09-16T07:23:37.334034+0000","last_fullsized":"2024-09-16T07:23:37.334034+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352659+0000","last_change":"2024-09-16T07:23:38.348792+0000","last_active":"2024-09-16T07:23:38.352659+0000","last_peered":"2024-09-16T07:23:38.352659+0000","last_clean":"2024-09-16T07:23:38.352659+0000","last_became_active":"2024-09-16T07:23:38.345353+0000","last_became_peered":"2024-09-16T07:23:38.345353+0000","last_unstale":"2024-09-16T07:23:38.352659+0000","last_undegraded":"2024-09-16T07:23:38.352659+0000","last_fullsized":"2024-09-16T07:23:38.352659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:32.489176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615553+0000","last_change":"2024-09-16T07:23:38.351284+0000","last_active":"2024-09-16T07:23:40.615553+0000","last_peered":"2024-09-16T07:23:40.615553+0000","last_clean":"2024-09-16T07:23:40.615553+0000","last_became_active":"2024-09-16T07:23:38.350972+0000","last_became_peered":"2024-09-16T07:23:38.350972+0000","last_unstale":"2024-09-16T07:23:40.615553+0000","last_undegraded":"2024-09-16T07:23:40.615553+0000","last_fullsized":"2024-09-16T07:23:40.615553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:25.493457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615534+0000","last_change":"2024-09-16T07:23:38.356598+0000","last_active":"2024-09-16T07:23:40.615534+0000","last_peered":"2024-09-16T07:23:40.615534+0000","last_clean":"2024-09-16T07:23:40.615534+0000","last_became_active":"2024-09-16T07:23:38.356177+0000","last_became_peered":"2024-09-16T07:23:38.356177+0000","last_unstale":"2024-09-16T07:23:40.615534+0000","last_undegraded":"2024-09-16T07:23:40.615534+0000","last_fullsized":"2024-09-16T07:23:40.615534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:42.647875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.330534+0000","last_change":"2024-09-16T07:23:37.330534+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.330534+0000","last_undegraded":"2024-09-16T07:23:37.330534+0000","last_fullsized":"2024-09-16T07:23:37.330534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.364143+0000","last_change":"2024-09-16T07:23:38.362005+0000","last_active":"2024-09-16T07:23:38.364143+0000","last_peered":"2024-09-16T07:23:38.364143+0000","last_clean":"2024-09-16T07:23:38.364143+0000","last_became_active":"2024-09-16T07:23:38.361748+0000","last_became_peered":"2024-09-16T07:23:38.361748+0000","last_unstale":"2024-09-16T07:23:38.364143+0000","last_undegraded":"2024-09-16T07:23:38.364143+0000","last_fullsized":"2024-09-16T07:23:38.364143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:17.507704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.343486+0000","last_change":"2024-09-16T07:23:38.337747+0000","last_active":"2024-09-16T07:23:38.343486+0000","last_peered":"2024-09-16T07:23:38.343486+0000","last_clean":"2024-09-16T07:23:38.343486+0000","last_became_active":"2024-09-16T07:23:38.337455+0000","last_became_peered":"2024-09-16T07:23:38.337455+0000","last_unstale":"2024-09-16T07:23:38.343486+0000","last_undegraded":"2024-09-16T07:23:38.343486+0000","last_fullsized":"2024-09-16T07:23:38.343486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:21.370546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.351724+0000","last_change":"2024-09-16T07:23:38.348476+0000","last_active":"2024-09-16T07:23:38.351724+0000","last_peered":"2024-09-16T07:23:38.351724+0000","last_clean":"2024-09-16T07:23:38.351724+0000","last_became_active":"2024-09-16T07:23:38.348263+0000","last_became_peered":"2024-09-16T07:23:38.348263+0000","last_unstale":"2024-09-16T07:23:38.351724+0000","last_undegraded":"2024-09-16T07:23:38.351724+0000","last_fullsized":"2024-09-16T07:23:38.351724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:24.558417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.358843+0000","last_change":"2024-09-16T07:23:38.356216+0000","last_active":"2024-09-16T07:23:38.358843+0000","last_peered":"2024-09-16T07:23:38.358843+0000","last_clean":"2024-09-16T07:23:38.358843+0000","last_became_active":"2024-09-16T07:23:38.355971+0000","last_became_peered":"2024-09-16T07:23:38.355971+0000","last_unstale":"2024-09-16T07:23:38.358843+0000","last_undegraded":"2024-09-16T07:23:38.358843+0000","last_fullsized":"2024-09-16T07:23:38.358843+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:25.955048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.346389+0000","last_change":"2024-09-16T07:23:38.342538+0000","last_active":"2024-09-16T07:23:38.346389+0000","last_peered":"2024-09-16T07:23:38.346389+0000","last_clean":"2024-09-16T07:23:38.346389+0000","last_became_active":"2024-09-16T07:23:38.342085+0000","last_became_peered":"2024-09-16T07:23:38.342085+0000","last_unstale":"2024-09-16T07:23:38.346389+0000","last_undegraded":"2024-09-16T07:23:38.346389+0000","last_fullsized":"2024-09-16T07:23:38.346389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:56.231147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333820+0000","last_change":"2024-09-16T07:23:37.333820+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333820+0000","last_undegraded":"2024-09-16T07:23:37.333820+0000","last_fullsized":"2024-09-16T07:23:37.333820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.375763+0000","last_change":"2024-09-16T07:23:37.346853+0000","last_active":"2024-09-16T07:23:38.375763+0000","last_peered":"2024-09-16T07:23:38.375763+0000","last_clean":"2024-09-16T07:23:38.375763+0000","last_became_active":"2024-09-16T07:23:37.344215+0000","last_became_peered":"2024-09-16T07:23:37.344215+0000","last_unstale":"2024-09-16T07:23:38.375763+0000","last_undegraded":"2024-09-16T07:23:38.375763+0000","last_fullsized":"2024-09-16T07:23:38.375763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:55.470787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.350348+0000","last_change":"2024-09-16T07:23:38.347363+0000","last_active":"2024-09-16T07:23:38.350348+0000","last_peered":"2024-09-16T07:23:38.350348+0000","last_clean":"2024-09-16T07:23:38.350348+0000","last_became_active":"2024-09-16T07:23:38.347091+0000","last_became_peered":"2024-09-16T07:23:38.347091+0000","last_unstale":"2024-09-16T07:23:38.350348+0000","last_undegraded":"2024-09-16T07:23:38.350348+0000","last_fullsized":"2024-09-16T07:23:38.350348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:16.751332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615579+0000","last_change":"2024-09-16T07:23:37.349796+0000","last_active":"2024-09-16T07:23:40.615579+0000","last_peered":"2024-09-16T07:23:40.615579+0000","last_clean":"2024-09-16T07:23:40.615579+0000","last_became_active":"2024-09-16T07:23:37.349505+0000","last_became_peered":"2024-09-16T07:23:37.349505+0000","last_unstale":"2024-09-16T07:23:40.615579+0000","last_undegraded":"2024-09-16T07:23:40.615579+0000","last_fullsized":"2024-09-16T07:23:40.615579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:04.156099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.349933+0000","last_change":"2024-09-16T07:23:38.344847+0000","last_active":"2024-09-16T07:23:38.349933+0000","last_peered":"2024-09-16T07:23:38.349933+0000","last_clean":"2024-09-16T07:23:38.349933+0000","last_became_active":"2024-09-16T07:23:38.344502+0000","last_became_peered":"2024-09-16T07:23:38.344502+0000","last_unstale":"2024-09-16T07:23:38.349933+0000","last_undegraded":"2024-09-16T07:23:38.349933+0000","last_fullsized":"2024-09-16T07:23:38.349933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:26.398231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350064+0000","last_change":"2024-09-16T07:23:37.347456+0000","last_active":"2024-09-16T07:23:37.350064+0000","last_peered":"2024-09-16T07:23:37.350064+0000","last_clean":"2024-09-16T07:23:37.350064+0000","last_became_active":"2024-09-16T07:23:37.347152+0000","last_became_peered":"2024-09-16T07:23:37.347152+0000","last_unstale":"2024-09-16T07:23:37.350064+0000","last_undegraded":"2024-09-16T07:23:37.350064+0000","last_fullsized":"2024-09-16T07:23:37.350064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:51:05.553785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615486+0000","last_change":"2024-09-16T07:23:38.353195+0000","last_active":"2024-09-16T07:23:40.615486+0000","last_peered":"2024-09-16T07:23:40.615486+0000","last_clean":"2024-09-16T07:23:40.615486+0000","last_became_active":"2024-09-16T07:23:38.350558+0000","last_became_peered":"2024-09-16T07:23:38.350558+0000","last_unstale":"2024-09-16T07:23:40.615486+0000","last_undegraded":"2024-09-16T07:23:40.615486+0000","last_fullsized":"2024-09-16T07:23:40.615486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:23.647474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615384+0000","last_change":"2024-09-16T07:23:37.348375+0000","last_active":"2024-09-16T07:23:40.615384+0000","last_peered":"2024-09-16T07:23:40.615384+0000","last_clean":"2024-09-16T07:23:40.615384+0000","last_became_active":"2024-09-16T07:23:37.348096+0000","last_became_peered":"2024-09-16T07:23:37.348096+0000","last_unstale":"2024-09-16T07:23:40.615384+0000","last_undegraded":"2024-09-16T07:23:40.615384+0000","last_fullsized":"2024-09-16T07:23:40.615384+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:37.348737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359143+0000","last_change":"2024-09-16T07:23:38.351112+0000","last_active":"2024-09-16T07:23:38.359143+0000","last_peered":"2024-09-16T07:23:38.359143+0000","last_clean":"2024-09-16T07:23:38.359143+0000","last_became_active":"2024-09-16T07:23:38.350832+0000","last_became_peered":"2024-09-16T07:23:38.350832+0000","last_unstale":"2024-09-16T07:23:38.359143+0000","last_undegraded":"2024-09-16T07:23:38.359143+0000","last_fullsized":"2024-09-16T07:23:38.359143+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:44.019182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359294+0000","last_change":"2024-09-16T07:23:37.347090+0000","last_active":"2024-09-16T07:23:38.359294+0000","last_peered":"2024-09-16T07:23:38.359294+0000","last_clean":"2024-09-16T07:23:38.359294+0000","last_became_active":"2024-09-16T07:23:37.346832+0000","last_became_peered":"2024-09-16T07:23:37.346832+0000","last_unstale":"2024-09-16T07:23:38.359294+0000","last_undegraded":"2024-09-16T07:23:38.359294+0000","last_fullsized":"2024-09-16T07:23:38.359294+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:41.066939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.364807+0000","last_change":"2024-09-16T07:23:38.363179+0000","last_active":"2024-09-16T07:23:38.364807+0000","last_peered":"2024-09-16T07:23:38.364807+0000","last_clean":"2024-09-16T07:23:38.364807+0000","last_became_active":"2024-09-16T07:23:38.360647+0000","last_became_peered":"2024-09-16T07:23:38.360647+0000","last_unstale":"2024-09-16T07:23:38.364807+0000","last_undegraded":"2024-09-16T07:23:38.364807+0000","last_fullsized":"2024-09-16T07:23:38.364807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:09:24.558417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.363776+0000","last_change":"2024-09-16T07:23:37.344503+0000","last_active":"2024-09-16T07:23:38.363776+0000","last_peered":"2024-09-16T07:23:38.363776+0000","last_clean":"2024-09-16T07:23:38.363776+0000","last_became_active":"2024-09-16T07:23:37.344139+0000","last_became_peered":"2024-09-16T07:23:37.344139+0000","last_unstale":"2024-09-16T07:23:38.363776+0000","last_undegraded":"2024-09-16T07:23:38.363776+0000","last_fullsized":"2024-09-16T07:23:38.363776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:59.430414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352723+0000","last_change":"2024-09-16T07:23:38.348789+0000","last_active":"2024-09-16T07:23:38.352723+0000","last_peered":"2024-09-16T07:23:38.352723+0000","last_clean":"2024-09-16T07:23:38.352723+0000","last_became_active":"2024-09-16T07:23:38.348529+0000","last_became_peered":"2024-09-16T07:23:38.348529+0000","last_unstale":"2024-09-16T07:23:38.352723+0000","last_undegraded":"2024-09-16T07:23:38.352723+0000","last_fullsized":"2024-09-16T07:23:38.352723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:16.637145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350090+0000","last_change":"2024-09-16T07:23:37.348311+0000","last_active":"2024-09-16T07:23:37.350090+0000","last_peered":"2024-09-16T07:23:37.350090+0000","last_clean":"2024-09-16T07:23:37.350090+0000","last_became_active":"2024-09-16T07:23:37.348006+0000","last_became_peered":"2024-09-16T07:23:37.348006+0000","last_unstale":"2024-09-16T07:23:37.350090+0000","last_undegraded":"2024-09-16T07:23:37.350090+0000","last_fullsized":"2024-09-16T07:23:37.350090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:29.668774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614873+0000","last_change":"2024-09-16T07:23:38.352705+0000","last_active":"2024-09-16T07:23:40.614873+0000","last_peered":"2024-09-16T07:23:40.614873+0000","last_clean":"2024-09-16T07:23:40.614873+0000","last_became_active":"2024-09-16T07:23:38.352415+0000","last_became_peered":"2024-09-16T07:23:38.352415+0000","last_unstale":"2024-09-16T07:23:40.614873+0000","last_undegraded":"2024-09-16T07:23:40.614873+0000","last_fullsized":"2024-09-16T07:23:40.614873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:10.916617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.335903+0000","last_change":"2024-09-16T07:23:37.353028+0000","last_active":"2024-09-16T07:23:38.335903+0000","last_peered":"2024-09-16T07:23:38.335903+0000","last_clean":"2024-09-16T07:23:38.335903+0000","last_became_active":"2024-09-16T07:23:37.352718+0000","last_became_peered":"2024-09-16T07:23:37.352718+0000","last_unstale":"2024-09-16T07:23:38.335903+0000","last_undegraded":"2024-09-16T07:23:38.335903+0000","last_fullsized":"2024-09-16T07:23:38.335903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:41.965086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.360894+0000","last_change":"2024-09-16T07:23:38.353299+0000","last_active":"2024-09-16T07:23:38.360894+0000","last_peered":"2024-09-16T07:23:38.360894+0000","last_clean":"2024-09-16T07:23:38.360894+0000","last_became_active":"2024-09-16T07:23:38.352818+0000","last_became_peered":"2024-09-16T07:23:38.352818+0000","last_unstale":"2024-09-16T07:23:38.360894+0000","last_undegraded":"2024-09-16T07:23:38.360894+0000","last_fullsized":"2024-09-16T07:23:38.360894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:02.666636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.349313+0000","last_change":"2024-09-16T07:23:37.346845+0000","last_active":"2024-09-16T07:23:37.349313+0000","last_peered":"2024-09-16T07:23:37.349313+0000","last_clean":"2024-09-16T07:23:37.349313+0000","last_became_active":"2024-09-16T07:23:37.346574+0000","last_became_peered":"2024-09-16T07:23:37.346574+0000","last_unstale":"2024-09-16T07:23:37.349313+0000","last_undegraded":"2024-09-16T07:23:37.349313+0000","last_fullsized":"2024-09-16T07:23:37.349313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:30.460837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615689+0000","last_change":"2024-09-16T07:23:38.358332+0000","last_active":"2024-09-16T07:23:40.615689+0000","last_peered":"2024-09-16T07:23:40.615689+0000","last_clean":"2024-09-16T07:23:40.615689+0000","last_became_active":"2024-09-16T07:23:38.355145+0000","last_became_peered":"2024-09-16T07:23:38.355145+0000","last_unstale":"2024-09-16T07:23:40.615689+0000","last_undegraded":"2024-09-16T07:23:40.615689+0000","last_fullsized":"2024-09-16T07:23:40.615689+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.765410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346902+0000","last_change":"2024-09-16T07:23:37.345039+0000","last_active":"2024-09-16T07:23:37.346902+0000","last_peered":"2024-09-16T07:23:37.346902+0000","last_clean":"2024-09-16T07:23:37.346902+0000","last_became_active":"2024-09-16T07:23:37.344529+0000","last_became_peered":"2024-09-16T07:23:37.344529+0000","last_unstale":"2024-09-16T07:23:37.346902+0000","last_undegraded":"2024-09-16T07:23:37.346902+0000","last_fullsized":"2024-09-16T07:23:37.346902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:18.045656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615409+0000","last_change":"2024-09-16T07:23:38.344916+0000","last_active":"2024-09-16T07:23:40.615409+0000","last_peered":"2024-09-16T07:23:40.615409+0000","last_clean":"2024-09-16T07:23:40.615409+0000","last_became_active":"2024-09-16T07:23:38.344552+0000","last_became_peered":"2024-09-16T07:23:38.344552+0000","last_unstale":"2024-09-16T07:23:40.615409+0000","last_undegraded":"2024-09-16T07:23:40.615409+0000","last_fullsized":"2024-09-16T07:23:40.615409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:07:30.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.374362+0000","last_change":"2024-09-16T07:23:37.337958+0000","last_active":"2024-09-16T07:23:38.374362+0000","last_peered":"2024-09-16T07:23:38.374362+0000","last_clean":"2024-09-16T07:23:38.374362+0000","last_became_active":"2024-09-16T07:23:37.337681+0000","last_became_peered":"2024-09-16T07:23:37.337681+0000","last_unstale":"2024-09-16T07:23:38.374362+0000","last_undegraded":"2024-09-16T07:23:38.374362+0000","last_fullsized":"2024-09-16T07:23:38.374362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:31.334181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352729+0000","last_change":"2024-09-16T07:23:38.350404+0000","last_active":"2024-09-16T07:23:38.352729+0000","last_peered":"2024-09-16T07:23:38.352729+0000","last_clean":"2024-09-16T07:23:38.352729+0000","last_became_active":"2024-09-16T07:23:38.350111+0000","last_became_peered":"2024-09-16T07:23:38.350111+0000","last_unstale":"2024-09-16T07:23:38.352729+0000","last_undegraded":"2024-09-16T07:23:38.352729+0000","last_fullsized":"2024-09-16T07:23:38.352729+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:22.548419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.354301+0000","last_change":"2024-09-16T07:23:37.353132+0000","last_active":"2024-09-16T07:23:37.354301+0000","last_peered":"2024-09-16T07:23:37.354301+0000","last_clean":"2024-09-16T07:23:37.354301+0000","last_became_active":"2024-09-16T07:23:37.352306+0000","last_became_peered":"2024-09-16T07:23:37.352306+0000","last_unstale":"2024-09-16T07:23:37.354301+0000","last_undegraded":"2024-09-16T07:23:37.354301+0000","last_fullsized":"2024-09-16T07:23:37.354301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:00:30.195425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333905+0000","last_change":"2024-09-16T07:23:37.333905+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333905+0000","last_undegraded":"2024-09-16T07:23:37.333905+0000","last_fullsized":"2024-09-16T07:23:37.333905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.365888+0000","last_change":"2024-09-16T07:23:37.347964+0000","last_active":"2024-09-16T07:23:38.365888+0000","last_peered":"2024-09-16T07:23:38.365888+0000","last_clean":"2024-09-16T07:23:38.365888+0000","last_became_active":"2024-09-16T07:23:37.347687+0000","last_became_peered":"2024-09-16T07:23:37.347687+0000","last_unstale":"2024-09-16T07:23:38.365888+0000","last_undegraded":"2024-09-16T07:23:38.365888+0000","last_fullsized":"2024-09-16T07:23:38.365888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:24.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615132+0000","last_change":"2024-09-16T07:23:38.352259+0000","last_active":"2024-09-16T07:23:40.615132+0000","last_peered":"2024-09-16T07:23:40.615132+0000","last_clean":"2024-09-16T07:23:40.615132+0000","last_became_active":"2024-09-16T07:23:38.351935+0000","last_became_peered":"2024-09-16T07:23:38.351935+0000","last_unstale":"2024-09-16T07:23:40.615132+0000","last_undegraded":"2024-09-16T07:23:40.615132+0000","last_fullsized":"2024-09-16T07:23:40.615132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:09:10.158498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614756+0000","last_change":"2024-09-16T07:23:37.350953+0000","last_active":"2024-09-16T07:23:40.614756+0000","last_peered":"2024-09-16T07:23:40.614756+0000","last_clean":"2024-09-16T07:23:40.614756+0000","last_became_active":"2024-09-16T07:23:37.350621+0000","last_became_peered":"2024-09-16T07:23:37.350621+0000","last_unstale":"2024-09-16T07:23:40.614756+0000","last_undegraded":"2024-09-16T07:23:40.614756+0000","last_fullsized":"2024-09-16T07:23:40.614756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:34.314424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.352682+0000","last_change":"2024-09-16T07:23:38.348443+0000","last_active":"2024-09-16T07:23:38.352682+0000","last_peered":"2024-09-16T07:23:38.352682+0000","last_clean":"2024-09-16T07:23:38.352682+0000","last_became_active":"2024-09-16T07:23:38.347974+0000","last_became_peered":"2024-09-16T07:23:38.347974+0000","last_unstale":"2024-09-16T07:23:38.352682+0000","last_undegraded":"2024-09-16T07:23:38.352682+0000","last_fullsized":"2024-09-16T07:23:38.352682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:25:27.231251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614598+0000","last_change":"2024-09-16T07:23:37.352837+0000","last_active":"2024-09-16T07:23:40.614598+0000","last_peered":"2024-09-16T07:23:40.614598+0000","last_clean":"2024-09-16T07:23:40.614598+0000","last_became_active":"2024-09-16T07:23:37.352566+0000","last_became_peered":"2024-09-16T07:23:37.352566+0000","last_unstale":"2024-09-16T07:23:40.614598+0000","last_undegraded":"2024-09-16T07:23:40.614598+0000","last_fullsized":"2024-09-16T07:23:40.614598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:20.188668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.347236+0000","last_change":"2024-09-16T07:23:38.344224+0000","last_active":"2024-09-16T07:23:38.347236+0000","last_peered":"2024-09-16T07:23:38.347236+0000","last_clean":"2024-09-16T07:23:38.347236+0000","last_became_active":"2024-09-16T07:23:38.343918+0000","last_became_peered":"2024-09-16T07:23:38.343918+0000","last_unstale":"2024-09-16T07:23:38.347236+0000","last_undegraded":"2024-09-16T07:23:38.347236+0000","last_fullsized":"2024-09-16T07:23:38.347236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:55.009205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350824+0000","last_change":"2024-09-16T07:23:37.349547+0000","last_active":"2024-09-16T07:23:37.350824+0000","last_peered":"2024-09-16T07:23:37.350824+0000","last_clean":"2024-09-16T07:23:37.350824+0000","last_became_active":"2024-09-16T07:23:37.349283+0000","last_became_peered":"2024-09-16T07:23:37.349283+0000","last_unstale":"2024-09-16T07:23:37.350824+0000","last_undegraded":"2024-09-16T07:23:37.350824+0000","last_fullsized":"2024-09-16T07:23:37.350824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:41.491685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.353115+0000","last_change":"2024-09-16T07:23:38.349714+0000","last_active":"2024-09-16T07:23:38.353115+0000","last_peered":"2024-09-16T07:23:38.353115+0000","last_clean":"2024-09-16T07:23:38.353115+0000","last_became_active":"2024-09-16T07:23:38.349431+0000","last_became_peered":"2024-09-16T07:23:38.349431+0000","last_unstale":"2024-09-16T07:23:38.353115+0000","last_undegraded":"2024-09-16T07:23:38.353115+0000","last_fullsized":"2024-09-16T07:23:38.353115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:28.910006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.353383+0000","last_change":"2024-09-16T07:23:37.352287+0000","last_active":"2024-09-16T07:23:37.353383+0000","last_peered":"2024-09-16T07:23:37.353383+0000","last_clean":"2024-09-16T07:23:37.353383+0000","last_became_active":"2024-09-16T07:23:37.351844+0000","last_became_peered":"2024-09-16T07:23:37.351844+0000","last_unstale":"2024-09-16T07:23:37.353383+0000","last_undegraded":"2024-09-16T07:23:37.353383+0000","last_fullsized":"2024-09-16T07:23:37.353383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:24.047148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359294+0000","last_change":"2024-09-16T07:23:38.356161+0000","last_active":"2024-09-16T07:23:38.359294+0000","last_peered":"2024-09-16T07:23:38.359294+0000","last_clean":"2024-09-16T07:23:38.359294+0000","last_became_active":"2024-09-16T07:23:38.354062+0000","last_became_peered":"2024-09-16T07:23:38.354062+0000","last_unstale":"2024-09-16T07:23:38.359294+0000","last_undegraded":"2024-09-16T07:23:38.359294+0000","last_fullsized":"2024-09-16T07:23:38.359294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:36.964109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.366712+0000","last_change":"2024-09-16T07:23:37.349714+0000","last_active":"2024-09-16T07:23:38.366712+0000","last_peered":"2024-09-16T07:23:38.366712+0000","last_clean":"2024-09-16T07:23:38.366712+0000","last_became_active":"2024-09-16T07:23:37.349169+0000","last_became_peered":"2024-09-16T07:23:37.349169+0000","last_unstale":"2024-09-16T07:23:38.366712+0000","last_undegraded":"2024-09-16T07:23:38.366712+0000","last_fullsized":"2024-09-16T07:23:38.366712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:36.809629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.331286+0000","last_change":"2024-09-16T07:23:37.331286+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.331286+0000","last_undegraded":"2024-09-16T07:23:37.331286+0000","last_fullsized":"2024-09-16T07:23:37.331286+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.374468+0000","last_change":"2024-09-16T07:23:37.345215+0000","last_active":"2024-09-16T07:23:38.374468+0000","last_peered":"2024-09-16T07:23:38.374468+0000","last_clean":"2024-09-16T07:23:38.374468+0000","last_became_active":"2024-09-16T07:23:37.344006+0000","last_became_peered":"2024-09-16T07:23:37.344006+0000","last_unstale":"2024-09-16T07:23:38.374468+0000","last_undegraded":"2024-09-16T07:23:38.374468+0000","last_fullsized":"2024-09-16T07:23:38.374468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:43.819327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334029+0000","last_change":"2024-09-16T07:23:37.334029+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334029+0000","last_undegraded":"2024-09-16T07:23:37.334029+0000","last_fullsized":"2024-09-16T07:23:37.334029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.330726+0000","last_change":"2024-09-16T07:23:37.353056+0000","last_active":"2024-09-16T07:23:38.330726+0000","last_peered":"2024-09-16T07:23:38.330726+0000","last_clean":"2024-09-16T07:23:38.330726+0000","last_became_active":"2024-09-16T07:23:37.352723+0000","last_became_peered":"2024-09-16T07:23:37.352723+0000","last_unstale":"2024-09-16T07:23:38.330726+0000","last_undegraded":"2024-09-16T07:23:38.330726+0000","last_fullsized":"2024-09-16T07:23:38.330726+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:24.951401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.334406+0000","last_change":"2024-09-16T07:23:37.334406+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.334406+0000","last_undegraded":"2024-09-16T07:23:37.334406+0000","last_fullsized":"2024-09-16T07:23:37.334406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.350390+0000","last_change":"2024-09-16T07:23:37.347166+0000","last_active":"2024-09-16T07:23:37.350390+0000","last_peered":"2024-09-16T07:23:37.350390+0000","last_clean":"2024-09-16T07:23:37.350390+0000","last_became_active":"2024-09-16T07:23:37.346890+0000","last_became_peered":"2024-09-16T07:23:37.346890+0000","last_unstale":"2024-09-16T07:23:37.350390+0000","last_undegraded":"2024-09-16T07:23:37.350390+0000","last_fullsized":"2024-09-16T07:23:37.350390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:09.516212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.330484+0000","last_change":"2024-09-16T07:23:37.330484+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.330484+0000","last_undegraded":"2024-09-16T07:23:37.330484+0000","last_fullsized":"2024-09-16T07:23:37.330484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.336973+0000","last_change":"2024-09-16T07:23:37.349623+0000","last_active":"2024-09-16T07:23:38.336973+0000","last_peered":"2024-09-16T07:23:38.336973+0000","last_clean":"2024-09-16T07:23:38.336973+0000","last_became_active":"2024-09-16T07:23:37.349374+0000","last_became_peered":"2024-09-16T07:23:37.349374+0000","last_unstale":"2024-09-16T07:23:38.336973+0000","last_undegraded":"2024-09-16T07:23:38.336973+0000","last_fullsized":"2024-09-16T07:23:38.336973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:46.136303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.336554+0000","last_change":"2024-09-16T07:23:37.336554+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.336554+0000","last_undegraded":"2024-09-16T07:23:37.336554+0000","last_fullsized":"2024-09-16T07:23:37.336554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.343402+0000","last_change":"2024-09-16T07:23:37.347390+0000","last_active":"2024-09-16T07:23:38.343402+0000","last_peered":"2024-09-16T07:23:38.343402+0000","last_clean":"2024-09-16T07:23:38.343402+0000","last_became_active":"2024-09-16T07:23:37.346226+0000","last_became_peered":"2024-09-16T07:23:37.346226+0000","last_unstale":"2024-09-16T07:23:38.343402+0000","last_undegraded":"2024-09-16T07:23:38.343402+0000","last_fullsized":"2024-09-16T07:23:38.343402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:54.186625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.359489+0000","last_change":"2024-09-16T07:23:38.353841+0000","last_active":"2024-09-16T07:23:38.359489+0000","last_peered":"2024-09-16T07:23:38.359489+0000","last_clean":"2024-09-16T07:23:38.359489+0000","last_became_active":"2024-09-16T07:23:38.352266+0000","last_became_peered":"2024-09-16T07:23:38.352266+0000","last_unstale":"2024-09-16T07:23:38.359489+0000","last_undegraded":"2024-09-16T07:23:38.359489+0000","last_fullsized":"2024-09-16T07:23:38.359489+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:01.093082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.334377+0000","last_change":"2024-09-16T07:23:37.339731+0000","last_active":"2024-09-16T07:23:38.334377+0000","last_peered":"2024-09-16T07:23:38.334377+0000","last_clean":"2024-09-16T07:23:38.334377+0000","last_became_active":"2024-09-16T07:23:37.339480+0000","last_became_peered":"2024-09-16T07:23:37.339480+0000","last_unstale":"2024-09-16T07:23:38.334377+0000","last_undegraded":"2024-09-16T07:23:38.334377+0000","last_fullsized":"2024-09-16T07:23:38.334377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:58.263118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.337719+0000","last_change":"2024-09-16T07:23:37.337719+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.337719+0000","last_undegraded":"2024-09-16T07:23:37.337719+0000","last_fullsized":"2024-09-16T07:23:37.337719+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346812+0000","last_change":"2024-09-16T07:23:37.344100+0000","last_active":"2024-09-16T07:23:37.346812+0000","last_peered":"2024-09-16T07:23:37.346812+0000","last_clean":"2024-09-16T07:23:37.346812+0000","last_became_active":"2024-09-16T07:23:37.343269+0000","last_became_peered":"2024-09-16T07:23:37.343269+0000","last_unstale":"2024-09-16T07:23:37.346812+0000","last_undegraded":"2024-09-16T07:23:37.346812+0000","last_fullsized":"2024-09-16T07:23:37.346812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:39.795699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362195+0000","last_change":"2024-09-16T07:23:38.354226+0000","last_active":"2024-09-16T07:23:38.362195+0000","last_peered":"2024-09-16T07:23:38.362195+0000","last_clean":"2024-09-16T07:23:38.362195+0000","last_became_active":"2024-09-16T07:23:38.354001+0000","last_became_peered":"2024-09-16T07:23:38.354001+0000","last_unstale":"2024-09-16T07:23:38.362195+0000","last_undegraded":"2024-09-16T07:23:38.362195+0000","last_fullsized":"2024-09-16T07:23:38.362195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:16.310897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.345689+0000","last_change":"2024-09-16T07:23:37.343487+0000","last_active":"2024-09-16T07:23:37.345689+0000","last_peered":"2024-09-16T07:23:37.345689+0000","last_clean":"2024-09-16T07:23:37.345689+0000","last_became_active":"2024-09-16T07:23:37.343170+0000","last_became_peered":"2024-09-16T07:23:37.343170+0000","last_unstale":"2024-09-16T07:23:37.345689+0000","last_undegraded":"2024-09-16T07:23:37.345689+0000","last_fullsized":"2024-09-16T07:23:37.345689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:10.785300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T07:23:37.333534+0000","last_change":"2024-09-16T07:23:37.333534+0000","last_active":"2024-09-16T07:23:37.322443+0000","last_peered":"2024-09-16T07:23:37.322443+0000","last_clean":"2024-09-16T07:23:37.322443+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:23:37.333534+0000","last_undegraded":"2024-09-16T07:23:37.333534+0000","last_fullsized":"2024-09-16T07:23:37.333534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.348747+0000","last_change":"2024-09-16T07:23:37.346738+0000","last_active":"2024-09-16T07:23:37.348747+0000","last_peered":"2024-09-16T07:23:37.348747+0000","last_clean":"2024-09-16T07:23:37.348747+0000","last_became_active":"2024-09-16T07:23:37.345610+0000","last_became_peered":"2024-09-16T07:23:37.345610+0000","last_unstale":"2024-09-16T07:23:37.348747+0000","last_undegraded":"2024-09-16T07:23:37.348747+0000","last_fullsized":"2024-09-16T07:23:37.348747+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:45.870123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362706+0000","last_change":"2024-09-16T07:23:38.358556+0000","last_active":"2024-09-16T07:23:38.362706+0000","last_peered":"2024-09-16T07:23:38.362706+0000","last_clean":"2024-09-16T07:23:38.362706+0000","last_became_active":"2024-09-16T07:23:38.358288+0000","last_became_peered":"2024-09-16T07:23:38.358288+0000","last_unstale":"2024-09-16T07:23:38.362706+0000","last_undegraded":"2024-09-16T07:23:38.362706+0000","last_fullsized":"2024-09-16T07:23:38.362706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:34.849304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.346559+0000","last_change":"2024-09-16T07:23:37.343946+0000","last_active":"2024-09-16T07:23:37.346559+0000","last_peered":"2024-09-16T07:23:37.346559+0000","last_clean":"2024-09-16T07:23:37.346559+0000","last_became_active":"2024-09-16T07:23:37.343628+0000","last_became_peered":"2024-09-16T07:23:37.343628+0000","last_unstale":"2024-09-16T07:23:37.346559+0000","last_undegraded":"2024-09-16T07:23:37.346559+0000","last_fullsized":"2024-09-16T07:23:37.346559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:00.089436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615243+0000","last_change":"2024-09-16T07:23:38.346303+0000","last_active":"2024-09-16T07:23:40.615243+0000","last_peered":"2024-09-16T07:23:40.615243+0000","last_clean":"2024-09-16T07:23:40.615243+0000","last_became_active":"2024-09-16T07:23:38.346039+0000","last_became_peered":"2024-09-16T07:23:38.346039+0000","last_unstale":"2024-09-16T07:23:40.615243+0000","last_undegraded":"2024-09-16T07:23:40.615243+0000","last_fullsized":"2024-09-16T07:23:40.615243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:47.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T07:23:37.342710+0000","last_change":"2024-09-16T07:23:37.340258+0000","last_active":"2024-09-16T07:23:37.342710+0000","last_peered":"2024-09-16T07:23:37.342710+0000","last_clean":"2024-09-16T07:23:37.342710+0000","last_became_active":"2024-09-16T07:23:37.339993+0000","last_became_peered":"2024-09-16T07:23:37.339993+0000","last_unstale":"2024-09-16T07:23:37.342710+0000","last_undegraded":"2024-09-16T07:23:37.342710+0000","last_fullsized":"2024-09-16T07:23:37.342710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:29.784689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614910+0000","last_change":"2024-09-16T07:23:38.342446+0000","last_active":"2024-09-16T07:23:40.614910+0000","last_peered":"2024-09-16T07:23:40.614910+0000","last_clean":"2024-09-16T07:23:40.614910+0000","last_became_active":"2024-09-16T07:23:38.341488+0000","last_became_peered":"2024-09-16T07:23:38.341488+0000","last_unstale":"2024-09-16T07:23:40.614910+0000","last_undegraded":"2024-09-16T07:23:40.614910+0000","last_fullsized":"2024-09-16T07:23:40.614910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:51.750252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.336765+0000","last_change":"2024-09-16T07:23:37.348185+0000","last_active":"2024-09-16T07:23:38.336765+0000","last_peered":"2024-09-16T07:23:38.336765+0000","last_clean":"2024-09-16T07:23:38.336765+0000","last_became_active":"2024-09-16T07:23:37.347959+0000","last_became_peered":"2024-09-16T07:23:37.347959+0000","last_unstale":"2024-09-16T07:23:38.336765+0000","last_undegraded":"2024-09-16T07:23:38.336765+0000","last_fullsized":"2024-09-16T07:23:38.336765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:21:54.247700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362963+0000","last_change":"2024-09-16T07:23:38.355905+0000","last_active":"2024-09-16T07:23:38.362963+0000","last_peered":"2024-09-16T07:23:38.362963+0000","last_clean":"2024-09-16T07:23:38.362963+0000","last_became_active":"2024-09-16T07:23:38.355546+0000","last_became_peered":"2024-09-16T07:23:38.355546+0000","last_unstale":"2024-09-16T07:23:38.362963+0000","last_undegraded":"2024-09-16T07:23:38.362963+0000","last_fullsized":"2024-09-16T07:23:38.362963+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:42.332816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614759+0000","last_change":"2024-09-16T07:23:37.345122+0000","last_active":"2024-09-16T07:23:40.614759+0000","last_peered":"2024-09-16T07:23:40.614759+0000","last_clean":"2024-09-16T07:23:40.614759+0000","last_became_active":"2024-09-16T07:23:37.344793+0000","last_became_peered":"2024-09-16T07:23:37.344793+0000","last_unstale":"2024-09-16T07:23:40.614759+0000","last_undegraded":"2024-09-16T07:23:40.614759+0000","last_fullsized":"2024-09-16T07:23:40.614759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:56.809362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.362575+0000","last_change":"2024-09-16T07:23:38.358386+0000","last_active":"2024-09-16T07:23:38.362575+0000","last_peered":"2024-09-16T07:23:38.362575+0000","last_clean":"2024-09-16T07:23:38.362575+0000","last_became_active":"2024-09-16T07:23:38.358092+0000","last_became_peered":"2024-09-16T07:23:38.358092+0000","last_unstale":"2024-09-16T07:23:38.362575+0000","last_undegraded":"2024-09-16T07:23:38.362575+0000","last_fullsized":"2024-09-16T07:23:38.362575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:58:30.618313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.338916+0000","last_change":"2024-09-16T07:23:37.345296+0000","last_active":"2024-09-16T07:23:38.338916+0000","last_peered":"2024-09-16T07:23:38.338916+0000","last_clean":"2024-09-16T07:23:38.338916+0000","last_became_active":"2024-09-16T07:23:37.343905+0000","last_became_peered":"2024-09-16T07:23:37.343905+0000","last_unstale":"2024-09-16T07:23:38.338916+0000","last_undegraded":"2024-09-16T07:23:38.338916+0000","last_fullsized":"2024-09-16T07:23:38.338916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:43.228036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":130,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T07:23:38.338930+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:23:38.338930+0000","last_peered":"2024-09-16T07:23:38.338930+0000","last_clean":"2024-09-16T07:23:38.338930+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:23:38.338930+0000","last_undegraded":"2024-09-16T07:23:38.338930+0000","last_fullsized":"2024-09-16T07:23:38.338930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476753,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476753,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476753,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":954368,"data_stored":727307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":130664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":721032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":151648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:23:47.319 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:23:47.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.488+0000 7f79e0400640 1 -- 172.21.15.71:0/4077667715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 msgr2=0x7f79d80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:47.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.488+0000 7f79e0400640 1 --2- 172.21.15.71:0/4077667715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 0x7f79d80b2440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f79cc005fd0 tx=0x7f79cc02a4b0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.488+0000 7f79e0400640 1 -- 172.21.15.71:0/4077667715 shutdown_connections 2024-09-16T07:23:47.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.488+0000 7f79e0400640 1 --2- 172.21.15.71:0/4077667715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d8073fb0 0x7f79d8074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.488+0000 7f79e0400640 1 --2- 172.21.15.71:0/4077667715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 0x7f79d80b2440 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.488+0000 7f79e0400640 1 -- 172.21.15.71:0/4077667715 >> 172.21.15.71:0/4077667715 conn(0x7f79d80b1950 msgr2=0x7f79d80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:47.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.488+0000 7f79e0400640 1 -- 172.21.15.71:0/4077667715 shutdown_connections 2024-09-16T07:23:47.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.488+0000 7f79e0400640 1 -- 172.21.15.71:0/4077667715 wait complete. 2024-09-16T07:23:47.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.490+0000 7f79e0400640 1 Processor -- start 2024-09-16T07:23:47.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.490+0000 7f79e0400640 1 -- start start 2024-09-16T07:23:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79e0400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d8073fb0 0x7f79d80cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79e0400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d80ccf00 0x7f79d81536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79e0400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 0x7f79d815e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79e0400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79d806f2e0 con 0x7f79d815f5a0 2024-09-16T07:23:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79e0400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f79d806f160 con 0x7f79d80ccf00 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79e0400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f79d806f460 con 0x7f79d8073fb0 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79de400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 0x7f79d815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79de400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 0x7f79d815e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50806/0 (socket says 172.21.15.71:50806) 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79de400640 1 -- 172.21.15.71:0/2808098304 learned_addr learned my addr 172.21.15.71:0/2808098304 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79dda00640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d8073fb0 0x7f79d80cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.491+0000 7f79dd000640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d80ccf00 0x7f79d81536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79de400640 1 -- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d8073fb0 msgr2=0x7f79d80cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79de400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d8073fb0 0x7f79d80cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79de400640 1 -- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d80ccf00 msgr2=0x7f79d81536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79de400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d80ccf00 0x7f79d81536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79de400640 1 -- 172.21.15.71:0/2808098304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d81f53a0 con 0x7f79d815f5a0 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79dd000640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d80ccf00 0x7f79d81536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79de400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 0x7f79d815e320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b9a0 tx=0x7f79d000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:47.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79d5600640 1 -- 172.21.15.71:0/2808098304 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0019070 con 0x7f79d815f5a0 2024-09-16T07:23:47.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79d5600640 1 -- 172.21.15.71:0/2808098304 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79d000cc50 con 0x7f79d815f5a0 2024-09-16T07:23:47.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79d5600640 1 -- 172.21.15.71:0/2808098304 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00146a0 con 0x7f79d815f5a0 2024-09-16T07:23:47.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d81f5680 con 0x7f79d815f5a0 2024-09-16T07:23:47.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.492+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d81f5c40 con 0x7f79d815f5a0 2024-09-16T07:23:47.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.494+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d806f9b0 con 0x7f79d815f5a0 2024-09-16T07:23:47.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.495+0000 7f79d5600640 1 -- 172.21.15.71:0/2808098304 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f79d001c070 con 0x7f79d815f5a0 2024-09-16T07:23:47.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.496+0000 7f79d5600640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79b8078850 0x7f79b807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:47.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.496+0000 7f79d5600640 1 -- 172.21.15.71:0/2808098304 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f79d009a590 con 0x7f79d815f5a0 2024-09-16T07:23:47.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.496+0000 7f79dda00640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79b8078850 0x7f79b807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:47.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.497+0000 7f79dda00640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79b8078850 0x7f79b807ad10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79cc033000 tx=0x7f79cc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:47.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.500+0000 7f79d5600640 1 -- 172.21.15.71:0/2808098304 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f79d00625b0 con 0x7f79d815f5a0 2024-09-16T07:23:47.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.679+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79d80cdd10 con 0x7f79b8078850 2024-09-16T07:23:47.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.683+0000 7f79d5600640 1 -- 172.21.15.71:0/2808098304 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f79d80cdd10 con 0x7f79b8078850 2024-09-16T07:23:47.686 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:47.687 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:23:47.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79b8078850 msgr2=0x7f79b807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:47.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79b8078850 0x7f79b807ad10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79cc033000 tx=0x7f79cc004730 comp rx=0 tx=0).stop 2024-09-16T07:23:47.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 msgr2=0x7f79d815e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 0x7f79d815e320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b9a0 tx=0x7f79d000be60 comp rx=0 tx=0).stop 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 shutdown_connections 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79b8078850 0x7f79b807ad10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79d815f5a0 0x7f79d815e320 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d80ccf00 0x7f79d81536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 --2- 172.21.15.71:0/2808098304 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d8073fb0 0x7f79d80cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 >> 172.21.15.71:0/2808098304 conn(0x7f79d80b1950 msgr2=0x7f79d80acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 shutdown_connections 2024-09-16T07:23:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.687+0000 7f79e0400640 1 -- 172.21.15.71:0/2808098304 wait complete. 2024-09-16T07:23:47.703 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-16T07:23:46.306264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224620,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750100,"statfs":{"total":773094113280,"available":772864102400,"internally_reserved":0,"allocated":4333568,"data_stored":2385668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002144"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768224+0000","last_change":"2024-09-16T07:23:38.347812+0000","last_active":"2024-09-16T07:23:41.768224+0000","last_peered":"2024-09-16T07:23:41.768224+0000","last_clean":"2024-09-16T07:23:41.768224+0000","last_became_active":"2024-09-16T07:23:38.347516+0000","last_became_peered":"2024-09-16T07:23:38.347516+0000","last_unstale":"2024-09-16T07:23:41.768224+0000","last_undegraded":"2024-09-16T07:23:41.768224+0000","last_fullsized":"2024-09-16T07:23:41.768224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:35.903322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687525+0000","last_change":"2024-09-16T07:23:38.364449+0000","last_active":"2024-09-16T07:23:39.687525+0000","last_peered":"2024-09-16T07:23:39.687525+0000","last_clean":"2024-09-16T07:23:39.687525+0000","last_became_active":"2024-09-16T07:23:38.363727+0000","last_became_peered":"2024-09-16T07:23:38.363727+0000","last_unstale":"2024-09-16T07:23:39.687525+0000","last_undegraded":"2024-09-16T07:23:39.687525+0000","last_fullsized":"2024-09-16T07:23:39.687525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:37:29.989581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.742876+0000","last_change":"2024-09-16T07:23:38.352895+0000","last_active":"2024-09-16T07:23:41.742876+0000","last_peered":"2024-09-16T07:23:41.742876+0000","last_clean":"2024-09-16T07:23:41.742876+0000","last_became_active":"2024-09-16T07:23:38.352621+0000","last_became_peered":"2024-09-16T07:23:38.352621+0000","last_unstale":"2024-09-16T07:23:41.742876+0000","last_undegraded":"2024-09-16T07:23:41.742876+0000","last_fullsized":"2024-09-16T07:23:41.742876+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:21:55.251346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615292+0000","last_change":"2024-09-16T07:23:38.350692+0000","last_active":"2024-09-16T07:23:40.615292+0000","last_peered":"2024-09-16T07:23:40.615292+0000","last_clean":"2024-09-16T07:23:40.615292+0000","last_became_active":"2024-09-16T07:23:38.347475+0000","last_became_peered":"2024-09-16T07:23:38.347475+0000","last_unstale":"2024-09-16T07:23:40.615292+0000","last_undegraded":"2024-09-16T07:23:40.615292+0000","last_fullsized":"2024-09-16T07:23:40.615292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:01.740131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687298+0000","last_change":"2024-09-16T07:23:38.360911+0000","last_active":"2024-09-16T07:23:39.687298+0000","last_peered":"2024-09-16T07:23:39.687298+0000","last_clean":"2024-09-16T07:23:39.687298+0000","last_became_active":"2024-09-16T07:23:38.360628+0000","last_became_peered":"2024-09-16T07:23:38.360628+0000","last_unstale":"2024-09-16T07:23:39.687298+0000","last_undegraded":"2024-09-16T07:23:39.687298+0000","last_fullsized":"2024-09-16T07:23:39.687298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:54.364706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352894+0000","last_change":"2024-09-16T07:23:38.361116+0000","last_active":"2024-09-16T07:23:39.352894+0000","last_peered":"2024-09-16T07:23:39.352894+0000","last_clean":"2024-09-16T07:23:39.352894+0000","last_became_active":"2024-09-16T07:23:38.360643+0000","last_became_peered":"2024-09-16T07:23:38.360643+0000","last_unstale":"2024-09-16T07:23:39.352894+0000","last_undegraded":"2024-09-16T07:23:39.352894+0000","last_fullsized":"2024-09-16T07:23:39.352894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:00.434060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767795+0000","last_change":"2024-09-16T07:23:38.367300+0000","last_active":"2024-09-16T07:23:41.767795+0000","last_peered":"2024-09-16T07:23:41.767795+0000","last_clean":"2024-09-16T07:23:41.767795+0000","last_became_active":"2024-09-16T07:23:38.365509+0000","last_became_peered":"2024-09-16T07:23:38.365509+0000","last_unstale":"2024-09-16T07:23:41.767795+0000","last_undegraded":"2024-09-16T07:23:41.767795+0000","last_fullsized":"2024-09-16T07:23:41.767795+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:48.547216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767990+0000","last_change":"2024-09-16T07:23:38.357105+0000","last_active":"2024-09-16T07:23:41.767990+0000","last_peered":"2024-09-16T07:23:41.767990+0000","last_clean":"2024-09-16T07:23:41.767990+0000","last_became_active":"2024-09-16T07:23:38.355543+0000","last_became_peered":"2024-09-16T07:23:38.355543+0000","last_unstale":"2024-09-16T07:23:41.767990+0000","last_undegraded":"2024-09-16T07:23:41.767990+0000","last_fullsized":"2024-09-16T07:23:41.767990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:44.822974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.743076+0000","last_change":"2024-09-16T07:23:38.356452+0000","last_active":"2024-09-16T07:23:41.743076+0000","last_peered":"2024-09-16T07:23:41.743076+0000","last_clean":"2024-09-16T07:23:41.743076+0000","last_became_active":"2024-09-16T07:23:38.355663+0000","last_became_peered":"2024-09-16T07:23:38.355663+0000","last_unstale":"2024-09-16T07:23:41.743076+0000","last_undegraded":"2024-09-16T07:23:41.743076+0000","last_fullsized":"2024-09-16T07:23:41.743076+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:37.813275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615484+0000","last_change":"2024-09-16T07:23:38.356446+0000","last_active":"2024-09-16T07:23:40.615484+0000","last_peered":"2024-09-16T07:23:40.615484+0000","last_clean":"2024-09-16T07:23:40.615484+0000","last_became_active":"2024-09-16T07:23:38.354502+0000","last_became_peered":"2024-09-16T07:23:38.354502+0000","last_unstale":"2024-09-16T07:23:40.615484+0000","last_undegraded":"2024-09-16T07:23:40.615484+0000","last_fullsized":"2024-09-16T07:23:40.615484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:27.342707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353288+0000","last_change":"2024-09-16T07:23:38.351223+0000","last_active":"2024-09-16T07:23:39.353288+0000","last_peered":"2024-09-16T07:23:39.353288+0000","last_clean":"2024-09-16T07:23:39.353288+0000","last_became_active":"2024-09-16T07:23:38.349810+0000","last_became_peered":"2024-09-16T07:23:38.349810+0000","last_unstale":"2024-09-16T07:23:39.353288+0000","last_undegraded":"2024-09-16T07:23:39.353288+0000","last_fullsized":"2024-09-16T07:23:39.353288+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:11.628499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.743054+0000","last_change":"2024-09-16T07:23:38.365322+0000","last_active":"2024-09-16T07:23:41.743054+0000","last_peered":"2024-09-16T07:23:41.743054+0000","last_clean":"2024-09-16T07:23:41.743054+0000","last_became_active":"2024-09-16T07:23:38.364995+0000","last_became_peered":"2024-09-16T07:23:38.364995+0000","last_unstale":"2024-09-16T07:23:41.743054+0000","last_undegraded":"2024-09-16T07:23:41.743054+0000","last_fullsized":"2024-09-16T07:23:41.743054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:47.139949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353996+0000","last_change":"2024-09-16T07:23:38.349688+0000","last_active":"2024-09-16T07:23:39.353996+0000","last_peered":"2024-09-16T07:23:39.353996+0000","last_clean":"2024-09-16T07:23:39.353996+0000","last_became_active":"2024-09-16T07:23:38.348923+0000","last_became_peered":"2024-09-16T07:23:38.348923+0000","last_unstale":"2024-09-16T07:23:39.353996+0000","last_undegraded":"2024-09-16T07:23:39.353996+0000","last_fullsized":"2024-09-16T07:23:39.353996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:43.106934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614877+0000","last_change":"2024-09-16T07:23:38.347109+0000","last_active":"2024-09-16T07:23:40.614877+0000","last_peered":"2024-09-16T07:23:40.614877+0000","last_clean":"2024-09-16T07:23:40.614877+0000","last_became_active":"2024-09-16T07:23:38.346821+0000","last_became_peered":"2024-09-16T07:23:38.346821+0000","last_unstale":"2024-09-16T07:23:40.614877+0000","last_undegraded":"2024-09-16T07:23:40.614877+0000","last_fullsized":"2024-09-16T07:23:40.614877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:59.555576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.135799+0000","last_change":"2024-09-16T07:23:38.341149+0000","last_active":"2024-09-16T07:23:41.135799+0000","last_peered":"2024-09-16T07:23:41.135799+0000","last_clean":"2024-09-16T07:23:41.135799+0000","last_became_active":"2024-09-16T07:23:38.340794+0000","last_became_peered":"2024-09-16T07:23:38.340794+0000","last_unstale":"2024-09-16T07:23:41.135799+0000","last_undegraded":"2024-09-16T07:23:41.135799+0000","last_fullsized":"2024-09-16T07:23:41.135799+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:14:10.307938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.135752+0000","last_change":"2024-09-16T07:23:38.360203+0000","last_active":"2024-09-16T07:23:41.135752+0000","last_peered":"2024-09-16T07:23:41.135752+0000","last_clean":"2024-09-16T07:23:41.135752+0000","last_became_active":"2024-09-16T07:23:38.359933+0000","last_became_peered":"2024-09-16T07:23:38.359933+0000","last_unstale":"2024-09-16T07:23:41.135752+0000","last_undegraded":"2024-09-16T07:23:41.135752+0000","last_fullsized":"2024-09-16T07:23:41.135752+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:56.360226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353579+0000","last_change":"2024-09-16T07:23:38.350866+0000","last_active":"2024-09-16T07:23:39.353579+0000","last_peered":"2024-09-16T07:23:39.353579+0000","last_clean":"2024-09-16T07:23:39.353579+0000","last_became_active":"2024-09-16T07:23:38.350617+0000","last_became_peered":"2024-09-16T07:23:38.350617+0000","last_unstale":"2024-09-16T07:23:39.353579+0000","last_undegraded":"2024-09-16T07:23:39.353579+0000","last_fullsized":"2024-09-16T07:23:39.353579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:44.435406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353420+0000","last_change":"2024-09-16T07:23:38.367989+0000","last_active":"2024-09-16T07:23:39.353420+0000","last_peered":"2024-09-16T07:23:39.353420+0000","last_clean":"2024-09-16T07:23:39.353420+0000","last_became_active":"2024-09-16T07:23:38.367700+0000","last_became_peered":"2024-09-16T07:23:38.367700+0000","last_unstale":"2024-09-16T07:23:39.353420+0000","last_undegraded":"2024-09-16T07:23:39.353420+0000","last_fullsized":"2024-09-16T07:23:39.353420+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:37.813275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352872+0000","last_change":"2024-09-16T07:23:38.370122+0000","last_active":"2024-09-16T07:23:39.352872+0000","last_peered":"2024-09-16T07:23:39.352872+0000","last_clean":"2024-09-16T07:23:39.352872+0000","last_became_active":"2024-09-16T07:23:38.369828+0000","last_became_peered":"2024-09-16T07:23:38.369828+0000","last_unstale":"2024-09-16T07:23:39.352872+0000","last_undegraded":"2024-09-16T07:23:39.352872+0000","last_fullsized":"2024-09-16T07:23:39.352872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:42.968732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.354140+0000","last_change":"2024-09-16T07:23:38.357696+0000","last_active":"2024-09-16T07:23:39.354140+0000","last_peered":"2024-09-16T07:23:39.354140+0000","last_clean":"2024-09-16T07:23:39.354140+0000","last_became_active":"2024-09-16T07:23:38.357307+0000","last_became_peered":"2024-09-16T07:23:38.357307+0000","last_unstale":"2024-09-16T07:23:39.354140+0000","last_undegraded":"2024-09-16T07:23:39.354140+0000","last_fullsized":"2024-09-16T07:23:39.354140+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:48.547216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.135921+0000","last_change":"2024-09-16T07:23:38.355303+0000","last_active":"2024-09-16T07:23:41.135921+0000","last_peered":"2024-09-16T07:23:41.135921+0000","last_clean":"2024-09-16T07:23:41.135921+0000","last_became_active":"2024-09-16T07:23:38.355033+0000","last_became_peered":"2024-09-16T07:23:38.355033+0000","last_unstale":"2024-09-16T07:23:41.135921+0000","last_undegraded":"2024-09-16T07:23:41.135921+0000","last_fullsized":"2024-09-16T07:23:41.135921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:28:01.064434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767912+0000","last_change":"2024-09-16T07:23:38.345255+0000","last_active":"2024-09-16T07:23:41.767912+0000","last_peered":"2024-09-16T07:23:41.767912+0000","last_clean":"2024-09-16T07:23:41.767912+0000","last_became_active":"2024-09-16T07:23:38.344965+0000","last_became_peered":"2024-09-16T07:23:38.344965+0000","last_unstale":"2024-09-16T07:23:41.767912+0000","last_undegraded":"2024-09-16T07:23:41.767912+0000","last_fullsized":"2024-09-16T07:23:41.767912+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:14:10.307938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352547+0000","last_change":"2024-09-16T07:23:38.359966+0000","last_active":"2024-09-16T07:23:39.352547+0000","last_peered":"2024-09-16T07:23:39.352547+0000","last_clean":"2024-09-16T07:23:39.352547+0000","last_became_active":"2024-09-16T07:23:38.359620+0000","last_became_peered":"2024-09-16T07:23:38.359620+0000","last_unstale":"2024-09-16T07:23:39.352547+0000","last_undegraded":"2024-09-16T07:23:39.352547+0000","last_fullsized":"2024-09-16T07:23:39.352547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:15.007111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767766+0000","last_change":"2024-09-16T07:23:38.348792+0000","last_active":"2024-09-16T07:23:41.767766+0000","last_peered":"2024-09-16T07:23:41.767766+0000","last_clean":"2024-09-16T07:23:41.767766+0000","last_became_active":"2024-09-16T07:23:38.345353+0000","last_became_peered":"2024-09-16T07:23:38.345353+0000","last_unstale":"2024-09-16T07:23:41.767766+0000","last_undegraded":"2024-09-16T07:23:41.767766+0000","last_fullsized":"2024-09-16T07:23:41.767766+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:43.106934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615553+0000","last_change":"2024-09-16T07:23:38.351284+0000","last_active":"2024-09-16T07:23:40.615553+0000","last_peered":"2024-09-16T07:23:40.615553+0000","last_clean":"2024-09-16T07:23:40.615553+0000","last_became_active":"2024-09-16T07:23:38.350972+0000","last_became_peered":"2024-09-16T07:23:38.350972+0000","last_unstale":"2024-09-16T07:23:40.615553+0000","last_undegraded":"2024-09-16T07:23:40.615553+0000","last_fullsized":"2024-09-16T07:23:40.615553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:25.493457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615534+0000","last_change":"2024-09-16T07:23:38.356598+0000","last_active":"2024-09-16T07:23:40.615534+0000","last_peered":"2024-09-16T07:23:40.615534+0000","last_clean":"2024-09-16T07:23:40.615534+0000","last_became_active":"2024-09-16T07:23:38.356177+0000","last_became_peered":"2024-09-16T07:23:38.356177+0000","last_unstale":"2024-09-16T07:23:40.615534+0000","last_undegraded":"2024-09-16T07:23:40.615534+0000","last_fullsized":"2024-09-16T07:23:40.615534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:42.647875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687389+0000","last_change":"2024-09-16T07:23:38.348680+0000","last_active":"2024-09-16T07:23:39.687389+0000","last_peered":"2024-09-16T07:23:39.687389+0000","last_clean":"2024-09-16T07:23:39.687389+0000","last_became_active":"2024-09-16T07:23:38.348431+0000","last_became_peered":"2024-09-16T07:23:38.348431+0000","last_unstale":"2024-09-16T07:23:39.687389+0000","last_undegraded":"2024-09-16T07:23:39.687389+0000","last_fullsized":"2024-09-16T07:23:39.687389+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:50.388181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768215+0000","last_change":"2024-09-16T07:23:38.362005+0000","last_active":"2024-09-16T07:23:41.768215+0000","last_peered":"2024-09-16T07:23:41.768215+0000","last_clean":"2024-09-16T07:23:41.768215+0000","last_became_active":"2024-09-16T07:23:38.361748+0000","last_became_peered":"2024-09-16T07:23:38.361748+0000","last_unstale":"2024-09-16T07:23:41.768215+0000","last_undegraded":"2024-09-16T07:23:41.768215+0000","last_fullsized":"2024-09-16T07:23:41.768215+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:34.731762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.136141+0000","last_change":"2024-09-16T07:23:38.337747+0000","last_active":"2024-09-16T07:23:41.136141+0000","last_peered":"2024-09-16T07:23:41.136141+0000","last_clean":"2024-09-16T07:23:41.136141+0000","last_became_active":"2024-09-16T07:23:38.337455+0000","last_became_peered":"2024-09-16T07:23:38.337455+0000","last_unstale":"2024-09-16T07:23:41.136141+0000","last_undegraded":"2024-09-16T07:23:41.136141+0000","last_fullsized":"2024-09-16T07:23:41.136141+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:52.201668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.135871+0000","last_change":"2024-09-16T07:23:38.348476+0000","last_active":"2024-09-16T07:23:41.135871+0000","last_peered":"2024-09-16T07:23:41.135871+0000","last_clean":"2024-09-16T07:23:41.135871+0000","last_became_active":"2024-09-16T07:23:38.348263+0000","last_became_peered":"2024-09-16T07:23:38.348263+0000","last_unstale":"2024-09-16T07:23:41.135871+0000","last_undegraded":"2024-09-16T07:23:41.135871+0000","last_fullsized":"2024-09-16T07:23:41.135871+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:31.734526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767844+0000","last_change":"2024-09-16T07:23:38.356216+0000","last_active":"2024-09-16T07:23:41.767844+0000","last_peered":"2024-09-16T07:23:41.767844+0000","last_clean":"2024-09-16T07:23:41.767844+0000","last_became_active":"2024-09-16T07:23:38.355971+0000","last_became_peered":"2024-09-16T07:23:38.355971+0000","last_unstale":"2024-09-16T07:23:41.767844+0000","last_undegraded":"2024-09-16T07:23:41.767844+0000","last_fullsized":"2024-09-16T07:23:41.767844+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:44.666810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352897+0000","last_change":"2024-09-16T07:23:38.342538+0000","last_active":"2024-09-16T07:23:39.352897+0000","last_peered":"2024-09-16T07:23:39.352897+0000","last_clean":"2024-09-16T07:23:39.352897+0000","last_became_active":"2024-09-16T07:23:38.342085+0000","last_became_peered":"2024-09-16T07:23:38.342085+0000","last_unstale":"2024-09-16T07:23:39.352897+0000","last_undegraded":"2024-09-16T07:23:39.352897+0000","last_fullsized":"2024-09-16T07:23:39.352897+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:07.537884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.354255+0000","last_change":"2024-09-16T07:23:38.350021+0000","last_active":"2024-09-16T07:23:39.354255+0000","last_peered":"2024-09-16T07:23:39.354255+0000","last_clean":"2024-09-16T07:23:39.354255+0000","last_became_active":"2024-09-16T07:23:38.349704+0000","last_became_peered":"2024-09-16T07:23:38.349704+0000","last_unstale":"2024-09-16T07:23:39.354255+0000","last_undegraded":"2024-09-16T07:23:39.354255+0000","last_fullsized":"2024-09-16T07:23:39.354255+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:44.666810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353712+0000","last_change":"2024-09-16T07:23:37.346853+0000","last_active":"2024-09-16T07:23:39.353712+0000","last_peered":"2024-09-16T07:23:39.353712+0000","last_clean":"2024-09-16T07:23:39.353712+0000","last_became_active":"2024-09-16T07:23:37.344215+0000","last_became_peered":"2024-09-16T07:23:37.344215+0000","last_unstale":"2024-09-16T07:23:39.353712+0000","last_undegraded":"2024-09-16T07:23:39.353712+0000","last_fullsized":"2024-09-16T07:23:39.353712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:55.470787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768346+0000","last_change":"2024-09-16T07:23:38.347363+0000","last_active":"2024-09-16T07:23:41.768346+0000","last_peered":"2024-09-16T07:23:41.768346+0000","last_clean":"2024-09-16T07:23:41.768346+0000","last_became_active":"2024-09-16T07:23:38.347091+0000","last_became_peered":"2024-09-16T07:23:38.347091+0000","last_unstale":"2024-09-16T07:23:41.768346+0000","last_undegraded":"2024-09-16T07:23:41.768346+0000","last_fullsized":"2024-09-16T07:23:41.768346+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:52.201668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615579+0000","last_change":"2024-09-16T07:23:37.349796+0000","last_active":"2024-09-16T07:23:40.615579+0000","last_peered":"2024-09-16T07:23:40.615579+0000","last_clean":"2024-09-16T07:23:40.615579+0000","last_became_active":"2024-09-16T07:23:37.349505+0000","last_became_peered":"2024-09-16T07:23:37.349505+0000","last_unstale":"2024-09-16T07:23:40.615579+0000","last_undegraded":"2024-09-16T07:23:40.615579+0000","last_fullsized":"2024-09-16T07:23:40.615579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:04.156099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768228+0000","last_change":"2024-09-16T07:23:38.344847+0000","last_active":"2024-09-16T07:23:41.768228+0000","last_peered":"2024-09-16T07:23:41.768228+0000","last_clean":"2024-09-16T07:23:41.768228+0000","last_became_active":"2024-09-16T07:23:38.344502+0000","last_became_peered":"2024-09-16T07:23:38.344502+0000","last_unstale":"2024-09-16T07:23:41.768228+0000","last_undegraded":"2024-09-16T07:23:41.768228+0000","last_fullsized":"2024-09-16T07:23:41.768228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:44.863772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353138+0000","last_change":"2024-09-16T07:23:37.347456+0000","last_active":"2024-09-16T07:23:39.353138+0000","last_peered":"2024-09-16T07:23:39.353138+0000","last_clean":"2024-09-16T07:23:39.353138+0000","last_became_active":"2024-09-16T07:23:37.347152+0000","last_became_peered":"2024-09-16T07:23:37.347152+0000","last_unstale":"2024-09-16T07:23:39.353138+0000","last_undegraded":"2024-09-16T07:23:39.353138+0000","last_fullsized":"2024-09-16T07:23:39.353138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:15.633499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615486+0000","last_change":"2024-09-16T07:23:38.353195+0000","last_active":"2024-09-16T07:23:40.615486+0000","last_peered":"2024-09-16T07:23:40.615486+0000","last_clean":"2024-09-16T07:23:40.615486+0000","last_became_active":"2024-09-16T07:23:38.350558+0000","last_became_peered":"2024-09-16T07:23:38.350558+0000","last_unstale":"2024-09-16T07:23:40.615486+0000","last_undegraded":"2024-09-16T07:23:40.615486+0000","last_fullsized":"2024-09-16T07:23:40.615486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:23.647474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615384+0000","last_change":"2024-09-16T07:23:37.348375+0000","last_active":"2024-09-16T07:23:40.615384+0000","last_peered":"2024-09-16T07:23:40.615384+0000","last_clean":"2024-09-16T07:23:40.615384+0000","last_became_active":"2024-09-16T07:23:37.348096+0000","last_became_peered":"2024-09-16T07:23:37.348096+0000","last_unstale":"2024-09-16T07:23:40.615384+0000","last_undegraded":"2024-09-16T07:23:40.615384+0000","last_fullsized":"2024-09-16T07:23:40.615384+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:37.348737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.742783+0000","last_change":"2024-09-16T07:23:38.351112+0000","last_active":"2024-09-16T07:23:41.742783+0000","last_peered":"2024-09-16T07:23:41.742783+0000","last_clean":"2024-09-16T07:23:41.742783+0000","last_became_active":"2024-09-16T07:23:38.350832+0000","last_became_peered":"2024-09-16T07:23:38.350832+0000","last_unstale":"2024-09-16T07:23:41.742783+0000","last_undegraded":"2024-09-16T07:23:41.742783+0000","last_fullsized":"2024-09-16T07:23:41.742783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:42.968732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768153+0000","last_change":"2024-09-16T07:23:37.347090+0000","last_active":"2024-09-16T07:23:41.768153+0000","last_peered":"2024-09-16T07:23:41.768153+0000","last_clean":"2024-09-16T07:23:41.768153+0000","last_became_active":"2024-09-16T07:23:37.346832+0000","last_became_peered":"2024-09-16T07:23:37.346832+0000","last_unstale":"2024-09-16T07:23:41.768153+0000","last_undegraded":"2024-09-16T07:23:41.768153+0000","last_fullsized":"2024-09-16T07:23:41.768153+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:41.066939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767944+0000","last_change":"2024-09-16T07:23:38.363179+0000","last_active":"2024-09-16T07:23:41.767944+0000","last_peered":"2024-09-16T07:23:41.767944+0000","last_clean":"2024-09-16T07:23:41.767944+0000","last_became_active":"2024-09-16T07:23:38.360647+0000","last_became_peered":"2024-09-16T07:23:38.360647+0000","last_unstale":"2024-09-16T07:23:41.767944+0000","last_undegraded":"2024-09-16T07:23:41.767944+0000","last_fullsized":"2024-09-16T07:23:41.767944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:31.734526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767833+0000","last_change":"2024-09-16T07:23:37.344503+0000","last_active":"2024-09-16T07:23:41.767833+0000","last_peered":"2024-09-16T07:23:41.767833+0000","last_clean":"2024-09-16T07:23:41.767833+0000","last_became_active":"2024-09-16T07:23:37.344139+0000","last_became_peered":"2024-09-16T07:23:37.344139+0000","last_unstale":"2024-09-16T07:23:41.767833+0000","last_undegraded":"2024-09-16T07:23:41.767833+0000","last_fullsized":"2024-09-16T07:23:41.767833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:59.430414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.742675+0000","last_change":"2024-09-16T07:23:38.348789+0000","last_active":"2024-09-16T07:23:41.742675+0000","last_peered":"2024-09-16T07:23:41.742675+0000","last_clean":"2024-09-16T07:23:41.742675+0000","last_became_active":"2024-09-16T07:23:38.348529+0000","last_became_peered":"2024-09-16T07:23:38.348529+0000","last_unstale":"2024-09-16T07:23:41.742675+0000","last_undegraded":"2024-09-16T07:23:41.742675+0000","last_fullsized":"2024-09-16T07:23:41.742675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:25.854127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352729+0000","last_change":"2024-09-16T07:23:37.348311+0000","last_active":"2024-09-16T07:23:39.352729+0000","last_peered":"2024-09-16T07:23:39.352729+0000","last_clean":"2024-09-16T07:23:39.352729+0000","last_became_active":"2024-09-16T07:23:37.348006+0000","last_became_peered":"2024-09-16T07:23:37.348006+0000","last_unstale":"2024-09-16T07:23:39.352729+0000","last_undegraded":"2024-09-16T07:23:39.352729+0000","last_fullsized":"2024-09-16T07:23:39.352729+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:54.186625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614873+0000","last_change":"2024-09-16T07:23:38.352705+0000","last_active":"2024-09-16T07:23:40.614873+0000","last_peered":"2024-09-16T07:23:40.614873+0000","last_clean":"2024-09-16T07:23:40.614873+0000","last_became_active":"2024-09-16T07:23:38.352415+0000","last_became_peered":"2024-09-16T07:23:38.352415+0000","last_unstale":"2024-09-16T07:23:40.614873+0000","last_undegraded":"2024-09-16T07:23:40.614873+0000","last_fullsized":"2024-09-16T07:23:40.614873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:10.916617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768341+0000","last_change":"2024-09-16T07:23:37.353028+0000","last_active":"2024-09-16T07:23:41.768341+0000","last_peered":"2024-09-16T07:23:41.768341+0000","last_clean":"2024-09-16T07:23:41.768341+0000","last_became_active":"2024-09-16T07:23:37.352718+0000","last_became_peered":"2024-09-16T07:23:37.352718+0000","last_unstale":"2024-09-16T07:23:41.768341+0000","last_undegraded":"2024-09-16T07:23:41.768341+0000","last_fullsized":"2024-09-16T07:23:41.768341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:41.965086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353516+0000","last_change":"2024-09-16T07:23:38.353299+0000","last_active":"2024-09-16T07:23:39.353516+0000","last_peered":"2024-09-16T07:23:39.353516+0000","last_clean":"2024-09-16T07:23:39.353516+0000","last_became_active":"2024-09-16T07:23:38.352818+0000","last_became_peered":"2024-09-16T07:23:38.352818+0000","last_unstale":"2024-09-16T07:23:39.353516+0000","last_undegraded":"2024-09-16T07:23:39.353516+0000","last_fullsized":"2024-09-16T07:23:39.353516+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:24:28.804097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687605+0000","last_change":"2024-09-16T07:23:37.346845+0000","last_active":"2024-09-16T07:23:39.687605+0000","last_peered":"2024-09-16T07:23:39.687605+0000","last_clean":"2024-09-16T07:23:39.687605+0000","last_became_active":"2024-09-16T07:23:37.346574+0000","last_became_peered":"2024-09-16T07:23:37.346574+0000","last_unstale":"2024-09-16T07:23:39.687605+0000","last_undegraded":"2024-09-16T07:23:39.687605+0000","last_fullsized":"2024-09-16T07:23:39.687605+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:04:57.815997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615689+0000","last_change":"2024-09-16T07:23:38.358332+0000","last_active":"2024-09-16T07:23:40.615689+0000","last_peered":"2024-09-16T07:23:40.615689+0000","last_clean":"2024-09-16T07:23:40.615689+0000","last_became_active":"2024-09-16T07:23:38.355145+0000","last_became_peered":"2024-09-16T07:23:38.355145+0000","last_unstale":"2024-09-16T07:23:40.615689+0000","last_undegraded":"2024-09-16T07:23:40.615689+0000","last_fullsized":"2024-09-16T07:23:40.615689+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:01.765410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687575+0000","last_change":"2024-09-16T07:23:37.345039+0000","last_active":"2024-09-16T07:23:39.687575+0000","last_peered":"2024-09-16T07:23:39.687575+0000","last_clean":"2024-09-16T07:23:39.687575+0000","last_became_active":"2024-09-16T07:23:37.344529+0000","last_became_peered":"2024-09-16T07:23:37.344529+0000","last_unstale":"2024-09-16T07:23:39.687575+0000","last_undegraded":"2024-09-16T07:23:39.687575+0000","last_fullsized":"2024-09-16T07:23:39.687575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:48.888558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615409+0000","last_change":"2024-09-16T07:23:38.344916+0000","last_active":"2024-09-16T07:23:40.615409+0000","last_peered":"2024-09-16T07:23:40.615409+0000","last_clean":"2024-09-16T07:23:40.615409+0000","last_became_active":"2024-09-16T07:23:38.344552+0000","last_became_peered":"2024-09-16T07:23:38.344552+0000","last_unstale":"2024-09-16T07:23:40.615409+0000","last_undegraded":"2024-09-16T07:23:40.615409+0000","last_fullsized":"2024-09-16T07:23:40.615409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:07:30.581718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353165+0000","last_change":"2024-09-16T07:23:37.337958+0000","last_active":"2024-09-16T07:23:39.353165+0000","last_peered":"2024-09-16T07:23:39.353165+0000","last_clean":"2024-09-16T07:23:39.353165+0000","last_became_active":"2024-09-16T07:23:37.337681+0000","last_became_peered":"2024-09-16T07:23:37.337681+0000","last_unstale":"2024-09-16T07:23:39.353165+0000","last_undegraded":"2024-09-16T07:23:39.353165+0000","last_fullsized":"2024-09-16T07:23:39.353165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:31.334181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353091+0000","last_change":"2024-09-16T07:23:38.350404+0000","last_active":"2024-09-16T07:23:39.353091+0000","last_peered":"2024-09-16T07:23:39.353091+0000","last_clean":"2024-09-16T07:23:39.353091+0000","last_became_active":"2024-09-16T07:23:38.350111+0000","last_became_peered":"2024-09-16T07:23:38.350111+0000","last_unstale":"2024-09-16T07:23:39.353091+0000","last_undegraded":"2024-09-16T07:23:39.353091+0000","last_fullsized":"2024-09-16T07:23:39.353091+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:52:23.894268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.354131+0000","last_change":"2024-09-16T07:23:37.353132+0000","last_active":"2024-09-16T07:23:39.354131+0000","last_peered":"2024-09-16T07:23:39.354131+0000","last_clean":"2024-09-16T07:23:39.354131+0000","last_became_active":"2024-09-16T07:23:37.352306+0000","last_became_peered":"2024-09-16T07:23:37.352306+0000","last_unstale":"2024-09-16T07:23:39.354131+0000","last_undegraded":"2024-09-16T07:23:39.354131+0000","last_fullsized":"2024-09-16T07:23:39.354131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:20:09.003771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353254+0000","last_change":"2024-09-16T07:23:38.359979+0000","last_active":"2024-09-16T07:23:39.353254+0000","last_peered":"2024-09-16T07:23:39.353254+0000","last_clean":"2024-09-16T07:23:39.353254+0000","last_became_active":"2024-09-16T07:23:38.359467+0000","last_became_peered":"2024-09-16T07:23:38.359467+0000","last_unstale":"2024-09-16T07:23:39.353254+0000","last_undegraded":"2024-09-16T07:23:39.353254+0000","last_fullsized":"2024-09-16T07:23:39.353254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:30.204706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768214+0000","last_change":"2024-09-16T07:23:37.347964+0000","last_active":"2024-09-16T07:23:41.768214+0000","last_peered":"2024-09-16T07:23:41.768214+0000","last_clean":"2024-09-16T07:23:41.768214+0000","last_became_active":"2024-09-16T07:23:37.347687+0000","last_became_peered":"2024-09-16T07:23:37.347687+0000","last_unstale":"2024-09-16T07:23:41.768214+0000","last_undegraded":"2024-09-16T07:23:41.768214+0000","last_fullsized":"2024-09-16T07:23:41.768214+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:24.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615132+0000","last_change":"2024-09-16T07:23:38.352259+0000","last_active":"2024-09-16T07:23:40.615132+0000","last_peered":"2024-09-16T07:23:40.615132+0000","last_clean":"2024-09-16T07:23:40.615132+0000","last_became_active":"2024-09-16T07:23:38.351935+0000","last_became_peered":"2024-09-16T07:23:38.351935+0000","last_unstale":"2024-09-16T07:23:40.615132+0000","last_undegraded":"2024-09-16T07:23:40.615132+0000","last_fullsized":"2024-09-16T07:23:40.615132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:09:10.158498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614756+0000","last_change":"2024-09-16T07:23:37.350953+0000","last_active":"2024-09-16T07:23:40.614756+0000","last_peered":"2024-09-16T07:23:40.614756+0000","last_clean":"2024-09-16T07:23:40.614756+0000","last_became_active":"2024-09-16T07:23:37.350621+0000","last_became_peered":"2024-09-16T07:23:37.350621+0000","last_unstale":"2024-09-16T07:23:40.614756+0000","last_undegraded":"2024-09-16T07:23:40.614756+0000","last_fullsized":"2024-09-16T07:23:40.614756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:34.314424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767780+0000","last_change":"2024-09-16T07:23:38.348443+0000","last_active":"2024-09-16T07:23:41.767780+0000","last_peered":"2024-09-16T07:23:41.767780+0000","last_clean":"2024-09-16T07:23:41.767780+0000","last_became_active":"2024-09-16T07:23:38.347974+0000","last_became_peered":"2024-09-16T07:23:38.347974+0000","last_unstale":"2024-09-16T07:23:41.767780+0000","last_undegraded":"2024-09-16T07:23:41.767780+0000","last_fullsized":"2024-09-16T07:23:41.767780+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:56.360226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614598+0000","last_change":"2024-09-16T07:23:37.352837+0000","last_active":"2024-09-16T07:23:40.614598+0000","last_peered":"2024-09-16T07:23:40.614598+0000","last_clean":"2024-09-16T07:23:40.614598+0000","last_became_active":"2024-09-16T07:23:37.352566+0000","last_became_peered":"2024-09-16T07:23:37.352566+0000","last_unstale":"2024-09-16T07:23:40.614598+0000","last_undegraded":"2024-09-16T07:23:40.614598+0000","last_fullsized":"2024-09-16T07:23:40.614598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:20.188668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352938+0000","last_change":"2024-09-16T07:23:38.344224+0000","last_active":"2024-09-16T07:23:39.352938+0000","last_peered":"2024-09-16T07:23:39.352938+0000","last_clean":"2024-09-16T07:23:39.352938+0000","last_became_active":"2024-09-16T07:23:38.343918+0000","last_became_peered":"2024-09-16T07:23:38.343918+0000","last_unstale":"2024-09-16T07:23:39.352938+0000","last_undegraded":"2024-09-16T07:23:39.352938+0000","last_fullsized":"2024-09-16T07:23:39.352938+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:53.895630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352654+0000","last_change":"2024-09-16T07:23:37.349547+0000","last_active":"2024-09-16T07:23:39.352654+0000","last_peered":"2024-09-16T07:23:39.352654+0000","last_clean":"2024-09-16T07:23:39.352654+0000","last_became_active":"2024-09-16T07:23:37.349283+0000","last_became_peered":"2024-09-16T07:23:37.349283+0000","last_unstale":"2024-09-16T07:23:39.352654+0000","last_undegraded":"2024-09-16T07:23:39.352654+0000","last_fullsized":"2024-09-16T07:23:39.352654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:52.709610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353514+0000","last_change":"2024-09-16T07:23:38.349714+0000","last_active":"2024-09-16T07:23:39.353514+0000","last_peered":"2024-09-16T07:23:39.353514+0000","last_clean":"2024-09-16T07:23:39.353514+0000","last_became_active":"2024-09-16T07:23:38.349431+0000","last_became_peered":"2024-09-16T07:23:38.349431+0000","last_unstale":"2024-09-16T07:23:39.353514+0000","last_undegraded":"2024-09-16T07:23:39.353514+0000","last_fullsized":"2024-09-16T07:23:39.353514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:01:17.985373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353473+0000","last_change":"2024-09-16T07:23:37.352287+0000","last_active":"2024-09-16T07:23:39.353473+0000","last_peered":"2024-09-16T07:23:39.353473+0000","last_clean":"2024-09-16T07:23:39.353473+0000","last_became_active":"2024-09-16T07:23:37.351844+0000","last_became_peered":"2024-09-16T07:23:37.351844+0000","last_unstale":"2024-09-16T07:23:39.353473+0000","last_undegraded":"2024-09-16T07:23:39.353473+0000","last_fullsized":"2024-09-16T07:23:39.353473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:38:43.015535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.743089+0000","last_change":"2024-09-16T07:23:38.356161+0000","last_active":"2024-09-16T07:23:41.743089+0000","last_peered":"2024-09-16T07:23:41.743089+0000","last_clean":"2024-09-16T07:23:41.743089+0000","last_became_active":"2024-09-16T07:23:38.354062+0000","last_became_peered":"2024-09-16T07:23:38.354062+0000","last_unstale":"2024-09-16T07:23:41.743089+0000","last_undegraded":"2024-09-16T07:23:41.743089+0000","last_fullsized":"2024-09-16T07:23:41.743089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:26.398231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768070+0000","last_change":"2024-09-16T07:23:37.349714+0000","last_active":"2024-09-16T07:23:41.768070+0000","last_peered":"2024-09-16T07:23:41.768070+0000","last_clean":"2024-09-16T07:23:41.768070+0000","last_became_active":"2024-09-16T07:23:37.349169+0000","last_became_peered":"2024-09-16T07:23:37.349169+0000","last_unstale":"2024-09-16T07:23:41.768070+0000","last_undegraded":"2024-09-16T07:23:41.768070+0000","last_fullsized":"2024-09-16T07:23:41.768070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:36.809629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687617+0000","last_change":"2024-09-16T07:23:38.353931+0000","last_active":"2024-09-16T07:23:39.687617+0000","last_peered":"2024-09-16T07:23:39.687617+0000","last_clean":"2024-09-16T07:23:39.687617+0000","last_became_active":"2024-09-16T07:23:38.353662+0000","last_became_peered":"2024-09-16T07:23:38.353662+0000","last_unstale":"2024-09-16T07:23:39.687617+0000","last_undegraded":"2024-09-16T07:23:39.687617+0000","last_fullsized":"2024-09-16T07:23:39.687617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:43.762454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353260+0000","last_change":"2024-09-16T07:23:37.345215+0000","last_active":"2024-09-16T07:23:39.353260+0000","last_peered":"2024-09-16T07:23:39.353260+0000","last_clean":"2024-09-16T07:23:39.353260+0000","last_became_active":"2024-09-16T07:23:37.344006+0000","last_became_peered":"2024-09-16T07:23:37.344006+0000","last_unstale":"2024-09-16T07:23:39.353260+0000","last_undegraded":"2024-09-16T07:23:39.353260+0000","last_fullsized":"2024-09-16T07:23:39.353260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:43.819327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352981+0000","last_change":"2024-09-16T07:23:38.365841+0000","last_active":"2024-09-16T07:23:39.352981+0000","last_peered":"2024-09-16T07:23:39.352981+0000","last_clean":"2024-09-16T07:23:39.352981+0000","last_became_active":"2024-09-16T07:23:38.365523+0000","last_became_peered":"2024-09-16T07:23:38.365523+0000","last_unstale":"2024-09-16T07:23:39.352981+0000","last_undegraded":"2024-09-16T07:23:39.352981+0000","last_fullsized":"2024-09-16T07:23:39.352981+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:21:55.251346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.135973+0000","last_change":"2024-09-16T07:23:37.353056+0000","last_active":"2024-09-16T07:23:41.135973+0000","last_peered":"2024-09-16T07:23:41.135973+0000","last_clean":"2024-09-16T07:23:41.135973+0000","last_became_active":"2024-09-16T07:23:37.352723+0000","last_became_peered":"2024-09-16T07:23:37.352723+0000","last_unstale":"2024-09-16T07:23:41.135973+0000","last_undegraded":"2024-09-16T07:23:41.135973+0000","last_fullsized":"2024-09-16T07:23:41.135973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:24.951401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353018+0000","last_change":"2024-09-16T07:23:38.348138+0000","last_active":"2024-09-16T07:23:39.353018+0000","last_peered":"2024-09-16T07:23:39.353018+0000","last_clean":"2024-09-16T07:23:39.353018+0000","last_became_active":"2024-09-16T07:23:38.347729+0000","last_became_peered":"2024-09-16T07:23:38.347729+0000","last_unstale":"2024-09-16T07:23:39.353018+0000","last_undegraded":"2024-09-16T07:23:39.353018+0000","last_fullsized":"2024-09-16T07:23:39.353018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:06.039368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687519+0000","last_change":"2024-09-16T07:23:37.347166+0000","last_active":"2024-09-16T07:23:39.687519+0000","last_peered":"2024-09-16T07:23:39.687519+0000","last_clean":"2024-09-16T07:23:39.687519+0000","last_became_active":"2024-09-16T07:23:37.346890+0000","last_became_peered":"2024-09-16T07:23:37.346890+0000","last_unstale":"2024-09-16T07:23:39.687519+0000","last_undegraded":"2024-09-16T07:23:39.687519+0000","last_fullsized":"2024-09-16T07:23:39.687519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:38.765216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687444+0000","last_change":"2024-09-16T07:23:38.361667+0000","last_active":"2024-09-16T07:23:39.687444+0000","last_peered":"2024-09-16T07:23:39.687444+0000","last_clean":"2024-09-16T07:23:39.687444+0000","last_became_active":"2024-09-16T07:23:38.361404+0000","last_became_peered":"2024-09-16T07:23:38.361404+0000","last_unstale":"2024-09-16T07:23:39.687444+0000","last_undegraded":"2024-09-16T07:23:39.687444+0000","last_fullsized":"2024-09-16T07:23:39.687444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:17.352879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.768068+0000","last_change":"2024-09-16T07:23:37.349623+0000","last_active":"2024-09-16T07:23:41.768068+0000","last_peered":"2024-09-16T07:23:41.768068+0000","last_clean":"2024-09-16T07:23:41.768068+0000","last_became_active":"2024-09-16T07:23:37.349374+0000","last_became_peered":"2024-09-16T07:23:37.349374+0000","last_unstale":"2024-09-16T07:23:41.768068+0000","last_undegraded":"2024-09-16T07:23:41.768068+0000","last_fullsized":"2024-09-16T07:23:41.768068+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:46.136303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352527+0000","last_change":"2024-09-16T07:23:38.348671+0000","last_active":"2024-09-16T07:23:39.352527+0000","last_peered":"2024-09-16T07:23:39.352527+0000","last_clean":"2024-09-16T07:23:39.352527+0000","last_became_active":"2024-09-16T07:23:38.348392+0000","last_became_peered":"2024-09-16T07:23:38.348392+0000","last_unstale":"2024-09-16T07:23:39.352527+0000","last_undegraded":"2024-09-16T07:23:39.352527+0000","last_fullsized":"2024-09-16T07:23:39.352527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:26.118688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.742792+0000","last_change":"2024-09-16T07:23:37.347390+0000","last_active":"2024-09-16T07:23:41.742792+0000","last_peered":"2024-09-16T07:23:41.742792+0000","last_clean":"2024-09-16T07:23:41.742792+0000","last_became_active":"2024-09-16T07:23:37.346226+0000","last_became_peered":"2024-09-16T07:23:37.346226+0000","last_unstale":"2024-09-16T07:23:41.742792+0000","last_undegraded":"2024-09-16T07:23:41.742792+0000","last_fullsized":"2024-09-16T07:23:41.742792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:54.186625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.742834+0000","last_change":"2024-09-16T07:23:38.353841+0000","last_active":"2024-09-16T07:23:41.742834+0000","last_peered":"2024-09-16T07:23:41.742834+0000","last_clean":"2024-09-16T07:23:41.742834+0000","last_became_active":"2024-09-16T07:23:38.352266+0000","last_became_peered":"2024-09-16T07:23:38.352266+0000","last_unstale":"2024-09-16T07:23:41.742834+0000","last_undegraded":"2024-09-16T07:23:41.742834+0000","last_fullsized":"2024-09-16T07:23:41.742834+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:00.434060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352874+0000","last_change":"2024-09-16T07:23:37.339731+0000","last_active":"2024-09-16T07:23:39.352874+0000","last_peered":"2024-09-16T07:23:39.352874+0000","last_clean":"2024-09-16T07:23:39.352874+0000","last_became_active":"2024-09-16T07:23:37.339480+0000","last_became_peered":"2024-09-16T07:23:37.339480+0000","last_unstale":"2024-09-16T07:23:39.352874+0000","last_undegraded":"2024-09-16T07:23:39.352874+0000","last_fullsized":"2024-09-16T07:23:39.352874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:58.263118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353604+0000","last_change":"2024-09-16T07:23:38.365172+0000","last_active":"2024-09-16T07:23:39.353604+0000","last_peered":"2024-09-16T07:23:39.353604+0000","last_clean":"2024-09-16T07:23:39.353604+0000","last_became_active":"2024-09-16T07:23:38.364751+0000","last_became_peered":"2024-09-16T07:23:38.364751+0000","last_unstale":"2024-09-16T07:23:39.353604+0000","last_undegraded":"2024-09-16T07:23:39.353604+0000","last_fullsized":"2024-09-16T07:23:39.353604+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:26.398231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.354143+0000","last_change":"2024-09-16T07:23:37.344100+0000","last_active":"2024-09-16T07:23:39.354143+0000","last_peered":"2024-09-16T07:23:39.354143+0000","last_clean":"2024-09-16T07:23:39.354143+0000","last_became_active":"2024-09-16T07:23:37.343269+0000","last_became_peered":"2024-09-16T07:23:37.343269+0000","last_unstale":"2024-09-16T07:23:39.354143+0000","last_undegraded":"2024-09-16T07:23:39.354143+0000","last_fullsized":"2024-09-16T07:23:39.354143+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:00.897722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353573+0000","last_change":"2024-09-16T07:23:38.354226+0000","last_active":"2024-09-16T07:23:39.353573+0000","last_peered":"2024-09-16T07:23:39.353573+0000","last_clean":"2024-09-16T07:23:39.353573+0000","last_became_active":"2024-09-16T07:23:38.354001+0000","last_became_peered":"2024-09-16T07:23:38.354001+0000","last_unstale":"2024-09-16T07:23:39.353573+0000","last_undegraded":"2024-09-16T07:23:39.353573+0000","last_fullsized":"2024-09-16T07:23:39.353573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:33.416944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.687694+0000","last_change":"2024-09-16T07:23:37.343487+0000","last_active":"2024-09-16T07:23:39.687694+0000","last_peered":"2024-09-16T07:23:39.687694+0000","last_clean":"2024-09-16T07:23:39.687694+0000","last_became_active":"2024-09-16T07:23:37.343170+0000","last_became_peered":"2024-09-16T07:23:37.343170+0000","last_unstale":"2024-09-16T07:23:39.687694+0000","last_undegraded":"2024-09-16T07:23:39.687694+0000","last_fullsized":"2024-09-16T07:23:39.687694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:59.434043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353071+0000","last_change":"2024-09-16T07:23:38.373696+0000","last_active":"2024-09-16T07:23:39.353071+0000","last_peered":"2024-09-16T07:23:39.353071+0000","last_clean":"2024-09-16T07:23:39.353071+0000","last_became_active":"2024-09-16T07:23:38.373424+0000","last_became_peered":"2024-09-16T07:23:38.373424+0000","last_unstale":"2024-09-16T07:23:39.353071+0000","last_undegraded":"2024-09-16T07:23:39.353071+0000","last_fullsized":"2024-09-16T07:23:39.353071+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:47.139949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.354280+0000","last_change":"2024-09-16T07:23:37.346738+0000","last_active":"2024-09-16T07:23:39.354280+0000","last_peered":"2024-09-16T07:23:39.354280+0000","last_clean":"2024-09-16T07:23:39.354280+0000","last_became_active":"2024-09-16T07:23:37.345610+0000","last_became_peered":"2024-09-16T07:23:37.345610+0000","last_unstale":"2024-09-16T07:23:39.354280+0000","last_undegraded":"2024-09-16T07:23:39.354280+0000","last_fullsized":"2024-09-16T07:23:39.354280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:50.918751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.135868+0000","last_change":"2024-09-16T07:23:38.358556+0000","last_active":"2024-09-16T07:23:41.135868+0000","last_peered":"2024-09-16T07:23:41.135868+0000","last_clean":"2024-09-16T07:23:41.135868+0000","last_became_active":"2024-09-16T07:23:38.358288+0000","last_became_peered":"2024-09-16T07:23:38.358288+0000","last_unstale":"2024-09-16T07:23:41.135868+0000","last_undegraded":"2024-09-16T07:23:41.135868+0000","last_fullsized":"2024-09-16T07:23:41.135868+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:32.337827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.354168+0000","last_change":"2024-09-16T07:23:37.343946+0000","last_active":"2024-09-16T07:23:39.354168+0000","last_peered":"2024-09-16T07:23:39.354168+0000","last_clean":"2024-09-16T07:23:39.354168+0000","last_became_active":"2024-09-16T07:23:37.343628+0000","last_became_peered":"2024-09-16T07:23:37.343628+0000","last_unstale":"2024-09-16T07:23:39.354168+0000","last_undegraded":"2024-09-16T07:23:39.354168+0000","last_fullsized":"2024-09-16T07:23:39.354168+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:14:49.081277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.615243+0000","last_change":"2024-09-16T07:23:38.346303+0000","last_active":"2024-09-16T07:23:40.615243+0000","last_peered":"2024-09-16T07:23:40.615243+0000","last_clean":"2024-09-16T07:23:40.615243+0000","last_became_active":"2024-09-16T07:23:38.346039+0000","last_became_peered":"2024-09-16T07:23:38.346039+0000","last_unstale":"2024-09-16T07:23:40.615243+0000","last_undegraded":"2024-09-16T07:23:40.615243+0000","last_fullsized":"2024-09-16T07:23:40.615243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:47.994689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353406+0000","last_change":"2024-09-16T07:23:37.340258+0000","last_active":"2024-09-16T07:23:39.353406+0000","last_peered":"2024-09-16T07:23:39.353406+0000","last_clean":"2024-09-16T07:23:39.353406+0000","last_became_active":"2024-09-16T07:23:37.339993+0000","last_became_peered":"2024-09-16T07:23:37.339993+0000","last_unstale":"2024-09-16T07:23:39.353406+0000","last_undegraded":"2024-09-16T07:23:39.353406+0000","last_fullsized":"2024-09-16T07:23:39.353406+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:00:40.963415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614910+0000","last_change":"2024-09-16T07:23:38.342446+0000","last_active":"2024-09-16T07:23:40.614910+0000","last_peered":"2024-09-16T07:23:40.614910+0000","last_clean":"2024-09-16T07:23:40.614910+0000","last_became_active":"2024-09-16T07:23:38.341488+0000","last_became_peered":"2024-09-16T07:23:38.341488+0000","last_unstale":"2024-09-16T07:23:40.614910+0000","last_undegraded":"2024-09-16T07:23:40.614910+0000","last_fullsized":"2024-09-16T07:23:40.614910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:51.750252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.767970+0000","last_change":"2024-09-16T07:23:37.348185+0000","last_active":"2024-09-16T07:23:41.767970+0000","last_peered":"2024-09-16T07:23:41.767970+0000","last_clean":"2024-09-16T07:23:41.767970+0000","last_became_active":"2024-09-16T07:23:37.347959+0000","last_became_peered":"2024-09-16T07:23:37.347959+0000","last_unstale":"2024-09-16T07:23:41.767970+0000","last_undegraded":"2024-09-16T07:23:41.767970+0000","last_fullsized":"2024-09-16T07:23:41.767970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:21:54.247700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.352978+0000","last_change":"2024-09-16T07:23:38.355905+0000","last_active":"2024-09-16T07:23:39.352978+0000","last_peered":"2024-09-16T07:23:39.352978+0000","last_clean":"2024-09-16T07:23:39.352978+0000","last_became_active":"2024-09-16T07:23:38.355546+0000","last_became_peered":"2024-09-16T07:23:38.355546+0000","last_unstale":"2024-09-16T07:23:39.352978+0000","last_undegraded":"2024-09-16T07:23:39.352978+0000","last_fullsized":"2024-09-16T07:23:39.352978+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:59.228018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:40.614759+0000","last_change":"2024-09-16T07:23:37.345122+0000","last_active":"2024-09-16T07:23:40.614759+0000","last_peered":"2024-09-16T07:23:40.614759+0000","last_clean":"2024-09-16T07:23:40.614759+0000","last_became_active":"2024-09-16T07:23:37.344793+0000","last_became_peered":"2024-09-16T07:23:37.344793+0000","last_unstale":"2024-09-16T07:23:40.614759+0000","last_undegraded":"2024-09-16T07:23:40.614759+0000","last_fullsized":"2024-09-16T07:23:40.614759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:56.809362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.135929+0000","last_change":"2024-09-16T07:23:38.358386+0000","last_active":"2024-09-16T07:23:41.135929+0000","last_peered":"2024-09-16T07:23:41.135929+0000","last_clean":"2024-09-16T07:23:41.135929+0000","last_became_active":"2024-09-16T07:23:38.358092+0000","last_became_peered":"2024-09-16T07:23:38.358092+0000","last_unstale":"2024-09-16T07:23:41.135929+0000","last_undegraded":"2024-09-16T07:23:41.135929+0000","last_fullsized":"2024-09-16T07:23:41.135929+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:37.322443+0000","last_clean_scrub_stamp":"2024-09-16T07:23:37.322443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:44.231682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:39.353351+0000","last_change":"2024-09-16T07:23:37.345296+0000","last_active":"2024-09-16T07:23:39.353351+0000","last_peered":"2024-09-16T07:23:39.353351+0000","last_clean":"2024-09-16T07:23:39.353351+0000","last_became_active":"2024-09-16T07:23:37.343905+0000","last_became_peered":"2024-09-16T07:23:37.343905+0000","last_unstale":"2024-09-16T07:23:39.353351+0000","last_undegraded":"2024-09-16T07:23:39.353351+0000","last_fullsized":"2024-09-16T07:23:39.353351+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:23:36.318797+0000","last_clean_scrub_stamp":"2024-09-16T07:23:36.318797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:43.228036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":132,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T07:23:41.742691+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:23:41.742691+0000","last_peered":"2024-09-16T07:23:41.742691+0000","last_clean":"2024-09-16T07:23:41.742691+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:23:41.742691+0000","last_undegraded":"2024-09-16T07:23:41.742691+0000","last_fullsized":"2024-09-16T07:23:41.742691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":150790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476754,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476754,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":740452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476753,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":151626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":740964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":151648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:23:47.707 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:23:47.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:23:47.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.908+0000 7f5d05a00640 1 -- 172.21.15.71:0/3965002905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 msgr2=0x7f5d000688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:47.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.908+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3965002905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 0x7f5d000688b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0005fd0 tx=0x7f5cf002a440 comp rx=0 tx=0).stop 2024-09-16T07:23:47.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.908+0000 7f5d05a00640 1 -- 172.21.15.71:0/3965002905 shutdown_connections 2024-09-16T07:23:47.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.908+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3965002905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d000b15c0 0x7f5d0015f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.908+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3965002905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 0x7f5d000688b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.908+0000 7f5d05a00640 1 -- 172.21.15.71:0/3965002905 >> 172.21.15.71:0/3965002905 conn(0x7f5d000d0330 msgr2=0x7f5d000d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:47.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.908+0000 7f5d05a00640 1 -- 172.21.15.71:0/3965002905 shutdown_connections 2024-09-16T07:23:47.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.908+0000 7f5d05a00640 1 -- 172.21.15.71:0/3965002905 wait complete. 2024-09-16T07:23:47.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.910+0000 7f5d05a00640 1 Processor -- start 2024-09-16T07:23:47.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.910+0000 7f5d05a00640 1 -- start start 2024-09-16T07:23:47.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d000b15c0 0x7f5d000c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:47.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d000d0fa0 0x7f5d000ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 0x7f5d000ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0015fe10 con 0x7f5d0015f0f0 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0015f6e0 con 0x7f5d000d0fa0 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0015ff90 con 0x7f5d000b15c0 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 0x7f5d000ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 0x7f5d000ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50828/0 (socket says 172.21.15.71:50828) 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05000640 1 -- 172.21.15.71:0/3824755589 learned_addr learned my addr 172.21.15.71:0/3824755589 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5cffe00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d000b15c0 0x7f5d000c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05000640 1 -- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d000b15c0 msgr2=0x7f5d000c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05000640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d000b15c0 0x7f5d000c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05000640 1 -- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d000d0fa0 msgr2=0x7f5d000ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5cf7400640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d000d0fa0 0x7f5d000ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:47.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05000640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d000d0fa0 0x7f5d000ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.911+0000 7f5d05000640 1 -- 172.21.15.71:0/3824755589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0015b0d0 con 0x7f5d0015f0f0 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.912+0000 7f5cffe00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d000b15c0 0x7f5d000c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.912+0000 7f5cf7400640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d000d0fa0 0x7f5d000ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.912+0000 7f5d05000640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 0x7f5d000ce650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800eaa0 tx=0x7f5cf800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.912+0000 7f5cfcc00640 1 -- 172.21.15.71:0/3824755589 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800cdb0 con 0x7f5d0015f0f0 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.912+0000 7f5cfcc00640 1 -- 172.21.15.71:0/3824755589 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cf8004c10 con 0x7f5d0015f0f0 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.912+0000 7f5cfcc00640 1 -- 172.21.15.71:0/3824755589 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8005020 con 0x7f5d0015f0f0 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.912+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d000b44a0 con 0x7f5d0015f0f0 2024-09-16T07:23:47.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.912+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d000b4a60 con 0x7f5d0015f0f0 2024-09-16T07:23:47.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.913+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0015f8d0 con 0x7f5d0015f0f0 2024-09-16T07:23:47.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.914+0000 7f5cfcc00640 1 -- 172.21.15.71:0/3824755589 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5cf80051c0 con 0x7f5d0015f0f0 2024-09-16T07:23:47.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.915+0000 7f5cfcc00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5cdc078880 0x7f5cdc07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:47.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.915+0000 7f5cfcc00640 1 -- 172.21.15.71:0/3824755589 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5cf809b340 con 0x7f5d0015f0f0 2024-09-16T07:23:47.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.916+0000 7f5cffe00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5cdc078880 0x7f5cdc07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:47.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.916+0000 7f5cffe00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5cdc078880 0x7f5cdc07ad40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0033000 tx=0x7f5cf002c040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:47.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:47.919+0000 7f5cfcc00640 1 -- 172.21.15.71:0/3824755589 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5cf8016030 con 0x7f5d0015f0f0 2024-09-16T07:23:48.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.174+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5d00068bd0 con 0x7f5d0015f0f0 2024-09-16T07:23:48.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.177+0000 7f5cfcc00640 1 -- 172.21.15.71:0/3824755589 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f5cf8063910 con 0x7f5d0015f0f0 2024-09-16T07:23:48.176 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:23:48.177 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:23:48.177 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:23:48.177 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:23:48.177 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:23:48.177 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:23:48.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5cdc078880 msgr2=0x7f5cdc07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5cdc078880 0x7f5cdc07ad40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0033000 tx=0x7f5cf002c040 comp rx=0 tx=0).stop 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 msgr2=0x7f5d000ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 0x7f5d000ce650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800eaa0 tx=0x7f5cf800ef60 comp rx=0 tx=0).stop 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 shutdown_connections 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5cdc078880 0x7f5cdc07ad40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0015f0f0 0x7f5d000ce650 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d000d0fa0 0x7f5d000ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 --2- 172.21.15.71:0/3824755589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d000b15c0 0x7f5d000c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 >> 172.21.15.71:0/3824755589 conn(0x7f5d000d0330 msgr2=0x7f5d00070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 shutdown_connections 2024-09-16T07:23:48.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.180+0000 7f5d05a00640 1 -- 172.21.15.71:0/3824755589 wait complete. 2024-09-16T07:23:48.195 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:23:48.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.354+0000 7fc774400640 1 -- 172.21.15.71:0/1396387989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc754004d40 msgr2=0x7fc7540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:48.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.354+0000 7fc774400640 1 --2- 172.21.15.71:0/1396387989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc754004d40 0x7fc7540051c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc764001e30 tx=0x7fc754005830 comp rx=0 tx=0).stop 2024-09-16T07:23:48.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.354+0000 7fc772400640 1 -- 172.21.15.71:0/1396387989 reap_dead start 2024-09-16T07:23:48.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.354+0000 7fc774400640 1 -- 172.21.15.71:0/1396387989 shutdown_connections 2024-09-16T07:23:48.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.354+0000 7fc774400640 1 -- 172.21.15.71:0/1396387989 >> 172.21.15.71:0/1396387989 conn(0x7fc76c0b1950 msgr2=0x7fc76c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:48.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.354+0000 7fc774400640 1 -- 172.21.15.71:0/1396387989 shutdown_connections 2024-09-16T07:23:48.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.354+0000 7fc774400640 1 -- 172.21.15.71:0/1396387989 wait complete. 2024-09-16T07:23:48.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.356+0000 7fc774400640 1 Processor -- start 2024-09-16T07:23:48.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.356+0000 7fc774400640 1 -- start start 2024-09-16T07:23:48.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc774400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc754004d40 0x7fc76c1f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:48.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc774400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc76c0b10d0 0x7fc76c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:48.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc774400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c1f8000 0x7fc76c15b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:48.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc774400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c06f9f0 con 0x7fc76c0b10d0 2024-09-16T07:23:48.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc774400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c06f870 con 0x7fc754004d40 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc774400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c0b2440 con 0x7fc76c1f8000 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc771000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc76c0b10d0 0x7fc76c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc771000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc76c0b10d0 0x7fc76c0cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50852/0 (socket says 172.21.15.71:50852) 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc771000640 1 -- 172.21.15.71:0/3777735134 learned_addr learned my addr 172.21.15.71:0/3777735134 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc771a00640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc754004d40 0x7fc76c1f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.357+0000 7fc772400640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c1f8000 0x7fc76c15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc771000640 1 -- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c1f8000 msgr2=0x7fc76c15b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc771000640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c1f8000 0x7fc76c15b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc771000640 1 -- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc754004d40 msgr2=0x7fc76c1f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc771000640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc754004d40 0x7fc76c1f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc771000640 1 -- 172.21.15.71:0/3777735134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c1f8930 con 0x7fc76c0b10d0 2024-09-16T07:23:48.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc771000640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc76c0b10d0 0x7fc76c0cc400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc758011a50 tx=0x7fc758011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc769600640 1 -- 172.21.15.71:0/3777735134 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75801f070 con 0x7fc76c0b10d0 2024-09-16T07:23:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc769600640 1 -- 172.21.15.71:0/3777735134 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc75800fe80 con 0x7fc76c0b10d0 2024-09-16T07:23:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc769600640 1 -- 172.21.15.71:0/3777735134 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758012690 con 0x7fc76c0b10d0 2024-09-16T07:23:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c1f8c10 con 0x7fc76c0b10d0 2024-09-16T07:23:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.358+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c1f58f0 con 0x7fc76c0b10d0 2024-09-16T07:23:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.359+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c074440 con 0x7fc76c0b10d0 2024-09-16T07:23:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.360+0000 7fc769600640 1 -- 172.21.15.71:0/3777735134 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc758022070 con 0x7fc76c0b10d0 2024-09-16T07:23:48.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.362+0000 7fc769600640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc74c078850 0x7fc74c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:48.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.362+0000 7fc769600640 1 -- 172.21.15.71:0/3777735134 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc75809b440 con 0x7fc76c0b10d0 2024-09-16T07:23:48.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.362+0000 7fc771a00640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc74c078850 0x7fc74c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:48.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.363+0000 7fc771a00640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc74c078850 0x7fc74c07ad10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc760006c30 tx=0x7fc760001080 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:48.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.366+0000 7fc769600640 1 -- 172.21.15.71:0/3777735134 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc7580633c0 con 0x7fc76c0b10d0 2024-09-16T07:23:48.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.587+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc76c0cf800 con 0x7fc76c0b10d0 2024-09-16T07:23:48.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.588+0000 7fc769600640 1 -- 172.21.15.71:0/3777735134 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7fc758067df0 con 0x7fc76c0b10d0 2024-09-16T07:23:48.588 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:23:48.588 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"btime":"2024-09-16T07:23:41:354934+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4834,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3085563418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3085563418},{"type":"v1","addr":"172.21.15.71:6834","nonce":3085563418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3025435692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3025435692},{"type":"v1","addr":"172.21.15.71:6835","nonce":3025435692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3253921551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":3253921551},{"type":"v1","addr":"172.21.15.177:6836","nonce":3253921551}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:23:37.752030+0000","modified":"2024-09-16T07:23:40.774583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1652086843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1652086843},{"type":"v1","addr":"172.21.15.177:6837","nonce":1652086843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4917,"qdb_cluster":[4917]},"id":2}]} 2024-09-16T07:23:48.588 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 17 2024-09-16T07:23:48.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc74c078850 msgr2=0x7fc74c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:48.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc74c078850 0x7fc74c07ad10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc760006c30 tx=0x7fc760001080 comp rx=0 tx=0).stop 2024-09-16T07:23:48.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc76c0b10d0 msgr2=0x7fc76c0cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:48.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc76c0b10d0 0x7fc76c0cc400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc758011a50 tx=0x7fc758011f10 comp rx=0 tx=0).stop 2024-09-16T07:23:48.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 shutdown_connections 2024-09-16T07:23:48.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc74c078850 0x7fc74c07ad10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c1f8000 0x7fc76c15b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc76c0b10d0 0x7fc76c0cc400 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 --2- 172.21.15.71:0/3777735134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc754004d40 0x7fc76c1f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:48.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 >> 172.21.15.71:0/3777735134 conn(0x7fc76c0b1950 msgr2=0x7fc76c1556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:48.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 shutdown_connections 2024-09-16T07:23:48.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:48.590+0000 7fc774400640 1 -- 172.21.15.71:0/3777735134 wait complete. 2024-09-16T07:23:49.008 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:23:37.752030+0000', 'modified': '2024-09-16T07:23:40.774583+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4917}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4917': {'gid': 4917, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6837/1652086843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 1652086843}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 1652086843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4917, 'qdb_cluster': [4917]} 2024-09-16T07:23:49.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:23:49.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-16T07:23:49.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.169+0000 7f1a00c00640 1 -- 172.21.15.71:0/2036648559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc0ccf00 msgr2=0x7f19fc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.169+0000 7f1a00c00640 1 --2- 172.21.15.71:0/2036648559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc0ccf00 0x7f19fc0b2440 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f19e4005fd0 tx=0x7f19e402a540 comp rx=0 tx=0).stop 2024-09-16T07:23:49.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.169+0000 7f1a00c00640 1 -- 172.21.15.71:0/2036648559 shutdown_connections 2024-09-16T07:23:49.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.169+0000 7f1a00c00640 1 --2- 172.21.15.71:0/2036648559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc0ccf00 0x7f19fc0b2440 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.169+0000 7f1a00c00640 1 -- 172.21.15.71:0/2036648559 >> 172.21.15.71:0/2036648559 conn(0x7f19fc0b1950 msgr2=0x7f19fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:49.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.169+0000 7f1a00c00640 1 -- 172.21.15.71:0/2036648559 shutdown_connections 2024-09-16T07:23:49.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.169+0000 7f1a00c00640 1 -- 172.21.15.71:0/2036648559 wait complete. 2024-09-16T07:23:49.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.171+0000 7f1a00c00640 1 Processor -- start 2024-09-16T07:23:49.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.171+0000 7f1a00c00640 1 -- start start 2024-09-16T07:23:49.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f1a00c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc073fb0 0x7f19fc09cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f1a00c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19fc0ccf00 0x7f19fc0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f1a00c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19fc15f5a0 0x7f19fc1f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f1a00c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc06f9f0 con 0x7f19fc073fb0 2024-09-16T07:23:49.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f1a00c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc06f870 con 0x7f19fc0ccf00 2024-09-16T07:23:49.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f1a00c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f19fc074420 con 0x7f19fc15f5a0 2024-09-16T07:23:49.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc073fb0 0x7f19fc09cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc073fb0 0x7f19fc09cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50860/0 (socket says 172.21.15.71:50860) 2024-09-16T07:23:49.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f9e00640 1 -- 172.21.15.71:0/2812315800 learned_addr learned my addr 172.21.15.71:0/2812315800 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:49.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f3e00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19fc0ccf00 0x7f19fc0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f9e00640 1 -- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19fc15f5a0 msgr2=0x7f19fc1f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19fa800640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19fc15f5a0 0x7f19fc1f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f9e00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19fc15f5a0 0x7f19fc1f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f9e00640 1 -- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19fc0ccf00 msgr2=0x7f19fc0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f9e00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19fc0ccf00 0x7f19fc0b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f9e00640 1 -- 172.21.15.71:0/2812315800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19fc09d250 con 0x7f19fc073fb0 2024-09-16T07:23:49.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19f3e00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19fc0ccf00 0x7f19fc0b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:49.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.172+0000 7f19fa800640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19fc15f5a0 0x7f19fc1f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:49.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.173+0000 7f19f9e00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc073fb0 0x7f19fc09cd10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b3f0 tx=0x7f19ec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:49.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.173+0000 7f19f2000640 1 -- 172.21.15.71:0/2812315800 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec015020 con 0x7f19fc073fb0 2024-09-16T07:23:49.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.173+0000 7f19f2000640 1 -- 172.21.15.71:0/2812315800 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19ec011920 con 0x7f19fc073fb0 2024-09-16T07:23:49.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.173+0000 7f1a00c00640 1 -- 172.21.15.71:0/2812315800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19fc1f59f0 con 0x7f19fc073fb0 2024-09-16T07:23:49.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.173+0000 7f19f2000640 1 -- 172.21.15.71:0/2812315800 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec011d70 con 0x7f19fc073fb0 2024-09-16T07:23:49.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.173+0000 7f1a00c00640 1 -- 172.21.15.71:0/2812315800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19fc09c510 con 0x7f19fc073fb0 2024-09-16T07:23:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.175+0000 7f19f2000640 1 -- 172.21.15.71:0/2812315800 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f19ec018020 con 0x7f19fc073fb0 2024-09-16T07:23:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.176+0000 7f19f2000640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19d8078850 0x7f19d807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.176+0000 7f19f2000640 1 -- 172.21.15.71:0/2812315800 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f19ec09a630 con 0x7f19fc073fb0 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.177+0000 7f1a03a3c740 1 Processor -- start 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.177+0000 7f1a03a3c740 1 -- start start 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.177+0000 7f19f3e00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19d8078850 0x7f19d807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.177+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a608c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.177+0000 7f19f3e00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19d8078850 0x7f19d807ad10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f19e4033000 tx=0x7f19e4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3a20 0x55c75a607390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a6078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c75a546430 con 0x55c75a5c3fa0 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c75a5365d0 con 0x55c75a6088b0 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55c75a525a00 con 0x55c75a5c3a20 2024-09-16T07:23:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19f9e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a608c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a6078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19f9e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a608c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:53970/0 (socket says 172.21.15.71:53970) 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19f3e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3a20 0x55c75a607390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19f9e00640 1 -- 172.21.15.71:0/2465545213 learned_addr learned my addr 172.21.15.71:0/2465545213 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19fa800640 1 -- 172.21.15.71:0/2465545213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3a20 msgr2=0x55c75a607390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19fa800640 1 --2- 172.21.15.71:0/2465545213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3a20 0x55c75a607390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19fa800640 1 -- 172.21.15.71:0/2465545213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 msgr2=0x55c75a608c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19fa800640 1 --2- 172.21.15.71:0/2465545213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a608c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19fa800640 1 -- 172.21.15.71:0/2465545213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c75a4ee520 con 0x55c75a5c3fa0 2024-09-16T07:23:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.178+0000 7f19fa800640 1 --2- 172.21.15.71:0/2465545213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a6078d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19f4001d60 tx=0x7f19f400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fe943643711d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f19ea600640 1 -- 172.21.15.71:0/2465545213 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4016020 con 0x55c75a5c3fa0 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f19ea600640 1 -- 172.21.15.71:0/2465545213 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19f40123f0 con 0x55c75a5c3fa0 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f19ea600640 1 -- 172.21.15.71:0/2465545213 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4012800 con 0x55c75a5c3fa0 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f1a03a3c740 1 -- 172.21.15.71:0/2465545213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 msgr2=0x55c75a6078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/2465545213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a6078d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19f4001d60 tx=0x7f19f400d150 comp rx=0 tx=0).stop 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f1a03a3c740 1 -- 172.21.15.71:0/2465545213 shutdown_connections 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/2465545213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a6078d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/2465545213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3a20 0x55c75a607390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/2465545213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a608c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.179+0000 7f1a03a3c740 1 -- 172.21.15.71:0/2465545213 >> 172.21.15.71:0/2465545213 conn(0x55c75a5beff0 msgr2=0x55c75a5c2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.180+0000 7f1a03a3c740 1 -- 172.21.15.71:0/2465545213 shutdown_connections 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.180+0000 7f1a03a3c740 1 -- 172.21.15.71:0/2465545213 wait complete. 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.180+0000 7f1a03a3c740 1 -- start start 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.180+0000 7f1a03a3c740 1 Processor -- start 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.181+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a60ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.181+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 0x55c75a61a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.181+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a60b240 0x55c75a6002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.181+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c75a4cc330 con 0x55c75a5c3fa0 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.181+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c75a529810 con 0x55c75a60b240 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.181+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55c75a296620 con 0x55c75a6088b0 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.181+0000 7f1a03a3c740 10 client.0 _ll_register_callbacks cb 0x55c75a4d1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a60ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a60ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50872/0 (socket says 172.21.15.71:50872) 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 -- 172.21.15.71:0/3696263885 learned_addr learned my addr 172.21.15.71:0/3696263885 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19fa800640 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a60b240 0x55c75a6002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f3e00640 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 0x55c75a61a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 -- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 msgr2=0x55c75a61a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 0x55c75a61a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 -- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a60b240 msgr2=0x55c75a6002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a60b240 0x55c75a6002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 -- 172.21.15.71:0/3696263885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c75a287850 con 0x55c75a5c3fa0 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19f9e00640 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a60ad00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19ec05e5b0 tx=0x7f19ec062c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21429f9ab4eed793 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19dea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19dea00640 1 -- 172.21.15.71:0/3696263885 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec088020 con 0x55c75a5c3fa0 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f19dea00640 1 -- 172.21.15.71:0/3696263885 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f19ec0691e0 con 0x55c75a5c3fa0 2024-09-16T07:23:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.182+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3696263885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f40123f0 con 0x55c75a5c3fa0 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f19dea00640 1 -- 172.21.15.71:0/3696263885 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f19ec082050 con 0x55c75a5c3fa0 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f1a03a3c740 10 client.5124 fetch_fsmap learned FSMap version 17 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3696263885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c75a2fe6b0 con 0x55c75a5c3fa0 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f19dea00640 1 -- 172.21.15.71:0/3696263885 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19f40123f0 con 0x55c75a5c3fa0 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f19dea00640 1 -- 172.21.15.71:0/3696263885 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f19ec05de50 con 0x55c75a5c3fa0 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f1a03a3c740 10 client.5124 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f1a03a3c740 10 client.5124 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f1a03a3c740 20 client.5124 populate_metadata read hostname 'smithi071' 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] conn(0x55c75a5ec140 0x55c75a5ec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f1a03a3c740 4 client.5124 mds_command: new command op to 4917 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.183+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3696263885 --> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c75a538520 con 0x55c75a5ec140 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.184+0000 7f19f3e00640 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] conn(0x55c75a5ec140 0x55c75a5ec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.184+0000 7f19f3e00640 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] conn(0x55c75a5ec140 0x55c75a5ec520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7cbb61ddf17cefc server_cookie=12cb6b168699ccef in_seq=0 out_seq=0 2024-09-16T07:23:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.184+0000 7f19dea00640 10 client.5124 ms_handle_connect on v2:172.21.15.177:6834/1652086843 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.185+0000 7f19dea00640 1 -- 172.21.15.71:0/3696263885 <== mds.0 v2:172.21.15.177:6834/1652086843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c75a538520 con 0x55c75a5ec140 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.185+0000 7f19dea00640 10 client.5124 handle_command_reply: tid=0 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 client.5124 shutdown 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3696263885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 msgr2=0x55c75a60ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a60ad00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f19ec05e5b0 tx=0x7f19ec062c00 comp rx=0 tx=0).stop 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3696263885 shutdown_connections 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] conn(0x55c75a5ec140 0x55c75a5ec520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a60b240 0x55c75a6002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 0x55c75a61a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3696263885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a5c3fa0 0x55c75a60ad00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f19dea00640 0 client.5124 ms_handle_reset on v2:172.21.15.177:6834/1652086843 2024-09-16T07:23:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3696263885 >> 172.21.15.71:0/3696263885 conn(0x55c75a5c3a20 msgr2=0x55c75a5fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3696263885 shutdown_connections 2024-09-16T07:23:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3696263885 wait complete. 2024-09-16T07:23:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.186+0000 7f1a03a3c740 20 client.5124 trim_cache size 0 max 16384 2024-09-16T07:23:49.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.204+0000 7f1a03a3c740 1 Processor -- start 2024-09-16T07:23:49.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.204+0000 7f1a03a3c740 1 -- start start 2024-09-16T07:23:49.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.205+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3fa0 0x55c75a600050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.205+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a619690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.205+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a619bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.205+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc09b6f0 con 0x55c75a60b240 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.205+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc09aa50 con 0x55c75a6088b0 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.205+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f19fc09b0a0 con 0x55c75a5c3fa0 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.205+0000 7f19fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a619bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a619bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50882/0 (socket says 172.21.15.71:50882) 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19fa800640 1 -- 172.21.15.71:0/878772676 learned_addr learned my addr 172.21.15.71:0/878772676 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19f9e00640 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3fa0 0x55c75a600050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19f3e00640 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a619690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19fa800640 1 -- 172.21.15.71:0/878772676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3fa0 msgr2=0x55c75a600050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19fa800640 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3fa0 0x55c75a600050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19fa800640 1 -- 172.21.15.71:0/878772676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 msgr2=0x55c75a619690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:49.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19fa800640 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a619690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19fa800640 1 -- 172.21.15.71:0/878772676 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c75a62bc20 con 0x55c75a60b240 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19fa800640 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a619bd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19f400e010 tx=0x7f19f4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3792e12039acc6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19e9c00640 1 -- 172.21.15.71:0/878772676 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4016040 con 0x55c75a60b240 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19e9c00640 1 -- 172.21.15.71:0/878772676 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19f4006b50 con 0x55c75a60b240 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.206+0000 7f19e9c00640 1 -- 172.21.15.71:0/878772676 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4003970 con 0x55c75a60b240 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 -- 172.21.15.71:0/878772676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 msgr2=0x55c75a619bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a619bd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f19f400e010 tx=0x7f19f4001250 comp rx=0 tx=0).stop 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 -- 172.21.15.71:0/878772676 shutdown_connections 2024-09-16T07:23:49.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a619bd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a6088b0 0x55c75a619690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/878772676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a5c3fa0 0x55c75a600050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 -- 172.21.15.71:0/878772676 >> 172.21.15.71:0/878772676 conn(0x55c75a5c3a20 msgr2=0x55c75a60aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 -- 172.21.15.71:0/878772676 shutdown_connections 2024-09-16T07:23:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.207+0000 7f1a03a3c740 1 -- 172.21.15.71:0/878772676 wait complete. 2024-09-16T07:23:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.208+0000 7f1a03a3c740 1 -- start start 2024-09-16T07:23:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.208+0000 7f1a03a3c740 1 Processor -- start 2024-09-16T07:23:49.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a5c3fa0 0x55c75a5c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 0x55c75a6713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f1a03a3c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a68bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c75a600aa0 con 0x55c75a60b240 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c75a600ef0 con 0x55c75a5c3fa0 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f1a03a3c740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55c75a5d8ca0 con 0x55c75a6088b0 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f1a03a3c740 10 client.0 _ll_register_callbacks cb 0x7f19ec062010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a68bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a68bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50894/0 (socket says 172.21.15.71:50894) 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19fa800640 1 -- 172.21.15.71:0/3069008235 learned_addr learned my addr 172.21.15.71:0/3069008235 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19f9e00640 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a5c3fa0 0x55c75a5c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19f3e00640 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 0x55c75a6713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19fa800640 1 -- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 msgr2=0x55c75a6713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19fa800640 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 0x55c75a6713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19fa800640 1 -- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a5c3fa0 msgr2=0x55c75a5c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19fa800640 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a5c3fa0 0x55c75a5c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.209+0000 7f19fa800640 1 -- 172.21.15.71:0/3069008235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c75a671910 con 0x55c75a60b240 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f19fa800640 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a68bea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f19f4001ad0 tx=0x7f19f4001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83197121ea412a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f19dea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f19dea00640 1 -- 172.21.15.71:0/3069008235 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4016050 con 0x55c75a60b240 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f19dea00640 1 -- 172.21.15.71:0/3069008235 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f19f4005030 con 0x55c75a60b240 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3069008235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f4006b50 con 0x55c75a60b240 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f19dea00640 1 -- 172.21.15.71:0/3069008235 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f19f4005230 con 0x55c75a60b240 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f1a03a3c740 10 client.5130 fetch_fsmap learned FSMap version 17 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3069008235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c75a5c0960 con 0x55c75a60b240 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.210+0000 7f19dea00640 1 -- 172.21.15.71:0/3069008235 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19f4006b50 con 0x55c75a60b240 2024-09-16T07:23:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.211+0000 7f19dea00640 1 -- 172.21.15.71:0/3069008235 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f19f4038080 con 0x55c75a60b240 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.211+0000 7f1a03a3c740 10 client.5130 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.211+0000 7f1a03a3c740 10 client.5130 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.211+0000 7f1a03a3c740 20 client.5130 populate_metadata read hostname 'smithi071' 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.211+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] conn(0x55c75a619ce0 0x55c75a61a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.211+0000 7f1a03a3c740 4 client.5130 mds_command: new command op to 4917 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.211+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3069008235 --> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] -- command(tid 0: {"prefix": "status"}) -- 0x55c75a533fd0 con 0x55c75a619ce0 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.211+0000 7f19f9e00640 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] conn(0x55c75a619ce0 0x55c75a61a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f19f9e00640 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] conn(0x55c75a619ce0 0x55c75a61a0c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee4775717201d63 server_cookie=5a4430607c78c095 in_seq=0 out_seq=0 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f19dea00640 10 client.5130 ms_handle_connect on v2:172.21.15.177:6834/1652086843 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f19dea00640 1 -- 172.21.15.71:0/3069008235 <== mds.0 v2:172.21.15.177:6834/1652086843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55c75a533fd0 con 0x55c75a619ce0 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f19dea00640 10 client.5130 handle_command_reply: tid=0 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 client.5130 shutdown 2024-09-16T07:23:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3069008235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 msgr2=0x55c75a68bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a68bea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f19f4001ad0 tx=0x7f19f4001b00 comp rx=0 tx=0).stop 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3069008235 shutdown_connections 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:6834/1652086843,v1:172.21.15.177:6837/1652086843] conn(0x55c75a619ce0 0x55c75a61a0c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c75a60b240 0x55c75a68bea0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c75a6088b0 0x55c75a6713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 --2- 172.21.15.71:0/3069008235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c75a5c3fa0 0x55c75a5c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3069008235 >> 172.21.15.71:0/3069008235 conn(0x55c75a5c3a20 msgr2=0x55c75a5b8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f19dea00640 0 client.5130 ms_handle_reset on v2:172.21.15.177:6834/1652086843 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3069008235 shutdown_connections 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 1 -- 172.21.15.71:0/3069008235 wait complete. 2024-09-16T07:23:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.212+0000 7f1a03a3c740 20 client.5130 trim_cache size 0 max 16384 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4917, 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 10.884747624999999, 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 17, 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 35, 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 35, 2024-09-16T07:23:49.219 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 17.802335170999999 2024-09-16T07:23:49.220 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:23:49.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 -- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19d8078850 msgr2=0x7f19d807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19d8078850 0x7f19d807ad10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f19e4033000 tx=0x7f19e4004730 comp rx=0 tx=0).stop 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 -- 172.21.15.71:0/2812315800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc073fb0 msgr2=0x7f19fc09cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc073fb0 0x7f19fc09cd10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b3f0 tx=0x7f19ec00b8b0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 -- 172.21.15.71:0/2812315800 shutdown_connections 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19d8078850 0x7f19d807ad10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19fc15f5a0 0x7f19fc1f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19fc0ccf00 0x7f19fc0b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 --2- 172.21.15.71:0/2812315800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc073fb0 0x7f19fc09cd10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 -- 172.21.15.71:0/2812315800 >> 172.21.15.71:0/2812315800 conn(0x7f19fc0b1950 msgr2=0x7f19fc0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 -- 172.21.15.71:0/2812315800 shutdown_connections 2024-09-16T07:23:49.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.213+0000 7f1a00c00640 1 -- 172.21.15.71:0/2812315800 wait complete. 2024-09-16T07:23:49.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:23:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.386+0000 7f0bd3000640 1 -- 172.21.15.71:0/4007907623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc0b4100 msgr2=0x7f0bcc0b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.386+0000 7f0bd3000640 1 --2- 172.21.15.71:0/4007907623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc0b4100 0x7f0bcc0b1770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009830 tx=0x7f0bc402a480 comp rx=0 tx=0).stop 2024-09-16T07:23:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.386+0000 7f0bd3000640 1 -- 172.21.15.71:0/4007907623 shutdown_connections 2024-09-16T07:23:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.386+0000 7f0bd3000640 1 --2- 172.21.15.71:0/4007907623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc0b4100 0x7f0bcc0b1770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.386+0000 7f0bd3000640 1 --2- 172.21.15.71:0/4007907623 >> v2:172.21.15.177:3300/0 conn(0x7f0bcc0b5370 0x7f0bcc0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.386+0000 7f0bd3000640 1 -- 172.21.15.71:0/4007907623 >> 172.21.15.71:0/4007907623 conn(0x7f0bcc0d3df0 msgr2=0x7f0bcc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.386+0000 7f0bd3000640 1 -- 172.21.15.71:0/4007907623 shutdown_connections 2024-09-16T07:23:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.386+0000 7f0bd3000640 1 -- 172.21.15.71:0/4007907623 wait complete. 2024-09-16T07:23:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.388+0000 7f0bd3000640 1 Processor -- start 2024-09-16T07:23:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.388+0000 7f0bd3000640 1 -- start start 2024-09-16T07:23:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd3000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc0b4100 0x7f0bcc1f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc0b5370 0x7f0bcc0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd3000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc161440 0x7f0bcc15d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b2040 con 0x7f0bcc0b5370 2024-09-16T07:23:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd3000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b1ec0 con 0x7f0bcc0b4100 2024-09-16T07:23:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd3000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b21c0 con 0x7f0bcc161440 2024-09-16T07:23:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc0b5370 0x7f0bcc0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc0b5370 0x7f0bcc0caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50898/0 (socket says 172.21.15.71:50898) 2024-09-16T07:23:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd1200640 1 -- 172.21.15.71:0/1680241623 learned_addr learned my addr 172.21.15.71:0/1680241623 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd2600640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc161440 0x7f0bcc15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.389+0000 7f0bd1c00640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc0b4100 0x7f0bcc1f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd1200640 1 -- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc161440 msgr2=0x7f0bcc15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd1200640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc161440 0x7f0bcc15d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd1200640 1 -- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc0b4100 msgr2=0x7f0bcc1f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd1200640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc0b4100 0x7f0bcc1f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd1200640 1 -- 172.21.15.71:0/1680241623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc1f5640 con 0x7f0bcc0b5370 2024-09-16T07:23:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd2600640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc161440 0x7f0bcc15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:23:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd1200640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc0b5370 0x7f0bcc0caec0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800e9f0 tx=0x7f0bb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bc9600640 1 -- 172.21.15.71:0/1680241623 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800cde0 con 0x7f0bcc0b5370 2024-09-16T07:23:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bc9600640 1 -- 172.21.15.71:0/1680241623 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bb8004020 con 0x7f0bcc0b5370 2024-09-16T07:23:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bc9600640 1 -- 172.21.15.71:0/1680241623 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8004450 con 0x7f0bcc0b5370 2024-09-16T07:23:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc0b8630 con 0x7f0bcc0b5370 2024-09-16T07:23:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.390+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc0b8bf0 con 0x7f0bcc0b5370 2024-09-16T07:23:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.391+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc0b1770 con 0x7f0bcc0b5370 2024-09-16T07:23:49.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.392+0000 7f0bc9600640 1 -- 172.21.15.71:0/1680241623 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0bb80035f0 con 0x7f0bcc0b5370 2024-09-16T07:23:49.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.393+0000 7f0bc9600640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bac078850 0x7f0bac07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:49.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.393+0000 7f0bc9600640 1 -- 172.21.15.71:0/1680241623 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0bb809b450 con 0x7f0bcc0b5370 2024-09-16T07:23:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.394+0000 7f0bd1c00640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bac078850 0x7f0bac07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.395+0000 7f0bd1c00640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bac078850 0x7f0bac07ad10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0b2f10 tx=0x7f0bc0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.397+0000 7f0bc9600640 1 -- 172.21.15.71:0/1680241623 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0bb8063a20 con 0x7f0bcc0b5370 2024-09-16T07:23:49.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.617+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0bcc0ce390 con 0x7f0bcc0b5370 2024-09-16T07:23:49.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.617+0000 7f0bc9600640 1 -- 172.21.15.71:0/1680241623 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f0bb8063320 con 0x7f0bcc0b5370 2024-09-16T07:23:49.617 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:23:49.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bac078850 msgr2=0x7f0bac07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bac078850 0x7f0bac07ad10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0b2f10 tx=0x7f0bc0009040 comp rx=0 tx=0).stop 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc0b5370 msgr2=0x7f0bcc0caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc0b5370 0x7f0bcc0caec0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800e9f0 tx=0x7f0bb800eeb0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 shutdown_connections 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bac078850 0x7f0bac07ad10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bcc161440 0x7f0bcc15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc0b5370 0x7f0bcc0caec0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 --2- 172.21.15.71:0/1680241623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc0b4100 0x7f0bcc1f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 >> 172.21.15.71:0/1680241623 conn(0x7f0bcc0d3df0 msgr2=0x7f0bcc160430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 shutdown_connections 2024-09-16T07:23:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49.620+0000 7f0bd3000640 1 -- 172.21.15.71:0/1680241623 wait complete. 2024-09-16T07:23:49.635 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:23:49.636 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:23:49.636 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:23:49.636 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:23:49.636 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:49.636 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:23:49.636 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:23:49.636 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:23:49.636 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:23:49.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:49.637 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:23:49.672 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:23:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:23:49.672 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:23:49.673 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42647sec preferred_lft 42647sec 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:23:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:23:49.675 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:23:49.676 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:23:49.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:49.676 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:23:49.766 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:23:49.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:49.768 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:23:49.806 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:23:49.806 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:23:49.806 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:23:49.806 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:23:49.806 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:23:49.807 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:23:49.807 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:23:49.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:23:49.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:23:49.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:23:49.807 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:23:49.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:23:49.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:50.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:50.002 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:23:50.037 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:23:50.037 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:23:50.037 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:23:50.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:50.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:23:50.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:50.136 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:50.170 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:23:50.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:23:50.171 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:50.245 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:23:50.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:50.246 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:50.374 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:50.374 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:50.403 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:23:50.404 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:23:50.404 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:23:50.404 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:23:50.404 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:23:50.406 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:23:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:23:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:23:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:23:50.407 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:23:50.409 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:23:50.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:50.410 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:23:50.466 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:23:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:23:50.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:23:50.564 INFO:teuthology.orchestra.run.smithi071.stdout:client4808 2024-09-16T07:23:50.564 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:50.564 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:50.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:23:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.800+0000 7f1c22000640 1 -- 172.21.15.71:0/2788729039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c04005260 msgr2=0x7f1c0400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.800+0000 7f1c22000640 1 --2- 172.21.15.71:0/2788729039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c04005260 0x7f1c0400b9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400c640 tx=0x7f1c04005830 comp rx=0 tx=0).stop 2024-09-16T07:23:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.800+0000 7f1c21600640 1 -- 172.21.15.71:0/2788729039 reap_dead start 2024-09-16T07:23:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.801+0000 7f1c22000640 1 -- 172.21.15.71:0/2788729039 shutdown_connections 2024-09-16T07:23:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.801+0000 7f1c22000640 1 -- 172.21.15.71:0/2788729039 >> 172.21.15.71:0/2788729039 conn(0x7f1c1c0b1510 msgr2=0x7f1c1c0b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.801+0000 7f1c22000640 1 -- 172.21.15.71:0/2788729039 shutdown_connections 2024-09-16T07:23:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.801+0000 7f1c22000640 1 -- 172.21.15.71:0/2788729039 wait complete. 2024-09-16T07:23:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.802+0000 7f1c22000640 1 Processor -- start 2024-09-16T07:23:50.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.802+0000 7f1c22000640 1 -- start start 2024-09-16T07:23:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.803+0000 7f1c22000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c04005260 0x7f1c1c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.803+0000 7f1c22000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c1c1f3390 0x7f1c1c14cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.803+0000 7f1c22000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c1c0b30b0 0x7f1c1c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.803+0000 7f1c22000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c0737f0 con 0x7f1c1c1f3390 2024-09-16T07:23:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.803+0000 7f1c22000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c073970 con 0x7f1c04005260 2024-09-16T07:23:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.803+0000 7f1c22000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c073af0 con 0x7f1c1c0b30b0 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c1c1f3390 0x7f1c1c14cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c1c1f3390 0x7f1c1c14cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50912/0 (socket says 172.21.15.71:50912) 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 -- 172.21.15.71:0/2731295293 learned_addr learned my addr 172.21.15.71:0/2731295293 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 -- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c1c0b30b0 msgr2=0x7f1c1c15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c1c0b30b0 0x7f1c1c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 -- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c04005260 msgr2=0x7f1c1c0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c04005260 0x7f1c1c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 -- 172.21.15.71:0/2731295293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c0b49a0 con 0x7f1c1c1f3390 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c1be00640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c1c1f3390 0x7f1c1c14cd60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1c08011a30 tx=0x7f1c08011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c18c00640 1 -- 172.21.15.71:0/2731295293 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0801f070 con 0x7f1c1c1f3390 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c18c00640 1 -- 172.21.15.71:0/2731295293 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c0800fe20 con 0x7f1c1c1f3390 2024-09-16T07:23:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c18c00640 1 -- 172.21.15.71:0/2731295293 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c080126b0 con 0x7f1c1c1f3390 2024-09-16T07:23:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.804+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c0b43e0 con 0x7f1c1c1f3390 2024-09-16T07:23:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.805+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c0b28b0 con 0x7f1c1c1f3390 2024-09-16T07:23:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.806+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c074080 con 0x7f1c1c1f3390 2024-09-16T07:23:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.807+0000 7f1c18c00640 1 -- 172.21.15.71:0/2731295293 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1c08022070 con 0x7f1c1c1f3390 2024-09-16T07:23:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.808+0000 7f1c18c00640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1bfc078850 0x7f1bfc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.808+0000 7f1c18c00640 1 -- 172.21.15.71:0/2731295293 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1c0809b430 con 0x7f1c1c1f3390 2024-09-16T07:23:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.808+0000 7f1c20c00640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1bfc078850 0x7f1bfc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:50.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.809+0000 7f1c20c00640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1bfc078850 0x7f1bfc07ad10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c10006550 tx=0x7f1c10006060 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:50.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:50.812+0000 7f1c18c00640 1 -- 172.21.15.71:0/2731295293 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1c080633c0 con 0x7f1c1c1f3390 2024-09-16T07:23:51.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.036+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f1c1c0aeab0 con 0x7f1c1c1f3390 2024-09-16T07:23:51.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.242+0000 7f1c18c00640 1 -- 172.21.15.71:0/2731295293 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7f1c08067df0 con 0x7f1c1c1f3390 2024-09-16T07:23:51.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1bfc078850 msgr2=0x7f1bfc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1bfc078850 0x7f1bfc07ad10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c10006550 tx=0x7f1c10006060 comp rx=0 tx=0).stop 2024-09-16T07:23:51.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c1c1f3390 msgr2=0x7f1c1c14cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c1c1f3390 0x7f1c1c14cd60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1c08011a30 tx=0x7f1c08011ef0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 shutdown_connections 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1bfc078850 0x7f1bfc07ad10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c1c0b30b0 0x7f1c1c15d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c1c1f3390 0x7f1c1c14cd60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 --2- 172.21.15.71:0/2731295293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c04005260 0x7f1c1c0b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 >> 172.21.15.71:0/2731295293 conn(0x7f1c1c0b1510 msgr2=0x7f1c1c0cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 shutdown_connections 2024-09-16T07:23:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.245+0000 7f1c22000640 1 -- 172.21.15.71:0/2731295293 wait complete. 2024-09-16T07:23:51.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:23:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.418+0000 7f8de4200640 1 -- 172.21.15.71:0/3050580215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 msgr2=0x7f8ddc06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.418+0000 7f8de4200640 1 --2- 172.21.15.71:0/3050580215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 0x7f8ddc06f490 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc06d9e0 tx=0x7f8dc802a560 comp rx=0 tx=0).stop 2024-09-16T07:23:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.418+0000 7f8de4200640 1 -- 172.21.15.71:0/3050580215 shutdown_connections 2024-09-16T07:23:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.418+0000 7f8de4200640 1 --2- 172.21.15.71:0/3050580215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 0x7f8ddc06f490 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.418+0000 7f8de4200640 1 --2- 172.21.15.71:0/3050580215 >> v2:172.21.15.177:3300/0 conn(0x7f8ddc161550 0x7f8ddc06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.418+0000 7f8de4200640 1 -- 172.21.15.71:0/3050580215 >> 172.21.15.71:0/3050580215 conn(0x7f8ddc0b1950 msgr2=0x7f8ddc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:51.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.418+0000 7f8de4200640 1 -- 172.21.15.71:0/3050580215 shutdown_connections 2024-09-16T07:23:51.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.418+0000 7f8de4200640 1 -- 172.21.15.71:0/3050580215 wait complete. 2024-09-16T07:23:51.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.420+0000 7f8de4200640 1 Processor -- start 2024-09-16T07:23:51.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.420+0000 7f8de4200640 1 -- start start 2024-09-16T07:23:51.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de4200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc06f9d0 0x7f8ddc1483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de4200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 0x7f8ddc0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de4200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc161550 0x7f8ddc1601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de4200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc073f80 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de4200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc073e00 con 0x7f8ddc06f9d0 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de4200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8ddc074100 con 0x7f8ddc161550 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 0x7f8ddc0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 0x7f8ddc0cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50936/0 (socket says 172.21.15.71:50936) 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de0e00640 1 -- 172.21.15.71:0/4086208060 learned_addr learned my addr 172.21.15.71:0/4086208060 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de1800640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc06f9d0 0x7f8ddc1483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de2200640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc161550 0x7f8ddc1601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de0e00640 1 -- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc161550 msgr2=0x7f8ddc1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.421+0000 7f8de0e00640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc161550 0x7f8ddc1601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8de0e00640 1 -- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc06f9d0 msgr2=0x7f8ddc1483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8de0e00640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc06f9d0 0x7f8ddc1483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8de0e00640 1 -- 172.21.15.71:0/4086208060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc0d0480 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8de2200640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc161550 0x7f8ddc1601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8de0e00640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 0x7f8ddc0cf9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80099c0 tx=0x7f8dc80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8dd9600640 1 -- 172.21.15.71:0/4086208060 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc803f070 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8dd9600640 1 -- 172.21.15.71:0/4086208060 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dc80047a0 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8dd9600640 1 -- 172.21.15.71:0/4086208060 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8035030 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ddc09ba50 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.422+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ddc09c2f0 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.423+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ddc0b3310 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.424+0000 7f8dd9600640 1 -- 172.21.15.71:0/4086208060 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8dc8042070 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.425+0000 7f8dd9600640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8dbc078850 0x7f8dbc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.425+0000 7f8dd9600640 1 -- 172.21.15.71:0/4086208060 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8dc80bbae0 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.426+0000 7f8de1800640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8dbc078850 0x7f8dbc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:51.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.427+0000 7f8de1800640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8dbc078850 0x7f8dbc07ad10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009ac0 tx=0x7f8dd0005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.429+0000 7f8dd9600640 1 -- 172.21.15.71:0/4086208060 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ddc0b3310 con 0x7f8ddc0af3d0 2024-09-16T07:23:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.637+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f8ddc09bc20 con 0x7f8dbc078850 2024-09-16T07:23:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.639+0000 7f8dd9600640 1 -- 172.21.15.71:0/4086208060 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f8ddc09bc20 con 0x7f8dbc078850 2024-09-16T07:23:51.638 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:23:51.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:23:51.638 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:23:51.639 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:23:51.639 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:23:51.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8dbc078850 msgr2=0x7f8dbc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8dbc078850 0x7f8dbc07ad10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009ac0 tx=0x7f8dd0005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:51.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 msgr2=0x7f8ddc0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 0x7f8ddc0cf9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80099c0 tx=0x7f8dc80038a0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 shutdown_connections 2024-09-16T07:23:51.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8dbc078850 0x7f8dbc07ad10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc161550 0x7f8ddc1601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ddc0af3d0 0x7f8ddc0cf9c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 --2- 172.21.15.71:0/4086208060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc06f9d0 0x7f8ddc1483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 >> 172.21.15.71:0/4086208060 conn(0x7f8ddc0b1950 msgr2=0x7f8ddc06ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:51.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 shutdown_connections 2024-09-16T07:23:51.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.641+0000 7f8de4200640 1 -- 172.21.15.71:0/4086208060 wait complete. 2024-09-16T07:23:51.657 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:23:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.819+0000 7fe876200640 1 -- 172.21.15.71:0/3136068360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87006efa0 msgr2=0x7fe8700cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.819+0000 7fe876200640 1 --2- 172.21.15.71:0/3136068360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87006efa0 0x7fe8700cdbb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe868005fd0 tx=0x7fe86802aa10 comp rx=0 tx=0).stop 2024-09-16T07:23:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.819+0000 7fe876200640 1 -- 172.21.15.71:0/3136068360 shutdown_connections 2024-09-16T07:23:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.819+0000 7fe876200640 1 --2- 172.21.15.71:0/3136068360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87006efa0 0x7fe8700cdbb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.819+0000 7fe876200640 1 --2- 172.21.15.71:0/3136068360 >> v2:172.21.15.177:3300/0 conn(0x7fe8700cccd0 0x7fe870073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.819+0000 7fe876200640 1 -- 172.21.15.71:0/3136068360 >> 172.21.15.71:0/3136068360 conn(0x7fe8700b1840 msgr2=0x7fe8700b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.819+0000 7fe876200640 1 -- 172.21.15.71:0/3136068360 shutdown_connections 2024-09-16T07:23:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.819+0000 7fe876200640 1 -- 172.21.15.71:0/3136068360 wait complete. 2024-09-16T07:23:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.820+0000 7fe876200640 1 Processor -- start 2024-09-16T07:23:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.821+0000 7fe876200640 1 -- start start 2024-09-16T07:23:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.821+0000 7fe876200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87006efa0 0x7fe870151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.821+0000 7fe876200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8700cccd0 0x7fe8701e8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.821+0000 7fe876200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870145750 0x7fe870140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.821+0000 7fe876200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe8700ce480 con 0x7fe870145750 2024-09-16T07:23:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.821+0000 7fe876200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe8700ce300 con 0x7fe87006efa0 2024-09-16T07:23:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.821+0000 7fe876200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe8700ce600 con 0x7fe8700cccd0 2024-09-16T07:23:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870145750 0x7fe870140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86f400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87006efa0 0x7fe870151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870145750 0x7fe870140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50940/0 (socket says 172.21.15.71:50940) 2024-09-16T07:23:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86fe00640 1 -- 172.21.15.71:0/2731150186 learned_addr learned my addr 172.21.15.71:0/2731150186 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86f400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87006efa0 0x7fe870151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:54066/0 (socket says 172.21.15.71:54066) 2024-09-16T07:23:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86ea00640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8700cccd0 0x7fe8701e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86f400640 1 -- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8700cccd0 msgr2=0x7fe8701e8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86f400640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8700cccd0 0x7fe8701e8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86f400640 1 -- 172.21.15.71:0/2731150186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870145750 msgr2=0x7fe870140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86f400640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870145750 0x7fe870140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86f400640 1 -- 172.21.15.71:0/2731150186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8701e8d10 con 0x7fe87006efa0 2024-09-16T07:23:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.822+0000 7fe86fe00640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870145750 0x7fe870140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:23:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.823+0000 7fe86f400640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87006efa0 0x7fe870151660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c3c0 tx=0x7fe86000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.823+0000 7fe867400640 1 -- 172.21.15.71:0/2731150186 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860017070 con 0x7fe87006efa0 2024-09-16T07:23:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.823+0000 7fe867400640 1 -- 172.21.15.71:0/2731150186 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8600053e0 con 0x7fe87006efa0 2024-09-16T07:23:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.823+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8701e8ff0 con 0x7fe87006efa0 2024-09-16T07:23:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.823+0000 7fe867400640 1 -- 172.21.15.71:0/2731150186 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860003850 con 0x7fe87006efa0 2024-09-16T07:23:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.823+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe870142b20 con 0x7fe87006efa0 2024-09-16T07:23:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.824+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8700cdbb0 con 0x7fe87006efa0 2024-09-16T07:23:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.825+0000 7fe867400640 1 -- 172.21.15.71:0/2731150186 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe86001a070 con 0x7fe87006efa0 2024-09-16T07:23:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.826+0000 7fe867400640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe850078850 0x7fe85007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.826+0000 7fe867400640 1 -- 172.21.15.71:0/2731150186 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe860099b30 con 0x7fe87006efa0 2024-09-16T07:23:51.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.827+0000 7fe86ea00640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe850078850 0x7fe85007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:51.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.827+0000 7fe86ea00640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe850078850 0x7fe85007ad10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8701523a0 tx=0x7fe85c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:51.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:51.831+0000 7fe867400640 1 -- 172.21.15.71:0/2731150186 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe860061ac0 con 0x7fe87006efa0 2024-09-16T07:23:52.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.014+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe8701e6eb0 con 0x7fe87006efa0 2024-09-16T07:23:52.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.016+0000 7fe867400640 1 -- 172.21.15.71:0/2731150186 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fe8600664f0 con 0x7fe87006efa0 2024-09-16T07:23:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe850078850 msgr2=0x7fe85007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe850078850 0x7fe85007ad10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8701523a0 tx=0x7fe85c005f90 comp rx=0 tx=0).stop 2024-09-16T07:23:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87006efa0 msgr2=0x7fe870151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87006efa0 0x7fe870151660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c3c0 tx=0x7fe86000c880 comp rx=0 tx=0).stop 2024-09-16T07:23:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 shutdown_connections 2024-09-16T07:23:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe850078850 0x7fe85007ad10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870145750 0x7fe870140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8700cccd0 0x7fe8701e8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 --2- 172.21.15.71:0/2731150186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87006efa0 0x7fe870151660 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 >> 172.21.15.71:0/2731150186 conn(0x7fe8700b1840 msgr2=0x7fe8701e73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:52.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 shutdown_connections 2024-09-16T07:23:52.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.018+0000 7fe876200640 1 -- 172.21.15.71:0/2731150186 wait complete. 2024-09-16T07:23:52.035 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:23:52.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.194+0000 7f9e91600640 1 -- 172.21.15.71:0/2041438762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e70004800 msgr2=0x7f9e70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.194+0000 7f9e91600640 1 --2- 172.21.15.71:0/2041438762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e70004800 0x7f9e70004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e74006f70 tx=0x7f9e70005830 comp rx=0 tx=0).stop 2024-09-16T07:23:52.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.195+0000 7f9e91600640 1 -- 172.21.15.71:0/2041438762 shutdown_connections 2024-09-16T07:23:52.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.195+0000 7f9e91600640 1 --2- 172.21.15.71:0/2041438762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e70004800 0x7f9e70004c80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.195+0000 7f9e91600640 1 -- 172.21.15.71:0/2041438762 >> 172.21.15.71:0/2041438762 conn(0x7f9e8c0b1840 msgr2=0x7f9e8c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:52.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.195+0000 7f9e91600640 1 -- 172.21.15.71:0/2041438762 shutdown_connections 2024-09-16T07:23:52.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.195+0000 7f9e91600640 1 -- 172.21.15.71:0/2041438762 wait complete. 2024-09-16T07:23:52.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.196+0000 7f9e91600640 1 Processor -- start 2024-09-16T07:23:52.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.196+0000 7f9e91600640 1 -- start start 2024-09-16T07:23:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.197+0000 7f9e91600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e70004800 0x7f9e8c13ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.197+0000 7f9e91600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8c15e6b0 0x7f9e8c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.197+0000 7f9e91600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c15a8a0 0x7f9e8c143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.197+0000 7f9e91600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c0737d0 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.197+0000 7f9e91600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c073650 con 0x7f9e8c15e6b0 2024-09-16T07:23:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.197+0000 7f9e91600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8c0cc5c0 con 0x7f9e70004800 2024-09-16T07:23:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c15a8a0 0x7f9e8c143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c15a8a0 0x7f9e8c143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50954/0 (socket says 172.21.15.71:50954) 2024-09-16T07:23:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8aa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e70004800 0x7f9e8c13ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 -- 172.21.15.71:0/505998236 learned_addr learned my addr 172.21.15.71:0/505998236 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e83e00640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8c15e6b0 0x7f9e8c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 -- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e70004800 msgr2=0x7f9e8c13ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e70004800 0x7f9e8c13ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 -- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8c15e6b0 msgr2=0x7f9e8c0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8c15e6b0 0x7f9e8c0b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 -- 172.21.15.71:0/505998236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8c0b3180 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.198+0000 7f9e8b400640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c15a8a0 0x7f9e8c143c90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0af750 tx=0x7f9e8400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.199+0000 7f9e82a00640 1 -- 172.21.15.71:0/505998236 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84003d80 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.199+0000 7f9e82a00640 1 -- 172.21.15.71:0/505998236 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e84004b20 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.199+0000 7f9e82a00640 1 -- 172.21.15.71:0/505998236 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84004050 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.199+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8c0b3450 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.199+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8c0b3a10 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.200+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8c15f9f0 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.201+0000 7f9e82a00640 1 -- 172.21.15.71:0/505998236 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9e840041f0 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.202+0000 7f9e82a00640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9e68078850 0x7f9e6807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.202+0000 7f9e82a00640 1 -- 172.21.15.71:0/505998236 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9e8409afd0 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.202+0000 7f9e8aa00640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9e68078850 0x7f9e6807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.203+0000 7f9e8aa00640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9e68078850 0x7f9e6807ad10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c006fd0 tx=0x7f9e7c006b10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:52.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.206+0000 7f9e82a00640 1 -- 172.21.15.71:0/505998236 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9e84062ff0 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.392+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f9e8c0b0d10 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.392+0000 7f9e82a00640 1 -- 172.21.15.71:0/505998236 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v28) ==== 116+0+2 (secure 0 0 0) 0x7f9e84067a20 con 0x7f9e8c15a8a0 2024-09-16T07:23:52.393 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9e68078850 msgr2=0x7f9e6807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9e68078850 0x7f9e6807ad10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c006fd0 tx=0x7f9e7c006b10 comp rx=0 tx=0).stop 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c15a8a0 msgr2=0x7f9e8c143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c15a8a0 0x7f9e8c143c90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0af750 tx=0x7f9e8400f790 comp rx=0 tx=0).stop 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 shutdown_connections 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9e68078850 0x7f9e6807ad10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c15a8a0 0x7f9e8c143c90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8c15e6b0 0x7f9e8c0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 --2- 172.21.15.71:0/505998236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e70004800 0x7f9e8c13ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 >> 172.21.15.71:0/505998236 conn(0x7f9e8c0b1840 msgr2=0x7f9e8c0aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 shutdown_connections 2024-09-16T07:23:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.395+0000 7f9e91600640 1 -- 172.21.15.71:0/505998236 wait complete. 2024-09-16T07:23:52.411 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:23:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.570+0000 7fef72800640 1 -- 172.21.15.71:0/1410165180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef6c1437c0 msgr2=0x7fef6c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.570+0000 7fef72800640 1 --2- 172.21.15.71:0/1410165180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef6c1437c0 0x7fef6c0cdad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00a970 tx=0x7fef5c02b8d0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.570+0000 7fef72800640 1 -- 172.21.15.71:0/1410165180 shutdown_connections 2024-09-16T07:23:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.570+0000 7fef72800640 1 --2- 172.21.15.71:0/1410165180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef6c1437c0 0x7fef6c0cdad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.570+0000 7fef72800640 1 -- 172.21.15.71:0/1410165180 >> 172.21.15.71:0/1410165180 conn(0x7fef6c0b1840 msgr2=0x7fef6c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.570+0000 7fef72800640 1 -- 172.21.15.71:0/1410165180 shutdown_connections 2024-09-16T07:23:52.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.570+0000 7fef72800640 1 -- 172.21.15.71:0/1410165180 wait complete. 2024-09-16T07:23:52.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.571+0000 7fef72800640 1 Processor -- start 2024-09-16T07:23:52.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.572+0000 7fef72800640 1 -- start start 2024-09-16T07:23:52.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.572+0000 7fef72800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef6c0b45d0 0x7fef6c1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.572+0000 7fef72800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef6c0b4bd0 0x7fef6c1f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.572+0000 7fef72800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c1437c0 0x7fef6c13b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef72800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c06f0f0 con 0x7fef6c1437c0 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef72800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c06ef70 con 0x7fef6c0b4bd0 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef72800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c06f270 con 0x7fef6c0b45d0 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c1437c0 0x7fef6c13b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c1437c0 0x7fef6c13b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50960/0 (socket says 172.21.15.71:50960) 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6be00640 1 -- 172.21.15.71:0/2918962505 learned_addr learned my addr 172.21.15.71:0/2918962505 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6b400640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef6c0b45d0 0x7fef6c1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6aa00640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef6c0b4bd0 0x7fef6c1f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6be00640 1 -- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef6c0b45d0 msgr2=0x7fef6c1f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6be00640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef6c0b45d0 0x7fef6c1f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6be00640 1 -- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef6c0b4bd0 msgr2=0x7fef6c1f8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6be00640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef6c0b4bd0 0x7fef6c1f8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.573+0000 7fef6be00640 1 -- 172.21.15.71:0/2918962505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c1f78c0 con 0x7fef6c1437c0 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.574+0000 7fef6be00640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c1437c0 0x7fef6c13b2b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fef6400cba0 tx=0x7fef6400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.574+0000 7fef63400640 1 -- 172.21.15.71:0/2918962505 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64016020 con 0x7fef6c1437c0 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.574+0000 7fef63400640 1 -- 172.21.15.71:0/2918962505 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef6400ae50 con 0x7fef6c1437c0 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.574+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c15a970 con 0x7fef6c1437c0 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.574+0000 7fef63400640 1 -- 172.21.15.71:0/2918962505 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64012680 con 0x7fef6c1437c0 2024-09-16T07:23:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.574+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1f5e10 con 0x7fef6c1437c0 2024-09-16T07:23:52.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.575+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c0cdad0 con 0x7fef6c1437c0 2024-09-16T07:23:52.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.576+0000 7fef63400640 1 -- 172.21.15.71:0/2918962505 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fef64019020 con 0x7fef6c1437c0 2024-09-16T07:23:52.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.577+0000 7fef63400640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef4c078850 0x7fef4c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.577+0000 7fef63400640 1 -- 172.21.15.71:0/2918962505 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fef6409af60 con 0x7fef6c1437c0 2024-09-16T07:23:52.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.577+0000 7fef6b400640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef4c078850 0x7fef4c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.578+0000 7fef6b400640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef4c078850 0x7fef4c07ad10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef5c036000 tx=0x7fef5c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:52.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.581+0000 7fef63400640 1 -- 172.21.15.71:0/2918962505 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fef64020020 con 0x7fef6c1437c0 2024-09-16T07:23:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.766+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fef6c06e990 con 0x7fef6c1437c0 2024-09-16T07:23:52.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.770+0000 7fef63400640 1 -- 172.21.15.71:0/2918962505 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7fef64062e60 con 0x7fef6c1437c0 2024-09-16T07:23:52.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef4c078850 msgr2=0x7fef4c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef4c078850 0x7fef4c07ad10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef5c036000 tx=0x7fef5c004730 comp rx=0 tx=0).stop 2024-09-16T07:23:52.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c1437c0 msgr2=0x7fef6c13b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c1437c0 0x7fef6c13b2b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fef6400cba0 tx=0x7fef6400a430 comp rx=0 tx=0).stop 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 shutdown_connections 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef4c078850 0x7fef4c07ad10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef6c1437c0 0x7fef6c13b2b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef6c0b4bd0 0x7fef6c1f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 --2- 172.21.15.71:0/2918962505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef6c0b45d0 0x7fef6c1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.772+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 >> 172.21.15.71:0/2918962505 conn(0x7fef6c0b1840 msgr2=0x7fef6c15d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.773+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 shutdown_connections 2024-09-16T07:23:52.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.773+0000 7fef72800640 1 -- 172.21.15.71:0/2918962505 wait complete. 2024-09-16T07:23:52.788 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group1 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 -- 172.21.15.71:0/2327976437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f406efa0 msgr2=0x7fc8f40cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 --2- 172.21.15.71:0/2327976437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f406efa0 0x7fc8f40cc5c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40cc550 tx=0x7fc8e002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 -- 172.21.15.71:0/2327976437 shutdown_connections 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 --2- 172.21.15.71:0/2327976437 >> v2:172.21.15.177:3300/0 conn(0x7fc8f40ccb00 0x7fc8f415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 --2- 172.21.15.71:0/2327976437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f406efa0 0x7fc8f40cc5c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 --2- 172.21.15.71:0/2327976437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f4145750 0x7fc8f4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 -- 172.21.15.71:0/2327976437 >> 172.21.15.71:0/2327976437 conn(0x7fc8f40b1840 msgr2=0x7fc8f40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 -- 172.21.15.71:0/2327976437 shutdown_connections 2024-09-16T07:23:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.948+0000 7fc8fb200640 1 -- 172.21.15.71:0/2327976437 wait complete. 2024-09-16T07:23:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.950+0000 7fc8fb200640 1 Processor -- start 2024-09-16T07:23:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.950+0000 7fc8fb200640 1 -- start start 2024-09-16T07:23:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8fb200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f406efa0 0x7fc8f40b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8fb200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40ccb00 0x7fc8f415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8fb200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4145750 0x7fc8f40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8fb200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41600b0 con 0x7fc8f4145750 2024-09-16T07:23:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8fb200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f415ff30 con 0x7fc8f406efa0 2024-09-16T07:23:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8fb200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f4160230 con 0x7fc8f40ccb00 2024-09-16T07:23:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f9200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4145750 0x7fc8f40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f9200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4145750 0x7fc8f40b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50992/0 (socket says 172.21.15.71:50992) 2024-09-16T07:23:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f9200640 1 -- 172.21.15.71:0/3692403117 learned_addr learned my addr 172.21.15.71:0/3692403117 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f3400640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40ccb00 0x7fc8f415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f3e00640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f406efa0 0x7fc8f40b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f9200640 1 -- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40ccb00 msgr2=0x7fc8f415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f9200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40ccb00 0x7fc8f415f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f9200640 1 -- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f406efa0 msgr2=0x7fc8f40b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f9200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f406efa0 0x7fc8f40b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.951+0000 7fc8f9200640 1 -- 172.21.15.71:0/3692403117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f41f8260 con 0x7fc8f4145750 2024-09-16T07:23:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.952+0000 7fc8f9200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4145750 0x7fc8f40b4590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00cce0 tx=0x7fc8ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.952+0000 7fc8ebe00640 1 -- 172.21.15.71:0/3692403117 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec00f450 con 0x7fc8f4145750 2024-09-16T07:23:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.952+0000 7fc8ebe00640 1 -- 172.21.15.71:0/3692403117 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8ec00f6e0 con 0x7fc8f4145750 2024-09-16T07:23:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.952+0000 7fc8ebe00640 1 -- 172.21.15.71:0/3692403117 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec00fb30 con 0x7fc8f4145750 2024-09-16T07:23:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.952+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f41f8540 con 0x7fc8f4145750 2024-09-16T07:23:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.952+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41f8b00 con 0x7fc8f4145750 2024-09-16T07:23:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.953+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f4160710 con 0x7fc8f4145750 2024-09-16T07:23:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.954+0000 7fc8ebe00640 1 -- 172.21.15.71:0/3692403117 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc8ec00fcd0 con 0x7fc8f4145750 2024-09-16T07:23:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.955+0000 7fc8ebe00640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d4078850 0x7fc8d407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.955+0000 7fc8ebe00640 1 -- 172.21.15.71:0/3692403117 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc8ec09b360 con 0x7fc8f4145750 2024-09-16T07:23:52.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.956+0000 7fc8f3e00640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d4078850 0x7fc8d407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:52.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.956+0000 7fc8f3e00640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d4078850 0x7fc8d407ad10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4005fd0 tx=0x7fc8e400a250 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:52.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:52.959+0000 7fc8ebe00640 1 -- 172.21.15.71:0/3692403117 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc8ec014030 con 0x7fc8f4145750 2024-09-16T07:23:53.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.168+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fc8f40b0900 con 0x7fc8d4078850 2024-09-16T07:23:53.175 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:23:53.175+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:23:53.175 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:23:53.175+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:23:53.175 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:23:53.175+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:23:53.175 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:23:53.175+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:23:53.175 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:23:53.175+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:23:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.195+0000 7fc8ebe00640 1 -- 172.21.15.71:0/3692403117 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc8f40b0900 con 0x7fc8d4078850 2024-09-16T07:23:53.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d4078850 msgr2=0x7fc8d407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d4078850 0x7fc8d407ad10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4005fd0 tx=0x7fc8e400a250 comp rx=0 tx=0).stop 2024-09-16T07:23:53.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4145750 msgr2=0x7fc8f40b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4145750 0x7fc8f40b4590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00cce0 tx=0x7fc8ec00a620 comp rx=0 tx=0).stop 2024-09-16T07:23:53.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 shutdown_connections 2024-09-16T07:23:53.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d4078850 0x7fc8d407ad10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4145750 0x7fc8f40b4590 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40ccb00 0x7fc8f415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 --2- 172.21.15.71:0/3692403117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f406efa0 0x7fc8f40b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 >> 172.21.15.71:0/3692403117 conn(0x7fc8f40b1840 msgr2=0x7fc8f40cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:53.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 shutdown_connections 2024-09-16T07:23:53.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.197+0000 7fc8fb200640 1 -- 172.21.15.71:0/3692403117 wait complete. 2024-09-16T07:23:53.212 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 group1 --mode=777 2024-09-16T07:23:53.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.377+0000 7f8cc3000640 1 -- 172.21.15.71:0/1812587643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc15cff0 msgr2=0x7f8cbc06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.377+0000 7f8cc3000640 1 --2- 172.21.15.71:0/1812587643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc15cff0 0x7f8cbc06e6f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc075110 tx=0x7f8ca402a540 comp rx=0 tx=0).stop 2024-09-16T07:23:53.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.377+0000 7f8cc3000640 1 -- 172.21.15.71:0/1812587643 shutdown_connections 2024-09-16T07:23:53.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.377+0000 7f8cc3000640 1 --2- 172.21.15.71:0/1812587643 >> v2:172.21.15.177:3300/0 conn(0x7f8cbc06f220 0x7f8cbc06f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.377+0000 7f8cc3000640 1 --2- 172.21.15.71:0/1812587643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc15cff0 0x7f8cbc06e6f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.377+0000 7f8cc3000640 1 -- 172.21.15.71:0/1812587643 >> 172.21.15.71:0/1812587643 conn(0x7f8cbc0b18e0 msgr2=0x7f8cbc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:53.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.377+0000 7f8cc3000640 1 -- 172.21.15.71:0/1812587643 shutdown_connections 2024-09-16T07:23:53.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.377+0000 7f8cc3000640 1 -- 172.21.15.71:0/1812587643 wait complete. 2024-09-16T07:23:53.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.378+0000 7f8cc3000640 1 Processor -- start 2024-09-16T07:23:53.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.379+0000 7f8cc3000640 1 -- start start 2024-09-16T07:23:53.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.379+0000 7f8cc3000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cbc06f220 0x7f8cbc1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:53.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.379+0000 7f8cc3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc159aa0 0x7f8cbc0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.379+0000 7f8cc3000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cbc15cff0 0x7f8cbc0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.379+0000 7f8cc3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc0cc520 con 0x7f8cbc159aa0 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.379+0000 7f8cc3000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc0cc3a0 con 0x7f8cbc06f220 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.379+0000 7f8cc3000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8cbc0cc6a0 con 0x7f8cbc15cff0 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc159aa0 0x7f8cbc0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc159aa0 0x7f8cbc0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51022/0 (socket says 172.21.15.71:51022) 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 -- 172.21.15.71:0/3585876579 learned_addr learned my addr 172.21.15.71:0/3585876579 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 -- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cbc15cff0 msgr2=0x7f8cbc0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cbc15cff0 0x7f8cbc0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 -- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cbc06f220 msgr2=0x7f8cbc1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cbbe00640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cbc06f220 0x7f8cbc1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cbc06f220 0x7f8cbc1f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 -- 172.21.15.71:0/3585876579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc0b3fb0 con 0x7f8cbc159aa0 2024-09-16T07:23:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cbbe00640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cbc06f220 0x7f8cbc1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:23:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.380+0000 7f8cb3e00640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc159aa0 0x7f8cbc0cafe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009920 tx=0x7f8ca4033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.381+0000 7f8cb8c00640 1 -- 172.21.15.71:0/3585876579 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4031d60 con 0x7f8cbc159aa0 2024-09-16T07:23:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.381+0000 7f8cb8c00640 1 -- 172.21.15.71:0/3585876579 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ca4005020 con 0x7f8cbc159aa0 2024-09-16T07:23:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.381+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc0aced0 con 0x7f8cbc159aa0 2024-09-16T07:23:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.381+0000 7f8cb8c00640 1 -- 172.21.15.71:0/3585876579 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4005470 con 0x7f8cbc159aa0 2024-09-16T07:23:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.381+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc0ad490 con 0x7f8cbc159aa0 2024-09-16T07:23:53.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.382+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cbc069360 con 0x7f8cbc159aa0 2024-09-16T07:23:53.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.383+0000 7f8cb8c00640 1 -- 172.21.15.71:0/3585876579 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8ca40047a0 con 0x7f8cbc159aa0 2024-09-16T07:23:53.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.384+0000 7f8cb8c00640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c98078850 0x7f8c9807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:53.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.384+0000 7f8cb8c00640 1 -- 172.21.15.71:0/3585876579 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8ca40bbb50 con 0x7f8cbc159aa0 2024-09-16T07:23:53.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.384+0000 7f8cbbe00640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c98078850 0x7f8c9807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:53.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.385+0000 7f8cbbe00640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c98078850 0x7f8c9807ad10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8cac007fd0 tx=0x7f8cac009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:53.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.388+0000 7f8cb8c00640 1 -- 172.21.15.71:0/3585876579 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ca403f030 con 0x7f8cbc159aa0 2024-09-16T07:23:53.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.599+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f8cbc0b4c80 con 0x7f8c98078850 2024-09-16T07:23:53.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.632+0000 7f8cb8c00640 1 -- 172.21.15.71:0/3585876579 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8cac001340 con 0x7f8c98078850 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c98078850 msgr2=0x7f8c9807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c98078850 0x7f8c9807ad10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8cac007fd0 tx=0x7f8cac009040 comp rx=0 tx=0).stop 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc159aa0 msgr2=0x7f8cbc0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc159aa0 0x7f8cbc0cafe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009920 tx=0x7f8ca4033d60 comp rx=0 tx=0).stop 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 shutdown_connections 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c98078850 0x7f8c9807ad10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cbc15cff0 0x7f8cbc0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cbc159aa0 0x7f8cbc0cafe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 --2- 172.21.15.71:0/3585876579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cbc06f220 0x7f8cbc1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 >> 172.21.15.71:0/3585876579 conn(0x7f8cbc0b18e0 msgr2=0x7f8cbc15a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 shutdown_connections 2024-09-16T07:23:53.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.634+0000 7f8cc3000640 1 -- 172.21.15.71:0/3585876579 wait complete. 2024-09-16T07:23:53.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-16T07:23:53.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.809+0000 7fc291a00640 1 -- 172.21.15.71:0/56138419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15e710 msgr2=0x7fc28c0d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.809+0000 7fc291a00640 1 --2- 172.21.15.71:0/56138419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15e710 0x7fc28c0d4a20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005fd0 tx=0x7fc27c02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.809+0000 7fc291a00640 1 -- 172.21.15.71:0/56138419 shutdown_connections 2024-09-16T07:23:53.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.809+0000 7fc291a00640 1 --2- 172.21.15.71:0/56138419 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc28c0d0fe0 0x7fc28c15ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.809+0000 7fc291a00640 1 --2- 172.21.15.71:0/56138419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15e710 0x7fc28c0d4a20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.809+0000 7fc291a00640 1 -- 172.21.15.71:0/56138419 >> 172.21.15.71:0/56138419 conn(0x7fc28c0d3f00 msgr2=0x7fc28c0d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:53.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.810+0000 7fc291a00640 1 -- 172.21.15.71:0/56138419 shutdown_connections 2024-09-16T07:23:53.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.810+0000 7fc291a00640 1 -- 172.21.15.71:0/56138419 wait complete. 2024-09-16T07:23:53.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.811+0000 7fc291a00640 1 Processor -- start 2024-09-16T07:23:53.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.811+0000 7fc291a00640 1 -- start start 2024-09-16T07:23:53.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.812+0000 7fc291a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc28c0d0fe0 0x7fc28c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:53.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.812+0000 7fc291a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc28c15e710 0x7fc28c0b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:53.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.812+0000 7fc291a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15eaf0 0x7fc28c0b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.812+0000 7fc291a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0d1e60 con 0x7fc28c15eaf0 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.812+0000 7fc291a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c0d1ce0 con 0x7fc28c0d0fe0 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.812+0000 7fc291a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c0d1fe0 con 0x7fc28c15e710 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15eaf0 0x7fc28c0b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15eaf0 0x7fc28c0b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51032/0 (socket says 172.21.15.71:51032) 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 -- 172.21.15.71:0/2754478040 learned_addr learned my addr 172.21.15.71:0/2754478040 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc28be00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc28c0d0fe0 0x7fc28c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc28b400640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc28c15e710 0x7fc28c0b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 -- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc28c15e710 msgr2=0x7fc28c0b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc28c15e710 0x7fc28c0b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 -- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc28c0d0fe0 msgr2=0x7fc28c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc28c0d0fe0 0x7fc28c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:53.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 -- 172.21.15.71:0/2754478040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c1f8e00 con 0x7fc28c15eaf0 2024-09-16T07:23:53.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.813+0000 7fc291000640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15eaf0 0x7fc28c0b4c30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc28400b9e0 tx=0x7fc28400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:53.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.814+0000 7fc283e00640 1 -- 172.21.15.71:0/2754478040 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284019070 con 0x7fc28c15eaf0 2024-09-16T07:23:53.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.814+0000 7fc283e00640 1 -- 172.21.15.71:0/2754478040 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc28400cbd0 con 0x7fc28c15eaf0 2024-09-16T07:23:53.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.814+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c1f4df0 con 0x7fc28c15eaf0 2024-09-16T07:23:53.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.814+0000 7fc283e00640 1 -- 172.21.15.71:0/2754478040 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284014440 con 0x7fc28c15eaf0 2024-09-16T07:23:53.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.814+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1f53b0 con 0x7fc28c15eaf0 2024-09-16T07:23:53.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.815+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c15ef80 con 0x7fc28c15eaf0 2024-09-16T07:23:53.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.816+0000 7fc283e00640 1 -- 172.21.15.71:0/2754478040 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc28401c070 con 0x7fc28c15eaf0 2024-09-16T07:23:53.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.817+0000 7fc283e00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26c078850 0x7fc26c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:23:53.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.817+0000 7fc283e00640 1 -- 172.21.15.71:0/2754478040 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc28409a660 con 0x7fc28c15eaf0 2024-09-16T07:23:53.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.817+0000 7fc28be00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26c078850 0x7fc26c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:23:53.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.818+0000 7fc28be00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26c078850 0x7fc26c07ad10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc27c033000 tx=0x7fc27c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:23:53.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:53.821+0000 7fc283e00640 1 -- 172.21.15.71:0/2754478040 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc284062550 con 0x7fc28c15eaf0 2024-09-16T07:23:54.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.031+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fc28c1f5580 con 0x7fc26c078850 2024-09-16T07:23:54.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.038+0000 7fc283e00640 1 -- 172.21.15.71:0/2754478040 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7fc28c1f5580 con 0x7fc26c078850 2024-09-16T07:23:54.037 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03 2024-09-16T07:23:54.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.039+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26c078850 msgr2=0x7fc26c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:54.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.039+0000 7fc291a00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26c078850 0x7fc26c07ad10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc27c033000 tx=0x7fc27c004730 comp rx=0 tx=0).stop 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.039+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15eaf0 msgr2=0x7fc28c0b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.039+0000 7fc291a00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15eaf0 0x7fc28c0b4c30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc28400b9e0 tx=0x7fc28400bea0 comp rx=0 tx=0).stop 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.040+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 shutdown_connections 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.040+0000 7fc291a00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26c078850 0x7fc26c07ad10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.040+0000 7fc291a00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc28c15eaf0 0x7fc28c0b4c30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.040+0000 7fc291a00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc28c15e710 0x7fc28c0b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.040+0000 7fc291a00640 1 --2- 172.21.15.71:0/2754478040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc28c0d0fe0 0x7fc28c15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.040+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 >> 172.21.15.71:0/2754478040 conn(0x7fc28c0d3f00 msgr2=0x7fc28c15f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.040+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 shutdown_connections 2024-09-16T07:23:54.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:23:54.040+0000 7fc291a00640 1 -- 172.21.15.71:0/2754478040 wait complete. 2024-09-16T07:23:54.054 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03 2024-09-16T07:23:54.054 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:23:54.055 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:23:54.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:23:54.090 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:24:01.400 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:24:01.400 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:24:01.400 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.22293 s, 149 MB/s 2024-09-16T07:24:01.401 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:24:01.401 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:24:01.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:24:01.437 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:24:08.875 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:24:08.876 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:24:08.876 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.36158 s, 146 MB/s 2024-09-16T07:24:08.878 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:24:08.878 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:24:08.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:24:08.914 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:24:13.624 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:24:13.629 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:24:16.408 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:24:16.408 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:24:16.408 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.41721 s, 145 MB/s 2024-09-16T07:24:16.409 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:24:16.410 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:24:16.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:24:16.446 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:24:24.036 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:24:24.037 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:24:24.037 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.5135 s, 143 MB/s 2024-09-16T07:24:24.038 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:24:24.038 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:24:24.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:24:24.073 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:24:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:24:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:24:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.54148 s, 142 MB/s 2024-09-16T07:24:31.693 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:24:31.693 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:24:31.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:24:31.730 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:24:39.426 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:24:39.426 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:24:39.426 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.61867 s, 141 MB/s 2024-09-16T07:24:39.428 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:24:39.428 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:24:39.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:24:39.463 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:24:43.720 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:24:43.725 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:24:46.913 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:24:46.913 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:24:46.913 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.37356 s, 146 MB/s 2024-09-16T07:24:46.914 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:24:46.914 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:24:46.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:24:46.951 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:24:54.550 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:24:54.550 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:24:54.550 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.52263 s, 143 MB/s 2024-09-16T07:24:54.552 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:24:54.552 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:24:54.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:24:54.590 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:25:02.113 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:25:02.113 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:25:02.113 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.44636 s, 144 MB/s 2024-09-16T07:25:02.114 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:02.114 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:02.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:02.150 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:25:09.584 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:25:09.585 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:25:09.585 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.35819 s, 146 MB/s 2024-09-16T07:25:09.586 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 group1 2024-09-16T07:25:09.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.751+0000 7fb424e00640 1 -- 172.21.15.71:0/4071524515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb420161760 msgr2=0x7fb4200ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.751+0000 7fb424e00640 1 --2- 172.21.15.71:0/4071524515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb420161760 0x7fb4200ce4a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb4200cd800 tx=0x7fb41002aac0 comp rx=0 tx=0).stop 2024-09-16T07:25:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.751+0000 7fb424e00640 1 -- 172.21.15.71:0/4071524515 shutdown_connections 2024-09-16T07:25:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.751+0000 7fb424e00640 1 --2- 172.21.15.71:0/4071524515 >> v2:172.21.15.177:3300/0 conn(0x7fb4200cec60 0x7fb4200cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.751+0000 7fb424e00640 1 --2- 172.21.15.71:0/4071524515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb420161760 0x7fb4200ce4a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.751+0000 7fb424e00640 1 -- 172.21.15.71:0/4071524515 >> 172.21.15.71:0/4071524515 conn(0x7fb4200b1840 msgr2=0x7fb4200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.751+0000 7fb424e00640 1 -- 172.21.15.71:0/4071524515 shutdown_connections 2024-09-16T07:25:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.751+0000 7fb424e00640 1 -- 172.21.15.71:0/4071524515 wait complete. 2024-09-16T07:25:09.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.752+0000 7fb424e00640 1 Processor -- start 2024-09-16T07:25:09.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.753+0000 7fb424e00640 1 -- start start 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.753+0000 7fb424e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4200af190 0x7fb42015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.753+0000 7fb424e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cec60 0x7fb4201f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.753+0000 7fb424e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420161760 0x7fb4201f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.753+0000 7fb424e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb420162310 con 0x7fb4200af190 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.753+0000 7fb424e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb420162190 con 0x7fb4200cec60 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.753+0000 7fb424e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb420162490 con 0x7fb420161760 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4200af190 0x7fb42015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4200af190 0x7fb42015b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54422/0 (socket says 172.21.15.71:54422) 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 -- 172.21.15.71:0/3825534230 learned_addr learned my addr 172.21.15.71:0/3825534230 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:09.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb415400640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cec60 0x7fb4201f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:09.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41e800640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420161760 0x7fb4201f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:09.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 -- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420161760 msgr2=0x7fb4201f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:09.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420161760 0x7fb4201f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:09.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 -- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cec60 msgr2=0x7fb4201f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:09.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cec60 0x7fb4201f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:09.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 -- 172.21.15.71:0/3825534230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42015b660 con 0x7fb4200af190 2024-09-16T07:25:09.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.754+0000 7fb41de00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4200af190 0x7fb42015b120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb410009bf0 tx=0x7fb410003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:09.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.755+0000 7fb416a00640 1 -- 172.21.15.71:0/3825534230 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410044070 con 0x7fb4200af190 2024-09-16T07:25:09.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.755+0000 7fb416a00640 1 -- 172.21.15.71:0/3825534230 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb410004250 con 0x7fb4200af190 2024-09-16T07:25:09.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.755+0000 7fb416a00640 1 -- 172.21.15.71:0/3825534230 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410004680 con 0x7fb4200af190 2024-09-16T07:25:09.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.755+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42015b8e0 con 0x7fb4200af190 2024-09-16T07:25:09.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.755+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42015bea0 con 0x7fb4200af190 2024-09-16T07:25:09.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.756+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4201629f0 con 0x7fb4200af190 2024-09-16T07:25:09.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.757+0000 7fb416a00640 1 -- 172.21.15.71:0/3825534230 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 7) ==== 101287+0+0 (secure 0 0 0) 0x7fb41002c8c0 con 0x7fb4200af190 2024-09-16T07:25:09.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.758+0000 7fb416a00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3fc078850 0x7fb3fc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:09.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.758+0000 7fb416a00640 1 -- 172.21.15.71:0/3825534230 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fb4100baf40 con 0x7fb4200af190 2024-09-16T07:25:09.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.758+0000 7fb415400640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3fc078850 0x7fb3fc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:09.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.759+0000 7fb415400640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3fc078850 0x7fb3fc07ad10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb42015d370 tx=0x7fb408009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:09.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.762+0000 7fb416a00640 1 -- 172.21.15.71:0/3825534230 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb42015b660 con 0x7fb4200af190 2024-09-16T07:25:09.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:09.977+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fb4200b2de0 con 0x7fb3fc078850 2024-09-16T07:25:09.982 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:09.982+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:09.982 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:09.982+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:09.982 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:09.982+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:09.982 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:09.982+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:09.982 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:09.982+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:10.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.004+0000 7fb416a00640 1 -- 172.21.15.71:0/3825534230 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb4200b2de0 con 0x7fb3fc078850 2024-09-16T07:25:10.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3fc078850 msgr2=0x7fb3fc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:10.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3fc078850 0x7fb3fc07ad10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb42015d370 tx=0x7fb408009040 comp rx=0 tx=0).stop 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4200af190 msgr2=0x7fb42015b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4200af190 0x7fb42015b120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb410009bf0 tx=0x7fb410003f10 comp rx=0 tx=0).stop 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 shutdown_connections 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3fc078850 0x7fb3fc07ad10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420161760 0x7fb4201f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cec60 0x7fb4201f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 --2- 172.21.15.71:0/3825534230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4200af190 0x7fb42015b120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.006+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 >> 172.21.15.71:0/3825534230 conn(0x7fb4200b1840 msgr2=0x7fb420073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.007+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 shutdown_connections 2024-09-16T07:25:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.007+0000 7fb424e00640 1 -- 172.21.15.71:0/3825534230 wait complete. 2024-09-16T07:25:10.022 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-16T07:25:10.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.186+0000 7f0e16000640 1 -- 172.21.15.71:0/2069843081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1006e590 msgr2=0x7f0e1006fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:10.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.186+0000 7f0e16000640 1 --2- 172.21.15.71:0/2069843081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1006e590 0x7f0e1006fe40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005fd0 tx=0x7f0dfc02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.186+0000 7f0e16000640 1 -- 172.21.15.71:0/2069843081 shutdown_connections 2024-09-16T07:25:10.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.186+0000 7f0e16000640 1 --2- 172.21.15.71:0/2069843081 >> v2:172.21.15.177:3300/0 conn(0x7f0e10070380 0x7f0e100cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.186+0000 7f0e16000640 1 --2- 172.21.15.71:0/2069843081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1006e590 0x7f0e1006fe40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.186+0000 7f0e16000640 1 -- 172.21.15.71:0/2069843081 >> 172.21.15.71:0/2069843081 conn(0x7f0e100cfa70 msgr2=0x7f0e100cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:10.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.186+0000 7f0e16000640 1 -- 172.21.15.71:0/2069843081 shutdown_connections 2024-09-16T07:25:10.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.186+0000 7f0e16000640 1 -- 172.21.15.71:0/2069843081 wait complete. 2024-09-16T07:25:10.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.188+0000 7f0e16000640 1 Processor -- start 2024-09-16T07:25:10.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.188+0000 7f0e16000640 1 -- start start 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e16000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1006e590 0x7f0e100ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e16000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e10070380 0x7f0e101f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e16000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e1015bda0 0x7f0e10152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e16000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0e100cb910 con 0x7f0e1015bda0 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e16000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0e100708f0 con 0x7f0e10070380 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e16000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0e100cba90 con 0x7f0e1006e590 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e15600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e1015bda0 0x7f0e10152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e10070380 0x7f0e101f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e14c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1006e590 0x7f0e100ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e10070380 0x7f0e101f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:37766/0 (socket says 172.21.15.71:37766) 2024-09-16T07:25:10.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e0fe00640 1 -- 172.21.15.71:0/1767224431 learned_addr learned my addr 172.21.15.71:0/1767224431 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.189+0000 7f0e15600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e1015bda0 0x7f0e10152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54434/0 (socket says 172.21.15.71:54434) 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0fe00640 1 -- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1006e590 msgr2=0x7f0e100ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0fe00640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1006e590 0x7f0e100ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0fe00640 1 -- 172.21.15.71:0/1767224431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e1015bda0 msgr2=0x7f0e10152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0fe00640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e1015bda0 0x7f0e10152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0fe00640 1 -- 172.21.15.71:0/1767224431 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e101f8870 con 0x7f0e10070380 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e15600640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e1015bda0 0x7f0e10152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0fe00640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e10070380 0x7f0e101f7e90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009850 tx=0x7f0dfc0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0cc00640 1 -- 172.21.15.71:0/1767224431 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc03f070 con 0x7f0e10070380 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0cc00640 1 -- 172.21.15.71:0/1767224431 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dfc003ce0 con 0x7f0e10070380 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e101f8b50 con 0x7f0e10070380 2024-09-16T07:25:10.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e0cc00640 1 -- 172.21.15.71:0/1767224431 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc035030 con 0x7f0e10070380 2024-09-16T07:25:10.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.190+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e100cdf90 con 0x7f0e10070380 2024-09-16T07:25:10.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.191+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e101f8d90 con 0x7f0e10070380 2024-09-16T07:25:10.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.193+0000 7f0e0cc00640 1 -- 172.21.15.71:0/1767224431 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 7) ==== 101287+0+0 (secure 0 0 0) 0x7f0dfc042070 con 0x7f0e10070380 2024-09-16T07:25:10.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.194+0000 7f0e0cc00640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0df0078850 0x7f0df007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:10.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.194+0000 7f0e0cc00640 1 -- 172.21.15.71:0/1767224431 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f0dfc0bb4b0 con 0x7f0e10070380 2024-09-16T07:25:10.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.194+0000 7f0e14c00640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0df0078850 0x7f0df007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:10.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.195+0000 7f0e14c00640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0df0078850 0x7f0df007ad10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0e100cd880 tx=0x7f0e04009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:10.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.199+0000 7f0e0cc00640 1 -- 172.21.15.71:0/1767224431 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0e101f8d90 con 0x7f0e10070380 2024-09-16T07:25:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.410+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f0e100cf630 con 0x7f0df0078850 2024-09-16T07:25:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.416+0000 7f0e0cc00640 1 -- 172.21.15.71:0/1767224431 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f0e100cf630 con 0x7f0df0078850 2024-09-16T07:25:10.415 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03 2024-09-16T07:25:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0df0078850 msgr2=0x7f0df007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0df0078850 0x7f0df007ad10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0e100cd880 tx=0x7f0e04009040 comp rx=0 tx=0).stop 2024-09-16T07:25:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e10070380 msgr2=0x7f0e101f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e10070380 0x7f0e101f7e90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009850 tx=0x7f0dfc0035f0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 shutdown_connections 2024-09-16T07:25:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0df0078850 0x7f0df007ad10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e1015bda0 0x7f0e10152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e10070380 0x7f0e101f7e90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 --2- 172.21.15.71:0/1767224431 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1006e590 0x7f0e100ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 >> 172.21.15.71:0/1767224431 conn(0x7f0e100cfa70 msgr2=0x7f0e100737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 shutdown_connections 2024-09-16T07:25:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:10.418+0000 7f0e16000640 1 -- 172.21.15.71:0/1767224431 wait complete. 2024-09-16T07:25:10.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:10.433 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03 2024-09-16T07:25:10.433 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:25:10.472 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03 2024-09-16T07:25:10.472 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T07:25:10.473 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:10.474 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T07:25:12.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:12.475 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03 2024-09-16T07:25:12.475 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:25:12.514 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/group1/sv1/25069363-59ef-463e-8d37-79e0d6b10c03 2024-09-16T07:25:12.514 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="10737418240" 2024-09-16T07:25:12.514 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:12.515 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-16T07:25:12.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 --group-name group1 2024-09-16T07:25:12.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.727+0000 7ff2ada00640 1 -- 172.21.15.71:0/3115034407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a80cf3d0 msgr2=0x7ff2a80d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:12.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.727+0000 7ff2ada00640 1 --2- 172.21.15.71:0/3115034407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a80cf3d0 0x7ff2a80d0600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0005fd0 tx=0x7ff2a002aa90 comp rx=0 tx=0).stop 2024-09-16T07:25:12.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.727+0000 7ff2ada00640 1 -- 172.21.15.71:0/3115034407 shutdown_connections 2024-09-16T07:25:12.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.727+0000 7ff2ada00640 1 --2- 172.21.15.71:0/3115034407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a80cf3d0 0x7ff2a80d0600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:12.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.727+0000 7ff2ada00640 1 --2- 172.21.15.71:0/3115034407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a8152e30 0x7ff2a80d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:12.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.727+0000 7ff2ada00640 1 -- 172.21.15.71:0/3115034407 >> 172.21.15.71:0/3115034407 conn(0x7ff2a806cf90 msgr2=0x7ff2a806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:12.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.727+0000 7ff2ada00640 1 -- 172.21.15.71:0/3115034407 shutdown_connections 2024-09-16T07:25:12.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.727+0000 7ff2ada00640 1 -- 172.21.15.71:0/3115034407 wait complete. 2024-09-16T07:25:12.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.729+0000 7ff2ada00640 1 Processor -- start 2024-09-16T07:25:12.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.729+0000 7ff2ada00640 1 -- start start 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2ada00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a80cd210 0x7ff2a80b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2ada00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a80cf3d0 0x7ff2a80af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2ada00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a8152e30 0x7ff2a81e8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2ada00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a806f940 con 0x7ff2a8152e30 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2ada00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a806f7c0 con 0x7ff2a80cd210 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2ada00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a806fac0 con 0x7ff2a80cf3d0 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a8152e30 0x7ff2a81e8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a8152e30 0x7ff2a81e8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54454/0 (socket says 172.21.15.71:54454) 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2a6a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a80cd210 0x7ff2a80b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:12.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2a7400640 1 -- 172.21.15.71:0/464323195 learned_addr learned my addr 172.21.15.71:0/464323195 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:12.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.730+0000 7ff2a6000640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a80cf3d0 0x7ff2a80af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff2a7400640 1 -- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a80cf3d0 msgr2=0x7ff2a80af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff2a7400640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a80cf3d0 0x7ff2a80af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff2a7400640 1 -- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a80cd210 msgr2=0x7ff2a80b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff2a7400640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a80cd210 0x7ff2a80b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff2a7400640 1 -- 172.21.15.71:0/464323195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a81534e0 con 0x7ff2a8152e30 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff2a7400640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a8152e30 0x7ff2a81e8930 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0006010 tx=0x7ff2a0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff29ea00640 1 -- 172.21.15.71:0/464323195 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0046070 con 0x7ff2a8152e30 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff29ea00640 1 -- 172.21.15.71:0/464323195 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2a0003e50 con 0x7ff2a8152e30 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a81537c0 con 0x7ff2a8152e30 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.731+0000 7ff29ea00640 1 -- 172.21.15.71:0/464323195 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00044b0 con 0x7ff2a8152e30 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.732+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a8154060 con 0x7ff2a8152e30 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.733+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a806ff70 con 0x7ff2a8152e30 2024-09-16T07:25:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.734+0000 7ff29ea00640 1 -- 172.21.15.71:0/464323195 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 101338+0+0 (secure 0 0 0) 0x7ff2a0049070 con 0x7ff2a8152e30 2024-09-16T07:25:12.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.735+0000 7ff29ea00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2880788e0 0x7ff28807ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:12.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.735+0000 7ff29ea00640 1 -- 172.21.15.71:0/464323195 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7ff2a00bb900 con 0x7ff2a8152e30 2024-09-16T07:25:12.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.735+0000 7ff2a6a00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2880788e0 0x7ff28807ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:12.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.736+0000 7ff2a6a00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2880788e0 0x7ff28807ada0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80cc3a0 tx=0x7ff29800b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:12.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.739+0000 7ff29ea00640 1 -- 172.21.15.71:0/464323195 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff2a0032030 con 0x7ff2a8152e30 2024-09-16T07:25:12.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:12.956+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7ff2a8153990 con 0x7ff2880788e0 2024-09-16T07:25:13.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.036+0000 7ff29ea00640 1 -- 172.21.15.71:0/464323195 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2a8153990 con 0x7ff2880788e0 2024-09-16T07:25:13.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2880788e0 msgr2=0x7ff28807ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:13.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2880788e0 0x7ff28807ada0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80cc3a0 tx=0x7ff29800b2e0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a8152e30 msgr2=0x7ff2a81e8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:13.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a8152e30 0x7ff2a81e8930 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0006010 tx=0x7ff2a0003ce0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 shutdown_connections 2024-09-16T07:25:13.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2880788e0 0x7ff28807ada0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a8152e30 0x7ff2a81e8930 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a80cf3d0 0x7ff2a80af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 --2- 172.21.15.71:0/464323195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a80cd210 0x7ff2a80b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.038+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 >> 172.21.15.71:0/464323195 conn(0x7ff2a806cf90 msgr2=0x7ff2a80b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.039+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 shutdown_connections 2024-09-16T07:25:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.039+0000 7ff2ada00640 1 -- 172.21.15.71:0/464323195 wait complete. 2024-09-16T07:25:13.039 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.039+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.039 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.039+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.039 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.039+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.039 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.039+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.040 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.039+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.054 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:25:13.057 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.057+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.057 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.057+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.057 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.057+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.057 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.057+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.057 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:13.057+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:13.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.211+0000 7f0585400640 1 -- 172.21.15.71:0/1402476822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 msgr2=0x7f058006eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.211+0000 7f0585400640 1 --2- 172.21.15.71:0/1402476822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 0x7f058006eb70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0578005fd0 tx=0x7f057802a820 comp rx=0 tx=0).stop 2024-09-16T07:25:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.211+0000 7f0585400640 1 -- 172.21.15.71:0/1402476822 shutdown_connections 2024-09-16T07:25:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.211+0000 7f0585400640 1 --2- 172.21.15.71:0/1402476822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 0x7f058006eb70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.211+0000 7f0585400640 1 --2- 172.21.15.71:0/1402476822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f058015f9a0 0x7f058015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.211+0000 7f0585400640 1 -- 172.21.15.71:0/1402476822 >> 172.21.15.71:0/1402476822 conn(0x7f05800b18e0 msgr2=0x7f05800b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.211+0000 7f0585400640 1 -- 172.21.15.71:0/1402476822 shutdown_connections 2024-09-16T07:25:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.211+0000 7f0585400640 1 -- 172.21.15.71:0/1402476822 wait complete. 2024-09-16T07:25:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.212+0000 7f0585400640 1 Processor -- start 2024-09-16T07:25:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.213+0000 7f0585400640 1 -- start start 2024-09-16T07:25:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.213+0000 7f0585400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 0x7f05801f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.213+0000 7f0585400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0580073de0 0x7f0580155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.213+0000 7f0585400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f058015f9a0 0x7f05801ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.213+0000 7f0585400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f058006f0b0 con 0x7f058006e6f0 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.213+0000 7f0585400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f05800cd5e0 con 0x7f0580073de0 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.213+0000 7f0585400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f058006f230 con 0x7f058015f9a0 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 0x7f05801f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 0x7f05801f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54482/0 (socket says 172.21.15.71:54482) 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 -- 172.21.15.71:0/1340666380 learned_addr learned my addr 172.21.15.71:0/1340666380 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f0577e00640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0580073de0 0x7f0580155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057f200640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f058015f9a0 0x7f05801ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 -- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f058015f9a0 msgr2=0x7f05801ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f058015f9a0 0x7f05801ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 -- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0580073de0 msgr2=0x7f0580155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0580073de0 0x7f0580155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 -- 172.21.15.71:0/1340666380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05800b3010 con 0x7f058006e6f0 2024-09-16T07:25:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057f200640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f058015f9a0 0x7f05801ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.214+0000 7f057e800640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 0x7f05801f5830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f057000d740 tx=0x7f057000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.215+0000 7f0576a00640 1 -- 172.21.15.71:0/1340666380 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570017070 con 0x7f058006e6f0 2024-09-16T07:25:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.215+0000 7f0576a00640 1 -- 172.21.15.71:0/1340666380 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0570009d20 con 0x7f058006e6f0 2024-09-16T07:25:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.215+0000 7f0576a00640 1 -- 172.21.15.71:0/1340666380 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05700106c0 con 0x7f058006e6f0 2024-09-16T07:25:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.215+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05800b32f0 con 0x7f058006e6f0 2024-09-16T07:25:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.215+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05800b38b0 con 0x7f058006e6f0 2024-09-16T07:25:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.216+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0580069360 con 0x7f058006e6f0 2024-09-16T07:25:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.217+0000 7f0576a00640 1 -- 172.21.15.71:0/1340666380 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 101338+0+0 (secure 0 0 0) 0x7f057001a070 con 0x7f058006e6f0 2024-09-16T07:25:13.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.218+0000 7f0576a00640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c0788e0 0x7f055c07ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:13.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.218+0000 7f0576a00640 1 -- 172.21.15.71:0/1340666380 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f057009ac10 con 0x7f058006e6f0 2024-09-16T07:25:13.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.218+0000 7f0577e00640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c0788e0 0x7f055c07ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:13.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.219+0000 7f0577e00640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c0788e0 0x7f055c07ada0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f05800b4ec0 tx=0x7f0568007eb0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:13.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.222+0000 7f0576a00640 1 -- 172.21.15.71:0/1340666380 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0570062a40 con 0x7f058006e6f0 2024-09-16T07:25:13.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.480+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f05800cd7b0 con 0x7f058006e6f0 2024-09-16T07:25:13.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.480+0000 7f0576a00640 1 -- 172.21.15.71:0/1340666380 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1882 (secure 0 0 0) 0x7f0570067470 con 0x7f058006e6f0 2024-09-16T07:25:13.480 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:25:13.481 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 170, 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:25:13.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 37, 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:13.483 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 2449, 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 10168120677, 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 20406374400, 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 752687738880, 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 131391956, 2024-09-16T07:25:13.484 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 0, 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 33 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 18, 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:23:51:237136+0000", 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 2, 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 2, 2024-09-16T07:25:13.485 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "b", 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 4917 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:25:13.486 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 8, 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:24:44.321131+0000", 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": {} 2024-09-16T07:25:13.487 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:25:13.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c0788e0 msgr2=0x7f055c07ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c0788e0 0x7f055c07ada0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f05800b4ec0 tx=0x7f0568007eb0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 msgr2=0x7f05801f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 0x7f05801f5830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f057000d740 tx=0x7f057000dc00 comp rx=0 tx=0).stop 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 shutdown_connections 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c0788e0 0x7f055c07ada0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f058015f9a0 0x7f05801ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0580073de0 0x7f0580155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 --2- 172.21.15.71:0/1340666380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f058006e6f0 0x7f05801f5830 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 >> 172.21.15.71:0/1340666380 conn(0x7f05800b18e0 msgr2=0x7f05800b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 shutdown_connections 2024-09-16T07:25:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:13.483+0000 7f0585400640 1 -- 172.21.15.71:0/1340666380 wait complete. 2024-09-16T07:25:13.817 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:25:13.821 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:25:14.043 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:14.044+0000 7fe75d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:14.044 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:14.044+0000 7fe75d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:14.044 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:14.044+0000 7fe75d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:14.044 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:14.044+0000 7fe75d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:14.044 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:14.044+0000 7fe75d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:14.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:25:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.735+0000 7f2b02e00640 1 -- 172.21.15.71:0/3006384501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 msgr2=0x7f2afc0cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.735+0000 7f2b02e00640 1 --2- 172.21.15.71:0/3006384501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 0x7f2afc0cc540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8005fd0 tx=0x7f2ae802a5d0 comp rx=0 tx=0).stop 2024-09-16T07:25:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.735+0000 7f2b02e00640 1 -- 172.21.15.71:0/3006384501 shutdown_connections 2024-09-16T07:25:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.735+0000 7f2b02e00640 1 --2- 172.21.15.71:0/3006384501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc0cca80 0x7f2afc06eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.735+0000 7f2b02e00640 1 --2- 172.21.15.71:0/3006384501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 0x7f2afc0cc540 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.735+0000 7f2b02e00640 1 -- 172.21.15.71:0/3006384501 >> 172.21.15.71:0/3006384501 conn(0x7f2afc0b18e0 msgr2=0x7f2afc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.736+0000 7f2b02e00640 1 -- 172.21.15.71:0/3006384501 shutdown_connections 2024-09-16T07:25:14.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.736+0000 7f2b02e00640 1 -- 172.21.15.71:0/3006384501 wait complete. 2024-09-16T07:25:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.737+0000 7f2b02e00640 1 Processor -- start 2024-09-16T07:25:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.737+0000 7f2b02e00640 1 -- start start 2024-09-16T07:25:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b02e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0cca80 0x7f2afc145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b02e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 0x7f2afc14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b02e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc14c020 0x7f2afc073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b02e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06f260 con 0x7f2afc15afa0 2024-09-16T07:25:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b02e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06f0e0 con 0x7f2afc0cca80 2024-09-16T07:25:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b02e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc06f3e0 con 0x7f2afc14c020 2024-09-16T07:25:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b01000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 0x7f2afc14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b01000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 0x7f2afc14bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54456/0 (socket says 172.21.15.71:54456) 2024-09-16T07:25:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b01000640 1 -- 172.21.15.71:0/2147332193 learned_addr learned my addr 172.21.15.71:0/2147332193 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b02400640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc14c020 0x7f2afc073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.738+0000 7f2b01a00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0cca80 0x7f2afc145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b01000640 1 -- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc14c020 msgr2=0x7f2afc073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b01000640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc14c020 0x7f2afc073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b01000640 1 -- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0cca80 msgr2=0x7f2afc145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b01000640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0cca80 0x7f2afc145060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b01000640 1 -- 172.21.15.71:0/2147332193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc14c5a0 con 0x7f2afc15afa0 2024-09-16T07:25:14.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b01a00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0cca80 0x7f2afc145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:14.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b01000640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 0x7f2afc14bae0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8006010 tx=0x7f2ae80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:14.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2af9600640 1 -- 172.21.15.71:0/2147332193 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae803e030 con 0x7f2afc15afa0 2024-09-16T07:25:14.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2af9600640 1 -- 172.21.15.71:0/2147332193 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ae8003ca0 con 0x7f2afc15afa0 2024-09-16T07:25:14.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2af9600640 1 -- 172.21.15.71:0/2147332193 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8035040 con 0x7f2afc15afa0 2024-09-16T07:25:14.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1f5a90 con 0x7f2afc15afa0 2024-09-16T07:25:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.739+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc14c880 con 0x7f2afc15afa0 2024-09-16T07:25:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.740+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc06eba0 con 0x7f2afc15afa0 2024-09-16T07:25:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.741+0000 7f2af9600640 1 -- 172.21.15.71:0/2147332193 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f2ae8041030 con 0x7f2afc15afa0 2024-09-16T07:25:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.742+0000 7f2af9600640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc078ac0 0x7f2adc07af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.742+0000 7f2af9600640 1 -- 172.21.15.71:0/2147332193 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f2ae80bb230 con 0x7f2afc15afa0 2024-09-16T07:25:14.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.743+0000 7f2b01a00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc078ac0 0x7f2adc07af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:14.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.743+0000 7f2b01a00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc078ac0 0x7f2adc07af80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2af0001e40 tx=0x7f2af0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:14.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:14.747+0000 7f2af9600640 1 -- 172.21.15.71:0/2147332193 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ae8082e20 con 0x7f2afc15afa0 2024-09-16T07:25:15.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.138+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2afc0cd220 con 0x7f2afc15afa0 2024-09-16T07:25:15.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.138+0000 7f2af9600640 1 -- 172.21.15.71:0/2147332193 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2138 (secure 0 0 0) 0x7f2ae8087850 con 0x7f2afc15afa0 2024-09-16T07:25:15.235 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:15.235 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:25:15.235 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:25:15.235 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:25:15.236 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 172, 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 37, 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:25:15.237 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:25:15.238 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 2587, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 10738576860, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 21740208128, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 751353905152, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 85, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 168784941, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 0, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 43 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 18, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:23:51:237136+0000", 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 2, 2024-09-16T07:25:15.239 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:25:15.240 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:25:15.240 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:25:15.240 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:25:15.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:15.240 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 2, 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "b", 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 4917 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:25:15.241 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 8, 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:24:44.321131+0000", 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:25:15.242 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:25:15.243 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-16T07:25:15.243 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0, 2024-09-16T07:25:15.243 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:25:15.243 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:15.243 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:15.243 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:25:15.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc078ac0 msgr2=0x7f2adc07af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:15.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc078ac0 0x7f2adc07af80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2af0001e40 tx=0x7f2af0009040 comp rx=0 tx=0).stop 2024-09-16T07:25:15.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 msgr2=0x7f2afc14bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 0x7f2afc14bae0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8006010 tx=0x7f2ae80038a0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 shutdown_connections 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc078ac0 0x7f2adc07af80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc14c020 0x7f2afc073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc15afa0 0x7f2afc14bae0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 --2- 172.21.15.71:0/2147332193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0cca80 0x7f2afc145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 >> 172.21.15.71:0/2147332193 conn(0x7f2afc0b18e0 msgr2=0x7f2afc0cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 shutdown_connections 2024-09-16T07:25:15.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.141+0000 7f2b02e00640 1 -- 172.21.15.71:0/2147332193 wait complete. 2024-09-16T07:25:15.246 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-16T07:25:15.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.521+0000 7f8622200640 1 -- 172.21.15.71:0/3598084450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c0af190 msgr2=0x7f861c0b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:15.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.521+0000 7f8622200640 1 --2- 172.21.15.71:0/3598084450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c0af190 0x7f861c0b2b40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8604005fd0 tx=0x7f860402a5c0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.521+0000 7f8622200640 1 -- 172.21.15.71:0/3598084450 shutdown_connections 2024-09-16T07:25:15.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.521+0000 7f8622200640 1 --2- 172.21.15.71:0/3598084450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f861c0cec60 0x7f861c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.521+0000 7f8622200640 1 --2- 172.21.15.71:0/3598084450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c0af190 0x7f861c0b2b40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.521+0000 7f8622200640 1 -- 172.21.15.71:0/3598084450 >> 172.21.15.71:0/3598084450 conn(0x7f861c0b1840 msgr2=0x7f861c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:15.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.522+0000 7f8622200640 1 -- 172.21.15.71:0/3598084450 shutdown_connections 2024-09-16T07:25:15.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.522+0000 7f8622200640 1 -- 172.21.15.71:0/3598084450 wait complete. 2024-09-16T07:25:15.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.523+0000 7f8622200640 1 Processor -- start 2024-09-16T07:25:15.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.524+0000 7f8622200640 1 -- start start 2024-09-16T07:25:15.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.525+0000 7f8622200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f861c0af190 0x7f861c1475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:15.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.525+0000 7f8622200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f861c0cec60 0x7f861c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:15.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.525+0000 7f8622200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c161760 0x7f861c15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:15.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.525+0000 7f8622200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f861c162310 con 0x7f861c161760 2024-09-16T07:25:15.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.525+0000 7f8622200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f861c162190 con 0x7f861c0af190 2024-09-16T07:25:15.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.525+0000 7f8622200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f861c162490 con 0x7f861c0cec60 2024-09-16T07:25:15.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.526+0000 7f8613e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f861c0cec60 0x7f861c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:15.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.526+0000 7f8613e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f861c0cec60 0x7f861c15c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:53272/0 (socket says 172.21.15.71:53272) 2024-09-16T07:25:15.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.526+0000 7f8613e00640 1 -- 172.21.15.71:0/1322803284 learned_addr learned my addr 172.21.15.71:0/1322803284 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:15.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.526+0000 7f861b400640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f861c0af190 0x7f861c1475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:15.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.526+0000 7f861be00640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c161760 0x7f861c15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:15.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.527+0000 7f861be00640 1 -- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f861c0cec60 msgr2=0x7f861c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:15.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.527+0000 7f861be00640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f861c0cec60 0x7f861c15c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.527+0000 7f861be00640 1 -- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f861c0af190 msgr2=0x7f861c1475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:15.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.527+0000 7f861be00640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f861c0af190 0x7f861c1475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:15.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.527+0000 7f861be00640 1 -- 172.21.15.71:0/1322803284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c148210 con 0x7f861c161760 2024-09-16T07:25:15.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.527+0000 7f861b400640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f861c0af190 0x7f861c1475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:15.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.527+0000 7f861be00640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c161760 0x7f861c15cd60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f861400ea80 tx=0x7f861400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:15.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.527+0000 7f8613400640 1 -- 172.21.15.71:0/1322803284 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861400cea0 con 0x7f861c161760 2024-09-16T07:25:15.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.528+0000 7f8613400640 1 -- 172.21.15.71:0/1322803284 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8614004c10 con 0x7f861c161760 2024-09-16T07:25:15.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.528+0000 7f8613400640 1 -- 172.21.15.71:0/1322803284 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614005020 con 0x7f861c161760 2024-09-16T07:25:15.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.528+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c1484f0 con 0x7f861c161760 2024-09-16T07:25:15.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.528+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c148ab0 con 0x7f861c161760 2024-09-16T07:25:15.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.529+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c162970 con 0x7f861c161760 2024-09-16T07:25:15.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.532+0000 7f8613400640 1 -- 172.21.15.71:0/1322803284 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f86140051c0 con 0x7f861c161760 2024-09-16T07:25:15.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.534+0000 7f8613400640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f85f8078a90 0x7f85f807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:15.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.534+0000 7f8613400640 1 -- 172.21.15.71:0/1322803284 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f861409bc30 con 0x7f861c161760 2024-09-16T07:25:15.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.537+0000 7f861b400640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f85f8078a90 0x7f85f807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:15.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.538+0000 7f861b400640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f85f8078a90 0x7f85f807af50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f860c000f60 tx=0x7f860c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:15.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.543+0000 7f8613400640 1 -- 172.21.15.71:0/1322803284 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8614063da0 con 0x7f861c161760 2024-09-16T07:25:15.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:15.888+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f861c073af0 con 0x7f85f8078a90 2024-09-16T07:25:16.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.040+0000 7f8613400640 1 -- 172.21.15.71:0/1322803284 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f861c073af0 con 0x7f85f8078a90 2024-09-16T07:25:16.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f85f8078a90 msgr2=0x7f85f807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f85f8078a90 0x7f85f807af50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f860c000f60 tx=0x7f860c009040 comp rx=0 tx=0).stop 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c161760 msgr2=0x7f861c15cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c161760 0x7f861c15cd60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f861400ea80 tx=0x7f861400ef40 comp rx=0 tx=0).stop 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 shutdown_connections 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f85f8078a90 0x7f85f807af50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f861c161760 0x7f861c15cd60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f861c0cec60 0x7f861c15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 --2- 172.21.15.71:0/1322803284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f861c0af190 0x7f861c1475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 >> 172.21.15.71:0/1322803284 conn(0x7f861c0b1840 msgr2=0x7f861c072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 shutdown_connections 2024-09-16T07:25:16.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.043+0000 7f8622200640 1 -- 172.21.15.71:0/1322803284 wait complete. 2024-09-16T07:25:16.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:25:16.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.218+0000 7f56bc400640 1 -- 172.21.15.71:0/2328294366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f569c003a00 msgr2=0x7f569c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.218+0000 7f56bc400640 1 --2- 172.21.15.71:0/2328294366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f569c003a00 0x7f569c003de0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c610 tx=0x7f569c005830 comp rx=0 tx=0).stop 2024-09-16T07:25:16.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.218+0000 7f56bc400640 1 -- 172.21.15.71:0/2328294366 shutdown_connections 2024-09-16T07:25:16.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.218+0000 7f56bc400640 1 --2- 172.21.15.71:0/2328294366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f569c003a00 0x7f569c003de0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.218+0000 7f56bc400640 1 -- 172.21.15.71:0/2328294366 >> 172.21.15.71:0/2328294366 conn(0x7f56b40b1950 msgr2=0x7f56b40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:16.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.218+0000 7f56bc400640 1 -- 172.21.15.71:0/2328294366 shutdown_connections 2024-09-16T07:25:16.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.218+0000 7f56bc400640 1 -- 172.21.15.71:0/2328294366 wait complete. 2024-09-16T07:25:16.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.219+0000 7f56bc400640 1 Processor -- start 2024-09-16T07:25:16.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.220+0000 7f56bc400640 1 -- start start 2024-09-16T07:25:16.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.220+0000 7f56bc400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f569c003a00 0x7f56b4157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:16.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.220+0000 7f56bc400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b409d130 0x7f56b41f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:16.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.220+0000 7f56bc400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b41f4ee0 0x7f56b40b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.220+0000 7f56bc400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4074420 con 0x7f56b41f4ee0 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56bc400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56b40b2700 con 0x7f56b409d130 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56bc400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f56b40745a0 con 0x7f569c003a00 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b41f4ee0 0x7f56b40b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b41f4ee0 0x7f56b40b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54472/0 (socket says 172.21.15.71:54472) 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 -- 172.21.15.71:0/16827922 learned_addr learned my addr 172.21.15.71:0/16827922 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56b9a00640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f569c003a00 0x7f56b4157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56b9000640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b409d130 0x7f56b41f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 -- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f569c003a00 msgr2=0x7f56b4157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f569c003a00 0x7f56b4157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 -- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b409d130 msgr2=0x7f56b41f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b409d130 0x7f56b41f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 -- 172.21.15.71:0/16827922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b41e6870 con 0x7f56b41f4ee0 2024-09-16T07:25:16.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56b9a00640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f569c003a00 0x7f56b4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:16.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.221+0000 7f56ba400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b41f4ee0 0x7f56b40b2d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56b406da40 tx=0x7f56ac001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:16.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.222+0000 7f56b1600640 1 -- 172.21.15.71:0/16827922 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac011640 con 0x7f56b41f4ee0 2024-09-16T07:25:16.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.222+0000 7f56b1600640 1 -- 172.21.15.71:0/16827922 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56ac011800 con 0x7f56b41f4ee0 2024-09-16T07:25:16.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.222+0000 7f56b1600640 1 -- 172.21.15.71:0/16827922 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac011c50 con 0x7f56b41f4ee0 2024-09-16T07:25:16.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.222+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41e6b50 con 0x7f56b41f4ee0 2024-09-16T07:25:16.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.222+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41e7110 con 0x7f56b41f4ee0 2024-09-16T07:25:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.223+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b40acd30 con 0x7f56b41f4ee0 2024-09-16T07:25:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.224+0000 7f56b1600640 1 -- 172.21.15.71:0/16827922 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f56ac019020 con 0x7f56b41f4ee0 2024-09-16T07:25:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.225+0000 7f56b1600640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5694078ac0 0x7f569407af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.225+0000 7f56b1600640 1 -- 172.21.15.71:0/16827922 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f56ac09b090 con 0x7f56b41f4ee0 2024-09-16T07:25:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.225+0000 7f56b9a00640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5694078ac0 0x7f569407af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:16.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.226+0000 7f56b9a00640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5694078ac0 0x7f569407af80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f56a8033000 tx=0x7f56a8001de0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:16.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.229+0000 7f56b1600640 1 -- 172.21.15.71:0/16827922 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f56ac004e20 con 0x7f56b41f4ee0 2024-09-16T07:25:16.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.435+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f56b40ceb60 con 0x7f5694078ac0 2024-09-16T07:25:16.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.437+0000 7f56b1600640 1 -- 172.21.15.71:0/16827922 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f56b40ceb60 con 0x7f5694078ac0 2024-09-16T07:25:16.437 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:25:16.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:16.437 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:25:16.437 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:16.437 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5694078ac0 msgr2=0x7f569407af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5694078ac0 0x7f569407af80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f56a8033000 tx=0x7f56a8001de0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b41f4ee0 msgr2=0x7f56b40b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b41f4ee0 0x7f56b40b2d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56b406da40 tx=0x7f56ac001730 comp rx=0 tx=0).stop 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 shutdown_connections 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5694078ac0 0x7f569407af80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b41f4ee0 0x7f56b40b2d30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b409d130 0x7f56b41f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 --2- 172.21.15.71:0/16827922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f569c003a00 0x7f56b4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 >> 172.21.15.71:0/16827922 conn(0x7f56b40b1950 msgr2=0x7f56b40ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:16.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 shutdown_connections 2024-09-16T07:25:16.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.440+0000 7f56bc400640 1 -- 172.21.15.71:0/16827922 wait complete. 2024-09-16T07:25:16.455 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:25:16.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 -- 172.21.15.71:0/2670978603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd80ccb00 msgr2=0x7f3cd815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 --2- 172.21.15.71:0/2670978603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd80ccb00 0x7f3cd815f9f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8074ff0 tx=0x7f3cd002a980 comp rx=0 tx=0).stop 2024-09-16T07:25:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 -- 172.21.15.71:0/2670978603 shutdown_connections 2024-09-16T07:25:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 --2- 172.21.15.71:0/2670978603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd80ccb00 0x7f3cd815f9f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 --2- 172.21.15.71:0/2670978603 >> v2:172.21.15.177:3300/0 conn(0x7f3cd806efa0 0x7f3cd80cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 --2- 172.21.15.71:0/2670978603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cd8145750 0x7f3cd8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 -- 172.21.15.71:0/2670978603 >> 172.21.15.71:0/2670978603 conn(0x7f3cd80b1840 msgr2=0x7f3cd80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 -- 172.21.15.71:0/2670978603 shutdown_connections 2024-09-16T07:25:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.613+0000 7f3cdce00640 1 -- 172.21.15.71:0/2670978603 wait complete. 2024-09-16T07:25:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.615+0000 7f3cdce00640 1 Processor -- start 2024-09-16T07:25:16.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.615+0000 7f3cdce00640 1 -- start start 2024-09-16T07:25:16.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cdce00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cd806efa0 0x7f3cd80cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:16.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cdce00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cd80ccb00 0x7f3cd80ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:16.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cdce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8145750 0x7f3cd80aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:16.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cdce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81600b0 con 0x7f3cd8145750 2024-09-16T07:25:16.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cdce00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd815ff30 con 0x7f3cd806efa0 2024-09-16T07:25:16.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cdce00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd8160230 con 0x7f3cd80ccb00 2024-09-16T07:25:16.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cd6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8145750 0x7f3cd80aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cd6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8145750 0x7f3cd80aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54484/0 (socket says 172.21.15.71:54484) 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cd6800640 1 -- 172.21.15.71:0/3402035438 learned_addr learned my addr 172.21.15.71:0/3402035438 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3cd5e00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cd806efa0 0x7f3cd80cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.616+0000 7f3ccfe00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cd80ccb00 0x7f3cd80ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cd6800640 1 -- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cd80ccb00 msgr2=0x7f3cd80ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cd6800640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cd80ccb00 0x7f3cd80ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cd6800640 1 -- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cd806efa0 msgr2=0x7f3cd80cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cd6800640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cd806efa0 0x7f3cd80cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cd6800640 1 -- 172.21.15.71:0/3402035438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd814c410 con 0x7f3cd8145750 2024-09-16T07:25:16.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cd6800640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8145750 0x7f3cd80aecf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3cd002af70 tx=0x7f3cd0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:16.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cce000640 1 -- 172.21.15.71:0/3402035438 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0034050 con 0x7f3cd8145750 2024-09-16T07:25:16.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cce000640 1 -- 172.21.15.71:0/3402035438 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cd0004450 con 0x7f3cd8145750 2024-09-16T07:25:16.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd80aaeb0 con 0x7f3cd8145750 2024-09-16T07:25:16.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.617+0000 7f3cce000640 1 -- 172.21.15.71:0/3402035438 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0043440 con 0x7f3cd8145750 2024-09-16T07:25:16.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.618+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd80ab190 con 0x7f3cd8145750 2024-09-16T07:25:16.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.619+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd8160710 con 0x7f3cd8145750 2024-09-16T07:25:16.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.620+0000 7f3cce000640 1 -- 172.21.15.71:0/3402035438 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f3cd0037020 con 0x7f3cd8145750 2024-09-16T07:25:16.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.620+0000 7f3cce000640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cb4078a90 0x7f3cb407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:16.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.621+0000 7f3cce000640 1 -- 172.21.15.71:0/3402035438 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f3cd00bc250 con 0x7f3cd8145750 2024-09-16T07:25:16.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.621+0000 7f3cd5e00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cb4078a90 0x7f3cb407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:16.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.621+0000 7f3cd5e00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cb4078a90 0x7f3cb407af50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8004630 tx=0x7f3cc800b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:16.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.624+0000 7f3cce000640 1 -- 172.21.15.71:0/3402035438 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3cd0083cf0 con 0x7f3cd8145750 2024-09-16T07:25:16.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.837+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3cd80b0220 con 0x7f3cb4078a90 2024-09-16T07:25:16.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.842+0000 7f3cce000640 1 -- 172.21.15.71:0/3402035438 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7f3cd80b0220 con 0x7f3cb4078a90 2024-09-16T07:25:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:16.842 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:25:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1clone1" 2024-09-16T07:25:16.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:16.842 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:25:16.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cb4078a90 msgr2=0x7f3cb407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cb4078a90 0x7f3cb407af50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8004630 tx=0x7f3cc800b2e0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8145750 msgr2=0x7f3cd80aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:16.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8145750 0x7f3cd80aecf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3cd002af70 tx=0x7f3cd0003f10 comp rx=0 tx=0).stop 2024-09-16T07:25:16.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 shutdown_connections 2024-09-16T07:25:16.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cb4078a90 0x7f3cb407af50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8145750 0x7f3cd80aecf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cd80ccb00 0x7f3cd80ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 --2- 172.21.15.71:0/3402035438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cd806efa0 0x7f3cd80cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:16.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 >> 172.21.15.71:0/3402035438 conn(0x7f3cd80b1840 msgr2=0x7f3cd8142030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:16.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 shutdown_connections 2024-09-16T07:25:16.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:16.844+0000 7f3cdce00640 1 -- 172.21.15.71:0/3402035438 wait complete. 2024-09-16T07:25:16.860 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-16T07:25:17.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.020+0000 7f5da3c00640 1 -- 172.21.15.71:0/1157231604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c1565b0 msgr2=0x7f5d9c06f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:17.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.020+0000 7f5da3c00640 1 --2- 172.21.15.71:0/1157231604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c1565b0 0x7f5d9c06f810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5d88005fd0 tx=0x7f5d8802a540 comp rx=0 tx=0).stop 2024-09-16T07:25:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.020+0000 7f5da3c00640 1 -- 172.21.15.71:0/1157231604 shutdown_connections 2024-09-16T07:25:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.020+0000 7f5da3c00640 1 --2- 172.21.15.71:0/1157231604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d9c0ccb00 0x7f5d9c0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.020+0000 7f5da3c00640 1 --2- 172.21.15.71:0/1157231604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c1565b0 0x7f5d9c06f810 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.020+0000 7f5da3c00640 1 -- 172.21.15.71:0/1157231604 >> 172.21.15.71:0/1157231604 conn(0x7f5d9c0b18e0 msgr2=0x7f5d9c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.020+0000 7f5da3c00640 1 -- 172.21.15.71:0/1157231604 shutdown_connections 2024-09-16T07:25:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.020+0000 7f5da3c00640 1 -- 172.21.15.71:0/1157231604 wait complete. 2024-09-16T07:25:17.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.022+0000 7f5da3c00640 1 Processor -- start 2024-09-16T07:25:17.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.022+0000 7f5da3c00640 1 -- start start 2024-09-16T07:25:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da3c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d9c0ccb00 0x7f5d9c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da3c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d9c1565b0 0x7f5d9c141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da3c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c15f870 0x7f5d9c14ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da3c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c073620 con 0x7f5d9c15f870 2024-09-16T07:25:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da3c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c0734a0 con 0x7f5d9c0ccb00 2024-09-16T07:25:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da3c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9c0737a0 con 0x7f5d9c1565b0 2024-09-16T07:25:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c15f870 0x7f5d9c14ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c15f870 0x7f5d9c14ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54504/0 (socket says 172.21.15.71:54504) 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da1c00640 1 -- 172.21.15.71:0/3970039678 learned_addr learned my addr 172.21.15.71:0/3970039678 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5da1200640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d9c0ccb00 0x7f5d9c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.023+0000 7f5d9be00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d9c1565b0 0x7f5d9c141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5da1c00640 1 -- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d9c1565b0 msgr2=0x7f5d9c141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5da1c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d9c1565b0 0x7f5d9c141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5da1c00640 1 -- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d9c0ccb00 msgr2=0x7f5d9c151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5da1c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d9c0ccb00 0x7f5d9c151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5da1c00640 1 -- 172.21.15.71:0/3970039678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c1521f0 con 0x7f5d9c15f870 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5da1c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c15f870 0x7f5d9c14ff90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e970 tx=0x7f5d9400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5d98c00640 1 -- 172.21.15.71:0/3970039678 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400ccc0 con 0x7f5d9c15f870 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5d98c00640 1 -- 172.21.15.71:0/3970039678 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d9400ce60 con 0x7f5d9c15f870 2024-09-16T07:25:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c1447a0 con 0x7f5d9c15f870 2024-09-16T07:25:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.024+0000 7f5d98c00640 1 -- 172.21.15.71:0/3970039678 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94005280 con 0x7f5d9c15f870 2024-09-16T07:25:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.025+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c144d60 con 0x7f5d9c15f870 2024-09-16T07:25:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.025+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9c073cd0 con 0x7f5d9c15f870 2024-09-16T07:25:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.027+0000 7f5d98c00640 1 -- 172.21.15.71:0/3970039678 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f5d94005420 con 0x7f5d9c15f870 2024-09-16T07:25:17.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.027+0000 7f5d98c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d7c078a90 0x7f5d7c07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:17.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.028+0000 7f5d98c00640 1 -- 172.21.15.71:0/3970039678 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f5d9409b870 con 0x7f5d9c15f870 2024-09-16T07:25:17.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.028+0000 7f5da1200640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d7c078a90 0x7f5d7c07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.029+0000 7f5da1200640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d7c078a90 0x7f5d7c07af50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0b29a0 tx=0x7f5d8c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.032+0000 7f5d98c00640 1 -- 172.21.15.71:0/3970039678 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5d9c1521f0 con 0x7f5d9c15f870 2024-09-16T07:25:17.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:17.241+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f5d9c070980 con 0x7f5d7c078a90 2024-09-16T07:25:21.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.443+0000 7f5d98c00640 1 -- 172.21.15.71:0/3970039678 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5d9c070980 con 0x7f5d7c078a90 2024-09-16T07:25:21.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d7c078a90 msgr2=0x7f5d7c07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d7c078a90 0x7f5d7c07af50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0b29a0 tx=0x7f5d8c009040 comp rx=0 tx=0).stop 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c15f870 msgr2=0x7f5d9c14ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c15f870 0x7f5d9c14ff90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e970 tx=0x7f5d9400ed70 comp rx=0 tx=0).stop 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 shutdown_connections 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d7c078a90 0x7f5d7c07af50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d9c15f870 0x7f5d9c14ff90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d9c1565b0 0x7f5d9c141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 --2- 172.21.15.71:0/3970039678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d9c0ccb00 0x7f5d9c151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 >> 172.21.15.71:0/3970039678 conn(0x7f5d9c0b18e0 msgr2=0x7f5d9c1538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 shutdown_connections 2024-09-16T07:25:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.446+0000 7f5da3c00640 1 -- 172.21.15.71:0/3970039678 wait complete. 2024-09-16T07:25:21.447 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.447+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.447 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.447+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.447 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.447+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.447 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.447+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.447 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.447+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.464 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:25:21.465 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.464+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.465 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.464+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.465 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.464+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.465 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.464+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.465 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:21.464+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:21.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:25:21.554 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:25:21.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:25:21.563 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:25:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.722+0000 7f9027200640 1 -- 172.21.15.71:0/609061695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028074280 msgr2=0x7f902806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.722+0000 7f9027200640 1 --2- 172.21.15.71:0/609061695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028074280 0x7f902806ed60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f90280cd0e0 tx=0x7f902002a560 comp rx=0 tx=0).stop 2024-09-16T07:25:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.722+0000 7f9027200640 1 -- 172.21.15.71:0/609061695 shutdown_connections 2024-09-16T07:25:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.722+0000 7f9027200640 1 --2- 172.21.15.71:0/609061695 >> v2:172.21.15.177:3300/0 conn(0x7f90280b2810 0x7f9028073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.722+0000 7f9027200640 1 --2- 172.21.15.71:0/609061695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028074280 0x7f902806ed60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.722+0000 7f9027200640 1 -- 172.21.15.71:0/609061695 >> 172.21.15.71:0/609061695 conn(0x7f90280b1510 msgr2=0x7f90280b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.722+0000 7f9027200640 1 -- 172.21.15.71:0/609061695 shutdown_connections 2024-09-16T07:25:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.722+0000 7f9027200640 1 -- 172.21.15.71:0/609061695 wait complete. 2024-09-16T07:25:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.724+0000 7f9027200640 1 Processor -- start 2024-09-16T07:25:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.724+0000 7f9027200640 1 -- start start 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9027200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9028074280 0x7f90281f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9027200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90280b2810 0x7f902815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9027200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f902815bb90 0x7f90280c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9027200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f902806f2a0 con 0x7f902815bb90 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9027200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f902815bf70 con 0x7f9028074280 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9027200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f902806f420 con 0x7f90280b2810 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9026800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f902815bb90 0x7f90280c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9026800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f902815bb90 0x7f90280c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54520/0 (socket says 172.21.15.71:54520) 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9026800640 1 -- 172.21.15.71:0/1666269971 learned_addr learned my addr 172.21.15.71:0/1666269971 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9025e00640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9028074280 0x7f90281f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.725+0000 7f9025400640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90280b2810 0x7f902815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f9026800640 1 -- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90280b2810 msgr2=0x7f902815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f9026800640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90280b2810 0x7f902815dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f9026800640 1 -- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9028074280 msgr2=0x7f90281f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f9026800640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9028074280 0x7f90281f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f9026800640 1 -- 172.21.15.71:0/1666269971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90281f2ab0 con 0x7f902815bb90 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f9026800640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f902815bb90 0x7f90280c89d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9020009230 tx=0x7f9020003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f901e000640 1 -- 172.21.15.71:0/1666269971 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902003f070 con 0x7f902815bb90 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f901e000640 1 -- 172.21.15.71:0/1666269971 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90200047a0 con 0x7f902815bb90 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f901e000640 1 -- 172.21.15.71:0/1666269971 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020035030 con 0x7f902815bb90 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90281f2d90 con 0x7f902815bb90 2024-09-16T07:25:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.726+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90281f3350 con 0x7f902815bb90 2024-09-16T07:25:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.727+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902806f940 con 0x7f902815bb90 2024-09-16T07:25:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.729+0000 7f901e000640 1 -- 172.21.15.71:0/1666269971 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f9020042070 con 0x7f902815bb90 2024-09-16T07:25:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.729+0000 7f901e000640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9008078a90 0x7f900807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.730+0000 7f901e000640 1 -- 172.21.15.71:0/1666269971 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f90200bbc80 con 0x7f902815bb90 2024-09-16T07:25:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.730+0000 7f9025e00640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9008078a90 0x7f900807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.731+0000 7f9025e00640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9008078a90 0x7f900807af50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9018001e20 tx=0x7f9018009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:21.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.734+0000 7f901e000640 1 -- 172.21.15.71:0/1666269971 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f90200837e0 con 0x7f902815bb90 2024-09-16T07:25:21.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.915+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f9028069d20 con 0x7f902815bb90 2024-09-16T07:25:21.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.915+0000 7f901e000640 1 -- 172.21.15.71:0/1666269971 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7f9020088210 con 0x7f902815bb90 2024-09-16T07:25:21.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9008078a90 msgr2=0x7f900807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:21.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9008078a90 0x7f900807af50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9018001e20 tx=0x7f9018009040 comp rx=0 tx=0).stop 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f902815bb90 msgr2=0x7f90280c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f902815bb90 0x7f90280c89d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9020009230 tx=0x7f9020003960 comp rx=0 tx=0).stop 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 shutdown_connections 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9008078a90 0x7f900807af50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f902815bb90 0x7f90280c89d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90280b2810 0x7f902815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 --2- 172.21.15.71:0/1666269971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9028074280 0x7f90281f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 >> 172.21.15.71:0/1666269971 conn(0x7f90280b1510 msgr2=0x7f90280af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 shutdown_connections 2024-09-16T07:25:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:21.918+0000 7f9027200640 1 -- 172.21.15.71:0/1666269971 wait complete. 2024-09-16T07:25:21.934 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:25:21.934 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:25:22.013 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:25:22.097 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:25:22.097 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:25:22.097 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:22.097 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:22.133 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:22.133 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:22.210 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:25:22.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.413+0000 7f5e1ca00640 1 -- 172.21.15.71:0/3960739923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e180725d0 msgr2=0x7f5e180b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:22.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.413+0000 7f5e1ca00640 1 --2- 172.21.15.71:0/3960739923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e180725d0 0x7f5e180b28f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5e00005fd0 tx=0x7f5e0002a670 comp rx=0 tx=0).stop 2024-09-16T07:25:22.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.413+0000 7f5e1ca00640 1 -- 172.21.15.71:0/3960739923 shutdown_connections 2024-09-16T07:25:22.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.413+0000 7f5e1ca00640 1 --2- 172.21.15.71:0/3960739923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e180725d0 0x7f5e180b28f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:22.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.413+0000 7f5e1ca00640 1 -- 172.21.15.71:0/3960739923 >> 172.21.15.71:0/3960739923 conn(0x7f5e180cfa20 msgr2=0x7f5e180cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:22.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.413+0000 7f5e1ca00640 1 -- 172.21.15.71:0/3960739923 shutdown_connections 2024-09-16T07:25:22.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.413+0000 7f5e1ca00640 1 -- 172.21.15.71:0/3960739923 wait complete. 2024-09-16T07:25:22.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.415+0000 7f5e1ca00640 1 Processor -- start 2024-09-16T07:25:22.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.415+0000 7f5e1ca00640 1 -- start start 2024-09-16T07:25:22.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e1ca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e180d0540 0x7f5e180c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:22.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e1ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e1815bc30 0x7f5e18150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e1ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18155d40 0x7f5e18156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e1ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18074660 con 0x7f5e1815bc30 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e1ca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1806fa30 con 0x7f5e18155d40 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e1ca00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e18152560 con 0x7f5e180d0540 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e0de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e1815bc30 0x7f5e18150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e0de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e1815bc30 0x7f5e18150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54546/0 (socket says 172.21.15.71:54546) 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e0de00640 1 -- 172.21.15.71:0/2872232647 learned_addr learned my addr 172.21.15.71:0/2872232647 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e16800640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e180d0540 0x7f5e180c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e17200640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18155d40 0x7f5e18156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e0de00640 1 -- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e180d0540 msgr2=0x7f5e180c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e0de00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e180d0540 0x7f5e180c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e0de00640 1 -- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18155d40 msgr2=0x7f5e18156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:22.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e0de00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18155d40 0x7f5e18156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e0de00640 1 -- 172.21.15.71:0/2872232647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e18156aa0 con 0x7f5e1815bc30 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e16800640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e180d0540 0x7f5e180c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.416+0000 7f5e17200640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18155d40 0x7f5e18156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.417+0000 7f5e0de00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e1815bc30 0x7f5e18150e60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000c00 tx=0x7f5e000039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.417+0000 7f5e0f400640 1 -- 172.21.15.71:0/2872232647 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0003f070 con 0x7f5e1815bc30 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.417+0000 7f5e0f400640 1 -- 172.21.15.71:0/2872232647 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e00004890 con 0x7f5e1815bc30 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.417+0000 7f5e0f400640 1 -- 172.21.15.71:0/2872232647 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00035030 con 0x7f5e1815bc30 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.417+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e180cd720 con 0x7f5e1815bc30 2024-09-16T07:25:22.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.417+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e181564e0 con 0x7f5e1815bc30 2024-09-16T07:25:22.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.418+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e180b28f0 con 0x7f5e1815bc30 2024-09-16T07:25:22.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.419+0000 7f5e0f400640 1 -- 172.21.15.71:0/2872232647 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f5e00042070 con 0x7f5e1815bc30 2024-09-16T07:25:22.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.420+0000 7f5e0f400640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5df4078a90 0x7f5df407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:22.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.420+0000 7f5e0f400640 1 -- 172.21.15.71:0/2872232647 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f5e000bbe00 con 0x7f5e1815bc30 2024-09-16T07:25:22.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.420+0000 7f5e16800640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5df4078a90 0x7f5df407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:22.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.421+0000 7f5e16800640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5df4078a90 0x7f5df407af50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5e08007fd0 tx=0x7f5e08009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:22.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.424+0000 7f5e0f400640 1 -- 172.21.15.71:0/2872232647 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5e00084040 con 0x7f5e1815bc30 2024-09-16T07:25:22.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.619+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e18156880 con 0x7f5e1815bc30 2024-09-16T07:25:22.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.619+0000 7f5e0f400640 1 -- 172.21.15.71:0/2872232647 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7f5e00083940 con 0x7f5e1815bc30 2024-09-16T07:25:22.619 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:25:22.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.621+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5df4078a90 msgr2=0x7f5df407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:22.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.621+0000 7f5e1ca00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5df4078a90 0x7f5df407af50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5e08007fd0 tx=0x7f5e08009040 comp rx=0 tx=0).stop 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.621+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e1815bc30 msgr2=0x7f5e18150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.621+0000 7f5e1ca00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e1815bc30 0x7f5e18150e60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5e00000c00 tx=0x7f5e000039e0 comp rx=0 tx=0).stop 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.622+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 shutdown_connections 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.622+0000 7f5e1ca00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5df4078a90 0x7f5df407af50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.622+0000 7f5e1ca00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18155d40 0x7f5e18156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.622+0000 7f5e1ca00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e1815bc30 0x7f5e18150e60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.622+0000 7f5e1ca00640 1 --2- 172.21.15.71:0/2872232647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e180d0540 0x7f5e180c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.622+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 >> 172.21.15.71:0/2872232647 conn(0x7f5e180cfa20 msgr2=0x7f5e180cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.622+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 shutdown_connections 2024-09-16T07:25:22.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:22.622+0000 7f5e1ca00640 1 -- 172.21.15.71:0/2872232647 wait complete. 2024-09-16T07:25:22.637 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:25:22.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:22.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:25:24.416 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:25:24.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:24.417 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:25:24.476 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:25:24.477 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:25:24.477 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:24.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:25:24.534 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:25:24.534 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:24.534 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:24.580 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:25:24.581 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:25:24.581 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:25:24.581 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:24.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:25:24.617 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:25:24.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.778+0000 7fadd7400640 1 -- 172.21.15.71:0/3780342293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 msgr2=0x7fadd00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:24.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.778+0000 7fadd7400640 1 --2- 172.21.15.71:0/3780342293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 0x7fadd00b2440 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fadd00cfbb0 tx=0x7fadc002aa30 comp rx=0 tx=0).stop 2024-09-16T07:25:24.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.778+0000 7fadd7400640 1 -- 172.21.15.71:0/3780342293 shutdown_connections 2024-09-16T07:25:24.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.778+0000 7fadd7400640 1 --2- 172.21.15.71:0/3780342293 >> v2:172.21.15.177:3300/0 conn(0x7fadd00ccf00 0x7fadd00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:24.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.778+0000 7fadd7400640 1 --2- 172.21.15.71:0/3780342293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 0x7fadd00b2440 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:24.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.778+0000 7fadd7400640 1 -- 172.21.15.71:0/3780342293 >> 172.21.15.71:0/3780342293 conn(0x7fadd00b1950 msgr2=0x7fadd00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:24.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.778+0000 7fadd7400640 1 -- 172.21.15.71:0/3780342293 shutdown_connections 2024-09-16T07:25:24.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.778+0000 7fadd7400640 1 -- 172.21.15.71:0/3780342293 wait complete. 2024-09-16T07:25:24.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.780+0000 7fadd7400640 1 Processor -- start 2024-09-16T07:25:24.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.780+0000 7fadd7400640 1 -- start start 2024-09-16T07:25:24.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd7400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd00ad270 0x7fadd014cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:24.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd7400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadd00ccf00 0x7fadd0152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:24.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 0x7fadd0153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:24.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd7400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fadd006f160 con 0x7fadd015f5a0 2024-09-16T07:25:24.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd7400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fadd006f2e0 con 0x7fadd00ad270 2024-09-16T07:25:24.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd7400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fadd006f460 con 0x7fadd00ccf00 2024-09-16T07:25:24.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 0x7fadd0153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:24.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 0x7fadd0153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60196/0 (socket says 172.21.15.71:60196) 2024-09-16T07:25:24.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd5400640 1 -- 172.21.15.71:0/520790205 learned_addr learned my addr 172.21.15.71:0/520790205 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:24.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd5400640 1 -- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadd00ccf00 msgr2=0x7fadd0152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:24.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadcfe00640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadd00ccf00 0x7fadd0152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:24.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd4a00640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd00ad270 0x7fadd014cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:24.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd5400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadd00ccf00 0x7fadd0152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:24.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd5400640 1 -- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd00ad270 msgr2=0x7fadd014cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:24.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd5400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd00ad270 0x7fadd014cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:24.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd5400640 1 -- 172.21.15.71:0/520790205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd014fc50 con 0x7fadd015f5a0 2024-09-16T07:25:24.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.781+0000 7fadd4a00640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd00ad270 0x7fadd014cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:24.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.782+0000 7fadd5400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 0x7fadd0153490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fadc800c3c0 tx=0x7fadc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:24.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.782+0000 7fadccc00640 1 -- 172.21.15.71:0/520790205 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8017070 con 0x7fadd015f5a0 2024-09-16T07:25:24.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.782+0000 7fadccc00640 1 -- 172.21.15.71:0/520790205 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadc80053c0 con 0x7fadd015f5a0 2024-09-16T07:25:24.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.782+0000 7fadccc00640 1 -- 172.21.15.71:0/520790205 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8003850 con 0x7fadd015f5a0 2024-09-16T07:25:24.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.782+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd014ff30 con 0x7fadd015f5a0 2024-09-16T07:25:24.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.782+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd01f3ab0 con 0x7fadd015f5a0 2024-09-16T07:25:24.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.783+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd006f9f0 con 0x7fadd015f5a0 2024-09-16T07:25:24.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.784+0000 7fadccc00640 1 -- 172.21.15.71:0/520790205 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fadc801a070 con 0x7fadd015f5a0 2024-09-16T07:25:24.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.785+0000 7fadccc00640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fadb0078be0 0x7fadb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:24.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.785+0000 7fadccc00640 1 -- 172.21.15.71:0/520790205 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fadc809a350 con 0x7fadd015f5a0 2024-09-16T07:25:24.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.786+0000 7fadd4a00640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fadb0078be0 0x7fadb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:24.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.786+0000 7fadd4a00640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fadb0078be0 0x7fadb007b0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fadc0035000 tx=0x7fadc0004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:24.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:24.789+0000 7fadccc00640 1 -- 172.21.15.71:0/520790205 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fadc80623e0 con 0x7fadd015f5a0 2024-09-16T07:25:25.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.007+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadd01f3c80 con 0x7fadd015f5a0 2024-09-16T07:25:25.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.007+0000 7fadccc00640 1 -- 172.21.15.71:0/520790205 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7fadc8061ce0 con 0x7fadd015f5a0 2024-09-16T07:25:25.008 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:25.008 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"btime":"2024-09-16T07:23:51:237136+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4834,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3085563418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3085563418},{"type":"v1","addr":"172.21.15.71:6834","nonce":3085563418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3025435692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3025435692},{"type":"v1","addr":"172.21.15.71:6835","nonce":3025435692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3253921551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":3253921551},{"type":"v1","addr":"172.21.15.177:6836","nonce":3253921551}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:23:37.752030+0000","modified":"2024-09-16T07:23:40.774583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1652086843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1652086843},{"type":"v1","addr":"172.21.15.177:6837","nonce":1652086843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4917,"qdb_cluster":[4917]},"id":2}]} 2024-09-16T07:25:25.008 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2024-09-16T07:25:25.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fadb0078be0 msgr2=0x7fadb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fadb0078be0 0x7fadb007b0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fadc0035000 tx=0x7fadc0004ab0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 msgr2=0x7fadd0153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 0x7fadd0153490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fadc800c3c0 tx=0x7fadc800c880 comp rx=0 tx=0).stop 2024-09-16T07:25:25.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 shutdown_connections 2024-09-16T07:25:25.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fadb0078be0 0x7fadb007b0a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadd015f5a0 0x7fadd0153490 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadd00ccf00 0x7fadd0152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 --2- 172.21.15.71:0/520790205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadd00ad270 0x7fadd014cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 >> 172.21.15.71:0/520790205 conn(0x7fadd00b1950 msgr2=0x7fadd00af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:25.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 shutdown_connections 2024-09-16T07:25:25.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.010+0000 7fadd7400640 1 -- 172.21.15.71:0/520790205 wait complete. 2024-09-16T07:25:25.026 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:25:25.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.182+0000 7fcf41000640 1 -- 172.21.15.71:0/4136250686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c073fb0 msgr2=0x7fcf3c074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.182+0000 7fcf41000640 1 --2- 172.21.15.71:0/4136250686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c073fb0 0x7fcf3c074460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcf34005fd0 tx=0x7fcf3402aa10 comp rx=0 tx=0).stop 2024-09-16T07:25:25.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.182+0000 7fcf41000640 1 -- 172.21.15.71:0/4136250686 shutdown_connections 2024-09-16T07:25:25.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.182+0000 7fcf41000640 1 --2- 172.21.15.71:0/4136250686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c073fb0 0x7fcf3c074460 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.182+0000 7fcf41000640 1 --2- 172.21.15.71:0/4136250686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf3c15f5a0 0x7fcf3c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.182+0000 7fcf41000640 1 -- 172.21.15.71:0/4136250686 >> 172.21.15.71:0/4136250686 conn(0x7fcf3c0b1950 msgr2=0x7fcf3c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:25.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.182+0000 7fcf41000640 1 -- 172.21.15.71:0/4136250686 shutdown_connections 2024-09-16T07:25:25.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.182+0000 7fcf41000640 1 -- 172.21.15.71:0/4136250686 wait complete. 2024-09-16T07:25:25.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.184+0000 7fcf41000640 1 Processor -- start 2024-09-16T07:25:25.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.184+0000 7fcf41000640 1 -- start start 2024-09-16T07:25:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf41000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf3c073fb0 0x7fcf3c1470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf41000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c0ccf00 0x7fcf3c15c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf41000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c15f5a0 0x7fcf3c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf41000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c06f2e0 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf41000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c06f160 con 0x7fcf3c0ccf00 2024-09-16T07:25:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf41000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c06f460 con 0x7fcf3c073fb0 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf3a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c15f5a0 0x7fcf3c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf3a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c15f5a0 0x7fcf3c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60216/0 (socket says 172.21.15.71:60216) 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf3a800640 1 -- 172.21.15.71:0/2163893106 learned_addr learned my addr 172.21.15.71:0/2163893106 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf31400640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c0ccf00 0x7fcf3c15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.185+0000 7fcf39e00640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf3c073fb0 0x7fcf3c1470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf3a800640 1 -- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf3c073fb0 msgr2=0x7fcf3c1470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf3a800640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf3c073fb0 0x7fcf3c1470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf3a800640 1 -- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c0ccf00 msgr2=0x7fcf3c15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf3a800640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c0ccf00 0x7fcf3c15c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf3a800640 1 -- 172.21.15.71:0/2163893106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c15a6c0 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf31400640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c0ccf00 0x7fcf3c15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf3a800640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c15f5a0 0x7fcf3c159ce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009d90 tx=0x7fcf34003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf32a00640 1 -- 172.21.15.71:0/2163893106 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34047070 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf32a00640 1 -- 172.21.15.71:0/2163893106 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf3402f670 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c15a9a0 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf32a00640 1 -- 172.21.15.71:0/2163893106 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34033050 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.186+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c15af60 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.187+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3c06f9f0 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.188+0000 7fcf32a00640 1 -- 172.21.15.71:0/2163893106 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fcf3404a070 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.189+0000 7fcf32a00640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf18078bb0 0x7fcf1807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.189+0000 7fcf32a00640 1 -- 172.21.15.71:0/2163893106 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fcf340bc060 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.190+0000 7fcf39e00640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf18078bb0 0x7fcf1807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.190+0000 7fcf39e00640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf18078bb0 0x7fcf1807b070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009b30 tx=0x7fcf2c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:25.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.193+0000 7fcf32a00640 1 -- 172.21.15.71:0/2163893106 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcf34083b30 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.415+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf3c0b13d0 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.415+0000 7fcf32a00640 1 -- 172.21.15.71:0/2163893106 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7fcf34088560 con 0x7fcf3c15f5a0 2024-09-16T07:25:25.415 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:25.415 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"btime":"2024-09-16T07:23:51:237136+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4834,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3085563418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3085563418},{"type":"v1","addr":"172.21.15.71:6834","nonce":3085563418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3025435692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3025435692},{"type":"v1","addr":"172.21.15.71:6835","nonce":3025435692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3253921551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":3253921551},{"type":"v1","addr":"172.21.15.177:6836","nonce":3253921551}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:23:37.752030+0000","modified":"2024-09-16T07:23:40.774583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1652086843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1652086843},{"type":"v1","addr":"172.21.15.177:6837","nonce":1652086843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4917,"qdb_cluster":[4917]},"id":2}]} 2024-09-16T07:25:25.415 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2024-09-16T07:25:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf18078bb0 msgr2=0x7fcf1807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf18078bb0 0x7fcf1807b070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009b30 tx=0x7fcf2c005f90 comp rx=0 tx=0).stop 2024-09-16T07:25:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c15f5a0 msgr2=0x7fcf3c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c15f5a0 0x7fcf3c159ce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009d90 tx=0x7fcf34003ef0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 shutdown_connections 2024-09-16T07:25:25.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf18078bb0 0x7fcf1807b070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf3c15f5a0 0x7fcf3c159ce0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c0ccf00 0x7fcf3c15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 --2- 172.21.15.71:0/2163893106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf3c073fb0 0x7fcf3c1470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 >> 172.21.15.71:0/2163893106 conn(0x7fcf3c0b1950 msgr2=0x7fcf3c0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:25.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 shutdown_connections 2024-09-16T07:25:25.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.418+0000 7fcf41000640 1 -- 172.21.15.71:0/2163893106 wait complete. 2024-09-16T07:25:25.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:25:25.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.591+0000 7f64a9400640 1 -- 172.21.15.71:0/4021873394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64a415f5a0 msgr2=0x7f64a40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.591+0000 7f64a9400640 1 --2- 172.21.15.71:0/4021873394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64a415f5a0 0x7f64a40b2440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6494005fd0 tx=0x7f649402a820 comp rx=0 tx=0).stop 2024-09-16T07:25:25.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.592+0000 7f64a9400640 1 -- 172.21.15.71:0/4021873394 shutdown_connections 2024-09-16T07:25:25.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.592+0000 7f64a9400640 1 --2- 172.21.15.71:0/4021873394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64a40ccf00 0x7f64a40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.592+0000 7f64a9400640 1 --2- 172.21.15.71:0/4021873394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64a415f5a0 0x7f64a40b2440 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.592+0000 7f64a9400640 1 -- 172.21.15.71:0/4021873394 >> 172.21.15.71:0/4021873394 conn(0x7f64a40b1950 msgr2=0x7f64a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:25.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.592+0000 7f64a9400640 1 -- 172.21.15.71:0/4021873394 shutdown_connections 2024-09-16T07:25:25.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.592+0000 7f64a9400640 1 -- 172.21.15.71:0/4021873394 wait complete. 2024-09-16T07:25:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.593+0000 7f64a9400640 1 Processor -- start 2024-09-16T07:25:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.593+0000 7f64a9400640 1 -- start start 2024-09-16T07:25:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.594+0000 7f64a9400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64a40ad270 0x7f64a415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.594+0000 7f64a9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64a40ccf00 0x7f64a40cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.594+0000 7f64a9400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64a415f5a0 0x7f64a40d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.594+0000 7f64a9400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f64a406f2e0 con 0x7f64a40ccf00 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.594+0000 7f64a9400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64a406f160 con 0x7f64a40ad270 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.594+0000 7f64a9400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64a406f460 con 0x7f64a415f5a0 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64a40ccf00 0x7f64a40cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64a40ccf00 0x7f64a40cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60226/0 (socket says 172.21.15.71:60226) 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 -- 172.21.15.71:0/2902058380 learned_addr learned my addr 172.21.15.71:0/2902058380 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a2800640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64a40ad270 0x7f64a415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a3200640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64a415f5a0 0x7f64a40d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 -- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64a415f5a0 msgr2=0x7f64a40d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64a415f5a0 0x7f64a40d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 -- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64a40ad270 msgr2=0x7f64a415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64a40ad270 0x7f64a415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 -- 172.21.15.71:0/2902058380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a41f4960 con 0x7f64a40ccf00 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a3200640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64a415f5a0 0x7f64a40d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.595+0000 7f64a1e00640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64a40ccf00 0x7f64a40cfcb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f649000e9f0 tx=0x7f649000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.596+0000 7f649aa00640 1 -- 172.21.15.71:0/2902058380 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649000cd80 con 0x7f64a40ccf00 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.596+0000 7f649aa00640 1 -- 172.21.15.71:0/2902058380 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6490010430 con 0x7f64a40ccf00 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.596+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a40c9d00 con 0x7f64a40ccf00 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.596+0000 7f649aa00640 1 -- 172.21.15.71:0/2902058380 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490010840 con 0x7f64a40ccf00 2024-09-16T07:25:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.596+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a41f4ec0 con 0x7f64a40ccf00 2024-09-16T07:25:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.597+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a406fa20 con 0x7f64a40ccf00 2024-09-16T07:25:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.598+0000 7f649aa00640 1 -- 172.21.15.71:0/2902058380 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f64900109e0 con 0x7f64a40ccf00 2024-09-16T07:25:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.599+0000 7f649aa00640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6484078be0 0x7f648407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.599+0000 7f649aa00640 1 -- 172.21.15.71:0/2902058380 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f649009bb10 con 0x7f64a40ccf00 2024-09-16T07:25:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.599+0000 7f64a2800640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6484078be0 0x7f648407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.600+0000 7f64a2800640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6484078be0 0x7f648407b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6494034000 tx=0x7f6494004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.603+0000 7f649aa00640 1 -- 172.21.15.71:0/2902058380 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f64a41f4960 con 0x7f64a40ccf00 2024-09-16T07:25:25.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.788+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64a40b41c0 con 0x7f64a40ccf00 2024-09-16T07:25:25.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.789+0000 7f649aa00640 1 -- 172.21.15.71:0/2902058380 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16931 (secure 0 0 0) 0x7f6490063590 con 0x7f64a40ccf00 2024-09-16T07:25:25.789 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:25.789 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":37,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:24:30.352331+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:23:36.070239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T07:23:36.713734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]},{"pgid":"5.35","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:25:25.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6484078be0 msgr2=0x7f648407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6484078be0 0x7f648407b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6494034000 tx=0x7f6494004ab0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64a40ccf00 msgr2=0x7f64a40cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64a40ccf00 0x7f64a40cfcb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f649000e9f0 tx=0x7f649000eeb0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 shutdown_connections 2024-09-16T07:25:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6484078be0 0x7f648407b0a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64a415f5a0 0x7f64a40d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64a40ccf00 0x7f64a40cfcb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 --2- 172.21.15.71:0/2902058380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64a40ad270 0x7f64a415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.791+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 >> 172.21.15.71:0/2902058380 conn(0x7f64a40b1950 msgr2=0x7f64a40b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.792+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 shutdown_connections 2024-09-16T07:25:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.792+0000 7f64a9400640 1 -- 172.21.15.71:0/2902058380 wait complete. 2024-09-16T07:25:25.808 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:25:25.808 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:25:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.966+0000 7fd706c00640 1 -- 172.21.15.71:0/1953593308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 msgr2=0x7fd7000b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.966+0000 7fd706c00640 1 --2- 172.21.15.71:0/1953593308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 0x7fd7000b1770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd7000d4ea0 tx=0x7fd6f802a540 comp rx=0 tx=0).stop 2024-09-16T07:25:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.966+0000 7fd706c00640 1 -- 172.21.15.71:0/1953593308 shutdown_connections 2024-09-16T07:25:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.966+0000 7fd706c00640 1 --2- 172.21.15.71:0/1953593308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 0x7fd7000b1770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.966+0000 7fd706c00640 1 --2- 172.21.15.71:0/1953593308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7000b5370 0x7fd7000b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.966+0000 7fd706c00640 1 -- 172.21.15.71:0/1953593308 >> 172.21.15.71:0/1953593308 conn(0x7fd7000d3df0 msgr2=0x7fd7000d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.966+0000 7fd706c00640 1 -- 172.21.15.71:0/1953593308 shutdown_connections 2024-09-16T07:25:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.966+0000 7fd706c00640 1 -- 172.21.15.71:0/1953593308 wait complete. 2024-09-16T07:25:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.968+0000 7fd706c00640 1 Processor -- start 2024-09-16T07:25:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.968+0000 7fd706c00640 1 -- start start 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd706c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 0x7fd7000b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd706c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7000b5370 0x7fd70014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd706c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd700161440 0x7fd7000d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd706c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd7000b2040 con 0x7fd7000b5370 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd706c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7000b1ec0 con 0x7fd7000b4100 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd706c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd7000b21c0 con 0x7fd700161440 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd705800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 0x7fd7000b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd704e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7000b5370 0x7fd70014d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd705800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 0x7fd7000b8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:52122/0 (socket says 172.21.15.71:52122) 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd704e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7000b5370 0x7fd70014d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60242/0 (socket says 172.21.15.71:60242) 2024-09-16T07:25:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd705800640 1 -- 172.21.15.71:0/2949241436 learned_addr learned my addr 172.21.15.71:0/2949241436 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.969+0000 7fd706200640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd700161440 0x7fd7000d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd704e00640 1 -- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd700161440 msgr2=0x7fd7000d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd704e00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd700161440 0x7fd7000d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd704e00640 1 -- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 msgr2=0x7fd7000b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd704e00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 0x7fd7000b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd704e00640 1 -- 172.21.15.71:0/2949241436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7000ce930 con 0x7fd7000b5370 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd705800640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 0x7fd7000b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd704e00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7000b5370 0x7fd70014d320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00e9f0 tx=0x7fd6ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd6fd600640 1 -- 172.21.15.71:0/2949241436 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00ccc0 con 0x7fd7000b5370 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd6fd600640 1 -- 172.21.15.71:0/2949241436 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6ec010430 con 0x7fd7000b5370 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd6fd600640 1 -- 172.21.15.71:0/2949241436 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec010860 con 0x7fd7000b5370 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7000b8eb0 con 0x7fd7000b5370 2024-09-16T07:25:25.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.970+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7000b8ba0 con 0x7fd7000b5370 2024-09-16T07:25:25.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.971+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7000b1770 con 0x7fd7000b5370 2024-09-16T07:25:25.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.973+0000 7fd6fd600640 1 -- 172.21.15.71:0/2949241436 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fd6ec010a00 con 0x7fd7000b5370 2024-09-16T07:25:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.974+0000 7fd6fd600640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd6e0078bb0 0x7fd6e007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.974+0000 7fd6fd600640 1 -- 172.21.15.71:0/2949241436 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fd6ec09b9a0 con 0x7fd7000b5370 2024-09-16T07:25:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.975+0000 7fd705800640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd6e0078bb0 0x7fd6e007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.975+0000 7fd705800640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd6e0078bb0 0x7fd6e007b070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0007fd0 tx=0x7fd6f0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:25.978+0000 7fd6fd600640 1 -- 172.21.15.71:0/2949241436 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd6ec063470 con 0x7fd7000b5370 2024-09-16T07:25:26.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.197+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd70014d860 con 0x7fd7000b5370 2024-09-16T07:25:26.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.198+0000 7fd6fd600640 1 -- 172.21.15.71:0/2949241436 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7fd6ec067ea0 con 0x7fd7000b5370 2024-09-16T07:25:26.198 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:26.198 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"btime":"2024-09-16T07:23:51:237136+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4834,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3085563418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3085563418},{"type":"v1","addr":"172.21.15.71:6834","nonce":3085563418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3025435692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3025435692},{"type":"v1","addr":"172.21.15.71:6835","nonce":3025435692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3253921551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":3253921551},{"type":"v1","addr":"172.21.15.177:6836","nonce":3253921551}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:23:37.752030+0000","modified":"2024-09-16T07:23:40.774583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1652086843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1652086843},{"type":"v1","addr":"172.21.15.177:6837","nonce":1652086843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4917,"qdb_cluster":[4917]},"id":2}]} 2024-09-16T07:25:26.198 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2024-09-16T07:25:26.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd6e0078bb0 msgr2=0x7fd6e007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd6e0078bb0 0x7fd6e007b070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0007fd0 tx=0x7fd6f0009040 comp rx=0 tx=0).stop 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7000b5370 msgr2=0x7fd70014d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7000b5370 0x7fd70014d320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00e9f0 tx=0x7fd6ec00eeb0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 shutdown_connections 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd6e0078bb0 0x7fd6e007b070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd700161440 0x7fd7000d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7000b5370 0x7fd70014d320 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 --2- 172.21.15.71:0/2949241436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7000b4100 0x7fd7000b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.200+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 >> 172.21.15.71:0/2949241436 conn(0x7fd7000d3df0 msgr2=0x7fd7000d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.201+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 shutdown_connections 2024-09-16T07:25:26.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.201+0000 7fd706c00640 1 -- 172.21.15.71:0/2949241436 wait complete. 2024-09-16T07:25:26.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:25:26.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.374+0000 7f0f7c600640 1 -- 172.21.15.71:0/3169316532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 msgr2=0x7f0f740b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.374+0000 7f0f7c600640 1 --2- 172.21.15.71:0/3169316532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 0x7f0f740b2440 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f7406dac0 tx=0x7f0f6802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.374+0000 7f0f7c600640 1 -- 172.21.15.71:0/3169316532 shutdown_connections 2024-09-16T07:25:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.374+0000 7f0f7c600640 1 --2- 172.21.15.71:0/3169316532 >> v2:172.21.15.177:3300/0 conn(0x7f0f74073fb0 0x7f0f74074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.374+0000 7f0f7c600640 1 --2- 172.21.15.71:0/3169316532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 0x7f0f740b2440 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.374+0000 7f0f7c600640 1 -- 172.21.15.71:0/3169316532 >> 172.21.15.71:0/3169316532 conn(0x7f0f740b1950 msgr2=0x7f0f740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.375+0000 7f0f7c600640 1 -- 172.21.15.71:0/3169316532 shutdown_connections 2024-09-16T07:25:26.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.375+0000 7f0f7c600640 1 -- 172.21.15.71:0/3169316532 wait complete. 2024-09-16T07:25:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.376+0000 7f0f7c600640 1 Processor -- start 2024-09-16T07:25:26.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.376+0000 7f0f7c600640 1 -- start start 2024-09-16T07:25:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7c600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f74073fb0 0x7f0f7414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7c600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f740ccf00 0x7f0f741f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7c600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 0x7f0f740af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:26.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7c600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7406f2e0 con 0x7f0f7415f5a0 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7c600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7406f160 con 0x7f0f740ccf00 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7c600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7406f460 con 0x7f0f74073fb0 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 0x7f0f740af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 0x7f0f740af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60258/0 (socket says 172.21.15.71:60258) 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.377+0000 7f0f7a600640 1 -- 172.21.15.71:0/2218542539 learned_addr learned my addr 172.21.15.71:0/2218542539 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f79200640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f740ccf00 0x7f0f741f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f79c00640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f74073fb0 0x7f0f7414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f7a600640 1 -- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f74073fb0 msgr2=0x7f0f7414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f7a600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f74073fb0 0x7f0f7414e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f7a600640 1 -- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f740ccf00 msgr2=0x7f0f741f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f7a600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f740ccf00 0x7f0f741f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f7a600640 1 -- 172.21.15.71:0/2218542539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f74157560 con 0x7f0f7415f5a0 2024-09-16T07:25:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f79c00640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f74073fb0 0x7f0f7414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:26.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f7a600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 0x7f0f740af8c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00eaa0 tx=0x7f0f6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:26.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f71600640 1 -- 172.21.15.71:0/2218542539 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00cdb0 con 0x7f0f7415f5a0 2024-09-16T07:25:26.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f71600640 1 -- 172.21.15.71:0/2218542539 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f6c004c10 con 0x7f0f7415f5a0 2024-09-16T07:25:26.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f71600640 1 -- 172.21.15.71:0/2218542539 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c005020 con 0x7f0f7415f5a0 2024-09-16T07:25:26.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.378+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7414ec20 con 0x7f0f7415f5a0 2024-09-16T07:25:26.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.379+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f741567a0 con 0x7f0f7415f5a0 2024-09-16T07:25:26.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.379+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7406f9f0 con 0x7f0f7415f5a0 2024-09-16T07:25:26.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.381+0000 7f0f71600640 1 -- 172.21.15.71:0/2218542539 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f0f6c0051c0 con 0x7f0f7415f5a0 2024-09-16T07:25:26.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.381+0000 7f0f71600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f54078bb0 0x7f0f5407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:26.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.382+0000 7f0f71600640 1 -- 172.21.15.71:0/2218542539 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f0f6c09b9a0 con 0x7f0f7415f5a0 2024-09-16T07:25:26.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.382+0000 7f0f79c00640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f54078bb0 0x7f0f5407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.383+0000 7f0f79c00640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f54078bb0 0x7f0f5407b070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f68033000 tx=0x7f0f68004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:26.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.385+0000 7f0f71600640 1 -- 172.21.15.71:0/2218542539 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0f6c016030 con 0x7f0f7415f5a0 2024-09-16T07:25:26.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.606+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f740cf7b0 con 0x7f0f7415f5a0 2024-09-16T07:25:26.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.606+0000 7f0f71600640 1 -- 172.21.15.71:0/2218542539 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7f0f6c063370 con 0x7f0f7415f5a0 2024-09-16T07:25:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:26.606 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"btime":"2024-09-16T07:23:51:237136+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4834,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3085563418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3085563418},{"type":"v1","addr":"172.21.15.71:6834","nonce":3085563418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3025435692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3025435692},{"type":"v1","addr":"172.21.15.71:6835","nonce":3025435692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3253921551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":3253921551},{"type":"v1","addr":"172.21.15.177:6836","nonce":3253921551}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:23:37.752030+0000","modified":"2024-09-16T07:23:40.774583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1652086843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1652086843},{"type":"v1","addr":"172.21.15.177:6837","nonce":1652086843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4917,"qdb_cluster":[4917]},"id":2}]} 2024-09-16T07:25:26.606 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2024-09-16T07:25:26.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f54078bb0 msgr2=0x7f0f5407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f54078bb0 0x7f0f5407b070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f68033000 tx=0x7f0f68004730 comp rx=0 tx=0).stop 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 msgr2=0x7f0f740af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 0x7f0f740af8c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00eaa0 tx=0x7f0f6c00ef60 comp rx=0 tx=0).stop 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 shutdown_connections 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0f54078bb0 0x7f0f5407b070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f7415f5a0 0x7f0f740af8c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f740ccf00 0x7f0f741f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 --2- 172.21.15.71:0/2218542539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f74073fb0 0x7f0f7414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 >> 172.21.15.71:0/2218542539 conn(0x7f0f740b1950 msgr2=0x7f0f740ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 shutdown_connections 2024-09-16T07:25:26.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.609+0000 7f0f7c600640 1 -- 172.21.15.71:0/2218542539 wait complete. 2024-09-16T07:25:26.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:25:26.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.782+0000 7fdf6d800640 1 -- 172.21.15.71:0/3395312315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 msgr2=0x7fdf680b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.782+0000 7fdf6d800640 1 --2- 172.21.15.71:0/3395312315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 0x7fdf680b2440 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdf680cfbb0 tx=0x7fdf5802a560 comp rx=0 tx=0).stop 2024-09-16T07:25:26.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.782+0000 7fdf6d800640 1 -- 172.21.15.71:0/3395312315 shutdown_connections 2024-09-16T07:25:26.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.782+0000 7fdf6d800640 1 --2- 172.21.15.71:0/3395312315 >> v2:172.21.15.177:3300/0 conn(0x7fdf680ccf00 0x7fdf680acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.782+0000 7fdf6d800640 1 --2- 172.21.15.71:0/3395312315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 0x7fdf680b2440 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.782+0000 7fdf6d800640 1 -- 172.21.15.71:0/3395312315 >> 172.21.15.71:0/3395312315 conn(0x7fdf680b1950 msgr2=0x7fdf680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:26.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.782+0000 7fdf6d800640 1 -- 172.21.15.71:0/3395312315 shutdown_connections 2024-09-16T07:25:26.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.782+0000 7fdf6d800640 1 -- 172.21.15.71:0/3395312315 wait complete. 2024-09-16T07:25:26.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.784+0000 7fdf6d800640 1 Processor -- start 2024-09-16T07:25:26.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.784+0000 7fdf6d800640 1 -- start start 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.785+0000 7fdf6d800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf680ad270 0x7fdf681548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.785+0000 7fdf6d800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf680ccf00 0x7fdf681e4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.785+0000 7fdf6d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 0x7fdf681f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.785+0000 7fdf6d800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6806f160 con 0x7fdf6815f5a0 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.785+0000 7fdf6d800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6806f2e0 con 0x7fdf680ccf00 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.785+0000 7fdf6d800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6806f460 con 0x7fdf680ad270 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.785+0000 7fdf67400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 0x7fdf681f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf67400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 0x7fdf681f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60276/0 (socket says 172.21.15.71:60276) 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf67400640 1 -- 172.21.15.71:0/2044251934 learned_addr learned my addr 172.21.15.71:0/2044251934 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf66a00640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf680ad270 0x7fdf681548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:26.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf5e000640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf680ccf00 0x7fdf681e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf67400640 1 -- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf680ad270 msgr2=0x7fdf681548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf67400640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf680ad270 0x7fdf681548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf67400640 1 -- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf680ccf00 msgr2=0x7fdf681e4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf67400640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf680ccf00 0x7fdf681e4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf67400640 1 -- 172.21.15.71:0/2044251934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf681f5cf0 con 0x7fdf6815f5a0 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf5e000640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf680ccf00 0x7fdf681e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf67400640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 0x7fdf681f57b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c7a0 tx=0x7fdf6000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf5f400640 1 -- 172.21.15.71:0/2044251934 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60003b70 con 0x7fdf6815f5a0 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf5f400640 1 -- 172.21.15.71:0/2044251934 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf60003d10 con 0x7fdf6815f5a0 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf680cde80 con 0x7fdf6815f5a0 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.786+0000 7fdf5f400640 1 -- 172.21.15.71:0/2044251934 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000aab0 con 0x7fdf6815f5a0 2024-09-16T07:25:26.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.787+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf680ce440 con 0x7fdf6815f5a0 2024-09-16T07:25:26.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.788+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6806f9f0 con 0x7fdf6815f5a0 2024-09-16T07:25:26.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.789+0000 7fdf5f400640 1 -- 172.21.15.71:0/2044251934 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fdf6000ac50 con 0x7fdf6815f5a0 2024-09-16T07:25:26.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.790+0000 7fdf5f400640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf44078bb0 0x7fdf4407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:26.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.790+0000 7fdf5f400640 1 -- 172.21.15.71:0/2044251934 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fdf6009af00 con 0x7fdf6815f5a0 2024-09-16T07:25:26.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.791+0000 7fdf66a00640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf44078bb0 0x7fdf4407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:26.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.791+0000 7fdf66a00640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf44078bb0 0x7fdf4407b070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdf58033000 tx=0x7fdf58004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:26.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.795+0000 7fdf5f400640 1 -- 172.21.15.71:0/2044251934 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdf681f5cf0 con 0x7fdf6815f5a0 2024-09-16T07:25:26.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.978+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf680cfa10 con 0x7fdf6815f5a0 2024-09-16T07:25:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.979+0000 7fdf5f400640 1 -- 172.21.15.71:0/2044251934 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16931 (secure 0 0 0) 0x7fdf60062e90 con 0x7fdf6815f5a0 2024-09-16T07:25:26.979 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:26.979 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":37,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:24:30.352331+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:23:36.070239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T07:23:36.713734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]},{"pgid":"5.35","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf44078bb0 msgr2=0x7fdf4407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf44078bb0 0x7fdf4407b070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdf58033000 tx=0x7fdf58004730 comp rx=0 tx=0).stop 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 msgr2=0x7fdf681f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 0x7fdf681f57b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c7a0 tx=0x7fdf6000cc60 comp rx=0 tx=0).stop 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 shutdown_connections 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdf44078bb0 0x7fdf4407b070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6815f5a0 0x7fdf681f57b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf680ccf00 0x7fdf681e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 --2- 172.21.15.71:0/2044251934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf680ad270 0x7fdf681548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:26.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 >> 172.21.15.71:0/2044251934 conn(0x7fdf680b1950 msgr2=0x7fdf680cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:26.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 shutdown_connections 2024-09-16T07:25:26.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:26.981+0000 7fdf6d800640 1 -- 172.21.15.71:0/2044251934 wait complete. 2024-09-16T07:25:26.996 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:25:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.154+0000 7f6dc0e00640 1 -- 172.21.15.71:0/3590115355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc15f5a0 msgr2=0x7f6dbc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.154+0000 7f6dc0e00640 1 --2- 172.21.15.71:0/3590115355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc15f5a0 0x7f6dbc0b2440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6dac005fd0 tx=0x7f6dac02aa10 comp rx=0 tx=0).stop 2024-09-16T07:25:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.154+0000 7f6dc0e00640 1 -- 172.21.15.71:0/3590115355 shutdown_connections 2024-09-16T07:25:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.154+0000 7f6dc0e00640 1 --2- 172.21.15.71:0/3590115355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc15f5a0 0x7f6dbc0b2440 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.154+0000 7f6dc0e00640 1 -- 172.21.15.71:0/3590115355 >> 172.21.15.71:0/3590115355 conn(0x7f6dbc0b1950 msgr2=0x7f6dbc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.154+0000 7f6dc0e00640 1 -- 172.21.15.71:0/3590115355 shutdown_connections 2024-09-16T07:25:27.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.154+0000 7f6dc0e00640 1 -- 172.21.15.71:0/3590115355 wait complete. 2024-09-16T07:25:27.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.156+0000 7f6dc0e00640 1 Processor -- start 2024-09-16T07:25:27.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.156+0000 7f6dc0e00640 1 -- start start 2024-09-16T07:25:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6dc0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc073fb0 0x7f6dbc15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6dc0e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc0ccf00 0x7f6dbc146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6dc0e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc15f5a0 0x7f6dbc0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6dc0e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc06f9f0 con 0x7f6dbc073fb0 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6dc0e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc06f870 con 0x7f6dbc0ccf00 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6dc0e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6dbc074420 con 0x7f6dbc15f5a0 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc073fb0 0x7f6dbc15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc073fb0 0x7f6dbc15de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60290/0 (socket says 172.21.15.71:60290) 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9e00640 1 -- 172.21.15.71:0/4243551972 learned_addr learned my addr 172.21.15.71:0/4243551972 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9400640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc0ccf00 0x7f6dbc146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6dba800640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc15f5a0 0x7f6dbc0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9e00640 1 -- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc15f5a0 msgr2=0x7f6dbc0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:27.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc15f5a0 0x7f6dbc0cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9e00640 1 -- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc0ccf00 msgr2=0x7f6dbc146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc0ccf00 0x7f6dbc146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6db9e00640 1 -- 172.21.15.71:0/4243551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc0c60d0 con 0x7f6dbc073fb0 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.157+0000 7f6dba800640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc15f5a0 0x7f6dbc0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.158+0000 7f6db9e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc073fb0 0x7f6dbc15de90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6dac000f80 tx=0x7f6dac003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.158+0000 7f6db2000640 1 -- 172.21.15.71:0/4243551972 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac046070 con 0x7f6dbc073fb0 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.158+0000 7f6db2000640 1 -- 172.21.15.71:0/4243551972 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dac0046c0 con 0x7f6dbc073fb0 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.158+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc0c6370 con 0x7f6dbc073fb0 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.158+0000 7f6db2000640 1 -- 172.21.15.71:0/4243551972 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac033070 con 0x7f6dbc073fb0 2024-09-16T07:25:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.158+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc0c6c10 con 0x7f6dbc073fb0 2024-09-16T07:25:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.159+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc0ad140 con 0x7f6dbc073fb0 2024-09-16T07:25:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.160+0000 7f6db2000640 1 -- 172.21.15.71:0/4243551972 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f6dac049070 con 0x7f6dbc073fb0 2024-09-16T07:25:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.161+0000 7f6db2000640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d9c078bb0 0x7f6d9c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.161+0000 7f6db2000640 1 -- 172.21.15.71:0/4243551972 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f6dac0bc2c0 con 0x7f6dbc073fb0 2024-09-16T07:25:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.162+0000 7f6db9400640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d9c078bb0 0x7f6d9c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.162+0000 7f6db9400640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d9c078bb0 0x7f6d9c07b070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc1551b0 tx=0x7f6da8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:27.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.165+0000 7f6db2000640 1 -- 172.21.15.71:0/4243551972 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6dac03e070 con 0x7f6dbc073fb0 2024-09-16T07:25:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.387+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6dbc0c6540 con 0x7f6dbc073fb0 2024-09-16T07:25:27.387 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:25:27.388+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:25:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.390+0000 7f6db2000640 1 -- 172.21.15.71:0/4243551972 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f6dac084250 con 0x7f6dbc073fb0 2024-09-16T07:25:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:25:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d9c078bb0 msgr2=0x7f6d9c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d9c078bb0 0x7f6d9c07b070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc1551b0 tx=0x7f6da8009040 comp rx=0 tx=0).stop 2024-09-16T07:25:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc073fb0 msgr2=0x7f6dbc15de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc073fb0 0x7f6dbc15de90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6dac000f80 tx=0x7f6dac003f90 comp rx=0 tx=0).stop 2024-09-16T07:25:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 shutdown_connections 2024-09-16T07:25:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d9c078bb0 0x7f6d9c07b070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc15f5a0 0x7f6dbc0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc0ccf00 0x7f6dbc146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 --2- 172.21.15.71:0/4243551972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dbc073fb0 0x7f6dbc15de90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 >> 172.21.15.71:0/4243551972 conn(0x7f6dbc0b1950 msgr2=0x7f6dbc0ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 shutdown_connections 2024-09-16T07:25:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.393+0000 7f6dc0e00640 1 -- 172.21.15.71:0/4243551972 wait complete. 2024-09-16T07:25:27.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:25:27.461 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:25:27.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.569+0000 7f845e800640 1 -- 172.21.15.71:0/3308138578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8458145750 msgr2=0x7f8458073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:27.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.569+0000 7f845e800640 1 --2- 172.21.15.71:0/3308138578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8458145750 0x7f8458073440 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8448005fd0 tx=0x7f844802aa10 comp rx=0 tx=0).stop 2024-09-16T07:25:27.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.569+0000 7f845e800640 1 -- 172.21.15.71:0/3308138578 shutdown_connections 2024-09-16T07:25:27.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.569+0000 7f845e800640 1 --2- 172.21.15.71:0/3308138578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f845806efa0 0x7f84580cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.569+0000 7f845e800640 1 --2- 172.21.15.71:0/3308138578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8458145750 0x7f8458073440 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.569+0000 7f845e800640 1 -- 172.21.15.71:0/3308138578 >> 172.21.15.71:0/3308138578 conn(0x7f84580b1840 msgr2=0x7f84580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:27.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.569+0000 7f845e800640 1 -- 172.21.15.71:0/3308138578 shutdown_connections 2024-09-16T07:25:27.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.569+0000 7f845e800640 1 -- 172.21.15.71:0/3308138578 wait complete. 2024-09-16T07:25:27.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.571+0000 7f845e800640 1 Processor -- start 2024-09-16T07:25:27.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.571+0000 7f845e800640 1 -- start start 2024-09-16T07:25:27.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f845e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f845806efa0 0x7f84580c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:27.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f845e800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580ccb00 0x7f84581f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:27.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f845e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8458145750 0x7f84581f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:27.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f845e800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84580b26b0 con 0x7f845806efa0 2024-09-16T07:25:27.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f845e800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f84580cedd0 con 0x7f8458145750 2024-09-16T07:25:27.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f845e800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8458160100 con 0x7f84580ccb00 2024-09-16T07:25:27.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f845806efa0 0x7f84580c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:27.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f845806efa0 0x7f84580c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60312/0 (socket says 172.21.15.71:60312) 2024-09-16T07:25:27.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457400640 1 -- 172.21.15.71:0/1731455523 learned_addr learned my addr 172.21.15.71:0/1731455523 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8456a00640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580ccb00 0x7f84581f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457e00640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8458145750 0x7f84581f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457400640 1 -- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580ccb00 msgr2=0x7f84581f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457400640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580ccb00 0x7f84581f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457400640 1 -- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8458145750 msgr2=0x7f84581f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457400640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8458145750 0x7f84581f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457400640 1 -- 172.21.15.71:0/1731455523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84580af120 con 0x7f845806efa0 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8456a00640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580ccb00 0x7f84581f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.572+0000 7f8457e00640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8458145750 0x7f84581f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.573+0000 7f8457400640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f845806efa0 0x7f84580c4ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8448006010 tx=0x7f8448003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.573+0000 7f844f400640 1 -- 172.21.15.71:0/1731455523 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448046070 con 0x7f845806efa0 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.573+0000 7f844f400640 1 -- 172.21.15.71:0/1731455523 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8448004560 con 0x7f845806efa0 2024-09-16T07:25:27.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.573+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84580aa380 con 0x7f845806efa0 2024-09-16T07:25:27.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.573+0000 7f844f400640 1 -- 172.21.15.71:0/1731455523 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448033070 con 0x7f845806efa0 2024-09-16T07:25:27.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.573+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84580aa940 con 0x7f845806efa0 2024-09-16T07:25:27.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.574+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84581605b0 con 0x7f845806efa0 2024-09-16T07:25:27.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.575+0000 7f844f400640 1 -- 172.21.15.71:0/1731455523 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f8448049070 con 0x7f845806efa0 2024-09-16T07:25:27.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.576+0000 7f844f400640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8438078be0 0x7f843807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:27.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.576+0000 7f844f400640 1 -- 172.21.15.71:0/1731455523 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f84480bc020 con 0x7f845806efa0 2024-09-16T07:25:27.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.577+0000 7f8456a00640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8438078be0 0x7f843807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:27.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.577+0000 7f8456a00640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8438078be0 0x7f843807b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f84581523e0 tx=0x7f8444005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:27.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.580+0000 7f844f400640 1 -- 172.21.15.71:0/1731455523 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f844803e070 con 0x7f845806efa0 2024-09-16T07:25:27.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:27.806+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f84580b1fa0 con 0x7f845806efa0 2024-09-16T07:25:28.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.398+0000 7f844f400640 1 -- 172.21.15.71:0/1731455523 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f8448084030 con 0x7f845806efa0 2024-09-16T07:25:28.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.400+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8438078be0 msgr2=0x7f843807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:28.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.400+0000 7f845e800640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8438078be0 0x7f843807b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f84581523e0 tx=0x7f8444005f90 comp rx=0 tx=0).stop 2024-09-16T07:25:28.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.400+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f845806efa0 msgr2=0x7f84580c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:28.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.400+0000 7f845e800640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f845806efa0 0x7f84580c4ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8448006010 tx=0x7f8448003f10 comp rx=0 tx=0).stop 2024-09-16T07:25:28.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.401+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 shutdown_connections 2024-09-16T07:25:28.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.401+0000 7f845e800640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8438078be0 0x7f843807b0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.401+0000 7f845e800640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8458145750 0x7f84581f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.401+0000 7f845e800640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580ccb00 0x7f84581f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.401+0000 7f845e800640 1 --2- 172.21.15.71:0/1731455523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f845806efa0 0x7f84580c4ab0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.401+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 >> 172.21.15.71:0/1731455523 conn(0x7f84580b1840 msgr2=0x7f84580cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:28.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.401+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 shutdown_connections 2024-09-16T07:25:28.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.401+0000 7f845e800640 1 -- 172.21.15.71:0/1731455523 wait complete. 2024-09-16T07:25:28.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:25:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.582+0000 7faaf1c00640 1 -- 172.21.15.71:0/3221620883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec15dfc0 msgr2=0x7faaec06f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.582+0000 7faaf1c00640 1 --2- 172.21.15.71:0/3221620883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec15dfc0 0x7faaec06f300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faadc005fd0 tx=0x7faadc02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.582+0000 7faaf1c00640 1 -- 172.21.15.71:0/3221620883 shutdown_connections 2024-09-16T07:25:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.582+0000 7faaf1c00640 1 --2- 172.21.15.71:0/3221620883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaec06ee00 0x7faaec0ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.582+0000 7faaf1c00640 1 --2- 172.21.15.71:0/3221620883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec15dfc0 0x7faaec06f300 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.582+0000 7faaf1c00640 1 -- 172.21.15.71:0/3221620883 >> 172.21.15.71:0/3221620883 conn(0x7faaec0b18e0 msgr2=0x7faaec0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.582+0000 7faaf1c00640 1 -- 172.21.15.71:0/3221620883 shutdown_connections 2024-09-16T07:25:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.582+0000 7faaf1c00640 1 -- 172.21.15.71:0/3221620883 wait complete. 2024-09-16T07:25:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.584+0000 7faaf1c00640 1 Processor -- start 2024-09-16T07:25:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.584+0000 7faaf1c00640 1 -- start start 2024-09-16T07:25:28.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaf1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec06ee00 0x7faaec15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaf1c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaec0b4770 0x7faaec1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaf1c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaec15dfc0 0x7faaec15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaf1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0adfa0 con 0x7faaec06ee00 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaf1c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faaec0ade20 con 0x7faaec0b4770 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaf1c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7faaec0ae120 con 0x7faaec15dfc0 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaeaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec06ee00 0x7faaec15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaeaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec06ee00 0x7faaec15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60326/0 (socket says 172.21.15.71:60326) 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaeaa00640 1 -- 172.21.15.71:0/2196116597 learned_addr learned my addr 172.21.15.71:0/2196116597 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaeb400640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaec15dfc0 0x7faaec15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:28.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faae2000640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaec0b4770 0x7faaec1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:28.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaeaa00640 1 -- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaec15dfc0 msgr2=0x7faaec15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:28.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.585+0000 7faaeaa00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaec15dfc0 0x7faaec15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faaeaa00640 1 -- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaec0b4770 msgr2=0x7faaec1540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:28.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faaeaa00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaec0b4770 0x7faaec1540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:28.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faaeaa00640 1 -- 172.21.15.71:0/2196116597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec15b3d0 con 0x7faaec06ee00 2024-09-16T07:25:28.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faaeaa00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec06ee00 0x7faaec15ae90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faadc034040 tx=0x7faadc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:28.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faae3400640 1 -- 172.21.15.71:0/2196116597 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc03e030 con 0x7faaec06ee00 2024-09-16T07:25:28.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faae3400640 1 -- 172.21.15.71:0/2196116597 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faadc003ae0 con 0x7faaec06ee00 2024-09-16T07:25:28.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faae3400640 1 -- 172.21.15.71:0/2196116597 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc035040 con 0x7faaec06ee00 2024-09-16T07:25:28.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec15e790 con 0x7faaec06ee00 2024-09-16T07:25:28.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.586+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec15b950 con 0x7faaec06ee00 2024-09-16T07:25:28.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.587+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec069360 con 0x7faaec06ee00 2024-09-16T07:25:28.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.588+0000 7faae3400640 1 -- 172.21.15.71:0/2196116597 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7faadc041030 con 0x7faaec06ee00 2024-09-16T07:25:28.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.589+0000 7faae3400640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faac8078bb0 0x7faac807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:28.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.589+0000 7faae3400640 1 -- 172.21.15.71:0/2196116597 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7faadc0bb760 con 0x7faaec06ee00 2024-09-16T07:25:28.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.590+0000 7faae2000640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faac8078bb0 0x7faac807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:28.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.591+0000 7faae2000640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faac8078bb0 0x7faac807b070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faaec0d0120 tx=0x7faad4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:28.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.594+0000 7faae3400640 1 -- 172.21.15.71:0/2196116597 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faadc083200 con 0x7faaec06ee00 2024-09-16T07:25:28.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:28.779+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faaec0ad6d0 con 0x7faaec06ee00 2024-09-16T07:25:29.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.400+0000 7faae3400640 1 -- 172.21.15.71:0/2196116597 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) ==== 181+0+0 (secure 0 0 0) 0x7faadc087c30 con 0x7faaec06ee00 2024-09-16T07:25:29.399 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:25:29.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.402+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faac8078bb0 msgr2=0x7faac807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:29.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.402+0000 7faaf1c00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faac8078bb0 0x7faac807b070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faaec0d0120 tx=0x7faad4009040 comp rx=0 tx=0).stop 2024-09-16T07:25:29.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.402+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec06ee00 msgr2=0x7faaec15ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:29.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.402+0000 7faaf1c00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec06ee00 0x7faaec15ae90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faadc034040 tx=0x7faadc0035f0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.403+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 shutdown_connections 2024-09-16T07:25:29.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.403+0000 7faaf1c00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faac8078bb0 0x7faac807b070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.403+0000 7faaf1c00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaec15dfc0 0x7faaec15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.403+0000 7faaf1c00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaec0b4770 0x7faaec1540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.403+0000 7faaf1c00640 1 --2- 172.21.15.71:0/2196116597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaec06ee00 0x7faaec15ae90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.403+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 >> 172.21.15.71:0/2196116597 conn(0x7faaec0b18e0 msgr2=0x7faaec0739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:29.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.403+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 shutdown_connections 2024-09-16T07:25:29.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.403+0000 7faaf1c00640 1 -- 172.21.15.71:0/2196116597 wait complete. 2024-09-16T07:25:29.422 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:25:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.585+0000 7f20a5200640 1 -- 172.21.15.71:0/3341378434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 msgr2=0x7f20a006eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.585+0000 7f20a5200640 1 --2- 172.21.15.71:0/3341378434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 0x7f20a006eb80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2098005fd0 tx=0x7f209802aa00 comp rx=0 tx=0).stop 2024-09-16T07:25:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.585+0000 7f20a5200640 1 -- 172.21.15.71:0/3341378434 shutdown_connections 2024-09-16T07:25:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.585+0000 7f20a5200640 1 --2- 172.21.15.71:0/3341378434 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20a00ccca0 0x7f20a00cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.585+0000 7f20a5200640 1 --2- 172.21.15.71:0/3341378434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 0x7f20a006eb80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.585+0000 7f20a5200640 1 -- 172.21.15.71:0/3341378434 >> 172.21.15.71:0/3341378434 conn(0x7f20a00b18e0 msgr2=0x7f20a00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.585+0000 7f20a5200640 1 -- 172.21.15.71:0/3341378434 shutdown_connections 2024-09-16T07:25:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.585+0000 7f20a5200640 1 -- 172.21.15.71:0/3341378434 wait complete. 2024-09-16T07:25:29.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.587+0000 7f20a5200640 1 Processor -- start 2024-09-16T07:25:29.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.587+0000 7f20a5200640 1 -- start start 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f20a5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 0x7f20a01f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f20a5200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20a00ccca0 0x7f20a01e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f20a5200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20a0158900 0x7f20a01563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f20a5200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20a00ced60 con 0x7f20a006e6f0 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f20a5200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f20a00cebe0 con 0x7f20a00ccca0 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f20a5200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f20a00ceee0 con 0x7f20a0158900 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 0x7f20a01f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 0x7f20a01f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60334/0 (socket says 172.21.15.71:60334) 2024-09-16T07:25:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209de00640 1 -- 172.21.15.71:0/943328672 learned_addr learned my addr 172.21.15.71:0/943328672 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:29.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209d400640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20a00ccca0 0x7f20a01e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:29.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209e800640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20a0158900 0x7f20a01563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:29.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209de00640 1 -- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20a0158900 msgr2=0x7f20a01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:29.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209de00640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20a0158900 0x7f20a01563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209de00640 1 -- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20a00ccca0 msgr2=0x7f20a01e58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:29.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209de00640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20a00ccca0 0x7f20a01e58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:29.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.588+0000 7f209de00640 1 -- 172.21.15.71:0/943328672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a01e7df0 con 0x7f20a006e6f0 2024-09-16T07:25:29.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.589+0000 7f209de00640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 0x7f20a01f6790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f209000b3f0 tx=0x7f209000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:29.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.589+0000 7f2096000640 1 -- 172.21.15.71:0/943328672 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090015020 con 0x7f20a006e6f0 2024-09-16T07:25:29.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.589+0000 7f2096000640 1 -- 172.21.15.71:0/943328672 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2090011920 con 0x7f20a006e6f0 2024-09-16T07:25:29.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.589+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a01e80d0 con 0x7f20a006e6f0 2024-09-16T07:25:29.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.589+0000 7f2096000640 1 -- 172.21.15.71:0/943328672 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090011d70 con 0x7f20a006e6f0 2024-09-16T07:25:29.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.589+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a01e8690 con 0x7f20a006e6f0 2024-09-16T07:25:29.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.590+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a0069360 con 0x7f20a006e6f0 2024-09-16T07:25:29.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.591+0000 7f2096000640 1 -- 172.21.15.71:0/943328672 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f2090018020 con 0x7f20a006e6f0 2024-09-16T07:25:29.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.592+0000 7f2096000640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2080078bb0 0x7f208007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:29.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.592+0000 7f2096000640 1 -- 172.21.15.71:0/943328672 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5473+0+0 (secure 0 0 0) 0x7f209009ab10 con 0x7f20a006e6f0 2024-09-16T07:25:29.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.593+0000 7f209d400640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2080078bb0 0x7f208007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:29.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.593+0000 7f209d400640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2080078bb0 0x7f208007b070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f208c005fd0 tx=0x7f208c005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:29.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.596+0000 7f2096000640 1 -- 172.21.15.71:0/943328672 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f209001f030 con 0x7f20a006e6f0 2024-09-16T07:25:29.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:29.785+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20a00cf470 con 0x7f20a006e6f0 2024-09-16T07:25:30.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.406+0000 7f2096000640 1 -- 172.21.15.71:0/943328672 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) ==== 169+0+0 (secure 0 0 0) 0x7f2090062d90 con 0x7f20a006e6f0 2024-09-16T07:25:30.406 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:25:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.410+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2080078bb0 msgr2=0x7f208007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.410+0000 7f20a5200640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2080078bb0 0x7f208007b070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f208c005fd0 tx=0x7f208c005850 comp rx=0 tx=0).stop 2024-09-16T07:25:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.410+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 msgr2=0x7f20a01f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.410+0000 7f20a5200640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 0x7f20a01f6790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f209000b3f0 tx=0x7f209000b8b0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.411+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 shutdown_connections 2024-09-16T07:25:30.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.411+0000 7f20a5200640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2080078bb0 0x7f208007b070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.411+0000 7f20a5200640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20a0158900 0x7f20a01563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.411+0000 7f20a5200640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20a00ccca0 0x7f20a01e58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.411+0000 7f20a5200640 1 --2- 172.21.15.71:0/943328672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20a006e6f0 0x7f20a01f6790 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.411+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 >> 172.21.15.71:0/943328672 conn(0x7f20a00b18e0 msgr2=0x7f20a0158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:30.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.411+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 shutdown_connections 2024-09-16T07:25:30.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.411+0000 7f20a5200640 1 -- 172.21.15.71:0/943328672 wait complete. 2024-09-16T07:25:30.437 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:25:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.715+0000 7fa937e00640 1 -- 172.21.15.71:0/1074257484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 msgr2=0x7fa93806f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:30.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.715+0000 7fa937e00640 1 --2- 172.21.15.71:0/1074257484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 0x7fa93806f770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa930005fd0 tx=0x7fa93002a8d0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.715+0000 7fa937e00640 1 -- 172.21.15.71:0/1074257484 shutdown_connections 2024-09-16T07:25:30.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.715+0000 7fa937e00640 1 --2- 172.21.15.71:0/1074257484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 0x7fa93806f770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.715+0000 7fa937e00640 1 --2- 172.21.15.71:0/1074257484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa93815a620 0x7fa9380cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.715+0000 7fa937e00640 1 -- 172.21.15.71:0/1074257484 >> 172.21.15.71:0/1074257484 conn(0x7fa9380cfa20 msgr2=0x7fa9380cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:30.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.715+0000 7fa937e00640 1 -- 172.21.15.71:0/1074257484 shutdown_connections 2024-09-16T07:25:30.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.715+0000 7fa937e00640 1 -- 172.21.15.71:0/1074257484 wait complete. 2024-09-16T07:25:30.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.717+0000 7fa937e00640 1 Processor -- start 2024-09-16T07:25:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.718+0000 7fa937e00640 1 -- start start 2024-09-16T07:25:30.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.719+0000 7fa937e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9380aec20 0x7fa938073220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:30.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.719+0000 7fa937e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 0x7fa938073760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:30.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.719+0000 7fa937e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa93815a620 0x7fa9380b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:30.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa937400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa93815a620 0x7fa9380b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:30.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa936a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9380aec20 0x7fa938073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:30.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa936a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9380aec20 0x7fa938073220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:48876/0 (socket says 172.21.15.71:48876) 2024-09-16T07:25:30.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa937400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa93815a620 0x7fa9380b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60338/0 (socket says 172.21.15.71:60338) 2024-09-16T07:25:30.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa936000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 0x7fa938073760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:30.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa93806fe30 con 0x7fa93815a620 2024-09-16T07:25:30.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa93806fcb0 con 0x7fa9380af040 2024-09-16T07:25:30.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa93806ffb0 con 0x7fa9380aec20 2024-09-16T07:25:30.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.720+0000 7fa936a00640 1 -- 172.21.15.71:0/1768289919 learned_addr learned my addr 172.21.15.71:0/1768289919 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:30.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.721+0000 7fa936000640 1 -- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9380aec20 msgr2=0x7fa938073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:30.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.721+0000 7fa936000640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9380aec20 0x7fa938073220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.721+0000 7fa936000640 1 -- 172.21.15.71:0/1768289919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa93815a620 msgr2=0x7fa9380b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:30.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.721+0000 7fa936000640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa93815a620 0x7fa9380b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:30.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.721+0000 7fa936000640 1 -- 172.21.15.71:0/1768289919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380c3460 con 0x7fa9380af040 2024-09-16T07:25:30.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.722+0000 7fa937400640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa93815a620 0x7fa9380b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:30.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.723+0000 7fa936000640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 0x7fa938073760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa92800c400 tx=0x7fa92800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:30.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.724+0000 7fa92ea00640 1 -- 172.21.15.71:0/1768289919 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928011790 con 0x7fa9380af040 2024-09-16T07:25:30.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.724+0000 7fa92ea00640 1 -- 172.21.15.71:0/1768289919 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa928017070 con 0x7fa9380af040 2024-09-16T07:25:30.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.724+0000 7fa92ea00640 1 -- 172.21.15.71:0/1768289919 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928011c40 con 0x7fa9380af040 2024-09-16T07:25:30.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.724+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa938148a40 con 0x7fa9380af040 2024-09-16T07:25:30.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.725+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa938151350 con 0x7fa9380af040 2024-09-16T07:25:30.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.726+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93806f770 con 0x7fa9380af040 2024-09-16T07:25:30.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.730+0000 7fa92ea00640 1 -- 172.21.15.71:0/1768289919 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fa92801c460 con 0x7fa9380af040 2024-09-16T07:25:30.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.731+0000 7fa92ea00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa918078c90 0x7fa91807b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:30.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.731+0000 7fa92ea00640 1 -- 172.21.15.71:0/1768289919 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7fa92809b520 con 0x7fa9380af040 2024-09-16T07:25:30.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.731+0000 7fa936a00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa918078c90 0x7fa91807b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:30.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.733+0000 7fa936a00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa918078c90 0x7fa91807b150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa9240058c0 tx=0x7fa924005ef0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:30.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:30.744+0000 7fa92ea00640 1 -- 172.21.15.71:0/1768289919 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa9380c3460 con 0x7fa9380af040 2024-09-16T07:25:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.001+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa93815ad80 con 0x7fa9380af040 2024-09-16T07:25:31.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.007+0000 7fa92ea00640 1 -- 172.21.15.71:0/1768289919 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v30) ==== 71+0+0 (secure 0 0 0) 0x7fa9280639b0 con 0x7fa9380af040 2024-09-16T07:25:31.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa918078c90 msgr2=0x7fa91807b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:31.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa918078c90 0x7fa91807b150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa9240058c0 tx=0x7fa924005ef0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 msgr2=0x7fa938073760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:31.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 0x7fa938073760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa92800c400 tx=0x7fa92800c8c0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 shutdown_connections 2024-09-16T07:25:31.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa918078c90 0x7fa91807b150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa93815a620 0x7fa9380b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9380af040 0x7fa938073760 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 --2- 172.21.15.71:0/1768289919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9380aec20 0x7fa938073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 >> 172.21.15.71:0/1768289919 conn(0x7fa9380cfa20 msgr2=0x7fa9380afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:31.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 shutdown_connections 2024-09-16T07:25:31.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.009+0000 7fa937e00640 1 -- 172.21.15.71:0/1768289919 wait complete. 2024-09-16T07:25:31.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-16T07:25:31.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.181+0000 7f5486c00640 1 -- 172.21.15.71:0/618379328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 msgr2=0x7f54800b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:31.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.181+0000 7f5486c00640 1 --2- 172.21.15.71:0/618379328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 0x7f54800b47d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f54800af840 tx=0x7f547002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.181+0000 7f5486c00640 1 -- 172.21.15.71:0/618379328 shutdown_connections 2024-09-16T07:25:31.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.181+0000 7f5486c00640 1 --2- 172.21.15.71:0/618379328 >> v2:172.21.15.177:3300/0 conn(0x7f5480074150 0x7f548006e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.181+0000 7f5486c00640 1 --2- 172.21.15.71:0/618379328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 0x7f54800b47d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.181+0000 7f5486c00640 1 -- 172.21.15.71:0/618379328 >> 172.21.15.71:0/618379328 conn(0x7f54800b1950 msgr2=0x7f54800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:31.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.181+0000 7f5486c00640 1 -- 172.21.15.71:0/618379328 shutdown_connections 2024-09-16T07:25:31.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.181+0000 7f5486c00640 1 -- 172.21.15.71:0/618379328 wait complete. 2024-09-16T07:25:31.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.183+0000 7f5486c00640 1 Processor -- start 2024-09-16T07:25:31.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.183+0000 7f5486c00640 1 -- start start 2024-09-16T07:25:31.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5486c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5480074150 0x7f548015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:31.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5486c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54800746a0 0x7f5480157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5486c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 0x7f5480145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5486c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f548006f3b0 con 0x7f54800b43f0 2024-09-16T07:25:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5486c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f548015dd20 con 0x7f5480074150 2024-09-16T07:25:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5486c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f548006f530 con 0x7f54800746a0 2024-09-16T07:25:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5484c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 0x7f5480145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5484c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 0x7f5480145320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60354/0 (socket says 172.21.15.71:60354) 2024-09-16T07:25:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.184+0000 7f5484c00640 1 -- 172.21.15.71:0/3327137106 learned_addr learned my addr 172.21.15.71:0/3327137106 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f547f400640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54800746a0 0x7f5480157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f547fe00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5480074150 0x7f548015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f5484c00640 1 -- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54800746a0 msgr2=0x7f5480157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f5484c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54800746a0 0x7f5480157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f5484c00640 1 -- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5480074150 msgr2=0x7f548015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f5484c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5480074150 0x7f548015a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:31.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f5484c00640 1 -- 172.21.15.71:0/3327137106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5480157390 con 0x7f54800b43f0 2024-09-16T07:25:31.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f5484c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 0x7f5480145320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f547800c7d0 tx=0x7f547800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:31.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f5477e00640 1 -- 172.21.15.71:0/3327137106 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478003b70 con 0x7f54800b43f0 2024-09-16T07:25:31.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.185+0000 7f5477e00640 1 -- 172.21.15.71:0/3327137106 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5478003d30 con 0x7f54800b43f0 2024-09-16T07:25:31.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.186+0000 7f5477e00640 1 -- 172.21.15.71:0/3327137106 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547800aa50 con 0x7f54800b43f0 2024-09-16T07:25:31.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.186+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54801570b0 con 0x7f54800b43f0 2024-09-16T07:25:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.186+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5480159d70 con 0x7f54800b43f0 2024-09-16T07:25:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.187+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54800760b0 con 0x7f54800b43f0 2024-09-16T07:25:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.188+0000 7f5477e00640 1 -- 172.21.15.71:0/3327137106 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f547800abf0 con 0x7f54800b43f0 2024-09-16T07:25:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.189+0000 7f5477e00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5460078bb0 0x7f546007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.189+0000 7f5477e00640 1 -- 172.21.15.71:0/3327137106 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f547809a9f0 con 0x7f54800b43f0 2024-09-16T07:25:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.189+0000 7f547fe00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5460078bb0 0x7f546007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.190+0000 7f547fe00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5460078bb0 0x7f546007b070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5470033000 tx=0x7f5470004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:31.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.193+0000 7f5477e00640 1 -- 172.21.15.71:0/3327137106 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f54800760b0 con 0x7f54800b43f0 2024-09-16T07:25:31.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:31.447+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f54800cdd10 con 0x7f54800b43f0 2024-09-16T07:25:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.414+0000 7f5477e00640 1 -- 172.21.15.71:0/3327137106 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v147) ==== 184+0+0 (secure 0 0 0) 0x7f54800cdd10 con 0x7f54800b43f0 2024-09-16T07:25:32.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.417+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5460078bb0 msgr2=0x7f546007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.417+0000 7f5486c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5460078bb0 0x7f546007b070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5470033000 tx=0x7f5470004730 comp rx=0 tx=0).stop 2024-09-16T07:25:32.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.417+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 msgr2=0x7f5480145320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.417+0000 7f5486c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 0x7f5480145320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f547800c7d0 tx=0x7f547800cc90 comp rx=0 tx=0).stop 2024-09-16T07:25:32.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.418+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 shutdown_connections 2024-09-16T07:25:32.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.418+0000 7f5486c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5460078bb0 0x7f546007b070 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.418+0000 7f5486c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54800b43f0 0x7f5480145320 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.418+0000 7f5486c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54800746a0 0x7f5480157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.418+0000 7f5486c00640 1 --2- 172.21.15.71:0/3327137106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5480074150 0x7f548015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.418+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 >> 172.21.15.71:0/3327137106 conn(0x7f54800b1950 msgr2=0x7f54800cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:32.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.418+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 shutdown_connections 2024-09-16T07:25:32.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.418+0000 7f5486c00640 1 -- 172.21.15.71:0/3327137106 wait complete. 2024-09-16T07:25:32.437 INFO:tasks.cephfs_test_runner:Initiate cloning where clone subvolume and source subvolume are located ... ok 2024-09-16T07:25:32.437 INFO:tasks.cephfs_test_runner:Starting test: test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-16T07:25:32.437 INFO:tasks.cephfs_test_runner:test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:25:32.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:25:32.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.599+0000 7fa906200640 1 -- 172.21.15.71:0/832129472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 msgr2=0x7fa9000b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.599+0000 7fa906200640 1 --2- 172.21.15.71:0/832129472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 0x7fa9000b2440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009830 tx=0x7fa8f002a480 comp rx=0 tx=0).stop 2024-09-16T07:25:32.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.599+0000 7fa906200640 1 -- 172.21.15.71:0/832129472 shutdown_connections 2024-09-16T07:25:32.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.599+0000 7fa906200640 1 --2- 172.21.15.71:0/832129472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 0x7fa9000b2440 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.599+0000 7fa906200640 1 -- 172.21.15.71:0/832129472 >> 172.21.15.71:0/832129472 conn(0x7fa9000b1950 msgr2=0x7fa9000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:32.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.599+0000 7fa906200640 1 -- 172.21.15.71:0/832129472 shutdown_connections 2024-09-16T07:25:32.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.599+0000 7fa906200640 1 -- 172.21.15.71:0/832129472 wait complete. 2024-09-16T07:25:32.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.601+0000 7fa906200640 1 Processor -- start 2024-09-16T07:25:32.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.601+0000 7fa906200640 1 -- start start 2024-09-16T07:25:32.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa906200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa900073fb0 0x7fa9000c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:32.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa906200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9000ccf00 0x7fa900146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa906200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 0x7fa900146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa906200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa90006f9f0 con 0x7fa90015f5a0 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa906200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa90006f870 con 0x7fa9000ccf00 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa906200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa900074420 con 0x7fa900073fb0 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 0x7fa900146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 0x7fa900146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60374/0 (socket says 172.21.15.71:60374) 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ffe00640 1 -- 172.21.15.71:0/2716375584 learned_addr learned my addr 172.21.15.71:0/2716375584 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ff400640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa900073fb0 0x7fa9000c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8fea00640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9000ccf00 0x7fa900146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ffe00640 1 -- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa900073fb0 msgr2=0x7fa9000c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ffe00640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa900073fb0 0x7fa9000c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ffe00640 1 -- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9000ccf00 msgr2=0x7fa900146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ffe00640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9000ccf00 0x7fa900146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.602+0000 7fa8ffe00640 1 -- 172.21.15.71:0/2716375584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9000c9890 con 0x7fa90015f5a0 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.603+0000 7fa8ffe00640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 0x7fa900146770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800cbc0 tx=0x7fa8f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.603+0000 7fa8f7400640 1 -- 172.21.15.71:0/2716375584 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8016020 con 0x7fa90015f5a0 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.603+0000 7fa8f7400640 1 -- 172.21.15.71:0/2716375584 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8f800ae30 con 0x7fa90015f5a0 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.603+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9000cacf0 con 0x7fa90015f5a0 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.603+0000 7fa8f7400640 1 -- 172.21.15.71:0/2716375584 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8012660 con 0x7fa90015f5a0 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.603+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9000caf90 con 0x7fa90015f5a0 2024-09-16T07:25:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.604+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9000b2440 con 0x7fa90015f5a0 2024-09-16T07:25:32.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.605+0000 7fa8f7400640 1 -- 172.21.15.71:0/2716375584 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fa8f8019020 con 0x7fa90015f5a0 2024-09-16T07:25:32.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.606+0000 7fa8f7400640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e0078bb0 0x7fa8e007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:32.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.606+0000 7fa8f7400640 1 -- 172.21.15.71:0/2716375584 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7fa8f809b070 con 0x7fa90015f5a0 2024-09-16T07:25:32.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.606+0000 7fa8ff400640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e0078bb0 0x7fa8e007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:32.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.607+0000 7fa8ff400640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e0078bb0 0x7fa8e007b070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0034000 tx=0x7fa8f0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:32.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.610+0000 7fa8f7400640 1 -- 172.21.15.71:0/2716375584 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8f8020030 con 0x7fa90015f5a0 2024-09-16T07:25:32.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.794+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9000b1ed0 con 0x7fa90015f5a0 2024-09-16T07:25:32.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.795+0000 7fa8f7400640 1 -- 172.21.15.71:0/2716375584 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+13046 (secure 0 0 0) 0x7fa9000c9890 con 0x7fa90015f5a0 2024-09-16T07:25:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:32.795 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":40,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:25:30.401167+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6837/1652086843":"2024-09-17T07:25:27.388783+0000","172.21.15.177:6834/1652086843":"2024-09-17T07:25:27.388783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:25:32.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e0078bb0 msgr2=0x7fa8e007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e0078bb0 0x7fa8e007b070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0034000 tx=0x7fa8f0004730 comp rx=0 tx=0).stop 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 msgr2=0x7fa900146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 0x7fa900146770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800cbc0 tx=0x7fa8f800a430 comp rx=0 tx=0).stop 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 shutdown_connections 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e0078bb0 0x7fa8e007b070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa90015f5a0 0x7fa900146770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9000ccf00 0x7fa900146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 --2- 172.21.15.71:0/2716375584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa900073fb0 0x7fa9000c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 >> 172.21.15.71:0/2716375584 conn(0x7fa9000b1950 msgr2=0x7fa9000af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 shutdown_connections 2024-09-16T07:25:32.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.798+0000 7fa906200640 1 -- 172.21.15.71:0/2716375584 wait complete. 2024-09-16T07:25:32.813 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:25:32.813 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:25:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.975+0000 7fc7a6800640 1 -- 172.21.15.71:0/3110395849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 msgr2=0x7fc78400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.975+0000 7fc7a6800640 1 --2- 172.21.15.71:0/3110395849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 0x7fc78400b9c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc798001dc0 tx=0x7fc784005830 comp rx=0 tx=0).stop 2024-09-16T07:25:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.975+0000 7fc7a6800640 1 -- 172.21.15.71:0/3110395849 shutdown_connections 2024-09-16T07:25:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.975+0000 7fc7a6800640 1 --2- 172.21.15.71:0/3110395849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 0x7fc78400b9c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.975+0000 7fc7a6800640 1 -- 172.21.15.71:0/3110395849 >> 172.21.15.71:0/3110395849 conn(0x7fc7a00b1840 msgr2=0x7fc7a00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.975+0000 7fc7a6800640 1 -- 172.21.15.71:0/3110395849 shutdown_connections 2024-09-16T07:25:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.975+0000 7fc7a6800640 1 -- 172.21.15.71:0/3110395849 wait complete. 2024-09-16T07:25:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.977+0000 7fc7a6800640 1 Processor -- start 2024-09-16T07:25:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.977+0000 7fc7a6800640 1 -- start start 2024-09-16T07:25:32.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc7a6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 0x7fc7a01f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:32.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc7a6800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a01f4c60 0x7fc7a01f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc7a6800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a00b0ae0 0x7fc7a00b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc7a6800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a00ce940 con 0x7fc7840052e0 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc7a6800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a00ce7c0 con 0x7fc7a00b0ae0 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc7a6800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a00cf070 con 0x7fc7a01f4c60 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 0x7fc7a01f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 0x7fc7a01f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60396/0 (socket says 172.21.15.71:60396) 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79f400640 1 -- 172.21.15.71:0/748401636 learned_addr learned my addr 172.21.15.71:0/748401636 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79ea00640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a01f4c60 0x7fc7a01f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79fe00640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a00b0ae0 0x7fc7a00b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79f400640 1 -- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a01f4c60 msgr2=0x7fc7a01f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79f400640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a01f4c60 0x7fc7a01f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79f400640 1 -- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a00b0ae0 msgr2=0x7fc7a00b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79f400640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a00b0ae0 0x7fc7a00b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.978+0000 7fc79f400640 1 -- 172.21.15.71:0/748401636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a01f91a0 con 0x7fc7840052e0 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.979+0000 7fc79f400640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 0x7fc7a01f7b50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc790006680 tx=0x7fc7900066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.979+0000 7fc797400640 1 -- 172.21.15.71:0/748401636 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790015070 con 0x7fc7840052e0 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.979+0000 7fc797400640 1 -- 172.21.15.71:0/748401636 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc790011860 con 0x7fc7840052e0 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.979+0000 7fc797400640 1 -- 172.21.15.71:0/748401636 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790011cb0 con 0x7fc7840052e0 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.979+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a01f5120 con 0x7fc7840052e0 2024-09-16T07:25:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.979+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a01f56e0 con 0x7fc7840052e0 2024-09-16T07:25:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.980+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a00b2b40 con 0x7fc7840052e0 2024-09-16T07:25:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.981+0000 7fc797400640 1 -- 172.21.15.71:0/748401636 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fc790018070 con 0x7fc7840052e0 2024-09-16T07:25:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.982+0000 7fc797400640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078bb0 0x7fc78007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.982+0000 7fc797400640 1 -- 172.21.15.71:0/748401636 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7fc79009b920 con 0x7fc7840052e0 2024-09-16T07:25:32.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.983+0000 7fc79ea00640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078bb0 0x7fc78007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:32.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.983+0000 7fc79ea00640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078bb0 0x7fc78007b070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0155f40 tx=0x7fc78c00b260 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:32.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:32.986+0000 7fc797400640 1 -- 172.21.15.71:0/748401636 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc7a01f91a0 con 0x7fc7840052e0 2024-09-16T07:25:33.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.172+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7a0072d40 con 0x7fc7840052e0 2024-09-16T07:25:33.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.173+0000 7fc797400640 1 -- 172.21.15.71:0/748401636 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7fc790063c30 con 0x7fc7840052e0 2024-09-16T07:25:33.172 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:25:33.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078bb0 msgr2=0x7fc78007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:33.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078bb0 0x7fc78007b070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0155f40 tx=0x7fc78c00b260 comp rx=0 tx=0).stop 2024-09-16T07:25:33.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 msgr2=0x7fc7a01f7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:33.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 0x7fc7a01f7b50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc790006680 tx=0x7fc7900066b0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 shutdown_connections 2024-09-16T07:25:33.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc780078bb0 0x7fc78007b070 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a00b0ae0 0x7fc7a00b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a01f4c60 0x7fc7a01f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 --2- 172.21.15.71:0/748401636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7840052e0 0x7fc7a01f7b50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.175+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 >> 172.21.15.71:0/748401636 conn(0x7fc7a00b1840 msgr2=0x7fc7a00cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:33.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.176+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 shutdown_connections 2024-09-16T07:25:33.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.176+0000 7fc7a6800640 1 -- 172.21.15.71:0/748401636 wait complete. 2024-09-16T07:25:33.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-16T07:25:33.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.347+0000 7f85bd600640 1 -- 172.21.15.71:0/1264347248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b815f5a0 msgr2=0x7f85b80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:33.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.347+0000 7f85bd600640 1 --2- 172.21.15.71:0/1264347248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b815f5a0 0x7f85b80b2440 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85b806dac0 tx=0x7f85a802a820 comp rx=0 tx=0).stop 2024-09-16T07:25:33.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.347+0000 7f85bd600640 1 -- 172.21.15.71:0/1264347248 shutdown_connections 2024-09-16T07:25:33.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.347+0000 7f85bd600640 1 --2- 172.21.15.71:0/1264347248 >> v2:172.21.15.177:3300/0 conn(0x7f85b8073fb0 0x7f85b8074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.347+0000 7f85bd600640 1 --2- 172.21.15.71:0/1264347248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b815f5a0 0x7f85b80b2440 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.347+0000 7f85bd600640 1 -- 172.21.15.71:0/1264347248 >> 172.21.15.71:0/1264347248 conn(0x7f85b80b1950 msgr2=0x7f85b80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:33.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.347+0000 7f85bd600640 1 -- 172.21.15.71:0/1264347248 shutdown_connections 2024-09-16T07:25:33.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.348+0000 7f85bd600640 1 -- 172.21.15.71:0/1264347248 wait complete. 2024-09-16T07:25:33.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.349+0000 7f85bd600640 1 Processor -- start 2024-09-16T07:25:33.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.349+0000 7f85bd600640 1 -- start start 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85bd600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85b8073fb0 0x7f85b81503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85bd600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b80ccf00 0x7f85b81f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85bd600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85b815f5a0 0x7f85b81f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85bd600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f85b806f2e0 con 0x7f85b80ccf00 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85bd600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85b806f160 con 0x7f85b815f5a0 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85bd600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f85b806f460 con 0x7f85b8073fb0 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85b6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b80ccf00 0x7f85b81f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85b6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b80ccf00 0x7f85b81f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60414/0 (socket says 172.21.15.71:60414) 2024-09-16T07:25:33.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.350+0000 7f85b6000640 1 -- 172.21.15.71:0/2390082887 learned_addr learned my addr 172.21.15.71:0/2390082887 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b7400640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85b815f5a0 0x7f85b81f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b6a00640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85b8073fb0 0x7f85b81503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b6000640 1 -- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85b8073fb0 msgr2=0x7f85b81503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b6000640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85b8073fb0 0x7f85b81503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b6000640 1 -- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85b815f5a0 msgr2=0x7f85b81f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b6000640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85b815f5a0 0x7f85b81f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b6000640 1 -- 172.21.15.71:0/2390082887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b81508e0 con 0x7f85b80ccf00 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b6a00640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85b8073fb0 0x7f85b81503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85b6000640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b80ccf00 0x7f85b81f2ed0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c530 tx=0x7f85a400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:33.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85aea00640 1 -- 172.21.15.71:0/2390082887 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4016020 con 0x7f85b80ccf00 2024-09-16T07:25:33.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85aea00640 1 -- 172.21.15.71:0/2390082887 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85a4012920 con 0x7f85b80ccf00 2024-09-16T07:25:33.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b8150bd0 con 0x7f85b80ccf00 2024-09-16T07:25:33.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.351+0000 7f85aea00640 1 -- 172.21.15.71:0/2390082887 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4012d50 con 0x7f85b80ccf00 2024-09-16T07:25:33.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.352+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b8151190 con 0x7f85b80ccf00 2024-09-16T07:25:33.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.353+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b806f9f0 con 0x7f85b80ccf00 2024-09-16T07:25:33.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.354+0000 7f85aea00640 1 -- 172.21.15.71:0/2390082887 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f85a4019020 con 0x7f85b80ccf00 2024-09-16T07:25:33.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.355+0000 7f85aea00640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8598078bb0 0x7f859807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:33.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.355+0000 7f85aea00640 1 -- 172.21.15.71:0/2390082887 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f85a409a920 con 0x7f85b80ccf00 2024-09-16T07:25:33.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.356+0000 7f85b6a00640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8598078bb0 0x7f859807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:33.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.356+0000 7f85b6a00640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8598078bb0 0x7f859807b070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85a8034000 tx=0x7f85a8004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.360+0000 7f85aea00640 1 -- 172.21.15.71:0/2390082887 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f85a4020030 con 0x7f85b80ccf00 2024-09-16T07:25:33.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:33.619+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7f85b8155180 con 0x7f85b80ccf00 2024-09-16T07:25:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.422+0000 7f85aea00640 1 -- 172.21.15.71:0/2390082887 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v149) ==== 169+0+0 (secure 0 0 0) 0x7f85a4062dc0 con 0x7f85b80ccf00 2024-09-16T07:25:34.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8598078bb0 msgr2=0x7f859807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8598078bb0 0x7f859807b070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85a8034000 tx=0x7f85a8004ab0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b80ccf00 msgr2=0x7f85b81f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b80ccf00 0x7f85b81f2ed0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c530 tx=0x7f85a400c9f0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 shutdown_connections 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8598078bb0 0x7f859807b070 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85b815f5a0 0x7f85b81f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85b80ccf00 0x7f85b81f2ed0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 --2- 172.21.15.71:0/2390082887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85b8073fb0 0x7f85b81503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 >> 172.21.15.71:0/2390082887 conn(0x7f85b80b1950 msgr2=0x7f85b8154150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 shutdown_connections 2024-09-16T07:25:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.425+0000 7f85bd600640 1 -- 172.21.15.71:0/2390082887 wait complete. 2024-09-16T07:25:34.443 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:25:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.603+0000 7fa923200640 1 -- 172.21.15.71:0/1149454346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 msgr2=0x7fa9240b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.603+0000 7fa923200640 1 --2- 172.21.15.71:0/1149454346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 0x7fa9240b28f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa924073b00 tx=0x7fa91402a540 comp rx=0 tx=0).stop 2024-09-16T07:25:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.603+0000 7fa923200640 1 -- 172.21.15.71:0/1149454346 shutdown_connections 2024-09-16T07:25:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.603+0000 7fa923200640 1 --2- 172.21.15.71:0/1149454346 >> v2:172.21.15.177:3300/0 conn(0x7fa9240725d0 0x7fa924073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.603+0000 7fa923200640 1 --2- 172.21.15.71:0/1149454346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 0x7fa9240b28f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.603+0000 7fa923200640 1 -- 172.21.15.71:0/1149454346 >> 172.21.15.71:0/1149454346 conn(0x7fa9240cfa20 msgr2=0x7fa9240cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.603+0000 7fa923200640 1 -- 172.21.15.71:0/1149454346 shutdown_connections 2024-09-16T07:25:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.603+0000 7fa923200640 1 -- 172.21.15.71:0/1149454346 wait complete. 2024-09-16T07:25:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.605+0000 7fa923200640 1 Processor -- start 2024-09-16T07:25:34.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.605+0000 7fa923200640 1 -- start start 2024-09-16T07:25:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa923200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9240d0540 0x7fa92414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa923200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 0x7fa924157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa923200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa924158080 0x7fa9240c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa923200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa924074610 con 0x7fa92415bc30 2024-09-16T07:25:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa923200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa924074490 con 0x7fa924158080 2024-09-16T07:25:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa923200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa924074790 con 0x7fa9240d0540 2024-09-16T07:25:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa921400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 0x7fa924157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa921400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 0x7fa924157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60442/0 (socket says 172.21.15.71:60442) 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa921400640 1 -- 172.21.15.71:0/4035844821 learned_addr learned my addr 172.21.15.71:0/4035844821 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.606+0000 7fa921e00640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9240d0540 0x7fa92414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa922800640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa924158080 0x7fa9240c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa921400640 1 -- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9240d0540 msgr2=0x7fa92414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa921400640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9240d0540 0x7fa92414f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa921400640 1 -- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa924158080 msgr2=0x7fa9240c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa921400640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa924158080 0x7fa9240c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa921400640 1 -- 172.21.15.71:0/4035844821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9241f7f40 con 0x7fa92415bc30 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa921e00640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9240d0540 0x7fa92414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa921400640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 0x7fa924157b40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa91000d770 tx=0x7fa91000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa91a000640 1 -- 172.21.15.71:0/4035844821 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910017070 con 0x7fa92415bc30 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa91a000640 1 -- 172.21.15.71:0/4035844821 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa910009d20 con 0x7fa92415bc30 2024-09-16T07:25:34.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa91a000640 1 -- 172.21.15.71:0/4035844821 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9100106c0 con 0x7fa92415bc30 2024-09-16T07:25:34.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9241f8220 con 0x7fa92415bc30 2024-09-16T07:25:34.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.607+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9241f87e0 con 0x7fa92415bc30 2024-09-16T07:25:34.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.608+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa92406fd20 con 0x7fa92415bc30 2024-09-16T07:25:34.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.609+0000 7fa91a000640 1 -- 172.21.15.71:0/4035844821 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fa91001a070 con 0x7fa92415bc30 2024-09-16T07:25:34.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.610+0000 7fa91a000640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa904078bb0 0x7fa90407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:34.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.610+0000 7fa91a000640 1 -- 172.21.15.71:0/4035844821 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7fa91009af90 con 0x7fa92415bc30 2024-09-16T07:25:34.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.611+0000 7fa921e00640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa904078bb0 0x7fa90407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:34.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.612+0000 7fa921e00640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa904078bb0 0x7fa90407b070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa914034000 tx=0x7fa914004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:34.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.615+0000 7fa91a000640 1 -- 172.21.15.71:0/4035844821 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa9100632a0 con 0x7fa92415bc30 2024-09-16T07:25:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.801+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa9241e6cc0 con 0x7fa92415bc30 2024-09-16T07:25:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.801+0000 7fa91a000640 1 -- 172.21.15.71:0/4035844821 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) ==== 59+0+1998 (secure 0 0 0) 0x7fa910062ba0 con 0x7fa92415bc30 2024-09-16T07:25:34.801 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:25:34.801 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:25:34.801 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:25:34.801 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:25:34.801 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:25:34.802 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:25:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:25:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:25:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:25:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:25:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:25:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:25:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:25:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:25:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa904078bb0 msgr2=0x7fa90407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa904078bb0 0x7fa90407b070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa914034000 tx=0x7fa914004730 comp rx=0 tx=0).stop 2024-09-16T07:25:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 msgr2=0x7fa924157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 0x7fa924157b40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa91000d770 tx=0x7fa91000dc30 comp rx=0 tx=0).stop 2024-09-16T07:25:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 shutdown_connections 2024-09-16T07:25:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa904078bb0 0x7fa90407b070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa924158080 0x7fa9240c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa92415bc30 0x7fa924157b40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 --2- 172.21.15.71:0/4035844821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9240d0540 0x7fa92414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 >> 172.21.15.71:0/4035844821 conn(0x7fa9240cfa20 msgr2=0x7fa924159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:34.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 shutdown_connections 2024-09-16T07:25:34.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.804+0000 7fa923200640 1 -- 172.21.15.71:0/4035844821 wait complete. 2024-09-16T07:25:34.820 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:25:34.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.980+0000 7f0be3200640 1 -- 172.21.15.71:0/2236087050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be40b3450 msgr2=0x7f0be40b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.980+0000 7f0be3200640 1 --2- 172.21.15.71:0/2236087050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be40b3450 0x7f0be40b3890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009830 tx=0x7f0bd002a410 comp rx=0 tx=0).stop 2024-09-16T07:25:34.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.980+0000 7f0be3200640 1 -- 172.21.15.71:0/2236087050 shutdown_connections 2024-09-16T07:25:34.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.980+0000 7f0be3200640 1 --2- 172.21.15.71:0/2236087050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be40cf4b0 0x7f0be40cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.980+0000 7f0be3200640 1 --2- 172.21.15.71:0/2236087050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be40b3450 0x7f0be40b3890 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.980+0000 7f0be3200640 1 -- 172.21.15.71:0/2236087050 >> 172.21.15.71:0/2236087050 conn(0x7f0be40d3d50 msgr2=0x7f0be40d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:34.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.980+0000 7f0be3200640 1 -- 172.21.15.71:0/2236087050 shutdown_connections 2024-09-16T07:25:34.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.980+0000 7f0be3200640 1 -- 172.21.15.71:0/2236087050 wait complete. 2024-09-16T07:25:34.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.982+0000 7f0be3200640 1 Processor -- start 2024-09-16T07:25:34.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.982+0000 7f0be3200640 1 -- start start 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be40b3450 0x7f0be40b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be3200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be40cf4b0 0x7f0be40b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be3200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be415df60 0x7f0be41f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0be40d24b0 con 0x7f0be40b3450 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be3200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0be40d2330 con 0x7f0be40cf4b0 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be3200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0be40d2630 con 0x7f0be415df60 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be40b3450 0x7f0be40b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be40b3450 0x7f0be40b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44082/0 (socket says 172.21.15.71:44082) 2024-09-16T07:25:34.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be1e00640 1 -- 172.21.15.71:0/1381530771 learned_addr learned my addr 172.21.15.71:0/1381530771 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be1400640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be40cf4b0 0x7f0be40b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.983+0000 7f0be2800640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be415df60 0x7f0be41f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be1e00640 1 -- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be415df60 msgr2=0x7f0be41f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be1e00640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be415df60 0x7f0be41f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be1e00640 1 -- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be40cf4b0 msgr2=0x7f0be40b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be1e00640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be40cf4b0 0x7f0be40b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be1e00640 1 -- 172.21.15.71:0/1381530771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be40c7e70 con 0x7f0be40b3450 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be2800640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be415df60 0x7f0be41f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:34.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be1e00640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be40b3450 0x7f0be40b1950 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400cbc0 tx=0x7f0bd400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:34.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0bda000640 1 -- 172.21.15.71:0/1381530771 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4016020 con 0x7f0be40b3450 2024-09-16T07:25:34.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0bda000640 1 -- 172.21.15.71:0/1381530771 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bd400ae50 con 0x7f0be40b3450 2024-09-16T07:25:34.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0bda000640 1 -- 172.21.15.71:0/1381530771 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4012660 con 0x7f0be40b3450 2024-09-16T07:25:34.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be40c8150 con 0x7f0be40b3450 2024-09-16T07:25:34.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.984+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be40c8710 con 0x7f0be40b3450 2024-09-16T07:25:34.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.985+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be40c78e0 con 0x7f0be40b3450 2024-09-16T07:25:34.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.986+0000 7f0bda000640 1 -- 172.21.15.71:0/1381530771 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f0bd4019020 con 0x7f0be40b3450 2024-09-16T07:25:34.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.988+0000 7f0bda000640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc4078bb0 0x7f0bc407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:34.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.988+0000 7f0bda000640 1 -- 172.21.15.71:0/1381530771 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f0bd409b2d0 con 0x7f0be40b3450 2024-09-16T07:25:34.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.988+0000 7f0be1400640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc4078bb0 0x7f0bc407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:34.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.989+0000 7f0be1400640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc4078bb0 0x7f0bc407b070 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0034000 tx=0x7f0bd0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:34.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:34.992+0000 7f0bda000640 1 -- 172.21.15.71:0/1381530771 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0bd4020020 con 0x7f0be40b3450 2024-09-16T07:25:35.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.174+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0be40d2b50 con 0x7f0be40b3450 2024-09-16T07:25:35.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.178+0000 7f0bda000640 1 -- 172.21.15.71:0/1381530771 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f0bd4063770 con 0x7f0be40b3450 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc4078bb0 msgr2=0x7f0bc407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc4078bb0 0x7f0bc407b070 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0034000 tx=0x7f0bd0004730 comp rx=0 tx=0).stop 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be40b3450 msgr2=0x7f0be40b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be40b3450 0x7f0be40b1950 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400cbc0 tx=0x7f0bd400a430 comp rx=0 tx=0).stop 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 shutdown_connections 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc4078bb0 0x7f0bc407b070 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be415df60 0x7f0be41f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be40cf4b0 0x7f0be40b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 --2- 172.21.15.71:0/1381530771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be40b3450 0x7f0be40b1950 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 >> 172.21.15.71:0/1381530771 conn(0x7f0be40d3d50 msgr2=0x7f0be406e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:35.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 shutdown_connections 2024-09-16T07:25:35.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.180+0000 7f0be3200640 1 -- 172.21.15.71:0/1381530771 wait complete. 2024-09-16T07:25:35.195 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:25:35.195 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:25:35.274 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:25:35.368 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:25:35.368 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:35.368 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:35.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:25:35.404 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:25:35.404 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:35.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:25:35.442 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:35.442 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:35.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:25:35.480 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:25:35.480 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:35.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:25:35.518 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:25:35.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:25:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.678+0000 7f7762000640 1 -- 172.21.15.71:0/2212446005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f775c073fb0 msgr2=0x7f775c074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.678+0000 7f7762000640 1 --2- 172.21.15.71:0/2212446005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f775c073fb0 0x7f775c074440 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7754005fd0 tx=0x7f775402aa00 comp rx=0 tx=0).stop 2024-09-16T07:25:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.678+0000 7f7762000640 1 -- 172.21.15.71:0/2212446005 shutdown_connections 2024-09-16T07:25:35.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.678+0000 7f7762000640 1 --2- 172.21.15.71:0/2212446005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f775c073fb0 0x7f775c074440 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.678+0000 7f7762000640 1 --2- 172.21.15.71:0/2212446005 >> v2:172.21.15.177:3300/0 conn(0x7f775c15f5a0 0x7f775c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.678+0000 7f7762000640 1 -- 172.21.15.71:0/2212446005 >> 172.21.15.71:0/2212446005 conn(0x7f775c0b1950 msgr2=0x7f775c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:35.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.679+0000 7f7762000640 1 -- 172.21.15.71:0/2212446005 shutdown_connections 2024-09-16T07:25:35.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.679+0000 7f7762000640 1 -- 172.21.15.71:0/2212446005 wait complete. 2024-09-16T07:25:35.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.680+0000 7f7762000640 1 Processor -- start 2024-09-16T07:25:35.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.680+0000 7f7762000640 1 -- start start 2024-09-16T07:25:35.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.681+0000 7f7762000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f775c073fb0 0x7f775c149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:35.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.681+0000 7f7762000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f775c0ccf00 0x7f775c09d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:35.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.681+0000 7f7762000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f775c15f5a0 0x7f775c152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:35.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.681+0000 7f7762000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f775c06f2e0 con 0x7f775c073fb0 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.681+0000 7f7762000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f775c06f160 con 0x7f775c0ccf00 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.681+0000 7f7762000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f775c06f460 con 0x7f775c15f5a0 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.681+0000 7f775b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f775c073fb0 0x7f775c149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f775c073fb0 0x7f775c149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44088/0 (socket says 172.21.15.71:44088) 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775b400640 1 -- 172.21.15.71:0/2438425921 learned_addr learned my addr 172.21.15.71:0/2438425921 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775aa00640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f775c0ccf00 0x7f775c09d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775be00640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f775c15f5a0 0x7f775c152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775b400640 1 -- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f775c15f5a0 msgr2=0x7f775c152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775b400640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f775c15f5a0 0x7f775c152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775b400640 1 -- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f775c0ccf00 msgr2=0x7f775c09d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775b400640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f775c0ccf00 0x7f775c09d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775b400640 1 -- 172.21.15.71:0/2438425921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c1516a0 con 0x7f775c073fb0 2024-09-16T07:25:35.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.682+0000 7f775b400640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f775c073fb0 0x7f775c149940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f774c00e9f0 tx=0x7f774c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:35.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.683+0000 7f7753400640 1 -- 172.21.15.71:0/2438425921 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c00cd80 con 0x7f775c073fb0 2024-09-16T07:25:35.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.683+0000 7f7753400640 1 -- 172.21.15.71:0/2438425921 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f774c004020 con 0x7f775c073fb0 2024-09-16T07:25:35.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.683+0000 7f7753400640 1 -- 172.21.15.71:0/2438425921 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c004470 con 0x7f775c073fb0 2024-09-16T07:25:35.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.683+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c151980 con 0x7f775c073fb0 2024-09-16T07:25:35.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.683+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c151f40 con 0x7f775c073fb0 2024-09-16T07:25:35.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.684+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775c06f980 con 0x7f775c073fb0 2024-09-16T07:25:35.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.685+0000 7f7753400640 1 -- 172.21.15.71:0/2438425921 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f774c0035f0 con 0x7f775c073fb0 2024-09-16T07:25:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.686+0000 7f7753400640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f773c078bb0 0x7f773c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.686+0000 7f7753400640 1 -- 172.21.15.71:0/2438425921 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f774c09b6c0 con 0x7f775c073fb0 2024-09-16T07:25:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.686+0000 7f775aa00640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f773c078bb0 0x7f773c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:35.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.687+0000 7f775aa00640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f773c078bb0 0x7f773c07b070 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7748007fd0 tx=0x7f7748009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:35.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.690+0000 7f7753400640 1 -- 172.21.15.71:0/2438425921 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f774c063450 con 0x7f775c073fb0 2024-09-16T07:25:35.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.911+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f775c0ad1d0 con 0x7f775c073fb0 2024-09-16T07:25:35.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.912+0000 7f7753400640 1 -- 172.21.15.71:0/2438425921 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3917 (secure 0 0 0) 0x7f774c067e80 con 0x7f775c073fb0 2024-09-16T07:25:35.912 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:35.912 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"btime":"2024-09-16T07:25:28:391778+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4834,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3085563418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3085563418},{"type":"v1","addr":"172.21.15.71:6834","nonce":3085563418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3025435692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3025435692},{"type":"v1","addr":"172.21.15.71:6835","nonce":3025435692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3253921551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":3253921551},{"type":"v1","addr":"172.21.15.177:6836","nonce":3253921551}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5179,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2166411535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2166411535},{"type":"v1","addr":"172.21.15.177:6837","nonce":2166411535}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-16T07:25:35.912 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f773c078bb0 msgr2=0x7f773c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f773c078bb0 0x7f773c07b070 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7748007fd0 tx=0x7f7748009040 comp rx=0 tx=0).stop 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f775c073fb0 msgr2=0x7f775c149940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f775c073fb0 0x7f775c149940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f774c00e9f0 tx=0x7f774c00eeb0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 shutdown_connections 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f773c078bb0 0x7f773c07b070 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f775c15f5a0 0x7f775c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f775c0ccf00 0x7f775c09d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 --2- 172.21.15.71:0/2438425921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f775c073fb0 0x7f775c149940 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 >> 172.21.15.71:0/2438425921 conn(0x7f775c0b1950 msgr2=0x7f775c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:35.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 shutdown_connections 2024-09-16T07:25:35.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:35.914+0000 7f7762000640 1 -- 172.21.15.71:0/2438425921 wait complete. 2024-09-16T07:25:35.931 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:25:35.931 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:25:35.932 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:25:35.932 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:25:35.932 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:25:35.932 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:25:35.933 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:25:35.933 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:25:35.933 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:25:35.933 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:25:35.934 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:25:35.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:25:35.934 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:25:35.934 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:25:35.935 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:25:35.935 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:25:35.947 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:25:35.948 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:25:35.949 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:25:35.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:25:35.953 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:25:35.954 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:25:35.954 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:25:35.955 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:25:35.991 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:25:35.992 DEBUG:teuthology.parallel:result is None 2024-09-16T07:25:35.994 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:25:35.994 DEBUG:teuthology.parallel:result is None 2024-09-16T07:25:35.998 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:25:35.998 DEBUG:teuthology.parallel:result is None 2024-09-16T07:25:36.000 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:25:36.000 DEBUG:teuthology.parallel:result is None 2024-09-16T07:25:36.000 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:25:36.122 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:25:36.122 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:25:36.124 INFO:tasks.ceph.mds.c.smithi071.stderr:2024-09-16T07:25:36.124+0000 7fe13ee00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-16T07:25:36.130 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:25:36.130 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:25:36.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.165+0000 7fb3c7200640 1 -- 172.21.15.71:0/2103012996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0073fb0 msgr2=0x7fb3c0074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.165+0000 7fb3c7200640 1 --2- 172.21.15.71:0/2103012996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0073fb0 0x7fb3c0074440 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8005fd0 tx=0x7fb3b802a820 comp rx=0 tx=0).stop 2024-09-16T07:25:36.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.165+0000 7fb3c7200640 1 -- 172.21.15.71:0/2103012996 shutdown_connections 2024-09-16T07:25:36.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.165+0000 7fb3c7200640 1 --2- 172.21.15.71:0/2103012996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0073fb0 0x7fb3c0074440 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.165+0000 7fb3c7200640 1 --2- 172.21.15.71:0/2103012996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3c00ccf00 0x7fb3c00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.165+0000 7fb3c7200640 1 -- 172.21.15.71:0/2103012996 >> 172.21.15.71:0/2103012996 conn(0x7fb3c00b1950 msgr2=0x7fb3c00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:36.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.165+0000 7fb3c7200640 1 -- 172.21.15.71:0/2103012996 shutdown_connections 2024-09-16T07:25:36.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.165+0000 7fb3c7200640 1 -- 172.21.15.71:0/2103012996 wait complete. 2024-09-16T07:25:36.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.167+0000 7fb3c7200640 1 Processor -- start 2024-09-16T07:25:36.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.167+0000 7fb3c7200640 1 -- start start 2024-09-16T07:25:36.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c7200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3c0073fb0 0x7fb3c0158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:36.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c7200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3c00ccf00 0x7fb3c015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:36.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c015f5a0 0x7fb3c014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:36.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c006f2e0 con 0x7fb3c015f5a0 2024-09-16T07:25:36.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c7200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c006f160 con 0x7fb3c00ccf00 2024-09-16T07:25:36.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c7200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c006f460 con 0x7fb3c0073fb0 2024-09-16T07:25:36.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c015f5a0 0x7fb3c014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:36.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c015f5a0 0x7fb3c014c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44142/0 (socket says 172.21.15.71:44142) 2024-09-16T07:25:36.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3c5200640 1 -- 172.21.15.71:0/1861586805 learned_addr learned my addr 172.21.15.71:0/1861586805 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:36.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3bf400640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3c00ccf00 0x7fb3c015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:36.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.168+0000 7fb3bfe00640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3c0073fb0 0x7fb3c0158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3c5200640 1 -- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3c0073fb0 msgr2=0x7fb3c0158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3c5200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3c0073fb0 0x7fb3c0158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3c5200640 1 -- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3c00ccf00 msgr2=0x7fb3c015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3c5200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3c00ccf00 0x7fb3c015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3c5200640 1 -- 172.21.15.71:0/1861586805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c00cdae0 con 0x7fb3c015f5a0 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3c5200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c015f5a0 0x7fb3c014c220 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009cd0 tx=0x7fb3b8032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3b7e00640 1 -- 172.21.15.71:0/1861586805 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8037040 con 0x7fb3c015f5a0 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3b7e00640 1 -- 172.21.15.71:0/1861586805 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3b8042070 con 0x7fb3c015f5a0 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c00cddc0 con 0x7fb3c015f5a0 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.169+0000 7fb3b7e00640 1 -- 172.21.15.71:0/1861586805 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8003e60 con 0x7fb3c015f5a0 2024-09-16T07:25:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.170+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c00ce660 con 0x7fb3c015f5a0 2024-09-16T07:25:36.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.171+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c006f9f0 con 0x7fb3c015f5a0 2024-09-16T07:25:36.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.172+0000 7fb3b7e00640 1 -- 172.21.15.71:0/1861586805 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fb3b8004280 con 0x7fb3c015f5a0 2024-09-16T07:25:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.172+0000 7fb3b7e00640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3a0078bb0 0x7fb3a007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.173+0000 7fb3b7e00640 1 -- 172.21.15.71:0/1861586805 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7fb3b80bb720 con 0x7fb3c015f5a0 2024-09-16T07:25:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.173+0000 7fb3bfe00640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3a0078bb0 0x7fb3a007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.174+0000 7fb3bfe00640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3a0078bb0 0x7fb3a007b070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0007fd0 tx=0x7fb3b0001370 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:36.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.177+0000 7fb3b7e00640 1 -- 172.21.15.71:0/1861586805 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb3b80834e0 con 0x7fb3c015f5a0 2024-09-16T07:25:36.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.374+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb3c0154da0 con 0x7fb3c015f5a0 2024-09-16T07:25:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.440+0000 7fb3b7e00640 1 -- 172.21.15.71:0/1861586805 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) ==== 97+0+0 (secure 0 0 0) 0x7fb3b8087f10 con 0x7fb3c015f5a0 2024-09-16T07:25:36.439 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:25:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3a0078bb0 msgr2=0x7fb3a007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3a0078bb0 0x7fb3a007b070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0007fd0 tx=0x7fb3b0001370 comp rx=0 tx=0).stop 2024-09-16T07:25:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c015f5a0 msgr2=0x7fb3c014c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c015f5a0 0x7fb3c014c220 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009cd0 tx=0x7fb3b8032c60 comp rx=0 tx=0).stop 2024-09-16T07:25:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 shutdown_connections 2024-09-16T07:25:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb3a0078bb0 0x7fb3a007b070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c015f5a0 0x7fb3c014c220 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3c00ccf00 0x7fb3c015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 --2- 172.21.15.71:0/1861586805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3c0073fb0 0x7fb3c0158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.442+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 >> 172.21.15.71:0/1861586805 conn(0x7fb3c00b1950 msgr2=0x7fb3c00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.443+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 shutdown_connections 2024-09-16T07:25:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.443+0000 7fb3c7200640 1 -- 172.21.15.71:0/1861586805 wait complete. 2024-09-16T07:25:36.460 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:25:36.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.619+0000 7fb811400640 1 -- 172.21.15.71:0/3761194910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0b28d0 msgr2=0x7fb80c162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.619+0000 7fb811400640 1 --2- 172.21.15.71:0/3761194910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0b28d0 0x7fb80c162730 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb804005fd0 tx=0x7fb80402a8a0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.619+0000 7fb811400640 1 -- 172.21.15.71:0/3761194910 shutdown_connections 2024-09-16T07:25:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.619+0000 7fb811400640 1 --2- 172.21.15.71:0/3761194910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0b28d0 0x7fb80c162730 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.619+0000 7fb811400640 1 --2- 172.21.15.71:0/3761194910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80c0d1ec0 0x7fb80c0d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.619+0000 7fb811400640 1 -- 172.21.15.71:0/3761194910 >> 172.21.15.71:0/3761194910 conn(0x7fb80c0d3de0 msgr2=0x7fb80c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.619+0000 7fb811400640 1 -- 172.21.15.71:0/3761194910 shutdown_connections 2024-09-16T07:25:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.619+0000 7fb811400640 1 -- 172.21.15.71:0/3761194910 wait complete. 2024-09-16T07:25:36.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.620+0000 7fb811400640 1 Processor -- start 2024-09-16T07:25:36.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.621+0000 7fb811400640 1 -- start start 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.621+0000 7fb811400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80c0b28d0 0x7fb80c1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.621+0000 7fb811400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb80c0b4d10 0x7fb80c0b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.621+0000 7fb811400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0d1ec0 0x7fb80c161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb811400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c0b3a80 con 0x7fb80c0b4d10 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb811400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c0b3900 con 0x7fb80c0d1ec0 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb811400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb80c0b3c00 con 0x7fb80c0b28d0 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb803e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb80c0b4d10 0x7fb80c0b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb803e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb80c0b4d10 0x7fb80c0b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44144/0 (socket says 172.21.15.71:44144) 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb803e00640 1 -- 172.21.15.71:0/1164050034 learned_addr learned my addr 172.21.15.71:0/1164050034 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb80be00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80c0b28d0 0x7fb80c1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb810a00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0d1ec0 0x7fb80c161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb803e00640 1 -- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80c0b28d0 msgr2=0x7fb80c1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb803e00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80c0b28d0 0x7fb80c1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb803e00640 1 -- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0d1ec0 msgr2=0x7fb80c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb803e00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0d1ec0 0x7fb80c161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb803e00640 1 -- 172.21.15.71:0/1164050034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c0b8df0 con 0x7fb80c0b4d10 2024-09-16T07:25:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb80be00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80c0b28d0 0x7fb80c1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.622+0000 7fb810a00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0d1ec0 0x7fb80c161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.623+0000 7fb803e00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb80c0b4d10 0x7fb80c0b8410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c490 tx=0x7fb7f400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.623+0000 7fb808c00640 1 -- 172.21.15.71:0/1164050034 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4016020 con 0x7fb80c0b4d10 2024-09-16T07:25:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.623+0000 7fb808c00640 1 -- 172.21.15.71:0/1164050034 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7f4012940 con 0x7fb80c0b4d10 2024-09-16T07:25:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.623+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c0b90d0 con 0x7fb80c0b4d10 2024-09-16T07:25:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.623+0000 7fb808c00640 1 -- 172.21.15.71:0/1164050034 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4012d90 con 0x7fb80c0b4d10 2024-09-16T07:25:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.623+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c15ec90 con 0x7fb80c0b4d10 2024-09-16T07:25:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.624+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80c15eab0 con 0x7fb80c0b4d10 2024-09-16T07:25:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.625+0000 7fb808c00640 1 -- 172.21.15.71:0/1164050034 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fb7f4019020 con 0x7fb80c0b4d10 2024-09-16T07:25:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.626+0000 7fb808c00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb7e8078be0 0x7fb7e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.626+0000 7fb808c00640 1 -- 172.21.15.71:0/1164050034 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7fb7f409a890 con 0x7fb80c0b4d10 2024-09-16T07:25:36.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.627+0000 7fb80be00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb7e8078be0 0x7fb7e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:36.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.627+0000 7fb80be00640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb7e8078be0 0x7fb7e807b0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009b80 tx=0x7fb7fc005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.631+0000 7fb808c00640 1 -- 172.21.15.71:0/1164050034 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb80c15eab0 con 0x7fb80c0b4d10 2024-09-16T07:25:36.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.883+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb80c08d470 con 0x7fb80c0b4d10 2024-09-16T07:25:36.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.884+0000 7fb808c00640 1 -- 172.21.15.71:0/1164050034 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7fb80c08d470 con 0x7fb80c0b4d10 2024-09-16T07:25:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:36.884 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:25:36.884 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:25:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:25:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:25:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.885 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:25:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:25:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:25:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:25:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:25:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:25:36.887 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:25:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:25:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:25:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:25:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:25:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:25:36.889 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:25:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.890 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:25:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:25:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:25:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:25:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:25:36.891 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:25:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:25:36.892 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:25:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:25:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:25:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:25:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:25:36.894 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:25:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:25:36.895 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:25:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:25:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:25:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.896 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:25:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:25:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:25:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:25:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:25:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:25:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:25:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:25:36.899 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:25:36.900 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:25:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:25:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:25:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:25:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:25:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.902 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:25:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:25:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:25:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:25:36.903 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:25:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:25:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:25:36.904 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:25:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:25:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2024-09-16T07:25:36.905 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T07:25:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:25:36.909 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:25:36.910 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:36.912 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:25:36.913 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:25:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb7e8078be0 msgr2=0x7fb7e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb7e8078be0 0x7fb7e807b0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009b80 tx=0x7fb7fc005f90 comp rx=0 tx=0).stop 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb80c0b4d10 msgr2=0x7fb80c0b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb80c0b4d10 0x7fb80c0b8410 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c490 tx=0x7fb7f400c950 comp rx=0 tx=0).stop 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 shutdown_connections 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb7e8078be0 0x7fb7e807b0a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c0d1ec0 0x7fb80c161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb80c0b4d10 0x7fb80c0b8410 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 --2- 172.21.15.71:0/1164050034 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80c0b28d0 0x7fb80c1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 >> 172.21.15.71:0/1164050034 conn(0x7fb80c0d3de0 msgr2=0x7fb80c06e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 shutdown_connections 2024-09-16T07:25:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:36.886+0000 7fb811400640 1 -- 172.21.15.71:0/1164050034 wait complete. 2024-09-16T07:25:36.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:25:37.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.072+0000 7f5293600640 1 -- 172.21.15.71:0/4217190376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 msgr2=0x7f528c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:37.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.072+0000 7f5293600640 1 --2- 172.21.15.71:0/4217190376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 0x7f528c06f490 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f528c06d9e0 tx=0x7f527802a580 comp rx=0 tx=0).stop 2024-09-16T07:25:37.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.072+0000 7f5293600640 1 -- 172.21.15.71:0/4217190376 shutdown_connections 2024-09-16T07:25:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.072+0000 7f5293600640 1 --2- 172.21.15.71:0/4217190376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 0x7f528c06f490 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.072+0000 7f5293600640 1 --2- 172.21.15.71:0/4217190376 >> v2:172.21.15.177:3300/0 conn(0x7f528c161550 0x7f528c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.072+0000 7f5293600640 1 -- 172.21.15.71:0/4217190376 >> 172.21.15.71:0/4217190376 conn(0x7f528c0b1950 msgr2=0x7f528c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.072+0000 7f5293600640 1 -- 172.21.15.71:0/4217190376 shutdown_connections 2024-09-16T07:25:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.072+0000 7f5293600640 1 -- 172.21.15.71:0/4217190376 wait complete. 2024-09-16T07:25:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.074+0000 7f5293600640 1 Processor -- start 2024-09-16T07:25:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.074+0000 7f5293600640 1 -- start start 2024-09-16T07:25:37.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f5293600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f528c06f9d0 0x7f528c1f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f5293600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 0x7f528c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f5293600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f528c161550 0x7f528c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f5293600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f528c073f80 con 0x7f528c0af3d0 2024-09-16T07:25:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f5293600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f528c073e00 con 0x7f528c161550 2024-09-16T07:25:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f5293600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f528c074100 con 0x7f528c06f9d0 2024-09-16T07:25:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f528be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 0x7f528c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f528be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 0x7f528c0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44172/0 (socket says 172.21.15.71:44172) 2024-09-16T07:25:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f528be00640 1 -- 172.21.15.71:0/1129653863 learned_addr learned my addr 172.21.15.71:0/1129653863 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f5290c00640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f528c06f9d0 0x7f528c1f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.075+0000 7f5291600640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f528c161550 0x7f528c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f528be00640 1 -- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f528c06f9d0 msgr2=0x7f528c1f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f528be00640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f528c06f9d0 0x7f528c1f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f528be00640 1 -- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f528c161550 msgr2=0x7f528c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f528be00640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f528c161550 0x7f528c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f528be00640 1 -- 172.21.15.71:0/1129653863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c1f5a50 con 0x7f528c0af3d0 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f5291600640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f528c161550 0x7f528c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f528be00640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 0x7f528c0cdae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5278009850 tx=0x7f5278003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f5288c00640 1 -- 172.21.15.71:0/1129653863 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527803f070 con 0x7f528c0af3d0 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f5288c00640 1 -- 172.21.15.71:0/1129653863 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52780047a0 con 0x7f528c0af3d0 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f5288c00640 1 -- 172.21.15.71:0/1129653863 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278035030 con 0x7f528c0af3d0 2024-09-16T07:25:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c0cf6e0 con 0x7f528c0af3d0 2024-09-16T07:25:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.076+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c0cf980 con 0x7f528c0af3d0 2024-09-16T07:25:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.077+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528c1626a0 con 0x7f528c0af3d0 2024-09-16T07:25:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.079+0000 7f5288c00640 1 -- 172.21.15.71:0/1129653863 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f5278042070 con 0x7f528c0af3d0 2024-09-16T07:25:37.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.079+0000 7f5288c00640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f526c078bb0 0x7f526c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:37.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.080+0000 7f5288c00640 1 -- 172.21.15.71:0/1129653863 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f52780bbac0 con 0x7f528c0af3d0 2024-09-16T07:25:37.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.080+0000 7f5290c00640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f526c078bb0 0x7f526c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:37.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.080+0000 7f5290c00640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f526c078bb0 0x7f526c07b070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f527c005fd0 tx=0x7f527c00b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:37.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.084+0000 7f5288c00640 1 -- 172.21.15.71:0/1129653863 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f52780838d0 con 0x7f528c0af3d0 2024-09-16T07:25:37.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.270+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f528c0adf50 con 0x7f528c0af3d0 2024-09-16T07:25:37.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.271+0000 7f5288c00640 1 -- 172.21.15.71:0/1129653863 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) ==== 84+0+65 (secure 0 0 0) 0x7f5278088300 con 0x7f528c0af3d0 2024-09-16T07:25:37.271 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:37.271 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:25:37.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:37.271 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:25:37.271 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:25:37.271 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:37.271 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:25:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f526c078bb0 msgr2=0x7f526c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f526c078bb0 0x7f526c07b070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f527c005fd0 tx=0x7f527c00b2e0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 msgr2=0x7f528c0cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 0x7f528c0cdae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5278009850 tx=0x7f5278003910 comp rx=0 tx=0).stop 2024-09-16T07:25:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 shutdown_connections 2024-09-16T07:25:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f526c078bb0 0x7f526c07b070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f528c161550 0x7f528c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f528c0af3d0 0x7f528c0cdae0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 --2- 172.21.15.71:0/1129653863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f528c06f9d0 0x7f528c1f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 >> 172.21.15.71:0/1129653863 conn(0x7f528c0b1950 msgr2=0x7f528c15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:37.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 shutdown_connections 2024-09-16T07:25:37.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.273+0000 7f5293600640 1 -- 172.21.15.71:0/1129653863 wait complete. 2024-09-16T07:25:37.288 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:25:37.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:25:37.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 -- 172.21.15.71:0/1263805549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415e710 msgr2=0x7f70c40d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 --2- 172.21.15.71:0/1263805549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415e710 0x7f70c40d4a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f70c40b59a0 tx=0x7f70b402a560 comp rx=0 tx=0).stop 2024-09-16T07:25:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 -- 172.21.15.71:0/1263805549 shutdown_connections 2024-09-16T07:25:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 --2- 172.21.15.71:0/1263805549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70c40d08d0 0x7f70c40d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 --2- 172.21.15.71:0/1263805549 >> v2:172.21.15.177:3300/0 conn(0x7f70c415eaf0 0x7f70c415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 --2- 172.21.15.71:0/1263805549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415e710 0x7f70c40d4a20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 -- 172.21.15.71:0/1263805549 >> 172.21.15.71:0/1263805549 conn(0x7f70c40d3f00 msgr2=0x7f70c40d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 -- 172.21.15.71:0/1263805549 shutdown_connections 2024-09-16T07:25:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.452+0000 7f70c3200640 1 -- 172.21.15.71:0/1263805549 wait complete. 2024-09-16T07:25:37.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.453+0000 7f70c3200640 1 Processor -- start 2024-09-16T07:25:37.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.454+0000 7f70c3200640 1 -- start start 2024-09-16T07:25:37.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.454+0000 7f70c3200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c40d08d0 0x7f70c414e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.454+0000 7f70c3200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70c415e710 0x7f70c40ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.454+0000 7f70c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415eaf0 0x7f70c40ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.454+0000 7f70c3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4161eb0 con 0x7f70c415eaf0 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.454+0000 7f70c3200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4161d30 con 0x7f70c40d08d0 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.454+0000 7f70c3200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f70c4162030 con 0x7f70c415e710 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415eaf0 0x7f70c40ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c1400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70c415e710 0x7f70c40ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415eaf0 0x7f70c40ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44186/0 (socket says 172.21.15.71:44186) 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 -- 172.21.15.71:0/87887445 learned_addr learned my addr 172.21.15.71:0/87887445 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c1e00640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c40d08d0 0x7f70c414e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:37.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 -- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70c415e710 msgr2=0x7f70c40ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:37.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70c415e710 0x7f70c40ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 -- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c40d08d0 msgr2=0x7f70c414e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:37.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c40d08d0 0x7f70c414e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:37.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 -- 172.21.15.71:0/87887445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c41f4a60 con 0x7f70c415eaf0 2024-09-16T07:25:37.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c1e00640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c40d08d0 0x7f70c414e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:37.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.455+0000 7f70c2800640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415eaf0 0x7f70c40ca9d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00e8e0 tx=0x7f70bc00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.456+0000 7f70ba000640 1 -- 172.21.15.71:0/87887445 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc004d20 con 0x7f70c415eaf0 2024-09-16T07:25:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.456+0000 7f70ba000640 1 -- 172.21.15.71:0/87887445 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70bc00ca60 con 0x7f70c415eaf0 2024-09-16T07:25:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.456+0000 7f70ba000640 1 -- 172.21.15.71:0/87887445 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc005020 con 0x7f70c415eaf0 2024-09-16T07:25:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.456+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c41f4fc0 con 0x7f70c415eaf0 2024-09-16T07:25:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.456+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41f5580 con 0x7f70c415eaf0 2024-09-16T07:25:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.457+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c40d4a20 con 0x7f70c415eaf0 2024-09-16T07:25:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.458+0000 7f70ba000640 1 -- 172.21.15.71:0/87887445 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f70bc0051c0 con 0x7f70c415eaf0 2024-09-16T07:25:37.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.459+0000 7f70ba000640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70a4078bb0 0x7f70a407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:37.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.459+0000 7f70ba000640 1 -- 172.21.15.71:0/87887445 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f70bc09b580 con 0x7f70c415eaf0 2024-09-16T07:25:37.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.459+0000 7f70c1e00640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70a4078bb0 0x7f70a407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:37.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.460+0000 7f70c1e00640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70a4078bb0 0x7f70a407b070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f70b4033000 tx=0x7f70b4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:37.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.463+0000 7f70ba000640 1 -- 172.21.15.71:0/87887445 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f70c41f4a60 con 0x7f70c415eaf0 2024-09-16T07:25:37.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:37.647+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f70c40b9e80 con 0x7f70c415eaf0 2024-09-16T07:25:38.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.450+0000 7f70ba000640 1 -- 172.21.15.71:0/87887445 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) ==== 138+0+0 (secure 0 0 0) 0x7f70bc016030 con 0x7f70c415eaf0 2024-09-16T07:25:38.450 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70a4078bb0 msgr2=0x7f70a407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70a4078bb0 0x7f70a407b070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f70b4033000 tx=0x7f70b4004730 comp rx=0 tx=0).stop 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415eaf0 msgr2=0x7f70c40ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415eaf0 0x7f70c40ca9d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00e8e0 tx=0x7f70bc00eda0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 shutdown_connections 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70a4078bb0 0x7f70a407b070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70c415eaf0 0x7f70c40ca9d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70c415e710 0x7f70c40ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 --2- 172.21.15.71:0/87887445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c40d08d0 0x7f70c414e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 >> 172.21.15.71:0/87887445 conn(0x7f70c40d3f00 msgr2=0x7f70c415b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:38.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 shutdown_connections 2024-09-16T07:25:38.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.453+0000 7f70c3200640 1 -- 172.21.15.71:0/87887445 wait complete. 2024-09-16T07:25:38.474 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:25:38.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 -- 172.21.15.71:0/1486924227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 msgr2=0x7fe6140cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:38.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 --2- 172.21.15.71:0/1486924227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 0x7fe6140cc5c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe6140d0cd0 tx=0x7fe5fc02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 -- 172.21.15.71:0/1486924227 shutdown_connections 2024-09-16T07:25:38.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 --2- 172.21.15.71:0/1486924227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6140ccb00 0x7fe61415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 --2- 172.21.15.71:0/1486924227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 0x7fe6140cc5c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 --2- 172.21.15.71:0/1486924227 >> v2:172.21.15.177:3300/0 conn(0x7fe614145750 0x7fe614073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 -- 172.21.15.71:0/1486924227 >> 172.21.15.71:0/1486924227 conn(0x7fe6140b1840 msgr2=0x7fe6140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:38.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 -- 172.21.15.71:0/1486924227 shutdown_connections 2024-09-16T07:25:38.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.640+0000 7fe61bc00640 1 -- 172.21.15.71:0/1486924227 wait complete. 2024-09-16T07:25:38.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.642+0000 7fe61bc00640 1 Processor -- start 2024-09-16T07:25:38.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.642+0000 7fe61bc00640 1 -- start start 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe61bc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 0x7fe6141f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe61bc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6140ccb00 0x7fe61413dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe61bc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe614145750 0x7fe61413e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe61bc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6141602c0 con 0x7fe61406efa0 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe61bc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe614160140 con 0x7fe6140ccb00 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe61bc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe614160440 con 0x7fe614145750 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe619200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 0x7fe6141f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe619200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 0x7fe6141f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44204/0 (socket says 172.21.15.71:44204) 2024-09-16T07:25:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe619200640 1 -- 172.21.15.71:0/2240815904 learned_addr learned my addr 172.21.15.71:0/2240815904 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe60be00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6140ccb00 0x7fe61413dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.643+0000 7fe619c00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe614145750 0x7fe61413e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe619200640 1 -- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe614145750 msgr2=0x7fe61413e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe619200640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe614145750 0x7fe61413e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe619200640 1 -- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6140ccb00 msgr2=0x7fe61413dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe619200640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6140ccb00 0x7fe61413dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe619200640 1 -- 172.21.15.71:0/2240815904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61413ec60 con 0x7fe61406efa0 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe619200640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 0x7fe6141f72b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe60400d770 tx=0x7fe60400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe611600640 1 -- 172.21.15.71:0/2240815904 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604017070 con 0x7fe61406efa0 2024-09-16T07:25:38.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe611600640 1 -- 172.21.15.71:0/2240815904 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe604009d20 con 0x7fe61406efa0 2024-09-16T07:25:38.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61413ef40 con 0x7fe61406efa0 2024-09-16T07:25:38.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.644+0000 7fe611600640 1 -- 172.21.15.71:0/2240815904 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6040106c0 con 0x7fe61406efa0 2024-09-16T07:25:38.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.645+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61413f500 con 0x7fe61406efa0 2024-09-16T07:25:38.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.646+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61415f9f0 con 0x7fe61406efa0 2024-09-16T07:25:38.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.647+0000 7fe611600640 1 -- 172.21.15.71:0/2240815904 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fe60401a070 con 0x7fe61406efa0 2024-09-16T07:25:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.648+0000 7fe611600640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5f0078bb0 0x7fe5f007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.648+0000 7fe611600640 1 -- 172.21.15.71:0/2240815904 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5227+0+0 (secure 0 0 0) 0x7fe60409af70 con 0x7fe61406efa0 2024-09-16T07:25:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.648+0000 7fe60be00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5f0078bb0 0x7fe5f007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:38.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.649+0000 7fe60be00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5f0078bb0 0x7fe5f007b070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc033000 tx=0x7fe5fc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:38.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.653+0000 7fe611600640 1 -- 172.21.15.71:0/2240815904 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe604062b10 con 0x7fe61406efa0 2024-09-16T07:25:38.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:38.840+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe614069a10 con 0x7fe61406efa0 2024-09-16T07:25:39.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.459+0000 7fe611600640 1 -- 172.21.15.71:0/2240815904 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) ==== 170+0+0 (secure 0 0 0) 0x7fe604067540 con 0x7fe61406efa0 2024-09-16T07:25:39.459 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:25:39.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5f0078bb0 msgr2=0x7fe5f007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:39.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5f0078bb0 0x7fe5f007b070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc033000 tx=0x7fe5fc004730 comp rx=0 tx=0).stop 2024-09-16T07:25:39.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 msgr2=0x7fe6141f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:39.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 0x7fe6141f72b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe60400d770 tx=0x7fe60400dc30 comp rx=0 tx=0).stop 2024-09-16T07:25:39.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 shutdown_connections 2024-09-16T07:25:39.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5f0078bb0 0x7fe5f007b070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:39.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe614145750 0x7fe61413e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:39.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6140ccb00 0x7fe61413dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:39.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 --2- 172.21.15.71:0/2240815904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe61406efa0 0x7fe6141f72b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:39.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.463+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 >> 172.21.15.71:0/2240815904 conn(0x7fe6140b1840 msgr2=0x7fe6140d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:39.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.464+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 shutdown_connections 2024-09-16T07:25:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.464+0000 7fe61bc00640 1 -- 172.21.15.71:0/2240815904 wait complete. 2024-09-16T07:25:39.496 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:25:39.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.658+0000 7f551b600640 1 -- 172.21.15.71:0/416567004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f8003c10 msgr2=0x7f54f8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:39.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.658+0000 7f551b600640 1 --2- 172.21.15.71:0/416567004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f8003c10 0x7f54f8004010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f550400c640 tx=0x7f54f8005830 comp rx=0 tx=0).stop 2024-09-16T07:25:39.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.658+0000 7f5519600640 1 -- 172.21.15.71:0/416567004 reap_dead start 2024-09-16T07:25:39.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.658+0000 7f551b600640 1 -- 172.21.15.71:0/416567004 shutdown_connections 2024-09-16T07:25:39.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.658+0000 7f551b600640 1 -- 172.21.15.71:0/416567004 >> 172.21.15.71:0/416567004 conn(0x7f55140b1840 msgr2=0x7f55140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:39.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.658+0000 7f551b600640 1 -- 172.21.15.71:0/416567004 shutdown_connections 2024-09-16T07:25:39.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.658+0000 7f551b600640 1 -- 172.21.15.71:0/416567004 wait complete. 2024-09-16T07:25:39.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.660+0000 7f551b600640 1 Processor -- start 2024-09-16T07:25:39.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.660+0000 7f551b600640 1 -- start start 2024-09-16T07:25:39.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f551b600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8003c10 0x7f551413cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f551b600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f551415e810 0x7f551413b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f551b600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551413fb40 0x7f55141422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f551b600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55140737d0 con 0x7f551415e810 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f551b600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5514073650 con 0x7f54f8003c10 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f551b600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55140ce2c0 con 0x7f551413fb40 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f5513e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f551415e810 0x7f551413b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f5513e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f551415e810 0x7f551413b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44222/0 (socket says 172.21.15.71:44222) 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f5513e00640 1 -- 172.21.15.71:0/967163311 learned_addr learned my addr 172.21.15.71:0/967163311 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:39.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f5518c00640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8003c10 0x7f551413cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.661+0000 7f5519600640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551413fb40 0x7f55141422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5513e00640 1 -- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551413fb40 msgr2=0x7f55141422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5513e00640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551413fb40 0x7f55141422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5513e00640 1 -- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8003c10 msgr2=0x7f551413cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5513e00640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8003c10 0x7f551413cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5513e00640 1 -- 172.21.15.71:0/967163311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55141f91c0 con 0x7f551415e810 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5513e00640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f551415e810 0x7f551413b840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5500011ae0 tx=0x7f5500009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5510c00640 1 -- 172.21.15.71:0/967163311 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500004460 con 0x7f551415e810 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5510c00640 1 -- 172.21.15.71:0/967163311 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f550000fa00 con 0x7f551415e810 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f5510c00640 1 -- 172.21.15.71:0/967163311 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550000fe50 con 0x7f551415e810 2024-09-16T07:25:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5514141530 con 0x7f551415e810 2024-09-16T07:25:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.662+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5514141af0 con 0x7f551415e810 2024-09-16T07:25:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.663+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55140cdbb0 con 0x7f551415e810 2024-09-16T07:25:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.665+0000 7f5510c00640 1 -- 172.21.15.71:0/967163311 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f55000035f0 con 0x7f551415e810 2024-09-16T07:25:39.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.666+0000 7f5510c00640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f4078bb0 0x7f54f407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:39.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.666+0000 7f5510c00640 1 -- 172.21.15.71:0/967163311 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5609+0+0 (secure 0 0 0) 0x7f550009c3f0 con 0x7f551415e810 2024-09-16T07:25:39.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.667+0000 7f5518c00640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f4078bb0 0x7f54f407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:39.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.667+0000 7f5518c00640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f4078bb0 0x7f54f407b070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5504033000 tx=0x7f551406f410 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.670+0000 7f5510c00640 1 -- 172.21.15.71:0/967163311 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5500063f50 con 0x7f551415e810 2024-09-16T07:25:39.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:39.893+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f55141435f0 con 0x7f551415e810 2024-09-16T07:25:40.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.471+0000 7f5510c00640 1 -- 172.21.15.71:0/967163311 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7f5500068980 con 0x7f551415e810 2024-09-16T07:25:40.471 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 6 and data pool 7 2024-09-16T07:25:40.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.476+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f4078bb0 msgr2=0x7f54f407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:40.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.476+0000 7f551b600640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f4078bb0 0x7f54f407b070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5504033000 tx=0x7f551406f410 comp rx=0 tx=0).stop 2024-09-16T07:25:40.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.476+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f551415e810 msgr2=0x7f551413b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:40.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.476+0000 7f551b600640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f551415e810 0x7f551413b840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5500011ae0 tx=0x7f5500009e90 comp rx=0 tx=0).stop 2024-09-16T07:25:40.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.477+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 shutdown_connections 2024-09-16T07:25:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.477+0000 7f551b600640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f4078bb0 0x7f54f407b070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.477+0000 7f551b600640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551413fb40 0x7f55141422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.477+0000 7f551b600640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f551415e810 0x7f551413b840 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.477+0000 7f551b600640 1 --2- 172.21.15.71:0/967163311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8003c10 0x7f551413cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.477+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 >> 172.21.15.71:0/967163311 conn(0x7f55140b1840 msgr2=0x7f5514143df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.477+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 shutdown_connections 2024-09-16T07:25:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.477+0000 7f551b600640 1 -- 172.21.15.71:0/967163311 wait complete. 2024-09-16T07:25:40.505 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:25:40.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.663+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2698985381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 msgr2=0x7fa1f40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.663+0000 7fa1f8c00640 1 --2- 172.21.15.71:0/2698985381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 0x7fa1f40b28f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec005fd0 tx=0x7fa1ec02a8b0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.663+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2698985381 shutdown_connections 2024-09-16T07:25:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.663+0000 7fa1f8c00640 1 --2- 172.21.15.71:0/2698985381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 0x7fa1f40b28f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.663+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2698985381 >> 172.21.15.71:0/2698985381 conn(0x7fa1f40cfa20 msgr2=0x7fa1f40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.663+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2698985381 shutdown_connections 2024-09-16T07:25:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.663+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2698985381 wait complete. 2024-09-16T07:25:40.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.665+0000 7fa1f8c00640 1 Processor -- start 2024-09-16T07:25:40.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.665+0000 7fa1f8c00640 1 -- start start 2024-09-16T07:25:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 0x7fa1f40ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f8c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1f415bc30 0x7fa1f4154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f8c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1f40c9400 0x7fa1f40af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f8c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f4074660 con 0x7fa1f40d0540 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f8c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f406fa30 con 0x7fa1f415bc30 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f8c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f40c98b0 con 0x7fa1f40c9400 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 0x7fa1f40ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 0x7fa1f40ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44248/0 (socket says 172.21.15.71:44248) 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3400640 1 -- 172.21.15.71:0/2310927943 learned_addr learned my addr 172.21.15.71:0/2310927943 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f2a00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1f415bc30 0x7fa1f4154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3e00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1f40c9400 0x7fa1f40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3400640 1 -- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1f40c9400 msgr2=0x7fa1f40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3400640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1f40c9400 0x7fa1f40af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3400640 1 -- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1f415bc30 msgr2=0x7fa1f4154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3400640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1f415bc30 0x7fa1f4154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.666+0000 7fa1f3400640 1 -- 172.21.15.71:0/2310927943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f40ae950 con 0x7fa1f40d0540 2024-09-16T07:25:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.667+0000 7fa1f3400640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 0x7fa1f40ae290 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400b3f0 tx=0x7fa1e400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.667+0000 7fa1eb400640 1 -- 172.21.15.71:0/2310927943 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4015020 con 0x7fa1f40d0540 2024-09-16T07:25:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.667+0000 7fa1eb400640 1 -- 172.21.15.71:0/2310927943 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1e4011920 con 0x7fa1f40d0540 2024-09-16T07:25:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.667+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f40c6b90 con 0x7fa1f40d0540 2024-09-16T07:25:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.667+0000 7fa1eb400640 1 -- 172.21.15.71:0/2310927943 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4011d70 con 0x7fa1f40d0540 2024-09-16T07:25:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.667+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f40c7150 con 0x7fa1f40d0540 2024-09-16T07:25:40.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.668+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f40b28f0 con 0x7fa1f40d0540 2024-09-16T07:25:40.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.669+0000 7fa1eb400640 1 -- 172.21.15.71:0/2310927943 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fa1e4018020 con 0x7fa1f40d0540 2024-09-16T07:25:40.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.670+0000 7fa1eb400640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa1d4078bb0 0x7fa1d407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:40.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.670+0000 7fa1eb400640 1 -- 172.21.15.71:0/2310927943 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e409abb0 con 0x7fa1f40d0540 2024-09-16T07:25:40.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.671+0000 7fa1f2a00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa1d4078bb0 0x7fa1d407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:40.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.671+0000 7fa1f2a00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa1d4078bb0 0x7fa1d407b070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1f41f7cc0 tx=0x7fa1e0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:40.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.674+0000 7fa1eb400640 1 -- 172.21.15.71:0/2310927943 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa1e401f030 con 0x7fa1f40d0540 2024-09-16T07:25:40.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.860+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1f4073da0 con 0x7fa1f40d0540 2024-09-16T07:25:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.861+0000 7fa1eb400640 1 -- 172.21.15.71:0/2310927943 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16694 (secure 0 0 0) 0x7fa1e4062d90 con 0x7fa1f40d0540 2024-09-16T07:25:40.939 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:40.939 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":47,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:25:39.895219+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:25:37.657272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T07:25:38.850365+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:25:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa1d4078bb0 msgr2=0x7fa1d407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa1d4078bb0 0x7fa1d407b070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1f41f7cc0 tx=0x7fa1e0009040 comp rx=0 tx=0).stop 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 msgr2=0x7fa1f40ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 0x7fa1f40ae290 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400b3f0 tx=0x7fa1e400b8b0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 shutdown_connections 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa1d4078bb0 0x7fa1d407b070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1f40c9400 0x7fa1f40af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1f415bc30 0x7fa1f4154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 --2- 172.21.15.71:0/2310927943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1f40d0540 0x7fa1f40ae290 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.863+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 >> 172.21.15.71:0/2310927943 conn(0x7fa1f40cfa20 msgr2=0x7fa1f4157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.864+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 shutdown_connections 2024-09-16T07:25:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:40.864+0000 7fa1f8c00640 1 -- 172.21.15.71:0/2310927943 wait complete. 2024-09-16T07:25:40.942 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:25:41.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.099+0000 7f9cdf400640 1 -- 172.21.15.71:0/1496893797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 msgr2=0x7f9cd8074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:41.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.099+0000 7f9cdf400640 1 --2- 172.21.15.71:0/1496893797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 0x7f9cd8074460 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0005fd0 tx=0x7f9cd002a820 comp rx=0 tx=0).stop 2024-09-16T07:25:41.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.099+0000 7f9cdf400640 1 -- 172.21.15.71:0/1496893797 shutdown_connections 2024-09-16T07:25:41.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.099+0000 7f9cdf400640 1 --2- 172.21.15.71:0/1496893797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 0x7f9cd8074460 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.099+0000 7f9cdf400640 1 --2- 172.21.15.71:0/1496893797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cd815f5a0 0x7f9cd80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.099+0000 7f9cdf400640 1 -- 172.21.15.71:0/1496893797 >> 172.21.15.71:0/1496893797 conn(0x7f9cd80b1950 msgr2=0x7f9cd80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:41.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.099+0000 7f9cdf400640 1 -- 172.21.15.71:0/1496893797 shutdown_connections 2024-09-16T07:25:41.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.099+0000 7f9cdf400640 1 -- 172.21.15.71:0/1496893797 wait complete. 2024-09-16T07:25:41.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.101+0000 7f9cdf400640 1 Processor -- start 2024-09-16T07:25:41.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.101+0000 7f9cdf400640 1 -- start start 2024-09-16T07:25:41.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 0x7f9cd8145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdf400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cd80ccf00 0x7f9cd809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdf400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cd815f5a0 0x7f9cd809bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdf400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd806f2e0 con 0x7f9cd8073fb0 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdf400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd806f160 con 0x7f9cd815f5a0 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdf400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd806f460 con 0x7f9cd80ccf00 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 0x7f9cd8145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 0x7f9cd8145900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44266/0 (socket says 172.21.15.71:44266) 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdca00640 1 -- 172.21.15.71:0/3153281489 learned_addr learned my addr 172.21.15.71:0/3153281489 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cd7e00640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cd80ccf00 0x7f9cd809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.102+0000 7f9cdd400640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cd815f5a0 0x7f9cd809bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:41.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdca00640 1 -- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cd80ccf00 msgr2=0x7f9cd809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdca00640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cd80ccf00 0x7f9cd809ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdca00640 1 -- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cd815f5a0 msgr2=0x7f9cd809bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdca00640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cd815f5a0 0x7f9cd809bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdca00640 1 -- 172.21.15.71:0/3153281489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd815b920 con 0x7f9cd8073fb0 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdd400640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cd815f5a0 0x7f9cd809bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdca00640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 0x7f9cd8145900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800d780 tx=0x7f9cc800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cd4c00640 1 -- 172.21.15.71:0/3153281489 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8017070 con 0x7f9cd8073fb0 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cd4c00640 1 -- 172.21.15.71:0/3153281489 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cc800de90 con 0x7f9cd8073fb0 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cd4c00640 1 -- 172.21.15.71:0/3153281489 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80099d0 con 0x7f9cd8073fb0 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd815bba0 con 0x7f9cd8073fb0 2024-09-16T07:25:41.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.103+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd815c160 con 0x7f9cd8073fb0 2024-09-16T07:25:41.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.104+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd806f9f0 con 0x7f9cd8073fb0 2024-09-16T07:25:41.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.105+0000 7f9cd4c00640 1 -- 172.21.15.71:0/3153281489 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f9cc8009b70 con 0x7f9cd8073fb0 2024-09-16T07:25:41.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.106+0000 7f9cd4c00640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9cb8078bb0 0x7f9cb807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:41.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.106+0000 7f9cd4c00640 1 -- 172.21.15.71:0/3153281489 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f9cc809b230 con 0x7f9cd8073fb0 2024-09-16T07:25:41.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.107+0000 7f9cd7e00640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9cb8078bb0 0x7f9cb807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:41.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.108+0000 7f9cd7e00640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9cb8078bb0 0x7f9cb807b070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4007fd0 tx=0x7f9cc4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:41.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.111+0000 7f9cd4c00640 1 -- 172.21.15.71:0/3153281489 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9cc8020020 con 0x7f9cd8073fb0 2024-09-16T07:25:41.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.297+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cd80cbef0 con 0x7f9cd8073fb0 2024-09-16T07:25:41.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.298+0000 7f9cd4c00640 1 -- 172.21.15.71:0/3153281489 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16694 (secure 0 0 0) 0x7f9cd815b920 con 0x7f9cd8073fb0 2024-09-16T07:25:41.298 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:41.298 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":47,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:25:39.895219+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:25:37.657272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T07:25:38.850365+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:25:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.300+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9cb8078bb0 msgr2=0x7f9cb807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.300+0000 7f9cdf400640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9cb8078bb0 0x7f9cb807b070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4007fd0 tx=0x7f9cc4009040 comp rx=0 tx=0).stop 2024-09-16T07:25:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.300+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 msgr2=0x7f9cd8145900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.300+0000 7f9cdf400640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 0x7f9cd8145900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800d780 tx=0x7f9cc800dc40 comp rx=0 tx=0).stop 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.301+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 shutdown_connections 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.301+0000 7f9cdf400640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9cb8078bb0 0x7f9cb807b070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.301+0000 7f9cdf400640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cd815f5a0 0x7f9cd809bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.301+0000 7f9cdf400640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cd80ccf00 0x7f9cd809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.301+0000 7f9cdf400640 1 --2- 172.21.15.71:0/3153281489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd8073fb0 0x7f9cd8145900 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.301+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 >> 172.21.15.71:0/3153281489 conn(0x7f9cd80b1950 msgr2=0x7f9cd80acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.301+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 shutdown_connections 2024-09-16T07:25:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.301+0000 7f9cdf400640 1 -- 172.21.15.71:0/3153281489 wait complete. 2024-09-16T07:25:41.316 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:25:41.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.483+0000 7f470a000640 1 -- 172.21.15.71:0/1055208225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46e8004000 msgr2=0x7f46e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:41.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.483+0000 7f470a000640 1 --2- 172.21.15.71:0/1055208225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46e8004000 0x7f46e8004800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c310 tx=0x7f46e8005830 comp rx=0 tx=0).stop 2024-09-16T07:25:41.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.483+0000 7f4703e00640 1 -- 172.21.15.71:0/1055208225 reap_dead start 2024-09-16T07:25:41.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.483+0000 7f470a000640 1 -- 172.21.15.71:0/1055208225 shutdown_connections 2024-09-16T07:25:41.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.483+0000 7f470a000640 1 -- 172.21.15.71:0/1055208225 >> 172.21.15.71:0/1055208225 conn(0x7f47040d1e40 msgr2=0x7f47040d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:41.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.483+0000 7f470a000640 1 -- 172.21.15.71:0/1055208225 shutdown_connections 2024-09-16T07:25:41.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.483+0000 7f470a000640 1 -- 172.21.15.71:0/1055208225 wait complete. 2024-09-16T07:25:41.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.485+0000 7f470a000640 1 Processor -- start 2024-09-16T07:25:41.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.485+0000 7f470a000640 1 -- start start 2024-09-16T07:25:41.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.485+0000 7f470a000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46e8004000 0x7f47041513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:41.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f470a000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47041f4700 0x7f47041f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:41.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f470a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f470409bc00 0x7f470409c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:41.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f470a000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f47040b0eb0 con 0x7f470409bc00 2024-09-16T07:25:41.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f470a000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47040b0d30 con 0x7f47041f4700 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f470a000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f47040b1030 con 0x7f46e8004000 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f470409bc00 0x7f470409c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f470409bc00 0x7f470409c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44274/0 (socket says 172.21.15.71:44274) 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 -- 172.21.15.71:0/3999289298 learned_addr learned my addr 172.21.15.71:0/3999289298 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703400640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46e8004000 0x7f47041513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4702a00640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47041f4700 0x7f47041f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 -- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46e8004000 msgr2=0x7f47041513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46e8004000 0x7f47041513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 -- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47041f4700 msgr2=0x7f47041f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47041f4700 0x7f47041f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 -- 172.21.15.71:0/3999289298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470409dbd0 con 0x7f470409bc00 2024-09-16T07:25:41.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703400640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46e8004000 0x7f47041513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.486+0000 7f4703e00640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f470409bc00 0x7f470409c060 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f47040b1610 tx=0x7f46fc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.487+0000 7f46fb400640 1 -- 172.21.15.71:0/3999289298 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc004b20 con 0x7f470409bc00 2024-09-16T07:25:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.487+0000 7f46fb400640 1 -- 172.21.15.71:0/3999289298 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46fc004cc0 con 0x7f470409bc00 2024-09-16T07:25:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.487+0000 7f46fb400640 1 -- 172.21.15.71:0/3999289298 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc003bd0 con 0x7f470409bc00 2024-09-16T07:25:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.487+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4704151910 con 0x7f470409bc00 2024-09-16T07:25:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.487+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47041f5480 con 0x7f470409bc00 2024-09-16T07:25:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.488+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47041f56c0 con 0x7f470409bc00 2024-09-16T07:25:41.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.489+0000 7f46fb400640 1 -- 172.21.15.71:0/3999289298 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f46fc004030 con 0x7f470409bc00 2024-09-16T07:25:41.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.490+0000 7f46fb400640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f46e4078be0 0x7f46e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:41.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.490+0000 7f46fb400640 1 -- 172.21.15.71:0/3999289298 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f46fc09b970 con 0x7f470409bc00 2024-09-16T07:25:41.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.490+0000 7f4703400640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f46e4078be0 0x7f46e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:41.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.491+0000 7f4703400640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f46e4078be0 0x7f46e407b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46f4000c00 tx=0x7f46f4006b70 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:41.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.494+0000 7f46fb400640 1 -- 172.21.15.71:0/3999289298 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f47041f56c0 con 0x7f470409bc00 2024-09-16T07:25:41.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:41.714+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f47040d0d50 con 0x7f470409bc00 2024-09-16T07:25:42.446 INFO:tasks.ceph.mds.a.smithi071.stderr:2024-09-16T07:25:42.447+0000 7f7832800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:25:42.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.487+0000 7f46fb400640 1 -- 172.21.15.71:0/3999289298 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f46fc003d70 con 0x7f470409bc00 2024-09-16T07:25:42.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f46e4078be0 msgr2=0x7f46e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:42.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f46e4078be0 0x7f46e407b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46f4000c00 tx=0x7f46f4006b70 comp rx=0 tx=0).stop 2024-09-16T07:25:42.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f470409bc00 msgr2=0x7f470409c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f470409bc00 0x7f470409c060 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f47040b1610 tx=0x7f46fc00f3c0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 shutdown_connections 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f46e4078be0 0x7f46e407b0a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f470409bc00 0x7f470409c060 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47041f4700 0x7f47041f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 --2- 172.21.15.71:0/3999289298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46e8004000 0x7f47041513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 >> 172.21.15.71:0/3999289298 conn(0x7f47040d1e40 msgr2=0x7f4704161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 shutdown_connections 2024-09-16T07:25:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.490+0000 7f470a000640 1 -- 172.21.15.71:0/3999289298 wait complete. 2024-09-16T07:25:42.507 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:25:42.507 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:25:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.668+0000 7f4f45600640 1 -- 172.21.15.71:0/791425758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f40145750 msgr2=0x7f4f40073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.668+0000 7f4f45600640 1 --2- 172.21.15.71:0/791425758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f40145750 0x7f4f40073440 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4f30005fd0 tx=0x7f4f3002a8a0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.668+0000 7f4f45600640 1 -- 172.21.15.71:0/791425758 shutdown_connections 2024-09-16T07:25:42.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.668+0000 7f4f45600640 1 --2- 172.21.15.71:0/791425758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f40145750 0x7f4f40073440 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.668+0000 7f4f45600640 1 -- 172.21.15.71:0/791425758 >> 172.21.15.71:0/791425758 conn(0x7f4f400b1840 msgr2=0x7f4f400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:42.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.669+0000 7f4f45600640 1 -- 172.21.15.71:0/791425758 shutdown_connections 2024-09-16T07:25:42.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.669+0000 7f4f45600640 1 -- 172.21.15.71:0/791425758 wait complete. 2024-09-16T07:25:42.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.670+0000 7f4f45600640 1 Processor -- start 2024-09-16T07:25:42.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.670+0000 7f4f45600640 1 -- start start 2024-09-16T07:25:42.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f45600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f4006efa0 0x7f4f40141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:42.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f45600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f400cccd0 0x7f4f401f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:42.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f45600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f40145750 0x7f4f400c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:42.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f45600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4f400ce6a0 con 0x7f4f400cccd0 2024-09-16T07:25:42.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f45600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f400ce520 con 0x7f4f4006efa0 2024-09-16T07:25:42.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f45600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4f400ce820 con 0x7f4f40145750 2024-09-16T07:25:42.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f3e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f400cccd0 0x7f4f401f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:42.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f3e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f400cccd0 0x7f4f401f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44298/0 (socket says 172.21.15.71:44298) 2024-09-16T07:25:42.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.671+0000 7f4f3e000640 1 -- 172.21.15.71:0/191151358 learned_addr learned my addr 172.21.15.71:0/191151358 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:42.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f3e000640 1 -- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f40145750 msgr2=0x7f4f400c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:42.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f3e000640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f40145750 0x7f4f400c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f3e000640 1 -- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f4006efa0 msgr2=0x7f4f40141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:42.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f3e000640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f4006efa0 0x7f4f40141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:42.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f3e000640 1 -- 172.21.15.71:0/191151358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f40141e70 con 0x7f4f400cccd0 2024-09-16T07:25:42.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f3e000640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f400cccd0 0x7f4f401f5910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c4c0 tx=0x7f4f2800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:42.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f36a00640 1 -- 172.21.15.71:0/191151358 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28016020 con 0x7f4f400cccd0 2024-09-16T07:25:42.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f36a00640 1 -- 172.21.15.71:0/191151358 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f28012940 con 0x7f4f400cccd0 2024-09-16T07:25:42.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f400b44e0 con 0x7f4f400cccd0 2024-09-16T07:25:42.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f36a00640 1 -- 172.21.15.71:0/191151358 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28012d90 con 0x7f4f400cccd0 2024-09-16T07:25:42.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.672+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f400b4aa0 con 0x7f4f400cccd0 2024-09-16T07:25:42.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.673+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f400af310 con 0x7f4f400cccd0 2024-09-16T07:25:42.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.674+0000 7f4f36a00640 1 -- 172.21.15.71:0/191151358 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f4f28019020 con 0x7f4f400cccd0 2024-09-16T07:25:42.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.675+0000 7f4f36a00640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4f1c078bb0 0x7f4f1c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:42.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.675+0000 7f4f36a00640 1 -- 172.21.15.71:0/191151358 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f4f2809ac40 con 0x7f4f400cccd0 2024-09-16T07:25:42.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.676+0000 7f4f3ea00640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4f1c078bb0 0x7f4f1c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:42.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.676+0000 7f4f3ea00640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4f1c078bb0 0x7f4f1c07b070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4f30034000 tx=0x7f4f30004a40 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:42.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.679+0000 7f4f36a00640 1 -- 172.21.15.71:0/191151358 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4f400af310 con 0x7f4f400cccd0 2024-09-16T07:25:42.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:42.902+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4f400cecd0 con 0x7f4f400cccd0 2024-09-16T07:25:43.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.495+0000 7f4f36a00640 1 -- 172.21.15.71:0/191151358 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f4f28020030 con 0x7f4f400cccd0 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4f1c078bb0 msgr2=0x7f4f1c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4f1c078bb0 0x7f4f1c07b070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4f30034000 tx=0x7f4f30004a40 comp rx=0 tx=0).stop 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f400cccd0 msgr2=0x7f4f401f5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f400cccd0 0x7f4f401f5910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c4c0 tx=0x7f4f2800c980 comp rx=0 tx=0).stop 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 shutdown_connections 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4f1c078bb0 0x7f4f1c07b070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f40145750 0x7f4f400c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f400cccd0 0x7f4f401f5910 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 --2- 172.21.15.71:0/191151358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f4006efa0 0x7f4f40141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 >> 172.21.15.71:0/191151358 conn(0x7f4f400b1840 msgr2=0x7f4f400b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:43.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 shutdown_connections 2024-09-16T07:25:43.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.499+0000 7f4f45600640 1 -- 172.21.15.71:0/191151358 wait complete. 2024-09-16T07:25:43.516 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:25:43.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:25:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.677+0000 7fdb5fa00640 1 -- 172.21.15.71:0/3988233230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 msgr2=0x7fdb3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.677+0000 7fdb5fa00640 1 --2- 172.21.15.71:0/3988233230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 0x7fdb3c003e20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c2e0 tx=0x7fdb3c005830 comp rx=0 tx=0).stop 2024-09-16T07:25:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.677+0000 7fdb5da00640 1 -- 172.21.15.71:0/3988233230 reap_dead start 2024-09-16T07:25:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.677+0000 7fdb5fa00640 1 -- 172.21.15.71:0/3988233230 shutdown_connections 2024-09-16T07:25:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.677+0000 7fdb5fa00640 1 -- 172.21.15.71:0/3988233230 >> 172.21.15.71:0/3988233230 conn(0x7fdb580b1950 msgr2=0x7fdb580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.677+0000 7fdb5fa00640 1 -- 172.21.15.71:0/3988233230 shutdown_connections 2024-09-16T07:25:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.677+0000 7fdb5fa00640 1 -- 172.21.15.71:0/3988233230 wait complete. 2024-09-16T07:25:43.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.679+0000 7fdb5fa00640 1 Processor -- start 2024-09-16T07:25:43.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.679+0000 7fdb5fa00640 1 -- start start 2024-09-16T07:25:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5fa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 0x7fdb5814e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5fa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb581f8370 0x7fdb581f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5fa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806f2d0 0x7fdb5806f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5fa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdb580745c0 con 0x7fdb3c003a00 2024-09-16T07:25:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5fa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58074440 con 0x7fdb5806f2d0 2024-09-16T07:25:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5fa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdb58074740 con 0x7fdb581f8370 2024-09-16T07:25:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 0x7fdb5814e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 0x7fdb5814e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44314/0 (socket says 172.21.15.71:44314) 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.680+0000 7fdb5d000640 1 -- 172.21.15.71:0/1547620115 learned_addr learned my addr 172.21.15.71:0/1547620115 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5da00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806f2d0 0x7fdb5806f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb57e00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb581f8370 0x7fdb581f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5d000640 1 -- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb581f8370 msgr2=0x7fdb581f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5d000640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb581f8370 0x7fdb581f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5d000640 1 -- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806f2d0 msgr2=0x7fdb5806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5d000640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806f2d0 0x7fdb5806f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5d000640 1 -- 172.21.15.71:0/1547620115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb581f8e10 con 0x7fdb3c003a00 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5d000640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 0x7fdb5814e4d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c400 tx=0x7fdb48006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb54c00640 1 -- 172.21.15.71:0/1547620115 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48031040 con 0x7fdb3c003a00 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb54c00640 1 -- 172.21.15.71:0/1547620115 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb480065e0 con 0x7fdb3c003a00 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb54c00640 1 -- 172.21.15.71:0/1547620115 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48003a40 con 0x7fdb3c003a00 2024-09-16T07:25:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb581f90f0 con 0x7fdb3c003a00 2024-09-16T07:25:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.681+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb58157ca0 con 0x7fdb3c003a00 2024-09-16T07:25:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.682+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb580b2440 con 0x7fdb3c003a00 2024-09-16T07:25:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.684+0000 7fdb54c00640 1 -- 172.21.15.71:0/1547620115 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fdb48034070 con 0x7fdb3c003a00 2024-09-16T07:25:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.684+0000 7fdb54c00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb38078be0 0x7fdb3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.685+0000 7fdb54c00640 1 -- 172.21.15.71:0/1547620115 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fdb480baec0 con 0x7fdb3c003a00 2024-09-16T07:25:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.685+0000 7fdb57e00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb38078be0 0x7fdb3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.685+0000 7fdb57e00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb38078be0 0x7fdb3807b0a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb580cb580 tx=0x7fdb44005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.689+0000 7fdb54c00640 1 -- 172.21.15.71:0/1547620115 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdb48083060 con 0x7fdb3c003a00 2024-09-16T07:25:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.909+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb580b0760 con 0x7fdb3c003a00 2024-09-16T07:25:43.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.909+0000 7fdb54c00640 1 -- 172.21.15.71:0/1547620115 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fdb48082960 con 0x7fdb3c003a00 2024-09-16T07:25:43.909 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:43.909 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":27,"btime":"2024-09-16T07:25:43:488308+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1366102730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1366102730},{"type":"v1","addr":"172.21.15.177:6836","nonce":1366102730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/796136206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":796136206},{"type":"v1","addr":"172.21.15.177:6837","nonce":796136206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3187236541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3187236541},{"type":"v1","addr":"172.21.15.71:6834","nonce":3187236541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:25:39.895186+0000","modified":"2024-09-16T07:25:42.903826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2966230103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2966230103},{"type":"v1","addr":"172.21.15.71:6835","nonce":2966230103}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-16T07:25:43.910 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 27 2024-09-16T07:25:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb38078be0 msgr2=0x7fdb3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb38078be0 0x7fdb3807b0a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb580cb580 tx=0x7fdb44005f90 comp rx=0 tx=0).stop 2024-09-16T07:25:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 msgr2=0x7fdb5814e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 0x7fdb5814e4d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c400 tx=0x7fdb48006ce0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 shutdown_connections 2024-09-16T07:25:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb38078be0 0x7fdb3807b0a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806f2d0 0x7fdb5806f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb581f8370 0x7fdb581f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 --2- 172.21.15.71:0/1547620115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb3c003a00 0x7fdb5814e4d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 >> 172.21.15.71:0/1547620115 conn(0x7fdb580b1950 msgr2=0x7fdb580cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 shutdown_connections 2024-09-16T07:25:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:43.912+0000 7fdb5fa00640 1 -- 172.21.15.71:0/1547620115 wait complete. 2024-09-16T07:25:43.927 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:25:43.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:25:43.934 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:25:44.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.087+0000 7ff23b400640 1 -- 172.21.15.71:0/1587644851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff218003c30 msgr2=0x7ff218004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.087+0000 7ff23b400640 1 --2- 172.21.15.71:0/1587644851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff218003c30 0x7ff218004030 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c560 tx=0x7ff218005830 comp rx=0 tx=0).stop 2024-09-16T07:25:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.087+0000 7ff23b400640 1 -- 172.21.15.71:0/1587644851 shutdown_connections 2024-09-16T07:25:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.087+0000 7ff23b400640 1 --2- 172.21.15.71:0/1587644851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff218003c30 0x7ff218004030 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.087+0000 7ff23b400640 1 -- 172.21.15.71:0/1587644851 >> 172.21.15.71:0/1587644851 conn(0x7ff2340b1950 msgr2=0x7ff2340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.087+0000 7ff23b400640 1 -- 172.21.15.71:0/1587644851 shutdown_connections 2024-09-16T07:25:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.087+0000 7ff23b400640 1 -- 172.21.15.71:0/1587644851 wait complete. 2024-09-16T07:25:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.089+0000 7ff23b400640 1 Processor -- start 2024-09-16T07:25:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.089+0000 7ff23b400640 1 -- start start 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff23b400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff218003c30 0x7ff234152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff23b400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff234143fc0 0x7ff2340ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff23b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2340cf070 0x7ff2340cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff23b400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2340ad0c0 con 0x7ff2340cf070 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff23b400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2340acf40 con 0x7ff234143fc0 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff23b400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff2340ad240 con 0x7ff218003c30 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff239400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2340cf070 0x7ff2340cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff239400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2340cf070 0x7ff2340cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44330/0 (socket says 172.21.15.71:44330) 2024-09-16T07:25:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff239400640 1 -- 172.21.15.71:0/747906993 learned_addr learned my addr 172.21.15.71:0/747906993 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff238a00640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff218003c30 0x7ff234152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.090+0000 7ff233e00640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff234143fc0 0x7ff2340ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff239400640 1 -- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff218003c30 msgr2=0x7ff234152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff239400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff218003c30 0x7ff234152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff239400640 1 -- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff234143fc0 msgr2=0x7ff2340ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff239400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff234143fc0 0x7ff2340ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff239400640 1 -- 172.21.15.71:0/747906993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff234158de0 con 0x7ff2340cf070 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff238a00640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff218003c30 0x7ff234152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff233e00640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff234143fc0 0x7ff2340ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff239400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2340cf070 0x7ff2340cf4f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff234076720 tx=0x7ff22c011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff230c00640 1 -- 172.21.15.71:0/747906993 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c004440 con 0x7ff2340cf070 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff230c00640 1 -- 172.21.15.71:0/747906993 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22c00f920 con 0x7ff2340cf070 2024-09-16T07:25:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff230c00640 1 -- 172.21.15.71:0/747906993 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00fd70 con 0x7ff2340cf070 2024-09-16T07:25:44.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff234156e70 con 0x7ff2340cf070 2024-09-16T07:25:44.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.091+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff234159080 con 0x7ff2340cf070 2024-09-16T07:25:44.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.092+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff234157390 con 0x7ff2340cf070 2024-09-16T07:25:44.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.094+0000 7ff230c00640 1 -- 172.21.15.71:0/747906993 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7ff22c0035f0 con 0x7ff2340cf070 2024-09-16T07:25:44.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.095+0000 7ff230c00640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff214078be0 0x7ff21407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.095+0000 7ff230c00640 1 -- 172.21.15.71:0/747906993 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7ff22c09b480 con 0x7ff2340cf070 2024-09-16T07:25:44.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.095+0000 7ff238a00640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff214078be0 0x7ff21407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.096+0000 7ff238a00640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff214078be0 0x7ff21407b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff224033000 tx=0x7ff224038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:44.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.099+0000 7ff230c00640 1 -- 172.21.15.71:0/747906993 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff22c063620 con 0x7ff2340cf070 2024-09-16T07:25:44.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.285+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff234073fb0 con 0x7ff2340cf070 2024-09-16T07:25:44.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.286+0000 7ff230c00640 1 -- 172.21.15.71:0/747906993 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16681 (secure 0 0 0) 0x7ff22c062f20 con 0x7ff2340cf070 2024-09-16T07:25:44.286 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:44.286 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":48,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:25:41.471266+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:25:37.657272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T07:25:38.850365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:25:44.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff214078be0 msgr2=0x7ff21407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff214078be0 0x7ff21407b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff224033000 tx=0x7ff224038040 comp rx=0 tx=0).stop 2024-09-16T07:25:44.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2340cf070 msgr2=0x7ff2340cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2340cf070 0x7ff2340cf4f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff234076720 tx=0x7ff22c011c60 comp rx=0 tx=0).stop 2024-09-16T07:25:44.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 shutdown_connections 2024-09-16T07:25:44.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff214078be0 0x7ff21407b0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2340cf070 0x7ff2340cf4f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff234143fc0 0x7ff2340ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 --2- 172.21.15.71:0/747906993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff218003c30 0x7ff234152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 >> 172.21.15.71:0/747906993 conn(0x7ff2340b1950 msgr2=0x7ff23415d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:44.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.288+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 shutdown_connections 2024-09-16T07:25:44.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.289+0000 7ff23b400640 1 -- 172.21.15.71:0/747906993 wait complete. 2024-09-16T07:25:44.304 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:25:44.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:25:44.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.501+0000 7f01ed400640 1 -- 172.21.15.71:0/1073904268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e815f5a0 msgr2=0x7f01e80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.501+0000 7f01ed400640 1 --2- 172.21.15.71:0/1073904268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e815f5a0 0x7f01e80b2440 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f01e806dac0 tx=0x7f01d802a4a0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.501+0000 7f01ed400640 1 -- 172.21.15.71:0/1073904268 shutdown_connections 2024-09-16T07:25:44.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.501+0000 7f01ed400640 1 --2- 172.21.15.71:0/1073904268 >> v2:172.21.15.177:3300/0 conn(0x7f01e8073fb0 0x7f01e8074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.501+0000 7f01ed400640 1 --2- 172.21.15.71:0/1073904268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e815f5a0 0x7f01e80b2440 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.501+0000 7f01ed400640 1 -- 172.21.15.71:0/1073904268 >> 172.21.15.71:0/1073904268 conn(0x7f01e80b1950 msgr2=0x7f01e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:44.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.501+0000 7f01ed400640 1 -- 172.21.15.71:0/1073904268 shutdown_connections 2024-09-16T07:25:44.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.501+0000 7f01ed400640 1 -- 172.21.15.71:0/1073904268 wait complete. 2024-09-16T07:25:44.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.503+0000 7f01ed400640 1 Processor -- start 2024-09-16T07:25:44.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.503+0000 7f01ed400640 1 -- start start 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01ed400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8073fb0 0x7f01e81f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01ed400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01e80ccf00 0x7f01e814d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01ed400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01e815f5a0 0x7f01e81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01ed400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01e806f2e0 con 0x7f01e8073fb0 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01ed400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01e806f160 con 0x7f01e815f5a0 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01ed400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01e806f460 con 0x7f01e80ccf00 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8073fb0 0x7f01e81f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8073fb0 0x7f01e81f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44340/0 (socket says 172.21.15.71:44340) 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e6800640 1 -- 172.21.15.71:0/3448627088 learned_addr learned my addr 172.21.15.71:0/3448627088 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:44.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e7200640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01e815f5a0 0x7f01e81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01dfe00640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01e80ccf00 0x7f01e814d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e6800640 1 -- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01e80ccf00 msgr2=0x7f01e814d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e6800640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01e80ccf00 0x7f01e814d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e6800640 1 -- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01e815f5a0 msgr2=0x7f01e81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e6800640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01e815f5a0 0x7f01e81531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.504+0000 7f01e6800640 1 -- 172.21.15.71:0/3448627088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e814e060 con 0x7f01e8073fb0 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.505+0000 7f01e6800640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8073fb0 0x7f01e81f26c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009780 tx=0x7f01d8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.505+0000 7f01dea00640 1 -- 172.21.15.71:0/3448627088 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8031d60 con 0x7f01e8073fb0 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.505+0000 7f01dea00640 1 -- 172.21.15.71:0/3448627088 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01d8033e10 con 0x7f01e8073fb0 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.505+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e814e340 con 0x7f01e8073fb0 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.505+0000 7f01dea00640 1 -- 172.21.15.71:0/3448627088 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8005260 con 0x7f01e8073fb0 2024-09-16T07:25:44.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.505+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e814e900 con 0x7f01e8073fb0 2024-09-16T07:25:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.506+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e806f9f0 con 0x7f01e8073fb0 2024-09-16T07:25:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.507+0000 7f01dea00640 1 -- 172.21.15.71:0/3448627088 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f01d8005400 con 0x7f01e8073fb0 2024-09-16T07:25:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.508+0000 7f01dea00640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01c4078bb0 0x7f01c407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.508+0000 7f01dea00640 1 -- 172.21.15.71:0/3448627088 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f01d80bc090 con 0x7f01e8073fb0 2024-09-16T07:25:44.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.508+0000 7f01dfe00640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01c4078bb0 0x7f01c407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.509+0000 7f01dfe00640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01c4078bb0 0x7f01c407b070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f01e815eb80 tx=0x7f01d0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:44.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.512+0000 7f01dea00640 1 -- 172.21.15.71:0/3448627088 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f01d803f030 con 0x7f01e8073fb0 2024-09-16T07:25:44.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.694+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01e8154a50 con 0x7f01c4078bb0 2024-09-16T07:25:44.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.700+0000 7f01dea00640 1 -- 172.21.15.71:0/3448627088 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f01e8154a50 con 0x7f01c4078bb0 2024-09-16T07:25:44.703 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:44.704 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:25:44.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01c4078bb0 msgr2=0x7f01c407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01c4078bb0 0x7f01c407b070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f01e815eb80 tx=0x7f01d0009040 comp rx=0 tx=0).stop 2024-09-16T07:25:44.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8073fb0 msgr2=0x7f01e81f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8073fb0 0x7f01e81f26c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009780 tx=0x7f01d8033c20 comp rx=0 tx=0).stop 2024-09-16T07:25:44.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 shutdown_connections 2024-09-16T07:25:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01c4078bb0 0x7f01c407b070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01e815f5a0 0x7f01e81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01e80ccf00 0x7f01e814d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 --2- 172.21.15.71:0/3448627088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8073fb0 0x7f01e81f26c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.704+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 >> 172.21.15.71:0/3448627088 conn(0x7f01e80b1950 msgr2=0x7f01e8153f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.705+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 shutdown_connections 2024-09-16T07:25:44.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.705+0000 7f01ed400640 1 -- 172.21.15.71:0/3448627088 wait complete. 2024-09-16T07:25:44.719 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-16T07:25:44.336284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259240,"kb_used_data":5520,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715480,"statfs":{"total":773094113280,"available":772828651520,"internally_reserved":0,"allocated":5652480,"data_stored":2978524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.881579"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860419+0000","last_change":"2024-09-16T07:25:40.495566+0000","last_active":"2024-09-16T07:25:41.860419+0000","last_peered":"2024-09-16T07:25:41.860419+0000","last_clean":"2024-09-16T07:25:41.860419+0000","last_became_active":"2024-09-16T07:25:40.495279+0000","last_became_peered":"2024-09-16T07:25:40.495279+0000","last_unstale":"2024-09-16T07:25:41.860419+0000","last_undegraded":"2024-09-16T07:25:41.860419+0000","last_fullsized":"2024-09-16T07:25:41.860419+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:33.587362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859699+0000","last_change":"2024-09-16T07:25:40.488257+0000","last_active":"2024-09-16T07:25:41.859699+0000","last_peered":"2024-09-16T07:25:41.859699+0000","last_clean":"2024-09-16T07:25:41.859699+0000","last_became_active":"2024-09-16T07:25:40.487904+0000","last_became_peered":"2024-09-16T07:25:40.487904+0000","last_unstale":"2024-09-16T07:25:41.859699+0000","last_undegraded":"2024-09-16T07:25:41.859699+0000","last_fullsized":"2024-09-16T07:25:41.859699+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:07.902365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756584+0000","last_change":"2024-09-16T07:25:40.475079+0000","last_active":"2024-09-16T07:25:41.756584+0000","last_peered":"2024-09-16T07:25:41.756584+0000","last_clean":"2024-09-16T07:25:41.756584+0000","last_became_active":"2024-09-16T07:25:40.474811+0000","last_became_peered":"2024-09-16T07:25:40.474811+0000","last_unstale":"2024-09-16T07:25:41.756584+0000","last_undegraded":"2024-09-16T07:25:41.756584+0000","last_fullsized":"2024-09-16T07:25:41.756584+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:30.933339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491827+0000","last_change":"2024-09-16T07:25:40.487446+0000","last_active":"2024-09-16T07:25:41.491827+0000","last_peered":"2024-09-16T07:25:41.491827+0000","last_clean":"2024-09-16T07:25:41.491827+0000","last_became_active":"2024-09-16T07:25:40.487101+0000","last_became_peered":"2024-09-16T07:25:40.487101+0000","last_unstale":"2024-09-16T07:25:41.491827+0000","last_undegraded":"2024-09-16T07:25:41.491827+0000","last_fullsized":"2024-09-16T07:25:41.491827+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:50.977999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860231+0000","last_change":"2024-09-16T07:25:40.491063+0000","last_active":"2024-09-16T07:25:41.860231+0000","last_peered":"2024-09-16T07:25:41.860231+0000","last_clean":"2024-09-16T07:25:41.860231+0000","last_became_active":"2024-09-16T07:25:40.486241+0000","last_became_peered":"2024-09-16T07:25:40.486241+0000","last_unstale":"2024-09-16T07:25:41.860231+0000","last_undegraded":"2024-09-16T07:25:41.860231+0000","last_fullsized":"2024-09-16T07:25:41.860231+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:30.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860339+0000","last_change":"2024-09-16T07:25:40.493698+0000","last_active":"2024-09-16T07:25:41.860339+0000","last_peered":"2024-09-16T07:25:41.860339+0000","last_clean":"2024-09-16T07:25:41.860339+0000","last_became_active":"2024-09-16T07:25:40.493402+0000","last_became_peered":"2024-09-16T07:25:40.493402+0000","last_unstale":"2024-09-16T07:25:41.860339+0000","last_undegraded":"2024-09-16T07:25:41.860339+0000","last_fullsized":"2024-09-16T07:25:41.860339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:06.601739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481801+0000","last_change":"2024-09-16T07:25:40.488556+0000","last_active":"2024-09-16T07:25:41.481801+0000","last_peered":"2024-09-16T07:25:41.481801+0000","last_clean":"2024-09-16T07:25:41.481801+0000","last_became_active":"2024-09-16T07:25:40.488013+0000","last_became_peered":"2024-09-16T07:25:40.488013+0000","last_unstale":"2024-09-16T07:25:41.481801+0000","last_undegraded":"2024-09-16T07:25:41.481801+0000","last_fullsized":"2024-09-16T07:25:41.481801+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:49.291729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491506+0000","last_change":"2024-09-16T07:25:40.484006+0000","last_active":"2024-09-16T07:25:41.491506+0000","last_peered":"2024-09-16T07:25:41.491506+0000","last_clean":"2024-09-16T07:25:41.491506+0000","last_became_active":"2024-09-16T07:25:40.483193+0000","last_became_peered":"2024-09-16T07:25:40.483193+0000","last_unstale":"2024-09-16T07:25:41.491506+0000","last_undegraded":"2024-09-16T07:25:41.491506+0000","last_fullsized":"2024-09-16T07:25:41.491506+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:35.115044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860551+0000","last_change":"2024-09-16T07:25:40.494031+0000","last_active":"2024-09-16T07:25:41.860551+0000","last_peered":"2024-09-16T07:25:41.860551+0000","last_clean":"2024-09-16T07:25:41.860551+0000","last_became_active":"2024-09-16T07:25:40.493614+0000","last_became_peered":"2024-09-16T07:25:40.493614+0000","last_unstale":"2024-09-16T07:25:41.860551+0000","last_undegraded":"2024-09-16T07:25:41.860551+0000","last_fullsized":"2024-09-16T07:25:41.860551+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:06.173373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.490157+0000","last_change":"2024-09-16T07:25:40.484557+0000","last_active":"2024-09-16T07:25:41.490157+0000","last_peered":"2024-09-16T07:25:41.490157+0000","last_clean":"2024-09-16T07:25:41.490157+0000","last_became_active":"2024-09-16T07:25:40.481774+0000","last_became_peered":"2024-09-16T07:25:40.481774+0000","last_unstale":"2024-09-16T07:25:41.490157+0000","last_undegraded":"2024-09-16T07:25:41.490157+0000","last_fullsized":"2024-09-16T07:25:41.490157+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:44.296303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.492153+0000","last_change":"2024-09-16T07:25:40.487455+0000","last_active":"2024-09-16T07:25:41.492153+0000","last_peered":"2024-09-16T07:25:41.492153+0000","last_clean":"2024-09-16T07:25:41.492153+0000","last_became_active":"2024-09-16T07:25:40.487039+0000","last_became_peered":"2024-09-16T07:25:40.487039+0000","last_unstale":"2024-09-16T07:25:41.492153+0000","last_undegraded":"2024-09-16T07:25:41.492153+0000","last_fullsized":"2024-09-16T07:25:41.492153+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:29.161707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859262+0000","last_change":"2024-09-16T07:25:40.480528+0000","last_active":"2024-09-16T07:25:41.859262+0000","last_peered":"2024-09-16T07:25:41.859262+0000","last_clean":"2024-09-16T07:25:41.859262+0000","last_became_active":"2024-09-16T07:25:40.480249+0000","last_became_peered":"2024-09-16T07:25:40.480249+0000","last_unstale":"2024-09-16T07:25:41.859262+0000","last_undegraded":"2024-09-16T07:25:41.859262+0000","last_fullsized":"2024-09-16T07:25:41.859262+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:41:07.040537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860308+0000","last_change":"2024-09-16T07:25:40.491540+0000","last_active":"2024-09-16T07:25:41.860308+0000","last_peered":"2024-09-16T07:25:41.860308+0000","last_clean":"2024-09-16T07:25:41.860308+0000","last_became_active":"2024-09-16T07:25:40.491154+0000","last_became_peered":"2024-09-16T07:25:40.491154+0000","last_unstale":"2024-09-16T07:25:41.860308+0000","last_undegraded":"2024-09-16T07:25:41.860308+0000","last_fullsized":"2024-09-16T07:25:41.860308+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:17.218319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491590+0000","last_change":"2024-09-16T07:25:40.482281+0000","last_active":"2024-09-16T07:25:41.491590+0000","last_peered":"2024-09-16T07:25:41.491590+0000","last_clean":"2024-09-16T07:25:41.491590+0000","last_became_active":"2024-09-16T07:25:40.478298+0000","last_became_peered":"2024-09-16T07:25:40.478298+0000","last_unstale":"2024-09-16T07:25:41.491590+0000","last_undegraded":"2024-09-16T07:25:41.491590+0000","last_fullsized":"2024-09-16T07:25:41.491590+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:16.806061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491557+0000","last_change":"2024-09-16T07:25:40.489894+0000","last_active":"2024-09-16T07:25:41.491557+0000","last_peered":"2024-09-16T07:25:41.491557+0000","last_clean":"2024-09-16T07:25:41.491557+0000","last_became_active":"2024-09-16T07:25:40.489645+0000","last_became_peered":"2024-09-16T07:25:40.489645+0000","last_unstale":"2024-09-16T07:25:41.491557+0000","last_undegraded":"2024-09-16T07:25:41.491557+0000","last_fullsized":"2024-09-16T07:25:41.491557+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:59.517687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859529+0000","last_change":"2024-09-16T07:25:40.494128+0000","last_active":"2024-09-16T07:25:41.859529+0000","last_peered":"2024-09-16T07:25:41.859529+0000","last_clean":"2024-09-16T07:25:41.859529+0000","last_became_active":"2024-09-16T07:25:40.491538+0000","last_became_peered":"2024-09-16T07:25:40.491538+0000","last_unstale":"2024-09-16T07:25:41.859529+0000","last_undegraded":"2024-09-16T07:25:41.859529+0000","last_fullsized":"2024-09-16T07:25:41.859529+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:01.665049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859685+0000","last_change":"2024-09-16T07:25:40.485202+0000","last_active":"2024-09-16T07:25:41.859685+0000","last_peered":"2024-09-16T07:25:41.859685+0000","last_clean":"2024-09-16T07:25:41.859685+0000","last_became_active":"2024-09-16T07:25:40.484946+0000","last_became_peered":"2024-09-16T07:25:40.484946+0000","last_unstale":"2024-09-16T07:25:41.859685+0000","last_undegraded":"2024-09-16T07:25:41.859685+0000","last_fullsized":"2024-09-16T07:25:41.859685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:33.587362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756551+0000","last_change":"2024-09-16T07:25:40.487069+0000","last_active":"2024-09-16T07:25:41.756551+0000","last_peered":"2024-09-16T07:25:41.756551+0000","last_clean":"2024-09-16T07:25:41.756551+0000","last_became_active":"2024-09-16T07:25:40.486710+0000","last_became_peered":"2024-09-16T07:25:40.486710+0000","last_unstale":"2024-09-16T07:25:41.756551+0000","last_undegraded":"2024-09-16T07:25:41.756551+0000","last_fullsized":"2024-09-16T07:25:41.756551+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:20.114615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860303+0000","last_change":"2024-09-16T07:25:40.490170+0000","last_active":"2024-09-16T07:25:41.860303+0000","last_peered":"2024-09-16T07:25:41.860303+0000","last_clean":"2024-09-16T07:25:41.860303+0000","last_became_active":"2024-09-16T07:25:40.488820+0000","last_became_peered":"2024-09-16T07:25:40.488820+0000","last_unstale":"2024-09-16T07:25:41.860303+0000","last_undegraded":"2024-09-16T07:25:41.860303+0000","last_fullsized":"2024-09-16T07:25:41.860303+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:43.113303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489903+0000","last_change":"2024-09-16T07:25:40.484899+0000","last_active":"2024-09-16T07:25:41.489903+0000","last_peered":"2024-09-16T07:25:41.489903+0000","last_clean":"2024-09-16T07:25:41.489903+0000","last_became_active":"2024-09-16T07:25:40.484309+0000","last_became_peered":"2024-09-16T07:25:40.484309+0000","last_unstale":"2024-09-16T07:25:41.489903+0000","last_undegraded":"2024-09-16T07:25:41.489903+0000","last_fullsized":"2024-09-16T07:25:41.489903+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:02:18.015079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491896+0000","last_change":"2024-09-16T07:25:40.497488+0000","last_active":"2024-09-16T07:25:41.491896+0000","last_peered":"2024-09-16T07:25:41.491896+0000","last_clean":"2024-09-16T07:25:41.491896+0000","last_became_active":"2024-09-16T07:25:40.496238+0000","last_became_peered":"2024-09-16T07:25:40.496238+0000","last_unstale":"2024-09-16T07:25:41.491896+0000","last_undegraded":"2024-09-16T07:25:41.491896+0000","last_fullsized":"2024-09-16T07:25:41.491896+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:58.652817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.487646+0000","last_change":"2024-09-16T07:25:40.486029+0000","last_active":"2024-09-16T07:25:41.487646+0000","last_peered":"2024-09-16T07:25:41.487646+0000","last_clean":"2024-09-16T07:25:41.487646+0000","last_became_active":"2024-09-16T07:25:40.485754+0000","last_became_peered":"2024-09-16T07:25:40.485754+0000","last_unstale":"2024-09-16T07:25:41.487646+0000","last_undegraded":"2024-09-16T07:25:41.487646+0000","last_fullsized":"2024-09-16T07:25:41.487646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:39:01.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482085+0000","last_change":"2024-09-16T07:25:40.484665+0000","last_active":"2024-09-16T07:25:41.482085+0000","last_peered":"2024-09-16T07:25:41.482085+0000","last_clean":"2024-09-16T07:25:41.482085+0000","last_became_active":"2024-09-16T07:25:40.484420+0000","last_became_peered":"2024-09-16T07:25:40.484420+0000","last_unstale":"2024-09-16T07:25:41.482085+0000","last_undegraded":"2024-09-16T07:25:41.482085+0000","last_fullsized":"2024-09-16T07:25:41.482085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:50:44.550122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859452+0000","last_change":"2024-09-16T07:25:40.487780+0000","last_active":"2024-09-16T07:25:41.859452+0000","last_peered":"2024-09-16T07:25:41.859452+0000","last_clean":"2024-09-16T07:25:41.859452+0000","last_became_active":"2024-09-16T07:25:40.487515+0000","last_became_peered":"2024-09-16T07:25:40.487515+0000","last_unstale":"2024-09-16T07:25:41.859452+0000","last_undegraded":"2024-09-16T07:25:41.859452+0000","last_fullsized":"2024-09-16T07:25:41.859452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:21.609647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860434+0000","last_change":"2024-09-16T07:25:40.489633+0000","last_active":"2024-09-16T07:25:41.860434+0000","last_peered":"2024-09-16T07:25:41.860434+0000","last_clean":"2024-09-16T07:25:41.860434+0000","last_became_active":"2024-09-16T07:25:40.489378+0000","last_became_peered":"2024-09-16T07:25:40.489378+0000","last_unstale":"2024-09-16T07:25:41.860434+0000","last_undegraded":"2024-09-16T07:25:41.860434+0000","last_fullsized":"2024-09-16T07:25:41.860434+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:07.902365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.492261+0000","last_change":"2024-09-16T07:25:40.481397+0000","last_active":"2024-09-16T07:25:41.492261+0000","last_peered":"2024-09-16T07:25:41.492261+0000","last_clean":"2024-09-16T07:25:41.492261+0000","last_became_active":"2024-09-16T07:25:40.481067+0000","last_became_peered":"2024-09-16T07:25:40.481067+0000","last_unstale":"2024-09-16T07:25:41.492261+0000","last_undegraded":"2024-09-16T07:25:41.492261+0000","last_fullsized":"2024-09-16T07:25:41.492261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:17.111828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482077+0000","last_change":"2024-09-16T07:25:40.481231+0000","last_active":"2024-09-16T07:25:41.482077+0000","last_peered":"2024-09-16T07:25:41.482077+0000","last_clean":"2024-09-16T07:25:41.482077+0000","last_became_active":"2024-09-16T07:25:40.480949+0000","last_became_peered":"2024-09-16T07:25:40.480949+0000","last_unstale":"2024-09-16T07:25:41.482077+0000","last_undegraded":"2024-09-16T07:25:41.482077+0000","last_fullsized":"2024-09-16T07:25:41.482077+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:16.097342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482130+0000","last_change":"2024-09-16T07:25:40.479586+0000","last_active":"2024-09-16T07:25:41.482130+0000","last_peered":"2024-09-16T07:25:41.482130+0000","last_clean":"2024-09-16T07:25:41.482130+0000","last_became_active":"2024-09-16T07:25:40.479342+0000","last_became_peered":"2024-09-16T07:25:40.479342+0000","last_unstale":"2024-09-16T07:25:41.482130+0000","last_undegraded":"2024-09-16T07:25:41.482130+0000","last_fullsized":"2024-09-16T07:25:41.482130+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:49:43.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860322+0000","last_change":"2024-09-16T07:25:40.487105+0000","last_active":"2024-09-16T07:25:41.860322+0000","last_peered":"2024-09-16T07:25:41.860322+0000","last_clean":"2024-09-16T07:25:41.860322+0000","last_became_active":"2024-09-16T07:25:40.486852+0000","last_became_peered":"2024-09-16T07:25:40.486852+0000","last_unstale":"2024-09-16T07:25:41.860322+0000","last_undegraded":"2024-09-16T07:25:41.860322+0000","last_fullsized":"2024-09-16T07:25:41.860322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:35.339845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859183+0000","last_change":"2024-09-16T07:25:40.475324+0000","last_active":"2024-09-16T07:25:41.859183+0000","last_peered":"2024-09-16T07:25:41.859183+0000","last_clean":"2024-09-16T07:25:41.859183+0000","last_became_active":"2024-09-16T07:25:40.475067+0000","last_became_peered":"2024-09-16T07:25:40.475067+0000","last_unstale":"2024-09-16T07:25:41.859183+0000","last_undegraded":"2024-09-16T07:25:41.859183+0000","last_fullsized":"2024-09-16T07:25:41.859183+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:54.528096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860129+0000","last_change":"2024-09-16T07:25:40.488113+0000","last_active":"2024-09-16T07:25:41.860129+0000","last_peered":"2024-09-16T07:25:41.860129+0000","last_clean":"2024-09-16T07:25:41.860129+0000","last_became_active":"2024-09-16T07:25:40.487014+0000","last_became_peered":"2024-09-16T07:25:40.487014+0000","last_unstale":"2024-09-16T07:25:41.860129+0000","last_undegraded":"2024-09-16T07:25:41.860129+0000","last_fullsized":"2024-09-16T07:25:41.860129+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:01.414849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491659+0000","last_change":"2024-09-16T07:25:40.484455+0000","last_active":"2024-09-16T07:25:41.491659+0000","last_peered":"2024-09-16T07:25:41.491659+0000","last_clean":"2024-09-16T07:25:41.491659+0000","last_became_active":"2024-09-16T07:25:40.483683+0000","last_became_peered":"2024-09-16T07:25:40.483683+0000","last_unstale":"2024-09-16T07:25:41.491659+0000","last_undegraded":"2024-09-16T07:25:41.491659+0000","last_fullsized":"2024-09-16T07:25:41.491659+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:14.760336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488468+0000","last_change":"2024-09-16T07:25:40.488370+0000","last_active":"2024-09-16T07:25:41.488468+0000","last_peered":"2024-09-16T07:25:41.488468+0000","last_clean":"2024-09-16T07:25:41.488468+0000","last_became_active":"2024-09-16T07:25:40.488065+0000","last_became_peered":"2024-09-16T07:25:40.488065+0000","last_unstale":"2024-09-16T07:25:41.488468+0000","last_undegraded":"2024-09-16T07:25:41.488468+0000","last_fullsized":"2024-09-16T07:25:41.488468+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:30.933339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860504+0000","last_change":"2024-09-16T07:25:39.487079+0000","last_active":"2024-09-16T07:25:41.860504+0000","last_peered":"2024-09-16T07:25:41.860504+0000","last_clean":"2024-09-16T07:25:41.860504+0000","last_became_active":"2024-09-16T07:25:39.486746+0000","last_became_peered":"2024-09-16T07:25:39.486746+0000","last_unstale":"2024-09-16T07:25:41.860504+0000","last_undegraded":"2024-09-16T07:25:41.860504+0000","last_fullsized":"2024-09-16T07:25:41.860504+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:13.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860524+0000","last_change":"2024-09-16T07:25:40.483782+0000","last_active":"2024-09-16T07:25:41.860524+0000","last_peered":"2024-09-16T07:25:41.860524+0000","last_clean":"2024-09-16T07:25:41.860524+0000","last_became_active":"2024-09-16T07:25:40.483529+0000","last_became_peered":"2024-09-16T07:25:40.483529+0000","last_unstale":"2024-09-16T07:25:41.860524+0000","last_undegraded":"2024-09-16T07:25:41.860524+0000","last_fullsized":"2024-09-16T07:25:41.860524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:01.741045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482000+0000","last_change":"2024-09-16T07:25:39.475469+0000","last_active":"2024-09-16T07:25:41.482000+0000","last_peered":"2024-09-16T07:25:41.482000+0000","last_clean":"2024-09-16T07:25:41.482000+0000","last_became_active":"2024-09-16T07:25:39.474850+0000","last_became_peered":"2024-09-16T07:25:39.474850+0000","last_unstale":"2024-09-16T07:25:41.482000+0000","last_undegraded":"2024-09-16T07:25:41.482000+0000","last_fullsized":"2024-09-16T07:25:41.482000+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:14.885355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859354+0000","last_change":"2024-09-16T07:25:40.477709+0000","last_active":"2024-09-16T07:25:41.859354+0000","last_peered":"2024-09-16T07:25:41.859354+0000","last_clean":"2024-09-16T07:25:41.859354+0000","last_became_active":"2024-09-16T07:25:40.477468+0000","last_became_peered":"2024-09-16T07:25:40.477468+0000","last_unstale":"2024-09-16T07:25:41.859354+0000","last_undegraded":"2024-09-16T07:25:41.859354+0000","last_fullsized":"2024-09-16T07:25:41.859354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:01.665049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860480+0000","last_change":"2024-09-16T07:25:39.486193+0000","last_active":"2024-09-16T07:25:41.860480+0000","last_peered":"2024-09-16T07:25:41.860480+0000","last_clean":"2024-09-16T07:25:41.860480+0000","last_became_active":"2024-09-16T07:25:39.485907+0000","last_became_peered":"2024-09-16T07:25:39.485907+0000","last_unstale":"2024-09-16T07:25:41.860480+0000","last_undegraded":"2024-09-16T07:25:41.860480+0000","last_fullsized":"2024-09-16T07:25:41.860480+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:03.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.492140+0000","last_change":"2024-09-16T07:25:40.483247+0000","last_active":"2024-09-16T07:25:41.492140+0000","last_peered":"2024-09-16T07:25:41.492140+0000","last_clean":"2024-09-16T07:25:41.492140+0000","last_became_active":"2024-09-16T07:25:40.482916+0000","last_became_peered":"2024-09-16T07:25:40.482916+0000","last_unstale":"2024-09-16T07:25:41.492140+0000","last_undegraded":"2024-09-16T07:25:41.492140+0000","last_fullsized":"2024-09-16T07:25:41.492140+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:05.334204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482084+0000","last_change":"2024-09-16T07:25:39.476503+0000","last_active":"2024-09-16T07:25:41.482084+0000","last_peered":"2024-09-16T07:25:41.482084+0000","last_clean":"2024-09-16T07:25:41.482084+0000","last_became_active":"2024-09-16T07:25:39.475778+0000","last_became_peered":"2024-09-16T07:25:39.475778+0000","last_unstale":"2024-09-16T07:25:41.482084+0000","last_undegraded":"2024-09-16T07:25:41.482084+0000","last_fullsized":"2024-09-16T07:25:41.482084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:10.664491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481922+0000","last_change":"2024-09-16T07:25:40.483238+0000","last_active":"2024-09-16T07:25:41.481922+0000","last_peered":"2024-09-16T07:25:41.481922+0000","last_clean":"2024-09-16T07:25:41.481922+0000","last_became_active":"2024-09-16T07:25:40.481175+0000","last_became_peered":"2024-09-16T07:25:40.481175+0000","last_unstale":"2024-09-16T07:25:41.481922+0000","last_undegraded":"2024-09-16T07:25:41.481922+0000","last_fullsized":"2024-09-16T07:25:41.481922+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:40.233508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860517+0000","last_change":"2024-09-16T07:25:39.476280+0000","last_active":"2024-09-16T07:25:41.860517+0000","last_peered":"2024-09-16T07:25:41.860517+0000","last_clean":"2024-09-16T07:25:41.860517+0000","last_became_active":"2024-09-16T07:25:39.474941+0000","last_became_peered":"2024-09-16T07:25:39.474941+0000","last_unstale":"2024-09-16T07:25:41.860517+0000","last_undegraded":"2024-09-16T07:25:41.860517+0000","last_fullsized":"2024-09-16T07:25:41.860517+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:57.896707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489768+0000","last_change":"2024-09-16T07:25:40.491634+0000","last_active":"2024-09-16T07:25:41.489768+0000","last_peered":"2024-09-16T07:25:41.489768+0000","last_clean":"2024-09-16T07:25:41.489768+0000","last_became_active":"2024-09-16T07:25:40.490215+0000","last_became_peered":"2024-09-16T07:25:40.490215+0000","last_unstale":"2024-09-16T07:25:41.489768+0000","last_undegraded":"2024-09-16T07:25:41.489768+0000","last_fullsized":"2024-09-16T07:25:41.489768+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:55.418756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481981+0000","last_change":"2024-09-16T07:25:39.488231+0000","last_active":"2024-09-16T07:25:41.481981+0000","last_peered":"2024-09-16T07:25:41.481981+0000","last_clean":"2024-09-16T07:25:41.481981+0000","last_became_active":"2024-09-16T07:25:39.487929+0000","last_became_peered":"2024-09-16T07:25:39.487929+0000","last_unstale":"2024-09-16T07:25:41.481981+0000","last_undegraded":"2024-09-16T07:25:41.481981+0000","last_fullsized":"2024-09-16T07:25:41.481981+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:43:10.963901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489119+0000","last_change":"2024-09-16T07:25:40.475133+0000","last_active":"2024-09-16T07:25:41.489119+0000","last_peered":"2024-09-16T07:25:41.489119+0000","last_clean":"2024-09-16T07:25:41.489119+0000","last_became_active":"2024-09-16T07:25:40.474861+0000","last_became_peered":"2024-09-16T07:25:40.474861+0000","last_unstale":"2024-09-16T07:25:41.489119+0000","last_undegraded":"2024-09-16T07:25:41.489119+0000","last_fullsized":"2024-09-16T07:25:41.489119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:02.928953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859958+0000","last_change":"2024-09-16T07:25:39.481277+0000","last_active":"2024-09-16T07:25:41.859958+0000","last_peered":"2024-09-16T07:25:41.859958+0000","last_clean":"2024-09-16T07:25:41.859958+0000","last_became_active":"2024-09-16T07:25:39.480797+0000","last_became_peered":"2024-09-16T07:25:39.480797+0000","last_unstale":"2024-09-16T07:25:41.859958+0000","last_undegraded":"2024-09-16T07:25:41.859958+0000","last_fullsized":"2024-09-16T07:25:41.859958+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:52.068667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481831+0000","last_change":"2024-09-16T07:25:40.488267+0000","last_active":"2024-09-16T07:25:41.481831+0000","last_peered":"2024-09-16T07:25:41.481831+0000","last_clean":"2024-09-16T07:25:41.481831+0000","last_became_active":"2024-09-16T07:25:40.487649+0000","last_became_peered":"2024-09-16T07:25:40.487649+0000","last_unstale":"2024-09-16T07:25:41.481831+0000","last_undegraded":"2024-09-16T07:25:41.481831+0000","last_fullsized":"2024-09-16T07:25:41.481831+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:12.211195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489281+0000","last_change":"2024-09-16T07:25:39.477719+0000","last_active":"2024-09-16T07:25:41.489281+0000","last_peered":"2024-09-16T07:25:41.489281+0000","last_clean":"2024-09-16T07:25:41.489281+0000","last_became_active":"2024-09-16T07:25:39.477410+0000","last_became_peered":"2024-09-16T07:25:39.477410+0000","last_unstale":"2024-09-16T07:25:41.489281+0000","last_undegraded":"2024-09-16T07:25:41.489281+0000","last_fullsized":"2024-09-16T07:25:41.489281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:44.532555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756646+0000","last_change":"2024-09-16T07:25:40.483029+0000","last_active":"2024-09-16T07:25:41.756646+0000","last_peered":"2024-09-16T07:25:41.756646+0000","last_clean":"2024-09-16T07:25:41.756646+0000","last_became_active":"2024-09-16T07:25:40.482765+0000","last_became_peered":"2024-09-16T07:25:40.482765+0000","last_unstale":"2024-09-16T07:25:41.756646+0000","last_undegraded":"2024-09-16T07:25:41.756646+0000","last_fullsized":"2024-09-16T07:25:41.756646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:48.620121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756454+0000","last_change":"2024-09-16T07:25:39.483146+0000","last_active":"2024-09-16T07:25:41.756454+0000","last_peered":"2024-09-16T07:25:41.756454+0000","last_clean":"2024-09-16T07:25:41.756454+0000","last_became_active":"2024-09-16T07:25:39.482831+0000","last_became_peered":"2024-09-16T07:25:39.482831+0000","last_unstale":"2024-09-16T07:25:41.756454+0000","last_undegraded":"2024-09-16T07:25:41.756454+0000","last_fullsized":"2024-09-16T07:25:41.756454+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:40.176703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756653+0000","last_change":"2024-09-16T07:25:40.482140+0000","last_active":"2024-09-16T07:25:41.756653+0000","last_peered":"2024-09-16T07:25:41.756653+0000","last_clean":"2024-09-16T07:25:41.756653+0000","last_became_active":"2024-09-16T07:25:40.481943+0000","last_became_peered":"2024-09-16T07:25:40.481943+0000","last_unstale":"2024-09-16T07:25:41.756653+0000","last_undegraded":"2024-09-16T07:25:41.756653+0000","last_fullsized":"2024-09-16T07:25:41.756653+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:27.133749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.490270+0000","last_change":"2024-09-16T07:25:39.488482+0000","last_active":"2024-09-16T07:25:41.490270+0000","last_peered":"2024-09-16T07:25:41.490270+0000","last_clean":"2024-09-16T07:25:41.490270+0000","last_became_active":"2024-09-16T07:25:39.488116+0000","last_became_peered":"2024-09-16T07:25:39.488116+0000","last_unstale":"2024-09-16T07:25:41.490270+0000","last_undegraded":"2024-09-16T07:25:41.490270+0000","last_fullsized":"2024-09-16T07:25:41.490270+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:25.838519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491963+0000","last_change":"2024-09-16T07:25:40.482648+0000","last_active":"2024-09-16T07:25:41.491963+0000","last_peered":"2024-09-16T07:25:41.491963+0000","last_clean":"2024-09-16T07:25:41.491963+0000","last_became_active":"2024-09-16T07:25:40.482361+0000","last_became_peered":"2024-09-16T07:25:40.482361+0000","last_unstale":"2024-09-16T07:25:41.491963+0000","last_undegraded":"2024-09-16T07:25:41.491963+0000","last_fullsized":"2024-09-16T07:25:41.491963+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:21.955464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859223+0000","last_change":"2024-09-16T07:25:39.480570+0000","last_active":"2024-09-16T07:25:41.859223+0000","last_peered":"2024-09-16T07:25:41.859223+0000","last_clean":"2024-09-16T07:25:41.859223+0000","last_became_active":"2024-09-16T07:25:39.480254+0000","last_became_peered":"2024-09-16T07:25:39.480254+0000","last_unstale":"2024-09-16T07:25:41.859223+0000","last_undegraded":"2024-09-16T07:25:41.859223+0000","last_fullsized":"2024-09-16T07:25:41.859223+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:13.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482293+0000","last_change":"2024-09-16T07:25:40.489893+0000","last_active":"2024-09-16T07:25:41.482293+0000","last_peered":"2024-09-16T07:25:41.482293+0000","last_clean":"2024-09-16T07:25:41.482293+0000","last_became_active":"2024-09-16T07:25:40.489600+0000","last_became_peered":"2024-09-16T07:25:40.489600+0000","last_unstale":"2024-09-16T07:25:41.482293+0000","last_undegraded":"2024-09-16T07:25:41.482293+0000","last_fullsized":"2024-09-16T07:25:41.482293+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:46.750992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":21,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.490024+0000","last_change":"2024-09-16T07:25:39.484869+0000","last_active":"2024-09-16T07:25:41.490024+0000","last_peered":"2024-09-16T07:25:41.490024+0000","last_clean":"2024-09-16T07:25:41.490024+0000","last_became_active":"2024-09-16T07:25:39.484501+0000","last_became_peered":"2024-09-16T07:25:39.484501+0000","last_unstale":"2024-09-16T07:25:41.490024+0000","last_undegraded":"2024-09-16T07:25:41.490024+0000","last_fullsized":"2024-09-16T07:25:41.490024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:17.294961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756471+0000","last_change":"2024-09-16T07:25:40.476585+0000","last_active":"2024-09-16T07:25:41.756471+0000","last_peered":"2024-09-16T07:25:41.756471+0000","last_clean":"2024-09-16T07:25:41.756471+0000","last_became_active":"2024-09-16T07:25:40.476331+0000","last_became_peered":"2024-09-16T07:25:40.476331+0000","last_unstale":"2024-09-16T07:25:41.756471+0000","last_undegraded":"2024-09-16T07:25:41.756471+0000","last_fullsized":"2024-09-16T07:25:41.756471+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:13.757741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.487959+0000","last_change":"2024-09-16T07:25:39.484679+0000","last_active":"2024-09-16T07:25:41.487959+0000","last_peered":"2024-09-16T07:25:41.487959+0000","last_clean":"2024-09-16T07:25:41.487959+0000","last_became_active":"2024-09-16T07:25:39.484355+0000","last_became_peered":"2024-09-16T07:25:39.484355+0000","last_unstale":"2024-09-16T07:25:41.487959+0000","last_undegraded":"2024-09-16T07:25:41.487959+0000","last_fullsized":"2024-09-16T07:25:41.487959+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:16:28.972948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488019+0000","last_change":"2024-09-16T07:25:40.495608+0000","last_active":"2024-09-16T07:25:41.488019+0000","last_peered":"2024-09-16T07:25:41.488019+0000","last_clean":"2024-09-16T07:25:41.488019+0000","last_became_active":"2024-09-16T07:25:40.495294+0000","last_became_peered":"2024-09-16T07:25:40.495294+0000","last_unstale":"2024-09-16T07:25:41.488019+0000","last_undegraded":"2024-09-16T07:25:41.488019+0000","last_fullsized":"2024-09-16T07:25:41.488019+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:13.757741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860044+0000","last_change":"2024-09-16T07:25:39.483943+0000","last_active":"2024-09-16T07:25:41.860044+0000","last_peered":"2024-09-16T07:25:41.860044+0000","last_clean":"2024-09-16T07:25:41.860044+0000","last_became_active":"2024-09-16T07:25:39.483627+0000","last_became_peered":"2024-09-16T07:25:39.483627+0000","last_unstale":"2024-09-16T07:25:41.860044+0000","last_undegraded":"2024-09-16T07:25:41.860044+0000","last_fullsized":"2024-09-16T07:25:41.860044+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:49:08.008044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.487736+0000","last_change":"2024-09-16T07:25:40.488314+0000","last_active":"2024-09-16T07:25:41.487736+0000","last_peered":"2024-09-16T07:25:41.487736+0000","last_clean":"2024-09-16T07:25:41.487736+0000","last_became_active":"2024-09-16T07:25:40.488037+0000","last_became_peered":"2024-09-16T07:25:40.488037+0000","last_unstale":"2024-09-16T07:25:41.487736+0000","last_undegraded":"2024-09-16T07:25:41.487736+0000","last_fullsized":"2024-09-16T07:25:41.487736+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:26.023510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859986+0000","last_change":"2024-09-16T07:25:39.485838+0000","last_active":"2024-09-16T07:25:41.859986+0000","last_peered":"2024-09-16T07:25:41.859986+0000","last_clean":"2024-09-16T07:25:41.859986+0000","last_became_active":"2024-09-16T07:25:39.485534+0000","last_became_peered":"2024-09-16T07:25:39.485534+0000","last_unstale":"2024-09-16T07:25:41.859986+0000","last_undegraded":"2024-09-16T07:25:41.859986+0000","last_fullsized":"2024-09-16T07:25:41.859986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:34.536339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859450+0000","last_change":"2024-09-16T07:25:40.482807+0000","last_active":"2024-09-16T07:25:41.859450+0000","last_peered":"2024-09-16T07:25:41.859450+0000","last_clean":"2024-09-16T07:25:41.859450+0000","last_became_active":"2024-09-16T07:25:40.482529+0000","last_became_peered":"2024-09-16T07:25:40.482529+0000","last_unstale":"2024-09-16T07:25:41.859450+0000","last_undegraded":"2024-09-16T07:25:41.859450+0000","last_fullsized":"2024-09-16T07:25:41.859450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:30.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":23,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489024+0000","last_change":"2024-09-16T07:25:39.477719+0000","last_active":"2024-09-16T07:25:41.489024+0000","last_peered":"2024-09-16T07:25:41.489024+0000","last_clean":"2024-09-16T07:25:41.489024+0000","last_became_active":"2024-09-16T07:25:39.477400+0000","last_became_peered":"2024-09-16T07:25:39.477400+0000","last_unstale":"2024-09-16T07:25:41.489024+0000","last_undegraded":"2024-09-16T07:25:41.489024+0000","last_fullsized":"2024-09-16T07:25:41.489024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:58.457283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488483+0000","last_change":"2024-09-16T07:25:40.492290+0000","last_active":"2024-09-16T07:25:41.488483+0000","last_peered":"2024-09-16T07:25:41.488483+0000","last_clean":"2024-09-16T07:25:41.488483+0000","last_became_active":"2024-09-16T07:25:40.491991+0000","last_became_peered":"2024-09-16T07:25:40.491991+0000","last_unstale":"2024-09-16T07:25:41.488483+0000","last_undegraded":"2024-09-16T07:25:41.488483+0000","last_fullsized":"2024-09-16T07:25:41.488483+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:35.546186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491908+0000","last_change":"2024-09-16T07:25:39.475171+0000","last_active":"2024-09-16T07:25:41.491908+0000","last_peered":"2024-09-16T07:25:41.491908+0000","last_clean":"2024-09-16T07:25:41.491908+0000","last_became_active":"2024-09-16T07:25:39.474907+0000","last_became_peered":"2024-09-16T07:25:39.474907+0000","last_unstale":"2024-09-16T07:25:41.491908+0000","last_undegraded":"2024-09-16T07:25:41.491908+0000","last_fullsized":"2024-09-16T07:25:41.491908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:32.182081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488426+0000","last_change":"2024-09-16T07:25:40.487333+0000","last_active":"2024-09-16T07:25:41.488426+0000","last_peered":"2024-09-16T07:25:41.488426+0000","last_clean":"2024-09-16T07:25:41.488426+0000","last_became_active":"2024-09-16T07:25:40.486824+0000","last_became_peered":"2024-09-16T07:25:40.486824+0000","last_unstale":"2024-09-16T07:25:41.488426+0000","last_undegraded":"2024-09-16T07:25:41.488426+0000","last_fullsized":"2024-09-16T07:25:41.488426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:20.114615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756628+0000","last_change":"2024-09-16T07:25:39.487961+0000","last_active":"2024-09-16T07:25:41.756628+0000","last_peered":"2024-09-16T07:25:41.756628+0000","last_clean":"2024-09-16T07:25:41.756628+0000","last_became_active":"2024-09-16T07:25:39.487558+0000","last_became_peered":"2024-09-16T07:25:39.487558+0000","last_unstale":"2024-09-16T07:25:41.756628+0000","last_undegraded":"2024-09-16T07:25:41.756628+0000","last_fullsized":"2024-09-16T07:25:41.756628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:10.214643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859518+0000","last_change":"2024-09-16T07:25:40.487788+0000","last_active":"2024-09-16T07:25:41.859518+0000","last_peered":"2024-09-16T07:25:41.859518+0000","last_clean":"2024-09-16T07:25:41.859518+0000","last_became_active":"2024-09-16T07:25:40.486589+0000","last_became_peered":"2024-09-16T07:25:40.486589+0000","last_unstale":"2024-09-16T07:25:41.859518+0000","last_undegraded":"2024-09-16T07:25:41.859518+0000","last_fullsized":"2024-09-16T07:25:41.859518+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:35.339845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756653+0000","last_change":"2024-09-16T07:25:39.484643+0000","last_active":"2024-09-16T07:25:41.756653+0000","last_peered":"2024-09-16T07:25:41.756653+0000","last_clean":"2024-09-16T07:25:41.756653+0000","last_became_active":"2024-09-16T07:25:39.484462+0000","last_became_peered":"2024-09-16T07:25:39.484462+0000","last_unstale":"2024-09-16T07:25:41.756653+0000","last_undegraded":"2024-09-16T07:25:41.756653+0000","last_fullsized":"2024-09-16T07:25:41.756653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:45.983167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756633+0000","last_change":"2024-09-16T07:25:40.474807+0000","last_active":"2024-09-16T07:25:41.756633+0000","last_peered":"2024-09-16T07:25:41.756633+0000","last_clean":"2024-09-16T07:25:41.756633+0000","last_became_active":"2024-09-16T07:25:40.474541+0000","last_became_peered":"2024-09-16T07:25:40.474541+0000","last_unstale":"2024-09-16T07:25:41.756633+0000","last_undegraded":"2024-09-16T07:25:41.756633+0000","last_fullsized":"2024-09-16T07:25:41.756633+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:35.546186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860161+0000","last_change":"2024-09-16T07:25:39.487629+0000","last_active":"2024-09-16T07:25:41.860161+0000","last_peered":"2024-09-16T07:25:41.860161+0000","last_clean":"2024-09-16T07:25:41.860161+0000","last_became_active":"2024-09-16T07:25:39.487403+0000","last_became_peered":"2024-09-16T07:25:39.487403+0000","last_unstale":"2024-09-16T07:25:41.860161+0000","last_undegraded":"2024-09-16T07:25:41.860161+0000","last_fullsized":"2024-09-16T07:25:41.860161+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:45.554802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489452+0000","last_change":"2024-09-16T07:25:40.489384+0000","last_active":"2024-09-16T07:25:41.489452+0000","last_peered":"2024-09-16T07:25:41.489452+0000","last_clean":"2024-09-16T07:25:41.489452+0000","last_became_active":"2024-09-16T07:25:40.489093+0000","last_became_peered":"2024-09-16T07:25:40.489093+0000","last_unstale":"2024-09-16T07:25:41.489452+0000","last_undegraded":"2024-09-16T07:25:41.489452+0000","last_fullsized":"2024-09-16T07:25:41.489452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:34.681075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489780+0000","last_change":"2024-09-16T07:25:39.482276+0000","last_active":"2024-09-16T07:25:41.489780+0000","last_peered":"2024-09-16T07:25:41.489780+0000","last_clean":"2024-09-16T07:25:41.489780+0000","last_became_active":"2024-09-16T07:25:39.481941+0000","last_became_peered":"2024-09-16T07:25:39.481941+0000","last_unstale":"2024-09-16T07:25:41.489780+0000","last_undegraded":"2024-09-16T07:25:41.489780+0000","last_fullsized":"2024-09-16T07:25:41.489780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:07.997463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860548+0000","last_change":"2024-09-16T07:25:40.496625+0000","last_active":"2024-09-16T07:25:41.860548+0000","last_peered":"2024-09-16T07:25:41.860548+0000","last_clean":"2024-09-16T07:25:41.860548+0000","last_became_active":"2024-09-16T07:25:40.496223+0000","last_became_peered":"2024-09-16T07:25:40.496223+0000","last_unstale":"2024-09-16T07:25:41.860548+0000","last_undegraded":"2024-09-16T07:25:41.860548+0000","last_fullsized":"2024-09-16T07:25:41.860548+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:46.968839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859972+0000","last_change":"2024-09-16T07:25:39.485953+0000","last_active":"2024-09-16T07:25:41.859972+0000","last_peered":"2024-09-16T07:25:41.859972+0000","last_clean":"2024-09-16T07:25:41.859972+0000","last_became_active":"2024-09-16T07:25:39.485638+0000","last_became_peered":"2024-09-16T07:25:39.485638+0000","last_unstale":"2024-09-16T07:25:41.859972+0000","last_undegraded":"2024-09-16T07:25:41.859972+0000","last_fullsized":"2024-09-16T07:25:41.859972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:06.177260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491813+0000","last_change":"2024-09-16T07:25:40.482739+0000","last_active":"2024-09-16T07:25:41.491813+0000","last_peered":"2024-09-16T07:25:41.491813+0000","last_clean":"2024-09-16T07:25:41.491813+0000","last_became_active":"2024-09-16T07:25:40.475871+0000","last_became_peered":"2024-09-16T07:25:40.475871+0000","last_unstale":"2024-09-16T07:25:41.491813+0000","last_undegraded":"2024-09-16T07:25:41.491813+0000","last_fullsized":"2024-09-16T07:25:41.491813+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:04.018649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489253+0000","last_change":"2024-09-16T07:25:39.490345+0000","last_active":"2024-09-16T07:25:41.489253+0000","last_peered":"2024-09-16T07:25:41.489253+0000","last_clean":"2024-09-16T07:25:41.489253+0000","last_became_active":"2024-09-16T07:25:39.490012+0000","last_became_peered":"2024-09-16T07:25:39.490012+0000","last_unstale":"2024-09-16T07:25:41.489253+0000","last_undegraded":"2024-09-16T07:25:41.489253+0000","last_fullsized":"2024-09-16T07:25:41.489253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:44.255454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491840+0000","last_change":"2024-09-16T07:25:40.482424+0000","last_active":"2024-09-16T07:25:41.491840+0000","last_peered":"2024-09-16T07:25:41.491840+0000","last_clean":"2024-09-16T07:25:41.491840+0000","last_became_active":"2024-09-16T07:25:40.482140+0000","last_became_peered":"2024-09-16T07:25:40.482140+0000","last_unstale":"2024-09-16T07:25:41.491840+0000","last_undegraded":"2024-09-16T07:25:41.491840+0000","last_fullsized":"2024-09-16T07:25:41.491840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:57.758871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481650+0000","last_change":"2024-09-16T07:25:39.475101+0000","last_active":"2024-09-16T07:25:41.481650+0000","last_peered":"2024-09-16T07:25:41.481650+0000","last_clean":"2024-09-16T07:25:41.481650+0000","last_became_active":"2024-09-16T07:25:39.474794+0000","last_became_peered":"2024-09-16T07:25:39.474794+0000","last_unstale":"2024-09-16T07:25:41.481650+0000","last_undegraded":"2024-09-16T07:25:41.481650+0000","last_fullsized":"2024-09-16T07:25:41.481650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:57.043298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482243+0000","last_change":"2024-09-16T07:25:40.483146+0000","last_active":"2024-09-16T07:25:41.482243+0000","last_peered":"2024-09-16T07:25:41.482243+0000","last_clean":"2024-09-16T07:25:41.482243+0000","last_became_active":"2024-09-16T07:25:40.482882+0000","last_became_peered":"2024-09-16T07:25:40.482882+0000","last_unstale":"2024-09-16T07:25:41.482243+0000","last_undegraded":"2024-09-16T07:25:41.482243+0000","last_fullsized":"2024-09-16T07:25:41.482243+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:03:49.592147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.493902+0000","last_change":"2024-09-16T07:25:39.474412+0000","last_active":"2024-09-16T07:25:41.493902+0000","last_peered":"2024-09-16T07:25:41.493902+0000","last_clean":"2024-09-16T07:25:41.493902+0000","last_became_active":"2024-09-16T07:25:39.474136+0000","last_became_peered":"2024-09-16T07:25:39.474136+0000","last_unstale":"2024-09-16T07:25:41.493902+0000","last_undegraded":"2024-09-16T07:25:41.493902+0000","last_fullsized":"2024-09-16T07:25:41.493902+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:18.249099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489988+0000","last_change":"2024-09-16T07:25:40.484599+0000","last_active":"2024-09-16T07:25:41.489988+0000","last_peered":"2024-09-16T07:25:41.489988+0000","last_clean":"2024-09-16T07:25:41.489988+0000","last_became_active":"2024-09-16T07:25:40.483875+0000","last_became_peered":"2024-09-16T07:25:40.483875+0000","last_unstale":"2024-09-16T07:25:41.489988+0000","last_undegraded":"2024-09-16T07:25:41.489988+0000","last_fullsized":"2024-09-16T07:25:41.489988+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:21.022228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860203+0000","last_change":"2024-09-16T07:25:39.488724+0000","last_active":"2024-09-16T07:25:41.860203+0000","last_peered":"2024-09-16T07:25:41.860203+0000","last_clean":"2024-09-16T07:25:41.860203+0000","last_became_active":"2024-09-16T07:25:39.488481+0000","last_became_peered":"2024-09-16T07:25:39.488481+0000","last_unstale":"2024-09-16T07:25:41.860203+0000","last_undegraded":"2024-09-16T07:25:41.860203+0000","last_fullsized":"2024-09-16T07:25:41.860203+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:21.673436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482295+0000","last_change":"2024-09-16T07:25:40.483603+0000","last_active":"2024-09-16T07:25:41.482295+0000","last_peered":"2024-09-16T07:25:41.482295+0000","last_clean":"2024-09-16T07:25:41.482295+0000","last_became_active":"2024-09-16T07:25:40.483338+0000","last_became_peered":"2024-09-16T07:25:40.483338+0000","last_unstale":"2024-09-16T07:25:41.482295+0000","last_undegraded":"2024-09-16T07:25:41.482295+0000","last_fullsized":"2024-09-16T07:25:41.482295+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:09:05.828290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488110+0000","last_change":"2024-09-16T07:25:39.489493+0000","last_active":"2024-09-16T07:25:41.488110+0000","last_peered":"2024-09-16T07:25:41.488110+0000","last_clean":"2024-09-16T07:25:41.488110+0000","last_became_active":"2024-09-16T07:25:39.489199+0000","last_became_peered":"2024-09-16T07:25:39.489199+0000","last_unstale":"2024-09-16T07:25:41.488110+0000","last_undegraded":"2024-09-16T07:25:41.488110+0000","last_fullsized":"2024-09-16T07:25:41.488110+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:30.029402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489443+0000","last_change":"2024-09-16T07:25:40.488447+0000","last_active":"2024-09-16T07:25:41.489443+0000","last_peered":"2024-09-16T07:25:41.489443+0000","last_clean":"2024-09-16T07:25:41.489443+0000","last_became_active":"2024-09-16T07:25:40.488091+0000","last_became_peered":"2024-09-16T07:25:40.488091+0000","last_unstale":"2024-09-16T07:25:41.489443+0000","last_undegraded":"2024-09-16T07:25:41.489443+0000","last_fullsized":"2024-09-16T07:25:41.489443+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:57.299647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860291+0000","last_change":"2024-09-16T07:25:39.481074+0000","last_active":"2024-09-16T07:25:41.860291+0000","last_peered":"2024-09-16T07:25:41.860291+0000","last_clean":"2024-09-16T07:25:41.860291+0000","last_became_active":"2024-09-16T07:25:39.480669+0000","last_became_peered":"2024-09-16T07:25:39.480669+0000","last_unstale":"2024-09-16T07:25:41.860291+0000","last_undegraded":"2024-09-16T07:25:41.860291+0000","last_fullsized":"2024-09-16T07:25:41.860291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:57.182409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491545+0000","last_change":"2024-09-16T07:25:40.496598+0000","last_active":"2024-09-16T07:25:41.491545+0000","last_peered":"2024-09-16T07:25:41.491545+0000","last_clean":"2024-09-16T07:25:41.491545+0000","last_became_active":"2024-09-16T07:25:40.496243+0000","last_became_peered":"2024-09-16T07:25:40.496243+0000","last_unstale":"2024-09-16T07:25:41.491545+0000","last_undegraded":"2024-09-16T07:25:41.491545+0000","last_fullsized":"2024-09-16T07:25:41.491545+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:07.409111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481994+0000","last_change":"2024-09-16T07:25:39.478723+0000","last_active":"2024-09-16T07:25:41.481994+0000","last_peered":"2024-09-16T07:25:41.481994+0000","last_clean":"2024-09-16T07:25:41.481994+0000","last_became_active":"2024-09-16T07:25:39.478455+0000","last_became_peered":"2024-09-16T07:25:39.478455+0000","last_unstale":"2024-09-16T07:25:41.481994+0000","last_undegraded":"2024-09-16T07:25:41.481994+0000","last_fullsized":"2024-09-16T07:25:41.481994+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:42.758473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756289+0000","last_change":"2024-09-16T07:25:40.487502+0000","last_active":"2024-09-16T07:25:41.756289+0000","last_peered":"2024-09-16T07:25:41.756289+0000","last_clean":"2024-09-16T07:25:41.756289+0000","last_became_active":"2024-09-16T07:25:40.487296+0000","last_became_peered":"2024-09-16T07:25:40.487296+0000","last_unstale":"2024-09-16T07:25:41.756289+0000","last_undegraded":"2024-09-16T07:25:41.756289+0000","last_fullsized":"2024-09-16T07:25:41.756289+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:26.023510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860084+0000","last_change":"2024-09-16T07:25:39.480660+0000","last_active":"2024-09-16T07:25:41.860084+0000","last_peered":"2024-09-16T07:25:41.860084+0000","last_clean":"2024-09-16T07:25:41.860084+0000","last_became_active":"2024-09-16T07:25:39.480069+0000","last_became_peered":"2024-09-16T07:25:39.480069+0000","last_unstale":"2024-09-16T07:25:41.860084+0000","last_undegraded":"2024-09-16T07:25:41.860084+0000","last_fullsized":"2024-09-16T07:25:41.860084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:49.345120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859209+0000","last_change":"2024-09-16T07:25:40.479796+0000","last_active":"2024-09-16T07:25:41.859209+0000","last_peered":"2024-09-16T07:25:41.859209+0000","last_clean":"2024-09-16T07:25:41.859209+0000","last_became_active":"2024-09-16T07:25:40.479530+0000","last_became_peered":"2024-09-16T07:25:40.479530+0000","last_unstale":"2024-09-16T07:25:41.859209+0000","last_undegraded":"2024-09-16T07:25:41.859209+0000","last_fullsized":"2024-09-16T07:25:41.859209+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:13.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.487757+0000","last_change":"2024-09-16T07:25:39.484122+0000","last_active":"2024-09-16T07:25:41.487757+0000","last_peered":"2024-09-16T07:25:41.487757+0000","last_clean":"2024-09-16T07:25:41.487757+0000","last_became_active":"2024-09-16T07:25:39.483874+0000","last_became_peered":"2024-09-16T07:25:39.483874+0000","last_unstale":"2024-09-16T07:25:41.487757+0000","last_undegraded":"2024-09-16T07:25:41.487757+0000","last_fullsized":"2024-09-16T07:25:41.487757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:23.667815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.492316+0000","last_change":"2024-09-16T07:25:40.487148+0000","last_active":"2024-09-16T07:25:41.492316+0000","last_peered":"2024-09-16T07:25:41.492316+0000","last_clean":"2024-09-16T07:25:41.492316+0000","last_became_active":"2024-09-16T07:25:40.486858+0000","last_became_peered":"2024-09-16T07:25:40.486858+0000","last_unstale":"2024-09-16T07:25:41.492316+0000","last_undegraded":"2024-09-16T07:25:41.492316+0000","last_fullsized":"2024-09-16T07:25:41.492316+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:16:42.941546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482201+0000","last_change":"2024-09-16T07:25:39.476976+0000","last_active":"2024-09-16T07:25:41.482201+0000","last_peered":"2024-09-16T07:25:41.482201+0000","last_clean":"2024-09-16T07:25:41.482201+0000","last_became_active":"2024-09-16T07:25:39.476605+0000","last_became_peered":"2024-09-16T07:25:39.476605+0000","last_unstale":"2024-09-16T07:25:41.482201+0000","last_undegraded":"2024-09-16T07:25:41.482201+0000","last_fullsized":"2024-09-16T07:25:41.482201+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:07.560167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":157,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756192+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:25:41.756192+0000","last_peered":"2024-09-16T07:25:41.756192+0000","last_clean":"2024-09-16T07:25:41.756192+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:25:41.756192+0000","last_undegraded":"2024-09-16T07:25:41.756192+0000","last_fullsized":"2024-09-16T07:25:41.756192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476778,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":565248,"data_stored":227308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476778,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":557056,"data_stored":225786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476778,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1150976,"data_stored":816134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476777,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":561152,"data_stored":226368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":557056,"data_stored":225730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":548864,"data_stored":220012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1150976,"data_stored":816086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":561152,"data_stored":221100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:25:44.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:25:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.917+0000 7ff36d200640 1 -- 172.21.15.71:0/1592963399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 msgr2=0x7ff368074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.917+0000 7ff36d200640 1 --2- 172.21.15.71:0/1592963399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 0x7ff368074420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff3680cfbb0 tx=0x7ff36002aa30 comp rx=0 tx=0).stop 2024-09-16T07:25:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.917+0000 7ff36d200640 1 -- 172.21.15.71:0/1592963399 shutdown_connections 2024-09-16T07:25:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.917+0000 7ff36d200640 1 --2- 172.21.15.71:0/1592963399 >> v2:172.21.15.177:3300/0 conn(0x7ff3680ccf00 0x7ff3680b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.917+0000 7ff36d200640 1 --2- 172.21.15.71:0/1592963399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 0x7ff368074420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.917+0000 7ff36d200640 1 -- 172.21.15.71:0/1592963399 >> 172.21.15.71:0/1592963399 conn(0x7ff3680b1950 msgr2=0x7ff3680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.917+0000 7ff36d200640 1 -- 172.21.15.71:0/1592963399 shutdown_connections 2024-09-16T07:25:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.917+0000 7ff36d200640 1 -- 172.21.15.71:0/1592963399 wait complete. 2024-09-16T07:25:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.919+0000 7ff36d200640 1 Processor -- start 2024-09-16T07:25:44.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.919+0000 7ff36d200640 1 -- start start 2024-09-16T07:25:44.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff36d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 0x7ff3680b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff36d200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3680ccf00 0x7ff3681f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff36d200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36815f5a0 0x7ff3681f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff36d200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff36806f2e0 con 0x7ff368073fb0 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff36d200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff36806f160 con 0x7ff3680ccf00 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff36d200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff36806f460 con 0x7ff36815f5a0 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 0x7ff3680b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 0x7ff3680b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52000/0 (socket says 172.21.15.71:52000) 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365e00640 1 -- 172.21.15.71:0/1648871795 learned_addr learned my addr 172.21.15.71:0/1648871795 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff366800640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36815f5a0 0x7ff3681f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365400640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3680ccf00 0x7ff3681f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365e00640 1 -- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36815f5a0 msgr2=0x7ff3681f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365e00640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36815f5a0 0x7ff3681f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365e00640 1 -- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3680ccf00 msgr2=0x7ff3681f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:44.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365e00640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3680ccf00 0x7ff3681f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:44.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365e00640 1 -- 172.21.15.71:0/1648871795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3680b4170 con 0x7ff368073fb0 2024-09-16T07:25:44.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.920+0000 7ff365400640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3680ccf00 0x7ff3681f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:44.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.921+0000 7ff365e00640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 0x7ff3680b3c30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff35800c3a0 tx=0x7ff35800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:44.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.921+0000 7ff35e000640 1 -- 172.21.15.71:0/1648871795 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358017070 con 0x7ff368073fb0 2024-09-16T07:25:44.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.921+0000 7ff35e000640 1 -- 172.21.15.71:0/1648871795 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3580053c0 con 0x7ff368073fb0 2024-09-16T07:25:44.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.921+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3680c9150 con 0x7ff368073fb0 2024-09-16T07:25:44.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.921+0000 7ff35e000640 1 -- 172.21.15.71:0/1648871795 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358003850 con 0x7ff368073fb0 2024-09-16T07:25:44.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.921+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3680c9710 con 0x7ff368073fb0 2024-09-16T07:25:44.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.922+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3680b1480 con 0x7ff368073fb0 2024-09-16T07:25:44.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.923+0000 7ff35e000640 1 -- 172.21.15.71:0/1648871795 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7ff35801a070 con 0x7ff368073fb0 2024-09-16T07:25:44.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.924+0000 7ff35e000640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff348078bb0 0x7ff34807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:44.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.924+0000 7ff35e000640 1 -- 172.21.15.71:0/1648871795 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7ff35809a260 con 0x7ff368073fb0 2024-09-16T07:25:44.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.925+0000 7ff365400640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff348078bb0 0x7ff34807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:44.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.925+0000 7ff365400640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff348078bb0 0x7ff34807b070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff36809d7e0 tx=0x7ff354005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:44.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:44.928+0000 7ff35e000640 1 -- 172.21.15.71:0/1648871795 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff358062440 con 0x7ff368073fb0 2024-09-16T07:25:45.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.109+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff36806f9f0 con 0x7ff348078bb0 2024-09-16T07:25:45.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.114+0000 7ff35e000640 1 -- 172.21.15.71:0/1648871795 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7ff36806f9f0 con 0x7ff348078bb0 2024-09-16T07:25:45.116 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:45.118 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:25:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff348078bb0 msgr2=0x7ff34807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff348078bb0 0x7ff34807b070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff36809d7e0 tx=0x7ff354005f90 comp rx=0 tx=0).stop 2024-09-16T07:25:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 msgr2=0x7ff3680b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 0x7ff3680b3c30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff35800c3a0 tx=0x7ff35800c860 comp rx=0 tx=0).stop 2024-09-16T07:25:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 shutdown_connections 2024-09-16T07:25:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff348078bb0 0x7ff34807b070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36815f5a0 0x7ff3681f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3680ccf00 0x7ff3681f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 --2- 172.21.15.71:0/1648871795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff368073fb0 0x7ff3680b3c30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 >> 172.21.15.71:0/1648871795 conn(0x7ff3680b1950 msgr2=0x7ff3680cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:45.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 shutdown_connections 2024-09-16T07:25:45.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.118+0000 7ff36d200640 1 -- 172.21.15.71:0/1648871795 wait complete. 2024-09-16T07:25:45.134 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-16T07:25:44.336284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259240,"kb_used_data":5520,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715480,"statfs":{"total":773094113280,"available":772828651520,"internally_reserved":0,"allocated":5652480,"data_stored":2978524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.881579"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860419+0000","last_change":"2024-09-16T07:25:40.495566+0000","last_active":"2024-09-16T07:25:41.860419+0000","last_peered":"2024-09-16T07:25:41.860419+0000","last_clean":"2024-09-16T07:25:41.860419+0000","last_became_active":"2024-09-16T07:25:40.495279+0000","last_became_peered":"2024-09-16T07:25:40.495279+0000","last_unstale":"2024-09-16T07:25:41.860419+0000","last_undegraded":"2024-09-16T07:25:41.860419+0000","last_fullsized":"2024-09-16T07:25:41.860419+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:33.587362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859699+0000","last_change":"2024-09-16T07:25:40.488257+0000","last_active":"2024-09-16T07:25:41.859699+0000","last_peered":"2024-09-16T07:25:41.859699+0000","last_clean":"2024-09-16T07:25:41.859699+0000","last_became_active":"2024-09-16T07:25:40.487904+0000","last_became_peered":"2024-09-16T07:25:40.487904+0000","last_unstale":"2024-09-16T07:25:41.859699+0000","last_undegraded":"2024-09-16T07:25:41.859699+0000","last_fullsized":"2024-09-16T07:25:41.859699+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:07.902365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756584+0000","last_change":"2024-09-16T07:25:40.475079+0000","last_active":"2024-09-16T07:25:41.756584+0000","last_peered":"2024-09-16T07:25:41.756584+0000","last_clean":"2024-09-16T07:25:41.756584+0000","last_became_active":"2024-09-16T07:25:40.474811+0000","last_became_peered":"2024-09-16T07:25:40.474811+0000","last_unstale":"2024-09-16T07:25:41.756584+0000","last_undegraded":"2024-09-16T07:25:41.756584+0000","last_fullsized":"2024-09-16T07:25:41.756584+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:30.933339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491827+0000","last_change":"2024-09-16T07:25:40.487446+0000","last_active":"2024-09-16T07:25:41.491827+0000","last_peered":"2024-09-16T07:25:41.491827+0000","last_clean":"2024-09-16T07:25:41.491827+0000","last_became_active":"2024-09-16T07:25:40.487101+0000","last_became_peered":"2024-09-16T07:25:40.487101+0000","last_unstale":"2024-09-16T07:25:41.491827+0000","last_undegraded":"2024-09-16T07:25:41.491827+0000","last_fullsized":"2024-09-16T07:25:41.491827+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:50.977999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860231+0000","last_change":"2024-09-16T07:25:40.491063+0000","last_active":"2024-09-16T07:25:41.860231+0000","last_peered":"2024-09-16T07:25:41.860231+0000","last_clean":"2024-09-16T07:25:41.860231+0000","last_became_active":"2024-09-16T07:25:40.486241+0000","last_became_peered":"2024-09-16T07:25:40.486241+0000","last_unstale":"2024-09-16T07:25:41.860231+0000","last_undegraded":"2024-09-16T07:25:41.860231+0000","last_fullsized":"2024-09-16T07:25:41.860231+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:30.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860339+0000","last_change":"2024-09-16T07:25:40.493698+0000","last_active":"2024-09-16T07:25:41.860339+0000","last_peered":"2024-09-16T07:25:41.860339+0000","last_clean":"2024-09-16T07:25:41.860339+0000","last_became_active":"2024-09-16T07:25:40.493402+0000","last_became_peered":"2024-09-16T07:25:40.493402+0000","last_unstale":"2024-09-16T07:25:41.860339+0000","last_undegraded":"2024-09-16T07:25:41.860339+0000","last_fullsized":"2024-09-16T07:25:41.860339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:06.601739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481801+0000","last_change":"2024-09-16T07:25:40.488556+0000","last_active":"2024-09-16T07:25:41.481801+0000","last_peered":"2024-09-16T07:25:41.481801+0000","last_clean":"2024-09-16T07:25:41.481801+0000","last_became_active":"2024-09-16T07:25:40.488013+0000","last_became_peered":"2024-09-16T07:25:40.488013+0000","last_unstale":"2024-09-16T07:25:41.481801+0000","last_undegraded":"2024-09-16T07:25:41.481801+0000","last_fullsized":"2024-09-16T07:25:41.481801+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:49.291729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491506+0000","last_change":"2024-09-16T07:25:40.484006+0000","last_active":"2024-09-16T07:25:41.491506+0000","last_peered":"2024-09-16T07:25:41.491506+0000","last_clean":"2024-09-16T07:25:41.491506+0000","last_became_active":"2024-09-16T07:25:40.483193+0000","last_became_peered":"2024-09-16T07:25:40.483193+0000","last_unstale":"2024-09-16T07:25:41.491506+0000","last_undegraded":"2024-09-16T07:25:41.491506+0000","last_fullsized":"2024-09-16T07:25:41.491506+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:35.115044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860551+0000","last_change":"2024-09-16T07:25:40.494031+0000","last_active":"2024-09-16T07:25:41.860551+0000","last_peered":"2024-09-16T07:25:41.860551+0000","last_clean":"2024-09-16T07:25:41.860551+0000","last_became_active":"2024-09-16T07:25:40.493614+0000","last_became_peered":"2024-09-16T07:25:40.493614+0000","last_unstale":"2024-09-16T07:25:41.860551+0000","last_undegraded":"2024-09-16T07:25:41.860551+0000","last_fullsized":"2024-09-16T07:25:41.860551+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:06.173373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.490157+0000","last_change":"2024-09-16T07:25:40.484557+0000","last_active":"2024-09-16T07:25:41.490157+0000","last_peered":"2024-09-16T07:25:41.490157+0000","last_clean":"2024-09-16T07:25:41.490157+0000","last_became_active":"2024-09-16T07:25:40.481774+0000","last_became_peered":"2024-09-16T07:25:40.481774+0000","last_unstale":"2024-09-16T07:25:41.490157+0000","last_undegraded":"2024-09-16T07:25:41.490157+0000","last_fullsized":"2024-09-16T07:25:41.490157+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:44.296303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.492153+0000","last_change":"2024-09-16T07:25:40.487455+0000","last_active":"2024-09-16T07:25:41.492153+0000","last_peered":"2024-09-16T07:25:41.492153+0000","last_clean":"2024-09-16T07:25:41.492153+0000","last_became_active":"2024-09-16T07:25:40.487039+0000","last_became_peered":"2024-09-16T07:25:40.487039+0000","last_unstale":"2024-09-16T07:25:41.492153+0000","last_undegraded":"2024-09-16T07:25:41.492153+0000","last_fullsized":"2024-09-16T07:25:41.492153+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:29.161707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859262+0000","last_change":"2024-09-16T07:25:40.480528+0000","last_active":"2024-09-16T07:25:41.859262+0000","last_peered":"2024-09-16T07:25:41.859262+0000","last_clean":"2024-09-16T07:25:41.859262+0000","last_became_active":"2024-09-16T07:25:40.480249+0000","last_became_peered":"2024-09-16T07:25:40.480249+0000","last_unstale":"2024-09-16T07:25:41.859262+0000","last_undegraded":"2024-09-16T07:25:41.859262+0000","last_fullsized":"2024-09-16T07:25:41.859262+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:41:07.040537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860308+0000","last_change":"2024-09-16T07:25:40.491540+0000","last_active":"2024-09-16T07:25:41.860308+0000","last_peered":"2024-09-16T07:25:41.860308+0000","last_clean":"2024-09-16T07:25:41.860308+0000","last_became_active":"2024-09-16T07:25:40.491154+0000","last_became_peered":"2024-09-16T07:25:40.491154+0000","last_unstale":"2024-09-16T07:25:41.860308+0000","last_undegraded":"2024-09-16T07:25:41.860308+0000","last_fullsized":"2024-09-16T07:25:41.860308+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:17.218319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491590+0000","last_change":"2024-09-16T07:25:40.482281+0000","last_active":"2024-09-16T07:25:41.491590+0000","last_peered":"2024-09-16T07:25:41.491590+0000","last_clean":"2024-09-16T07:25:41.491590+0000","last_became_active":"2024-09-16T07:25:40.478298+0000","last_became_peered":"2024-09-16T07:25:40.478298+0000","last_unstale":"2024-09-16T07:25:41.491590+0000","last_undegraded":"2024-09-16T07:25:41.491590+0000","last_fullsized":"2024-09-16T07:25:41.491590+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:16.806061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491557+0000","last_change":"2024-09-16T07:25:40.489894+0000","last_active":"2024-09-16T07:25:41.491557+0000","last_peered":"2024-09-16T07:25:41.491557+0000","last_clean":"2024-09-16T07:25:41.491557+0000","last_became_active":"2024-09-16T07:25:40.489645+0000","last_became_peered":"2024-09-16T07:25:40.489645+0000","last_unstale":"2024-09-16T07:25:41.491557+0000","last_undegraded":"2024-09-16T07:25:41.491557+0000","last_fullsized":"2024-09-16T07:25:41.491557+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:59.517687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859529+0000","last_change":"2024-09-16T07:25:40.494128+0000","last_active":"2024-09-16T07:25:41.859529+0000","last_peered":"2024-09-16T07:25:41.859529+0000","last_clean":"2024-09-16T07:25:41.859529+0000","last_became_active":"2024-09-16T07:25:40.491538+0000","last_became_peered":"2024-09-16T07:25:40.491538+0000","last_unstale":"2024-09-16T07:25:41.859529+0000","last_undegraded":"2024-09-16T07:25:41.859529+0000","last_fullsized":"2024-09-16T07:25:41.859529+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:01.665049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859685+0000","last_change":"2024-09-16T07:25:40.485202+0000","last_active":"2024-09-16T07:25:41.859685+0000","last_peered":"2024-09-16T07:25:41.859685+0000","last_clean":"2024-09-16T07:25:41.859685+0000","last_became_active":"2024-09-16T07:25:40.484946+0000","last_became_peered":"2024-09-16T07:25:40.484946+0000","last_unstale":"2024-09-16T07:25:41.859685+0000","last_undegraded":"2024-09-16T07:25:41.859685+0000","last_fullsized":"2024-09-16T07:25:41.859685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:33.587362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756551+0000","last_change":"2024-09-16T07:25:40.487069+0000","last_active":"2024-09-16T07:25:41.756551+0000","last_peered":"2024-09-16T07:25:41.756551+0000","last_clean":"2024-09-16T07:25:41.756551+0000","last_became_active":"2024-09-16T07:25:40.486710+0000","last_became_peered":"2024-09-16T07:25:40.486710+0000","last_unstale":"2024-09-16T07:25:41.756551+0000","last_undegraded":"2024-09-16T07:25:41.756551+0000","last_fullsized":"2024-09-16T07:25:41.756551+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:20.114615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860303+0000","last_change":"2024-09-16T07:25:40.490170+0000","last_active":"2024-09-16T07:25:41.860303+0000","last_peered":"2024-09-16T07:25:41.860303+0000","last_clean":"2024-09-16T07:25:41.860303+0000","last_became_active":"2024-09-16T07:25:40.488820+0000","last_became_peered":"2024-09-16T07:25:40.488820+0000","last_unstale":"2024-09-16T07:25:41.860303+0000","last_undegraded":"2024-09-16T07:25:41.860303+0000","last_fullsized":"2024-09-16T07:25:41.860303+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:43.113303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489903+0000","last_change":"2024-09-16T07:25:40.484899+0000","last_active":"2024-09-16T07:25:41.489903+0000","last_peered":"2024-09-16T07:25:41.489903+0000","last_clean":"2024-09-16T07:25:41.489903+0000","last_became_active":"2024-09-16T07:25:40.484309+0000","last_became_peered":"2024-09-16T07:25:40.484309+0000","last_unstale":"2024-09-16T07:25:41.489903+0000","last_undegraded":"2024-09-16T07:25:41.489903+0000","last_fullsized":"2024-09-16T07:25:41.489903+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:02:18.015079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491896+0000","last_change":"2024-09-16T07:25:40.497488+0000","last_active":"2024-09-16T07:25:41.491896+0000","last_peered":"2024-09-16T07:25:41.491896+0000","last_clean":"2024-09-16T07:25:41.491896+0000","last_became_active":"2024-09-16T07:25:40.496238+0000","last_became_peered":"2024-09-16T07:25:40.496238+0000","last_unstale":"2024-09-16T07:25:41.491896+0000","last_undegraded":"2024-09-16T07:25:41.491896+0000","last_fullsized":"2024-09-16T07:25:41.491896+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:58.652817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.487646+0000","last_change":"2024-09-16T07:25:40.486029+0000","last_active":"2024-09-16T07:25:41.487646+0000","last_peered":"2024-09-16T07:25:41.487646+0000","last_clean":"2024-09-16T07:25:41.487646+0000","last_became_active":"2024-09-16T07:25:40.485754+0000","last_became_peered":"2024-09-16T07:25:40.485754+0000","last_unstale":"2024-09-16T07:25:41.487646+0000","last_undegraded":"2024-09-16T07:25:41.487646+0000","last_fullsized":"2024-09-16T07:25:41.487646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:39:01.357260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482085+0000","last_change":"2024-09-16T07:25:40.484665+0000","last_active":"2024-09-16T07:25:41.482085+0000","last_peered":"2024-09-16T07:25:41.482085+0000","last_clean":"2024-09-16T07:25:41.482085+0000","last_became_active":"2024-09-16T07:25:40.484420+0000","last_became_peered":"2024-09-16T07:25:40.484420+0000","last_unstale":"2024-09-16T07:25:41.482085+0000","last_undegraded":"2024-09-16T07:25:41.482085+0000","last_fullsized":"2024-09-16T07:25:41.482085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:50:44.550122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859452+0000","last_change":"2024-09-16T07:25:40.487780+0000","last_active":"2024-09-16T07:25:41.859452+0000","last_peered":"2024-09-16T07:25:41.859452+0000","last_clean":"2024-09-16T07:25:41.859452+0000","last_became_active":"2024-09-16T07:25:40.487515+0000","last_became_peered":"2024-09-16T07:25:40.487515+0000","last_unstale":"2024-09-16T07:25:41.859452+0000","last_undegraded":"2024-09-16T07:25:41.859452+0000","last_fullsized":"2024-09-16T07:25:41.859452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:33:21.609647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860434+0000","last_change":"2024-09-16T07:25:40.489633+0000","last_active":"2024-09-16T07:25:41.860434+0000","last_peered":"2024-09-16T07:25:41.860434+0000","last_clean":"2024-09-16T07:25:41.860434+0000","last_became_active":"2024-09-16T07:25:40.489378+0000","last_became_peered":"2024-09-16T07:25:40.489378+0000","last_unstale":"2024-09-16T07:25:41.860434+0000","last_undegraded":"2024-09-16T07:25:41.860434+0000","last_fullsized":"2024-09-16T07:25:41.860434+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:07.902365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.492261+0000","last_change":"2024-09-16T07:25:40.481397+0000","last_active":"2024-09-16T07:25:41.492261+0000","last_peered":"2024-09-16T07:25:41.492261+0000","last_clean":"2024-09-16T07:25:41.492261+0000","last_became_active":"2024-09-16T07:25:40.481067+0000","last_became_peered":"2024-09-16T07:25:40.481067+0000","last_unstale":"2024-09-16T07:25:41.492261+0000","last_undegraded":"2024-09-16T07:25:41.492261+0000","last_fullsized":"2024-09-16T07:25:41.492261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:17.111828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482077+0000","last_change":"2024-09-16T07:25:40.481231+0000","last_active":"2024-09-16T07:25:41.482077+0000","last_peered":"2024-09-16T07:25:41.482077+0000","last_clean":"2024-09-16T07:25:41.482077+0000","last_became_active":"2024-09-16T07:25:40.480949+0000","last_became_peered":"2024-09-16T07:25:40.480949+0000","last_unstale":"2024-09-16T07:25:41.482077+0000","last_undegraded":"2024-09-16T07:25:41.482077+0000","last_fullsized":"2024-09-16T07:25:41.482077+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:16.097342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482130+0000","last_change":"2024-09-16T07:25:40.479586+0000","last_active":"2024-09-16T07:25:41.482130+0000","last_peered":"2024-09-16T07:25:41.482130+0000","last_clean":"2024-09-16T07:25:41.482130+0000","last_became_active":"2024-09-16T07:25:40.479342+0000","last_became_peered":"2024-09-16T07:25:40.479342+0000","last_unstale":"2024-09-16T07:25:41.482130+0000","last_undegraded":"2024-09-16T07:25:41.482130+0000","last_fullsized":"2024-09-16T07:25:41.482130+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:49:43.002414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860322+0000","last_change":"2024-09-16T07:25:40.487105+0000","last_active":"2024-09-16T07:25:41.860322+0000","last_peered":"2024-09-16T07:25:41.860322+0000","last_clean":"2024-09-16T07:25:41.860322+0000","last_became_active":"2024-09-16T07:25:40.486852+0000","last_became_peered":"2024-09-16T07:25:40.486852+0000","last_unstale":"2024-09-16T07:25:41.860322+0000","last_undegraded":"2024-09-16T07:25:41.860322+0000","last_fullsized":"2024-09-16T07:25:41.860322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:35.339845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859183+0000","last_change":"2024-09-16T07:25:40.475324+0000","last_active":"2024-09-16T07:25:41.859183+0000","last_peered":"2024-09-16T07:25:41.859183+0000","last_clean":"2024-09-16T07:25:41.859183+0000","last_became_active":"2024-09-16T07:25:40.475067+0000","last_became_peered":"2024-09-16T07:25:40.475067+0000","last_unstale":"2024-09-16T07:25:41.859183+0000","last_undegraded":"2024-09-16T07:25:41.859183+0000","last_fullsized":"2024-09-16T07:25:41.859183+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:54.528096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860129+0000","last_change":"2024-09-16T07:25:40.488113+0000","last_active":"2024-09-16T07:25:41.860129+0000","last_peered":"2024-09-16T07:25:41.860129+0000","last_clean":"2024-09-16T07:25:41.860129+0000","last_became_active":"2024-09-16T07:25:40.487014+0000","last_became_peered":"2024-09-16T07:25:40.487014+0000","last_unstale":"2024-09-16T07:25:41.860129+0000","last_undegraded":"2024-09-16T07:25:41.860129+0000","last_fullsized":"2024-09-16T07:25:41.860129+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:01.414849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491659+0000","last_change":"2024-09-16T07:25:40.484455+0000","last_active":"2024-09-16T07:25:41.491659+0000","last_peered":"2024-09-16T07:25:41.491659+0000","last_clean":"2024-09-16T07:25:41.491659+0000","last_became_active":"2024-09-16T07:25:40.483683+0000","last_became_peered":"2024-09-16T07:25:40.483683+0000","last_unstale":"2024-09-16T07:25:41.491659+0000","last_undegraded":"2024-09-16T07:25:41.491659+0000","last_fullsized":"2024-09-16T07:25:41.491659+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:46:14.760336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488468+0000","last_change":"2024-09-16T07:25:40.488370+0000","last_active":"2024-09-16T07:25:41.488468+0000","last_peered":"2024-09-16T07:25:41.488468+0000","last_clean":"2024-09-16T07:25:41.488468+0000","last_became_active":"2024-09-16T07:25:40.488065+0000","last_became_peered":"2024-09-16T07:25:40.488065+0000","last_unstale":"2024-09-16T07:25:41.488468+0000","last_undegraded":"2024-09-16T07:25:41.488468+0000","last_fullsized":"2024-09-16T07:25:41.488468+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:30.933339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860504+0000","last_change":"2024-09-16T07:25:39.487079+0000","last_active":"2024-09-16T07:25:41.860504+0000","last_peered":"2024-09-16T07:25:41.860504+0000","last_clean":"2024-09-16T07:25:41.860504+0000","last_became_active":"2024-09-16T07:25:39.486746+0000","last_became_peered":"2024-09-16T07:25:39.486746+0000","last_unstale":"2024-09-16T07:25:41.860504+0000","last_undegraded":"2024-09-16T07:25:41.860504+0000","last_fullsized":"2024-09-16T07:25:41.860504+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:13.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860524+0000","last_change":"2024-09-16T07:25:40.483782+0000","last_active":"2024-09-16T07:25:41.860524+0000","last_peered":"2024-09-16T07:25:41.860524+0000","last_clean":"2024-09-16T07:25:41.860524+0000","last_became_active":"2024-09-16T07:25:40.483529+0000","last_became_peered":"2024-09-16T07:25:40.483529+0000","last_unstale":"2024-09-16T07:25:41.860524+0000","last_undegraded":"2024-09-16T07:25:41.860524+0000","last_fullsized":"2024-09-16T07:25:41.860524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:01.741045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482000+0000","last_change":"2024-09-16T07:25:39.475469+0000","last_active":"2024-09-16T07:25:41.482000+0000","last_peered":"2024-09-16T07:25:41.482000+0000","last_clean":"2024-09-16T07:25:41.482000+0000","last_became_active":"2024-09-16T07:25:39.474850+0000","last_became_peered":"2024-09-16T07:25:39.474850+0000","last_unstale":"2024-09-16T07:25:41.482000+0000","last_undegraded":"2024-09-16T07:25:41.482000+0000","last_fullsized":"2024-09-16T07:25:41.482000+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:14.885355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859354+0000","last_change":"2024-09-16T07:25:40.477709+0000","last_active":"2024-09-16T07:25:41.859354+0000","last_peered":"2024-09-16T07:25:41.859354+0000","last_clean":"2024-09-16T07:25:41.859354+0000","last_became_active":"2024-09-16T07:25:40.477468+0000","last_became_peered":"2024-09-16T07:25:40.477468+0000","last_unstale":"2024-09-16T07:25:41.859354+0000","last_undegraded":"2024-09-16T07:25:41.859354+0000","last_fullsized":"2024-09-16T07:25:41.859354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:01.665049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860480+0000","last_change":"2024-09-16T07:25:39.486193+0000","last_active":"2024-09-16T07:25:41.860480+0000","last_peered":"2024-09-16T07:25:41.860480+0000","last_clean":"2024-09-16T07:25:41.860480+0000","last_became_active":"2024-09-16T07:25:39.485907+0000","last_became_peered":"2024-09-16T07:25:39.485907+0000","last_unstale":"2024-09-16T07:25:41.860480+0000","last_undegraded":"2024-09-16T07:25:41.860480+0000","last_fullsized":"2024-09-16T07:25:41.860480+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:03.242708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.492140+0000","last_change":"2024-09-16T07:25:40.483247+0000","last_active":"2024-09-16T07:25:41.492140+0000","last_peered":"2024-09-16T07:25:41.492140+0000","last_clean":"2024-09-16T07:25:41.492140+0000","last_became_active":"2024-09-16T07:25:40.482916+0000","last_became_peered":"2024-09-16T07:25:40.482916+0000","last_unstale":"2024-09-16T07:25:41.492140+0000","last_undegraded":"2024-09-16T07:25:41.492140+0000","last_fullsized":"2024-09-16T07:25:41.492140+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:05.334204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482084+0000","last_change":"2024-09-16T07:25:39.476503+0000","last_active":"2024-09-16T07:25:41.482084+0000","last_peered":"2024-09-16T07:25:41.482084+0000","last_clean":"2024-09-16T07:25:41.482084+0000","last_became_active":"2024-09-16T07:25:39.475778+0000","last_became_peered":"2024-09-16T07:25:39.475778+0000","last_unstale":"2024-09-16T07:25:41.482084+0000","last_undegraded":"2024-09-16T07:25:41.482084+0000","last_fullsized":"2024-09-16T07:25:41.482084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:10.664491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481922+0000","last_change":"2024-09-16T07:25:40.483238+0000","last_active":"2024-09-16T07:25:41.481922+0000","last_peered":"2024-09-16T07:25:41.481922+0000","last_clean":"2024-09-16T07:25:41.481922+0000","last_became_active":"2024-09-16T07:25:40.481175+0000","last_became_peered":"2024-09-16T07:25:40.481175+0000","last_unstale":"2024-09-16T07:25:41.481922+0000","last_undegraded":"2024-09-16T07:25:41.481922+0000","last_fullsized":"2024-09-16T07:25:41.481922+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:21:40.233508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860517+0000","last_change":"2024-09-16T07:25:39.476280+0000","last_active":"2024-09-16T07:25:41.860517+0000","last_peered":"2024-09-16T07:25:41.860517+0000","last_clean":"2024-09-16T07:25:41.860517+0000","last_became_active":"2024-09-16T07:25:39.474941+0000","last_became_peered":"2024-09-16T07:25:39.474941+0000","last_unstale":"2024-09-16T07:25:41.860517+0000","last_undegraded":"2024-09-16T07:25:41.860517+0000","last_fullsized":"2024-09-16T07:25:41.860517+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:57.896707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489768+0000","last_change":"2024-09-16T07:25:40.491634+0000","last_active":"2024-09-16T07:25:41.489768+0000","last_peered":"2024-09-16T07:25:41.489768+0000","last_clean":"2024-09-16T07:25:41.489768+0000","last_became_active":"2024-09-16T07:25:40.490215+0000","last_became_peered":"2024-09-16T07:25:40.490215+0000","last_unstale":"2024-09-16T07:25:41.489768+0000","last_undegraded":"2024-09-16T07:25:41.489768+0000","last_fullsized":"2024-09-16T07:25:41.489768+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:55.418756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481981+0000","last_change":"2024-09-16T07:25:39.488231+0000","last_active":"2024-09-16T07:25:41.481981+0000","last_peered":"2024-09-16T07:25:41.481981+0000","last_clean":"2024-09-16T07:25:41.481981+0000","last_became_active":"2024-09-16T07:25:39.487929+0000","last_became_peered":"2024-09-16T07:25:39.487929+0000","last_unstale":"2024-09-16T07:25:41.481981+0000","last_undegraded":"2024-09-16T07:25:41.481981+0000","last_fullsized":"2024-09-16T07:25:41.481981+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:43:10.963901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489119+0000","last_change":"2024-09-16T07:25:40.475133+0000","last_active":"2024-09-16T07:25:41.489119+0000","last_peered":"2024-09-16T07:25:41.489119+0000","last_clean":"2024-09-16T07:25:41.489119+0000","last_became_active":"2024-09-16T07:25:40.474861+0000","last_became_peered":"2024-09-16T07:25:40.474861+0000","last_unstale":"2024-09-16T07:25:41.489119+0000","last_undegraded":"2024-09-16T07:25:41.489119+0000","last_fullsized":"2024-09-16T07:25:41.489119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:02.928953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859958+0000","last_change":"2024-09-16T07:25:39.481277+0000","last_active":"2024-09-16T07:25:41.859958+0000","last_peered":"2024-09-16T07:25:41.859958+0000","last_clean":"2024-09-16T07:25:41.859958+0000","last_became_active":"2024-09-16T07:25:39.480797+0000","last_became_peered":"2024-09-16T07:25:39.480797+0000","last_unstale":"2024-09-16T07:25:41.859958+0000","last_undegraded":"2024-09-16T07:25:41.859958+0000","last_fullsized":"2024-09-16T07:25:41.859958+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:52.068667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481831+0000","last_change":"2024-09-16T07:25:40.488267+0000","last_active":"2024-09-16T07:25:41.481831+0000","last_peered":"2024-09-16T07:25:41.481831+0000","last_clean":"2024-09-16T07:25:41.481831+0000","last_became_active":"2024-09-16T07:25:40.487649+0000","last_became_peered":"2024-09-16T07:25:40.487649+0000","last_unstale":"2024-09-16T07:25:41.481831+0000","last_undegraded":"2024-09-16T07:25:41.481831+0000","last_fullsized":"2024-09-16T07:25:41.481831+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:12.211195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489281+0000","last_change":"2024-09-16T07:25:39.477719+0000","last_active":"2024-09-16T07:25:41.489281+0000","last_peered":"2024-09-16T07:25:41.489281+0000","last_clean":"2024-09-16T07:25:41.489281+0000","last_became_active":"2024-09-16T07:25:39.477410+0000","last_became_peered":"2024-09-16T07:25:39.477410+0000","last_unstale":"2024-09-16T07:25:41.489281+0000","last_undegraded":"2024-09-16T07:25:41.489281+0000","last_fullsized":"2024-09-16T07:25:41.489281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:44.532555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756646+0000","last_change":"2024-09-16T07:25:40.483029+0000","last_active":"2024-09-16T07:25:41.756646+0000","last_peered":"2024-09-16T07:25:41.756646+0000","last_clean":"2024-09-16T07:25:41.756646+0000","last_became_active":"2024-09-16T07:25:40.482765+0000","last_became_peered":"2024-09-16T07:25:40.482765+0000","last_unstale":"2024-09-16T07:25:41.756646+0000","last_undegraded":"2024-09-16T07:25:41.756646+0000","last_fullsized":"2024-09-16T07:25:41.756646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:48.620121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756454+0000","last_change":"2024-09-16T07:25:39.483146+0000","last_active":"2024-09-16T07:25:41.756454+0000","last_peered":"2024-09-16T07:25:41.756454+0000","last_clean":"2024-09-16T07:25:41.756454+0000","last_became_active":"2024-09-16T07:25:39.482831+0000","last_became_peered":"2024-09-16T07:25:39.482831+0000","last_unstale":"2024-09-16T07:25:41.756454+0000","last_undegraded":"2024-09-16T07:25:41.756454+0000","last_fullsized":"2024-09-16T07:25:41.756454+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:40.176703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756653+0000","last_change":"2024-09-16T07:25:40.482140+0000","last_active":"2024-09-16T07:25:41.756653+0000","last_peered":"2024-09-16T07:25:41.756653+0000","last_clean":"2024-09-16T07:25:41.756653+0000","last_became_active":"2024-09-16T07:25:40.481943+0000","last_became_peered":"2024-09-16T07:25:40.481943+0000","last_unstale":"2024-09-16T07:25:41.756653+0000","last_undegraded":"2024-09-16T07:25:41.756653+0000","last_fullsized":"2024-09-16T07:25:41.756653+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:27.133749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.490270+0000","last_change":"2024-09-16T07:25:39.488482+0000","last_active":"2024-09-16T07:25:41.490270+0000","last_peered":"2024-09-16T07:25:41.490270+0000","last_clean":"2024-09-16T07:25:41.490270+0000","last_became_active":"2024-09-16T07:25:39.488116+0000","last_became_peered":"2024-09-16T07:25:39.488116+0000","last_unstale":"2024-09-16T07:25:41.490270+0000","last_undegraded":"2024-09-16T07:25:41.490270+0000","last_fullsized":"2024-09-16T07:25:41.490270+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:25.838519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491963+0000","last_change":"2024-09-16T07:25:40.482648+0000","last_active":"2024-09-16T07:25:41.491963+0000","last_peered":"2024-09-16T07:25:41.491963+0000","last_clean":"2024-09-16T07:25:41.491963+0000","last_became_active":"2024-09-16T07:25:40.482361+0000","last_became_peered":"2024-09-16T07:25:40.482361+0000","last_unstale":"2024-09-16T07:25:41.491963+0000","last_undegraded":"2024-09-16T07:25:41.491963+0000","last_fullsized":"2024-09-16T07:25:41.491963+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:21.955464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859223+0000","last_change":"2024-09-16T07:25:39.480570+0000","last_active":"2024-09-16T07:25:41.859223+0000","last_peered":"2024-09-16T07:25:41.859223+0000","last_clean":"2024-09-16T07:25:41.859223+0000","last_became_active":"2024-09-16T07:25:39.480254+0000","last_became_peered":"2024-09-16T07:25:39.480254+0000","last_unstale":"2024-09-16T07:25:41.859223+0000","last_undegraded":"2024-09-16T07:25:41.859223+0000","last_fullsized":"2024-09-16T07:25:41.859223+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:13.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482293+0000","last_change":"2024-09-16T07:25:40.489893+0000","last_active":"2024-09-16T07:25:41.482293+0000","last_peered":"2024-09-16T07:25:41.482293+0000","last_clean":"2024-09-16T07:25:41.482293+0000","last_became_active":"2024-09-16T07:25:40.489600+0000","last_became_peered":"2024-09-16T07:25:40.489600+0000","last_unstale":"2024-09-16T07:25:41.482293+0000","last_undegraded":"2024-09-16T07:25:41.482293+0000","last_fullsized":"2024-09-16T07:25:41.482293+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:46.750992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":21,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.490024+0000","last_change":"2024-09-16T07:25:39.484869+0000","last_active":"2024-09-16T07:25:41.490024+0000","last_peered":"2024-09-16T07:25:41.490024+0000","last_clean":"2024-09-16T07:25:41.490024+0000","last_became_active":"2024-09-16T07:25:39.484501+0000","last_became_peered":"2024-09-16T07:25:39.484501+0000","last_unstale":"2024-09-16T07:25:41.490024+0000","last_undegraded":"2024-09-16T07:25:41.490024+0000","last_fullsized":"2024-09-16T07:25:41.490024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:17.294961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756471+0000","last_change":"2024-09-16T07:25:40.476585+0000","last_active":"2024-09-16T07:25:41.756471+0000","last_peered":"2024-09-16T07:25:41.756471+0000","last_clean":"2024-09-16T07:25:41.756471+0000","last_became_active":"2024-09-16T07:25:40.476331+0000","last_became_peered":"2024-09-16T07:25:40.476331+0000","last_unstale":"2024-09-16T07:25:41.756471+0000","last_undegraded":"2024-09-16T07:25:41.756471+0000","last_fullsized":"2024-09-16T07:25:41.756471+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:13.757741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.487959+0000","last_change":"2024-09-16T07:25:39.484679+0000","last_active":"2024-09-16T07:25:41.487959+0000","last_peered":"2024-09-16T07:25:41.487959+0000","last_clean":"2024-09-16T07:25:41.487959+0000","last_became_active":"2024-09-16T07:25:39.484355+0000","last_became_peered":"2024-09-16T07:25:39.484355+0000","last_unstale":"2024-09-16T07:25:41.487959+0000","last_undegraded":"2024-09-16T07:25:41.487959+0000","last_fullsized":"2024-09-16T07:25:41.487959+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:16:28.972948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488019+0000","last_change":"2024-09-16T07:25:40.495608+0000","last_active":"2024-09-16T07:25:41.488019+0000","last_peered":"2024-09-16T07:25:41.488019+0000","last_clean":"2024-09-16T07:25:41.488019+0000","last_became_active":"2024-09-16T07:25:40.495294+0000","last_became_peered":"2024-09-16T07:25:40.495294+0000","last_unstale":"2024-09-16T07:25:41.488019+0000","last_undegraded":"2024-09-16T07:25:41.488019+0000","last_fullsized":"2024-09-16T07:25:41.488019+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:13.757741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860044+0000","last_change":"2024-09-16T07:25:39.483943+0000","last_active":"2024-09-16T07:25:41.860044+0000","last_peered":"2024-09-16T07:25:41.860044+0000","last_clean":"2024-09-16T07:25:41.860044+0000","last_became_active":"2024-09-16T07:25:39.483627+0000","last_became_peered":"2024-09-16T07:25:39.483627+0000","last_unstale":"2024-09-16T07:25:41.860044+0000","last_undegraded":"2024-09-16T07:25:41.860044+0000","last_fullsized":"2024-09-16T07:25:41.860044+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:49:08.008044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.487736+0000","last_change":"2024-09-16T07:25:40.488314+0000","last_active":"2024-09-16T07:25:41.487736+0000","last_peered":"2024-09-16T07:25:41.487736+0000","last_clean":"2024-09-16T07:25:41.487736+0000","last_became_active":"2024-09-16T07:25:40.488037+0000","last_became_peered":"2024-09-16T07:25:40.488037+0000","last_unstale":"2024-09-16T07:25:41.487736+0000","last_undegraded":"2024-09-16T07:25:41.487736+0000","last_fullsized":"2024-09-16T07:25:41.487736+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:26.023510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859986+0000","last_change":"2024-09-16T07:25:39.485838+0000","last_active":"2024-09-16T07:25:41.859986+0000","last_peered":"2024-09-16T07:25:41.859986+0000","last_clean":"2024-09-16T07:25:41.859986+0000","last_became_active":"2024-09-16T07:25:39.485534+0000","last_became_peered":"2024-09-16T07:25:39.485534+0000","last_unstale":"2024-09-16T07:25:41.859986+0000","last_undegraded":"2024-09-16T07:25:41.859986+0000","last_fullsized":"2024-09-16T07:25:41.859986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:34.536339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859450+0000","last_change":"2024-09-16T07:25:40.482807+0000","last_active":"2024-09-16T07:25:41.859450+0000","last_peered":"2024-09-16T07:25:41.859450+0000","last_clean":"2024-09-16T07:25:41.859450+0000","last_became_active":"2024-09-16T07:25:40.482529+0000","last_became_peered":"2024-09-16T07:25:40.482529+0000","last_unstale":"2024-09-16T07:25:41.859450+0000","last_undegraded":"2024-09-16T07:25:41.859450+0000","last_fullsized":"2024-09-16T07:25:41.859450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:30.426145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":23,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489024+0000","last_change":"2024-09-16T07:25:39.477719+0000","last_active":"2024-09-16T07:25:41.489024+0000","last_peered":"2024-09-16T07:25:41.489024+0000","last_clean":"2024-09-16T07:25:41.489024+0000","last_became_active":"2024-09-16T07:25:39.477400+0000","last_became_peered":"2024-09-16T07:25:39.477400+0000","last_unstale":"2024-09-16T07:25:41.489024+0000","last_undegraded":"2024-09-16T07:25:41.489024+0000","last_fullsized":"2024-09-16T07:25:41.489024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:58.457283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488483+0000","last_change":"2024-09-16T07:25:40.492290+0000","last_active":"2024-09-16T07:25:41.488483+0000","last_peered":"2024-09-16T07:25:41.488483+0000","last_clean":"2024-09-16T07:25:41.488483+0000","last_became_active":"2024-09-16T07:25:40.491991+0000","last_became_peered":"2024-09-16T07:25:40.491991+0000","last_unstale":"2024-09-16T07:25:41.488483+0000","last_undegraded":"2024-09-16T07:25:41.488483+0000","last_fullsized":"2024-09-16T07:25:41.488483+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:35.546186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491908+0000","last_change":"2024-09-16T07:25:39.475171+0000","last_active":"2024-09-16T07:25:41.491908+0000","last_peered":"2024-09-16T07:25:41.491908+0000","last_clean":"2024-09-16T07:25:41.491908+0000","last_became_active":"2024-09-16T07:25:39.474907+0000","last_became_peered":"2024-09-16T07:25:39.474907+0000","last_unstale":"2024-09-16T07:25:41.491908+0000","last_undegraded":"2024-09-16T07:25:41.491908+0000","last_fullsized":"2024-09-16T07:25:41.491908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:32.182081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488426+0000","last_change":"2024-09-16T07:25:40.487333+0000","last_active":"2024-09-16T07:25:41.488426+0000","last_peered":"2024-09-16T07:25:41.488426+0000","last_clean":"2024-09-16T07:25:41.488426+0000","last_became_active":"2024-09-16T07:25:40.486824+0000","last_became_peered":"2024-09-16T07:25:40.486824+0000","last_unstale":"2024-09-16T07:25:41.488426+0000","last_undegraded":"2024-09-16T07:25:41.488426+0000","last_fullsized":"2024-09-16T07:25:41.488426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:20.114615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756628+0000","last_change":"2024-09-16T07:25:39.487961+0000","last_active":"2024-09-16T07:25:41.756628+0000","last_peered":"2024-09-16T07:25:41.756628+0000","last_clean":"2024-09-16T07:25:41.756628+0000","last_became_active":"2024-09-16T07:25:39.487558+0000","last_became_peered":"2024-09-16T07:25:39.487558+0000","last_unstale":"2024-09-16T07:25:41.756628+0000","last_undegraded":"2024-09-16T07:25:41.756628+0000","last_fullsized":"2024-09-16T07:25:41.756628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:10.214643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859518+0000","last_change":"2024-09-16T07:25:40.487788+0000","last_active":"2024-09-16T07:25:41.859518+0000","last_peered":"2024-09-16T07:25:41.859518+0000","last_clean":"2024-09-16T07:25:41.859518+0000","last_became_active":"2024-09-16T07:25:40.486589+0000","last_became_peered":"2024-09-16T07:25:40.486589+0000","last_unstale":"2024-09-16T07:25:41.859518+0000","last_undegraded":"2024-09-16T07:25:41.859518+0000","last_fullsized":"2024-09-16T07:25:41.859518+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:35.339845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756653+0000","last_change":"2024-09-16T07:25:39.484643+0000","last_active":"2024-09-16T07:25:41.756653+0000","last_peered":"2024-09-16T07:25:41.756653+0000","last_clean":"2024-09-16T07:25:41.756653+0000","last_became_active":"2024-09-16T07:25:39.484462+0000","last_became_peered":"2024-09-16T07:25:39.484462+0000","last_unstale":"2024-09-16T07:25:41.756653+0000","last_undegraded":"2024-09-16T07:25:41.756653+0000","last_fullsized":"2024-09-16T07:25:41.756653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:45.983167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756633+0000","last_change":"2024-09-16T07:25:40.474807+0000","last_active":"2024-09-16T07:25:41.756633+0000","last_peered":"2024-09-16T07:25:41.756633+0000","last_clean":"2024-09-16T07:25:41.756633+0000","last_became_active":"2024-09-16T07:25:40.474541+0000","last_became_peered":"2024-09-16T07:25:40.474541+0000","last_unstale":"2024-09-16T07:25:41.756633+0000","last_undegraded":"2024-09-16T07:25:41.756633+0000","last_fullsized":"2024-09-16T07:25:41.756633+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:35.546186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860161+0000","last_change":"2024-09-16T07:25:39.487629+0000","last_active":"2024-09-16T07:25:41.860161+0000","last_peered":"2024-09-16T07:25:41.860161+0000","last_clean":"2024-09-16T07:25:41.860161+0000","last_became_active":"2024-09-16T07:25:39.487403+0000","last_became_peered":"2024-09-16T07:25:39.487403+0000","last_unstale":"2024-09-16T07:25:41.860161+0000","last_undegraded":"2024-09-16T07:25:41.860161+0000","last_fullsized":"2024-09-16T07:25:41.860161+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:45.554802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489452+0000","last_change":"2024-09-16T07:25:40.489384+0000","last_active":"2024-09-16T07:25:41.489452+0000","last_peered":"2024-09-16T07:25:41.489452+0000","last_clean":"2024-09-16T07:25:41.489452+0000","last_became_active":"2024-09-16T07:25:40.489093+0000","last_became_peered":"2024-09-16T07:25:40.489093+0000","last_unstale":"2024-09-16T07:25:41.489452+0000","last_undegraded":"2024-09-16T07:25:41.489452+0000","last_fullsized":"2024-09-16T07:25:41.489452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:34.681075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489780+0000","last_change":"2024-09-16T07:25:39.482276+0000","last_active":"2024-09-16T07:25:41.489780+0000","last_peered":"2024-09-16T07:25:41.489780+0000","last_clean":"2024-09-16T07:25:41.489780+0000","last_became_active":"2024-09-16T07:25:39.481941+0000","last_became_peered":"2024-09-16T07:25:39.481941+0000","last_unstale":"2024-09-16T07:25:41.489780+0000","last_undegraded":"2024-09-16T07:25:41.489780+0000","last_fullsized":"2024-09-16T07:25:41.489780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:07.997463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860548+0000","last_change":"2024-09-16T07:25:40.496625+0000","last_active":"2024-09-16T07:25:41.860548+0000","last_peered":"2024-09-16T07:25:41.860548+0000","last_clean":"2024-09-16T07:25:41.860548+0000","last_became_active":"2024-09-16T07:25:40.496223+0000","last_became_peered":"2024-09-16T07:25:40.496223+0000","last_unstale":"2024-09-16T07:25:41.860548+0000","last_undegraded":"2024-09-16T07:25:41.860548+0000","last_fullsized":"2024-09-16T07:25:41.860548+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:46.968839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859972+0000","last_change":"2024-09-16T07:25:39.485953+0000","last_active":"2024-09-16T07:25:41.859972+0000","last_peered":"2024-09-16T07:25:41.859972+0000","last_clean":"2024-09-16T07:25:41.859972+0000","last_became_active":"2024-09-16T07:25:39.485638+0000","last_became_peered":"2024-09-16T07:25:39.485638+0000","last_unstale":"2024-09-16T07:25:41.859972+0000","last_undegraded":"2024-09-16T07:25:41.859972+0000","last_fullsized":"2024-09-16T07:25:41.859972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:06.177260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491813+0000","last_change":"2024-09-16T07:25:40.482739+0000","last_active":"2024-09-16T07:25:41.491813+0000","last_peered":"2024-09-16T07:25:41.491813+0000","last_clean":"2024-09-16T07:25:41.491813+0000","last_became_active":"2024-09-16T07:25:40.475871+0000","last_became_peered":"2024-09-16T07:25:40.475871+0000","last_unstale":"2024-09-16T07:25:41.491813+0000","last_undegraded":"2024-09-16T07:25:41.491813+0000","last_fullsized":"2024-09-16T07:25:41.491813+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:04.018649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489253+0000","last_change":"2024-09-16T07:25:39.490345+0000","last_active":"2024-09-16T07:25:41.489253+0000","last_peered":"2024-09-16T07:25:41.489253+0000","last_clean":"2024-09-16T07:25:41.489253+0000","last_became_active":"2024-09-16T07:25:39.490012+0000","last_became_peered":"2024-09-16T07:25:39.490012+0000","last_unstale":"2024-09-16T07:25:41.489253+0000","last_undegraded":"2024-09-16T07:25:41.489253+0000","last_fullsized":"2024-09-16T07:25:41.489253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:44.255454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491840+0000","last_change":"2024-09-16T07:25:40.482424+0000","last_active":"2024-09-16T07:25:41.491840+0000","last_peered":"2024-09-16T07:25:41.491840+0000","last_clean":"2024-09-16T07:25:41.491840+0000","last_became_active":"2024-09-16T07:25:40.482140+0000","last_became_peered":"2024-09-16T07:25:40.482140+0000","last_unstale":"2024-09-16T07:25:41.491840+0000","last_undegraded":"2024-09-16T07:25:41.491840+0000","last_fullsized":"2024-09-16T07:25:41.491840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:57.758871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481650+0000","last_change":"2024-09-16T07:25:39.475101+0000","last_active":"2024-09-16T07:25:41.481650+0000","last_peered":"2024-09-16T07:25:41.481650+0000","last_clean":"2024-09-16T07:25:41.481650+0000","last_became_active":"2024-09-16T07:25:39.474794+0000","last_became_peered":"2024-09-16T07:25:39.474794+0000","last_unstale":"2024-09-16T07:25:41.481650+0000","last_undegraded":"2024-09-16T07:25:41.481650+0000","last_fullsized":"2024-09-16T07:25:41.481650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:57.043298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482243+0000","last_change":"2024-09-16T07:25:40.483146+0000","last_active":"2024-09-16T07:25:41.482243+0000","last_peered":"2024-09-16T07:25:41.482243+0000","last_clean":"2024-09-16T07:25:41.482243+0000","last_became_active":"2024-09-16T07:25:40.482882+0000","last_became_peered":"2024-09-16T07:25:40.482882+0000","last_unstale":"2024-09-16T07:25:41.482243+0000","last_undegraded":"2024-09-16T07:25:41.482243+0000","last_fullsized":"2024-09-16T07:25:41.482243+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:03:49.592147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.493902+0000","last_change":"2024-09-16T07:25:39.474412+0000","last_active":"2024-09-16T07:25:41.493902+0000","last_peered":"2024-09-16T07:25:41.493902+0000","last_clean":"2024-09-16T07:25:41.493902+0000","last_became_active":"2024-09-16T07:25:39.474136+0000","last_became_peered":"2024-09-16T07:25:39.474136+0000","last_unstale":"2024-09-16T07:25:41.493902+0000","last_undegraded":"2024-09-16T07:25:41.493902+0000","last_fullsized":"2024-09-16T07:25:41.493902+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:18.249099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489988+0000","last_change":"2024-09-16T07:25:40.484599+0000","last_active":"2024-09-16T07:25:41.489988+0000","last_peered":"2024-09-16T07:25:41.489988+0000","last_clean":"2024-09-16T07:25:41.489988+0000","last_became_active":"2024-09-16T07:25:40.483875+0000","last_became_peered":"2024-09-16T07:25:40.483875+0000","last_unstale":"2024-09-16T07:25:41.489988+0000","last_undegraded":"2024-09-16T07:25:41.489988+0000","last_fullsized":"2024-09-16T07:25:41.489988+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:21.022228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860203+0000","last_change":"2024-09-16T07:25:39.488724+0000","last_active":"2024-09-16T07:25:41.860203+0000","last_peered":"2024-09-16T07:25:41.860203+0000","last_clean":"2024-09-16T07:25:41.860203+0000","last_became_active":"2024-09-16T07:25:39.488481+0000","last_became_peered":"2024-09-16T07:25:39.488481+0000","last_unstale":"2024-09-16T07:25:41.860203+0000","last_undegraded":"2024-09-16T07:25:41.860203+0000","last_fullsized":"2024-09-16T07:25:41.860203+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:21.673436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482295+0000","last_change":"2024-09-16T07:25:40.483603+0000","last_active":"2024-09-16T07:25:41.482295+0000","last_peered":"2024-09-16T07:25:41.482295+0000","last_clean":"2024-09-16T07:25:41.482295+0000","last_became_active":"2024-09-16T07:25:40.483338+0000","last_became_peered":"2024-09-16T07:25:40.483338+0000","last_unstale":"2024-09-16T07:25:41.482295+0000","last_undegraded":"2024-09-16T07:25:41.482295+0000","last_fullsized":"2024-09-16T07:25:41.482295+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:09:05.828290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.488110+0000","last_change":"2024-09-16T07:25:39.489493+0000","last_active":"2024-09-16T07:25:41.488110+0000","last_peered":"2024-09-16T07:25:41.488110+0000","last_clean":"2024-09-16T07:25:41.488110+0000","last_became_active":"2024-09-16T07:25:39.489199+0000","last_became_peered":"2024-09-16T07:25:39.489199+0000","last_unstale":"2024-09-16T07:25:41.488110+0000","last_undegraded":"2024-09-16T07:25:41.488110+0000","last_fullsized":"2024-09-16T07:25:41.488110+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:30.029402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.489443+0000","last_change":"2024-09-16T07:25:40.488447+0000","last_active":"2024-09-16T07:25:41.489443+0000","last_peered":"2024-09-16T07:25:41.489443+0000","last_clean":"2024-09-16T07:25:41.489443+0000","last_became_active":"2024-09-16T07:25:40.488091+0000","last_became_peered":"2024-09-16T07:25:40.488091+0000","last_unstale":"2024-09-16T07:25:41.489443+0000","last_undegraded":"2024-09-16T07:25:41.489443+0000","last_fullsized":"2024-09-16T07:25:41.489443+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:57.299647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860291+0000","last_change":"2024-09-16T07:25:39.481074+0000","last_active":"2024-09-16T07:25:41.860291+0000","last_peered":"2024-09-16T07:25:41.860291+0000","last_clean":"2024-09-16T07:25:41.860291+0000","last_became_active":"2024-09-16T07:25:39.480669+0000","last_became_peered":"2024-09-16T07:25:39.480669+0000","last_unstale":"2024-09-16T07:25:41.860291+0000","last_undegraded":"2024-09-16T07:25:41.860291+0000","last_fullsized":"2024-09-16T07:25:41.860291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:12:57.182409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.491545+0000","last_change":"2024-09-16T07:25:40.496598+0000","last_active":"2024-09-16T07:25:41.491545+0000","last_peered":"2024-09-16T07:25:41.491545+0000","last_clean":"2024-09-16T07:25:41.491545+0000","last_became_active":"2024-09-16T07:25:40.496243+0000","last_became_peered":"2024-09-16T07:25:40.496243+0000","last_unstale":"2024-09-16T07:25:41.491545+0000","last_undegraded":"2024-09-16T07:25:41.491545+0000","last_fullsized":"2024-09-16T07:25:41.491545+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:07.409111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.481994+0000","last_change":"2024-09-16T07:25:39.478723+0000","last_active":"2024-09-16T07:25:41.481994+0000","last_peered":"2024-09-16T07:25:41.481994+0000","last_clean":"2024-09-16T07:25:41.481994+0000","last_became_active":"2024-09-16T07:25:39.478455+0000","last_became_peered":"2024-09-16T07:25:39.478455+0000","last_unstale":"2024-09-16T07:25:41.481994+0000","last_undegraded":"2024-09-16T07:25:41.481994+0000","last_fullsized":"2024-09-16T07:25:41.481994+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:42.758473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756289+0000","last_change":"2024-09-16T07:25:40.487502+0000","last_active":"2024-09-16T07:25:41.756289+0000","last_peered":"2024-09-16T07:25:41.756289+0000","last_clean":"2024-09-16T07:25:41.756289+0000","last_became_active":"2024-09-16T07:25:40.487296+0000","last_became_peered":"2024-09-16T07:25:40.487296+0000","last_unstale":"2024-09-16T07:25:41.756289+0000","last_undegraded":"2024-09-16T07:25:41.756289+0000","last_fullsized":"2024-09-16T07:25:41.756289+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:26.023510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.860084+0000","last_change":"2024-09-16T07:25:39.480660+0000","last_active":"2024-09-16T07:25:41.860084+0000","last_peered":"2024-09-16T07:25:41.860084+0000","last_clean":"2024-09-16T07:25:41.860084+0000","last_became_active":"2024-09-16T07:25:39.480069+0000","last_became_peered":"2024-09-16T07:25:39.480069+0000","last_unstale":"2024-09-16T07:25:41.860084+0000","last_undegraded":"2024-09-16T07:25:41.860084+0000","last_fullsized":"2024-09-16T07:25:41.860084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:49.345120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.859209+0000","last_change":"2024-09-16T07:25:40.479796+0000","last_active":"2024-09-16T07:25:41.859209+0000","last_peered":"2024-09-16T07:25:41.859209+0000","last_clean":"2024-09-16T07:25:41.859209+0000","last_became_active":"2024-09-16T07:25:40.479530+0000","last_became_peered":"2024-09-16T07:25:40.479530+0000","last_unstale":"2024-09-16T07:25:41.859209+0000","last_undegraded":"2024-09-16T07:25:41.859209+0000","last_fullsized":"2024-09-16T07:25:41.859209+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:13.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.487757+0000","last_change":"2024-09-16T07:25:39.484122+0000","last_active":"2024-09-16T07:25:41.487757+0000","last_peered":"2024-09-16T07:25:41.487757+0000","last_clean":"2024-09-16T07:25:41.487757+0000","last_became_active":"2024-09-16T07:25:39.483874+0000","last_became_peered":"2024-09-16T07:25:39.483874+0000","last_unstale":"2024-09-16T07:25:41.487757+0000","last_undegraded":"2024-09-16T07:25:41.487757+0000","last_fullsized":"2024-09-16T07:25:41.487757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:23.667815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.492316+0000","last_change":"2024-09-16T07:25:40.487148+0000","last_active":"2024-09-16T07:25:41.492316+0000","last_peered":"2024-09-16T07:25:41.492316+0000","last_clean":"2024-09-16T07:25:41.492316+0000","last_became_active":"2024-09-16T07:25:40.486858+0000","last_became_peered":"2024-09-16T07:25:40.486858+0000","last_unstale":"2024-09-16T07:25:41.492316+0000","last_undegraded":"2024-09-16T07:25:41.492316+0000","last_fullsized":"2024-09-16T07:25:41.492316+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:39.451685+0000","last_clean_scrub_stamp":"2024-09-16T07:25:39.451685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:16:42.941546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.482201+0000","last_change":"2024-09-16T07:25:39.476976+0000","last_active":"2024-09-16T07:25:41.482201+0000","last_peered":"2024-09-16T07:25:41.482201+0000","last_clean":"2024-09-16T07:25:41.482201+0000","last_became_active":"2024-09-16T07:25:39.476605+0000","last_became_peered":"2024-09-16T07:25:39.476605+0000","last_unstale":"2024-09-16T07:25:41.482201+0000","last_undegraded":"2024-09-16T07:25:41.482201+0000","last_fullsized":"2024-09-16T07:25:41.482201+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:25:38.441839+0000","last_clean_scrub_stamp":"2024-09-16T07:25:38.441839+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:07.560167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":157,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T07:25:41.756192+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:25:41.756192+0000","last_peered":"2024-09-16T07:25:41.756192+0000","last_clean":"2024-09-16T07:25:41.756192+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:25:41.756192+0000","last_undegraded":"2024-09-16T07:25:41.756192+0000","last_fullsized":"2024-09-16T07:25:41.756192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476778,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":565248,"data_stored":227308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476778,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":557056,"data_stored":225786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476778,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1150976,"data_stored":816134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476777,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":561152,"data_stored":226368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":557056,"data_stored":225730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":548864,"data_stored":220012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1150976,"data_stored":816086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":561152,"data_stored":221100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:25:45.138 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:25:45.138 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:25:45.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.339+0000 7f38bd600640 1 -- 172.21.15.71:0/457785630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80d00b0 msgr2=0x7f38b815fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.339+0000 7f38bd600640 1 --2- 172.21.15.71:0/457785630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80d00b0 0x7f38b815fe00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f38b8074ca0 tx=0x7f38a802a680 comp rx=0 tx=0).stop 2024-09-16T07:25:45.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.339+0000 7f38bd600640 1 -- 172.21.15.71:0/457785630 shutdown_connections 2024-09-16T07:25:45.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.339+0000 7f38bd600640 1 --2- 172.21.15.71:0/457785630 >> v2:172.21.15.177:3300/0 conn(0x7f38b80d2960 0x7f38b8160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.339+0000 7f38bd600640 1 --2- 172.21.15.71:0/457785630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80d00b0 0x7f38b815fe00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.339+0000 7f38bd600640 1 -- 172.21.15.71:0/457785630 >> 172.21.15.71:0/457785630 conn(0x7f38b80d1e40 msgr2=0x7f38b80d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:45.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.339+0000 7f38bd600640 1 -- 172.21.15.71:0/457785630 shutdown_connections 2024-09-16T07:25:45.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.339+0000 7f38bd600640 1 -- 172.21.15.71:0/457785630 wait complete. 2024-09-16T07:25:45.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.340+0000 7f38bd600640 1 Processor -- start 2024-09-16T07:25:45.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.341+0000 7f38bd600640 1 -- start start 2024-09-16T07:25:45.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.341+0000 7f38bd600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80b0b30 0x7f38b81522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.341+0000 7f38bd600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38b80d00b0 0x7f38b80c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.341+0000 7f38bd600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38b80d2960 0x7f38b81f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.341+0000 7f38bd600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f38b80b0640 con 0x7f38b80b0b30 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.341+0000 7f38bd600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f38b80d06e0 con 0x7f38b80d2960 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.341+0000 7f38bd600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f38b8160880 con 0x7f38b80d00b0 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80b0b30 0x7f38b81522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80b0b30 0x7f38b81522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52026/0 (socket says 172.21.15.71:52026) 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6a00640 1 -- 172.21.15.71:0/3916554367 learned_addr learned my addr 172.21.15.71:0/3916554367 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6000640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38b80d00b0 0x7f38b80c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b7400640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38b80d2960 0x7f38b81f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6a00640 1 -- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38b80d00b0 msgr2=0x7f38b80c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6a00640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38b80d00b0 0x7f38b80c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6a00640 1 -- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38b80d2960 msgr2=0x7f38b81f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6a00640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38b80d2960 0x7f38b81f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.342+0000 7f38b6a00640 1 -- 172.21.15.71:0/3916554367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b815cd70 con 0x7f38b80b0b30 2024-09-16T07:25:45.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.343+0000 7f38b6a00640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80b0b30 0x7f38b81522e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009850 tx=0x7f38a8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:45.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.343+0000 7f38aea00640 1 -- 172.21.15.71:0/3916554367 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a803f070 con 0x7f38b80b0b30 2024-09-16T07:25:45.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.343+0000 7f38aea00640 1 -- 172.21.15.71:0/3916554367 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38a80047a0 con 0x7f38b80b0b30 2024-09-16T07:25:45.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.343+0000 7f38aea00640 1 -- 172.21.15.71:0/3916554367 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8035030 con 0x7f38b80b0b30 2024-09-16T07:25:45.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.343+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b815d050 con 0x7f38b80b0b30 2024-09-16T07:25:45.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.343+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b81f4f80 con 0x7f38b80b0b30 2024-09-16T07:25:45.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.344+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b81f4db0 con 0x7f38b80b0b30 2024-09-16T07:25:45.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.345+0000 7f38aea00640 1 -- 172.21.15.71:0/3916554367 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f38a8042070 con 0x7f38b80b0b30 2024-09-16T07:25:45.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.346+0000 7f38aea00640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3898078be0 0x7f389807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:45.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.346+0000 7f38aea00640 1 -- 172.21.15.71:0/3916554367 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f38a80bbd20 con 0x7f38b80b0b30 2024-09-16T07:25:45.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.347+0000 7f38b6000640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3898078be0 0x7f389807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:45.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.347+0000 7f38b6000640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3898078be0 0x7f389807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38b80cf940 tx=0x7f38a400b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:45.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.350+0000 7f38aea00640 1 -- 172.21.15.71:0/3916554367 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f38a8083820 con 0x7f38b80b0b30 2024-09-16T07:25:45.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.602+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f38b815d570 con 0x7f38b80b0b30 2024-09-16T07:25:45.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.604+0000 7f38aea00640 1 -- 172.21.15.71:0/3916554367 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f38b815d570 con 0x7f38b80b0b30 2024-09-16T07:25:45.604 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:25:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:25:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:25:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:25:45.604 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:25:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:25:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3898078be0 msgr2=0x7f389807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3898078be0 0x7f389807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38b80cf940 tx=0x7f38a400b2e0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80b0b30 msgr2=0x7f38b81522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80b0b30 0x7f38b81522e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009850 tx=0x7f38a8003910 comp rx=0 tx=0).stop 2024-09-16T07:25:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 shutdown_connections 2024-09-16T07:25:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3898078be0 0x7f389807b0a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38b80d2960 0x7f38b81f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38b80d00b0 0x7f38b80c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 --2- 172.21.15.71:0/3916554367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b80b0b30 0x7f38b81522e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 >> 172.21.15.71:0/3916554367 conn(0x7f38b80d1e40 msgr2=0x7f38b80b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 shutdown_connections 2024-09-16T07:25:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.607+0000 7f38bd600640 1 -- 172.21.15.71:0/3916554367 wait complete. 2024-09-16T07:25:45.623 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:25:45.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.782+0000 7fd861c00640 1 -- 172.21.15.71:0/1455253749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c073fb0 msgr2=0x7fd85c074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.782+0000 7fd861c00640 1 --2- 172.21.15.71:0/1455253749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c073fb0 0x7fd85c074440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd854005fd0 tx=0x7fd85402a810 comp rx=0 tx=0).stop 2024-09-16T07:25:45.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.783+0000 7fd861c00640 1 -- 172.21.15.71:0/1455253749 shutdown_connections 2024-09-16T07:25:45.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.783+0000 7fd861c00640 1 --2- 172.21.15.71:0/1455253749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c073fb0 0x7fd85c074440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.783+0000 7fd861c00640 1 --2- 172.21.15.71:0/1455253749 >> v2:172.21.15.177:3300/0 conn(0x7fd85c15f5a0 0x7fd85c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.783+0000 7fd861c00640 1 -- 172.21.15.71:0/1455253749 >> 172.21.15.71:0/1455253749 conn(0x7fd85c0b1950 msgr2=0x7fd85c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:45.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.783+0000 7fd861c00640 1 -- 172.21.15.71:0/1455253749 shutdown_connections 2024-09-16T07:25:45.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.783+0000 7fd861c00640 1 -- 172.21.15.71:0/1455253749 wait complete. 2024-09-16T07:25:45.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.784+0000 7fd861c00640 1 Processor -- start 2024-09-16T07:25:45.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.784+0000 7fd861c00640 1 -- start start 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.785+0000 7fd861c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c073fb0 0x7fd85c14c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.785+0000 7fd861c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0ccf00 0x7fd85c14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.785+0000 7fd861c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f5a0 0x7fd85c0c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.785+0000 7fd861c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c06f2e0 con 0x7fd85c15f5a0 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.785+0000 7fd861c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c06f160 con 0x7fd85c073fb0 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.785+0000 7fd861c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd85c06f460 con 0x7fd85c0ccf00 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f5a0 0x7fd85c0c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f5a0 0x7fd85c0c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52042/0 (socket says 172.21.15.71:52042) 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85b400640 1 -- 172.21.15.71:0/3570101878 learned_addr learned my addr 172.21.15.71:0/3570101878 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85aa00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c073fb0 0x7fd85c14c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd853e00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0ccf00 0x7fd85c14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:45.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85b400640 1 -- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0ccf00 msgr2=0x7fd85c14ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85b400640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0ccf00 0x7fd85c14ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85b400640 1 -- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c073fb0 msgr2=0x7fd85c14c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:45.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85b400640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c073fb0 0x7fd85c14c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:45.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd85b400640 1 -- 172.21.15.71:0/3570101878 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c14f350 con 0x7fd85c15f5a0 2024-09-16T07:25:45.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.786+0000 7fd853e00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0ccf00 0x7fd85c14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:45.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.787+0000 7fd85b400640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f5a0 0x7fd85c0c5460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd854009b30 tx=0x7fd8540315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:45.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.787+0000 7fd852a00640 1 -- 172.21.15.71:0/3570101878 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854041020 con 0x7fd85c15f5a0 2024-09-16T07:25:45.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.787+0000 7fd852a00640 1 -- 172.21.15.71:0/3570101878 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd854031ac0 con 0x7fd85c15f5a0 2024-09-16T07:25:45.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.787+0000 7fd852a00640 1 -- 172.21.15.71:0/3570101878 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854037040 con 0x7fd85c15f5a0 2024-09-16T07:25:45.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.787+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c14f630 con 0x7fd85c15f5a0 2024-09-16T07:25:45.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.787+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c0c4c60 con 0x7fd85c15f5a0 2024-09-16T07:25:45.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.788+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c06f9f0 con 0x7fd85c15f5a0 2024-09-16T07:25:45.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.789+0000 7fd852a00640 1 -- 172.21.15.71:0/3570101878 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fd854044020 con 0x7fd85c15f5a0 2024-09-16T07:25:45.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.790+0000 7fd852a00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd838078bb0 0x7fd83807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:45.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.790+0000 7fd852a00640 1 -- 172.21.15.71:0/3570101878 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fd8540bb9a0 con 0x7fd85c15f5a0 2024-09-16T07:25:45.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.791+0000 7fd85aa00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd838078bb0 0x7fd83807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:45.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.792+0000 7fd85aa00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd838078bb0 0x7fd83807b070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0095f0 tx=0x7fd84c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:45.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:45.794+0000 7fd852a00640 1 -- 172.21.15.71:0/3570101878 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd85404b070 con 0x7fd85c15f5a0 2024-09-16T07:25:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.015+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd85c0ad0b0 con 0x7fd85c15f5a0 2024-09-16T07:25:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.016+0000 7fd852a00640 1 -- 172.21.15.71:0/3570101878 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fd854083b40 con 0x7fd85c15f5a0 2024-09-16T07:25:46.016 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:25:46.016 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":27,"btime":"2024-09-16T07:25:43:488308+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1366102730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1366102730},{"type":"v1","addr":"172.21.15.177:6836","nonce":1366102730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/796136206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":796136206},{"type":"v1","addr":"172.21.15.177:6837","nonce":796136206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3187236541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3187236541},{"type":"v1","addr":"172.21.15.71:6834","nonce":3187236541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:25:39.895186+0000","modified":"2024-09-16T07:25:42.903826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2966230103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2966230103},{"type":"v1","addr":"172.21.15.71:6835","nonce":2966230103}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-16T07:25:46.016 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 27 2024-09-16T07:25:46.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd838078bb0 msgr2=0x7fd83807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd838078bb0 0x7fd83807b070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0095f0 tx=0x7fd84c005f90 comp rx=0 tx=0).stop 2024-09-16T07:25:46.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f5a0 msgr2=0x7fd85c0c5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f5a0 0x7fd85c0c5460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd854009b30 tx=0x7fd8540315d0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 shutdown_connections 2024-09-16T07:25:46.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd838078bb0 0x7fd83807b070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f5a0 0x7fd85c0c5460 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0ccf00 0x7fd85c14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 --2- 172.21.15.71:0/3570101878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c073fb0 0x7fd85c14c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.018+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 >> 172.21.15.71:0/3570101878 conn(0x7fd85c0b1950 msgr2=0x7fd85c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:46.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.019+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 shutdown_connections 2024-09-16T07:25:46.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.019+0000 7fd861c00640 1 -- 172.21.15.71:0/3570101878 wait complete. 2024-09-16T07:25:46.438 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:25:39.895186+0000', 'modified': '2024-09-16T07:25:42.903826+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5433}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5433': {'gid': 5433, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/2966230103', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 2966230103}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 2966230103}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5433, 'qdb_cluster': [5433]} 2024-09-16T07:25:46.438 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:25:46.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-16T07:25:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.596+0000 7f9226800640 1 -- 172.21.15.71:0/700577832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 msgr2=0x7f92200b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.596+0000 7f9226800640 1 --2- 172.21.15.71:0/700577832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 0x7f92200b2440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9210009830 tx=0x7f921002a480 comp rx=0 tx=0).stop 2024-09-16T07:25:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.596+0000 7f9226800640 1 -- 172.21.15.71:0/700577832 shutdown_connections 2024-09-16T07:25:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.596+0000 7f9226800640 1 --2- 172.21.15.71:0/700577832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 0x7f92200b2440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.596+0000 7f9226800640 1 -- 172.21.15.71:0/700577832 >> 172.21.15.71:0/700577832 conn(0x7f92200b1950 msgr2=0x7f92200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.596+0000 7f9226800640 1 -- 172.21.15.71:0/700577832 shutdown_connections 2024-09-16T07:25:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.596+0000 7f9226800640 1 -- 172.21.15.71:0/700577832 wait complete. 2024-09-16T07:25:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.598+0000 7f9226800640 1 Processor -- start 2024-09-16T07:25:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.598+0000 7f9226800640 1 -- start start 2024-09-16T07:25:46.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f9226800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9220073fb0 0x7f92200c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f9226800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92200ccf00 0x7f9220152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f9226800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 0x7f92201530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f9226800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f922006f9f0 con 0x7f922015f5a0 2024-09-16T07:25:46.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f9226800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f922006f870 con 0x7f92200ccf00 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f9226800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9220074420 con 0x7f9220073fb0 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f921fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 0x7f92201530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f921fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 0x7f92201530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52064/0 (socket says 172.21.15.71:52064) 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f921fe00640 1 -- 172.21.15.71:0/4160590510 learned_addr learned my addr 172.21.15.71:0/4160590510 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f921f400640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9220073fb0 0x7f92200c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.599+0000 7f921ea00640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92200ccf00 0x7f9220152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f921fe00640 1 -- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9220073fb0 msgr2=0x7f92200c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f921fe00640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9220073fb0 0x7f92200c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f921fe00640 1 -- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92200ccf00 msgr2=0x7f9220152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f921fe00640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92200ccf00 0x7f9220152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f921fe00640 1 -- 172.21.15.71:0/4160590510 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92201f2a90 con 0x7f922015f5a0 2024-09-16T07:25:46.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f921fe00640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 0x7f92201530e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f921800cbc0 tx=0x7f921800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:46.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f9217400640 1 -- 172.21.15.71:0/4160590510 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218016020 con 0x7f922015f5a0 2024-09-16T07:25:46.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f9217400640 1 -- 172.21.15.71:0/4160590510 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f921800ae30 con 0x7f922015f5a0 2024-09-16T07:25:46.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f9217400640 1 -- 172.21.15.71:0/4160590510 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218012660 con 0x7f922015f5a0 2024-09-16T07:25:46.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f9226800640 1 -- 172.21.15.71:0/4160590510 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92201f2d50 con 0x7f922015f5a0 2024-09-16T07:25:46.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.600+0000 7f9226800640 1 -- 172.21.15.71:0/4160590510 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201f3310 con 0x7f922015f5a0 2024-09-16T07:25:46.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.603+0000 7f9217400640 1 -- 172.21.15.71:0/4160590510 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f9218019020 con 0x7f922015f5a0 2024-09-16T07:25:46.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.604+0000 7f9217400640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9200078bb0 0x7f920007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.604+0000 7f9217400640 1 -- 172.21.15.71:0/4160590510 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f921809b490 con 0x7f922015f5a0 2024-09-16T07:25:46.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.604+0000 7f92296a9740 1 Processor -- start 2024-09-16T07:25:46.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.604+0000 7f92296a9740 1 -- start start 2024-09-16T07:25:46.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.604+0000 7f921f400640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9200078bb0 0x7f920007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f92296a9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c428c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f921f400640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9200078bb0 0x7f920007b070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9210034000 tx=0x7f9210004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f92296a9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3e3a20 0x55ca1c427390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f92296a9740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 0x55ca1c4278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f92296a9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ca1c366430 con 0x55ca1c4288b0 2024-09-16T07:25:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f92296a9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ca1c3565d0 con 0x55ca1c3e3a20 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f92296a9740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ca1c345a00 con 0x55ca1c3e3fa0 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f921f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c428c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f921f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c428c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52076/0 (socket says 172.21.15.71:52076) 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f921f400640 1 -- 172.21.15.71:0/2738176709 learned_addr learned my addr 172.21.15.71:0/2738176709 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f921ea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3e3a20 0x55ca1c427390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.605+0000 7f921fe00640 1 --2- 172.21.15.71:0/2738176709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 0x55ca1c4278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.606+0000 7f921f400640 1 -- 172.21.15.71:0/2738176709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 msgr2=0x55ca1c4278d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.606+0000 7f921f400640 1 --2- 172.21.15.71:0/2738176709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 0x55ca1c4278d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.606+0000 7f921f400640 1 -- 172.21.15.71:0/2738176709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3e3a20 msgr2=0x55ca1c427390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.606+0000 7f921f400640 1 --2- 172.21.15.71:0/2738176709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3e3a20 0x55ca1c427390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.606+0000 7f921f400640 1 -- 172.21.15.71:0/2738176709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1c30e520 con 0x55ca1c4288b0 2024-09-16T07:25:46.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.606+0000 7f921f400640 1 --2- 172.21.15.71:0/2738176709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c428c90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9210000c00 tx=0x7f921003d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daada24a1046e31 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:46.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.608+0000 7f920be00640 1 -- 172.21.15.71:0/2738176709 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210003cf0 con 0x55ca1c4288b0 2024-09-16T07:25:46.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.608+0000 7f920be00640 1 -- 172.21.15.71:0/2738176709 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9210048420 con 0x55ca1c4288b0 2024-09-16T07:25:46.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.608+0000 7f920be00640 1 -- 172.21.15.71:0/2738176709 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210048850 con 0x55ca1c4288b0 2024-09-16T07:25:46.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.608+0000 7f92296a9740 1 -- 172.21.15.71:0/2738176709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 msgr2=0x55ca1c428c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.608+0000 7f92296a9740 1 --2- 172.21.15.71:0/2738176709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c428c90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9210000c00 tx=0x7f921003d070 comp rx=0 tx=0).stop 2024-09-16T07:25:46.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.609+0000 7f92296a9740 1 -- 172.21.15.71:0/2738176709 shutdown_connections 2024-09-16T07:25:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.609+0000 7f92296a9740 1 --2- 172.21.15.71:0/2738176709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 0x55ca1c4278d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.609+0000 7f92296a9740 1 --2- 172.21.15.71:0/2738176709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3e3a20 0x55ca1c427390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.609+0000 7f92296a9740 1 --2- 172.21.15.71:0/2738176709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c428c90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.609+0000 7f92296a9740 1 -- 172.21.15.71:0/2738176709 >> 172.21.15.71:0/2738176709 conn(0x55ca1c3deff0 msgr2=0x55ca1c3e2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.609+0000 7f92296a9740 1 -- 172.21.15.71:0/2738176709 shutdown_connections 2024-09-16T07:25:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.609+0000 7f92296a9740 1 -- 172.21.15.71:0/2738176709 wait complete. 2024-09-16T07:25:46.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.609+0000 7f92296a9740 1 -- start start 2024-09-16T07:25:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.610+0000 7f92296a9740 1 Processor -- start 2024-09-16T07:25:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f92296a9740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 0x55ca1c43ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f92296a9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c4288b0 0x55ca1c438040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f92296a9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c40d100 0x55ca1c406d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f92296a9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ca1c2ec330 con 0x55ca1c40d100 2024-09-16T07:25:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f92296a9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ca1c349810 con 0x55ca1c4288b0 2024-09-16T07:25:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f92296a9740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ca1c0b6620 con 0x55ca1c3e3fa0 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f92296a9740 10 client.0 _ll_register_callbacks cb 0x55ca1c2f1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c40d100 0x55ca1c406d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c40d100 0x55ca1c406d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52078/0 (socket says 172.21.15.71:52078) 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 -- 172.21.15.71:0/207878367 learned_addr learned my addr 172.21.15.71:0/207878367 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921f400640 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 0x55ca1c43ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921ea00640 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c4288b0 0x55ca1c438040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 -- 172.21.15.71:0/207878367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 msgr2=0x55ca1c43ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 0x55ca1c43ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 -- 172.21.15.71:0/207878367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c4288b0 msgr2=0x55ca1c438040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c4288b0 0x55ca1c438040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 -- 172.21.15.71:0/207878367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1c0a7850 con 0x55ca1c40d100 2024-09-16T07:25:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f921fe00640 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c40d100 0x55ca1c406d30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f921800cb90 tx=0x7f9218063570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfc6e2b775adabb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.611+0000 7f9208c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f9208c00640 1 -- 172.21.15.71:0/207878367 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218089020 con 0x55ca1c40d100 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f9208c00640 1 -- 172.21.15.71:0/207878367 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f92180623c0 con 0x55ca1c40d100 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 1 -- 172.21.15.71:0/207878367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9210048420 con 0x55ca1c40d100 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f9208c00640 1 -- 172.21.15.71:0/207878367 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9218067980 con 0x55ca1c40d100 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 10 client.5559 fetch_fsmap learned FSMap version 27 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 1 -- 172.21.15.71:0/207878367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55ca1c11e6b0 con 0x55ca1c40d100 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f9208c00640 1 -- 172.21.15.71:0/207878367 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9210048420 con 0x55ca1c40d100 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f9208c00640 1 -- 172.21.15.71:0/207878367 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f9218068b10 con 0x55ca1c40d100 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 10 client.5559 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 10 client.5559 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 20 client.5559 populate_metadata read hostname 'smithi071' 2024-09-16T07:25:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] conn(0x55ca1c40a400 0x55ca1c40a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 4 client.5559 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:25:46.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.612+0000 7f92296a9740 1 -- 172.21.15.71:0/207878367 --> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca1c358520 con 0x55ca1c40a400 2024-09-16T07:25:46.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.613+0000 7f921f400640 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] conn(0x55ca1c40a400 0x55ca1c40a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.613+0000 7f921f400640 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] conn(0x55ca1c40a400 0x55ca1c40a7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b4474d929d009b server_cookie=6c5ca1703964b1c5 in_seq=0 out_seq=0 2024-09-16T07:25:46.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.613+0000 7f9208c00640 10 client.5559 ms_handle_connect on v2:172.21.15.71:6832/2966230103 2024-09-16T07:25:46.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f9208c00640 1 -- 172.21.15.71:0/207878367 <== mds.0 v2:172.21.15.71:6832/2966230103 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca1c358520 con 0x55ca1c40a400 2024-09-16T07:25:46.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f9208c00640 10 client.5559 handle_command_reply: tid=0 2024-09-16T07:25:46.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 client.5559 shutdown 2024-09-16T07:25:46.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 -- 172.21.15.71:0/207878367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c40d100 msgr2=0x55ca1c406d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c40d100 0x55ca1c406d30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f921800cb90 tx=0x7f9218063570 comp rx=0 tx=0).stop 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 -- 172.21.15.71:0/207878367 shutdown_connections 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] conn(0x55ca1c40a400 0x55ca1c40a7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c40d100 0x55ca1c406d30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c4288b0 0x55ca1c438040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 --2- 172.21.15.71:0/207878367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c3e3fa0 0x55ca1c43ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f9208c00640 0 client.5559 ms_handle_reset on v2:172.21.15.71:6832/2966230103 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 -- 172.21.15.71:0/207878367 >> 172.21.15.71:0/207878367 conn(0x55ca1c3e3a20 msgr2=0x55ca1c3ddbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 -- 172.21.15.71:0/207878367 shutdown_connections 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.614+0000 7f92296a9740 1 -- 172.21.15.71:0/207878367 wait complete. 2024-09-16T07:25:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.615+0000 7f92296a9740 20 client.5559 trim_cache size 0 max 16384 2024-09-16T07:25:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.633+0000 7f92296a9740 1 Processor -- start 2024-09-16T07:25:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.633+0000 7f92296a9740 1 -- start start 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f92296a9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c3e3fa0 0x55ca1c423d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f92296a9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c40d100 0x55ca1c40c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f92296a9740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c4288b0 0x55ca1c40ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f92296a9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f92200c5e90 con 0x55ca1c3e3fa0 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f92296a9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92201f6d00 con 0x55ca1c40d100 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f92296a9740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f92200c5840 con 0x55ca1c4288b0 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f921f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c3e3fa0 0x55ca1c423d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f921f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c3e3fa0 0x55ca1c423d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52082/0 (socket says 172.21.15.71:52082) 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.634+0000 7f921f400640 1 -- 172.21.15.71:0/2020310602 learned_addr learned my addr 172.21.15.71:0/2020310602 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:46.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921fe00640 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c4288b0 0x55ca1c40ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921ea00640 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c40d100 0x55ca1c40c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921f400640 1 -- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c4288b0 msgr2=0x55ca1c40ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921f400640 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c4288b0 0x55ca1c40ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921f400640 1 -- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c40d100 msgr2=0x55ca1c40c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921f400640 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c40d100 0x55ca1c40c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921f400640 1 -- 172.21.15.71:0/2020310602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1c3de1b0 con 0x55ca1c3e3fa0 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921fe00640 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c4288b0 0x55ca1c40ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f921f400640 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c3e3fa0 0x55ca1c423d60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9210009230 tx=0x7f921004a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db620811947afcba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f920b400640 1 -- 172.21.15.71:0/2020310602 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210004320 con 0x55ca1c3e3fa0 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f920b400640 1 -- 172.21.15.71:0/2020310602 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9210003b70 con 0x55ca1c3e3fa0 2024-09-16T07:25:46.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.635+0000 7f920b400640 1 -- 172.21.15.71:0/2020310602 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210048680 con 0x55ca1c3e3fa0 2024-09-16T07:25:46.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 -- 172.21.15.71:0/2020310602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c3e3fa0 msgr2=0x55ca1c423d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c3e3fa0 0x55ca1c423d60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9210009230 tx=0x7f921004a040 comp rx=0 tx=0).stop 2024-09-16T07:25:46.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 -- 172.21.15.71:0/2020310602 shutdown_connections 2024-09-16T07:25:46.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c4288b0 0x55ca1c40ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c40d100 0x55ca1c40c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 --2- 172.21.15.71:0/2020310602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c3e3fa0 0x55ca1c423d60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 -- 172.21.15.71:0/2020310602 >> 172.21.15.71:0/2020310602 conn(0x55ca1c3e3a20 msgr2=0x55ca1c408920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 -- 172.21.15.71:0/2020310602 shutdown_connections 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.636+0000 7f92296a9740 1 -- 172.21.15.71:0/2020310602 wait complete. 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.637+0000 7f92296a9740 1 -- start start 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.637+0000 7f92296a9740 1 Processor -- start 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f92296a9740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c40d100 0x55ca1c4aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f92296a9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c44a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f92296a9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3ddb30 0x55ca1c44a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f92296a9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ca1c437bc0 con 0x55ca1c4288b0 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f92296a9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ca1c43b8a0 con 0x55ca1c3ddb30 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f92296a9740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ca1c3e12a0 con 0x55ca1c40d100 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f92296a9740 10 client.0 _ll_register_callbacks cb 0x7f921805f3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:25:46.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f921ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c44a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f921ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c44a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52096/0 (socket says 172.21.15.71:52096) 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f921ea00640 1 -- 172.21.15.71:0/3822042082 learned_addr learned my addr 172.21.15.71:0/3822042082 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f921f400640 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c40d100 0x55ca1c4aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.638+0000 7f921fe00640 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3ddb30 0x55ca1c44a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f921ea00640 1 -- 172.21.15.71:0/3822042082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c40d100 msgr2=0x55ca1c4aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f921ea00640 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c40d100 0x55ca1c4aa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f921ea00640 1 -- 172.21.15.71:0/3822042082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3ddb30 msgr2=0x55ca1c44a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f921ea00640 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3ddb30 0x55ca1c44a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f921ea00640 1 -- 172.21.15.71:0/3822042082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1c441780 con 0x55ca1c4288b0 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f921ea00640 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c44a0d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f92201521f0 tx=0x7f920c016ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77576bd13b7321b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f9208c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f9208c00640 1 -- 172.21.15.71:0/3822042082 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c007110 con 0x55ca1c4288b0 2024-09-16T07:25:46.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f9208c00640 1 -- 172.21.15.71:0/3822042082 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f920c021ae0 con 0x55ca1c4288b0 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f92296a9740 1 -- 172.21.15.71:0/3822042082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9210003b70 con 0x55ca1c4288b0 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f9208c00640 1 -- 172.21.15.71:0/3822042082 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f920c021ce0 con 0x55ca1c4288b0 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f92296a9740 10 client.5565 fetch_fsmap learned FSMap version 27 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.639+0000 7f92296a9740 1 -- 172.21.15.71:0/3822042082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55ca1c41dc60 con 0x55ca1c4288b0 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f9208c00640 1 -- 172.21.15.71:0/3822042082 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9210003b70 con 0x55ca1c4288b0 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f9208c00640 1 -- 172.21.15.71:0/3822042082 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f920c028050 con 0x55ca1c4288b0 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f92296a9740 10 client.5565 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f92296a9740 10 client.5565 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f92296a9740 20 client.5565 populate_metadata read hostname 'smithi071' 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f92296a9740 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] conn(0x55ca1c434250 0x55ca1c434630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f92296a9740 4 client.5565 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f92296a9740 1 -- 172.21.15.71:0/3822042082 --> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] -- command(tid 0: {"prefix": "status"}) -- 0x55ca1c353fd0 con 0x55ca1c434250 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f921f400640 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] conn(0x55ca1c434250 0x55ca1c434630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f921f400640 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] conn(0x55ca1c434250 0x55ca1c434630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440d194d5fbd03a8 server_cookie=c1b83fe1c7f632e1 in_seq=0 out_seq=0 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.640+0000 7f9208c00640 10 client.5565 ms_handle_connect on v2:172.21.15.71:6832/2966230103 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f9208c00640 1 -- 172.21.15.71:0/3822042082 <== mds.0 v2:172.21.15.71:6832/2966230103 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ca1c353fd0 con 0x55ca1c434250 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f9208c00640 10 client.5565 handle_command_reply: tid=0 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 client.5565 shutdown 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 -- 172.21.15.71:0/3822042082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 msgr2=0x55ca1c44a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c44a0d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f92201521f0 tx=0x7f920c016ad0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 -- 172.21.15.71:0/3822042082 shutdown_connections 2024-09-16T07:25:46.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.71:6832/2966230103,v1:172.21.15.71:6835/2966230103] conn(0x55ca1c434250 0x55ca1c434630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca1c3ddb30 0x55ca1c44a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca1c4288b0 0x55ca1c44a0d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 --2- 172.21.15.71:0/3822042082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ca1c40d100 0x55ca1c4aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 -- 172.21.15.71:0/3822042082 >> 172.21.15.71:0/3822042082 conn(0x55ca1c3e3a20 msgr2=0x55ca1c44b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f9208c00640 0 client.5565 ms_handle_reset on v2:172.21.15.71:6832/2966230103 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 -- 172.21.15.71:0/3822042082 shutdown_connections 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 1 -- 172.21.15.71:0/3822042082 wait complete. 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.641+0000 7f92296a9740 20 client.5565 trim_cache size 0 max 16384 2024-09-16T07:25:46.646 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5433, 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 6.1691944220000003, 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 27, 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 48, 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 48, 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 10.513647763 2024-09-16T07:25:46.647 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:25:46.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 -- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9200078bb0 msgr2=0x7f920007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9200078bb0 0x7f920007b070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9210034000 tx=0x7f9210004730 comp rx=0 tx=0).stop 2024-09-16T07:25:46.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 -- 172.21.15.71:0/4160590510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 msgr2=0x7f92201530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 0x7f92201530e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f921800cbc0 tx=0x7f921800a430 comp rx=0 tx=0).stop 2024-09-16T07:25:46.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 -- 172.21.15.71:0/4160590510 shutdown_connections 2024-09-16T07:25:46.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9200078bb0 0x7f920007b070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f922015f5a0 0x7f92201530e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92200ccf00 0x7f9220152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 --2- 172.21.15.71:0/4160590510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9220073fb0 0x7f92200c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 -- 172.21.15.71:0/4160590510 >> 172.21.15.71:0/4160590510 conn(0x7f92200b1950 msgr2=0x7f92200af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:46.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 -- 172.21.15.71:0/4160590510 shutdown_connections 2024-09-16T07:25:46.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.642+0000 7f9226800640 1 -- 172.21.15.71:0/4160590510 wait complete. 2024-09-16T07:25:46.657 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:25:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.813+0000 7f1530600640 1 -- 172.21.15.71:0/134778322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 msgr2=0x7f1528074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.813+0000 7f1530600640 1 --2- 172.21.15.71:0/134778322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 0x7f1528074420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1528075130 tx=0x7f152002a980 comp rx=0 tx=0).stop 2024-09-16T07:25:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.813+0000 7f1530600640 1 -- 172.21.15.71:0/134778322 shutdown_connections 2024-09-16T07:25:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.813+0000 7f1530600640 1 --2- 172.21.15.71:0/134778322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 0x7f1528074420 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.813+0000 7f1530600640 1 --2- 172.21.15.71:0/134778322 >> v2:172.21.15.177:3300/0 conn(0x7f152815f5a0 0x7f15280b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.813+0000 7f1530600640 1 -- 172.21.15.71:0/134778322 >> 172.21.15.71:0/134778322 conn(0x7f15280b1950 msgr2=0x7f15280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.813+0000 7f1530600640 1 -- 172.21.15.71:0/134778322 shutdown_connections 2024-09-16T07:25:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.813+0000 7f1530600640 1 -- 172.21.15.71:0/134778322 wait complete. 2024-09-16T07:25:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.815+0000 7f1530600640 1 Processor -- start 2024-09-16T07:25:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.815+0000 7f1530600640 1 -- start start 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f1530600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 0x7f15281f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f1530600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280ccf00 0x7f152809cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f1530600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815f5a0 0x7f152809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f1530600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f152806f2e0 con 0x7f1528073fb0 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f1530600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f152806f160 con 0x7f15280ccf00 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f1530600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f152806f460 con 0x7f152815f5a0 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f152dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 0x7f15281f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f152dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 0x7f15281f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52118/0 (socket says 172.21.15.71:52118) 2024-09-16T07:25:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f152dc00640 1 -- 172.21.15.71:0/3429202291 learned_addr learned my addr 172.21.15.71:0/3429202291 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f152d200640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280ccf00 0x7f152809cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f152e600640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815f5a0 0x7f152809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.816+0000 7f152dc00640 1 -- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815f5a0 msgr2=0x7f152809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f152dc00640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815f5a0 0x7f152809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f152dc00640 1 -- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280ccf00 msgr2=0x7f152809cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f152dc00640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280ccf00 0x7f152809cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f152dc00640 1 -- 172.21.15.71:0/3429202291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15281f4e80 con 0x7f1528073fb0 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f152dc00640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 0x7f15281f4870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f151c00d700 tx=0x7f151c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f1525600640 1 -- 172.21.15.71:0/3429202291 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c017070 con 0x7f1528073fb0 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f1525600640 1 -- 172.21.15.71:0/3429202291 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f151c005310 con 0x7f1528073fb0 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f1525600640 1 -- 172.21.15.71:0/3429202291 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c009820 con 0x7f1528073fb0 2024-09-16T07:25:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15281f5160 con 0x7f1528073fb0 2024-09-16T07:25:46.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.817+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15281f5720 con 0x7f1528073fb0 2024-09-16T07:25:46.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.818+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152815a6c0 con 0x7f1528073fb0 2024-09-16T07:25:46.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.820+0000 7f1525600640 1 -- 172.21.15.71:0/3429202291 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f151c0099c0 con 0x7f1528073fb0 2024-09-16T07:25:46.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.820+0000 7f1525600640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1508078bb0 0x7f150807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:46.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.821+0000 7f1525600640 1 -- 172.21.15.71:0/3429202291 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f151c09b070 con 0x7f1528073fb0 2024-09-16T07:25:46.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.821+0000 7f152d200640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1508078bb0 0x7f150807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:46.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.822+0000 7f152d200640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1508078bb0 0x7f150807b070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f15281f4db0 tx=0x7f1514007e30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:46.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:46.825+0000 7f1525600640 1 -- 172.21.15.71:0/3429202291 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f152815a6c0 con 0x7f1528073fb0 2024-09-16T07:25:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.042+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f152806f9f0 con 0x7f1528073fb0 2024-09-16T07:25:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.043+0000 7f1525600640 1 -- 172.21.15.71:0/3429202291 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f151c062ac0 con 0x7f1528073fb0 2024-09-16T07:25:47.042 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1508078bb0 msgr2=0x7f150807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1508078bb0 0x7f150807b070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f15281f4db0 tx=0x7f1514007e30 comp rx=0 tx=0).stop 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 msgr2=0x7f15281f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 0x7f15281f4870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f151c00d700 tx=0x7f151c00dbc0 comp rx=0 tx=0).stop 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 shutdown_connections 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1508078bb0 0x7f150807b070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815f5a0 0x7f152809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280ccf00 0x7f152809cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 --2- 172.21.15.71:0/3429202291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1528073fb0 0x7f15281f4870 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 >> 172.21.15.71:0/3429202291 conn(0x7f15280b1950 msgr2=0x7f1528154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 shutdown_connections 2024-09-16T07:25:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47.045+0000 7f1530600640 1 -- 172.21.15.71:0/3429202291 wait complete. 2024-09-16T07:25:47.061 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:25:47.062 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:25:47.062 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:25:47.062 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:25:47.062 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:25:47.062 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:25:47.062 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:25:47.062 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:25:47.062 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:25:47.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:47.063 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:25:47.098 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42530sec preferred_lft 42530sec 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:25:47.099 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:25:47.100 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:25:47.101 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:25:47.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:47.101 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:25:47.191 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:25:47.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:47.194 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:25:47.231 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:25:47.231 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:25:47.231 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:25:47.231 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:25:47.231 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:25:47.231 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:25:47.232 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:25:47.232 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:25:47.232 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:25:47.232 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:25:47.232 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:25:47.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:25:47.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:25:47.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:47.426 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:25:47.461 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:25:47.461 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:25:47.462 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:25:47.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:47.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:25:47.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:25:47.558 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:25:47.593 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:25:47.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:25:47.593 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:25:47.668 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:25:47.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:47.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:25:47.796 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:25:47.796 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:25:47.824 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:25:47.824 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:25:47.824 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:25:47.824 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:25:47.824 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:25:47.825 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:25:47.825 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:25:47.826 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:25:47.826 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:25:47.826 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:25:47.828 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:25:47.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:47.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:25:47.885 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:25:47.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:25:47.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:25:47.980 INFO:teuthology.orchestra.run.smithi071.stdout:client5386 2024-09-16T07:25:47.980 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:47.980 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:48.017 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:25:48.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.220+0000 7fb1c6c00640 1 -- 172.21.15.71:0/741496441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a4003a50 msgr2=0x7fb1a4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:48.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.220+0000 7fb1c6c00640 1 --2- 172.21.15.71:0/741496441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a4003a50 0x7fb1a4003e70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0001690 tx=0x7fb1a4005830 comp rx=0 tx=0).stop 2024-09-16T07:25:48.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.220+0000 7fb1c6c00640 1 -- 172.21.15.71:0/741496441 shutdown_connections 2024-09-16T07:25:48.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.220+0000 7fb1c6c00640 1 --2- 172.21.15.71:0/741496441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a4003a50 0x7fb1a4003e70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.220+0000 7fb1c6c00640 1 -- 172.21.15.71:0/741496441 >> 172.21.15.71:0/741496441 conn(0x7fb1c00b18e0 msgr2=0x7fb1c00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:48.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.220+0000 7fb1c6c00640 1 -- 172.21.15.71:0/741496441 shutdown_connections 2024-09-16T07:25:48.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.220+0000 7fb1c6c00640 1 -- 172.21.15.71:0/741496441 wait complete. 2024-09-16T07:25:48.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.222+0000 7fb1c6c00640 1 Processor -- start 2024-09-16T07:25:48.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.222+0000 7fb1c6c00640 1 -- start start 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1c6c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1a4003a50 0x7fb1c01502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1c6c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0151800 0x7fb1c00cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1c6c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c0148700 0x7fb1c00af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1c6c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c00ccc20 con 0x7fb1c0151800 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1c6c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c00cd5e0 con 0x7fb1c0148700 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1c6c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c0074240 con 0x7fb1a4003a50 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1bf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0151800 0x7fb1c00cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1bf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0151800 0x7fb1c00cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52134/0 (socket says 172.21.15.71:52134) 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1bf400640 1 -- 172.21.15.71:0/1352487210 learned_addr learned my addr 172.21.15.71:0/1352487210 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1bfe00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1a4003a50 0x7fb1c01502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:48.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.223+0000 7fb1c4c00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c0148700 0x7fb1c00af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:48.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1bf400640 1 -- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1a4003a50 msgr2=0x7fb1c01502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:48.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1bf400640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1a4003a50 0x7fb1c01502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1bf400640 1 -- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c0148700 msgr2=0x7fb1c00af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:48.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1bf400640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c0148700 0x7fb1c00af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1bf400640 1 -- 172.21.15.71:0/1352487210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c014fa20 con 0x7fb1c0151800 2024-09-16T07:25:48.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1bf400640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0151800 0x7fb1c00cabc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac011a10 tx=0x7fb1ac011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:48.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1b7e00640 1 -- 172.21.15.71:0/1352487210 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac01e070 con 0x7fb1c0151800 2024-09-16T07:25:48.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1b7e00640 1 -- 172.21.15.71:0/1352487210 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1ac00fbb0 con 0x7fb1c0151800 2024-09-16T07:25:48.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1b7e00640 1 -- 172.21.15.71:0/1352487210 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac019410 con 0x7fb1c0151800 2024-09-16T07:25:48.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c0148b80 con 0x7fb1c0151800 2024-09-16T07:25:48.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.224+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c00c9e00 con 0x7fb1c0151800 2024-09-16T07:25:48.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.225+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c0069360 con 0x7fb1c0151800 2024-09-16T07:25:48.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.226+0000 7fb1b7e00640 1 -- 172.21.15.71:0/1352487210 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fb1ac021070 con 0x7fb1c0151800 2024-09-16T07:25:48.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.228+0000 7fb1b7e00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb1a0078b50 0x7fb1a007b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:48.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.228+0000 7fb1b7e00640 1 -- 172.21.15.71:0/1352487210 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fb1ac09b6d0 con 0x7fb1c0151800 2024-09-16T07:25:48.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.228+0000 7fb1bfe00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb1a0078b50 0x7fb1a007b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.229+0000 7fb1bfe00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb1a0078b50 0x7fb1a007b010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0032000 tx=0x7fb1b002c040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:48.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.232+0000 7fb1b7e00640 1 -- 172.21.15.71:0/1352487210 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb1c014fa20 con 0x7fb1c0151800 2024-09-16T07:25:48.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.455+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb1c0074730 con 0x7fb1c0151800 2024-09-16T07:25:48.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.614+0000 7fb1b7e00640 1 -- 172.21.15.71:0/1352487210 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) ==== 136+0+0 (secure 0 0 0) 0x7fb1ac0630b0 con 0x7fb1c0151800 2024-09-16T07:25:48.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb1a0078b50 msgr2=0x7fb1a007b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:48.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb1a0078b50 0x7fb1a007b010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0032000 tx=0x7fb1b002c040 comp rx=0 tx=0).stop 2024-09-16T07:25:48.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0151800 msgr2=0x7fb1c00cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:48.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0151800 0x7fb1c00cabc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac011a10 tx=0x7fb1ac011ed0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 shutdown_connections 2024-09-16T07:25:48.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb1a0078b50 0x7fb1a007b010 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1c0148700 0x7fb1c00af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0151800 0x7fb1c00cabc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 --2- 172.21.15.71:0/1352487210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1a4003a50 0x7fb1c01502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.616+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 >> 172.21.15.71:0/1352487210 conn(0x7fb1c00b18e0 msgr2=0x7fb1c015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:48.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.617+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 shutdown_connections 2024-09-16T07:25:48.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.617+0000 7fb1c6c00640 1 -- 172.21.15.71:0/1352487210 wait complete. 2024-09-16T07:25:48.632 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:25:48.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.791+0000 7f8723600640 1 -- 172.21.15.71:0/1966404505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 msgr2=0x7f8704003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:48.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.791+0000 7f8723600640 1 --2- 172.21.15.71:0/1966404505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 0x7f8704003e20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8710009b50 tx=0x7f8704005830 comp rx=0 tx=0).stop 2024-09-16T07:25:48.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.791+0000 7f8721600640 1 -- 172.21.15.71:0/1966404505 reap_dead start 2024-09-16T07:25:48.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.791+0000 7f8723600640 1 -- 172.21.15.71:0/1966404505 shutdown_connections 2024-09-16T07:25:48.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.791+0000 7f8723600640 1 -- 172.21.15.71:0/1966404505 >> 172.21.15.71:0/1966404505 conn(0x7f871c0b1950 msgr2=0x7f871c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:48.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.791+0000 7f8723600640 1 -- 172.21.15.71:0/1966404505 shutdown_connections 2024-09-16T07:25:48.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.791+0000 7f8723600640 1 -- 172.21.15.71:0/1966404505 wait complete. 2024-09-16T07:25:48.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.792+0000 7f8723600640 1 Processor -- start 2024-09-16T07:25:48.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.793+0000 7f8723600640 1 -- start start 2024-09-16T07:25:48.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.793+0000 7f8723600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 0x7f871c1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:48.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8723600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f871c1556b0 0x7f871c0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:48.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8723600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871c0cfd60 0x7f871c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:48.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8723600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f871c0745c0 con 0x7f8704003a00 2024-09-16T07:25:48.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8723600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f871c074440 con 0x7f871c1556b0 2024-09-16T07:25:48.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8723600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f871c074740 con 0x7f871c0cfd60 2024-09-16T07:25:48.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 0x7f871c1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:48.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 0x7f871c1494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52150/0 (socket says 172.21.15.71:52150) 2024-09-16T07:25:48.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 -- 172.21.15.71:0/498667174 learned_addr learned my addr 172.21.15.71:0/498667174 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:48.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f871be00640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f871c1556b0 0x7f871c0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:48.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8721600640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871c0cfd60 0x7f871c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:48.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 -- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871c0cfd60 msgr2=0x7f871c157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:48.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871c0cfd60 0x7f871c157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 -- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f871c1556b0 msgr2=0x7f871c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:48.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f871c1556b0 0x7f871c0cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:48.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 -- 172.21.15.71:0/498667174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c159650 con 0x7f8704003a00 2024-09-16T07:25:48.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f871be00640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f871c1556b0 0x7f871c0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:48.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8721600640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871c0cfd60 0x7f871c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:25:48.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.794+0000 7f8720c00640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 0x7f871c1494d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8710009a70 tx=0x7f8710033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:48.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.795+0000 7f8718c00640 1 -- 172.21.15.71:0/498667174 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710038040 con 0x7f8704003a00 2024-09-16T07:25:48.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.795+0000 7f8718c00640 1 -- 172.21.15.71:0/498667174 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8710004110 con 0x7f8704003a00 2024-09-16T07:25:48.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.795+0000 7f8718c00640 1 -- 172.21.15.71:0/498667174 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100337e0 con 0x7f8704003a00 2024-09-16T07:25:48.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.795+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c1566f0 con 0x7f8704003a00 2024-09-16T07:25:48.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.795+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c159370 con 0x7f8704003a00 2024-09-16T07:25:48.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.796+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c0b2440 con 0x7f8704003a00 2024-09-16T07:25:48.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.797+0000 7f8718c00640 1 -- 172.21.15.71:0/498667174 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f8710045020 con 0x7f8704003a00 2024-09-16T07:25:48.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.798+0000 7f8718c00640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f86fc078bb0 0x7f86fc07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:48.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.798+0000 7f8718c00640 1 -- 172.21.15.71:0/498667174 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f87100bb830 con 0x7f8704003a00 2024-09-16T07:25:48.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.798+0000 7f871be00640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f86fc078bb0 0x7f86fc07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:48.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.799+0000 7f871be00640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f86fc078bb0 0x7f86fc07b070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8708001d50 tx=0x7f8708000ed0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:48.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:48.802+0000 7f8718c00640 1 -- 172.21.15.71:0/498667174 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8710083980 con 0x7f8704003a00 2024-09-16T07:25:49.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.008+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f871c0b2e80 con 0x7f86fc078bb0 2024-09-16T07:25:49.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.011+0000 7f8718c00640 1 -- 172.21.15.71:0/498667174 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f871c0b2e80 con 0x7f86fc078bb0 2024-09-16T07:25:49.010 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:25:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:25:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:25:49.010 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:25:49.010 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:25:49.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.012+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f86fc078bb0 msgr2=0x7f86fc07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.012+0000 7f8723600640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f86fc078bb0 0x7f86fc07b070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8708001d50 tx=0x7f8708000ed0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 msgr2=0x7f871c1494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 0x7f871c1494d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8710009a70 tx=0x7f8710033670 comp rx=0 tx=0).stop 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 shutdown_connections 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f86fc078bb0 0x7f86fc07b070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871c0cfd60 0x7f871c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f871c1556b0 0x7f871c0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 --2- 172.21.15.71:0/498667174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8704003a00 0x7f871c1494d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 >> 172.21.15.71:0/498667174 conn(0x7f871c0b1950 msgr2=0x7f871c09ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 shutdown_connections 2024-09-16T07:25:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.013+0000 7f8723600640 1 -- 172.21.15.71:0/498667174 wait complete. 2024-09-16T07:25:49.028 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:25:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.189+0000 7f6f5e600640 1 -- 172.21.15.71:0/913887742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f5815df60 msgr2=0x7f6f580d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.189+0000 7f6f5e600640 1 --2- 172.21.15.71:0/913887742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f5815df60 0x7f6f580d0fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f48005fd0 tx=0x7f6f4802aa00 comp rx=0 tx=0).stop 2024-09-16T07:25:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.189+0000 7f6f5e600640 1 -- 172.21.15.71:0/913887742 shutdown_connections 2024-09-16T07:25:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.189+0000 7f6f5e600640 1 --2- 172.21.15.71:0/913887742 >> v2:172.21.15.177:3300/0 conn(0x7f6f580b3450 0x7f6f580b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.189+0000 7f6f5e600640 1 --2- 172.21.15.71:0/913887742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f5815df60 0x7f6f580d0fa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.189+0000 7f6f5e600640 1 -- 172.21.15.71:0/913887742 >> 172.21.15.71:0/913887742 conn(0x7f6f580d3d50 msgr2=0x7f6f580d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.189+0000 7f6f5e600640 1 -- 172.21.15.71:0/913887742 shutdown_connections 2024-09-16T07:25:49.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.189+0000 7f6f5e600640 1 -- 172.21.15.71:0/913887742 wait complete. 2024-09-16T07:25:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.191+0000 7f6f5e600640 1 Processor -- start 2024-09-16T07:25:49.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.191+0000 7f6f5e600640 1 -- start start 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5e600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f580b3450 0x7f6f5815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5e600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f580cf4b0 0x7f6f580cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f5815df60 0x7f6f580cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6f580d2a40 con 0x7f6f5815df60 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5e600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6f580d2bc0 con 0x7f6f580cf4b0 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5e600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6f580cf960 con 0x7f6f580b3450 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f5815df60 0x7f6f580cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5d200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f580b3450 0x7f6f5815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f5815df60 0x7f6f580cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52158/0 (socket says 172.21.15.71:52158) 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5dc00640 1 -- 172.21.15.71:0/3898284379 learned_addr learned my addr 172.21.15.71:0/3898284379 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f57e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f580cf4b0 0x7f6f580cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5d200640 1 -- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f580cf4b0 msgr2=0x7f6f580cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5d200640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f580cf4b0 0x7f6f580cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5d200640 1 -- 172.21.15.71:0/3898284379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f5815df60 msgr2=0x7f6f580cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5d200640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f5815df60 0x7f6f580cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5d200640 1 -- 172.21.15.71:0/3898284379 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f580cc550 con 0x7f6f580b3450 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.192+0000 7f6f5dc00640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f5815df60 0x7f6f580cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:25:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.193+0000 7f6f5d200640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f580b3450 0x7f6f5815b240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6f580c7850 tx=0x7f6f48003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.193+0000 7f6f54c00640 1 -- 172.21.15.71:0/3898284379 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48033050 con 0x7f6f580b3450 2024-09-16T07:25:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.193+0000 7f6f54c00640 1 -- 172.21.15.71:0/3898284379 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f48004450 con 0x7f6f580b3450 2024-09-16T07:25:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.193+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f580cc830 con 0x7f6f580b3450 2024-09-16T07:25:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.193+0000 7f6f54c00640 1 -- 172.21.15.71:0/3898284379 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48038450 con 0x7f6f580b3450 2024-09-16T07:25:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.193+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5815eb20 con 0x7f6f580b3450 2024-09-16T07:25:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.194+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f580b3890 con 0x7f6f580b3450 2024-09-16T07:25:49.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.195+0000 7f6f54c00640 1 -- 172.21.15.71:0/3898284379 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f6f48036070 con 0x7f6f580b3450 2024-09-16T07:25:49.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.196+0000 7f6f54c00640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f38078bb0 0x7f6f3807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.196+0000 7f6f54c00640 1 -- 172.21.15.71:0/3898284379 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f6f480bc680 con 0x7f6f580b3450 2024-09-16T07:25:49.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.197+0000 7f6f57e00640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f38078bb0 0x7f6f3807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.198+0000 7f6f57e00640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f38078bb0 0x7f6f3807b070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f580cd050 tx=0x7f6f44005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:49.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.201+0000 7f6f54c00640 1 -- 172.21.15.71:0/3898284379 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6f480840d0 con 0x7f6f580b3450 2024-09-16T07:25:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.383+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6f580d16f0 con 0x7f6f580b3450 2024-09-16T07:25:49.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.385+0000 7f6f54c00640 1 -- 172.21.15.71:0/3898284379 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f6f48088b00 con 0x7f6f580b3450 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f38078bb0 msgr2=0x7f6f3807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f38078bb0 0x7f6f3807b070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f580cd050 tx=0x7f6f44005f90 comp rx=0 tx=0).stop 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f580b3450 msgr2=0x7f6f5815b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f580b3450 0x7f6f5815b240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6f580c7850 tx=0x7f6f48003f90 comp rx=0 tx=0).stop 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 shutdown_connections 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6f38078bb0 0x7f6f3807b070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f5815df60 0x7f6f580cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f580cf4b0 0x7f6f580cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 --2- 172.21.15.71:0/3898284379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f580b3450 0x7f6f5815b240 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 >> 172.21.15.71:0/3898284379 conn(0x7f6f580d3d50 msgr2=0x7f6f580b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 shutdown_connections 2024-09-16T07:25:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.387+0000 7f6f5e600640 1 -- 172.21.15.71:0/3898284379 wait complete. 2024-09-16T07:25:49.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:25:49.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.563+0000 7fa90fe00640 1 -- 172.21.15.71:0/33736954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa91015df60 msgr2=0x7fa9100d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.564+0000 7fa90fe00640 1 --2- 172.21.15.71:0/33736954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa91015df60 0x7fa9100d0740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa900009830 tx=0x7fa90002a510 comp rx=0 tx=0).stop 2024-09-16T07:25:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.564+0000 7fa90fe00640 1 -- 172.21.15.71:0/33736954 shutdown_connections 2024-09-16T07:25:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.564+0000 7fa90fe00640 1 --2- 172.21.15.71:0/33736954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa91015df60 0x7fa9100d0740 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.564+0000 7fa90fe00640 1 -- 172.21.15.71:0/33736954 >> 172.21.15.71:0/33736954 conn(0x7fa9100d3de0 msgr2=0x7fa9100d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.564+0000 7fa90fe00640 1 -- 172.21.15.71:0/33736954 shutdown_connections 2024-09-16T07:25:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.564+0000 7fa90fe00640 1 -- 172.21.15.71:0/33736954 wait complete. 2024-09-16T07:25:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.565+0000 7fa90fe00640 1 Processor -- start 2024-09-16T07:25:49.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.565+0000 7fa90fe00640 1 -- start start 2024-09-16T07:25:49.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.566+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9100d0c80 0x7fa91015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.566+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9100d29c0 0x7fa9100b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.566+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa91015df60 0x7fa9100cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.566+0000 7fa90fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100d14a0 con 0x7fa9100d29c0 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.566+0000 7fa90fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100d1320 con 0x7fa91015df60 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.566+0000 7fa90fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100c8ff0 con 0x7fa9100d0c80 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9100d29c0 0x7fa9100b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9100d29c0 0x7fa9100b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52170/0 (socket says 172.21.15.71:52170) 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 -- 172.21.15.71:0/2521531982 learned_addr learned my addr 172.21.15.71:0/2521531982 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90ea00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9100d0c80 0x7fa91015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90f400640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa91015df60 0x7fa9100cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 -- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9100d0c80 msgr2=0x7fa91015c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9100d0c80 0x7fa91015c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 -- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa91015df60 msgr2=0x7fa9100cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa91015df60 0x7fa9100cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 -- 172.21.15.71:0/2521531982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9100b7b50 con 0x7fa9100d29c0 2024-09-16T07:25:49.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90f400640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa91015df60 0x7fa9100cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:49.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.567+0000 7fa90e000640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9100d29c0 0x7fa9100b1530 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c430 tx=0x7fa8fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:49.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.568+0000 7fa906a00640 1 -- 172.21.15.71:0/2521531982 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011790 con 0x7fa9100d29c0 2024-09-16T07:25:49.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.568+0000 7fa906a00640 1 -- 172.21.15.71:0/2521531982 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8fc017070 con 0x7fa9100d29c0 2024-09-16T07:25:49.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.568+0000 7fa906a00640 1 -- 172.21.15.71:0/2521531982 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011cc0 con 0x7fa9100d29c0 2024-09-16T07:25:49.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.568+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9100b4230 con 0x7fa9100d29c0 2024-09-16T07:25:49.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.568+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9100b7e00 con 0x7fa9100d29c0 2024-09-16T07:25:49.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.569+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9100d0740 con 0x7fa9100d29c0 2024-09-16T07:25:49.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.570+0000 7fa906a00640 1 -- 172.21.15.71:0/2521531982 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fa8fc01c460 con 0x7fa9100d29c0 2024-09-16T07:25:49.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.571+0000 7fa906a00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8f0078bb0 0x7fa8f007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.571+0000 7fa906a00640 1 -- 172.21.15.71:0/2521531982 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa8fc09ace0 con 0x7fa9100d29c0 2024-09-16T07:25:49.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.571+0000 7fa90ea00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8f0078bb0 0x7fa8f007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.572+0000 7fa90ea00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8f0078bb0 0x7fa8f007b070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa900034000 tx=0x7fa900004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:49.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.575+0000 7fa906a00640 1 -- 172.21.15.71:0/2521531982 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8fc062ec0 con 0x7fa9100d29c0 2024-09-16T07:25:49.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.764+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fa91006d4e0 con 0x7fa9100d29c0 2024-09-16T07:25:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.764+0000 7fa906a00640 1 -- 172.21.15.71:0/2521531982 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v31) ==== 116+0+2 (secure 0 0 0) 0x7fa8fc0627c0 con 0x7fa9100d29c0 2024-09-16T07:25:49.765 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:25:49.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8f0078bb0 msgr2=0x7fa8f007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8f0078bb0 0x7fa8f007b070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa900034000 tx=0x7fa900004730 comp rx=0 tx=0).stop 2024-09-16T07:25:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9100d29c0 msgr2=0x7fa9100b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9100d29c0 0x7fa9100b1530 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c430 tx=0x7fa8fc00c8f0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 shutdown_connections 2024-09-16T07:25:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8f0078bb0 0x7fa8f007b070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa91015df60 0x7fa9100cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9100d29c0 0x7fa9100b1530 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 --2- 172.21.15.71:0/2521531982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9100d0c80 0x7fa91015c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 >> 172.21.15.71:0/2521531982 conn(0x7fa9100d3de0 msgr2=0x7fa9100b6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:49.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 shutdown_connections 2024-09-16T07:25:49.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.767+0000 7fa90fe00640 1 -- 172.21.15.71:0/2521531982 wait complete. 2024-09-16T07:25:49.783 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:25:49.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.943+0000 7fd5aa200640 1 -- 172.21.15.71:0/4043854622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a40cefc0 msgr2=0x7fd5a4073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.943+0000 7fd5aa200640 1 --2- 172.21.15.71:0/4043854622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a40cefc0 0x7fd5a4073d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd590009830 tx=0x7fd59002a510 comp rx=0 tx=0).stop 2024-09-16T07:25:49.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.943+0000 7fd5aa200640 1 -- 172.21.15.71:0/4043854622 shutdown_connections 2024-09-16T07:25:49.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.943+0000 7fd5aa200640 1 --2- 172.21.15.71:0/4043854622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a40cefc0 0x7fd5a4073d40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.943+0000 7fd5aa200640 1 --2- 172.21.15.71:0/4043854622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5a415bb90 0x7fd5a40b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.943+0000 7fd5aa200640 1 -- 172.21.15.71:0/4043854622 >> 172.21.15.71:0/4043854622 conn(0x7fd5a40b1510 msgr2=0x7fd5a40b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:49.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.943+0000 7fd5aa200640 1 -- 172.21.15.71:0/4043854622 shutdown_connections 2024-09-16T07:25:49.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.943+0000 7fd5aa200640 1 -- 172.21.15.71:0/4043854622 wait complete. 2024-09-16T07:25:49.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.945+0000 7fd5aa200640 1 Processor -- start 2024-09-16T07:25:49.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.945+0000 7fd5aa200640 1 -- start start 2024-09-16T07:25:49.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5aa200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5a40cefc0 0x7fd5a41e8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5aa200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5a40cf3e0 0x7fd5a40c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5aa200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a415bb90 0x7fd5a40c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5aa200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4074610 con 0x7fd5a415bb90 2024-09-16T07:25:49.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5aa200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4074490 con 0x7fd5a40cf3e0 2024-09-16T07:25:49.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5aa200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a4074790 con 0x7fd5a40cefc0 2024-09-16T07:25:49.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a415bb90 0x7fd5a40c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a415bb90 0x7fd5a40c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52184/0 (socket says 172.21.15.71:52184) 2024-09-16T07:25:49.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a9800640 1 -- 172.21.15.71:0/142088206 learned_addr learned my addr 172.21.15.71:0/142088206 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:49.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a8e00640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5a40cefc0 0x7fd5a41e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a3e00640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5a40cf3e0 0x7fd5a40c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a9800640 1 -- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5a40cefc0 msgr2=0x7fd5a41e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a9800640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5a40cefc0 0x7fd5a41e8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a9800640 1 -- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5a40cf3e0 msgr2=0x7fd5a40c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:49.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.946+0000 7fd5a9800640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5a40cf3e0 0x7fd5a40c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:49.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.947+0000 7fd5a9800640 1 -- 172.21.15.71:0/142088206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a40cbee0 con 0x7fd5a415bb90 2024-09-16T07:25:49.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.947+0000 7fd5a9800640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a415bb90 0x7fd5a40c6f70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00cbc0 tx=0x7fd59c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:49.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.947+0000 7fd5a0c00640 1 -- 172.21.15.71:0/142088206 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c016020 con 0x7fd5a415bb90 2024-09-16T07:25:49.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.947+0000 7fd5a0c00640 1 -- 172.21.15.71:0/142088206 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd59c00ae30 con 0x7fd5a415bb90 2024-09-16T07:25:49.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.947+0000 7fd5a0c00640 1 -- 172.21.15.71:0/142088206 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c012660 con 0x7fd5a415bb90 2024-09-16T07:25:49.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.947+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a40c7880 con 0x7fd5a415bb90 2024-09-16T07:25:49.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.947+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a40c7e40 con 0x7fd5a415bb90 2024-09-16T07:25:49.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.948+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a406f710 con 0x7fd5a415bb90 2024-09-16T07:25:49.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.949+0000 7fd5a0c00640 1 -- 172.21.15.71:0/142088206 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fd59c019020 con 0x7fd5a415bb90 2024-09-16T07:25:49.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.951+0000 7fd5a0c00640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd584078bb0 0x7fd58407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:49.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.951+0000 7fd5a0c00640 1 -- 172.21.15.71:0/142088206 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fd59c09b5f0 con 0x7fd5a415bb90 2024-09-16T07:25:49.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.951+0000 7fd5a8e00640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd584078bb0 0x7fd58407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:49.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.952+0000 7fd5a8e00640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd584078bb0 0x7fd58407b070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd594005fd0 tx=0x7fd594005c50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:49.955+0000 7fd5a0c00640 1 -- 172.21.15.71:0/142088206 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd59c020020 con 0x7fd5a415bb90 2024-09-16T07:25:50.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.139+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fd5a40ae6f0 con 0x7fd5a415bb90 2024-09-16T07:25:50.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.143+0000 7fd5a0c00640 1 -- 172.21.15.71:0/142088206 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7fd59c063700 con 0x7fd5a415bb90 2024-09-16T07:25:50.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd584078bb0 msgr2=0x7fd58407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd584078bb0 0x7fd58407b070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd594005fd0 tx=0x7fd594005c50 comp rx=0 tx=0).stop 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a415bb90 msgr2=0x7fd5a40c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a415bb90 0x7fd5a40c6f70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00cbc0 tx=0x7fd59c00a430 comp rx=0 tx=0).stop 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 shutdown_connections 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd584078bb0 0x7fd58407b070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a415bb90 0x7fd5a40c6f70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5a40cf3e0 0x7fd5a40c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 --2- 172.21.15.71:0/142088206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5a40cefc0 0x7fd5a41e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 >> 172.21.15.71:0/142088206 conn(0x7fd5a40b1510 msgr2=0x7fd5a40b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 shutdown_connections 2024-09-16T07:25:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.146+0000 7fd5aa200640 1 -- 172.21.15.71:0/142088206 wait complete. 2024-09-16T07:25:50.160 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:50.160+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:50.160 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:50.160+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:50.160 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:50.160+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:50.160 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:50.160+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:50.160 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:50.160+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:50.162 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T07:25:50.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.322+0000 7f41e6600640 1 -- 172.21.15.71:0/3897544245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e015db70 msgr2=0x7f41e00b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:50.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.322+0000 7f41e6600640 1 --2- 172.21.15.71:0/3897544245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e015db70 0x7f41e00b43a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009830 tx=0x7f41d002a3f0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.322+0000 7f41e6600640 1 -- 172.21.15.71:0/3897544245 shutdown_connections 2024-09-16T07:25:50.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.322+0000 7f41e6600640 1 --2- 172.21.15.71:0/3897544245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41e0074760 0x7f41e00b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.322+0000 7f41e6600640 1 --2- 172.21.15.71:0/3897544245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e015db70 0x7f41e00b43a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.322+0000 7f41e6600640 1 -- 172.21.15.71:0/3897544245 >> 172.21.15.71:0/3897544245 conn(0x7f41e00b18e0 msgr2=0x7f41e00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:50.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.322+0000 7f41e6600640 1 -- 172.21.15.71:0/3897544245 shutdown_connections 2024-09-16T07:25:50.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.322+0000 7f41e6600640 1 -- 172.21.15.71:0/3897544245 wait complete. 2024-09-16T07:25:50.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.324+0000 7f41e6600640 1 Processor -- start 2024-09-16T07:25:50.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.324+0000 7f41e6600640 1 -- start start 2024-09-16T07:25:50.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41e6600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e0074760 0x7f41e015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:50.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41e6600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e00b2be0 0x7f41e013c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:50.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41e6600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41e015db70 0x7f41e0155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:50.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41e6600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f41e006ee00 con 0x7f41e0074760 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41e6600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f41e0073c10 con 0x7f41e00b2be0 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41e6600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f41e006ef80 con 0x7f41e015db70 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41df400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e0074760 0x7f41e015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41df400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e0074760 0x7f41e015aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52204/0 (socket says 172.21.15.71:52204) 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41df400640 1 -- 172.21.15.71:0/4225775255 learned_addr learned my addr 172.21.15.71:0/4225775255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41dea00640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e00b2be0 0x7f41e013c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41dfe00640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41e015db70 0x7f41e0155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41df400640 1 -- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41e015db70 msgr2=0x7f41e0155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41df400640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41e015db70 0x7f41e0155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41df400640 1 -- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e00b2be0 msgr2=0x7f41e013c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41df400640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e00b2be0 0x7f41e013c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41df400640 1 -- 172.21.15.71:0/4225775255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e009cb70 con 0x7f41e0074760 2024-09-16T07:25:50.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.325+0000 7f41dfe00640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41e015db70 0x7f41e0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:50.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.326+0000 7f41df400640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e0074760 0x7f41e015aa50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009950 tx=0x7f41d0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:50.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.326+0000 7f41d7400640 1 -- 172.21.15.71:0/4225775255 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0031d60 con 0x7f41e0074760 2024-09-16T07:25:50.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.326+0000 7f41d7400640 1 -- 172.21.15.71:0/4225775255 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41d0033e10 con 0x7f41e0074760 2024-09-16T07:25:50.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.326+0000 7f41d7400640 1 -- 172.21.15.71:0/4225775255 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0005260 con 0x7f41e0074760 2024-09-16T07:25:50.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.326+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e009ce50 con 0x7f41e0074760 2024-09-16T07:25:50.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.326+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e009d410 con 0x7f41e0074760 2024-09-16T07:25:50.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.327+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e00b16d0 con 0x7f41e0074760 2024-09-16T07:25:50.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.328+0000 7f41d7400640 1 -- 172.21.15.71:0/4225775255 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f41d0005400 con 0x7f41e0074760 2024-09-16T07:25:50.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.329+0000 7f41d7400640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41c0078be0 0x7f41c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:50.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.329+0000 7f41d7400640 1 -- 172.21.15.71:0/4225775255 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f41d00bc1b0 con 0x7f41e0074760 2024-09-16T07:25:50.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.329+0000 7f41dea00640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41c0078be0 0x7f41c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:50.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.330+0000 7f41dea00640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41c0078be0 0x7f41c007b0a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0058c0 tx=0x7f41cc005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:50.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.333+0000 7f41d7400640 1 -- 172.21.15.71:0/4225775255 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f41e00b16d0 con 0x7f41e0074760 2024-09-16T07:25:50.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:50.545+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f41e00af2a0 con 0x7f41c0078be0 2024-09-16T07:25:51.059 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:51.059+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:51.059 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:51.059+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:51.059 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:51.059+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:51.059 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:51.059+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:51.059 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:25:51.059+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:25:51.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.109+0000 7f41d7400640 1 -- 172.21.15.71:0/4225775255 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41e00af2a0 con 0x7f41c0078be0 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41c0078be0 msgr2=0x7f41c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41c0078be0 0x7f41c007b0a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0058c0 tx=0x7f41cc005850 comp rx=0 tx=0).stop 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e0074760 msgr2=0x7f41e015aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e0074760 0x7f41e015aa50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009950 tx=0x7f41d0033c20 comp rx=0 tx=0).stop 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 shutdown_connections 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41c0078be0 0x7f41c007b0a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41e015db70 0x7f41e0155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e00b2be0 0x7f41e013c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 --2- 172.21.15.71:0/4225775255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e0074760 0x7f41e015aa50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 >> 172.21.15.71:0/4225775255 conn(0x7f41e00b18e0 msgr2=0x7f41e00cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:51.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.112+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 shutdown_connections 2024-09-16T07:25:51.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.113+0000 7f41e6600640 1 -- 172.21.15.71:0/4225775255 wait complete. 2024-09-16T07:25:51.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:25:51.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.294+0000 7fa7df800640 1 -- 172.21.15.71:0/2972772131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc004d40 msgr2=0x7fa7bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:51.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.294+0000 7fa7df800640 1 --2- 172.21.15.71:0/2972772131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc004d40 0x7fa7bc0051c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c310 tx=0x7fa7bc005830 comp rx=0 tx=0).stop 2024-09-16T07:25:51.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.294+0000 7fa7dd800640 1 -- 172.21.15.71:0/2972772131 reap_dead start 2024-09-16T07:25:51.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.294+0000 7fa7df800640 1 -- 172.21.15.71:0/2972772131 shutdown_connections 2024-09-16T07:25:51.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.294+0000 7fa7df800640 1 -- 172.21.15.71:0/2972772131 >> 172.21.15.71:0/2972772131 conn(0x7fa7d80b1840 msgr2=0x7fa7d80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:51.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.294+0000 7fa7df800640 1 -- 172.21.15.71:0/2972772131 shutdown_connections 2024-09-16T07:25:51.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.294+0000 7fa7df800640 1 -- 172.21.15.71:0/2972772131 wait complete. 2024-09-16T07:25:51.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.295+0000 7fa7df800640 1 Processor -- start 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.296+0000 7fa7df800640 1 -- start start 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.296+0000 7fa7df800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc004d40 0x7fa7d8158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.296+0000 7fa7df800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d81577c0 0x7fa7d8157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.296+0000 7fa7df800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d8141e50 0x7fa7d81422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.296+0000 7fa7df800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d80ce2c0 con 0x7fa7d8141e50 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7df800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d80736d0 con 0x7fa7bc004d40 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7df800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d80ce440 con 0x7fa7d81577c0 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d8141e50 0x7fa7d81422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d8141e50 0x7fa7d81422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52216/0 (socket says 172.21.15.71:52216) 2024-09-16T07:25:51.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 -- 172.21.15.71:0/2641439024 learned_addr learned my addr 172.21.15.71:0/2641439024 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dce00640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc004d40 0x7fa7d8158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7d7e00640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d81577c0 0x7fa7d8157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 -- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d81577c0 msgr2=0x7fa7d8157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d81577c0 0x7fa7d8157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 -- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc004d40 msgr2=0x7fa7d8158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc004d40 0x7fa7d8158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 -- 172.21.15.71:0/2641439024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d8142bc0 con 0x7fa7d8141e50 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7d7e00640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d81577c0 0x7fa7d8157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.297+0000 7fa7dd800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d8141e50 0x7fa7d81422d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c400 tx=0x7fa7d002ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.298+0000 7fa7d4c00640 1 -- 172.21.15.71:0/2641439024 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0031070 con 0x7fa7d8141e50 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.298+0000 7fa7d4c00640 1 -- 172.21.15.71:0/2641439024 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7d00014f0 con 0x7fa7d8141e50 2024-09-16T07:25:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.298+0000 7fa7d4c00640 1 -- 172.21.15.71:0/2641439024 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0006c70 con 0x7fa7d8141e50 2024-09-16T07:25:51.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.298+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d8142ea0 con 0x7fa7d8141e50 2024-09-16T07:25:51.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.298+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d80c9d00 con 0x7fa7d8141e50 2024-09-16T07:25:51.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.299+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d80c9f00 con 0x7fa7d8141e50 2024-09-16T07:25:51.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.300+0000 7fa7d4c00640 1 -- 172.21.15.71:0/2641439024 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 11) ==== 101644+0+0 (secure 0 0 0) 0x7fa7d0034070 con 0x7fa7d8141e50 2024-09-16T07:25:51.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.301+0000 7fa7d4c00640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b8078c40 0x7fa7b807b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:25:51.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.301+0000 7fa7d4c00640 1 -- 172.21.15.71:0/2641439024 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa7d00babd0 con 0x7fa7d8141e50 2024-09-16T07:25:51.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.301+0000 7fa7dce00640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b8078c40 0x7fa7b807b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:25:51.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.302+0000 7fa7dce00640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b8078c40 0x7fa7b807b100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8000f80 tx=0x7fa7c8006ef0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:25:51.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.305+0000 7fa7d4c00640 1 -- 172.21.15.71:0/2641439024 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa7d80c9f00 con 0x7fa7d8141e50 2024-09-16T07:25:51.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.518+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa7d8143070 con 0x7fa7b8078c40 2024-09-16T07:25:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.524+0000 7fa7d4c00640 1 -- 172.21.15.71:0/2641439024 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa7d8143070 con 0x7fa7b8078c40 2024-09-16T07:25:51.543 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:25:51.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b8078c40 msgr2=0x7fa7b807b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:51.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b8078c40 0x7fa7b807b100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8000f80 tx=0x7fa7c8006ef0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d8141e50 msgr2=0x7fa7d81422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:25:51.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d8141e50 0x7fa7d81422d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c400 tx=0x7fa7d002ec40 comp rx=0 tx=0).stop 2024-09-16T07:25:51.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 shutdown_connections 2024-09-16T07:25:51.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b8078c40 0x7fa7b807b100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d8141e50 0x7fa7d81422d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d81577c0 0x7fa7d8157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.526+0000 7fa7df800640 1 --2- 172.21.15.71:0/2641439024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc004d40 0x7fa7d8158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:25:51.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.527+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 >> 172.21.15.71:0/2641439024 conn(0x7fa7d80b1840 msgr2=0x7fa7d80aa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:25:51.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.527+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 shutdown_connections 2024-09-16T07:25:51.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:25:51.527+0000 7fa7df800640 1 -- 172.21.15.71:0/2641439024 wait complete. 2024-09-16T07:25:51.545 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:25:51.546 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:51.546 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:51.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:51.581 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:25:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:25:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:25:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.51669 s, 143 MB/s 2024-09-16T07:25:59.181 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:25:59.181 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:25:59.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:25:59.217 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:26:06.721 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:26:06.721 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:26:06.721 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.42702 s, 145 MB/s 2024-09-16T07:26:06.722 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:26:06.722 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:26:06.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:26:06.758 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:26:14.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:26:14.060 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:26:14.135 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:26:14.135 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:26:14.135 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.30042 s, 147 MB/s 2024-09-16T07:26:14.136 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:26:14.137 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:26:14.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:26:14.172 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:26:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:26:21.684 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:26:21.684 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.43339 s, 144 MB/s 2024-09-16T07:26:21.685 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:26:21.685 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:26:21.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:26:21.721 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:26:29.161 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:26:29.161 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:26:29.161 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.3629 s, 146 MB/s 2024-09-16T07:26:29.162 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:26:29.163 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:26:29.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:26:29.198 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:26:37.074 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:26:37.075 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:26:37.075 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.79948 s, 138 MB/s 2024-09-16T07:26:37.077 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:26:37.077 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:26:37.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:26:37.114 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:26:44.175 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:26:44.180 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:26:44.621 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:26:44.621 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:26:44.621 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.4309 s, 144 MB/s 2024-09-16T07:26:44.622 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:26:44.622 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:26:44.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:26:44.658 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:26:52.076 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:26:52.077 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:26:52.077 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.34227 s, 146 MB/s 2024-09-16T07:26:52.078 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:26:52.078 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:26:52.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:26:52.114 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:26:59.628 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:26:59.628 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:26:59.628 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.43701 s, 144 MB/s 2024-09-16T07:26:59.629 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:26:59.629 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:26:59.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:26:59.665 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:27:07.145 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:27:07.146 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:27:07.146 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.40525 s, 145 MB/s 2024-09-16T07:27:07.147 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T07:27:07.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.311+0000 7f9491000640 1 -- 172.21.15.71:0/2298500703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9470003a50 msgr2=0x7f9470003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.311+0000 7f9491000640 1 --2- 172.21.15.71:0/2298500703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9470003a50 0x7f9470003e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f947c009b50 tx=0x7f9470005830 comp rx=0 tx=0).stop 2024-09-16T07:27:07.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.311+0000 7f9491000640 1 -- 172.21.15.71:0/2298500703 shutdown_connections 2024-09-16T07:27:07.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.311+0000 7f9491000640 1 --2- 172.21.15.71:0/2298500703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9470003a50 0x7f9470003e70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.311+0000 7f9491000640 1 -- 172.21.15.71:0/2298500703 >> 172.21.15.71:0/2298500703 conn(0x7f948c0b18e0 msgr2=0x7f948c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:07.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.311+0000 7f9491000640 1 -- 172.21.15.71:0/2298500703 shutdown_connections 2024-09-16T07:27:07.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.311+0000 7f9491000640 1 -- 172.21.15.71:0/2298500703 wait complete. 2024-09-16T07:27:07.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.313+0000 7f9491000640 1 Processor -- start 2024-09-16T07:27:07.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.313+0000 7f9491000640 1 -- start start 2024-09-16T07:27:07.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.313+0000 7f9491000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9470003a50 0x7f948c149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:07.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9491000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f948c14a9a0 0x7f948c14a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:07.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9491000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c09cac0 0x7f948c09cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:07.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9491000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f948c06f830 con 0x7f948c14a9a0 2024-09-16T07:27:07.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9491000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f948c06f6b0 con 0x7f9470003a50 2024-09-16T07:27:07.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9491000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f948c06f9b0 con 0x7f948c09cac0 2024-09-16T07:27:07.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9481400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f948c14a9a0 0x7f948c14a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9481400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f948c14a9a0 0x7f948c14a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58898/0 (socket says 172.21.15.71:58898) 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9481400640 1 -- 172.21.15.71:0/2424374061 learned_addr learned my addr 172.21.15.71:0/2424374061 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9489e00640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9470003a50 0x7f948c149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f948a800640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c09cac0 0x7f948c09cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9481400640 1 -- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c09cac0 msgr2=0x7f948c09cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9481400640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c09cac0 0x7f948c09cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9481400640 1 -- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9470003a50 msgr2=0x7f948c149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9481400640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9470003a50 0x7f948c149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.314+0000 7f9481400640 1 -- 172.21.15.71:0/2424374061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c09b4b0 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.315+0000 7f9481400640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f948c14a9a0 0x7f948c14a350 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9474004d60 tx=0x7f9474001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.315+0000 7f9482a00640 1 -- 172.21.15.71:0/2424374061 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94740107d0 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.315+0000 7f9482a00640 1 -- 172.21.15.71:0/2424374061 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9474016070 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.315+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948c09b790 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.315+0000 7f9482a00640 1 -- 172.21.15.71:0/2424374061 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474010d00 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.315+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948c09bd50 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.316+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948c069360 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.317+0000 7f9482a00640 1 -- 172.21.15.71:0/2424374061 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 14) ==== 101593+0+0 (secure 0 0 0) 0x7f947401b460 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.318+0000 7f9482a00640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9468078be0 0x7f946807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.318+0000 7f9482a00640 1 -- 172.21.15.71:0/2424374061 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f947409b560 con 0x7f948c14a9a0 2024-09-16T07:27:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.318+0000 7f9489e00640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9468078be0 0x7f946807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:07.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.319+0000 7f9489e00640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9468078be0 0x7f946807b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f947c035000 tx=0x7f947c0046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:07.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.322+0000 7f9482a00640 1 -- 172.21.15.71:0/2424374061 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9474063020 con 0x7f948c14a9a0 2024-09-16T07:27:07.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.540+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f948c0746a0 con 0x7f9468078be0 2024-09-16T07:27:07.544 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:07.545+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:07.545 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:07.545+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:07.545 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:07.545+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:07.545 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:07.545+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:07.545 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:07.545+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:07.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.566+0000 7f9482a00640 1 -- 172.21.15.71:0/2424374061 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f948c0746a0 con 0x7f9468078be0 2024-09-16T07:27:07.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9468078be0 msgr2=0x7f946807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9468078be0 0x7f946807b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f947c035000 tx=0x7f947c0046c0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f948c14a9a0 msgr2=0x7f948c14a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f948c14a9a0 0x7f948c14a350 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9474004d60 tx=0x7f9474001630 comp rx=0 tx=0).stop 2024-09-16T07:27:07.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 shutdown_connections 2024-09-16T07:27:07.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9468078be0 0x7f946807b0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c09cac0 0x7f948c09cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f948c14a9a0 0x7f948c14a350 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 --2- 172.21.15.71:0/2424374061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9470003a50 0x7f948c149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 >> 172.21.15.71:0/2424374061 conn(0x7f948c0b18e0 msgr2=0x7f948c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:07.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 shutdown_connections 2024-09-16T07:27:07.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.568+0000 7f9491000640 1 -- 172.21.15.71:0/2424374061 wait complete. 2024-09-16T07:27:07.584 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:27:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.743+0000 7f7083200640 1 -- 172.21.15.71:0/2418697811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7084074450 msgr2=0x7f708406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.743+0000 7f7083200640 1 --2- 172.21.15.71:0/2418697811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7084074450 0x7f708406ed60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f707c005fd0 tx=0x7f707c02aaa0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.743+0000 7f7083200640 1 -- 172.21.15.71:0/2418697811 shutdown_connections 2024-09-16T07:27:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.743+0000 7f7083200640 1 --2- 172.21.15.71:0/2418697811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7084074450 0x7f708406ed60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.743+0000 7f7083200640 1 --2- 172.21.15.71:0/2418697811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f708415bb90 0x7f70840b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.743+0000 7f7083200640 1 -- 172.21.15.71:0/2418697811 >> 172.21.15.71:0/2418697811 conn(0x7f70840b1510 msgr2=0x7f70840b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.743+0000 7f7083200640 1 -- 172.21.15.71:0/2418697811 shutdown_connections 2024-09-16T07:27:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.743+0000 7f7083200640 1 -- 172.21.15.71:0/2418697811 wait complete. 2024-09-16T07:27:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.745+0000 7f7083200640 1 Processor -- start 2024-09-16T07:27:07.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.745+0000 7f7083200640 1 -- start start 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7083200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084074450 0x7f70841f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7083200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840cefc0 0x7f70840cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7083200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708415bb90 0x7f7084158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7083200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70840cf770 con 0x7f708415bb90 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7083200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70840cf5f0 con 0x7f70840cefc0 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7083200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f708406f2a0 con 0x7f7084074450 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7082800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708415bb90 0x7f7084158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7082800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708415bb90 0x7f7084158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58916/0 (socket says 172.21.15.71:58916) 2024-09-16T07:27:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7082800640 1 -- 172.21.15.71:0/466054122 learned_addr learned my addr 172.21.15.71:0/466054122 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:07.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7081e00640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084074450 0x7f70841f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:07.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.746+0000 7f7081400640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840cefc0 0x7f70840cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f7082800640 1 -- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084074450 msgr2=0x7f70841f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f7082800640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084074450 0x7f70841f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f7082800640 1 -- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840cefc0 msgr2=0x7f70840cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f7082800640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840cefc0 0x7f70840cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f7082800640 1 -- 172.21.15.71:0/466054122 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708415d2b0 con 0x7f708415bb90 2024-09-16T07:27:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f7082800640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708415bb90 0x7f7084158c20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f707c006010 tx=0x7f707c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f707a000640 1 -- 172.21.15.71:0/466054122 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c046070 con 0x7f708415bb90 2024-09-16T07:27:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f707a000640 1 -- 172.21.15.71:0/466054122 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f707c004040 con 0x7f708415bb90 2024-09-16T07:27:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708415d590 con 0x7f708415bb90 2024-09-16T07:27:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f707a000640 1 -- 172.21.15.71:0/466054122 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c004430 con 0x7f708415bb90 2024-09-16T07:27:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.747+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708415db50 con 0x7f708415bb90 2024-09-16T07:27:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.749+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708415e320 con 0x7f708415bb90 2024-09-16T07:27:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.750+0000 7f707a000640 1 -- 172.21.15.71:0/466054122 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 14) ==== 101593+0+0 (secure 0 0 0) 0x7f707c049070 con 0x7f708415bb90 2024-09-16T07:27:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.751+0000 7f707a000640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7064078bb0 0x7f706407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.751+0000 7f707a000640 1 -- 172.21.15.71:0/466054122 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f707c0bbdd0 con 0x7f708415bb90 2024-09-16T07:27:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.751+0000 7f7081e00640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7064078bb0 0x7f706407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.752+0000 7f7081e00640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7064078bb0 0x7f706407b070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7074009b30 tx=0x7f7074005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:07.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.755+0000 7f707a000640 1 -- 172.21.15.71:0/466054122 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f707c032030 con 0x7f708415bb90 2024-09-16T07:27:07.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.970+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f70840b2940 con 0x7f7064078bb0 2024-09-16T07:27:07.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.977+0000 7f707a000640 1 -- 172.21.15.71:0/466054122 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f70840b2940 con 0x7f7064078bb0 2024-09-16T07:27:07.976 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7064078bb0 msgr2=0x7f706407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7064078bb0 0x7f706407b070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7074009b30 tx=0x7f7074005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708415bb90 msgr2=0x7f7084158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708415bb90 0x7f7084158c20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f707c006010 tx=0x7f707c003ce0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 shutdown_connections 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7064078bb0 0x7f706407b070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708415bb90 0x7f7084158c20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840cefc0 0x7f70840cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 --2- 172.21.15.71:0/466054122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084074450 0x7f70841f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:07.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 >> 172.21.15.71:0/466054122 conn(0x7f70840b1510 msgr2=0x7f70840afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:07.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 shutdown_connections 2024-09-16T07:27:07.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:07.979+0000 7f7083200640 1 -- 172.21.15.71:0/466054122 wait complete. 2024-09-16T07:27:07.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:27:07.994 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:27:07.994 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:27:08.030 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:27:08.030 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T07:27:08.030 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:08.032 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T07:27:10.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:27:10.033 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:27:10.033 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:27:10.074 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:27:10.074 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T07:27:10.074 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:10.075 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T07:27:12.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:27:12.077 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:27:12.077 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:27:12.117 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/cdba332b-c183-4953-a8d5-9c9bf1e099bb 2024-09-16T07:27:12.117 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="10737418240" 2024-09-16T07:27:12.117 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:12.118 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-16T07:27:12.119 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone28890 2024-09-16T07:27:12.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.329+0000 7f48cae00640 1 -- 172.21.15.71:0/2522841488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac003c10 msgr2=0x7f48ac004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:12.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.329+0000 7f48cae00640 1 --2- 172.21.15.71:0/2522841488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48ac003c10 0x7f48ac004010 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c610 tx=0x7f48ac005830 comp rx=0 tx=0).stop 2024-09-16T07:27:12.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.329+0000 7f48ca400640 1 -- 172.21.15.71:0/2522841488 reap_dead start 2024-09-16T07:27:12.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.329+0000 7f48cae00640 1 -- 172.21.15.71:0/2522841488 shutdown_connections 2024-09-16T07:27:12.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.329+0000 7f48cae00640 1 -- 172.21.15.71:0/2522841488 >> 172.21.15.71:0/2522841488 conn(0x7f48c40cc490 msgr2=0x7f48c40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:12.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.329+0000 7f48cae00640 1 -- 172.21.15.71:0/2522841488 shutdown_connections 2024-09-16T07:27:12.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.329+0000 7f48cae00640 1 -- 172.21.15.71:0/2522841488 wait complete. 2024-09-16T07:27:12.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.331+0000 7f48cae00640 1 Processor -- start 2024-09-16T07:27:12.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.331+0000 7f48cae00640 1 -- start start 2024-09-16T07:27:12.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48cae00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac003c10 0x7f48c41f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:12.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48cae00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48c41f4780 0x7f48c41f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:12.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48cae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c41f5120 0x7f48c41f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:12.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48cae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f48c406f220 con 0x7f48c41f5120 2024-09-16T07:27:12.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48cae00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48c40cd2b0 con 0x7f48c41f4780 2024-09-16T07:27:12.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48cae00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f48c406f3a0 con 0x7f48ac003c10 2024-09-16T07:27:12.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48ca400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c41f5120 0x7f48c41f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48ca400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c41f5120 0x7f48c41f7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58944/0 (socket says 172.21.15.71:58944) 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48ca400640 1 -- 172.21.15.71:0/1881252398 learned_addr learned my addr 172.21.15.71:0/1881252398 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48c9a00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac003c10 0x7f48c41f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.332+0000 7f48c9000640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48c41f4780 0x7f48c41f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48ca400640 1 -- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac003c10 msgr2=0x7f48c41f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48ca400640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac003c10 0x7f48c41f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48ca400640 1 -- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48c41f4780 msgr2=0x7f48c41f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48ca400640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48c41f4780 0x7f48c41f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48ca400640 1 -- 172.21.15.71:0/1881252398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c40c78b0 con 0x7f48c41f5120 2024-09-16T07:27:12.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48ca400640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c41f5120 0x7f48c41f7d70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f48c40b2720 tx=0x7f48bc006f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:12.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48c1600640 1 -- 172.21.15.71:0/1881252398 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc011660 con 0x7f48c41f5120 2024-09-16T07:27:12.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48c1600640 1 -- 172.21.15.71:0/1881252398 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48bc017070 con 0x7f48c41f5120 2024-09-16T07:27:12.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c40c7b90 con 0x7f48c41f5120 2024-09-16T07:27:12.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48c1600640 1 -- 172.21.15.71:0/1881252398 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc011bd0 con 0x7f48c41f5120 2024-09-16T07:27:12.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.333+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c40c8150 con 0x7f48c41f5120 2024-09-16T07:27:12.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.334+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c40c6510 con 0x7f48c41f5120 2024-09-16T07:27:12.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.335+0000 7f48c1600640 1 -- 172.21.15.71:0/1881252398 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) ==== 101644+0+0 (secure 0 0 0) 0x7f48bc01c460 con 0x7f48c41f5120 2024-09-16T07:27:12.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.336+0000 7f48c1600640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48a4078c40 0x7f48a407b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:12.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.336+0000 7f48c1600640 1 -- 172.21.15.71:0/1881252398 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f48bc09b6e0 con 0x7f48c41f5120 2024-09-16T07:27:12.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.337+0000 7f48c9a00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48a4078c40 0x7f48a407b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:12.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.337+0000 7f48c9a00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48a4078c40 0x7f48a407b100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c610 tx=0x7f48c40c4e20 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.341+0000 7f48c1600640 1 -- 172.21.15.71:0/1881252398 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f48bc063500 con 0x7f48c41f5120 2024-09-16T07:27:12.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.560+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone28890", "target": ["mon-mgr", ""]}) -- 0x7f48c40ccc60 con 0x7f48a4078c40 2024-09-16T07:27:12.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.643+0000 7f48c1600640 1 -- 172.21.15.71:0/1881252398 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f48c40ccc60 con 0x7f48a4078c40 2024-09-16T07:27:12.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48a4078c40 msgr2=0x7f48a407b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:12.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48a4078c40 0x7f48a407b100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c610 tx=0x7f48c40c4e20 comp rx=0 tx=0).stop 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c41f5120 msgr2=0x7f48c41f7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c41f5120 0x7f48c41f7d70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f48c40b2720 tx=0x7f48bc006f80 comp rx=0 tx=0).stop 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 shutdown_connections 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48a4078c40 0x7f48a407b100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c41f5120 0x7f48c41f7d70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48c41f4780 0x7f48c41f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 --2- 172.21.15.71:0/1881252398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac003c10 0x7f48c41f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 >> 172.21.15.71:0/1881252398 conn(0x7f48c40cc490 msgr2=0x7f48c40725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 shutdown_connections 2024-09-16T07:27:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.646+0000 7f48cae00640 1 -- 172.21.15.71:0/1881252398 wait complete. 2024-09-16T07:27:12.647 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.647+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.647 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.647+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.647 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.647+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.647 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.647+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.647 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.647+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.661 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone16181 2024-09-16T07:27:12.665 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.665+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.665 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.665+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.665 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.665+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.665 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.665+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.665 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:12.665+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:12.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.829+0000 7efc37200640 1 -- 172.21.15.71:0/202161901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc1c004d40 msgr2=0x7efc1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:12.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.829+0000 7efc37200640 1 --2- 172.21.15.71:0/202161901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc1c004d40 0x7efc1c0051c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efc30001e10 tx=0x7efc1c005830 comp rx=0 tx=0).stop 2024-09-16T07:27:12.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.829+0000 7efc36800640 1 -- 172.21.15.71:0/202161901 reap_dead start 2024-09-16T07:27:12.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.829+0000 7efc37200640 1 -- 172.21.15.71:0/202161901 shutdown_connections 2024-09-16T07:27:12.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.829+0000 7efc37200640 1 -- 172.21.15.71:0/202161901 >> 172.21.15.71:0/202161901 conn(0x7efc380cc490 msgr2=0x7efc380cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:12.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.829+0000 7efc37200640 1 -- 172.21.15.71:0/202161901 shutdown_connections 2024-09-16T07:27:12.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.829+0000 7efc37200640 1 -- 172.21.15.71:0/202161901 wait complete. 2024-09-16T07:27:12.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.831+0000 7efc37200640 1 Processor -- start 2024-09-16T07:27:12.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.831+0000 7efc37200640 1 -- start start 2024-09-16T07:27:12.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc37200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc381f72d0 0x7efc381f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:12.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc37200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc380b53a0 0x7efc38153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:12.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc37200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc381f7ab0 0x7efc381f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:12.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc37200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc3806f220 con 0x7efc381f72d0 2024-09-16T07:27:12.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc37200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efc380cd2b0 con 0x7efc381f7ab0 2024-09-16T07:27:12.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc37200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efc3806f3a0 con 0x7efc380b53a0 2024-09-16T07:27:12.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc381f72d0 0x7efc381f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:12.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc381f72d0 0x7efc381f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58962/0 (socket says 172.21.15.71:58962) 2024-09-16T07:27:12.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35e00640 1 -- 172.21.15.71:0/2161626073 learned_addr learned my addr 172.21.15.71:0/2161626073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:12.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35400640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc380b53a0 0x7efc38153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:12.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc36800640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc381f7ab0 0x7efc381f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35e00640 1 -- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc380b53a0 msgr2=0x7efc38153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35e00640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc380b53a0 0x7efc38153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35e00640 1 -- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc381f7ab0 msgr2=0x7efc381f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35e00640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc381f7ab0 0x7efc381f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc35e00640 1 -- 172.21.15.71:0/2161626073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc381f5b10 con 0x7efc381f72d0 2024-09-16T07:27:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.832+0000 7efc36800640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc381f7ab0 0x7efc381f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.833+0000 7efc35e00640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc381f72d0 0x7efc381f55d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efc28006480 tx=0x7efc28003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.833+0000 7efc2e000640 1 -- 172.21.15.71:0/2161626073 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28015070 con 0x7efc381f72d0 2024-09-16T07:27:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.833+0000 7efc2e000640 1 -- 172.21.15.71:0/2161626073 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc28011940 con 0x7efc381f72d0 2024-09-16T07:27:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.833+0000 7efc2e000640 1 -- 172.21.15.71:0/2161626073 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28011d90 con 0x7efc381f72d0 2024-09-16T07:27:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.833+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc381540d0 con 0x7efc381f72d0 2024-09-16T07:27:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.833+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc380b2de0 con 0x7efc381f72d0 2024-09-16T07:27:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.834+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc380c6510 con 0x7efc381f72d0 2024-09-16T07:27:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.835+0000 7efc2e000640 1 -- 172.21.15.71:0/2161626073 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) ==== 101644+0+0 (secure 0 0 0) 0x7efc28018070 con 0x7efc381f72d0 2024-09-16T07:27:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.837+0000 7efc2e000640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc18078c70 0x7efc1807b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.837+0000 7efc2e000640 1 -- 172.21.15.71:0/2161626073 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7efc2809c000 con 0x7efc381f72d0 2024-09-16T07:27:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.837+0000 7efc35400640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc18078c70 0x7efc1807b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.838+0000 7efc35400640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc18078c70 0x7efc1807b130 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc3815ecd0 tx=0x7efc24005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:12.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:12.841+0000 7efc2e000640 1 -- 172.21.15.71:0/2161626073 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc28063eb0 con 0x7efc381f72d0 2024-09-16T07:27:13.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.058+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone16181", "target": ["mon-mgr", ""]}) -- 0x7efc380cbfc0 con 0x7efc18078c70 2024-09-16T07:27:13.648 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:13.648+0000 7fe75e400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:13.648 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:13.648+0000 7fe75e400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:13.648 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:13.648+0000 7fe75e400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:13.648 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:13.648+0000 7fe75e400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:13.648 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:13.648+0000 7fe75e400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:13.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.746+0000 7efc2e000640 1 -- 172.21.15.71:0/2161626073 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc380cbfc0 con 0x7efc18078c70 2024-09-16T07:27:13.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc18078c70 msgr2=0x7efc1807b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:13.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc18078c70 0x7efc1807b130 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc3815ecd0 tx=0x7efc24005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:13.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc381f72d0 msgr2=0x7efc381f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:13.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc381f72d0 0x7efc381f55d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efc28006480 tx=0x7efc28003aa0 comp rx=0 tx=0).stop 2024-09-16T07:27:13.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 shutdown_connections 2024-09-16T07:27:13.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc18078c70 0x7efc1807b130 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:13.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc381f7ab0 0x7efc381f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:13.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc380b53a0 0x7efc38153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:13.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 --2- 172.21.15.71:0/2161626073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc381f72d0 0x7efc381f55d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:13.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.749+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 >> 172.21.15.71:0/2161626073 conn(0x7efc380cc490 msgr2=0x7efc38074380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:13.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.750+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 shutdown_connections 2024-09-16T07:27:13.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:13.750+0000 7efc37200640 1 -- 172.21.15.71:0/2161626073 wait complete. 2024-09-16T07:27:13.773 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6962 2024-09-16T07:27:14.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.008+0000 7f0a1d000640 1 -- 172.21.15.71:0/2359710592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a1815bf10 msgr2=0x7f0a181604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:14.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.008+0000 7f0a1d000640 1 --2- 172.21.15.71:0/2359710592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a1815bf10 0x7f0a181604b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0a180cc400 tx=0x7f0a1002a8e0 comp rx=0 tx=0).stop 2024-09-16T07:27:14.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.008+0000 7f0a1d000640 1 -- 172.21.15.71:0/2359710592 shutdown_connections 2024-09-16T07:27:14.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.008+0000 7f0a1d000640 1 --2- 172.21.15.71:0/2359710592 >> v2:172.21.15.177:3300/0 conn(0x7f0a180cd6f0 0x7f0a180d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:14.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.008+0000 7f0a1d000640 1 --2- 172.21.15.71:0/2359710592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a1815bf10 0x7f0a181604b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:14.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.008+0000 7f0a1d000640 1 -- 172.21.15.71:0/2359710592 >> 172.21.15.71:0/2359710592 conn(0x7f0a180d1e20 msgr2=0x7f0a180d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:14.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.008+0000 7f0a1d000640 1 -- 172.21.15.71:0/2359710592 shutdown_connections 2024-09-16T07:27:14.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.008+0000 7f0a1d000640 1 -- 172.21.15.71:0/2359710592 wait complete. 2024-09-16T07:27:14.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.010+0000 7f0a1d000640 1 Processor -- start 2024-09-16T07:27:14.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.010+0000 7f0a1d000640 1 -- start start 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a1d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a180cd6f0 0x7f0a1815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a1d000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a180d0710 0x7f0a18159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a1d000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a1815bf10 0x7f0a181f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a1d000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0a180cdce0 con 0x7f0a180cd6f0 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a1d000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a180cdb60 con 0x7f0a1815bf10 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a1d000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a180cf9f0 con 0x7f0a180d0710 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a15e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a180cd6f0 0x7f0a1815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a15e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a180cd6f0 0x7f0a1815f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58990/0 (socket says 172.21.15.71:58990) 2024-09-16T07:27:14.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a15e00640 1 -- 172.21.15.71:0/188825511 learned_addr learned my addr 172.21.15.71:0/188825511 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:14.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a15e00640 1 -- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a180d0710 msgr2=0x7f0a18159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:14.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a15e00640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a180d0710 0x7f0a18159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:14.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a15e00640 1 -- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a1815bf10 msgr2=0x7f0a181f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:14.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a15e00640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a1815bf10 0x7f0a181f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:14.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.011+0000 7f0a15e00640 1 -- 172.21.15.71:0/188825511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a18159950 con 0x7f0a180cd6f0 2024-09-16T07:27:14.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.012+0000 7f0a15e00640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a180cd6f0 0x7f0a1815f010 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ca0 tx=0x7f0a10030680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:14.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.012+0000 7f0a0e000640 1 -- 172.21.15.71:0/188825511 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10041070 con 0x7f0a180cd6f0 2024-09-16T07:27:14.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.012+0000 7f0a0e000640 1 -- 172.21.15.71:0/188825511 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a10030d70 con 0x7f0a180cd6f0 2024-09-16T07:27:14.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.012+0000 7f0a0e000640 1 -- 172.21.15.71:0/188825511 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1003b680 con 0x7f0a180cd6f0 2024-09-16T07:27:14.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.012+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1815e4a0 con 0x7f0a180cd6f0 2024-09-16T07:27:14.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.012+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a180c5d20 con 0x7f0a180cd6f0 2024-09-16T07:27:14.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.014+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a180d01d0 con 0x7f0a180cd6f0 2024-09-16T07:27:14.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.015+0000 7f0a0e000640 1 -- 172.21.15.71:0/188825511 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) ==== 101644+0+0 (secure 0 0 0) 0x7f0a10044070 con 0x7f0a180cd6f0 2024-09-16T07:27:14.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.018+0000 7f0a0e000640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f09f4078cc0 0x7f09f407b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:14.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.018+0000 7f0a0e000640 1 -- 172.21.15.71:0/188825511 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f0a100bc470 con 0x7f0a180cd6f0 2024-09-16T07:27:14.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.021+0000 7f0a0e000640 1 -- 172.21.15.71:0/188825511 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0a18159950 con 0x7f0a180cd6f0 2024-09-16T07:27:14.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.024+0000 7f0a15400640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f09f4078cc0 0x7f09f407b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:14.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.030+0000 7f0a15400640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f09f4078cc0 0x7f09f407b180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a180b36a0 tx=0x7f0a00006c90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:14.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:27:14.313 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:27:14.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.350+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6962", "target": ["mon-mgr", ""]}) -- 0x7f0a180cbf40 con 0x7f09f4078cc0 2024-09-16T07:27:14.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:14.539+0000 7f0a0e000640 1 -- 172.21.15.71:0/188825511 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 16) ==== 101797+0+0 (secure 0 0 0) 0x7f0a10039040 con 0x7f0a180cd6f0 2024-09-16T07:27:15.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.085+0000 7f0a0e000640 1 -- 172.21.15.71:0/188825511 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a180cbf40 con 0x7f09f4078cc0 2024-09-16T07:27:15.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f09f4078cc0 msgr2=0x7f09f407b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:15.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f09f4078cc0 0x7f09f407b180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a180b36a0 tx=0x7f0a00006c90 comp rx=0 tx=0).stop 2024-09-16T07:27:15.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a180cd6f0 msgr2=0x7f0a1815f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:15.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a180cd6f0 0x7f0a1815f010 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ca0 tx=0x7f0a10030680 comp rx=0 tx=0).stop 2024-09-16T07:27:15.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 shutdown_connections 2024-09-16T07:27:15.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f09f4078cc0 0x7f09f407b180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:15.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a1815bf10 0x7f0a181f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:15.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a180d0710 0x7f0a18159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:15.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 --2- 172.21.15.71:0/188825511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a180cd6f0 0x7f0a1815f010 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:15.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 >> 172.21.15.71:0/188825511 conn(0x7f0a180d1e20 msgr2=0x7f0a180709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:15.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 shutdown_connections 2024-09-16T07:27:15.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.088+0000 7f0a1d000640 1 -- 172.21.15.71:0/188825511 wait complete. 2024-09-16T07:27:15.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone29773 2024-09-16T07:27:15.142 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:15.142+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:15.142 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:15.142+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:15.142 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:15.142+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:15.142 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:15.142+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:15.142 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:15.142+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:15.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.354+0000 7fa878a00640 1 -- 172.21.15.71:0/1230381174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa87406ed00 msgr2=0x7fa8740ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:15.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.354+0000 7fa878a00640 1 --2- 172.21.15.71:0/1230381174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa87406ed00 0x7fa8740ce800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa864009830 tx=0x7fa86402a480 comp rx=0 tx=0).stop 2024-09-16T07:27:15.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.355+0000 7fa878a00640 1 -- 172.21.15.71:0/1230381174 shutdown_connections 2024-09-16T07:27:15.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.355+0000 7fa878a00640 1 --2- 172.21.15.71:0/1230381174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa87406ed00 0x7fa8740ce800 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:15.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.355+0000 7fa878a00640 1 --2- 172.21.15.71:0/1230381174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa874161e90 0x7fa8740d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:15.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.355+0000 7fa878a00640 1 -- 172.21.15.71:0/1230381174 >> 172.21.15.71:0/1230381174 conn(0x7fa8740d1e20 msgr2=0x7fa8740d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:15.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.355+0000 7fa878a00640 1 -- 172.21.15.71:0/1230381174 shutdown_connections 2024-09-16T07:27:15.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.355+0000 7fa878a00640 1 -- 172.21.15.71:0/1230381174 wait complete. 2024-09-16T07:27:15.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.358+0000 7fa878a00640 1 Processor -- start 2024-09-16T07:27:15.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.359+0000 7fa878a00640 1 -- start start 2024-09-16T07:27:15.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.361+0000 7fa878a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa87406ed00 0x7fa87415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:15.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.361+0000 7fa878a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8740d1000 0x7fa87415f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:15.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.361+0000 7fa878a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa874161e90 0x7fa87415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:15.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.361+0000 7fa878a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8741623f0 con 0x7fa874161e90 2024-09-16T07:27:15.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.361+0000 7fa878a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa874162270 con 0x7fa8740d1000 2024-09-16T07:27:15.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.361+0000 7fa878a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa874162570 con 0x7fa87406ed00 2024-09-16T07:27:15.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.362+0000 7fa872800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa874161e90 0x7fa87415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:15.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.362+0000 7fa872800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa874161e90 0x7fa87415be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35882/0 (socket says 172.21.15.71:35882) 2024-09-16T07:27:15.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.362+0000 7fa872800640 1 -- 172.21.15.71:0/4062529251 learned_addr learned my addr 172.21.15.71:0/4062529251 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:15.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.362+0000 7fa871400640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8740d1000 0x7fa87415f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:15.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.362+0000 7fa872800640 1 -- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa87406ed00 msgr2=0x7fa87415cd60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:27:15.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.362+0000 7fa872800640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa87406ed00 0x7fa87415cd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:15.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.363+0000 7fa872800640 1 -- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8740d1000 msgr2=0x7fa87415f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:15.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.363+0000 7fa872800640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8740d1000 0x7fa87415f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:15.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.363+0000 7fa872800640 1 -- 172.21.15.71:0/4062529251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa874160250 con 0x7fa874161e90 2024-09-16T07:27:15.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.363+0000 7fa872800640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa874161e90 0x7fa87415be30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00caa0 tx=0x7fa86c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:15.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.364+0000 7fa86a000640 1 -- 172.21.15.71:0/4062529251 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c003d80 con 0x7fa874161e90 2024-09-16T07:27:15.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.364+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa874160530 con 0x7fa874161e90 2024-09-16T07:27:15.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.364+0000 7fa86a000640 1 -- 172.21.15.71:0/4062529251 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa86c00a730 con 0x7fa874161e90 2024-09-16T07:27:15.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.364+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87409f920 con 0x7fa874161e90 2024-09-16T07:27:15.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.365+0000 7fa86a000640 1 -- 172.21.15.71:0/4062529251 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c00abc0 con 0x7fa874161e90 2024-09-16T07:27:15.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.366+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8740d0ac0 con 0x7fa874161e90 2024-09-16T07:27:15.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.372+0000 7fa86a000640 1 -- 172.21.15.71:0/4062529251 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 101797+0+0 (secure 0 0 0) 0x7fa86c005070 con 0x7fa874161e90 2024-09-16T07:27:15.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.375+0000 7fa86a000640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa854078df0 0x7fa85407b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:15.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.376+0000 7fa86a000640 1 -- 172.21.15.71:0/4062529251 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fa86c09c160 con 0x7fa874161e90 2024-09-16T07:27:15.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.379+0000 7fa86a000640 1 -- 172.21.15.71:0/4062529251 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa86c063e10 con 0x7fa874161e90 2024-09-16T07:27:15.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.386+0000 7fa871e00640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa854078df0 0x7fa85407b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:15.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.391+0000 7fa871e00640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa854078df0 0x7fa85407b2b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa86000b990 tx=0x7fa86000b370 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:15.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:15.761+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone29773", "target": ["mon-mgr", ""]}) -- 0x7fa874069630 con 0x7fa854078df0 2024-09-16T07:27:16.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:16.604+0000 7fa86a000640 1 -- 172.21.15.71:0/4062529251 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 17) ==== 101848+0+0 (secure 0 0 0) 0x7fa86c05f980 con 0x7fa874161e90 2024-09-16T07:27:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.348+0000 7fa86a000640 1 -- 172.21.15.71:0/4062529251 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa874069630 con 0x7fa854078df0 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa854078df0 msgr2=0x7fa85407b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa854078df0 0x7fa85407b2b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa86000b990 tx=0x7fa86000b370 comp rx=0 tx=0).stop 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa874161e90 msgr2=0x7fa87415be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa874161e90 0x7fa87415be30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00caa0 tx=0x7fa86c00cf60 comp rx=0 tx=0).stop 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 shutdown_connections 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa854078df0 0x7fa85407b2b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa874161e90 0x7fa87415be30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8740d1000 0x7fa87415f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 --2- 172.21.15.71:0/4062529251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa87406ed00 0x7fa87415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 >> 172.21.15.71:0/4062529251 conn(0x7fa8740d1e20 msgr2=0x7fa8740b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 shutdown_connections 2024-09-16T07:27:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.353+0000 7fa878a00640 1 -- 172.21.15.71:0/4062529251 wait complete. 2024-09-16T07:27:17.379 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:27:17.421 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:17.421+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:17.421 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:17.421+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:17.421 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:17.421+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:17.422 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:17.421+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:17.422 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:17.421+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:17.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.632+0000 7f11ed400640 1 -- 172.21.15.71:0/385163345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11e80d25d0 msgr2=0x7f11e80cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:17.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.632+0000 7f11ed400640 1 --2- 172.21.15.71:0/385163345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11e80d25d0 0x7f11e80cf9b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005fd0 tx=0x7f11d402a540 comp rx=0 tx=0).stop 2024-09-16T07:27:17.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.633+0000 7f11ed400640 1 -- 172.21.15.71:0/385163345 shutdown_connections 2024-09-16T07:27:17.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.633+0000 7f11ed400640 1 --2- 172.21.15.71:0/385163345 >> v2:172.21.15.177:3300/0 conn(0x7f11e80d1260 0x7f11e80d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:17.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.633+0000 7f11ed400640 1 --2- 172.21.15.71:0/385163345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11e80d25d0 0x7f11e80cf9b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:17.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.633+0000 7f11ed400640 1 -- 172.21.15.71:0/385163345 >> 172.21.15.71:0/385163345 conn(0x7f11e80d3e40 msgr2=0x7f11e80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:17.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.633+0000 7f11ed400640 1 -- 172.21.15.71:0/385163345 shutdown_connections 2024-09-16T07:27:17.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.633+0000 7f11ed400640 1 -- 172.21.15.71:0/385163345 wait complete. 2024-09-16T07:27:17.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.636+0000 7f11ed400640 1 Processor -- start 2024-09-16T07:27:17.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.636+0000 7f11ed400640 1 -- start start 2024-09-16T07:27:17.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.637+0000 7f11ed400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11e80d1260 0x7f11e806edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:17.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.637+0000 7f11ed400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e80d25d0 0x7f11e8159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:17.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.637+0000 7f11ed400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e815e050 0x7f11e81e9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:17.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.637+0000 7f11ed400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80b7120 con 0x7f11e80d25d0 2024-09-16T07:27:17.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.637+0000 7f11ed400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80b6fa0 con 0x7f11e815e050 2024-09-16T07:27:17.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.637+0000 7f11ed400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11e80b72a0 con 0x7f11e80d1260 2024-09-16T07:27:17.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.638+0000 7f11e7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e80d25d0 0x7f11e8159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:17.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.638+0000 7f11e7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e80d25d0 0x7f11e8159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35896/0 (socket says 172.21.15.71:35896) 2024-09-16T07:27:17.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.638+0000 7f11e7400640 1 -- 172.21.15.71:0/3427370683 learned_addr learned my addr 172.21.15.71:0/3427370683 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:17.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.639+0000 7f11e7400640 1 -- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11e80d1260 msgr2=0x7f11e806edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.639+0000 7f11e7400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11e80d1260 0x7f11e806edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.639+0000 7f11e7400640 1 -- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e815e050 msgr2=0x7f11e81e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.639+0000 7f11e7400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e815e050 0x7f11e81e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.639+0000 7f11e7400640 1 -- 172.21.15.71:0/3427370683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e81f8480 con 0x7f11e80d25d0 2024-09-16T07:27:17.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.640+0000 7f11e7400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e80d25d0 0x7f11e8159a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f11d4006010 tx=0x7f11d40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:17.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.641+0000 7f11dfe00640 1 -- 172.21.15.71:0/3427370683 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d403f070 con 0x7f11e80d25d0 2024-09-16T07:27:17.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.641+0000 7f11dfe00640 1 -- 172.21.15.71:0/3427370683 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11d4004890 con 0x7f11e80d25d0 2024-09-16T07:27:17.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.641+0000 7f11dfe00640 1 -- 172.21.15.71:0/3427370683 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4035030 con 0x7f11e80d25d0 2024-09-16T07:27:17.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.641+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e81f8760 con 0x7f11e80d25d0 2024-09-16T07:27:17.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.641+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81f8d20 con 0x7f11e80d25d0 2024-09-16T07:27:17.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.643+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e80b7790 con 0x7f11e80d25d0 2024-09-16T07:27:17.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.645+0000 7f11dfe00640 1 -- 172.21.15.71:0/3427370683 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 101848+0+0 (secure 0 0 0) 0x7f11d4042070 con 0x7f11e80d25d0 2024-09-16T07:27:17.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.647+0000 7f11dfe00640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11c8078ed0 0x7f11c807b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:17.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.647+0000 7f11dfe00640 1 -- 172.21.15.71:0/3427370683 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f11d40bc300 con 0x7f11e80d25d0 2024-09-16T07:27:17.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.653+0000 7f11dfe00640 1 -- 172.21.15.71:0/3427370683 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f11d4084100 con 0x7f11e80d25d0 2024-09-16T07:27:17.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.668+0000 7f11e7e00640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11c8078ed0 0x7f11c807b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:17.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:17.718+0000 7f11e7e00640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11c8078ed0 0x7f11c807b390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009650 tx=0x7f11d8006ba0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:18.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.022+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f11e80d0630 con 0x7f11e80d25d0 2024-09-16T07:27:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.022+0000 7f11dfe00640 1 -- 172.21.15.71:0/3427370683 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2161 (secure 0 0 0) 0x7f11d4083a00 con 0x7f11e80d25d0 2024-09-16T07:27:18.022 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:18.022 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:27:18.022 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:27:18.022 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:27:18.023 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 294, 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 50, 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:27:18.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 2587, 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 10738581207, 2024-09-16T07:27:18.025 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 21775687680, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 751318425600, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 170, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 89430577, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 0, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 22 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 28, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:25:48:609067+0000", 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 3, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:27:18.026 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 3, 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "a", 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 5433 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:27:18.027 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 10, 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:26:48.354224+0000", 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:18.028 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:27:18.029 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:27:18.029 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "1 ongoing clones - average progress is 0.859% (0s)\n [............................] ", 2024-09-16T07:27:18.029 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.0085899997502565384, 2024-09-16T07:27:18.029 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:27:18.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:18.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:18.029 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11c8078ed0 msgr2=0x7f11c807b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11c8078ed0 0x7f11c807b390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009650 tx=0x7f11d8006ba0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e80d25d0 msgr2=0x7f11e8159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e80d25d0 0x7f11e8159a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f11d4006010 tx=0x7f11d40039e0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 shutdown_connections 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11c8078ed0 0x7f11c807b390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e815e050 0x7f11e81e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e80d25d0 0x7f11e8159a10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 --2- 172.21.15.71:0/3427370683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11e80d1260 0x7f11e806edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 >> 172.21.15.71:0/3427370683 conn(0x7f11e80d3e40 msgr2=0x7f11e80b8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:18.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 shutdown_connections 2024-09-16T07:27:18.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.026+0000 7f11ed400640 1 -- 172.21.15.71:0/3427370683 wait complete. 2024-09-16T07:27:18.052 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone28890 2024-09-16T07:27:18.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.303+0000 7fc27c800640 1 -- 172.21.15.71:0/133109530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 msgr2=0x7fc2740ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:18.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.303+0000 7fc27c800640 1 --2- 172.21.15.71:0/133109530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 0x7fc2740ce4a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc268009830 tx=0x7fc26802a520 comp rx=0 tx=0).stop 2024-09-16T07:27:18.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.304+0000 7fc27c800640 1 -- 172.21.15.71:0/133109530 shutdown_connections 2024-09-16T07:27:18.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.304+0000 7fc27c800640 1 --2- 172.21.15.71:0/133109530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2740af190 0x7fc2740b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.304+0000 7fc27c800640 1 --2- 172.21.15.71:0/133109530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 0x7fc2740ce4a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.304+0000 7fc27c800640 1 -- 172.21.15.71:0/133109530 >> 172.21.15.71:0/133109530 conn(0x7fc2740b1840 msgr2=0x7fc2740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:18.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.304+0000 7fc27c800640 1 -- 172.21.15.71:0/133109530 shutdown_connections 2024-09-16T07:27:18.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.304+0000 7fc27c800640 1 -- 172.21.15.71:0/133109530 wait complete. 2024-09-16T07:27:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.305+0000 7fc27c800640 1 Processor -- start 2024-09-16T07:27:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.306+0000 7fc27c800640 1 -- start start 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27c800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2740af190 0x7fc274158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27c800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2740cec60 0x7fc274073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27c800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 0x7fc2741f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27c800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc274162a20 con 0x7fc274161760 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27c800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741628a0 con 0x7fc2740cec60 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27c800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc274162ba0 con 0x7fc2740af190 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 0x7fc2741f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 0x7fc2741f8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35910/0 (socket says 172.21.15.71:35910) 2024-09-16T07:27:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27a800640 1 -- 172.21.15.71:0/3226447180 learned_addr learned my addr 172.21.15.71:0/3226447180 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27a800640 1 -- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2740af190 msgr2=0x7fc274158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27a800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2740af190 0x7fc274158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27a800640 1 -- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2740cec60 msgr2=0x7fc274073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27a800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2740cec60 0x7fc274073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.307+0000 7fc27a800640 1 -- 172.21.15.71:0/3226447180 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc274072990 con 0x7fc274161760 2024-09-16T07:27:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.308+0000 7fc27a800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 0x7fc2741f8280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c770 tx=0x7fc27000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.308+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270017070 con 0x7fc274161760 2024-09-16T07:27:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.308+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc270005070 con 0x7fc274161760 2024-09-16T07:27:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.308+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc274072c70 con 0x7fc274161760 2024-09-16T07:27:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.308+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2700054c0 con 0x7fc274161760 2024-09-16T07:27:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.308+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc274073510 con 0x7fc274161760 2024-09-16T07:27:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.309+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc274073710 con 0x7fc274161760 2024-09-16T07:27:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.311+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 101848+0+0 (secure 0 0 0) 0x7fc27001a070 con 0x7fc274161760 2024-09-16T07:27:18.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.312+0000 7fc26e000640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc258078e80 0x7fc25807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:18.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.312+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fc27009ac50 con 0x7fc274161760 2024-09-16T07:27:18.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.317+0000 7fc279e00640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc258078e80 0x7fc25807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:18.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.318+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc270062870 con 0x7fc274161760 2024-09-16T07:27:18.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.333+0000 7fc279e00640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc258078e80 0x7fc25807b340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc268034000 tx=0x7fc2680046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:18.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.654+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 18) ==== 101899+0+0 (secure 0 0 0) 0x7fc27005e3e0 con 0x7fc274161760 2024-09-16T07:27:18.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:18.665+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone28890", "target": ["mon-mgr", ""]}) -- 0x7fc2740cf290 con 0x7fc258078e80 2024-09-16T07:27:19.035 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:19.035+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:19.036 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:19.036+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:19.036 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:19.036+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:19.036 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:19.036+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:19.036 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:19.036+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:20.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:20.697+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7fc2700054a0 con 0x7fc274161760 2024-09-16T07:27:21.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.475+0000 7fc26e000640 1 -- 172.21.15.71:0/3226447180 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc2740cf290 con 0x7fc258078e80 2024-09-16T07:27:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.479+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc258078e80 msgr2=0x7fc25807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.479+0000 7fc27c800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc258078e80 0x7fc25807b340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc268034000 tx=0x7fc2680046c0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.479+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 msgr2=0x7fc2741f8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.479+0000 7fc27c800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 0x7fc2741f8280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c770 tx=0x7fc27000cc30 comp rx=0 tx=0).stop 2024-09-16T07:27:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.480+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 shutdown_connections 2024-09-16T07:27:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.480+0000 7fc27c800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc258078e80 0x7fc25807b340 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.480+0000 7fc27c800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc274161760 0x7fc2741f8280 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.480+0000 7fc27c800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2740cec60 0x7fc274073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.480+0000 7fc27c800640 1 --2- 172.21.15.71:0/3226447180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2740af190 0x7fc274158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.480+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 >> 172.21.15.71:0/3226447180 conn(0x7fc2740b1840 msgr2=0x7fc2740ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.481+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 shutdown_connections 2024-09-16T07:27:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.481+0000 7fc27c800640 1 -- 172.21.15.71:0/3226447180 wait complete. 2024-09-16T07:27:21.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone16181 2024-09-16T07:27:21.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.786+0000 7f9a4e200640 1 -- 172.21.15.71:0/3425500637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4815c080 msgr2=0x7f9a480cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:21.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.786+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3425500637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4815c080 0x7f9a480cf120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9a30005fd0 tx=0x7f9a3002a5d0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.786+0000 7f9a4e200640 1 -- 172.21.15.71:0/3425500637 shutdown_connections 2024-09-16T07:27:21.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.786+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3425500637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4815c080 0x7f9a480cf120 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.786+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3425500637 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a480b3370 0x7f9a480cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.786+0000 7f9a4e200640 1 -- 172.21.15.71:0/3425500637 >> 172.21.15.71:0/3425500637 conn(0x7f9a480b1840 msgr2=0x7f9a480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:21.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.786+0000 7f9a4e200640 1 -- 172.21.15.71:0/3425500637 shutdown_connections 2024-09-16T07:27:21.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.786+0000 7f9a4e200640 1 -- 172.21.15.71:0/3425500637 wait complete. 2024-09-16T07:27:21.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.788+0000 7f9a4e200640 1 Processor -- start 2024-09-16T07:27:21.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.789+0000 7f9a4e200640 1 -- start start 2024-09-16T07:27:21.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a4e200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a480b3370 0x7f9a4813a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:21.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a4e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4815c080 0x7f9a4815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:21.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a4e200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4815e980 0x7f9a4815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:21.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a4e200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48164e20 con 0x7f9a4815c080 2024-09-16T07:27:21.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a4e200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48164ca0 con 0x7f9a480b3370 2024-09-16T07:27:21.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a4e200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9a48164fa0 con 0x7f9a4815e980 2024-09-16T07:27:21.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a46a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4815c080 0x7f9a4815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:21.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a46a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4815c080 0x7f9a4815d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35914/0 (socket says 172.21.15.71:35914) 2024-09-16T07:27:21.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.790+0000 7f9a46a00640 1 -- 172.21.15.71:0/3124232192 learned_addr learned my addr 172.21.15.71:0/3124232192 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:21.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.791+0000 7f9a46a00640 1 -- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4815e980 msgr2=0x7f9a4815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:21.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.791+0000 7f9a46a00640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4815e980 0x7f9a4815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.791+0000 7f9a46a00640 1 -- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a480b3370 msgr2=0x7f9a4813a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:21.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.791+0000 7f9a46a00640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a480b3370 0x7f9a4813a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:21.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.791+0000 7f9a46a00640 1 -- 172.21.15.71:0/3124232192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a481f52f0 con 0x7f9a4815c080 2024-09-16T07:27:21.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.791+0000 7f9a46a00640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4815c080 0x7f9a4815d5f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009850 tx=0x7f9a300035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:21.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.792+0000 7f9a3f400640 1 -- 172.21.15.71:0/3124232192 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3003f070 con 0x7f9a4815c080 2024-09-16T07:27:21.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.792+0000 7f9a3f400640 1 -- 172.21.15.71:0/3124232192 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a30003ce0 con 0x7f9a4815c080 2024-09-16T07:27:21.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.792+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a481f55d0 con 0x7f9a4815c080 2024-09-16T07:27:21.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.792+0000 7f9a3f400640 1 -- 172.21.15.71:0/3124232192 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30035030 con 0x7f9a4815c080 2024-09-16T07:27:21.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.792+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a481f5b90 con 0x7f9a4815c080 2024-09-16T07:27:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.794+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4815ee10 con 0x7f9a4815c080 2024-09-16T07:27:21.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.795+0000 7f9a3f400640 1 -- 172.21.15.71:0/3124232192 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7f9a30042070 con 0x7f9a4815c080 2024-09-16T07:27:21.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.796+0000 7f9a3f400640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9a24078fa0 0x7f9a2407b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:21.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.796+0000 7f9a3f400640 1 -- 172.21.15.71:0/3124232192 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f9a300bc700 con 0x7f9a4815c080 2024-09-16T07:27:21.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.802+0000 7f9a3f400640 1 -- 172.21.15.71:0/3124232192 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9a30083e20 con 0x7f9a4815c080 2024-09-16T07:27:21.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.804+0000 7f9a47400640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9a24078fa0 0x7f9a2407b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:21.811+0000 7f9a47400640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9a24078fa0 0x7f9a2407b460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a38005c40 tx=0x7f9a38005bd0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:22.151+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone16181", "target": ["mon-mgr", ""]}) -- 0x7f9a4813c880 con 0x7f9a24078fa0 2024-09-16T07:27:23.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.003+0000 7f9a3f400640 1 -- 172.21.15.71:0/3124232192 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9a4813c880 con 0x7f9a24078fa0 2024-09-16T07:27:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9a24078fa0 msgr2=0x7f9a2407b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9a24078fa0 0x7f9a2407b460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a38005c40 tx=0x7f9a38005bd0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4815c080 msgr2=0x7f9a4815d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4815c080 0x7f9a4815d5f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009850 tx=0x7f9a300035f0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 shutdown_connections 2024-09-16T07:27:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9a24078fa0 0x7f9a2407b460 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a38005c40 tx=0x7f9a38005bd0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4815e980 0x7f9a4815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4815c080 0x7f9a4815d5f0 secure :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009850 tx=0x7f9a300035f0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 --2- 172.21.15.71:0/3124232192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a480b3370 0x7f9a4813a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 >> 172.21.15.71:0/3124232192 conn(0x7f9a480b1840 msgr2=0x7f9a480b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 shutdown_connections 2024-09-16T07:27:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.009+0000 7f9a4e200640 1 -- 172.21.15.71:0/3124232192 wait complete. 2024-09-16T07:27:23.049 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone6962 2024-09-16T07:27:23.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 -- 172.21.15.71:0/275372295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 msgr2=0x7fb3800cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:23.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 --2- 172.21.15.71:0/275372295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 0x7fb3800cebe0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb380152410 tx=0x7fb36802a580 comp rx=0 tx=0).stop 2024-09-16T07:27:23.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 -- 172.21.15.71:0/275372295 shutdown_connections 2024-09-16T07:27:23.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 --2- 172.21.15.71:0/275372295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb38015e980 0x7fb38015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 --2- 172.21.15.71:0/275372295 >> v2:172.21.15.177:3300/0 conn(0x7fb38015c080 0x7fb3800cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 --2- 172.21.15.71:0/275372295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 0x7fb3800cebe0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 -- 172.21.15.71:0/275372295 >> 172.21.15.71:0/275372295 conn(0x7fb3800b1840 msgr2=0x7fb3800b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:23.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 -- 172.21.15.71:0/275372295 shutdown_connections 2024-09-16T07:27:23.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.271+0000 7fb384c00640 1 -- 172.21.15.71:0/275372295 wait complete. 2024-09-16T07:27:23.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.274+0000 7fb384c00640 1 Processor -- start 2024-09-16T07:27:23.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.274+0000 7fb384c00640 1 -- start start 2024-09-16T07:27:23.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.275+0000 7fb384c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 0x7fb380073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:23.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.275+0000 7fb384c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb38015c080 0x7fb38006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:23.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.275+0000 7fb384c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb38015e980 0x7fb3801f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.275+0000 7fb384c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb380140770 con 0x7fb3800b3370 2024-09-16T07:27:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.275+0000 7fb384c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801405f0 con 0x7fb38015e980 2024-09-16T07:27:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.275+0000 7fb384c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb3801408f0 con 0x7fb38015c080 2024-09-16T07:27:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.276+0000 7fb37de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 0x7fb380073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.276+0000 7fb37de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 0x7fb380073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35932/0 (socket says 172.21.15.71:35932) 2024-09-16T07:27:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.276+0000 7fb37de00640 1 -- 172.21.15.71:0/4147768927 learned_addr learned my addr 172.21.15.71:0/4147768927 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:23.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.276+0000 7fb37de00640 1 -- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb38015c080 msgr2=0x7fb38006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:23.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.276+0000 7fb37de00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb38015c080 0x7fb38006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.276+0000 7fb37de00640 1 -- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb38015e980 msgr2=0x7fb3801f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:23.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.276+0000 7fb37de00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb38015e980 0x7fb3801f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:23.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.276+0000 7fb37de00640 1 -- 172.21.15.71:0/4147768927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38006e7e0 con 0x7fb3800b3370 2024-09-16T07:27:23.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.277+0000 7fb37de00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 0x7fb380073a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb368009850 tx=0x7fb3680039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:23.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.277+0000 7fb376000640 1 -- 172.21.15.71:0/4147768927 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36803f070 con 0x7fb3800b3370 2024-09-16T07:27:23.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.277+0000 7fb376000640 1 -- 172.21.15.71:0/4147768927 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb368004890 con 0x7fb3800b3370 2024-09-16T07:27:23.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.277+0000 7fb376000640 1 -- 172.21.15.71:0/4147768927 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368035030 con 0x7fb3800b3370 2024-09-16T07:27:23.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.277+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38006eac0 con 0x7fb3800b3370 2024-09-16T07:27:23.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.277+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38006f080 con 0x7fb3800b3370 2024-09-16T07:27:23.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.278+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38006f280 con 0x7fb3800b3370 2024-09-16T07:27:23.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.281+0000 7fb376000640 1 -- 172.21.15.71:0/4147768927 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7fb368042070 con 0x7fb3800b3370 2024-09-16T07:27:23.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.282+0000 7fb376000640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb35c078fa0 0x7fb35c07b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:23.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.282+0000 7fb376000640 1 -- 172.21.15.71:0/4147768927 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fb3680bcac0 con 0x7fb3800b3370 2024-09-16T07:27:23.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.289+0000 7fb376000640 1 -- 172.21.15.71:0/4147768927 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb368084120 con 0x7fb3800b3370 2024-09-16T07:27:23.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.314+0000 7fb377e00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb35c078fa0 0x7fb35c07b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:23.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.317+0000 7fb377e00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb35c078fa0 0x7fb35c07b460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3801ebf10 tx=0x7fb370006c90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:23.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:23.691+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone6962", "target": ["mon-mgr", ""]}) -- 0x7fb38015ee30 con 0x7fb35c078fa0 2024-09-16T07:27:24.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.314+0000 7fb376000640 1 -- 172.21.15.71:0/4147768927 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb38015ee30 con 0x7fb35c078fa0 2024-09-16T07:27:24.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb35c078fa0 msgr2=0x7fb35c07b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:24.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb35c078fa0 0x7fb35c07b460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3801ebf10 tx=0x7fb370006c90 comp rx=0 tx=0).stop 2024-09-16T07:27:24.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 msgr2=0x7fb380073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:24.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 0x7fb380073a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb368009850 tx=0x7fb3680039e0 comp rx=0 tx=0).stop 2024-09-16T07:27:24.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 shutdown_connections 2024-09-16T07:27:24.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb35c078fa0 0x7fb35c07b460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:24.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb38015e980 0x7fb3801f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:24.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb38015c080 0x7fb38006fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:24.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 --2- 172.21.15.71:0/4147768927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3800b3370 0x7fb380073a40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:24.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.317+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 >> 172.21.15.71:0/4147768927 conn(0x7fb3800b1840 msgr2=0x7fb3800adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:24.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.318+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 shutdown_connections 2024-09-16T07:27:24.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.318+0000 7fb384c00640 1 -- 172.21.15.71:0/4147768927 wait complete. 2024-09-16T07:27:24.344 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone29773 2024-09-16T07:27:24.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.603+0000 7f0fffe00640 1 -- 172.21.15.71:0/560902951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fe0003a50 msgr2=0x7f0fe0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:24.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.603+0000 7f0fffe00640 1 --2- 172.21.15.71:0/560902951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fe0003a50 0x7f0fe0003e70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0099d0 tx=0x7f0fe0005830 comp rx=0 tx=0).stop 2024-09-16T07:27:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.603+0000 7f0ffde00640 1 -- 172.21.15.71:0/560902951 reap_dead start 2024-09-16T07:27:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.604+0000 7f0fffe00640 1 -- 172.21.15.71:0/560902951 shutdown_connections 2024-09-16T07:27:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.604+0000 7f0fffe00640 1 -- 172.21.15.71:0/560902951 >> 172.21.15.71:0/560902951 conn(0x7f0ff80b1840 msgr2=0x7f0ff80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.604+0000 7f0fffe00640 1 -- 172.21.15.71:0/560902951 shutdown_connections 2024-09-16T07:27:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.604+0000 7f0fffe00640 1 -- 172.21.15.71:0/560902951 wait complete. 2024-09-16T07:27:24.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.607+0000 7f0fffe00640 1 Processor -- start 2024-09-16T07:27:24.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.607+0000 7f0fffe00640 1 -- start start 2024-09-16T07:27:24.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.609+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff80b43e0 0x7f0ff80c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:24.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.609+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ff80c5ce0 0x7f0ff813fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:24.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.609+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ff80b47c0 0x7f0ff8140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:24.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.609+0000 7f0fffe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff80b26b0 con 0x7f0ff80b43e0 2024-09-16T07:27:24.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.609+0000 7f0fffe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff80cedd0 con 0x7f0ff80b47c0 2024-09-16T07:27:24.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.609+0000 7f0fffe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff80cc5c0 con 0x7f0ff80c5ce0 2024-09-16T07:27:24.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.611+0000 7f0ffde00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ff80b47c0 0x7f0ff8140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:24.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.611+0000 7f0ffca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ff80c5ce0 0x7f0ff813fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:24.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.611+0000 7f0ffca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ff80c5ce0 0x7f0ff813fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:48604/0 (socket says 172.21.15.71:48604) 2024-09-16T07:27:24.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.611+0000 7f0ffca00640 1 -- 172.21.15.71:0/3600084757 learned_addr learned my addr 172.21.15.71:0/3600084757 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.612+0000 7f0ffde00640 1 -- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ff80c5ce0 msgr2=0x7f0ff813fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.612+0000 7f0ffde00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ff80c5ce0 0x7f0ff813fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.613+0000 7f0ffde00640 1 -- 172.21.15.71:0/3600084757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff80b43e0 msgr2=0x7f0ff80c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.613+0000 7f0ffde00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff80b43e0 0x7f0ff80c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.613+0000 7f0ffde00640 1 -- 172.21.15.71:0/3600084757 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff81f5820 con 0x7f0ff80b47c0 2024-09-16T07:27:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.613+0000 7f0ffca00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ff80c5ce0 0x7f0ff813fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:27:24.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.615+0000 7f0ffde00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ff80b47c0 0x7f0ff8140040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8074ff0 tx=0x7f0ff000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:24.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.617+0000 7f0ff5600640 1 -- 172.21.15.71:0/3600084757 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0017070 con 0x7f0ff80b47c0 2024-09-16T07:27:24.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.617+0000 7f0ff5600640 1 -- 172.21.15.71:0/3600084757 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ff000acf0 con 0x7f0ff80b47c0 2024-09-16T07:27:24.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.617+0000 7f0ff5600640 1 -- 172.21.15.71:0/3600084757 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00072d0 con 0x7f0ff80b47c0 2024-09-16T07:27:24.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.617+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff80c4cc0 con 0x7f0ff80b47c0 2024-09-16T07:27:24.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.617+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff81f4ac0 con 0x7f0ff80b47c0 2024-09-16T07:27:24.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.619+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff80c4ab0 con 0x7f0ff80b47c0 2024-09-16T07:27:24.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.624+0000 7f0ff5600640 1 -- 172.21.15.71:0/3600084757 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7f0ff001a070 con 0x7f0ff80b47c0 2024-09-16T07:27:24.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.626+0000 7f0ff5600640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fd8078fa0 0x7f0fd807b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:24.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.626+0000 7f0ff5600640 1 -- 172.21.15.71:0/3600084757 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f0ff009bbb0 con 0x7f0ff80b47c0 2024-09-16T07:27:24.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.628+0000 7f0ffd400640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fd8078fa0 0x7f0fd807b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:24.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.630+0000 7f0ffd400640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fd8078fa0 0x7f0fd807b460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fec034000 tx=0x7f0fec0047d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:24.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.632+0000 7f0ff5600640 1 -- 172.21.15.71:0/3600084757 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0ff0021020 con 0x7f0ff80b47c0 2024-09-16T07:27:24.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:24.993+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone29773", "target": ["mon-mgr", ""]}) -- 0x7f0ff80d04d0 con 0x7f0fd8078fa0 2024-09-16T07:27:25.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.127+0000 7f0ff5600640 1 -- 172.21.15.71:0/3600084757 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ff80d04d0 con 0x7f0fd8078fa0 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fd8078fa0 msgr2=0x7f0fd807b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fd8078fa0 0x7f0fd807b460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fec034000 tx=0x7f0fec0047d0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ff80b47c0 msgr2=0x7f0ff8140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ff80b47c0 0x7f0ff8140040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8074ff0 tx=0x7f0ff000cf90 comp rx=0 tx=0).stop 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 shutdown_connections 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fd8078fa0 0x7f0fd807b460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ff80b47c0 0x7f0ff8140040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ff80c5ce0 0x7f0ff813fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 --2- 172.21.15.71:0/3600084757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff80b43e0 0x7f0ff80c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 >> 172.21.15.71:0/3600084757 conn(0x7f0ff80b1840 msgr2=0x7f0ff80aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 shutdown_connections 2024-09-16T07:27:25.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.130+0000 7f0fffe00640 1 -- 172.21.15.71:0/3600084757 wait complete. 2024-09-16T07:27:25.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:27:25.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.305+0000 7f3d43200640 1 -- 172.21.15.71:0/491862662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c0ccf00 msgr2=0x7f3d3c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.305+0000 7f3d43200640 1 --2- 172.21.15.71:0/491862662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c0ccf00 0x7f3d3c0b2440 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009830 tx=0x7f3d2802a530 comp rx=0 tx=0).stop 2024-09-16T07:27:25.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.305+0000 7f3d43200640 1 -- 172.21.15.71:0/491862662 shutdown_connections 2024-09-16T07:27:25.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.305+0000 7f3d43200640 1 --2- 172.21.15.71:0/491862662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c0ccf00 0x7f3d3c0b2440 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.305+0000 7f3d43200640 1 -- 172.21.15.71:0/491862662 >> 172.21.15.71:0/491862662 conn(0x7f3d3c0b1950 msgr2=0x7f3d3c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:25.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.305+0000 7f3d43200640 1 -- 172.21.15.71:0/491862662 shutdown_connections 2024-09-16T07:27:25.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.305+0000 7f3d43200640 1 -- 172.21.15.71:0/491862662 wait complete. 2024-09-16T07:27:25.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.307+0000 7f3d43200640 1 Processor -- start 2024-09-16T07:27:25.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.307+0000 7f3d43200640 1 -- start start 2024-09-16T07:27:25.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d43200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c073fb0 0x7f3d3c15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d43200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d3c0ccf00 0x7f3d3c1f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d43200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15f5a0 0x7f3d3c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d43200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c06f9f0 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d43200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c06f870 con 0x7f3d3c073fb0 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d43200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3c074420 con 0x7f3d3c0ccf00 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d3be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c073fb0 0x7f3d3c15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d41200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15f5a0 0x7f3d3c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d3be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c073fb0 0x7f3d3c15d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:50986/0 (socket says 172.21.15.71:50986) 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d3be00640 1 -- 172.21.15.71:0/3093689448 learned_addr learned my addr 172.21.15.71:0/3093689448 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.308+0000 7f3d3b400640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d3c0ccf00 0x7f3d3c1f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d41200640 1 -- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d3c0ccf00 msgr2=0x7f3d3c1f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d41200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d3c0ccf00 0x7f3d3c1f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d41200640 1 -- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c073fb0 msgr2=0x7f3d3c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d41200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c073fb0 0x7f3d3c15d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d41200640 1 -- 172.21.15.71:0/3093689448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c1f5500 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d3be00640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c073fb0 0x7f3d3c15d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d41200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15f5a0 0x7f3d3c0cfcb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c3a0 tx=0x7f3d3400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d33e00640 1 -- 172.21.15.71:0/3093689448 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34017070 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d33e00640 1 -- 172.21.15.71:0/3093689448 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d340035f0 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d33e00640 1 -- 172.21.15.71:0/3093689448 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34003a40 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c1f57e0 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.309+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c1f6100 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.310+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3c0b2440 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.311+0000 7f3d33e00640 1 -- 172.21.15.71:0/3093689448 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7f3d3401a070 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.312+0000 7f3d33e00640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d1c078fa0 0x7f3d1c07b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.312+0000 7f3d33e00640 1 -- 172.21.15.71:0/3093689448 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f3d3409add0 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.313+0000 7f3d3be00640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d1c078fa0 0x7f3d1c07b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.313+0000 7f3d3be00640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d1c078fa0 0x7f3d1c07b460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0012e0 tx=0x7f3d2c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:25.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.316+0000 7f3d33e00640 1 -- 172.21.15.71:0/3093689448 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3d34061d10 con 0x7f3d3c15f5a0 2024-09-16T07:27:25.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.525+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3d3c09dd80 con 0x7f3d1c078fa0 2024-09-16T07:27:25.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.527+0000 7f3d33e00640 1 -- 172.21.15.71:0/3093689448 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3d3c09dd80 con 0x7f3d1c078fa0 2024-09-16T07:27:25.526 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:27:25.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:25.526 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:27:25.526 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:25.526 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d1c078fa0 msgr2=0x7f3d1c07b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d1c078fa0 0x7f3d1c07b460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0012e0 tx=0x7f3d2c009040 comp rx=0 tx=0).stop 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15f5a0 msgr2=0x7f3d3c0cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15f5a0 0x7f3d3c0cfcb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c3a0 tx=0x7f3d3400c860 comp rx=0 tx=0).stop 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 shutdown_connections 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d1c078fa0 0x7f3d1c07b460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15f5a0 0x7f3d3c0cfcb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d3c0ccf00 0x7f3d3c1f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 --2- 172.21.15.71:0/3093689448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d3c073fb0 0x7f3d3c15d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 >> 172.21.15.71:0/3093689448 conn(0x7f3d3c0b1950 msgr2=0x7f3d3c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:25.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 shutdown_connections 2024-09-16T07:27:25.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.529+0000 7f3d43200640 1 -- 172.21.15.71:0/3093689448 wait complete. 2024-09-16T07:27:25.544 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:27:25.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.706+0000 7f2394a00640 1 -- 172.21.15.71:0/3764693976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23900cbc00 msgr2=0x7f239006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.706+0000 7f2394a00640 1 --2- 172.21.15.71:0/3764693976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23900cbc00 0x7f239006fce0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2378005fd0 tx=0x7f237802a5d0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.706+0000 7f2394a00640 1 -- 172.21.15.71:0/3764693976 shutdown_connections 2024-09-16T07:27:25.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.706+0000 7f2394a00640 1 --2- 172.21.15.71:0/3764693976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390072560 0x7f2390073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.706+0000 7f2394a00640 1 --2- 172.21.15.71:0/3764693976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23900cbc00 0x7f239006fce0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.706+0000 7f2394a00640 1 -- 172.21.15.71:0/3764693976 >> 172.21.15.71:0/3764693976 conn(0x7f23900cf9f0 msgr2=0x7f23900cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:25.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.706+0000 7f2394a00640 1 -- 172.21.15.71:0/3764693976 shutdown_connections 2024-09-16T07:27:25.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.706+0000 7f2394a00640 1 -- 172.21.15.71:0/3764693976 wait complete. 2024-09-16T07:27:25.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.707+0000 7f2394a00640 1 Processor -- start 2024-09-16T07:27:25.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.708+0000 7f2394a00640 1 -- start start 2024-09-16T07:27:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.708+0000 7f2394a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2390072560 0x7f239014b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.708+0000 7f2394a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23900cbc00 0x7f23901e4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.708+0000 7f2394a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f239015bec0 0x7f239014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.708+0000 7f2394a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f23900ae960 con 0x7f2390072560 2024-09-16T07:27:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f2394a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23900ae7e0 con 0x7f239015bec0 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f2394a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f23900aeae0 con 0x7f23900cbc00 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2390072560 0x7f239014b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2390072560 0x7f239014b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46142/0 (socket says 172.21.15.71:46142) 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238e800640 1 -- 172.21.15.71:0/3605108371 learned_addr learned my addr 172.21.15.71:0/3605108371 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238f200640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f239015bec0 0x7f239014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f2387e00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23900cbc00 0x7f23901e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238e800640 1 -- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23900cbc00 msgr2=0x7f23901e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238e800640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23900cbc00 0x7f23901e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238e800640 1 -- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f239015bec0 msgr2=0x7f239014a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238e800640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f239015bec0 0x7f239014a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.709+0000 7f238e800640 1 -- 172.21.15.71:0/3605108371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239014e0b0 con 0x7f2390072560 2024-09-16T07:27:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.710+0000 7f238e800640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2390072560 0x7f239014b180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f238000c3a0 tx=0x7f238000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.710+0000 7f2386a00640 1 -- 172.21.15.71:0/3605108371 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23800052d0 con 0x7f2390072560 2024-09-16T07:27:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.710+0000 7f2386a00640 1 -- 172.21.15.71:0/3605108371 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2380007020 con 0x7f2390072560 2024-09-16T07:27:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.710+0000 7f2386a00640 1 -- 172.21.15.71:0/3605108371 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380007470 con 0x7f2390072560 2024-09-16T07:27:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.710+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239015d4e0 con 0x7f2390072560 2024-09-16T07:27:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.710+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239015daa0 con 0x7f2390072560 2024-09-16T07:27:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.711+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23900af000 con 0x7f2390072560 2024-09-16T07:27:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.712+0000 7f2386a00640 1 -- 172.21.15.71:0/3605108371 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7f23800067d0 con 0x7f2390072560 2024-09-16T07:27:25.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.713+0000 7f2386a00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f236c078fa0 0x7f236c07b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:25.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.713+0000 7f2386a00640 1 -- 172.21.15.71:0/3605108371 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f238009b070 con 0x7f2390072560 2024-09-16T07:27:25.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.714+0000 7f2387e00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f236c078fa0 0x7f236c07b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:25.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.714+0000 7f2387e00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f236c078fa0 0x7f236c07b460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2378033000 tx=0x7f2378004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:25.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.717+0000 7f2386a00640 1 -- 172.21.15.71:0/3605108371 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2380061fb0 con 0x7f2390072560 2024-09-16T07:27:25.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.930+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23900b0260 con 0x7f236c078fa0 2024-09-16T07:27:25.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.937+0000 7f2386a00640 1 -- 172.21.15.71:0/3605108371 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+233 (secure 0 0 0) 0x7f23900b0260 con 0x7f236c078fa0 2024-09-16T07:27:25.936 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:25.936 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sv1" 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone16181" 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone28890" 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone29773" 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:25.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone6962" 2024-09-16T07:27:25.938 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:25.938 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f236c078fa0 msgr2=0x7f236c07b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f236c078fa0 0x7f236c07b460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2378033000 tx=0x7f2378004730 comp rx=0 tx=0).stop 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2390072560 msgr2=0x7f239014b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2390072560 0x7f239014b180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f238000c3a0 tx=0x7f238000c860 comp rx=0 tx=0).stop 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 shutdown_connections 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f236c078fa0 0x7f236c07b460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f239015bec0 0x7f239014a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23900cbc00 0x7f23901e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 --2- 172.21.15.71:0/3605108371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2390072560 0x7f239014b180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:25.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 >> 172.21.15.71:0/3605108371 conn(0x7f23900cf9f0 msgr2=0x7f23900af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:25.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 shutdown_connections 2024-09-16T07:27:25.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:25.939+0000 7f2394a00640 1 -- 172.21.15.71:0/3605108371 wait complete. 2024-09-16T07:27:25.955 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T07:27:26.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.117+0000 7fd84d400640 1 -- 172.21.15.71:0/3948053122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84806f300 msgr2=0x7fd8480b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.117+0000 7fd84d400640 1 --2- 172.21.15.71:0/3948053122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84806f300 0x7fd8480b4770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd840009be0 tx=0x7fd84002a920 comp rx=0 tx=0).stop 2024-09-16T07:27:26.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.117+0000 7fd84d400640 1 -- 172.21.15.71:0/3948053122 shutdown_connections 2024-09-16T07:27:26.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.117+0000 7fd84d400640 1 --2- 172.21.15.71:0/3948053122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84806f300 0x7fd8480b4770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.117+0000 7fd84d400640 1 -- 172.21.15.71:0/3948053122 >> 172.21.15.71:0/3948053122 conn(0x7fd8480b18e0 msgr2=0x7fd8480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:26.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.117+0000 7fd84d400640 1 -- 172.21.15.71:0/3948053122 shutdown_connections 2024-09-16T07:27:26.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.117+0000 7fd84d400640 1 -- 172.21.15.71:0/3948053122 wait complete. 2024-09-16T07:27:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.119+0000 7fd84d400640 1 Processor -- start 2024-09-16T07:27:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.119+0000 7fd84d400640 1 -- start start 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd84d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd84806ee00 0x7fd8481542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd84d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd84806f300 0x7fd8481f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd84d400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815dfc0 0x7fd8481f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd84d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480ae2b0 con 0x7fd84806ee00 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd84d400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480ae130 con 0x7fd84806f300 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd84d400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd8480ae430 con 0x7fd84815dfc0 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd846800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd84806ee00 0x7fd8481542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd846800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd84806ee00 0x7fd8481542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46146/0 (socket says 172.21.15.71:46146) 2024-09-16T07:27:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd846800640 1 -- 172.21.15.71:0/1789629003 learned_addr learned my addr 172.21.15.71:0/1789629003 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:26.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd846800640 1 -- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815dfc0 msgr2=0x7fd8481f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd846800640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815dfc0 0x7fd8481f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd846800640 1 -- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd84806f300 msgr2=0x7fd8481f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd846800640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd84806f300 0x7fd8481f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.120+0000 7fd846800640 1 -- 172.21.15.71:0/1789629003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd848156c90 con 0x7fd84806ee00 2024-09-16T07:27:26.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.121+0000 7fd846800640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd84806ee00 0x7fd8481542d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c430 tx=0x7fd83800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:26.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.121+0000 7fd83ea00640 1 -- 172.21.15.71:0/1789629003 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838011790 con 0x7fd84806ee00 2024-09-16T07:27:26.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.121+0000 7fd83ea00640 1 -- 172.21.15.71:0/1789629003 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd838017070 con 0x7fd84806ee00 2024-09-16T07:27:26.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.121+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd848156f70 con 0x7fd84806ee00 2024-09-16T07:27:26.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.121+0000 7fd83ea00640 1 -- 172.21.15.71:0/1789629003 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838011ce0 con 0x7fd84806ee00 2024-09-16T07:27:26.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.121+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84815b720 con 0x7fd84806ee00 2024-09-16T07:27:26.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.122+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8480ae8b0 con 0x7fd84806ee00 2024-09-16T07:27:26.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.123+0000 7fd83ea00640 1 -- 172.21.15.71:0/1789629003 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7fd83801c460 con 0x7fd84806ee00 2024-09-16T07:27:26.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.124+0000 7fd83ea00640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828078fa0 0x7fd82807b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.124+0000 7fd83ea00640 1 -- 172.21.15.71:0/1789629003 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fd83809ba20 con 0x7fd84806ee00 2024-09-16T07:27:26.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.125+0000 7fd845e00640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828078fa0 0x7fd82807b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.126+0000 7fd845e00640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828078fa0 0x7fd82807b460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd84815f010 tx=0x7fd834007820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:26.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.128+0000 7fd83ea00640 1 -- 172.21.15.71:0/1789629003 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd838063040 con 0x7fd84806ee00 2024-09-16T07:27:26.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.345+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8480cd3d0 con 0x7fd828078fa0 2024-09-16T07:27:26.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.360+0000 7fd83ea00640 1 -- 172.21.15.71:0/1789629003 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd8480cd3d0 con 0x7fd828078fa0 2024-09-16T07:27:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:26.359 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:27:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:26.359 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1" 2024-09-16T07:27:26.360 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:26.360 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828078fa0 msgr2=0x7fd82807b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828078fa0 0x7fd82807b460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd84815f010 tx=0x7fd834007820 comp rx=0 tx=0).stop 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd84806ee00 msgr2=0x7fd8481542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd84806ee00 0x7fd8481542d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c430 tx=0x7fd83800c8f0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 shutdown_connections 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828078fa0 0x7fd82807b460 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815dfc0 0x7fd8481f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd84806f300 0x7fd8481f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 --2- 172.21.15.71:0/1789629003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd84806ee00 0x7fd8481542d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 >> 172.21.15.71:0/1789629003 conn(0x7fd8480b18e0 msgr2=0x7fd8480af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:26.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 shutdown_connections 2024-09-16T07:27:26.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.362+0000 7fd84d400640 1 -- 172.21.15.71:0/1789629003 wait complete. 2024-09-16T07:27:26.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T07:27:26.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.541+0000 7f25d7000640 1 -- 172.21.15.71:0/2581650764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d0145750 msgr2=0x7f25d0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.541+0000 7f25d7000640 1 --2- 172.21.15.71:0/2581650764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d0145750 0x7f25d0073440 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f25d00cfb70 tx=0x7f25c002aa30 comp rx=0 tx=0).stop 2024-09-16T07:27:26.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.541+0000 7f25d7000640 1 -- 172.21.15.71:0/2581650764 shutdown_connections 2024-09-16T07:27:26.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.541+0000 7f25d7000640 1 --2- 172.21.15.71:0/2581650764 >> v2:172.21.15.177:3300/0 conn(0x7f25d00cccd0 0x7f25d00cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.541+0000 7f25d7000640 1 --2- 172.21.15.71:0/2581650764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d0145750 0x7f25d0073440 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.541+0000 7f25d7000640 1 -- 172.21.15.71:0/2581650764 >> 172.21.15.71:0/2581650764 conn(0x7f25d00b1840 msgr2=0x7f25d00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:26.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.541+0000 7f25d7000640 1 -- 172.21.15.71:0/2581650764 shutdown_connections 2024-09-16T07:27:26.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.541+0000 7f25d7000640 1 -- 172.21.15.71:0/2581650764 wait complete. 2024-09-16T07:27:26.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.543+0000 7f25d7000640 1 Processor -- start 2024-09-16T07:27:26.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.543+0000 7f25d7000640 1 -- start start 2024-09-16T07:27:26.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25d7000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d006efa0 0x7f25d013bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25d7000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d00cccd0 0x7f25d01426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25d7000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25d0145750 0x7f25d01f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25d7000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00ce480 con 0x7f25d006efa0 2024-09-16T07:27:26.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25d7000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00ce300 con 0x7f25d00cccd0 2024-09-16T07:27:26.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25d7000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f25d00ce600 con 0x7f25d0145750 2024-09-16T07:27:26.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d006efa0 0x7f25d013bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d006efa0 0x7f25d013bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46172/0 (socket says 172.21.15.71:46172) 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cfe00640 1 -- 172.21.15.71:0/4280351213 learned_addr learned my addr 172.21.15.71:0/4280351213 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25d5000640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25d0145750 0x7f25d01f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cf400640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d00cccd0 0x7f25d01426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cfe00640 1 -- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25d0145750 msgr2=0x7f25d01f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cfe00640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25d0145750 0x7f25d01f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cfe00640 1 -- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d00cccd0 msgr2=0x7f25d01426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cfe00640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d00cccd0 0x7f25d01426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cfe00640 1 -- 172.21.15.71:0/4280351213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d00b3650 con 0x7f25d006efa0 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25d5000640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25d0145750 0x7f25d01f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.544+0000 7f25cf400640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d00cccd0 0x7f25d01426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.545+0000 7f25cfe00640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d006efa0 0x7f25d013bcd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009bf0 tx=0x7f25c0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:26.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.545+0000 7f25c7e00640 1 -- 172.21.15.71:0/4280351213 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0033050 con 0x7f25d006efa0 2024-09-16T07:27:26.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.545+0000 7f25c7e00640 1 -- 172.21.15.71:0/4280351213 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c00044d0 con 0x7f25d006efa0 2024-09-16T07:27:26.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.545+0000 7f25c7e00640 1 -- 172.21.15.71:0/4280351213 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0038450 con 0x7f25d006efa0 2024-09-16T07:27:26.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.545+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d00b3930 con 0x7f25d006efa0 2024-09-16T07:27:26.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.545+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d00b3ef0 con 0x7f25d006efa0 2024-09-16T07:27:26.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.546+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d00cdbb0 con 0x7f25d006efa0 2024-09-16T07:27:26.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.547+0000 7f25c7e00640 1 -- 172.21.15.71:0/4280351213 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7f25c0036070 con 0x7f25d006efa0 2024-09-16T07:27:26.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.548+0000 7f25c7e00640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f25b0078fa0 0x7f25b007b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.548+0000 7f25c7e00640 1 -- 172.21.15.71:0/4280351213 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f25c00bccd0 con 0x7f25d006efa0 2024-09-16T07:27:26.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.548+0000 7f25cf400640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f25b0078fa0 0x7f25b007b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.549+0000 7f25cf400640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f25b0078fa0 0x7f25b007b460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25d0144380 tx=0x7f25bc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:26.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.552+0000 7f25c7e00640 1 -- 172.21.15.71:0/4280351213 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f25c0084670 con 0x7f25d006efa0 2024-09-16T07:27:26.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.773+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25d0144ee0 con 0x7f25b0078fa0 2024-09-16T07:27:26.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.804+0000 7f25c7e00640 1 -- 172.21.15.71:0/4280351213 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25d0144ee0 con 0x7f25b0078fa0 2024-09-16T07:27:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f25b0078fa0 msgr2=0x7f25b007b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f25b0078fa0 0x7f25b007b460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25d0144380 tx=0x7f25bc009040 comp rx=0 tx=0).stop 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d006efa0 msgr2=0x7f25d013bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d006efa0 0x7f25d013bcd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009bf0 tx=0x7f25c0003f10 comp rx=0 tx=0).stop 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 shutdown_connections 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f25b0078fa0 0x7f25b007b460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25d0145750 0x7f25d01f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25d00cccd0 0x7f25d01426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 --2- 172.21.15.71:0/4280351213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d006efa0 0x7f25d013bcd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 >> 172.21.15.71:0/4280351213 conn(0x7f25d00b1840 msgr2=0x7f25d00b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 shutdown_connections 2024-09-16T07:27:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.807+0000 7f25d7000640 1 -- 172.21.15.71:0/4280351213 wait complete. 2024-09-16T07:27:26.822 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T07:27:26.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.983+0000 7fb22be00640 1 -- 172.21.15.71:0/4040840532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c072560 msgr2=0x7fb22c073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.983+0000 7fb22be00640 1 --2- 172.21.15.71:0/4040840532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c072560 0x7fb22c073ef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0c9ff0 tx=0x7fb21402a530 comp rx=0 tx=0).stop 2024-09-16T07:27:26.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.983+0000 7fb22be00640 1 -- 172.21.15.71:0/4040840532 shutdown_connections 2024-09-16T07:27:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.983+0000 7fb22be00640 1 --2- 172.21.15.71:0/4040840532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c072560 0x7fb22c073ef0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.983+0000 7fb22be00640 1 --2- 172.21.15.71:0/4040840532 >> v2:172.21.15.177:3300/0 conn(0x7fb22c15bec0 0x7fb22c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.983+0000 7fb22be00640 1 -- 172.21.15.71:0/4040840532 >> 172.21.15.71:0/4040840532 conn(0x7fb22c0cf9f0 msgr2=0x7fb22c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.983+0000 7fb22be00640 1 -- 172.21.15.71:0/4040840532 shutdown_connections 2024-09-16T07:27:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.983+0000 7fb22be00640 1 -- 172.21.15.71:0/4040840532 wait complete. 2024-09-16T07:27:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.984+0000 7fb22be00640 1 Processor -- start 2024-09-16T07:27:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.985+0000 7fb22be00640 1 -- start start 2024-09-16T07:27:26.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.985+0000 7fb22be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c072560 0x7fb22c1f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.985+0000 7fb22be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c0cb4f0 0x7fb22c14dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb22be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c15bec0 0x7fb22c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb22be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c0cbdf0 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb22be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c0cbc70 con 0x7fb22c072560 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb22be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb22c0aebd0 con 0x7fb22c15bec0 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb223e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c0cb4f0 0x7fb22c14dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb223e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c0cb4f0 0x7fb22c14dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46192/0 (socket says 172.21.15.71:46192) 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb223e00640 1 -- 172.21.15.71:0/2310191828 learned_addr learned my addr 172.21.15.71:0/2310191828 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb22aa00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c072560 0x7fb22c1f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb22b400640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c15bec0 0x7fb22c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb223e00640 1 -- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c15bec0 msgr2=0x7fb22c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb223e00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c15bec0 0x7fb22c0ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb223e00640 1 -- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c072560 msgr2=0x7fb22c1f82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb223e00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c072560 0x7fb22c1f82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb223e00640 1 -- 172.21.15.71:0/2310191828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c1f5250 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb22b400640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c15bec0 0x7fb22c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:26.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.986+0000 7fb22aa00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c072560 0x7fb22c1f82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:26.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.987+0000 7fb223e00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c0cb4f0 0x7fb22c14dc40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb214009920 tx=0x7fb214032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:26.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.987+0000 7fb222a00640 1 -- 172.21.15.71:0/2310191828 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214040070 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.987+0000 7fb222a00640 1 -- 172.21.15.71:0/2310191828 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb214005310 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.987+0000 7fb222a00640 1 -- 172.21.15.71:0/2310191828 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2140047a0 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.987+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22c1f46d0 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.987+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22c1f4c90 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.988+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22c0af050 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.989+0000 7fb222a00640 1 -- 172.21.15.71:0/2310191828 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7fb214039040 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.990+0000 7fb222a00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb208078fa0 0x7fb20807b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:26.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.990+0000 7fb222a00640 1 -- 172.21.15.71:0/2310191828 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fb2140bce00 con 0x7fb22c0cb4f0 2024-09-16T07:27:26.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.991+0000 7fb22aa00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb208078fa0 0x7fb20807b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:26.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.992+0000 7fb22aa00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb208078fa0 0x7fb20807b460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009b80 tx=0x7fb21c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:26.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:26.994+0000 7fb222a00640 1 -- 172.21.15.71:0/2310191828 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb214084460 con 0x7fb22c0cb4f0 2024-09-16T07:27:27.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.204+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb22c14e180 con 0x7fb208078fa0 2024-09-16T07:27:27.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.225+0000 7fb222a00640 1 -- 172.21.15.71:0/2310191828 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb22c14e180 con 0x7fb208078fa0 2024-09-16T07:27:27.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb208078fa0 msgr2=0x7fb20807b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:27.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb208078fa0 0x7fb20807b460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009b80 tx=0x7fb21c005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:27.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c0cb4f0 msgr2=0x7fb22c14dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:27.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c0cb4f0 0x7fb22c14dc40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb214009920 tx=0x7fb214032c10 comp rx=0 tx=0).stop 2024-09-16T07:27:27.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 shutdown_connections 2024-09-16T07:27:27.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb208078fa0 0x7fb20807b460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c15bec0 0x7fb22c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c0cb4f0 0x7fb22c14dc40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.227+0000 7fb22be00640 1 --2- 172.21.15.71:0/2310191828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c072560 0x7fb22c1f82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.228+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 >> 172.21.15.71:0/2310191828 conn(0x7fb22c0cf9f0 msgr2=0x7fb22c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:27.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.228+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 shutdown_connections 2024-09-16T07:27:27.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.228+0000 7fb22be00640 1 -- 172.21.15.71:0/2310191828 wait complete. 2024-09-16T07:27:27.229 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.231+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.230 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.231+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.230 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.231+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.230 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.231+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.230 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.231+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.245 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone16181 2024-09-16T07:27:27.248 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.249+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.248 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.249+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.248 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.249+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.248 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.249+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.248 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:27:27.249+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:27:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.414+0000 7f8a09600640 1 -- 172.21.15.71:0/2560458861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 msgr2=0x7f8a040b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.414+0000 7f8a09600640 1 --2- 172.21.15.71:0/2560458861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 0x7f8a040b4350 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005fd0 tx=0x7f89f402a5c0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.414+0000 7f8a09600640 1 -- 172.21.15.71:0/2560458861 shutdown_connections 2024-09-16T07:27:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.414+0000 7f8a09600640 1 --2- 172.21.15.71:0/2560458861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a0406faf0 0x7f8a0415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.414+0000 7f8a09600640 1 --2- 172.21.15.71:0/2560458861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 0x7f8a040b4350 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.414+0000 7f8a09600640 1 -- 172.21.15.71:0/2560458861 >> 172.21.15.71:0/2560458861 conn(0x7f8a040b18e0 msgr2=0x7f8a040b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.414+0000 7f8a09600640 1 -- 172.21.15.71:0/2560458861 shutdown_connections 2024-09-16T07:27:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.414+0000 7f8a09600640 1 -- 172.21.15.71:0/2560458861 wait complete. 2024-09-16T07:27:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.416+0000 7f8a09600640 1 Processor -- start 2024-09-16T07:27:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.416+0000 7f8a09600640 1 -- start start 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a09600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a0406faf0 0x7f8a04159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a09600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a040727c0 0x7f8a041f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a09600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 0x7f8a041f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a09600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0415eb50 con 0x7f8a041550c0 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a09600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0415e9d0 con 0x7f8a0406faf0 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a09600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0415ecd0 con 0x7f8a040727c0 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a03400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 0x7f8a041f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a03400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 0x7f8a041f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46212/0 (socket says 172.21.15.71:46212) 2024-09-16T07:27:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a03400640 1 -- 172.21.15.71:0/1754375853 learned_addr learned my addr 172.21.15.71:0/1754375853 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f89fbe00640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a040727c0 0x7f8a041f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.417+0000 7f8a02a00640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a0406faf0 0x7f8a04159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f8a03400640 1 -- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a040727c0 msgr2=0x7f8a041f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f8a03400640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a040727c0 0x7f8a041f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f8a03400640 1 -- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a0406faf0 msgr2=0x7f8a04159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f8a03400640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a0406faf0 0x7f8a04159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f8a03400640 1 -- 172.21.15.71:0/1754375853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0415f960 con 0x7f8a041550c0 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f8a03400640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 0x7f8a041f8610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b9a0 tx=0x7f89fc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f89faa00640 1 -- 172.21.15.71:0/1754375853 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc019070 con 0x7f8a041550c0 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f89faa00640 1 -- 172.21.15.71:0/1754375853 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89fc00ccb0 con 0x7f8a041550c0 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0415fc40 con 0x7f8a041550c0 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.418+0000 7f89faa00640 1 -- 172.21.15.71:0/1754375853 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0146a0 con 0x7f8a041550c0 2024-09-16T07:27:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.419+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041f8e30 con 0x7f8a041550c0 2024-09-16T07:27:27.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.420+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a04069360 con 0x7f8a041550c0 2024-09-16T07:27:27.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.421+0000 7f89faa00640 1 -- 172.21.15.71:0/1754375853 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7f89fc01c070 con 0x7f8a041550c0 2024-09-16T07:27:27.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.421+0000 7f89faa00640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f89e0078fa0 0x7f89e007b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:27.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.422+0000 7f89faa00640 1 -- 172.21.15.71:0/1754375853 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f89fc09b910 con 0x7f8a041550c0 2024-09-16T07:27:27.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.422+0000 7f8a02a00640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f89e0078fa0 0x7f89e007b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:27.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.423+0000 7f8a02a00640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f89e0078fa0 0x7f89e007b460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f89f4033000 tx=0x7f89f4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:27.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.425+0000 7f89faa00640 1 -- 172.21.15.71:0/1754375853 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f89fc0628e0 con 0x7f8a041550c0 2024-09-16T07:27:27.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:27.642+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone16181", "target": ["mon-mgr", ""]}) -- 0x7f8a040cdf50 con 0x7f89e0078fa0 2024-09-16T07:27:29.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.127+0000 7f89faa00640 1 -- 172.21.15.71:0/1754375853 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a040cdf50 con 0x7f89e0078fa0 2024-09-16T07:27:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f89e0078fa0 msgr2=0x7f89e007b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f89e0078fa0 0x7f89e007b460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f89f4033000 tx=0x7f89f4004730 comp rx=0 tx=0).stop 2024-09-16T07:27:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 msgr2=0x7f8a041f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 0x7f8a041f8610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b9a0 tx=0x7f89fc00be60 comp rx=0 tx=0).stop 2024-09-16T07:27:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 shutdown_connections 2024-09-16T07:27:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f89e0078fa0 0x7f89e007b460 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a041550c0 0x7f8a041f8610 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a040727c0 0x7f8a041f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 --2- 172.21.15.71:0/1754375853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a0406faf0 0x7f8a04159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 >> 172.21.15.71:0/1754375853 conn(0x7f8a040b18e0 msgr2=0x7f8a0406f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.130+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 shutdown_connections 2024-09-16T07:27:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.131+0000 7f8a09600640 1 -- 172.21.15.71:0/1754375853 wait complete. 2024-09-16T07:27:29.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone28890 2024-09-16T07:27:29.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.311+0000 7fa693e00640 1 -- 172.21.15.71:0/776483790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940d0500 msgr2=0x7fa678003a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.311+0000 7fa693e00640 1 --2- 172.21.15.71:0/776483790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940d0500 0x7fa678003a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa684009b50 tx=0x7fa678005830 comp rx=0 tx=0).stop 2024-09-16T07:27:29.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.311+0000 7fa693400640 1 -- 172.21.15.71:0/776483790 reap_dead start 2024-09-16T07:27:29.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.311+0000 7fa693e00640 1 -- 172.21.15.71:0/776483790 shutdown_connections 2024-09-16T07:27:29.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.311+0000 7fa693e00640 1 -- 172.21.15.71:0/776483790 >> 172.21.15.71:0/776483790 conn(0x7fa69409d180 msgr2=0x7fa6940d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:29.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.311+0000 7fa693e00640 1 -- 172.21.15.71:0/776483790 shutdown_connections 2024-09-16T07:27:29.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.311+0000 7fa693e00640 1 -- 172.21.15.71:0/776483790 wait complete. 2024-09-16T07:27:29.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.313+0000 7fa693e00640 1 Processor -- start 2024-09-16T07:27:29.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.313+0000 7fa693e00640 1 -- start start 2024-09-16T07:27:29.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa693e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940d0500 0x7fa6940ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa693e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa69414ce50 0x7fa6940c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa693e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa69414d2d0 0x7fa6941f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa693e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940d0b10 con 0x7fa69414ce50 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa693e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940d0990 con 0x7fa69414d2d0 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa693e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa6940d0c90 con 0x7fa6940d0500 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa68a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa69414ce50 0x7fa6940c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa68a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa69414ce50 0x7fa6940c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46230/0 (socket says 172.21.15.71:46230) 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa68a000640 1 -- 172.21.15.71:0/3840857382 learned_addr learned my addr 172.21.15.71:0/3840857382 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:29.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa692a00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940d0500 0x7fa6940ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa693400640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa69414d2d0 0x7fa6941f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa68a000640 1 -- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940d0500 msgr2=0x7fa6940ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa68a000640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940d0500 0x7fa6940ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa68a000640 1 -- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa69414d2d0 msgr2=0x7fa6941f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa68a000640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa69414d2d0 0x7fa6941f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.314+0000 7fa68a000640 1 -- 172.21.15.71:0/3840857382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6940cbfe0 con 0x7fa69414ce50 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.315+0000 7fa68a000640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa69414ce50 0x7fa6940c9db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa67c000fb0 tx=0x7fa67c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.315+0000 7fa68b400640 1 -- 172.21.15.71:0/3840857382 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c015070 con 0x7fa69414ce50 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.315+0000 7fa68b400640 1 -- 172.21.15.71:0/3840857382 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa67c00fa40 con 0x7fa69414ce50 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.315+0000 7fa68b400640 1 -- 172.21.15.71:0/3840857382 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00fe90 con 0x7fa69414ce50 2024-09-16T07:27:29.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.315+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6940cc2c0 con 0x7fa69414ce50 2024-09-16T07:27:29.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.315+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6940cc880 con 0x7fa69414ce50 2024-09-16T07:27:29.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.316+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69408df30 con 0x7fa69414ce50 2024-09-16T07:27:29.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.317+0000 7fa68b400640 1 -- 172.21.15.71:0/3840857382 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) ==== 101950+0+0 (secure 0 0 0) 0x7fa67c011430 con 0x7fa69414ce50 2024-09-16T07:27:29.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.318+0000 7fa68b400640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa670079140 0x7fa67007b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:29.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.318+0000 7fa68b400640 1 -- 172.21.15.71:0/3840857382 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fa67c09c960 con 0x7fa69414ce50 2024-09-16T07:27:29.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.319+0000 7fa692a00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa670079140 0x7fa67007b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:29.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.320+0000 7fa692a00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa670079140 0x7fa67007b600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa684035000 tx=0x7fa6840046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:29.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.323+0000 7fa68b400640 1 -- 172.21.15.71:0/3840857382 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa67c063930 con 0x7fa69414ce50 2024-09-16T07:27:29.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.361+0000 7fa68b400640 1 -- 172.21.15.71:0/3840857382 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fa67c05fba0 con 0x7fa69414ce50 2024-09-16T07:27:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.540+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone28890", "target": ["mon-mgr", ""]}) -- 0x7fa6941611b0 con 0x7fa670079140 2024-09-16T07:27:29.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.565+0000 7fa68b400640 1 -- 172.21.15.71:0/3840857382 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa6941611b0 con 0x7fa670079140 2024-09-16T07:27:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa670079140 msgr2=0x7fa67007b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa670079140 0x7fa67007b600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa684035000 tx=0x7fa6840046c0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa69414ce50 msgr2=0x7fa6940c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa69414ce50 0x7fa6940c9db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa67c000fb0 tx=0x7fa67c003a80 comp rx=0 tx=0).stop 2024-09-16T07:27:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 shutdown_connections 2024-09-16T07:27:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa670079140 0x7fa67007b600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa69414d2d0 0x7fa6941f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa69414ce50 0x7fa6940c9db0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 --2- 172.21.15.71:0/3840857382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940d0500 0x7fa6940ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 >> 172.21.15.71:0/3840857382 conn(0x7fa69409d180 msgr2=0x7fa694160270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 shutdown_connections 2024-09-16T07:27:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.568+0000 7fa693e00640 1 -- 172.21.15.71:0/3840857382 wait complete. 2024-09-16T07:27:29.594 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone29773 2024-09-16T07:27:29.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.871+0000 7f9373e00640 1 -- 172.21.15.71:0/376675763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f936c0d0ab0 msgr2=0x7f936c0d4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.871+0000 7f9373e00640 1 --2- 172.21.15.71:0/376675763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f936c0d0ab0 0x7f936c0d4b20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9360009830 tx=0x7f936002a520 comp rx=0 tx=0).stop 2024-09-16T07:27:29.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.871+0000 7f9373e00640 1 -- 172.21.15.71:0/376675763 shutdown_connections 2024-09-16T07:27:29.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.871+0000 7f9373e00640 1 --2- 172.21.15.71:0/376675763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f936c0d0ab0 0x7f936c0d4b20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.871+0000 7f9373e00640 1 -- 172.21.15.71:0/376675763 >> 172.21.15.71:0/376675763 conn(0x7f936c0d4000 msgr2=0x7f936c0d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:29.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.871+0000 7f9373e00640 1 -- 172.21.15.71:0/376675763 shutdown_connections 2024-09-16T07:27:29.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.872+0000 7f9373e00640 1 -- 172.21.15.71:0/376675763 wait complete. 2024-09-16T07:27:29.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.874+0000 7f9373e00640 1 Processor -- start 2024-09-16T07:27:29.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.874+0000 7f9373e00640 1 -- start start 2024-09-16T07:27:29.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.875+0000 7f9373e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936c0b4de0 0x7f936c151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:29.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.875+0000 7f9373e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f936c0d0ab0 0x7f936c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:29.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.875+0000 7f9373e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f936c161270 0x7f936c0b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:29.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.875+0000 7f9373e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f936c169350 con 0x7f936c0b4de0 2024-09-16T07:27:29.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.875+0000 7f9373e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1691d0 con 0x7f936c161270 2024-09-16T07:27:29.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.875+0000 7f9373e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f936c1694d0 con 0x7f936c0d0ab0 2024-09-16T07:27:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.876+0000 7f9371400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936c0b4de0 0x7f936c151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.876+0000 7f9371400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936c0b4de0 0x7f936c151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46236/0 (socket says 172.21.15.71:46236) 2024-09-16T07:27:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.876+0000 7f9371400640 1 -- 172.21.15.71:0/699990800 learned_addr learned my addr 172.21.15.71:0/699990800 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.876+0000 7f9370a00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f936c0d0ab0 0x7f936c15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.876+0000 7f9371e00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f936c161270 0x7f936c0b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:29.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.877+0000 7f9371400640 1 -- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f936c0d0ab0 msgr2=0x7f936c15e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.877+0000 7f9371400640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f936c0d0ab0 0x7f936c15e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.877+0000 7f9371400640 1 -- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f936c161270 msgr2=0x7f936c0b8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:29.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.877+0000 7f9371400640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f936c161270 0x7f936c0b8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:29.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.877+0000 7f9371400640 1 -- 172.21.15.71:0/699990800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c15d7e0 con 0x7f936c0b4de0 2024-09-16T07:27:29.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.878+0000 7f9371400640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936c0b4de0 0x7f936c151400 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9360000c00 tx=0x7f9360032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:29.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.878+0000 7f9369600640 1 -- 172.21.15.71:0/699990800 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360040070 con 0x7f936c0b4de0 2024-09-16T07:27:29.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.878+0000 7f9369600640 1 -- 172.21.15.71:0/699990800 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9360003af0 con 0x7f936c0b4de0 2024-09-16T07:27:29.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.878+0000 7f9369600640 1 -- 172.21.15.71:0/699990800 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360045460 con 0x7f936c0b4de0 2024-09-16T07:27:29.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.878+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c0b8020 con 0x7f936c0b4de0 2024-09-16T07:27:29.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.878+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c0b85e0 con 0x7f936c0b4de0 2024-09-16T07:27:29.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.880+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936c0d4b20 con 0x7f936c0b4de0 2024-09-16T07:27:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.881+0000 7f9369600640 1 -- 172.21.15.71:0/699990800 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f9360039040 con 0x7f936c0b4de0 2024-09-16T07:27:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.884+0000 7f9369600640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f934c0790c0 0x7f934c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.884+0000 7f9369600640 1 -- 172.21.15.71:0/699990800 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f93600bd2b0 con 0x7f936c0b4de0 2024-09-16T07:27:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.886+0000 7f9370a00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f934c0790c0 0x7f934c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:29.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.887+0000 7f9370a00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f934c0790c0 0x7f934c07b580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f936c0b3360 tx=0x7f9358007e90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:29.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:29.894+0000 7f9369600640 1 -- 172.21.15.71:0/699990800 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9360084860 con 0x7f936c0b4de0 2024-09-16T07:27:30.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:30.301+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone29773", "target": ["mon-mgr", ""]}) -- 0x7f936c0b2d60 con 0x7f934c0790c0 2024-09-16T07:27:31.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.152+0000 7f9369600640 1 -- 172.21.15.71:0/699990800 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f936c0b2d60 con 0x7f934c0790c0 2024-09-16T07:27:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f934c0790c0 msgr2=0x7f934c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f934c0790c0 0x7f934c07b580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f936c0b3360 tx=0x7f9358007e90 comp rx=0 tx=0).stop 2024-09-16T07:27:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936c0b4de0 msgr2=0x7f936c151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936c0b4de0 0x7f936c151400 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9360000c00 tx=0x7f9360032cd0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 shutdown_connections 2024-09-16T07:27:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f934c0790c0 0x7f934c07b580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f936c161270 0x7f936c0b8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f936c0d0ab0 0x7f936c15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 --2- 172.21.15.71:0/699990800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936c0b4de0 0x7f936c151400 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 >> 172.21.15.71:0/699990800 conn(0x7f936c0d4000 msgr2=0x7f936c06e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 shutdown_connections 2024-09-16T07:27:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.155+0000 7f9373e00640 1 -- 172.21.15.71:0/699990800 wait complete. 2024-09-16T07:27:31.179 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6962 2024-09-16T07:27:31.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.475+0000 7fab2b200640 1 -- 172.21.15.71:0/3235272220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2c15abd0 msgr2=0x7fab2c072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:31.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.475+0000 7fab2b200640 1 --2- 172.21.15.71:0/3235272220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2c15abd0 0x7fab2c072d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009860 tx=0x7fab1c02a5a0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.476+0000 7fab2b200640 1 -- 172.21.15.71:0/3235272220 shutdown_connections 2024-09-16T07:27:31.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.476+0000 7fab2b200640 1 --2- 172.21.15.71:0/3235272220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2c15abd0 0x7fab2c072d00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.476+0000 7fab2b200640 1 -- 172.21.15.71:0/3235272220 >> 172.21.15.71:0/3235272220 conn(0x7fab2c0cfa70 msgr2=0x7fab2c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:31.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.476+0000 7fab2b200640 1 -- 172.21.15.71:0/3235272220 shutdown_connections 2024-09-16T07:27:31.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.476+0000 7fab2b200640 1 -- 172.21.15.71:0/3235272220 wait complete. 2024-09-16T07:27:31.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.477+0000 7fab2b200640 1 Processor -- start 2024-09-16T07:27:31.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.477+0000 7fab2b200640 1 -- start start 2024-09-16T07:27:31.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.478+0000 7fab2b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab2c073cf0 0x7fab2c14a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:31.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.478+0000 7fab2b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2c15abd0 0x7fab2c1e3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:31.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.478+0000 7fab2b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2c0b43a0 0x7fab2c0ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.478+0000 7fab2b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c0cda80 con 0x7fab2c073cf0 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.478+0000 7fab2b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c0cd900 con 0x7fab2c15abd0 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.478+0000 7fab2b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fab2c0cdc00 con 0x7fab2c0b43a0 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.478+0000 7fab29e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab2c073cf0 0x7fab2c14a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.478+0000 7fab29e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab2c073cf0 0x7fab2c14a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46252/0 (socket says 172.21.15.71:46252) 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29e00640 1 -- 172.21.15.71:0/2006237655 learned_addr learned my addr 172.21.15.71:0/2006237655 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29400640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2c15abd0 0x7fab2c1e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29e00640 1 -- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2c0b43a0 msgr2=0x7fab2c0ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29e00640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2c0b43a0 0x7fab2c0ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29e00640 1 -- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2c15abd0 msgr2=0x7fab2c1e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29e00640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2c15abd0 0x7fab2c1e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29e00640 1 -- 172.21.15.71:0/2006237655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c1f7ee0 con 0x7fab2c073cf0 2024-09-16T07:27:31.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29400640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2c15abd0 0x7fab2c1e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:31.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab29e00640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab2c073cf0 0x7fab2c14a240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009980 tx=0x7fab1c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:31.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab22000640 1 -- 172.21.15.71:0/2006237655 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c031b50 con 0x7fab2c073cf0 2024-09-16T07:27:31.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab22000640 1 -- 172.21.15.71:0/2006237655 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab1c031de0 con 0x7fab2c073cf0 2024-09-16T07:27:31.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab22000640 1 -- 172.21.15.71:0/2006237655 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c033330 con 0x7fab2c073cf0 2024-09-16T07:27:31.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.479+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c1f81c0 con 0x7fab2c073cf0 2024-09-16T07:27:31.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.480+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c14a780 con 0x7fab2c073cf0 2024-09-16T07:27:31.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.481+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c0ce0d0 con 0x7fab2c073cf0 2024-09-16T07:27:31.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.482+0000 7fab22000640 1 -- 172.21.15.71:0/2006237655 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fab1c0334d0 con 0x7fab2c073cf0 2024-09-16T07:27:31.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.483+0000 7fab22000640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab0c0790c0 0x7fab0c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:31.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.483+0000 7fab22000640 1 -- 172.21.15.71:0/2006237655 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7fab1c0bcea0 con 0x7fab2c073cf0 2024-09-16T07:27:31.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.484+0000 7fab29400640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab0c0790c0 0x7fab0c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:31.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.484+0000 7fab29400640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab0c0790c0 0x7fab0c07b580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab18001220 tx=0x7fab18009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:31.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.488+0000 7fab22000640 1 -- 172.21.15.71:0/2006237655 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fab1c03f030 con 0x7fab2c073cf0 2024-09-16T07:27:31.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.807+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6962", "target": ["mon-mgr", ""]}) -- 0x7fab2c1f7c90 con 0x7fab0c0790c0 2024-09-16T07:27:31.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.842+0000 7fab22000640 1 -- 172.21.15.71:0/2006237655 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab2c1f7c90 con 0x7fab0c0790c0 2024-09-16T07:27:31.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.846+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab0c0790c0 msgr2=0x7fab0c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:31.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.846+0000 7fab2b200640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab0c0790c0 0x7fab0c07b580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab18001220 tx=0x7fab18009040 comp rx=0 tx=0).stop 2024-09-16T07:27:31.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.846+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab2c073cf0 msgr2=0x7fab2c14a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:31.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.846+0000 7fab2b200640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab2c073cf0 0x7fab2c14a240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009980 tx=0x7fab1c033ba0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.847+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 shutdown_connections 2024-09-16T07:27:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.847+0000 7fab2b200640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fab0c0790c0 0x7fab0c07b580 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.847+0000 7fab2b200640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2c0b43a0 0x7fab2c0ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.847+0000 7fab2b200640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2c15abd0 0x7fab2c1e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.847+0000 7fab2b200640 1 --2- 172.21.15.71:0/2006237655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab2c073cf0 0x7fab2c14a240 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.847+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 >> 172.21.15.71:0/2006237655 conn(0x7fab2c0cfa70 msgr2=0x7fab2c06f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.847+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 shutdown_connections 2024-09-16T07:27:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31.847+0000 7fab2b200640 1 -- 172.21.15.71:0/2006237655 wait complete. 2024-09-16T07:27:31.874 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:27:31.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:27:32.001 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:27:32.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:27:32.010 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:27:32.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.172+0000 7fc99dc00640 1 -- 172.21.15.71:0/2657276387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc97c003a50 msgr2=0x7fc97c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:32.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.172+0000 7fc99dc00640 1 --2- 172.21.15.71:0/2657276387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc97c003a50 0x7fc97c003e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc988009b50 tx=0x7fc97c005830 comp rx=0 tx=0).stop 2024-09-16T07:27:32.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.172+0000 7fc99d200640 1 -- 172.21.15.71:0/2657276387 reap_dead start 2024-09-16T07:27:32.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.172+0000 7fc99dc00640 1 -- 172.21.15.71:0/2657276387 shutdown_connections 2024-09-16T07:27:32.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.172+0000 7fc99dc00640 1 -- 172.21.15.71:0/2657276387 >> 172.21.15.71:0/2657276387 conn(0x7fc9980b1510 msgr2=0x7fc9980b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:32.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.172+0000 7fc99dc00640 1 -- 172.21.15.71:0/2657276387 shutdown_connections 2024-09-16T07:27:32.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.172+0000 7fc99dc00640 1 -- 172.21.15.71:0/2657276387 wait complete. 2024-09-16T07:27:32.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.173+0000 7fc99dc00640 1 Processor -- start 2024-09-16T07:27:32.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.174+0000 7fc99dc00640 1 -- start start 2024-09-16T07:27:32.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.174+0000 7fc99dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c003a50 0x7fc998152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:32.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.174+0000 7fc99dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc998151ce0 0x7fc9980b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.174+0000 7fc99dc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc99814b890 0x7fc99814bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.174+0000 7fc99dc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9980b2810 con 0x7fc998151ce0 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.174+0000 7fc99dc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9980cf6d0 con 0x7fc97c003a50 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.174+0000 7fc99dc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9980b2990 con 0x7fc99814b890 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c003a50 0x7fc998152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc998151ce0 0x7fc9980b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c003a50 0x7fc998152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:51132/0 (socket says 172.21.15.71:51132) 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997e00640 1 -- 172.21.15.71:0/2681627821 learned_addr learned my addr 172.21.15.71:0/2681627821 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc99d200640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc99814b890 0x7fc99814bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997e00640 1 -- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc99814b890 msgr2=0x7fc99814bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997e00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc99814b890 0x7fc99814bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997e00640 1 -- 172.21.15.71:0/2681627821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc998151ce0 msgr2=0x7fc9980b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:32.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997e00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc998151ce0 0x7fc9980b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997e00640 1 -- 172.21.15.71:0/2681627821 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9981f8d60 con 0x7fc97c003a50 2024-09-16T07:27:32.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.175+0000 7fc997400640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc998151ce0 0x7fc9980b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:27:32.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.176+0000 7fc997e00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c003a50 0x7fc998152150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc988009a70 tx=0x7fc988033670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:32.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.176+0000 7fc98fe00640 1 -- 172.21.15.71:0/2681627821 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988038040 con 0x7fc97c003a50 2024-09-16T07:27:32.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.176+0000 7fc98fe00640 1 -- 172.21.15.71:0/2681627821 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc988004020 con 0x7fc97c003a50 2024-09-16T07:27:32.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.176+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9981f81e0 con 0x7fc97c003a50 2024-09-16T07:27:32.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.176+0000 7fc98fe00640 1 -- 172.21.15.71:0/2681627821 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988004470 con 0x7fc97c003a50 2024-09-16T07:27:32.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.176+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981f87a0 con 0x7fc97c003a50 2024-09-16T07:27:32.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.177+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc998072560 con 0x7fc97c003a50 2024-09-16T07:27:32.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.178+0000 7fc98fe00640 1 -- 172.21.15.71:0/2681627821 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fc988045020 con 0x7fc97c003a50 2024-09-16T07:27:32.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.179+0000 7fc98fe00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc9780790c0 0x7fc97807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:32.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.179+0000 7fc98fe00640 1 -- 172.21.15.71:0/2681627821 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7fc9880bc480 con 0x7fc97c003a50 2024-09-16T07:27:32.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.179+0000 7fc997400640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc9780790c0 0x7fc97807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:32.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.180+0000 7fc997400640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc9780790c0 0x7fc97807b580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc99815eac0 tx=0x7fc984006720 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:32.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.184+0000 7fc98fe00640 1 -- 172.21.15.71:0/2681627821 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc998072560 con 0x7fc97c003a50 2024-09-16T07:27:32.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.367+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fc99806ed60 con 0x7fc97c003a50 2024-09-16T07:27:32.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.370+0000 7fc98fe00640 1 -- 172.21.15.71:0/2681627821 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7fc988083430 con 0x7fc97c003a50 2024-09-16T07:27:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc9780790c0 msgr2=0x7fc97807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc9780790c0 0x7fc97807b580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc99815eac0 tx=0x7fc984006720 comp rx=0 tx=0).stop 2024-09-16T07:27:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c003a50 msgr2=0x7fc998152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c003a50 0x7fc998152150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc988009a70 tx=0x7fc988033670 comp rx=0 tx=0).stop 2024-09-16T07:27:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 shutdown_connections 2024-09-16T07:27:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc9780790c0 0x7fc97807b580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc99814b890 0x7fc99814bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc998151ce0 0x7fc9980b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 --2- 172.21.15.71:0/2681627821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c003a50 0x7fc998152150 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 >> 172.21.15.71:0/2681627821 conn(0x7fc9980b1510 msgr2=0x7fc9980b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.372+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 shutdown_connections 2024-09-16T07:27:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.373+0000 7fc99dc00640 1 -- 172.21.15.71:0/2681627821 wait complete. 2024-09-16T07:27:32.388 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:27:32.388 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:27:32.475 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:27:32.571 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:27:32.571 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:27:32.571 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:27:32.571 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:32.633 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:27:32.633 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:32.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:27:32.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.886+0000 7f3deb200640 1 -- 172.21.15.71:0/2684610220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dcc005230 msgr2=0x7f3dcc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:32.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.886+0000 7f3deb200640 1 --2- 172.21.15.71:0/2684610220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dcc005230 0x7f3dcc00b9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c470 tx=0x7f3dcc005830 comp rx=0 tx=0).stop 2024-09-16T07:27:32.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.886+0000 7f3deb200640 1 -- 172.21.15.71:0/2684610220 shutdown_connections 2024-09-16T07:27:32.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.886+0000 7f3deb200640 1 --2- 172.21.15.71:0/2684610220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dcc005230 0x7f3dcc00b9c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.886+0000 7f3deb200640 1 -- 172.21.15.71:0/2684610220 >> 172.21.15.71:0/2684610220 conn(0x7f3de40cfa20 msgr2=0x7f3de40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:32.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.886+0000 7f3deb200640 1 -- 172.21.15.71:0/2684610220 shutdown_connections 2024-09-16T07:27:32.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.886+0000 7f3deb200640 1 -- 172.21.15.71:0/2684610220 wait complete. 2024-09-16T07:27:32.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.888+0000 7f3deb200640 1 Processor -- start 2024-09-16T07:27:32.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.888+0000 7f3deb200640 1 -- start start 2024-09-16T07:27:32.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3deb200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dcc005230 0x7f3de40cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:32.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3deb200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3de4155e00 0x7f3de40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3deb200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3de40cebf0 0x7f3de4151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3deb200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3de40b2c80 con 0x7f3de40cebf0 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3deb200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3de40b2b00 con 0x7f3dcc005230 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3deb200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3de4073d40 con 0x7f3de4155e00 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3dea800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3de40cebf0 0x7f3de4151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3dea800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3de40cebf0 0x7f3de4151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46298/0 (socket says 172.21.15.71:46298) 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3dea800640 1 -- 172.21.15.71:0/4116939202 learned_addr learned my addr 172.21.15.71:0/4116939202 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3de9e00640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dcc005230 0x7f3de40cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3de9400640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3de4155e00 0x7f3de40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3dea800640 1 -- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3de4155e00 msgr2=0x7f3de40b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3dea800640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3de4155e00 0x7f3de40b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3dea800640 1 -- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dcc005230 msgr2=0x7f3de40cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:32.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3dea800640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dcc005230 0x7f3de40cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:32.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.889+0000 7f3dea800640 1 -- 172.21.15.71:0/4116939202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de40cf140 con 0x7f3de40cebf0 2024-09-16T07:27:32.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.890+0000 7f3dea800640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3de40cebf0 0x7f3de4151ad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c560 tx=0x7f3de00021b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:32.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.890+0000 7f3dde000640 1 -- 172.21.15.71:0/4116939202 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0039070 con 0x7f3de40cebf0 2024-09-16T07:27:32.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.890+0000 7f3dde000640 1 -- 172.21.15.71:0/4116939202 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3de002bcd0 con 0x7f3de40cebf0 2024-09-16T07:27:32.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.890+0000 7f3dde000640 1 -- 172.21.15.71:0/4116939202 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0006330 con 0x7f3de40cebf0 2024-09-16T07:27:32.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.890+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de40cf420 con 0x7f3de40cebf0 2024-09-16T07:27:32.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.890+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de40ce4d0 con 0x7f3de40cebf0 2024-09-16T07:27:32.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.891+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de40742a0 con 0x7f3de40cebf0 2024-09-16T07:27:32.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.892+0000 7f3dde000640 1 -- 172.21.15.71:0/4116939202 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f3de00064d0 con 0x7f3de40cebf0 2024-09-16T07:27:32.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.893+0000 7f3dde000640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3dc80790c0 0x7f3dc807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:32.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.893+0000 7f3dde000640 1 -- 172.21.15.71:0/4116939202 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f3de00b8830 con 0x7f3de40cebf0 2024-09-16T07:27:32.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.894+0000 7f3de9e00640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3dc80790c0 0x7f3dc807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:32.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.894+0000 7f3de9e00640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3dc80790c0 0x7f3dc807b580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009bd0 tx=0x7f3dd8005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:32.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:32.898+0000 7f3dde000640 1 -- 172.21.15.71:0/4116939202 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3de0044030 con 0x7f3de40cebf0 2024-09-16T07:27:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.094+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3de40cf5f0 con 0x7f3de40cebf0 2024-09-16T07:27:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.095+0000 7f3dde000640 1 -- 172.21.15.71:0/4116939202 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) ==== 80+0+0 (secure 0 0 0) 0x7f3de0082a10 con 0x7f3de40cebf0 2024-09-16T07:27:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:27:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3dc80790c0 msgr2=0x7f3dc807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3dc80790c0 0x7f3dc807b580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009bd0 tx=0x7f3dd8005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3de40cebf0 msgr2=0x7f3de4151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3de40cebf0 0x7f3de4151ad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c560 tx=0x7f3de00021b0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 shutdown_connections 2024-09-16T07:27:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3dc80790c0 0x7f3dc807b580 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3de40cebf0 0x7f3de4151ad0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3de4155e00 0x7f3de40b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 --2- 172.21.15.71:0/4116939202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dcc005230 0x7f3de40cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.097+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 >> 172.21.15.71:0/4116939202 conn(0x7f3de40cfa20 msgr2=0x7f3de40c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:33.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.098+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 shutdown_connections 2024-09-16T07:27:33.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.098+0000 7f3deb200640 1 -- 172.21.15.71:0/4116939202 wait complete. 2024-09-16T07:27:33.113 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:27:33.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:27:33.113 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:27:33.210 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:27:33.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:27:33.210 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:27:33.253 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:27:33.253 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:27:33.254 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:33.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:27:33.289 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:27:33.289 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:27:33.289 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:33.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:27:33.328 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:27:33.328 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:27:33.329 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:33.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:27:33.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:27:33.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.525+0000 7f0b76800640 1 -- 172.21.15.71:0/2251767183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 msgr2=0x7f0b700d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:33.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.525+0000 7f0b76800640 1 --2- 172.21.15.71:0/2251767183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 0x7f0b700d2760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009830 tx=0x7f0b6402a510 comp rx=0 tx=0).stop 2024-09-16T07:27:33.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.526+0000 7f0b76800640 1 -- 172.21.15.71:0/2251767183 shutdown_connections 2024-09-16T07:27:33.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.526+0000 7f0b76800640 1 --2- 172.21.15.71:0/2251767183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 0x7f0b700d2760 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.526+0000 7f0b76800640 1 -- 172.21.15.71:0/2251767183 >> 172.21.15.71:0/2251767183 conn(0x7f0b700d1c40 msgr2=0x7f0b700d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:33.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.526+0000 7f0b76800640 1 -- 172.21.15.71:0/2251767183 shutdown_connections 2024-09-16T07:27:33.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.526+0000 7f0b76800640 1 -- 172.21.15.71:0/2251767183 wait complete. 2024-09-16T07:27:33.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.527+0000 7f0b76800640 1 Processor -- start 2024-09-16T07:27:33.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.527+0000 7f0b76800640 1 -- start start 2024-09-16T07:27:33.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.528+0000 7f0b76800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700b1680 0x7f0b700ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.528+0000 7f0b76800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700cff60 0x7f0b700cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.528+0000 7f0b76800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 0x7f0b7015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.528+0000 7f0b76800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0b700b23c0 con 0x7f0b7015bd30 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.528+0000 7f0b76800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b700b2240 con 0x7f0b700cff60 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.528+0000 7f0b76800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0b700b2540 con 0x7f0b700b1680 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 0x7f0b7015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 0x7f0b7015ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46324/0 (socket says 172.21.15.71:46324) 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 -- 172.21.15.71:0/660795118 learned_addr learned my addr 172.21.15.71:0/660795118 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75400640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700b1680 0x7f0b700ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b74a00640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700cff60 0x7f0b700cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 -- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700b1680 msgr2=0x7f0b700ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:33.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700b1680 0x7f0b700ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 -- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700cff60 msgr2=0x7f0b700cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:33.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700cff60 0x7f0b700cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 -- 172.21.15.71:0/660795118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b700cbff0 con 0x7f0b7015bd30 2024-09-16T07:27:33.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.529+0000 7f0b75e00640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 0x7f0b7015ee80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800cbc0 tx=0x7f0b6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.530+0000 7f0b6d600640 1 -- 172.21.15.71:0/660795118 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68016020 con 0x7f0b7015bd30 2024-09-16T07:27:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.530+0000 7f0b6d600640 1 -- 172.21.15.71:0/660795118 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b6800ae30 con 0x7f0b7015bd30 2024-09-16T07:27:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.530+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b700caa10 con 0x7f0b7015bd30 2024-09-16T07:27:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.530+0000 7f0b6d600640 1 -- 172.21.15.71:0/660795118 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68012660 con 0x7f0b7015bd30 2024-09-16T07:27:33.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.530+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b700cafd0 con 0x7f0b7015bd30 2024-09-16T07:27:33.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.531+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b700d2760 con 0x7f0b7015bd30 2024-09-16T07:27:33.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.532+0000 7f0b6d600640 1 -- 172.21.15.71:0/660795118 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f0b68019020 con 0x7f0b7015bd30 2024-09-16T07:27:33.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.533+0000 7f0b6d600640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b500790c0 0x7f0b5007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:33.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.533+0000 7f0b6d600640 1 -- 172.21.15.71:0/660795118 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f0b6809bed0 con 0x7f0b7015bd30 2024-09-16T07:27:33.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.533+0000 7f0b75400640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b500790c0 0x7f0b5007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:33.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.534+0000 7f0b75400640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b500790c0 0x7f0b5007b580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b64034000 tx=0x7f0b64004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:33.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.537+0000 7f0b6d600640 1 -- 172.21.15.71:0/660795118 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0b68020030 con 0x7f0b7015bd30 2024-09-16T07:27:33.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.840+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b700cde20 con 0x7f0b7015bd30 2024-09-16T07:27:33.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.841+0000 7f0b6d600640 1 -- 172.21.15.71:0/660795118 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5226 (secure 0 0 0) 0x7f0b68062e80 con 0x7f0b7015bd30 2024-09-16T07:27:33.840 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:33.841 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":28,"btime":"2024-09-16T07:25:48:609067+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1366102730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1366102730},{"type":"v1","addr":"172.21.15.177:6836","nonce":1366102730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/796136206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":796136206},{"type":"v1","addr":"172.21.15.177:6837","nonce":796136206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3187236541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3187236541},{"type":"v1","addr":"172.21.15.71:6834","nonce":3187236541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:25:39.895186+0000","modified":"2024-09-16T07:25:42.903826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2966230103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2966230103},{"type":"v1","addr":"172.21.15.71:6835","nonce":2966230103}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-16T07:27:33.841 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 28 2024-09-16T07:27:33.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b500790c0 msgr2=0x7f0b5007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:33.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b500790c0 0x7f0b5007b580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b64034000 tx=0x7f0b64004730 comp rx=0 tx=0).stop 2024-09-16T07:27:33.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 msgr2=0x7f0b7015ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:33.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 0x7f0b7015ee80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800cbc0 tx=0x7f0b6800a430 comp rx=0 tx=0).stop 2024-09-16T07:27:33.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 shutdown_connections 2024-09-16T07:27:33.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b500790c0 0x7f0b5007b580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b7015bd30 0x7f0b7015ee80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700cff60 0x7f0b700cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 --2- 172.21.15.71:0/660795118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700b1680 0x7f0b700ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:33.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.843+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 >> 172.21.15.71:0/660795118 conn(0x7f0b700d1c40 msgr2=0x7f0b700aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:33.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.844+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 shutdown_connections 2024-09-16T07:27:33.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:33.844+0000 7f0b76800640 1 -- 172.21.15.71:0/660795118 wait complete. 2024-09-16T07:27:33.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:27:34.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.015+0000 7f05bbe00640 1 -- 172.21.15.71:0/2343278875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc0725d0 msgr2=0x7f05bc0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.015+0000 7f05bbe00640 1 --2- 172.21.15.71:0/2343278875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc0725d0 0x7f05bc0b28f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05a8005fd0 tx=0x7f05a802a5d0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.015+0000 7f05bbe00640 1 -- 172.21.15.71:0/2343278875 shutdown_connections 2024-09-16T07:27:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.015+0000 7f05bbe00640 1 --2- 172.21.15.71:0/2343278875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05bc0d0540 0x7f05bc06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.015+0000 7f05bbe00640 1 --2- 172.21.15.71:0/2343278875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc0725d0 0x7f05bc0b28f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.015+0000 7f05bbe00640 1 -- 172.21.15.71:0/2343278875 >> 172.21.15.71:0/2343278875 conn(0x7f05bc0cfa20 msgr2=0x7f05bc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.015+0000 7f05bbe00640 1 -- 172.21.15.71:0/2343278875 shutdown_connections 2024-09-16T07:27:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.015+0000 7f05bbe00640 1 -- 172.21.15.71:0/2343278875 wait complete. 2024-09-16T07:27:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.017+0000 7f05bbe00640 1 Processor -- start 2024-09-16T07:27:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.017+0000 7f05bbe00640 1 -- start start 2024-09-16T07:27:34.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.018+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05bc0d0540 0x7f05bc1f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.018+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05bc15bc30 0x7f05bc0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.018+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc1f4bc0 0x7f05bc0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.018+0000 7f05bbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc074660 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.018+0000 7f05bbe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc06f860 con 0x7f05bc15bc30 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.018+0000 7f05bbe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f05bc1f5620 con 0x7f05bc0d0540 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.018+0000 7f05bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc1f4bc0 0x7f05bc0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc1f4bc0 0x7f05bc0cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46332/0 (socket says 172.21.15.71:46332) 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bb400640 1 -- 172.21.15.71:0/3776812740 learned_addr learned my addr 172.21.15.71:0/3776812740 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05ba000640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05bc15bc30 0x7f05bc0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05baa00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05bc0d0540 0x7f05bc1f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bb400640 1 -- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05bc0d0540 msgr2=0x7f05bc1f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bb400640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05bc0d0540 0x7f05bc1f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bb400640 1 -- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05bc15bc30 msgr2=0x7f05bc0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bb400640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05bc15bc30 0x7f05bc0cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bb400640 1 -- 172.21.15.71:0/3776812740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc1f5060 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05ba000640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05bc15bc30 0x7f05bc0cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:34.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bb400640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc1f4bc0 0x7f05bc0cce80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f05b400eaa0 tx=0x7f05b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:34.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05b2a00640 1 -- 172.21.15.71:0/3776812740 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b400cdb0 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05b2a00640 1 -- 172.21.15.71:0/3776812740 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05b4004c10 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.019+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc1f5340 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.020+0000 7f05b2a00640 1 -- 172.21.15.71:0/3776812740 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4005020 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.020+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc0c6f00 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.021+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05bc06fdf0 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.022+0000 7f05b2a00640 1 -- 172.21.15.71:0/3776812740 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f05b40051c0 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.023+0000 7f05b2a00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f059c0790c0 0x7f059c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.023+0000 7f05b2a00640 1 -- 172.21.15.71:0/3776812740 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f05b4099b90 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.023+0000 7f05baa00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f059c0790c0 0x7f059c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.024+0000 7f05baa00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f059c0790c0 0x7f059c07b580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0015e0 tx=0x7f05ac009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:34.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.027+0000 7f05b2a00640 1 -- 172.21.15.71:0/3776812740 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f05b4016030 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.249+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05bc15b2f0 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.251+0000 7f05b2a00640 1 -- 172.21.15.71:0/3776812740 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5226 (secure 0 0 0) 0x7f05b4063cb0 con 0x7f05bc1f4bc0 2024-09-16T07:27:34.251 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:34.251 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":28,"btime":"2024-09-16T07:25:48:609067+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1366102730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1366102730},{"type":"v1","addr":"172.21.15.177:6836","nonce":1366102730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/796136206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":796136206},{"type":"v1","addr":"172.21.15.177:6837","nonce":796136206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3187236541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3187236541},{"type":"v1","addr":"172.21.15.71:6834","nonce":3187236541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:25:39.895186+0000","modified":"2024-09-16T07:25:42.903826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2966230103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2966230103},{"type":"v1","addr":"172.21.15.71:6835","nonce":2966230103}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-16T07:27:34.251 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 28 2024-09-16T07:27:34.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f059c0790c0 msgr2=0x7f059c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f059c0790c0 0x7f059c07b580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0015e0 tx=0x7f05ac009040 comp rx=0 tx=0).stop 2024-09-16T07:27:34.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc1f4bc0 msgr2=0x7f05bc0cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc1f4bc0 0x7f05bc0cce80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f05b400eaa0 tx=0x7f05b400ef60 comp rx=0 tx=0).stop 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 shutdown_connections 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f059c0790c0 0x7f059c07b580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05bc1f4bc0 0x7f05bc0cce80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05bc15bc30 0x7f05bc0cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 --2- 172.21.15.71:0/3776812740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05bc0d0540 0x7f05bc1f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 >> 172.21.15.71:0/3776812740 conn(0x7f05bc0cfa20 msgr2=0x7f05bc15a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 shutdown_connections 2024-09-16T07:27:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.254+0000 7f05bbe00640 1 -- 172.21.15.71:0/3776812740 wait complete. 2024-09-16T07:27:34.269 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.427+0000 7f4973e00640 1 -- 172.21.15.71:0/1864817504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f497415bc30 msgr2=0x7f49740b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.427+0000 7f4973e00640 1 --2- 172.21.15.71:0/1864817504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f497415bc30 0x7f49740b28f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4974073b00 tx=0x7f496402a600 comp rx=0 tx=0).stop 2024-09-16T07:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.427+0000 7f4973e00640 1 -- 172.21.15.71:0/1864817504 shutdown_connections 2024-09-16T07:27:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.427+0000 7f4973e00640 1 --2- 172.21.15.71:0/1864817504 >> v2:172.21.15.177:3300/0 conn(0x7f49740725d0 0x7f4974073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.427+0000 7f4973e00640 1 --2- 172.21.15.71:0/1864817504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f497415bc30 0x7f49740b28f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.427+0000 7f4973e00640 1 -- 172.21.15.71:0/1864817504 >> 172.21.15.71:0/1864817504 conn(0x7f49740cfa20 msgr2=0x7f49740cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.428+0000 7f4973e00640 1 -- 172.21.15.71:0/1864817504 shutdown_connections 2024-09-16T07:27:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.428+0000 7f4973e00640 1 -- 172.21.15.71:0/1864817504 wait complete. 2024-09-16T07:27:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.429+0000 7f4973e00640 1 Processor -- start 2024-09-16T07:27:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.429+0000 7f4973e00640 1 -- start start 2024-09-16T07:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.430+0000 7f4973e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49740d0540 0x7f4974157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.430+0000 7f4973e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f497415bc30 0x7f497414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.430+0000 7f4973e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497414ebe0 0x7f49740ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.430+0000 7f4973e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4974074610 con 0x7f49740d0540 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.430+0000 7f4973e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4974074490 con 0x7f497415bc30 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.430+0000 7f4973e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4974074790 con 0x7f497414ebe0 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.430+0000 7f4972a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49740d0540 0x7f4974157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.430+0000 7f4972a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49740d0540 0x7f4974157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46348/0 (socket says 172.21.15.71:46348) 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4972a00640 1 -- 172.21.15.71:0/2934285489 learned_addr learned my addr 172.21.15.71:0/2934285489 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f496be00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f497415bc30 0x7f497414c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4973400640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497414ebe0 0x7f49740ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4972a00640 1 -- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497414ebe0 msgr2=0x7f49740ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4972a00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497414ebe0 0x7f49740ad9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4972a00640 1 -- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f497415bc30 msgr2=0x7f497414c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4972a00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f497415bc30 0x7f497414c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4972a00640 1 -- 172.21.15.71:0/2934285489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497414cdf0 con 0x7f49740d0540 2024-09-16T07:27:34.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4973400640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497414ebe0 0x7f49740ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:34.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f4972a00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49740d0540 0x7f4974157f80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4964034040 tx=0x7f49640039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:34.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.431+0000 7f496aa00640 1 -- 172.21.15.71:0/2934285489 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496403e030 con 0x7f49740d0540 2024-09-16T07:27:34.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.432+0000 7f496aa00640 1 -- 172.21.15.71:0/2934285489 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49640047a0 con 0x7f49740d0540 2024-09-16T07:27:34.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.432+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497414d0d0 con 0x7f49740d0540 2024-09-16T07:27:34.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.432+0000 7f496aa00640 1 -- 172.21.15.71:0/2934285489 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964035040 con 0x7f49740d0540 2024-09-16T07:27:34.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.432+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497414d690 con 0x7f49740d0540 2024-09-16T07:27:34.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.433+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f497406fd20 con 0x7f49740d0540 2024-09-16T07:27:34.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.434+0000 7f496aa00640 1 -- 172.21.15.71:0/2934285489 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f4964041030 con 0x7f49740d0540 2024-09-16T07:27:34.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.435+0000 7f496aa00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49500790c0 0x7f495007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.435+0000 7f496aa00640 1 -- 172.21.15.71:0/2934285489 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f49640b9720 con 0x7f49740d0540 2024-09-16T07:27:34.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.435+0000 7f496be00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49500790c0 0x7f495007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.436+0000 7f496be00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49500790c0 0x7f495007b580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49740b38b0 tx=0x7f495c00b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:34.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.439+0000 7f496aa00640 1 -- 172.21.15.71:0/2934285489 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f496402ae20 con 0x7f49740d0540 2024-09-16T07:27:34.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.627+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49740b51d0 con 0x7f49740d0540 2024-09-16T07:27:34.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.628+0000 7f496aa00640 1 -- 172.21.15.71:0/2934285489 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+17081 (secure 0 0 0) 0x7f49640831f0 con 0x7f49740d0540 2024-09-16T07:27:34.628 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:34.628 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:27:31.360603+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:25:37.657272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T07:25:38.850365+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":5,"to":1}]},{"pgid":"7.30","mappings":[{"from":5,"to":7}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:27:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49500790c0 msgr2=0x7f495007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49500790c0 0x7f495007b580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49740b38b0 tx=0x7f495c00b390 comp rx=0 tx=0).stop 2024-09-16T07:27:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49740d0540 msgr2=0x7f4974157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49740d0540 0x7f4974157f80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4964034040 tx=0x7f49640039e0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 shutdown_connections 2024-09-16T07:27:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49500790c0 0x7f495007b580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f497414ebe0 0x7f49740ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f497415bc30 0x7f497414c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 --2- 172.21.15.71:0/2934285489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49740d0540 0x7f4974157f80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.630+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 >> 172.21.15.71:0/2934285489 conn(0x7f49740cfa20 msgr2=0x7f49740c8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.631+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 shutdown_connections 2024-09-16T07:27:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.631+0000 7f4973e00640 1 -- 172.21.15.71:0/2934285489 wait complete. 2024-09-16T07:27:34.646 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:27:34.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:27:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.803+0000 7f318ea00640 1 -- 172.21.15.71:0/2943077312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31880ccf00 msgr2=0x7f31880b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.803+0000 7f318ea00640 1 --2- 172.21.15.71:0/2943077312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31880ccf00 0x7f31880b2440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f318806dac0 tx=0x7f317002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.803+0000 7f318ea00640 1 -- 172.21.15.71:0/2943077312 shutdown_connections 2024-09-16T07:27:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.803+0000 7f318ea00640 1 --2- 172.21.15.71:0/2943077312 >> v2:172.21.15.177:3300/0 conn(0x7f3188073fb0 0x7f3188074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.803+0000 7f318ea00640 1 --2- 172.21.15.71:0/2943077312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31880ccf00 0x7f31880b2440 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.803+0000 7f318ea00640 1 -- 172.21.15.71:0/2943077312 >> 172.21.15.71:0/2943077312 conn(0x7f31880b1950 msgr2=0x7f31880b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.803+0000 7f318ea00640 1 -- 172.21.15.71:0/2943077312 shutdown_connections 2024-09-16T07:27:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.803+0000 7f318ea00640 1 -- 172.21.15.71:0/2943077312 wait complete. 2024-09-16T07:27:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.805+0000 7f318ea00640 1 Processor -- start 2024-09-16T07:27:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.805+0000 7f318ea00640 1 -- start start 2024-09-16T07:27:34.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ea00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3188073fb0 0x7f31880cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31880ccf00 0x7f3188153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f318815f5a0 0x7f31880cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f318806f2e0 con 0x7f318815f5a0 2024-09-16T07:27:34.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ea00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f318806f160 con 0x7f31880ccf00 2024-09-16T07:27:34.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ea00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f318806f460 con 0x7f3188073fb0 2024-09-16T07:27:34.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f318815f5a0 0x7f31880cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f318815f5a0 0x7f31880cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40152/0 (socket says 172.21.15.71:40152) 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ca00640 1 -- 172.21.15.71:0/573364803 learned_addr learned my addr 172.21.15.71:0/573364803 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f317fe00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31880ccf00 0x7f3188153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f3187e00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3188073fb0 0x7f31880cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ca00640 1 -- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3188073fb0 msgr2=0x7f31880cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ca00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3188073fb0 0x7f31880cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ca00640 1 -- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31880ccf00 msgr2=0x7f3188153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.806+0000 7f318ca00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31880ccf00 0x7f3188153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.807+0000 7f318ca00640 1 -- 172.21.15.71:0/573364803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31880ce020 con 0x7f318815f5a0 2024-09-16T07:27:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.807+0000 7f318ca00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f318815f5a0 0x7f31880cb870 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f318000b9a0 tx=0x7f318000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.807+0000 7f3184c00640 1 -- 172.21.15.71:0/573364803 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180019070 con 0x7f318815f5a0 2024-09-16T07:27:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.807+0000 7f3184c00640 1 -- 172.21.15.71:0/573364803 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f318000ccb0 con 0x7f318815f5a0 2024-09-16T07:27:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.807+0000 7f3184c00640 1 -- 172.21.15.71:0/573364803 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31800146a0 con 0x7f318815f5a0 2024-09-16T07:27:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.807+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318815a660 con 0x7f318815f5a0 2024-09-16T07:27:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.807+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318815abe0 con 0x7f318815f5a0 2024-09-16T07:27:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.808+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318806fa30 con 0x7f318815f5a0 2024-09-16T07:27:34.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.809+0000 7f3184c00640 1 -- 172.21.15.71:0/573364803 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f318001c070 con 0x7f318815f5a0 2024-09-16T07:27:34.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.810+0000 7f3184c00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31640790c0 0x7f316407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.810+0000 7f3184c00640 1 -- 172.21.15.71:0/573364803 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f318009b970 con 0x7f318815f5a0 2024-09-16T07:27:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.811+0000 7f3187e00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31640790c0 0x7f316407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:34.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.811+0000 7f3187e00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31640790c0 0x7f316407b580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3178006fd0 tx=0x7f3178009060 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:34.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:34.815+0000 7f3184c00640 1 -- 172.21.15.71:0/573364803 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3180062890 con 0x7f318815f5a0 2024-09-16T07:27:35.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.035+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3188154f00 con 0x7f318815f5a0 2024-09-16T07:27:35.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.036+0000 7f3184c00640 1 -- 172.21.15.71:0/573364803 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5226 (secure 0 0 0) 0x7f318009b090 con 0x7f318815f5a0 2024-09-16T07:27:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:35.035 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":28,"btime":"2024-09-16T07:25:48:609067+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1366102730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1366102730},{"type":"v1","addr":"172.21.15.177:6836","nonce":1366102730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/796136206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":796136206},{"type":"v1","addr":"172.21.15.177:6837","nonce":796136206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3187236541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3187236541},{"type":"v1","addr":"172.21.15.71:6834","nonce":3187236541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:25:39.895186+0000","modified":"2024-09-16T07:25:42.903826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2966230103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2966230103},{"type":"v1","addr":"172.21.15.71:6835","nonce":2966230103}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-16T07:27:35.036 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 28 2024-09-16T07:27:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31640790c0 msgr2=0x7f316407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31640790c0 0x7f316407b580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3178006fd0 tx=0x7f3178009060 comp rx=0 tx=0).stop 2024-09-16T07:27:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f318815f5a0 msgr2=0x7f31880cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f318815f5a0 0x7f31880cb870 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f318000b9a0 tx=0x7f318000be60 comp rx=0 tx=0).stop 2024-09-16T07:27:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 shutdown_connections 2024-09-16T07:27:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31640790c0 0x7f316407b580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f318815f5a0 0x7f31880cb870 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31880ccf00 0x7f3188153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 --2- 172.21.15.71:0/573364803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3188073fb0 0x7f31880cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 >> 172.21.15.71:0/573364803 conn(0x7f31880b1950 msgr2=0x7f318809d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 shutdown_connections 2024-09-16T07:27:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.038+0000 7f318ea00640 1 -- 172.21.15.71:0/573364803 wait complete. 2024-09-16T07:27:35.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:27:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.212+0000 7f205ba00640 1 -- 172.21.15.71:0/4003755351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2054073fb0 msgr2=0x7f2054074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.212+0000 7f205ba00640 1 --2- 172.21.15.71:0/4003755351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2054073fb0 0x7f2054074440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f204c005fd0 tx=0x7f204c02a7a0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.212+0000 7f205ba00640 1 -- 172.21.15.71:0/4003755351 shutdown_connections 2024-09-16T07:27:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.212+0000 7f205ba00640 1 --2- 172.21.15.71:0/4003755351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2054073fb0 0x7f2054074440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.212+0000 7f205ba00640 1 --2- 172.21.15.71:0/4003755351 >> v2:172.21.15.177:3300/0 conn(0x7f20540ccf00 0x7f20540b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.212+0000 7f205ba00640 1 -- 172.21.15.71:0/4003755351 >> 172.21.15.71:0/4003755351 conn(0x7f20540b1950 msgr2=0x7f20540b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.212+0000 7f205ba00640 1 -- 172.21.15.71:0/4003755351 shutdown_connections 2024-09-16T07:27:35.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.212+0000 7f205ba00640 1 -- 172.21.15.71:0/4003755351 wait complete. 2024-09-16T07:27:35.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.214+0000 7f205ba00640 1 Processor -- start 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.214+0000 7f205ba00640 1 -- start start 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f205ba00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2054073fb0 0x7f20541f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f205ba00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20540ccf00 0x7f20540cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f205ba00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f205415f5a0 0x7f2054157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f205ba00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f205406f2e0 con 0x7f205415f5a0 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f205ba00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f205406f160 con 0x7f2054073fb0 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f205ba00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f205406f460 con 0x7f20540ccf00 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f205415f5a0 0x7f2054157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f205415f5a0 0x7f2054157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40164/0 (socket says 172.21.15.71:40164) 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059a00640 1 -- 172.21.15.71:0/1917183631 learned_addr learned my addr 172.21.15.71:0/1917183631 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059000640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2054073fb0 0x7f20541f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2053e00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20540ccf00 0x7f20540cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059a00640 1 -- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20540ccf00 msgr2=0x7f20540cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059a00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20540ccf00 0x7f20540cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059a00640 1 -- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2054073fb0 msgr2=0x7f20541f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059a00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2054073fb0 0x7f20541f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.215+0000 7f2059a00640 1 -- 172.21.15.71:0/1917183631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2054156470 con 0x7f205415f5a0 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.216+0000 7f2059a00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f205415f5a0 0x7f2054157a20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f204c005fd0 tx=0x7f204c032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.216+0000 7f2050c00640 1 -- 172.21.15.71:0/1917183631 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c041020 con 0x7f205415f5a0 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.216+0000 7f2050c00640 1 -- 172.21.15.71:0/1917183631 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f204c004040 con 0x7f205415f5a0 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.216+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2054156750 con 0x7f205415f5a0 2024-09-16T07:27:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.216+0000 7f2050c00640 1 -- 172.21.15.71:0/1917183631 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c036040 con 0x7f205415f5a0 2024-09-16T07:27:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.216+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2054156d10 con 0x7f205415f5a0 2024-09-16T07:27:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.217+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2054154ff0 con 0x7f205415f5a0 2024-09-16T07:27:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.218+0000 7f2050c00640 1 -- 172.21.15.71:0/1917183631 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f204c044020 con 0x7f205415f5a0 2024-09-16T07:27:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.219+0000 7f2050c00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f20340790c0 0x7f203407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.219+0000 7f2050c00640 1 -- 172.21.15.71:0/1917183631 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f204c0b9d90 con 0x7f205415f5a0 2024-09-16T07:27:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.220+0000 7f2059000640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f20340790c0 0x7f203407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.221+0000 7f2059000640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f20340790c0 0x7f203407b580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2044001eb0 tx=0x7f2044009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:35.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.223+0000 7f2050c00640 1 -- 172.21.15.71:0/1917183631 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2054154ff0 con 0x7f205415f5a0 2024-09-16T07:27:35.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.445+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20540acf80 con 0x7f205415f5a0 2024-09-16T07:27:35.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.446+0000 7f2050c00640 1 -- 172.21.15.71:0/1917183631 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5226 (secure 0 0 0) 0x7f204c04b070 con 0x7f205415f5a0 2024-09-16T07:27:35.445 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:35.445 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":28,"btime":"2024-09-16T07:25:48:609067+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1366102730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1366102730},{"type":"v1","addr":"172.21.15.177:6836","nonce":1366102730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/796136206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":796136206},{"type":"v1","addr":"172.21.15.177:6837","nonce":796136206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3187236541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3187236541},{"type":"v1","addr":"172.21.15.71:6834","nonce":3187236541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:25:39.895186+0000","modified":"2024-09-16T07:25:42.903826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2966230103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2966230103},{"type":"v1","addr":"172.21.15.71:6835","nonce":2966230103}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5433,"qdb_cluster":[5433]},"id":3}]} 2024-09-16T07:27:35.446 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 28 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f20340790c0 msgr2=0x7f203407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f20340790c0 0x7f203407b580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2044001eb0 tx=0x7f2044009040 comp rx=0 tx=0).stop 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f205415f5a0 msgr2=0x7f2054157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f205415f5a0 0x7f2054157a20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f204c005fd0 tx=0x7f204c032be0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 shutdown_connections 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f20340790c0 0x7f203407b580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f205415f5a0 0x7f2054157a20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20540ccf00 0x7f20540cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 --2- 172.21.15.71:0/1917183631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2054073fb0 0x7f20541f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 >> 172.21.15.71:0/1917183631 conn(0x7f20540b1950 msgr2=0x7f205409ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:35.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 shutdown_connections 2024-09-16T07:27:35.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.448+0000 7f205ba00640 1 -- 172.21.15.71:0/1917183631 wait complete. 2024-09-16T07:27:35.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:27:35.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.621+0000 7fc5b0000640 1 -- 172.21.15.71:0/2118264259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a81445d0 msgr2=0x7fc5a80cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.621+0000 7fc5b0000640 1 --2- 172.21.15.71:0/2118264259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a81445d0 0x7fc5a80cebf0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009830 tx=0x7fc59c02a5a0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.621+0000 7fc5b0000640 1 -- 172.21.15.71:0/2118264259 shutdown_connections 2024-09-16T07:27:35.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.621+0000 7fc5b0000640 1 --2- 172.21.15.71:0/2118264259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5a80ccfd0 0x7fc5a80cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.621+0000 7fc5b0000640 1 --2- 172.21.15.71:0/2118264259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a81445d0 0x7fc5a80cebf0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.621+0000 7fc5b0000640 1 -- 172.21.15.71:0/2118264259 >> 172.21.15.71:0/2118264259 conn(0x7fc5a80b1950 msgr2=0x7fc5a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:35.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.622+0000 7fc5b0000640 1 -- 172.21.15.71:0/2118264259 shutdown_connections 2024-09-16T07:27:35.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.622+0000 7fc5b0000640 1 -- 172.21.15.71:0/2118264259 wait complete. 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.623+0000 7fc5b0000640 1 Processor -- start 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.623+0000 7fc5b0000640 1 -- start start 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5b0000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5a80b4ca0 0x7fc5a80c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5b0000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a80ccfd0 0x7fc5a806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5b0000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a81445d0 0x7fc5a80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5b0000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a815f080 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5b0000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a8073bc0 con 0x7fc5a81445d0 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5b0000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a815f200 con 0x7fc5a80b4ca0 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5acc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a80ccfd0 0x7fc5a806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:35.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5acc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a80ccfd0 0x7fc5a806fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40174/0 (socket says 172.21.15.71:40174) 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.624+0000 7fc5acc00640 1 -- 172.21.15.71:0/3541420334 learned_addr learned my addr 172.21.15.71:0/3541420334 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5ad600640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5a80b4ca0 0x7fc5a80c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5ae000640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a81445d0 0x7fc5a80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5acc00640 1 -- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5a80b4ca0 msgr2=0x7fc5a80c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5acc00640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5a80b4ca0 0x7fc5a80c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5acc00640 1 -- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a81445d0 msgr2=0x7fc5a80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5acc00640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a81445d0 0x7fc5a80ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5acc00640 1 -- 172.21.15.71:0/3541420334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a8070230 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5ae000640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a81445d0 0x7fc5a80ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5acc00640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a80ccfd0 0x7fc5a806fcf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc59400d950 tx=0x7fc59400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5a5600640 1 -- 172.21.15.71:0/3541420334 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594017070 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5a5600640 1 -- 172.21.15.71:0/3541420334 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc594009d20 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a81f52e0 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5a5600640 1 -- 172.21.15.71:0/3541420334 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5940106c0 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.625+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a81f58a0 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.626+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a815f6b0 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.627+0000 7fc5a5600640 1 -- 172.21.15.71:0/3541420334 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fc594010860 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.628+0000 7fc5a5600640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5880790c0 0x7fc58807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.629+0000 7fc5a5600640 1 -- 172.21.15.71:0/3541420334 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7fc59409c090 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.629+0000 7fc5ad600640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5880790c0 0x7fc58807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:35.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.630+0000 7fc5ad600640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5880790c0 0x7fc58807b580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc59c034000 tx=0x7fc59c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:35.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.632+0000 7fc5a5600640 1 -- 172.21.15.71:0/3541420334 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc594063690 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.817+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5a80cc330 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.819+0000 7fc5a5600640 1 -- 172.21.15.71:0/3541420334 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+17081 (secure 0 0 0) 0x7fc594062f90 con 0x7fc5a80ccfd0 2024-09-16T07:27:35.819 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:35.819 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:27:31.360603+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:25:37.657272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T07:25:38.850365+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":5,"to":1}]},{"pgid":"7.30","mappings":[{"from":5,"to":7}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:27:35.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.822+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5880790c0 msgr2=0x7fc58807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.822+0000 7fc5b0000640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5880790c0 0x7fc58807b580 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc59c034000 tx=0x7fc59c004730 comp rx=0 tx=0).stop 2024-09-16T07:27:35.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.822+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a80ccfd0 msgr2=0x7fc5a806fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:35.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.822+0000 7fc5b0000640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a80ccfd0 0x7fc5a806fcf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc59400d950 tx=0x7fc59400de10 comp rx=0 tx=0).stop 2024-09-16T07:27:35.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.823+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 shutdown_connections 2024-09-16T07:27:35.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.823+0000 7fc5b0000640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5880790c0 0x7fc58807b580 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.823+0000 7fc5b0000640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5a81445d0 0x7fc5a80ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.823+0000 7fc5b0000640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5a80ccfd0 0x7fc5a806fcf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.823+0000 7fc5b0000640 1 --2- 172.21.15.71:0/3541420334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5a80b4ca0 0x7fc5a80c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:35.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.823+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 >> 172.21.15.71:0/3541420334 conn(0x7fc5a80b1950 msgr2=0x7fc5a8142210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:35.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.823+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 shutdown_connections 2024-09-16T07:27:35.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:35.823+0000 7fc5b0000640 1 -- 172.21.15.71:0/3541420334 wait complete. 2024-09-16T07:27:35.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:27:36.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.004+0000 7f6d8a400640 1 -- 172.21.15.71:0/2308461449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d840cc860 msgr2=0x7f6d840b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:36.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.004+0000 7f6d8a400640 1 --2- 172.21.15.71:0/2308461449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d840cc860 0x7f6d840b04f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c02a940 comp rx=0 tx=0).stop 2024-09-16T07:27:36.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.004+0000 7f6d8a400640 1 -- 172.21.15.71:0/2308461449 shutdown_connections 2024-09-16T07:27:36.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.004+0000 7f6d8a400640 1 --2- 172.21.15.71:0/2308461449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d840cc860 0x7f6d840b04f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.004+0000 7f6d8a400640 1 -- 172.21.15.71:0/2308461449 >> 172.21.15.71:0/2308461449 conn(0x7f6d840b1e90 msgr2=0x7f6d840b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:36.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.004+0000 7f6d8a400640 1 -- 172.21.15.71:0/2308461449 shutdown_connections 2024-09-16T07:27:36.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.004+0000 7f6d8a400640 1 -- 172.21.15.71:0/2308461449 wait complete. 2024-09-16T07:27:36.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.006+0000 7f6d8a400640 1 Processor -- start 2024-09-16T07:27:36.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.006+0000 7f6d8a400640 1 -- start start 2024-09-16T07:27:36.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d8a400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d84074ca0 0x7f6d840c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:36.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d8a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d840cc860 0x7f6d84155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:36.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d8a400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d840ce100 0x7f6d840b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:36.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d8a400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6d840b0f00 con 0x7f6d840cc860 2024-09-16T07:27:36.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d8a400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d840b0d80 con 0x7f6d84074ca0 2024-09-16T07:27:36.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d8a400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d840b1080 con 0x7f6d840ce100 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d840cc860 0x7f6d84155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d840cc860 0x7f6d84155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40180/0 (socket says 172.21.15.71:40180) 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d83e00640 1 -- 172.21.15.71:0/934687159 learned_addr learned my addr 172.21.15.71:0/934687159 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.007+0000 7f6d89000640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d84074ca0 0x7f6d840c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d89a00640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d840ce100 0x7f6d840b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d83e00640 1 -- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d840ce100 msgr2=0x7f6d840b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d83e00640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d840ce100 0x7f6d840b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d83e00640 1 -- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d84074ca0 msgr2=0x7f6d840c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d83e00640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d84074ca0 0x7f6d840c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d83e00640 1 -- 172.21.15.71:0/934687159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d840c5590 con 0x7f6d840cc860 2024-09-16T07:27:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d83e00640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d840cc860 0x7f6d84155900 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c460 tx=0x7f6d7000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:36.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d80c00640 1 -- 172.21.15.71:0/934687159 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70016020 con 0x7f6d840cc860 2024-09-16T07:27:36.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d80c00640 1 -- 172.21.15.71:0/934687159 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d70012a00 con 0x7f6d840cc860 2024-09-16T07:27:36.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d80c00640 1 -- 172.21.15.71:0/934687159 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70012e50 con 0x7f6d840cc860 2024-09-16T07:27:36.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d841563d0 con 0x7f6d840cc860 2024-09-16T07:27:36.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.008+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d840c8d20 con 0x7f6d840cc860 2024-09-16T07:27:36.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.009+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d840ce4e0 con 0x7f6d840cc860 2024-09-16T07:27:36.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.010+0000 7f6d80c00640 1 -- 172.21.15.71:0/934687159 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f6d70019020 con 0x7f6d840cc860 2024-09-16T07:27:36.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.011+0000 7f6d80c00640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d64079260 0x7f6d6407b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:36.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.011+0000 7f6d80c00640 1 -- 172.21.15.71:0/934687159 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f6d7009b680 con 0x7f6d840cc860 2024-09-16T07:27:36.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.012+0000 7f6d89000640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d64079260 0x7f6d6407b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.013+0000 7f6d89000640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d64079260 0x7f6d6407b720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d841f78c0 tx=0x7f6d78009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:36.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.015+0000 7f6d80c00640 1 -- 172.21.15.71:0/934687159 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6d70020070 con 0x7f6d840cc860 2024-09-16T07:27:36.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.238+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6d8406dea0 con 0x7f6d840cc860 2024-09-16T07:27:36.238 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:27:36.239+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:27:36.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.241+0000 7f6d80c00640 1 -- 172.21.15.71:0/934687159 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f6d70062c80 con 0x7f6d840cc860 2024-09-16T07:27:36.240 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:27:36.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d64079260 msgr2=0x7f6d6407b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:36.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d64079260 0x7f6d6407b720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d841f78c0 tx=0x7f6d78009040 comp rx=0 tx=0).stop 2024-09-16T07:27:36.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d840cc860 msgr2=0x7f6d84155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d840cc860 0x7f6d84155900 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c460 tx=0x7f6d7000c920 comp rx=0 tx=0).stop 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 shutdown_connections 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d64079260 0x7f6d6407b720 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d840ce100 0x7f6d840b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d840cc860 0x7f6d84155900 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 --2- 172.21.15.71:0/934687159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d84074ca0 0x7f6d840c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 >> 172.21.15.71:0/934687159 conn(0x7f6d840b1e90 msgr2=0x7f6d8406f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 shutdown_connections 2024-09-16T07:27:36.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.245+0000 7f6d8a400640 1 -- 172.21.15.71:0/934687159 wait complete. 2024-09-16T07:27:36.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:27:36.311 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:27:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.426+0000 7f0d8c800640 1 -- 172.21.15.71:0/3570627049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840cec60 msgr2=0x7f0d840cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.426+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3570627049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840cec60 0x7f0d840cf090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0d80005fd0 tx=0x7f0d8002a820 comp rx=0 tx=0).stop 2024-09-16T07:27:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.426+0000 7f0d8c800640 1 -- 172.21.15.71:0/3570627049 shutdown_connections 2024-09-16T07:27:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.426+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3570627049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840cec60 0x7f0d840cf090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.426+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3570627049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d84161760 0x7f0d840ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.426+0000 7f0d8c800640 1 -- 172.21.15.71:0/3570627049 >> 172.21.15.71:0/3570627049 conn(0x7f0d840b1840 msgr2=0x7f0d840b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.426+0000 7f0d8c800640 1 -- 172.21.15.71:0/3570627049 shutdown_connections 2024-09-16T07:27:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.426+0000 7f0d8c800640 1 -- 172.21.15.71:0/3570627049 wait complete. 2024-09-16T07:27:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.427+0000 7f0d8c800640 1 Processor -- start 2024-09-16T07:27:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.427+0000 7f0d8c800640 1 -- start start 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.428+0000 7f0d8c800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840af190 0x7f0d8415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.428+0000 7f0d8c800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d840cec60 0x7f0d841f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.428+0000 7f0d8c800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d84161760 0x7f0d840b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.428+0000 7f0d8c800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84162190 con 0x7f0d840cec60 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.428+0000 7f0d8c800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84070250 con 0x7f0d84161760 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.428+0000 7f0d8c800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d84162310 con 0x7f0d840af190 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d840cec60 0x7f0d841f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d840cec60 0x7f0d841f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40208/0 (socket says 172.21.15.71:40208) 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 -- 172.21.15.71:0/3191777357 learned_addr learned my addr 172.21.15.71:0/3191777357 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d8a800640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d84161760 0x7f0d840b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89e00640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840af190 0x7f0d8415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 -- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840af190 msgr2=0x7f0d8415c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840af190 0x7f0d8415c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 -- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d84161760 msgr2=0x7f0d840b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d84161760 0x7f0d840b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 -- 172.21.15.71:0/3191777357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d841f7f30 con 0x7f0d840cec60 2024-09-16T07:27:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d8a800640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d84161760 0x7f0d840b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.429+0000 7f0d89400640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d840cec60 0x7f0d841f5670 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c490 tx=0x7f0d7400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.430+0000 7f0d7e000640 1 -- 172.21.15.71:0/3191777357 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74016020 con 0x7f0d840cec60 2024-09-16T07:27:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.430+0000 7f0d7e000640 1 -- 172.21.15.71:0/3191777357 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d74012940 con 0x7f0d840cec60 2024-09-16T07:27:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.430+0000 7f0d7e000640 1 -- 172.21.15.71:0/3191777357 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74012d90 con 0x7f0d840cec60 2024-09-16T07:27:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.430+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d841f8210 con 0x7f0d840cec60 2024-09-16T07:27:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.430+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841f87d0 con 0x7f0d840cec60 2024-09-16T07:27:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.431+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d840cf090 con 0x7f0d840cec60 2024-09-16T07:27:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.432+0000 7f0d7e000640 1 -- 172.21.15.71:0/3191777357 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f0d74019020 con 0x7f0d840cec60 2024-09-16T07:27:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.433+0000 7f0d7e000640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d680790c0 0x7f0d6807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.433+0000 7f0d7e000640 1 -- 172.21.15.71:0/3191777357 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5968+0+0 (secure 0 0 0) 0x7f0d74098ee0 con 0x7f0d840cec60 2024-09-16T07:27:36.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.433+0000 7f0d89e00640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d680790c0 0x7f0d6807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.434+0000 7f0d89e00640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d680790c0 0x7f0d6807b580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d840b1020 tx=0x7f0d78005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:36.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.437+0000 7f0d7e000640 1 -- 172.21.15.71:0/3191777357 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0d74020030 con 0x7f0d840cec60 2024-09-16T07:27:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:36.664+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0d840b2c50 con 0x7f0d840cec60 2024-09-16T07:27:37.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.247+0000 7f0d7e000640 1 -- 172.21.15.71:0/3191777357 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f0d74062f40 con 0x7f0d840cec60 2024-09-16T07:27:37.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.249+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d680790c0 msgr2=0x7f0d6807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:37.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.249+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d680790c0 0x7f0d6807b580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d840b1020 tx=0x7f0d78005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:37.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.249+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d840cec60 msgr2=0x7f0d841f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.249+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d840cec60 0x7f0d841f5670 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c490 tx=0x7f0d7400c950 comp rx=0 tx=0).stop 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.250+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 shutdown_connections 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.250+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d680790c0 0x7f0d6807b580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.250+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d84161760 0x7f0d840b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.250+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d840cec60 0x7f0d841f5670 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.250+0000 7f0d8c800640 1 --2- 172.21.15.71:0/3191777357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840af190 0x7f0d8415c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.250+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 >> 172.21.15.71:0/3191777357 conn(0x7f0d840b1840 msgr2=0x7f0d8415dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.250+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 shutdown_connections 2024-09-16T07:27:37.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.250+0000 7f0d8c800640 1 -- 172.21.15.71:0/3191777357 wait complete. 2024-09-16T07:27:37.267 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:27:37.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.432+0000 7fecf1e00640 1 -- 172.21.15.71:0/440467207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec06ee00 msgr2=0x7fecec0ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:37.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.432+0000 7fecf1e00640 1 --2- 172.21.15.71:0/440467207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec06ee00 0x7fecec0ad6d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fece4005fd0 tx=0x7fece402a9f0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.432+0000 7fecf1e00640 1 -- 172.21.15.71:0/440467207 shutdown_connections 2024-09-16T07:27:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.432+0000 7fecf1e00640 1 --2- 172.21.15.71:0/440467207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec06ee00 0x7fecec0ad6d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.432+0000 7fecf1e00640 1 --2- 172.21.15.71:0/440467207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecec15dfc0 0x7fecec06f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.432+0000 7fecf1e00640 1 -- 172.21.15.71:0/440467207 >> 172.21.15.71:0/440467207 conn(0x7fecec0b18e0 msgr2=0x7fecec0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.432+0000 7fecf1e00640 1 -- 172.21.15.71:0/440467207 shutdown_connections 2024-09-16T07:27:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.432+0000 7fecf1e00640 1 -- 172.21.15.71:0/440467207 wait complete. 2024-09-16T07:27:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.434+0000 7fecf1e00640 1 Processor -- start 2024-09-16T07:27:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.434+0000 7fecf1e00640 1 -- start start 2024-09-16T07:27:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecf1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecec06ee00 0x7fecec1f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecf1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec0b4770 0x7fecec1f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecf1e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecec15dfc0 0x7fecec148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecf1e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fecec0ae020 con 0x7fecec0b4770 2024-09-16T07:27:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecf1e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fecec0adea0 con 0x7fecec06ee00 2024-09-16T07:27:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecf1e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fecec0ae1a0 con 0x7fecec15dfc0 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecea000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec0b4770 0x7fecec1f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecea000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec0b4770 0x7fecec1f88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40226/0 (socket says 172.21.15.71:40226) 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7fecea000640 1 -- 172.21.15.71:0/2899899111 learned_addr learned my addr 172.21.15.71:0/2899899111 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7feceaa00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecec06ee00 0x7fecec1f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.435+0000 7feceb400640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecec15dfc0 0x7fecec148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fecea000640 1 -- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecec15dfc0 msgr2=0x7fecec148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fecea000640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecec15dfc0 0x7fecec148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fecea000640 1 -- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecec06ee00 msgr2=0x7fecec1f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fecea000640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecec06ee00 0x7fecec1f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fecea000640 1 -- 172.21.15.71:0/2899899111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec1f49b0 con 0x7fecec0b4770 2024-09-16T07:27:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fecea000640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec0b4770 0x7fecec1f88f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fecd800c400 tx=0x7fecd800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fece3400640 1 -- 172.21.15.71:0/2899899111 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8011790 con 0x7fecec0b4770 2024-09-16T07:27:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fece3400640 1 -- 172.21.15.71:0/2899899111 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecd8017070 con 0x7fecec0b4770 2024-09-16T07:27:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fece3400640 1 -- 172.21.15.71:0/2899899111 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8011d00 con 0x7fecec0b4770 2024-09-16T07:27:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec1f8e30 con 0x7fecec0b4770 2024-09-16T07:27:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.436+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec148fc0 con 0x7fecec0b4770 2024-09-16T07:27:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.437+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecec069360 con 0x7fecec0b4770 2024-09-16T07:27:37.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.438+0000 7fece3400640 1 -- 172.21.15.71:0/2899899111 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fecd801c460 con 0x7fecec0b4770 2024-09-16T07:27:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.439+0000 7fece3400640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feccc0790f0 0x7feccc07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.440+0000 7fece3400640 1 -- 172.21.15.71:0/2899899111 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5968+0+0 (secure 0 0 0) 0x7fecd8098e70 con 0x7fecec0b4770 2024-09-16T07:27:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.440+0000 7feceaa00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feccc0790f0 0x7feccc07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.441+0000 7feceaa00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feccc0790f0 0x7feccc07b5b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fecdc005fd0 tx=0x7fecdc00a250 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:37.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.444+0000 7fece3400640 1 -- 172.21.15.71:0/2899899111 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fecec1f49b0 con 0x7fecec0b4770 2024-09-16T07:27:37.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:37.628+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fecec0ad6d0 con 0x7fecec0b4770 2024-09-16T07:27:38.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.146+0000 7fece3400640 1 -- 172.21.15.71:0/2899899111 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) ==== 181+0+0 (secure 0 0 0) 0x7fecd8062900 con 0x7fecec0b4770 2024-09-16T07:27:38.146 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:27:38.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feccc0790f0 msgr2=0x7feccc07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:38.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feccc0790f0 0x7feccc07b5b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fecdc005fd0 tx=0x7fecdc00a250 comp rx=0 tx=0).stop 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec0b4770 msgr2=0x7fecec1f88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec0b4770 0x7fecec1f88f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fecd800c400 tx=0x7fecd800c8c0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 shutdown_connections 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feccc0790f0 0x7feccc07b5b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecec15dfc0 0x7fecec148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecec0b4770 0x7fecec1f88f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 --2- 172.21.15.71:0/2899899111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecec06ee00 0x7fecec1f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 >> 172.21.15.71:0/2899899111 conn(0x7fecec0b18e0 msgr2=0x7fecec0b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 shutdown_connections 2024-09-16T07:27:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.149+0000 7fecf1e00640 1 -- 172.21.15.71:0/2899899111 wait complete. 2024-09-16T07:27:38.167 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:27:38.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.329+0000 7f94af800640 1 -- 172.21.15.71:0/2155031046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a806f920 msgr2=0x7f94a815e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:38.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.329+0000 7f94af800640 1 --2- 172.21.15.71:0/2155031046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a806f920 0x7f94a815e490 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009830 tx=0x7f94a002a500 comp rx=0 tx=0).stop 2024-09-16T07:27:38.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.329+0000 7f94af800640 1 -- 172.21.15.71:0/2155031046 shutdown_connections 2024-09-16T07:27:38.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.329+0000 7f94af800640 1 --2- 172.21.15.71:0/2155031046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a806f920 0x7f94a815e490 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.329+0000 7f94af800640 1 --2- 172.21.15.71:0/2155031046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94a80b4350 0x7f94a806f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.329+0000 7f94af800640 1 -- 172.21.15.71:0/2155031046 >> 172.21.15.71:0/2155031046 conn(0x7f94a80b18e0 msgr2=0x7f94a80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.330+0000 7f94af800640 1 -- 172.21.15.71:0/2155031046 shutdown_connections 2024-09-16T07:27:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.330+0000 7f94af800640 1 -- 172.21.15.71:0/2155031046 wait complete. 2024-09-16T07:27:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.331+0000 7f94af800640 1 Processor -- start 2024-09-16T07:27:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.331+0000 7f94af800640 1 -- start start 2024-09-16T07:27:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94af800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94a806f920 0x7f94a8147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94af800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a80b4350 0x7f94a809d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94af800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94a81550c0 0x7f94a81f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94af800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f94a815eb50 con 0x7f94a80b4350 2024-09-16T07:27:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94af800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f94a815e9d0 con 0x7f94a81550c0 2024-09-16T07:27:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94af800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f94a815ecd0 con 0x7f94a806f920 2024-09-16T07:27:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a80b4350 0x7f94a809d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a80b4350 0x7f94a809d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40244/0 (socket says 172.21.15.71:40244) 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.332+0000 7f94a7e00640 1 -- 172.21.15.71:0/734265091 learned_addr learned my addr 172.21.15.71:0/734265091 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94ad800640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94a81550c0 0x7f94a81f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94ace00640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94a806f920 0x7f94a8147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a7e00640 1 -- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94a806f920 msgr2=0x7f94a8147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a7e00640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94a806f920 0x7f94a8147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a7e00640 1 -- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94a81550c0 msgr2=0x7f94a81f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a7e00640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94a81550c0 0x7f94a81f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a7e00640 1 -- 172.21.15.71:0/734265091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a8148210 con 0x7f94a80b4350 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94ace00640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94a806f920 0x7f94a8147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94ad800640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94a81550c0 0x7f94a81f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a7e00640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a80b4350 0x7f94a809d570 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f949400e9c0 tx=0x7f949400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a4c00640 1 -- 172.21.15.71:0/734265091 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949400cd80 con 0x7f94a80b4350 2024-09-16T07:27:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a4c00640 1 -- 172.21.15.71:0/734265091 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9494004020 con 0x7f94a80b4350 2024-09-16T07:27:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94a4c00640 1 -- 172.21.15.71:0/734265091 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494004450 con 0x7f94a80b4350 2024-09-16T07:27:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.333+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a809dab0 con 0x7f94a80b4350 2024-09-16T07:27:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.334+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a8162110 con 0x7f94a80b4350 2024-09-16T07:27:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.334+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a8069360 con 0x7f94a80b4350 2024-09-16T07:27:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.335+0000 7f94a4c00640 1 -- 172.21.15.71:0/734265091 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f94940035f0 con 0x7f94a80b4350 2024-09-16T07:27:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.336+0000 7f94a4c00640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f94880790c0 0x7f948807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.337+0000 7f94a4c00640 1 -- 172.21.15.71:0/734265091 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5530+0+0 (secure 0 0 0) 0x7f949409c350 con 0x7f94a80b4350 2024-09-16T07:27:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.337+0000 7f94ace00640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f94880790c0 0x7f948807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:38.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.338+0000 7f94ace00640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f94880790c0 0x7f948807b580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94980017a0 tx=0x7f9498009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:38.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.341+0000 7f94a4c00640 1 -- 172.21.15.71:0/734265091 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f94a8148210 con 0x7f94a80b4350 2024-09-16T07:27:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:38.529+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f94a815e6b0 con 0x7f94a80b4350 2024-09-16T07:27:39.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.393+0000 7f94a4c00640 1 -- 172.21.15.71:0/734265091 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) ==== 169+0+0 (secure 0 0 0) 0x7f9494063e50 con 0x7f94a80b4350 2024-09-16T07:27:39.393 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:27:39.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f94880790c0 msgr2=0x7f948807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f94880790c0 0x7f948807b580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94980017a0 tx=0x7f9498009040 comp rx=0 tx=0).stop 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a80b4350 msgr2=0x7f94a809d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a80b4350 0x7f94a809d570 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f949400e9c0 tx=0x7f949400ee80 comp rx=0 tx=0).stop 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 shutdown_connections 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f94880790c0 0x7f948807b580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94a81550c0 0x7f94a81f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a80b4350 0x7f94a809d570 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 --2- 172.21.15.71:0/734265091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94a806f920 0x7f94a8147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 >> 172.21.15.71:0/734265091 conn(0x7f94a80b18e0 msgr2=0x7f94a8073ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 shutdown_connections 2024-09-16T07:27:39.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.396+0000 7f94af800640 1 -- 172.21.15.71:0/734265091 wait complete. 2024-09-16T07:27:39.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:27:39.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.576+0000 7fa857e00640 1 -- 172.21.15.71:0/1502701067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850072df0 msgr2=0x7fa8500732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.576+0000 7fa857e00640 1 --2- 172.21.15.71:0/1502701067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850072df0 0x7fa8500732a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa85006d9e0 tx=0x7fa84802a4a0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.576+0000 7fa857e00640 1 -- 172.21.15.71:0/1502701067 shutdown_connections 2024-09-16T07:27:39.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.576+0000 7fa857e00640 1 --2- 172.21.15.71:0/1502701067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850072df0 0x7fa8500732a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.576+0000 7fa857e00640 1 --2- 172.21.15.71:0/1502701067 >> v2:172.21.15.177:3300/0 conn(0x7fa850161550 0x7fa85006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.576+0000 7fa857e00640 1 -- 172.21.15.71:0/1502701067 >> 172.21.15.71:0/1502701067 conn(0x7fa8500b1950 msgr2=0x7fa8500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:39.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.576+0000 7fa857e00640 1 -- 172.21.15.71:0/1502701067 shutdown_connections 2024-09-16T07:27:39.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.576+0000 7fa857e00640 1 -- 172.21.15.71:0/1502701067 wait complete. 2024-09-16T07:27:39.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.578+0000 7fa857e00640 1 Processor -- start 2024-09-16T07:27:39.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.578+0000 7fa857e00640 1 -- start start 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa857e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850072df0 0x7fa85009c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa857e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500af3d0 0x7fa8500cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa857e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850161550 0x7fa8501f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa857e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8500737e0 con 0x7fa850161550 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa857e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa85006fdb0 con 0x7fa850072df0 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa857e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa850073960 con 0x7fa8500af3d0 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa855e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850161550 0x7fa8501f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa855e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850161550 0x7fa8501f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40254/0 (socket says 172.21.15.71:40254) 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa855e00640 1 -- 172.21.15.71:0/121063395 learned_addr learned my addr 172.21.15.71:0/121063395 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa855400640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850072df0 0x7fa85009c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:39.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.579+0000 7fa854a00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500af3d0 0x7fa8500cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:39.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa855e00640 1 -- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500af3d0 msgr2=0x7fa8500cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa855e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500af3d0 0x7fa8500cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa855e00640 1 -- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850072df0 msgr2=0x7fa85009c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa855e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850072df0 0x7fa85009c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa855e00640 1 -- 172.21.15.71:0/121063395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85009acb0 con 0x7fa850161550 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa855e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850161550 0x7fa8501f6290 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa84802aa90 tx=0x7fa848032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa84d600640 1 -- 172.21.15.71:0/121063395 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848040070 con 0x7fa850161550 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa84d600640 1 -- 172.21.15.71:0/121063395 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa848005310 con 0x7fa850161550 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa84d600640 1 -- 172.21.15.71:0/121063395 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480047a0 con 0x7fa850161550 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85009af90 con 0x7fa850161550 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.580+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85009b550 con 0x7fa850161550 2024-09-16T07:27:39.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.582+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8500b3310 con 0x7fa850161550 2024-09-16T07:27:39.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.583+0000 7fa84d600640 1 -- 172.21.15.71:0/121063395 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fa848039040 con 0x7fa850161550 2024-09-16T07:27:39.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.584+0000 7fa84d600640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8300790c0 0x7fa83007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:39.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.584+0000 7fa84d600640 1 -- 172.21.15.71:0/121063395 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7fa8480bcad0 con 0x7fa850161550 2024-09-16T07:27:39.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.584+0000 7fa855400640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8300790c0 0x7fa83007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:39.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.585+0000 7fa855400640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8300790c0 0x7fa83007b580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa844005fd0 tx=0x7fa844009660 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.588+0000 7fa84d600640 1 -- 172.21.15.71:0/121063395 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8500b3310 con 0x7fa850161550 2024-09-16T07:27:39.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.787+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa8500cd3e0 con 0x7fa850161550 2024-09-16T07:27:39.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.791+0000 7fa84d600640 1 -- 172.21.15.71:0/121063395 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7fa848083c40 con 0x7fa850161550 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8300790c0 msgr2=0x7fa83007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8300790c0 0x7fa83007b580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa844005fd0 tx=0x7fa844009660 comp rx=0 tx=0).stop 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850161550 msgr2=0x7fa8501f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850161550 0x7fa8501f6290 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa84802aa90 tx=0x7fa848032c10 comp rx=0 tx=0).stop 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 shutdown_connections 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8300790c0 0x7fa83007b580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa850161550 0x7fa8501f6290 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500af3d0 0x7fa8500cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 --2- 172.21.15.71:0/121063395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850072df0 0x7fa85009c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 >> 172.21.15.71:0/121063395 conn(0x7fa8500b1950 msgr2=0x7fa8500ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:39.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 shutdown_connections 2024-09-16T07:27:39.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.794+0000 7fa857e00640 1 -- 172.21.15.71:0/121063395 wait complete. 2024-09-16T07:27:39.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-16T07:27:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.970+0000 7f0d5de00640 1 -- 172.21.15.71:0/4233031846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d580b1680 msgr2=0x7f0d580b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.970+0000 7f0d5de00640 1 --2- 172.21.15.71:0/4233031846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d580b1680 0x7f0d580b1b30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0d58074100 tx=0x7f0d5002aac0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.971+0000 7f0d5de00640 1 -- 172.21.15.71:0/4233031846 shutdown_connections 2024-09-16T07:27:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.971+0000 7f0d5de00640 1 --2- 172.21.15.71:0/4233031846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d580b1680 0x7f0d580b1b30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.971+0000 7f0d5de00640 1 --2- 172.21.15.71:0/4233031846 >> v2:172.21.15.177:3300/0 conn(0x7f0d5815bd30 0x7f0d580d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.971+0000 7f0d5de00640 1 -- 172.21.15.71:0/4233031846 >> 172.21.15.71:0/4233031846 conn(0x7f0d580d1c40 msgr2=0x7f0d580d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:39.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.971+0000 7f0d5de00640 1 -- 172.21.15.71:0/4233031846 shutdown_connections 2024-09-16T07:27:39.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.971+0000 7f0d5de00640 1 -- 172.21.15.71:0/4233031846 wait complete. 2024-09-16T07:27:39.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.972+0000 7f0d5de00640 1 Processor -- start 2024-09-16T07:27:39.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.972+0000 7f0d5de00640 1 -- start start 2024-09-16T07:27:39.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.973+0000 7f0d5de00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d580b1680 0x7f0d581f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:39.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.973+0000 7f0d5de00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d580cff60 0x7f0d58158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:39.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.973+0000 7f0d5de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d5815bd30 0x7f0d5806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.973+0000 7f0d5de00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580b21f0 con 0x7f0d5815bd30 2024-09-16T07:27:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.973+0000 7f0d5de00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580b2070 con 0x7f0d580b1680 2024-09-16T07:27:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.973+0000 7f0d5de00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d580b2370 con 0x7f0d580cff60 2024-09-16T07:27:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d5815bd30 0x7f0d5806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d5815bd30 0x7f0d5806ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40274/0 (socket says 172.21.15.71:40274) 2024-09-16T07:27:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 -- 172.21.15.71:0/1222839507 learned_addr learned my addr 172.21.15.71:0/1222839507 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5ca00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d580b1680 0x7f0d581f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d57e00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d580cff60 0x7f0d58158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:39.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 -- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d580cff60 msgr2=0x7f0d58158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d580cff60 0x7f0d58158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 -- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d580b1680 msgr2=0x7f0d581f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:39.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d580b1680 0x7f0d581f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:39.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 -- 172.21.15.71:0/1222839507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5806f7c0 con 0x7f0d5815bd30 2024-09-16T07:27:39.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d57e00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d580cff60 0x7f0d58158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:39.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.974+0000 7f0d5d400640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d5815bd30 0x7f0d5806ee20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0d5002b0b0 tx=0x7f0d50003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:39.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.975+0000 7f0d54c00640 1 -- 172.21.15.71:0/1222839507 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50044070 con 0x7f0d5815bd30 2024-09-16T07:27:39.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.975+0000 7f0d54c00640 1 -- 172.21.15.71:0/1222839507 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d50003e00 con 0x7f0d5815bd30 2024-09-16T07:27:39.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.975+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d581f8330 con 0x7f0d5815bd30 2024-09-16T07:27:39.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.975+0000 7f0d54c00640 1 -- 172.21.15.71:0/1222839507 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d500044b0 con 0x7f0d5815bd30 2024-09-16T07:27:39.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.975+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5806faa0 con 0x7f0d5815bd30 2024-09-16T07:27:39.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.976+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d580cd860 con 0x7f0d5815bd30 2024-09-16T07:27:39.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.977+0000 7f0d54c00640 1 -- 172.21.15.71:0/1222839507 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f0d50004650 con 0x7f0d5815bd30 2024-09-16T07:27:39.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.978+0000 7f0d54c00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d380790c0 0x7f0d3807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:39.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.978+0000 7f0d54c00640 1 -- 172.21.15.71:0/1222839507 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f0d500bbef0 con 0x7f0d5815bd30 2024-09-16T07:27:39.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.978+0000 7f0d5ca00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d380790c0 0x7f0d3807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:39.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.979+0000 7f0d5ca00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d380790c0 0x7f0d3807b580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0d581e97c0 tx=0x7f0d48005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:39.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:39.982+0000 7f0d54c00640 1 -- 172.21.15.71:0/1222839507 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0d580cd860 con 0x7f0d5815bd30 2024-09-16T07:27:40.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.235+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7f0d58072710 con 0x7f0d5815bd30 2024-09-16T07:27:40.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.395+0000 7f0d54c00640 1 -- 172.21.15.71:0/1222839507 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v234) ==== 166+0+0 (secure 0 0 0) 0x7f0d58072710 con 0x7f0d5815bd30 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d380790c0 msgr2=0x7f0d3807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d380790c0 0x7f0d3807b580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0d581e97c0 tx=0x7f0d48005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d5815bd30 msgr2=0x7f0d5806ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d5815bd30 0x7f0d5806ee20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0d5002b0b0 tx=0x7f0d50003c10 comp rx=0 tx=0).stop 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 shutdown_connections 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d380790c0 0x7f0d3807b580 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d5815bd30 0x7f0d5806ee20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d580cff60 0x7f0d58158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 --2- 172.21.15.71:0/1222839507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d580b1680 0x7f0d581f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 >> 172.21.15.71:0/1222839507 conn(0x7f0d580d1c40 msgr2=0x7f0d580b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:40.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 shutdown_connections 2024-09-16T07:27:40.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.399+0000 7f0d5de00640 1 -- 172.21.15.71:0/1222839507 wait complete. 2024-09-16T07:27:40.415 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-16T07:27:40.415 INFO:tasks.cephfs_test_runner:Starting test: test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-16T07:27:40.415 INFO:tasks.cephfs_test_runner:test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:27:40.416 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:27:40.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.575+0000 7fe984c00640 1 -- 172.21.15.71:0/1620865206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9640043b0 msgr2=0x7fe964004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.575+0000 7fe984c00640 1 --2- 172.21.15.71:0/1620865206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9640043b0 0x7fe964004880 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009b50 tx=0x7fe964005830 comp rx=0 tx=0).stop 2024-09-16T07:27:40.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.575+0000 7fe97fe00640 1 -- 172.21.15.71:0/1620865206 reap_dead start 2024-09-16T07:27:40.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.575+0000 7fe984c00640 1 -- 172.21.15.71:0/1620865206 shutdown_connections 2024-09-16T07:27:40.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.575+0000 7fe984c00640 1 -- 172.21.15.71:0/1620865206 >> 172.21.15.71:0/1620865206 conn(0x7fe9800cfa20 msgr2=0x7fe9800cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:40.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.576+0000 7fe984c00640 1 -- 172.21.15.71:0/1620865206 shutdown_connections 2024-09-16T07:27:40.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.576+0000 7fe984c00640 1 -- 172.21.15.71:0/1620865206 wait complete. 2024-09-16T07:27:40.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.577+0000 7fe984c00640 1 Processor -- start 2024-09-16T07:27:40.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.577+0000 7fe984c00640 1 -- start start 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe984c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9640043b0 0x7fe98014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe984c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9801e5520 0x7fe9801e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe984c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9801e8050 0x7fe9800ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe984c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe980074660 con 0x7fe9801e8050 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe984c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9800b28f0 con 0x7fe9801e5520 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe984c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9801e4100 con 0x7fe9640043b0 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe97fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9801e8050 0x7fe9800ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe97fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9801e8050 0x7fe9800ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40290/0 (socket says 172.21.15.71:40290) 2024-09-16T07:27:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.578+0000 7fe97fe00640 1 -- 172.21.15.71:0/4182295142 learned_addr learned my addr 172.21.15.71:0/4182295142 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:40.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe97ea00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9801e5520 0x7fe9801e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:40.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe97f400640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9640043b0 0x7fe98014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe97fe00640 1 -- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9640043b0 msgr2=0x7fe98014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe97fe00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9640043b0 0x7fe98014f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe97fe00640 1 -- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9801e5520 msgr2=0x7fe9801e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe97fe00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9801e5520 0x7fe9801e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe97fe00640 1 -- 172.21.15.71:0/4182295142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9801e4350 con 0x7fe9801e8050 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe97fe00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9801e8050 0x7fe9800ce490 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe980074a80 tx=0x7fe978010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe977400640 1 -- 172.21.15.71:0/4182295142 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97800edb0 con 0x7fe9801e8050 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe977400640 1 -- 172.21.15.71:0/4182295142 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe978015070 con 0x7fe9801e8050 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe977400640 1 -- 172.21.15.71:0/4182295142 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978003bd0 con 0x7fe9801e8050 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.579+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9801e4630 con 0x7fe9801e8050 2024-09-16T07:27:40.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.580+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9801e4bf0 con 0x7fe9801e8050 2024-09-16T07:27:40.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.581+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9800725d0 con 0x7fe9801e8050 2024-09-16T07:27:40.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.582+0000 7fe977400640 1 -- 172.21.15.71:0/4182295142 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fe978004030 con 0x7fe9801e8050 2024-09-16T07:27:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.583+0000 7fe977400640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9600790c0 0x7fe96007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.583+0000 7fe977400640 1 -- 172.21.15.71:0/4182295142 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7fe978099b90 con 0x7fe9801e8050 2024-09-16T07:27:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.583+0000 7fe97f400640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9600790c0 0x7fe96007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:40.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.584+0000 7fe97f400640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9600790c0 0x7fe96007b580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe970001d50 tx=0x7fe970001ce0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:40.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.586+0000 7fe977400640 1 -- 172.21.15.71:0/4182295142 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe978063660 con 0x7fe9801e8050 2024-09-16T07:27:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.772+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9801ea660 con 0x7fe9801e8050 2024-09-16T07:27:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.773+0000 7fe977400640 1 -- 172.21.15.71:0/4182295142 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+13044 (secure 0 0 0) 0x7fe978068090 con 0x7fe9801e8050 2024-09-16T07:27:40.772 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:40.773 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":54,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:27:39.385765+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/2966230103":"2024-09-17T07:27:36.240112+0000","172.21.15.71:6832/2966230103":"2024-09-17T07:27:36.240112+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:27:40.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9600790c0 msgr2=0x7fe96007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9600790c0 0x7fe96007b580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe970001d50 tx=0x7fe970001ce0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9801e8050 msgr2=0x7fe9800ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9801e8050 0x7fe9800ce490 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe980074a80 tx=0x7fe978010e80 comp rx=0 tx=0).stop 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 shutdown_connections 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9600790c0 0x7fe96007b580 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9801e8050 0x7fe9800ce490 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9801e5520 0x7fe9801e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 --2- 172.21.15.71:0/4182295142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9640043b0 0x7fe98014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 >> 172.21.15.71:0/4182295142 conn(0x7fe9800cfa20 msgr2=0x7fe98006f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 shutdown_connections 2024-09-16T07:27:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.775+0000 7fe984c00640 1 -- 172.21.15.71:0/4182295142 wait complete. 2024-09-16T07:27:40.791 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:27:40.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:27:40.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.951+0000 7f3a11200640 1 -- 172.21.15.71:0/869268722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c06e780 msgr2=0x7f3a0c0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.951+0000 7f3a11200640 1 --2- 172.21.15.71:0/869268722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c06e780 0x7f3a0c0ce970 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f39f8005fd0 tx=0x7f39f802a4f0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.952+0000 7f3a11200640 1 -- 172.21.15.71:0/869268722 shutdown_connections 2024-09-16T07:27:40.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.952+0000 7f3a11200640 1 --2- 172.21.15.71:0/869268722 >> v2:172.21.15.177:3300/0 conn(0x7f3a0c0ceeb0 0x7f3a0c0730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.952+0000 7f3a11200640 1 --2- 172.21.15.71:0/869268722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c06e780 0x7f3a0c0ce970 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.952+0000 7f3a11200640 1 -- 172.21.15.71:0/869268722 >> 172.21.15.71:0/869268722 conn(0x7f3a0c0b1510 msgr2=0x7f3a0c0b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:40.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.952+0000 7f3a11200640 1 -- 172.21.15.71:0/869268722 shutdown_connections 2024-09-16T07:27:40.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.952+0000 7f3a11200640 1 -- 172.21.15.71:0/869268722 wait complete. 2024-09-16T07:27:40.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.953+0000 7f3a11200640 1 Processor -- start 2024-09-16T07:27:40.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.953+0000 7f3a11200640 1 -- start start 2024-09-16T07:27:40.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.954+0000 7f3a11200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a0c06e780 0x7f3a0c14d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:40.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.954+0000 7f3a11200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0c0ceeb0 0x7f3a0c0b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:40.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.954+0000 7f3a11200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c15acf0 0x7f3a0c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:40.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.954+0000 7f3a11200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c073b50 con 0x7f3a0c15acf0 2024-09-16T07:27:40.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.954+0000 7f3a11200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c0739d0 con 0x7f3a0c0ceeb0 2024-09-16T07:27:40.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.954+0000 7f3a11200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0c073cd0 con 0x7f3a0c06e780 2024-09-16T07:27:40.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c15acf0 0x7f3a0c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:40.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c15acf0 0x7f3a0c0b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40310/0 (socket says 172.21.15.71:40310) 2024-09-16T07:27:40.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 -- 172.21.15.71:0/2426668557 learned_addr learned my addr 172.21.15.71:0/2426668557 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:40.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0b400640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a0c06e780 0x7f3a0c14d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:40.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0aa00640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0c0ceeb0 0x7f3a0c0b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 -- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a0c06e780 msgr2=0x7f3a0c14d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a0c06e780 0x7f3a0c14d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 -- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0c0ceeb0 msgr2=0x7f3a0c0b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0c0ceeb0 0x7f3a0c0b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 -- 172.21.15.71:0/2426668557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c0c86d0 con 0x7f3a0c15acf0 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a0be00640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c15acf0 0x7f3a0c0b6250 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400eaa0 tx=0x7f3a0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.955+0000 7f3a03400640 1 -- 172.21.15.71:0/2426668557 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0400cdb0 con 0x7f3a0c15acf0 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.956+0000 7f3a03400640 1 -- 172.21.15.71:0/2426668557 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a04004c10 con 0x7f3a0c15acf0 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.956+0000 7f3a03400640 1 -- 172.21.15.71:0/2426668557 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04005020 con 0x7f3a0c15acf0 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.956+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0c0c8970 con 0x7f3a0c15acf0 2024-09-16T07:27:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.956+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0c0c8f30 con 0x7f3a0c15acf0 2024-09-16T07:27:40.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.957+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a0c0741f0 con 0x7f3a0c15acf0 2024-09-16T07:27:40.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.958+0000 7f3a03400640 1 -- 172.21.15.71:0/2426668557 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f3a040051c0 con 0x7f3a0c15acf0 2024-09-16T07:27:40.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.959+0000 7f3a03400640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f39ec0790c0 0x7f39ec07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:40.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.959+0000 7f3a03400640 1 -- 172.21.15.71:0/2426668557 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f3a0409c210 con 0x7f3a0c15acf0 2024-09-16T07:27:40.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.959+0000 7f3a0b400640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f39ec0790c0 0x7f39ec07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:40.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.960+0000 7f3a0b400640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f39ec0790c0 0x7f39ec07b580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39fc006fd0 tx=0x7f39fc009060 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:40.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:40.963+0000 7f3a03400640 1 -- 172.21.15.71:0/2426668557 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3a04016030 con 0x7f3a0c15acf0 2024-09-16T07:27:41.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.149+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a0c069d20 con 0x7f3a0c15acf0 2024-09-16T07:27:41.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.150+0000 7f3a03400640 1 -- 172.21.15.71:0/2426668557 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f3a0409ae20 con 0x7f3a0c15acf0 2024-09-16T07:27:41.149 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:27:41.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f39ec0790c0 msgr2=0x7f39ec07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f39ec0790c0 0x7f39ec07b580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39fc006fd0 tx=0x7f39fc009060 comp rx=0 tx=0).stop 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c15acf0 msgr2=0x7f3a0c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c15acf0 0x7f3a0c0b6250 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400eaa0 tx=0x7f3a0400ef60 comp rx=0 tx=0).stop 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 shutdown_connections 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f39ec0790c0 0x7f39ec07b580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c15acf0 0x7f3a0c0b6250 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0c0ceeb0 0x7f3a0c0b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 --2- 172.21.15.71:0/2426668557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a0c06e780 0x7f3a0c14d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 >> 172.21.15.71:0/2426668557 conn(0x7f3a0c0b1510 msgr2=0x7f3a0c0b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 shutdown_connections 2024-09-16T07:27:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.152+0000 7f3a11200640 1 -- 172.21.15.71:0/2426668557 wait complete. 2024-09-16T07:27:41.168 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-16T07:27:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.325+0000 7fc05e800640 1 -- 172.21.15.71:0/3700365236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc058073de0 msgr2=0x7fc058074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.325+0000 7fc05e800640 1 --2- 172.21.15.71:0/3700365236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc058073de0 0x7fc058074240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc040005fd0 tx=0x7fc04002a540 comp rx=0 tx=0).stop 2024-09-16T07:27:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.325+0000 7fc05e800640 1 -- 172.21.15.71:0/3700365236 shutdown_connections 2024-09-16T07:27:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.325+0000 7fc05e800640 1 --2- 172.21.15.71:0/3700365236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc058073de0 0x7fc058074240 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.325+0000 7fc05e800640 1 --2- 172.21.15.71:0/3700365236 >> v2:172.21.15.177:3300/0 conn(0x7fc05815f9a0 0x7fc05815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.325+0000 7fc05e800640 1 -- 172.21.15.71:0/3700365236 >> 172.21.15.71:0/3700365236 conn(0x7fc0580b18e0 msgr2=0x7fc0580b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.325+0000 7fc05e800640 1 -- 172.21.15.71:0/3700365236 shutdown_connections 2024-09-16T07:27:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.325+0000 7fc05e800640 1 -- 172.21.15.71:0/3700365236 wait complete. 2024-09-16T07:27:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.327+0000 7fc05e800640 1 Processor -- start 2024-09-16T07:27:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.327+0000 7fc05e800640 1 -- start start 2024-09-16T07:27:41.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc05e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058073de0 0x7fc058151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:41.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc05e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc05815c3f0 0x7fc058156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc05e800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05815f9a0 0x7fc058155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc05e800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc0580cd3d0 con 0x7fc05815c3f0 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc05e800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0580ccbd0 con 0x7fc058073de0 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc05e800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0580cd550 con 0x7fc05815f9a0 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc04ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc05815c3f0 0x7fc058156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc04ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc05815c3f0 0x7fc058156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40324/0 (socket says 172.21.15.71:40324) 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc04ea00640 1 -- 172.21.15.71:0/638650927 learned_addr learned my addr 172.21.15.71:0/638650927 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc057400640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058073de0 0x7fc058151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc057e00640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05815f9a0 0x7fc058155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:41.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc04ea00640 1 -- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05815f9a0 msgr2=0x7fc058155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc04ea00640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05815f9a0 0x7fc058155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc04ea00640 1 -- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058073de0 msgr2=0x7fc058151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc04ea00640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058073de0 0x7fc058151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.328+0000 7fc04ea00640 1 -- 172.21.15.71:0/638650927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc058152440 con 0x7fc05815c3f0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.329+0000 7fc04ea00640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc05815c3f0 0x7fc058156a70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc040000c00 tx=0x7fc0400038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.329+0000 7fc04fe00640 1 -- 172.21.15.71:0/638650927 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04003f070 con 0x7fc05815c3f0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.329+0000 7fc04fe00640 1 -- 172.21.15.71:0/638650927 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0400047a0 con 0x7fc05815c3f0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.329+0000 7fc04fe00640 1 -- 172.21.15.71:0/638650927 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040035030 con 0x7fc05815c3f0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.329+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0581534d0 con 0x7fc05815c3f0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.329+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc058153a90 con 0x7fc05815c3f0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.330+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05815d1e0 con 0x7fc05815c3f0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.331+0000 7fc04fe00640 1 -- 172.21.15.71:0/638650927 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fc040042070 con 0x7fc05815c3f0 2024-09-16T07:27:41.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.332+0000 7fc04fe00640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0340790c0 0x7fc03407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:41.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.332+0000 7fc04fe00640 1 -- 172.21.15.71:0/638650927 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7fc0400bc960 con 0x7fc05815c3f0 2024-09-16T07:27:41.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.333+0000 7fc057400640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0340790c0 0x7fc03407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:41.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.333+0000 7fc057400640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0340790c0 0x7fc03407b580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc048009b30 tx=0x7fc048005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:41.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.336+0000 7fc04fe00640 1 -- 172.21.15.71:0/638650927 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc040083c50 con 0x7fc05815c3f0 2024-09-16T07:27:41.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:41.587+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7fc0580cd7b0 con 0x7fc05815c3f0 2024-09-16T07:27:42.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.403+0000 7fc04fe00640 1 -- 172.21.15.71:0/638650927 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v236) ==== 170+0+0 (secure 0 0 0) 0x7fc040088680 con 0x7fc05815c3f0 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0340790c0 msgr2=0x7fc03407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0340790c0 0x7fc03407b580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc048009b30 tx=0x7fc048005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc05815c3f0 msgr2=0x7fc058156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc05815c3f0 0x7fc058156a70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc040000c00 tx=0x7fc0400038a0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 shutdown_connections 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0340790c0 0x7fc03407b580 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05815f9a0 0x7fc058155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc05815c3f0 0x7fc058156a70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 --2- 172.21.15.71:0/638650927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058073de0 0x7fc058151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 >> 172.21.15.71:0/638650927 conn(0x7fc0580b18e0 msgr2=0x7fc05815ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:42.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 shutdown_connections 2024-09-16T07:27:42.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.406+0000 7fc05e800640 1 -- 172.21.15.71:0/638650927 wait complete. 2024-09-16T07:27:42.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:27:42.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.584+0000 7fcfb3a00640 1 -- 172.21.15.71:0/3857001254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfac161550 msgr2=0x7fcfac06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.584+0000 7fcfb3a00640 1 --2- 172.21.15.71:0/3857001254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfac161550 0x7fcfac06ef50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c005fd0 tx=0x7fcf9c02a840 comp rx=0 tx=0).stop 2024-09-16T07:27:42.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.584+0000 7fcfb3a00640 1 -- 172.21.15.71:0/3857001254 shutdown_connections 2024-09-16T07:27:42.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.584+0000 7fcfb3a00640 1 --2- 172.21.15.71:0/3857001254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfac161550 0x7fcfac06ef50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.584+0000 7fcfb3a00640 1 -- 172.21.15.71:0/3857001254 >> 172.21.15.71:0/3857001254 conn(0x7fcfac0b1950 msgr2=0x7fcfac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:42.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.584+0000 7fcfb3a00640 1 -- 172.21.15.71:0/3857001254 shutdown_connections 2024-09-16T07:27:42.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.584+0000 7fcfb3a00640 1 -- 172.21.15.71:0/3857001254 wait complete. 2024-09-16T07:27:42.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.586+0000 7fcfb3a00640 1 Processor -- start 2024-09-16T07:27:42.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.586+0000 7fcfb3a00640 1 -- start start 2024-09-16T07:27:42.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb3a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfac072df0 0x7fcfac142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb3a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfac0af3d0 0x7fcfac145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb3a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfac161550 0x7fcfac1f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb3a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac0739b0 con 0x7fcfac161550 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb3a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac06fdb0 con 0x7fcfac072df0 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb3a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcfac073b30 con 0x7fcfac0af3d0 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfac161550 0x7fcfac1f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfac161550 0x7fcfac1f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40332/0 (socket says 172.21.15.71:40332) 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb1a00640 1 -- 172.21.15.71:0/2079840554 learned_addr learned my addr 172.21.15.71:0/2079840554 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.587+0000 7fcfb1000640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfac072df0 0x7fcfac142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfabe00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfac0af3d0 0x7fcfac145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfb1a00640 1 -- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfac0af3d0 msgr2=0x7fcfac145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfb1a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfac0af3d0 0x7fcfac145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfb1a00640 1 -- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfac072df0 msgr2=0x7fcfac142940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfb1a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfac072df0 0x7fcfac142940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfb1a00640 1 -- 172.21.15.71:0/2079840554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac145fe0 con 0x7fcfac161550 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfabe00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfac0af3d0 0x7fcfac145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfb1a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfac161550 0x7fcfac1f7ef0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400c4d0 tx=0x7fcfa400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfa8c00640 1 -- 172.21.15.71:0/2079840554 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4016020 con 0x7fcfac161550 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfa8c00640 1 -- 172.21.15.71:0/2079840554 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfa4012920 con 0x7fcfac161550 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfa8c00640 1 -- 172.21.15.71:0/2079840554 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4012d70 con 0x7fcfac161550 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac0cdeb0 con 0x7fcfac161550 2024-09-16T07:27:42.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.588+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac0ce470 con 0x7fcfac161550 2024-09-16T07:27:42.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.589+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfac074040 con 0x7fcfac161550 2024-09-16T07:27:42.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.590+0000 7fcfa8c00640 1 -- 172.21.15.71:0/2079840554 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fcfa4019020 con 0x7fcfac161550 2024-09-16T07:27:42.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.591+0000 7fcfa8c00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf8c0790c0 0x7fcf8c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:42.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.591+0000 7fcfa8c00640 1 -- 172.21.15.71:0/2079840554 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7fcfa409b5e0 con 0x7fcfac161550 2024-09-16T07:27:42.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.592+0000 7fcfb1000640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf8c0790c0 0x7fcf8c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:42.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.592+0000 7fcfb1000640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf8c0790c0 0x7fcf8c07b580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c033000 tx=0x7fcf9c004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:42.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.595+0000 7fcfa8c00640 1 -- 172.21.15.71:0/2079840554 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcfa4020030 con 0x7fcfac161550 2024-09-16T07:27:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.780+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fcfac0744a0 con 0x7fcfac161550 2024-09-16T07:27:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.780+0000 7fcfa8c00640 1 -- 172.21.15.71:0/2079840554 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7fcfa4062fb0 con 0x7fcfac161550 2024-09-16T07:27:42.780 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:27:42.780 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:27:42.780 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:27:42.780 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:27:42.780 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:27:42.781 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:27:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:27:42.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.782+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf8c0790c0 msgr2=0x7fcf8c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.782+0000 7fcfb3a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf8c0790c0 0x7fcf8c07b580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c033000 tx=0x7fcf9c004ab0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.782+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfac161550 msgr2=0x7fcfac1f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.782+0000 7fcfb3a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfac161550 0x7fcfac1f7ef0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400c4d0 tx=0x7fcfa400c990 comp rx=0 tx=0).stop 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.783+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 shutdown_connections 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.783+0000 7fcfb3a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcf8c0790c0 0x7fcf8c07b580 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.783+0000 7fcfb3a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfac161550 0x7fcfac1f7ef0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.783+0000 7fcfb3a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfac0af3d0 0x7fcfac145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.783+0000 7fcfb3a00640 1 --2- 172.21.15.71:0/2079840554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfac072df0 0x7fcfac142940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.783+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 >> 172.21.15.71:0/2079840554 conn(0x7fcfac0b1950 msgr2=0x7fcfac06ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.783+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 shutdown_connections 2024-09-16T07:27:42.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.783+0000 7fcfb3a00640 1 -- 172.21.15.71:0/2079840554 wait complete. 2024-09-16T07:27:42.798 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:27:42.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.958+0000 7f1537800640 1 -- 172.21.15.71:0/2993260731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 msgr2=0x7f1530070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.958+0000 7f1537800640 1 --2- 172.21.15.71:0/2993260731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 0x7f1530070ba0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1528005fd0 tx=0x7f152802a940 comp rx=0 tx=0).stop 2024-09-16T07:27:42.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.958+0000 7f1537800640 1 -- 172.21.15.71:0/2993260731 shutdown_connections 2024-09-16T07:27:42.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.958+0000 7f1537800640 1 --2- 172.21.15.71:0/2993260731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 0x7f1530070ba0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.958+0000 7f1537800640 1 -- 172.21.15.71:0/2993260731 >> 172.21.15.71:0/2993260731 conn(0x7f15300b2130 msgr2=0x7f15300b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:42.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.958+0000 7f1537800640 1 -- 172.21.15.71:0/2993260731 shutdown_connections 2024-09-16T07:27:42.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.958+0000 7f1537800640 1 -- 172.21.15.71:0/2993260731 wait complete. 2024-09-16T07:27:42.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.960+0000 7f1537800640 1 Processor -- start 2024-09-16T07:27:42.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.960+0000 7f1537800640 1 -- start start 2024-09-16T07:27:42.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1537800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15300b3b40 0x7f1530145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:42.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1537800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1530154bc0 0x7f15300cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:42.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1537800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 0x7f1530160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:42.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1537800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f153006f650 con 0x7f1530154fa0 2024-09-16T07:27:42.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1537800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f153006f4d0 con 0x7f15300b3b40 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1537800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f153006f7d0 con 0x7f1530154bc0 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1535800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 0x7f1530160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1535800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 0x7f1530160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40352/0 (socket says 172.21.15.71:40352) 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1535800640 1 -- 172.21.15.71:0/2230321541 learned_addr learned my addr 172.21.15.71:0/2230321541 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1527e00640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1530154bc0 0x7f15300cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.961+0000 7f1534e00640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15300b3b40 0x7f1530145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f1535800640 1 -- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1530154bc0 msgr2=0x7f15300cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f1535800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1530154bc0 0x7f15300cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f1535800640 1 -- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15300b3b40 msgr2=0x7f1530145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f1535800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15300b3b40 0x7f1530145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f1535800640 1 -- 172.21.15.71:0/2230321541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1530161180 con 0x7f1530154fa0 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f1534e00640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15300b3b40 0x7f1530145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:42.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f1535800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 0x7f1530160810 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1528006010 tx=0x7f1528031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f152d600640 1 -- 172.21.15.71:0/2230321541 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152803e070 con 0x7f1530154fa0 2024-09-16T07:27:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f152d600640 1 -- 172.21.15.71:0/2230321541 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1528004540 con 0x7f1530154fa0 2024-09-16T07:27:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f152d600640 1 -- 172.21.15.71:0/2230321541 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528043460 con 0x7f1530154fa0 2024-09-16T07:27:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.962+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1530161460 con 0x7f1530154fa0 2024-09-16T07:27:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.963+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15300cd5d0 con 0x7f1530154fa0 2024-09-16T07:27:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.964+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153006fcf0 con 0x7f1530154fa0 2024-09-16T07:27:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.965+0000 7f152d600640 1 -- 172.21.15.71:0/2230321541 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f1528043600 con 0x7f1530154fa0 2024-09-16T07:27:42.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.966+0000 7f152d600640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f150c0790c0 0x7f150c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:42.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.966+0000 7f152d600640 1 -- 172.21.15.71:0/2230321541 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f15280bd390 con 0x7f1530154fa0 2024-09-16T07:27:42.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.966+0000 7f1534e00640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f150c0790c0 0x7f150c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:42.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.967+0000 7f1534e00640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f150c0790c0 0x7f150c07b580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1520007fd0 tx=0x7f1520009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:42.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:42.970+0000 7f152d600640 1 -- 172.21.15.71:0/2230321541 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1528084e10 con 0x7f1530154fa0 2024-09-16T07:27:43.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.155+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f153015e060 con 0x7f1530154fa0 2024-09-16T07:27:43.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.159+0000 7f152d600640 1 -- 172.21.15.71:0/2230321541 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f1528041e40 con 0x7f1530154fa0 2024-09-16T07:27:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f150c0790c0 msgr2=0x7f150c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f150c0790c0 0x7f150c07b580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1520007fd0 tx=0x7f1520009040 comp rx=0 tx=0).stop 2024-09-16T07:27:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 msgr2=0x7f1530160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 0x7f1530160810 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1528006010 tx=0x7f1528031e20 comp rx=0 tx=0).stop 2024-09-16T07:27:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 shutdown_connections 2024-09-16T07:27:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f150c0790c0 0x7f150c07b580 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1530154fa0 0x7f1530160810 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1530154bc0 0x7f15300cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 --2- 172.21.15.71:0/2230321541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15300b3b40 0x7f1530145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 >> 172.21.15.71:0/2230321541 conn(0x7f15300b2130 msgr2=0x7f15300b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 shutdown_connections 2024-09-16T07:27:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.161+0000 7f1537800640 1 -- 172.21.15.71:0/2230321541 wait complete. 2024-09-16T07:27:43.178 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:27:43.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:27:43.269 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:27:43.372 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:27:43.372 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:27:43.372 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:43.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:27:43.408 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:27:43.409 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:43.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:27:43.445 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:27:43.446 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:43.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:27:43.483 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:27:43.483 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:27:43.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:27:43.521 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:27:43.521 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:27:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.680+0000 7f3ebb800640 1 -- 172.21.15.71:0/516849071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb40ccfd0 msgr2=0x7f3eb40cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.680+0000 7f3ebb800640 1 --2- 172.21.15.71:0/516849071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb40ccfd0 0x7f3eb40cf130 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009900 tx=0x7f3ea002a610 comp rx=0 tx=0).stop 2024-09-16T07:27:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.680+0000 7f3ebb800640 1 -- 172.21.15.71:0/516849071 shutdown_connections 2024-09-16T07:27:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.680+0000 7f3ebb800640 1 --2- 172.21.15.71:0/516849071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb40ccfd0 0x7f3eb40cf130 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.680+0000 7f3ebb800640 1 --2- 172.21.15.71:0/516849071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3eb41445d0 0x7f3eb40cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.680+0000 7f3ebb800640 1 -- 172.21.15.71:0/516849071 >> 172.21.15.71:0/516849071 conn(0x7f3eb40b1950 msgr2=0x7f3eb40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.680+0000 7f3ebb800640 1 -- 172.21.15.71:0/516849071 shutdown_connections 2024-09-16T07:27:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.680+0000 7f3ebb800640 1 -- 172.21.15.71:0/516849071 wait complete. 2024-09-16T07:27:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.682+0000 7f3ebb800640 1 Processor -- start 2024-09-16T07:27:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.682+0000 7f3ebb800640 1 -- start start 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3ebb800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3eb40b4ca0 0x7f3eb413d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3ebb800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3eb40ccfd0 0x7f3eb4140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3ebb800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb41445d0 0x7f3eb413f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3ebb800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb415f080 con 0x7f3eb41445d0 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3ebb800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4073bc0 con 0x7f3eb40b4ca0 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3ebb800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb415f200 con 0x7f3eb40ccfd0 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3eb9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb41445d0 0x7f3eb413f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3eb9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb41445d0 0x7f3eb413f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40358/0 (socket says 172.21.15.71:40358) 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.683+0000 7f3eb9800640 1 -- 172.21.15.71:0/3333630435 learned_addr learned my addr 172.21.15.71:0/3333630435 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb8e00640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3eb40b4ca0 0x7f3eb413d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb3e00640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3eb40ccfd0 0x7f3eb4140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb9800640 1 -- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3eb40ccfd0 msgr2=0x7f3eb4140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb9800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3eb40ccfd0 0x7f3eb4140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb9800640 1 -- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3eb40b4ca0 msgr2=0x7f3eb413d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb9800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3eb40b4ca0 0x7f3eb413d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb9800640 1 -- 172.21.15.71:0/3333630435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb4140ca0 con 0x7f3eb41445d0 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb8e00640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3eb40b4ca0 0x7f3eb413d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb3e00640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3eb40ccfd0 0x7f3eb4140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb9800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb41445d0 0x7f3eb413f470 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c3c0 tx=0x7f3eac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb0c00640 1 -- 172.21.15.71:0/3333630435 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac017070 con 0x7f3eb41445d0 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb0c00640 1 -- 172.21.15.71:0/3333630435 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3eac0053c0 con 0x7f3eb41445d0 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb409ad30 con 0x7f3eb41445d0 2024-09-16T07:27:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3eb0c00640 1 -- 172.21.15.71:0/3333630435 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac003850 con 0x7f3eb41445d0 2024-09-16T07:27:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.684+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb4140240 con 0x7f3eb41445d0 2024-09-16T07:27:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.686+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb415f6b0 con 0x7f3eb41445d0 2024-09-16T07:27:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.686+0000 7f3eb0c00640 1 -- 172.21.15.71:0/3333630435 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f3eac01a070 con 0x7f3eb41445d0 2024-09-16T07:27:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.687+0000 7f3eb0c00640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e940790c0 0x7f3e9407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.688+0000 7f3eb0c00640 1 -- 172.21.15.71:0/3333630435 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f3eac09ac80 con 0x7f3eb41445d0 2024-09-16T07:27:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.688+0000 7f3eb8e00640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e940790c0 0x7f3e9407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.689+0000 7f3eb8e00640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e940790c0 0x7f3e9407b580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005fd0 tx=0x7f3ea4009480 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.691+0000 7f3eb0c00640 1 -- 172.21.15.71:0/3333630435 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3eac0625c0 con 0x7f3eb41445d0 2024-09-16T07:27:43.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.910+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3eb41eba60 con 0x7f3eb41445d0 2024-09-16T07:27:43.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.910+0000 7f3eb0c00640 1 -- 172.21.15.71:0/3333630435 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+3914 (secure 0 0 0) 0x7f3eac061ec0 con 0x7f3eb41445d0 2024-09-16T07:27:43.910 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:43.910 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"btime":"2024-09-16T07:27:37:242478+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1366102730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1366102730},{"type":"v1","addr":"172.21.15.177:6836","nonce":1366102730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/796136206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":796136206},{"type":"v1","addr":"172.21.15.177:6837","nonce":796136206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3187236541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3187236541},{"type":"v1","addr":"172.21.15.71:6834","nonce":3187236541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1776513947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1776513947},{"type":"v1","addr":"172.21.15.71:6835","nonce":1776513947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-09-16T07:27:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.912+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e940790c0 msgr2=0x7f3e9407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.912+0000 7f3ebb800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e940790c0 0x7f3e9407b580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005fd0 tx=0x7f3ea4009480 comp rx=0 tx=0).stop 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.912+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb41445d0 msgr2=0x7f3eb413f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.912+0000 7f3ebb800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb41445d0 0x7f3eb413f470 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c3c0 tx=0x7f3eac00c880 comp rx=0 tx=0).stop 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.913+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 shutdown_connections 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.913+0000 7f3ebb800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e940790c0 0x7f3e9407b580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.913+0000 7f3ebb800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb41445d0 0x7f3eb413f470 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.913+0000 7f3ebb800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3eb40ccfd0 0x7f3eb4140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.913+0000 7f3ebb800640 1 --2- 172.21.15.71:0/3333630435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3eb40b4ca0 0x7f3eb413d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.913+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 >> 172.21.15.71:0/3333630435 conn(0x7f3eb40b1950 msgr2=0x7f3eb40afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.913+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 shutdown_connections 2024-09-16T07:27:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:43.913+0000 7f3ebb800640 1 -- 172.21.15.71:0/3333630435 wait complete. 2024-09-16T07:27:43.929 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:27:43.929 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:27:43.929 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:27:43.929 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:27:43.929 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:27:43.930 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:27:43.930 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:27:43.930 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:27:43.930 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:27:43.930 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:27:43.930 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:27:43.930 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:27:43.930 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:27:43.931 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:27:43.931 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:27:43.931 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:27:43.945 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:27:43.945 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:27:43.946 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:27:43.946 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:27:43.947 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:27:43.947 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:27:43.948 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:27:43.948 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:27:43.990 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:27:43.990 DEBUG:teuthology.parallel:result is None 2024-09-16T07:27:43.991 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:27:43.991 DEBUG:teuthology.parallel:result is None 2024-09-16T07:27:43.992 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:27:43.992 DEBUG:teuthology.parallel:result is None 2024-09-16T07:27:43.994 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:27:43.994 DEBUG:teuthology.parallel:result is None 2024-09-16T07:27:43.994 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:27:44.117 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:27:44.117 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:27:44.121 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:27:44.123 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:27:44.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.163+0000 7fbacfe00640 1 -- 172.21.15.71:0/917675521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbad0073ee0 msgr2=0x7fbad006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.163+0000 7fbacfe00640 1 --2- 172.21.15.71:0/917675521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbad0073ee0 0x7fbad006f740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbac8005fd0 tx=0x7fbac802aa90 comp rx=0 tx=0).stop 2024-09-16T07:27:44.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.163+0000 7fbacfe00640 1 -- 172.21.15.71:0/917675521 shutdown_connections 2024-09-16T07:27:44.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.163+0000 7fbacfe00640 1 --2- 172.21.15.71:0/917675521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbad0073ee0 0x7fbad006f740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.163+0000 7fbacfe00640 1 --2- 172.21.15.71:0/917675521 >> v2:172.21.15.177:3300/0 conn(0x7fbad00cc8d0 0x7fbad015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.163+0000 7fbacfe00640 1 -- 172.21.15.71:0/917675521 >> 172.21.15.71:0/917675521 conn(0x7fbad00d0b30 msgr2=0x7fbad00d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:44.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.163+0000 7fbacfe00640 1 -- 172.21.15.71:0/917675521 shutdown_connections 2024-09-16T07:27:44.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.163+0000 7fbacfe00640 1 -- 172.21.15.71:0/917675521 wait complete. 2024-09-16T07:27:44.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.165+0000 7fbacfe00640 1 Processor -- start 2024-09-16T07:27:44.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.165+0000 7fbacfe00640 1 -- start start 2024-09-16T07:27:44.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacfe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0073ee0 0x7fbad01419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:44.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacfe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbad00cc8d0 0x7fbad00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:44.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad00d1620 0x7fbad00b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacfe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbad00744f0 con 0x7fbad00d1620 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacfe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0074370 con 0x7fbad0073ee0 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacfe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbad0074670 con 0x7fbad00cc8d0 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad00d1620 0x7fbad00b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad00d1620 0x7fbad00b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40394/0 (socket says 172.21.15.71:40394) 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacf400640 1 -- 172.21.15.71:0/886064445 learned_addr learned my addr 172.21.15.71:0/886064445 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbacea00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0073ee0 0x7fbad01419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.166+0000 7fbac7e00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbad00cc8d0 0x7fbad00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbacf400640 1 -- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbad00cc8d0 msgr2=0x7fbad00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbacf400640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbad00cc8d0 0x7fbad00b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbacf400640 1 -- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0073ee0 msgr2=0x7fbad01419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbacf400640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0073ee0 0x7fbad01419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbacf400640 1 -- 172.21.15.71:0/886064445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad01f3610 con 0x7fbad00d1620 2024-09-16T07:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbacf400640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad00d1620 0x7fbad00b1e70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009d90 tx=0x7fbac8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:44.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbac7400640 1 -- 172.21.15.71:0/886064445 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8046070 con 0x7fbad00d1620 2024-09-16T07:27:44.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbac7400640 1 -- 172.21.15.71:0/886064445 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbac8004040 con 0x7fbad00d1620 2024-09-16T07:27:44.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad01f38f0 con 0x7fbad00d1620 2024-09-16T07:27:44.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.167+0000 7fbac7400640 1 -- 172.21.15.71:0/886064445 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8004430 con 0x7fbad00d1620 2024-09-16T07:27:44.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.168+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad01f3eb0 con 0x7fbad00d1620 2024-09-16T07:27:44.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.169+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad00c78c0 con 0x7fbad00d1620 2024-09-16T07:27:44.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.170+0000 7fbac7400640 1 -- 172.21.15.71:0/886064445 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fbac8049070 con 0x7fbad00d1620 2024-09-16T07:27:44.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.171+0000 7fbac7400640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbaac0790c0 0x7fbaac07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:44.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.171+0000 7fbac7400640 1 -- 172.21.15.71:0/886064445 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7fbac80b9e40 con 0x7fbad00d1620 2024-09-16T07:27:44.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.171+0000 7fbacea00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbaac0790c0 0x7fbaac07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:44.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.172+0000 7fbacea00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbaac0790c0 0x7fbaac07b580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009bd0 tx=0x7fbac0005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:44.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.175+0000 7fbac7400640 1 -- 172.21.15.71:0/886064445 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbad00c78c0 con 0x7fbad00d1620 2024-09-16T07:27:44.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.368+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbad006ffb0 con 0x7fbad00d1620 2024-09-16T07:27:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.419+0000 7fbac7400640 1 -- 172.21.15.71:0/886064445 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) ==== 97+0+0 (secure 0 0 0) 0x7fbac8032030 con 0x7fbad00d1620 2024-09-16T07:27:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:27:44.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbaac0790c0 msgr2=0x7fbaac07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbaac0790c0 0x7fbaac07b580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009bd0 tx=0x7fbac0005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad00d1620 msgr2=0x7fbad00b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad00d1620 0x7fbad00b1e70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009d90 tx=0x7fbac8003ce0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 shutdown_connections 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbaac0790c0 0x7fbaac07b580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad00d1620 0x7fbad00b1e70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbad00cc8d0 0x7fbad00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 --2- 172.21.15.71:0/886064445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0073ee0 0x7fbad01419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.421+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 >> 172.21.15.71:0/886064445 conn(0x7fbad00d0b30 msgr2=0x7fbad00d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.422+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 shutdown_connections 2024-09-16T07:27:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.422+0000 7fbacfe00640 1 -- 172.21.15.71:0/886064445 wait complete. 2024-09-16T07:27:44.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:27:44.485 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:27:44.489 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:27:44.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.600+0000 7fbcdf600640 1 -- 172.21.15.71:0/189228318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd8073fb0 msgr2=0x7fbcd8074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.600+0000 7fbcdf600640 1 --2- 172.21.15.71:0/189228318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd8073fb0 0x7fbcd8074420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8075130 tx=0x7fbcd002aa30 comp rx=0 tx=0).stop 2024-09-16T07:27:44.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.600+0000 7fbcdf600640 1 -- 172.21.15.71:0/189228318 shutdown_connections 2024-09-16T07:27:44.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.600+0000 7fbcdf600640 1 --2- 172.21.15.71:0/189228318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd8073fb0 0x7fbcd8074420 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.600+0000 7fbcdf600640 1 --2- 172.21.15.71:0/189228318 >> v2:172.21.15.177:3300/0 conn(0x7fbcd815f5a0 0x7fbcd80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.600+0000 7fbcdf600640 1 -- 172.21.15.71:0/189228318 >> 172.21.15.71:0/189228318 conn(0x7fbcd80b1950 msgr2=0x7fbcd80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.600+0000 7fbcdf600640 1 -- 172.21.15.71:0/189228318 shutdown_connections 2024-09-16T07:27:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.600+0000 7fbcdf600640 1 -- 172.21.15.71:0/189228318 wait complete. 2024-09-16T07:27:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.602+0000 7fbcdf600640 1 Processor -- start 2024-09-16T07:27:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.602+0000 7fbcdf600640 1 -- start start 2024-09-16T07:27:44.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdf600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcd8073fb0 0x7fbcd81456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdf600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcd80ccf00 0x7fbcd815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdf600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd815f5a0 0x7fbcd809b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdf600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd806f2e0 con 0x7fbcd815f5a0 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdf600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd806f160 con 0x7fbcd8073fb0 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdf600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd806f460 con 0x7fbcd80ccf00 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdd600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd815f5a0 0x7fbcd809b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdd600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd815f5a0 0x7fbcd809b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40422/0 (socket says 172.21.15.71:40422) 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdd600640 1 -- 172.21.15.71:0/1411288193 learned_addr learned my addr 172.21.15.71:0/1411288193 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbccfe00640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcd80ccf00 0x7fbcd815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdcc00640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcd8073fb0 0x7fbcd81456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:44.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdd600640 1 -- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcd80ccf00 msgr2=0x7fbcd815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdd600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcd80ccf00 0x7fbcd815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdd600640 1 -- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcd8073fb0 msgr2=0x7fbcd81456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdd600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcd8073fb0 0x7fbcd81456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.603+0000 7fbcdd600640 1 -- 172.21.15.71:0/1411288193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd815bac0 con 0x7fbcd815f5a0 2024-09-16T07:27:44.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.604+0000 7fbcdd600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd815f5a0 0x7fbcd809b800 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005fd0 tx=0x7fbcd0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:44.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.604+0000 7fbcd5600640 1 -- 172.21.15.71:0/1411288193 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0046070 con 0x7fbcd815f5a0 2024-09-16T07:27:44.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.604+0000 7fbcd5600640 1 -- 172.21.15.71:0/1411288193 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcd0004250 con 0x7fbcd815f5a0 2024-09-16T07:27:44.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.604+0000 7fbcd5600640 1 -- 172.21.15.71:0/1411288193 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0033050 con 0x7fbcd815f5a0 2024-09-16T07:27:44.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.604+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd815bda0 con 0x7fbcd815f5a0 2024-09-16T07:27:44.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.604+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd815c360 con 0x7fbcd815f5a0 2024-09-16T07:27:44.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.605+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd806f9f0 con 0x7fbcd815f5a0 2024-09-16T07:27:44.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.606+0000 7fbcd5600640 1 -- 172.21.15.71:0/1411288193 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fbcd0049070 con 0x7fbcd815f5a0 2024-09-16T07:27:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.607+0000 7fbcd5600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbcb40790c0 0x7fbcb407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.607+0000 7fbcd5600640 1 -- 172.21.15.71:0/1411288193 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4857+0+0 (secure 0 0 0) 0x7fbcd00bc320 con 0x7fbcd815f5a0 2024-09-16T07:27:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.608+0000 7fbcdcc00640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbcb40790c0 0x7fbcb407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.608+0000 7fbcdcc00640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbcb40790c0 0x7fbcb407b580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005fd0 tx=0x7fbcc8005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:44.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.611+0000 7fbcd5600640 1 -- 172.21.15.71:0/1411288193 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbcd0084200 con 0x7fbcd815f5a0 2024-09-16T07:27:44.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.864+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbcd80ae200 con 0x7fbcd815f5a0 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.864+0000 7fbcd5600640 1 -- 172.21.15.71:0/1411288193 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7fbcd0083b00 con 0x7fbcd815f5a0 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:27:44.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:27:44.867 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:27:44.868 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.869 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:27:44.870 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:27:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:27:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:27:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.871 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.873 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:27:44.874 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.876 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.877 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.878 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:44.879 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:27:44.879 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.866+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbcb40790c0 msgr2=0x7fbcb407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.866+0000 7fbcdf600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbcb40790c0 0x7fbcb407b580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005fd0 tx=0x7fbcc8005850 comp rx=0 tx=0).stop 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd815f5a0 msgr2=0x7fbcd809b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd815f5a0 0x7fbcd809b800 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005fd0 tx=0x7fbcd0003f10 comp rx=0 tx=0).stop 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 shutdown_connections 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbcb40790c0 0x7fbcb407b580 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd815f5a0 0x7fbcd809b800 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcd80ccf00 0x7fbcd815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 --2- 172.21.15.71:0/1411288193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcd8073fb0 0x7fbcd81456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:44.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 >> 172.21.15.71:0/1411288193 conn(0x7fbcd80b1950 msgr2=0x7fbcd80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:44.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 shutdown_connections 2024-09-16T07:27:44.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:44.867+0000 7fbcdf600640 1 -- 172.21.15.71:0/1411288193 wait complete. 2024-09-16T07:27:44.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:27:45.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.080+0000 7f5420a00640 1 -- 172.21.15.71:0/2374904463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5400003a00 msgr2=0x7f5400003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:45.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.080+0000 7f5420a00640 1 --2- 172.21.15.71:0/2374904463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5400003a00 0x7f5400003e20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c290 tx=0x7f5400005830 comp rx=0 tx=0).stop 2024-09-16T07:27:45.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.081+0000 7f541b200640 1 -- 172.21.15.71:0/2374904463 reap_dead start 2024-09-16T07:27:45.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.081+0000 7f5420a00640 1 -- 172.21.15.71:0/2374904463 shutdown_connections 2024-09-16T07:27:45.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.081+0000 7f5420a00640 1 -- 172.21.15.71:0/2374904463 >> 172.21.15.71:0/2374904463 conn(0x7f541c0cfa20 msgr2=0x7f541c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:45.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.081+0000 7f5420a00640 1 -- 172.21.15.71:0/2374904463 shutdown_connections 2024-09-16T07:27:45.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.081+0000 7f5420a00640 1 -- 172.21.15.71:0/2374904463 wait complete. 2024-09-16T07:27:45.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.082+0000 7f5420a00640 1 Processor -- start 2024-09-16T07:27:45.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.082+0000 7f5420a00640 1 -- start start 2024-09-16T07:27:45.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.083+0000 7f5420a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5400003a00 0x7f541c1f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:45.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.083+0000 7f5420a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c15d120 0x7f541c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:45.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.083+0000 7f5420a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c0cb000 0x7f541c0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.083+0000 7f5420a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f541c15e7f0 con 0x7f541c15d120 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.083+0000 7f5420a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f541c15e670 con 0x7f541c0cb000 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.083+0000 7f5420a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f541c15e970 con 0x7f5400003a00 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.083+0000 7f5411e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c15d120 0x7f541c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5411e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c15d120 0x7f541c0c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48286/0 (socket says 172.21.15.71:48286) 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5411e00640 1 -- 172.21.15.71:0/1887278838 learned_addr learned my addr 172.21.15.71:0/1887278838 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5411e00640 1 -- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5400003a00 msgr2=0x7f541c1f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5411e00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5400003a00 0x7f541c1f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5411e00640 1 -- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c0cb000 msgr2=0x7f541c0c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5411e00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c0cb000 0x7f541c0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5411e00640 1 -- 172.21.15.71:0/1887278838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c0c4da0 con 0x7f541c15d120 2024-09-16T07:27:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5411e00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c15d120 0x7f541c0c8cc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5404011a80 tx=0x7f5404011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5413400640 1 -- 172.21.15.71:0/1887278838 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400fcd0 con 0x7f541c15d120 2024-09-16T07:27:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5413400640 1 -- 172.21.15.71:0/1887278838 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5404016070 con 0x7f541c15d120 2024-09-16T07:27:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5413400640 1 -- 172.21.15.71:0/1887278838 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404012690 con 0x7f541c15d120 2024-09-16T07:27:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.084+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c0ca760 con 0x7f541c15d120 2024-09-16T07:27:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.085+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c0c5040 con 0x7f541c15d120 2024-09-16T07:27:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.086+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c0cac80 con 0x7f541c15d120 2024-09-16T07:27:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.087+0000 7f5413400640 1 -- 172.21.15.71:0/1887278838 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f5404018030 con 0x7f541c15d120 2024-09-16T07:27:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.088+0000 7f5413400640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53f80790f0 0x7f53f807b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.088+0000 7f5413400640 1 -- 172.21.15.71:0/1887278838 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4857+0+0 (secure 0 0 0) 0x7f540409bcb0 con 0x7f541c15d120 2024-09-16T07:27:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.088+0000 7f541a800640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53f80790f0 0x7f53f807b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:45.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.089+0000 7f541a800640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53f80790f0 0x7f53f807b5b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f540c033000 tx=0x7f541c0aeb40 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:45.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.092+0000 7f5413400640 1 -- 172.21.15.71:0/1887278838 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f541c0cac80 con 0x7f541c15d120 2024-09-16T07:27:45.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.277+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f541c15ee90 con 0x7f541c15d120 2024-09-16T07:27:45.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.278+0000 7f5413400640 1 -- 172.21.15.71:0/1887278838 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) ==== 84+0+65 (secure 0 0 0) 0x7f5404063a50 con 0x7f541c15d120 2024-09-16T07:27:45.277 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:45.277 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:27:45.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:27:45.278 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:27:45.278 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:27:45.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:27:45.278 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53f80790f0 msgr2=0x7f53f807b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53f80790f0 0x7f53f807b5b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f540c033000 tx=0x7f541c0aeb40 comp rx=0 tx=0).stop 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c15d120 msgr2=0x7f541c0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c15d120 0x7f541c0c8cc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5404011a80 tx=0x7f5404011f40 comp rx=0 tx=0).stop 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 shutdown_connections 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53f80790f0 0x7f53f807b5b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c0cb000 0x7f541c0c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c15d120 0x7f541c0c8cc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 --2- 172.21.15.71:0/1887278838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5400003a00 0x7f541c1f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 >> 172.21.15.71:0/1887278838 conn(0x7f541c0cfa20 msgr2=0x7f541c15d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:45.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 shutdown_connections 2024-09-16T07:27:45.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.280+0000 7f5420a00640 1 -- 172.21.15.71:0/1887278838 wait complete. 2024-09-16T07:27:45.295 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:27:45.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:27:45.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.463+0000 7f6662000640 1 -- 172.21.15.71:0/3658950389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f665c15cff0 msgr2=0x7f665c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:45.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.463+0000 7f6662000640 1 --2- 172.21.15.71:0/3658950389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f665c15cff0 0x7f665c06e6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6648009860 tx=0x7f664802a540 comp rx=0 tx=0).stop 2024-09-16T07:27:45.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.463+0000 7f6662000640 1 -- 172.21.15.71:0/3658950389 shutdown_connections 2024-09-16T07:27:45.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.463+0000 7f6662000640 1 --2- 172.21.15.71:0/3658950389 >> v2:172.21.15.177:3300/0 conn(0x7f665c06f220 0x7f665c06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.463+0000 7f6662000640 1 --2- 172.21.15.71:0/3658950389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f665c15cff0 0x7f665c06e6f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.463+0000 7f6662000640 1 -- 172.21.15.71:0/3658950389 >> 172.21.15.71:0/3658950389 conn(0x7f665c0b18e0 msgr2=0x7f665c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:45.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.463+0000 7f6662000640 1 -- 172.21.15.71:0/3658950389 shutdown_connections 2024-09-16T07:27:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.463+0000 7f6662000640 1 -- 172.21.15.71:0/3658950389 wait complete. 2024-09-16T07:27:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.464+0000 7f6662000640 1 Processor -- start 2024-09-16T07:27:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.465+0000 7f6662000640 1 -- start start 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.465+0000 7f6662000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f665c06f220 0x7f665c14b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.465+0000 7f6662000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665c159aa0 0x7f665c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.465+0000 7f6662000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f665c15cff0 0x7f665c15e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.465+0000 7f6662000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f665c0cc520 con 0x7f665c159aa0 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.465+0000 7f6662000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f665c0cc3a0 con 0x7f665c06f220 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.465+0000 7f6662000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f665c0cc6a0 con 0x7f665c15cff0 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665c159aa0 0x7f665c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665c159aa0 0x7f665c0acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48302/0 (socket says 172.21.15.71:48302) 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665aa00640 1 -- 172.21.15.71:0/519492044 learned_addr learned my addr 172.21.15.71:0/519492044 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665b400640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f665c06f220 0x7f665c14b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665be00640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f665c15cff0 0x7f665c15e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:45.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665aa00640 1 -- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f665c15cff0 msgr2=0x7f665c15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:45.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665aa00640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f665c15cff0 0x7f665c15e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665aa00640 1 -- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f665c06f220 msgr2=0x7f665c14b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:45.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665aa00640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f665c06f220 0x7f665c14b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:45.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.466+0000 7f665aa00640 1 -- 172.21.15.71:0/519492044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c154640 con 0x7f665c159aa0 2024-09-16T07:27:45.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.467+0000 7f665aa00640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665c159aa0 0x7f665c0acd30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6648009780 tx=0x7f6648033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:45.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.467+0000 7f6653400640 1 -- 172.21.15.71:0/519492044 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648031b50 con 0x7f665c159aa0 2024-09-16T07:27:45.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.467+0000 7f6653400640 1 -- 172.21.15.71:0/519492044 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6648031e00 con 0x7f665c159aa0 2024-09-16T07:27:45.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.467+0000 7f6653400640 1 -- 172.21.15.71:0/519492044 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648005280 con 0x7f665c159aa0 2024-09-16T07:27:45.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.467+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c154920 con 0x7f665c159aa0 2024-09-16T07:27:45.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.467+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c154ee0 con 0x7f665c159aa0 2024-09-16T07:27:45.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.468+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c069360 con 0x7f665c159aa0 2024-09-16T07:27:45.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.469+0000 7f6653400640 1 -- 172.21.15.71:0/519492044 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f6648005420 con 0x7f665c159aa0 2024-09-16T07:27:45.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.471+0000 7f6653400640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f663c0790f0 0x7f663c07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:45.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.471+0000 7f6653400640 1 -- 172.21.15.71:0/519492044 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4857+0+0 (secure 0 0 0) 0x7f66480bc7b0 con 0x7f665c159aa0 2024-09-16T07:27:45.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.471+0000 7f665b400640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f663c0790f0 0x7f663c07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:45.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.472+0000 7f665b400640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f663c0790f0 0x7f663c07b5b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f664c0017c0 tx=0x7f664c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:45.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.475+0000 7f6653400640 1 -- 172.21.15.71:0/519492044 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f664803f030 con 0x7f665c159aa0 2024-09-16T07:27:45.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:45.660+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f665c0cd380 con 0x7f665c159aa0 2024-09-16T07:27:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.426+0000 7f6653400640 1 -- 172.21.15.71:0/519492044 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) ==== 138+0+0 (secure 0 0 0) 0x7f6648083b80 con 0x7f665c159aa0 2024-09-16T07:27:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:27:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f663c0790f0 msgr2=0x7f663c07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f663c0790f0 0x7f663c07b5b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f664c0017c0 tx=0x7f664c009040 comp rx=0 tx=0).stop 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665c159aa0 msgr2=0x7f665c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665c159aa0 0x7f665c0acd30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6648009780 tx=0x7f6648033ba0 comp rx=0 tx=0).stop 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 shutdown_connections 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f663c0790f0 0x7f663c07b5b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f665c15cff0 0x7f665c15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665c159aa0 0x7f665c0acd30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 --2- 172.21.15.71:0/519492044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f665c06f220 0x7f665c14b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 >> 172.21.15.71:0/519492044 conn(0x7f665c0b18e0 msgr2=0x7f665c15b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 shutdown_connections 2024-09-16T07:27:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.430+0000 7f6662000640 1 -- 172.21.15.71:0/519492044 wait complete. 2024-09-16T07:27:46.451 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:27:46.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.620+0000 7f44a9c00640 1 -- 172.21.15.71:0/1900731922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488004d40 msgr2=0x7f44880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:46.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.620+0000 7f44a9c00640 1 --2- 172.21.15.71:0/1900731922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488004d40 0x7f44880051c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f449c00c610 tx=0x7f4488005830 comp rx=0 tx=0).stop 2024-09-16T07:27:46.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.620+0000 7f44a3400640 1 -- 172.21.15.71:0/1900731922 reap_dead start 2024-09-16T07:27:46.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.620+0000 7f44a9c00640 1 -- 172.21.15.71:0/1900731922 shutdown_connections 2024-09-16T07:27:46.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.620+0000 7f44a9c00640 1 -- 172.21.15.71:0/1900731922 >> 172.21.15.71:0/1900731922 conn(0x7f44a40b1840 msgr2=0x7f44a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:46.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.620+0000 7f44a9c00640 1 -- 172.21.15.71:0/1900731922 shutdown_connections 2024-09-16T07:27:46.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.620+0000 7f44a9c00640 1 -- 172.21.15.71:0/1900731922 wait complete. 2024-09-16T07:27:46.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.622+0000 7f44a9c00640 1 Processor -- start 2024-09-16T07:27:46.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.622+0000 7f44a9c00640 1 -- start start 2024-09-16T07:27:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a9c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4488004d40 0x7f44a4142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a4141310 0x7f44a413b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a9c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a40c4ce0 0x7f44a413f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a9c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f44a40737d0 con 0x7f44a4141310 2024-09-16T07:27:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a9c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f44a4073650 con 0x7f44a40c4ce0 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a9c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f44a40ce2c0 con 0x7f4488004d40 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a4141310 0x7f44a413b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a4141310 0x7f44a413b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48328/0 (socket says 172.21.15.71:48328) 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2000640 1 -- 172.21.15.71:0/1756607537 learned_addr learned my addr 172.21.15.71:0/1756607537 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2a00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4488004d40 0x7f44a4142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a3400640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a40c4ce0 0x7f44a413f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2000640 1 -- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4488004d40 msgr2=0x7f44a4142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2000640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4488004d40 0x7f44a4142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2000640 1 -- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a40c4ce0 msgr2=0x7f44a413f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2000640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a40c4ce0 0x7f44a413f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.623+0000 7f44a2000640 1 -- 172.21.15.71:0/1756607537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a413bd10 con 0x7f44a4141310 2024-09-16T07:27:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.624+0000 7f44a2000640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a4141310 0x7f44a413b7d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4490004e20 tx=0x7f4490001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.624+0000 7f449aa00640 1 -- 172.21.15.71:0/1756607537 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490015020 con 0x7f44a4141310 2024-09-16T07:27:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.624+0000 7f449aa00640 1 -- 172.21.15.71:0/1756607537 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4490004e50 con 0x7f44a4141310 2024-09-16T07:27:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.624+0000 7f449aa00640 1 -- 172.21.15.71:0/1756607537 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490011bc0 con 0x7f44a4141310 2024-09-16T07:27:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.624+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a41418b0 con 0x7f44a4141310 2024-09-16T07:27:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.624+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a4141e70 con 0x7f44a4141310 2024-09-16T07:27:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.625+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a40cc5c0 con 0x7f44a4141310 2024-09-16T07:27:46.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.626+0000 7f449aa00640 1 -- 172.21.15.71:0/1756607537 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f4490018020 con 0x7f44a4141310 2024-09-16T07:27:46.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.627+0000 7f449aa00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f44840790c0 0x7f448407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:46.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.627+0000 7f449aa00640 1 -- 172.21.15.71:0/1756607537 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5227+0+0 (secure 0 0 0) 0x7f449009c2e0 con 0x7f44a4141310 2024-09-16T07:27:46.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.628+0000 7f44a2a00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f44840790c0 0x7f448407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:46.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.628+0000 7f44a2a00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f44840790c0 0x7f448407b580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4494009c50 tx=0x7f4494005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:46.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.632+0000 7f449aa00640 1 -- 172.21.15.71:0/1756607537 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f449001f020 con 0x7f44a4141310 2024-09-16T07:27:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:46.817+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f44a40b0f60 con 0x7f44a4141310 2024-09-16T07:27:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.434+0000 7f449aa00640 1 -- 172.21.15.71:0/1756607537 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) ==== 170+0+0 (secure 0 0 0) 0x7f4490063b00 con 0x7f44a4141310 2024-09-16T07:27:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f44840790c0 msgr2=0x7f448407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f44840790c0 0x7f448407b580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4494009c50 tx=0x7f4494005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a4141310 msgr2=0x7f44a413b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a4141310 0x7f44a413b7d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4490004e20 tx=0x7f4490001630 comp rx=0 tx=0).stop 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 shutdown_connections 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f44840790c0 0x7f448407b580 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a40c4ce0 0x7f44a413f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a4141310 0x7f44a413b7d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 --2- 172.21.15.71:0/1756607537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4488004d40 0x7f44a4142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.437+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 >> 172.21.15.71:0/1756607537 conn(0x7f44a40b1840 msgr2=0x7f44a40b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:47.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.438+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 shutdown_connections 2024-09-16T07:27:47.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.438+0000 7f44a9c00640 1 -- 172.21.15.71:0/1756607537 wait complete. 2024-09-16T07:27:47.469 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:27:47.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.631+0000 7f53c8800640 1 -- 172.21.15.71:0/1386840435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 msgr2=0x7f53c0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:47.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.631+0000 7f53c8800640 1 --2- 172.21.15.71:0/1386840435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 0x7f53c0073440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009830 tx=0x7f53b402a410 comp rx=0 tx=0).stop 2024-09-16T07:27:47.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.631+0000 7f53c8800640 1 -- 172.21.15.71:0/1386840435 shutdown_connections 2024-09-16T07:27:47.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.631+0000 7f53c8800640 1 --2- 172.21.15.71:0/1386840435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c006efa0 0x7f53c00cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:47.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.631+0000 7f53c8800640 1 --2- 172.21.15.71:0/1386840435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 0x7f53c0073440 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:47.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.631+0000 7f53c8800640 1 -- 172.21.15.71:0/1386840435 >> 172.21.15.71:0/1386840435 conn(0x7f53c00b1840 msgr2=0x7f53c00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:47.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.631+0000 7f53c8800640 1 -- 172.21.15.71:0/1386840435 shutdown_connections 2024-09-16T07:27:47.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.631+0000 7f53c8800640 1 -- 172.21.15.71:0/1386840435 wait complete. 2024-09-16T07:27:47.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.632+0000 7f53c8800640 1 Processor -- start 2024-09-16T07:27:47.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.633+0000 7f53c8800640 1 -- start start 2024-09-16T07:27:47.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c8800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c006efa0 0x7f53c00ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:47.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c8800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c00ccb00 0x7f53c01f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:47.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c8800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 0x7f53c0141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:47.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c8800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53c00ced40 con 0x7f53c00ccb00 2024-09-16T07:27:47.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c8800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f53c00cebc0 con 0x7f53c0145750 2024-09-16T07:27:47.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c8800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f53c00ceec0 con 0x7f53c006efa0 2024-09-16T07:27:47.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c00ccb00 0x7f53c01f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:47.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c00ccb00 0x7f53c01f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48332/0 (socket says 172.21.15.71:48332) 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5400640 1 -- 172.21.15.71:0/306661797 learned_addr learned my addr 172.21.15.71:0/306661797 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5e00640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c006efa0 0x7f53c00ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c6800640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 0x7f53c0141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5400640 1 -- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c006efa0 msgr2=0x7f53c00ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5400640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c006efa0 0x7f53c00ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5400640 1 -- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 msgr2=0x7f53c0141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5400640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 0x7f53c0141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c5400640 1 -- 172.21.15.71:0/306661797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c01f37d0 con 0x7f53c00ccb00 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.634+0000 7f53c6800640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 0x7f53c0141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:47.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.635+0000 7f53c5400640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c00ccb00 0x7f53c01f3290 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f53b000ea20 tx=0x7f53b000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:47.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.635+0000 7f53ba000640 1 -- 172.21.15.71:0/306661797 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b000cd80 con 0x7f53c00ccb00 2024-09-16T07:27:47.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.635+0000 7f53ba000640 1 -- 172.21.15.71:0/306661797 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53b0010430 con 0x7f53c00ccb00 2024-09-16T07:27:47.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.635+0000 7f53ba000640 1 -- 172.21.15.71:0/306661797 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0010840 con 0x7f53c00ccb00 2024-09-16T07:27:47.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.635+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c00ab280 con 0x7f53c00ccb00 2024-09-16T07:27:47.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.635+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c00ab840 con 0x7f53c00ccb00 2024-09-16T07:27:47.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.636+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c00cc5c0 con 0x7f53c00ccb00 2024-09-16T07:27:47.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.638+0000 7f53ba000640 1 -- 172.21.15.71:0/306661797 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f53b00109e0 con 0x7f53c00ccb00 2024-09-16T07:27:47.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.639+0000 7f53ba000640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53a40790c0 0x7f53a407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:47.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.639+0000 7f53ba000640 1 -- 172.21.15.71:0/306661797 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5609+0+0 (secure 0 0 0) 0x7f53b009a4d0 con 0x7f53c00ccb00 2024-09-16T07:27:47.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.640+0000 7f53c5e00640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53a40790c0 0x7f53a407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:47.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.640+0000 7f53c5e00640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53a40790c0 0x7f53a407b580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f53b4034000 tx=0x7f53b4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.643+0000 7f53ba000640 1 -- 172.21.15.71:0/306661797 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f53c01f37d0 con 0x7f53c00ccb00 2024-09-16T07:27:47.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:47.863+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f53c00aa8c0 con 0x7f53c00ccb00 2024-09-16T07:27:48.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.450+0000 7f53ba000640 1 -- 172.21.15.71:0/306661797 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) ==== 172+0+0 (secure 0 0 0) 0x7f53b0063f80 con 0x7f53c00ccb00 2024-09-16T07:27:48.450 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 8 and data pool 9 2024-09-16T07:27:48.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.454+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53a40790c0 msgr2=0x7f53a407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:48.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.454+0000 7f53c8800640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53a40790c0 0x7f53a407b580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f53b4034000 tx=0x7f53b4004730 comp rx=0 tx=0).stop 2024-09-16T07:27:48.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.454+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c00ccb00 msgr2=0x7f53c01f3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:48.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.454+0000 7f53c8800640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c00ccb00 0x7f53c01f3290 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f53b000ea20 tx=0x7f53b000eee0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.456+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 shutdown_connections 2024-09-16T07:27:48.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.456+0000 7f53c8800640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53a40790c0 0x7f53a407b580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.456+0000 7f53c8800640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c0145750 0x7f53c0141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.456+0000 7f53c8800640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53c00ccb00 0x7f53c01f3290 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.456+0000 7f53c8800640 1 --2- 172.21.15.71:0/306661797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c006efa0 0x7f53c00ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.456+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 >> 172.21.15.71:0/306661797 conn(0x7f53c00b1840 msgr2=0x7f53c006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:48.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.456+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 shutdown_connections 2024-09-16T07:27:48.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.456+0000 7f53c8800640 1 -- 172.21.15.71:0/306661797 wait complete. 2024-09-16T07:27:48.482 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:27:48.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.640+0000 7fc52fc00640 1 -- 172.21.15.71:0/1215175925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 msgr2=0x7fc528074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:48.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.640+0000 7fc52fc00640 1 --2- 172.21.15.71:0/1215175925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 0x7fc528074440 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc520005fd0 tx=0x7fc52002aa10 comp rx=0 tx=0).stop 2024-09-16T07:27:48.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.640+0000 7fc52fc00640 1 -- 172.21.15.71:0/1215175925 shutdown_connections 2024-09-16T07:27:48.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.640+0000 7fc52fc00640 1 --2- 172.21.15.71:0/1215175925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 0x7fc528074440 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.640+0000 7fc52fc00640 1 --2- 172.21.15.71:0/1215175925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5280ccf00 0x7fc5280b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.640+0000 7fc52fc00640 1 -- 172.21.15.71:0/1215175925 >> 172.21.15.71:0/1215175925 conn(0x7fc5280b1950 msgr2=0x7fc5280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:48.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.640+0000 7fc52fc00640 1 -- 172.21.15.71:0/1215175925 shutdown_connections 2024-09-16T07:27:48.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.640+0000 7fc52fc00640 1 -- 172.21.15.71:0/1215175925 wait complete. 2024-09-16T07:27:48.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.642+0000 7fc52fc00640 1 Processor -- start 2024-09-16T07:27:48.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.642+0000 7fc52fc00640 1 -- start start 2024-09-16T07:27:48.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52fc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 0x7fc5281f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52fc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5280ccf00 0x7fc5280cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52fc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc52815f5a0 0x7fc5280b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52fc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc52806f2e0 con 0x7fc528073fb0 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52fc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc52806f160 con 0x7fc5280ccf00 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52fc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc52806f460 con 0x7fc52815f5a0 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 0x7fc5281f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 0x7fc5281f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48344/0 (socket says 172.21.15.71:48344) 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52d200640 1 -- 172.21.15.71:0/2145749992 learned_addr learned my addr 172.21.15.71:0/2145749992 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc52dc00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc52815f5a0 0x7fc5280b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.643+0000 7fc527e00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5280ccf00 0x7fc5280cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:48.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc52d200640 1 -- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc52815f5a0 msgr2=0x7fc5280b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:48.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc52d200640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc52815f5a0 0x7fc5280b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc52d200640 1 -- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5280ccf00 msgr2=0x7fc5280cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:48.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc52d200640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5280ccf00 0x7fc5280cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc52d200640 1 -- 172.21.15.71:0/2145749992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5280b39c0 con 0x7fc528073fb0 2024-09-16T07:27:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc52d200640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 0x7fc5281f4890 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c3a0 tx=0x7fc51800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc524c00640 1 -- 172.21.15.71:0/2145749992 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518017070 con 0x7fc528073fb0 2024-09-16T07:27:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc524c00640 1 -- 172.21.15.71:0/2145749992 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5180053c0 con 0x7fc528073fb0 2024-09-16T07:27:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc524c00640 1 -- 172.21.15.71:0/2145749992 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518003850 con 0x7fc528073fb0 2024-09-16T07:27:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5280b3ca0 con 0x7fc528073fb0 2024-09-16T07:27:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.644+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5280b4260 con 0x7fc528073fb0 2024-09-16T07:27:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.645+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52806f9f0 con 0x7fc528073fb0 2024-09-16T07:27:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.647+0000 7fc524c00640 1 -- 172.21.15.71:0/2145749992 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fc51801a070 con 0x7fc528073fb0 2024-09-16T07:27:48.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.647+0000 7fc524c00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5080790c0 0x7fc50807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:48.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.648+0000 7fc524c00640 1 -- 172.21.15.71:0/2145749992 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fc51809af20 con 0x7fc528073fb0 2024-09-16T07:27:48.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.648+0000 7fc527e00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5080790c0 0x7fc50807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:48.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.648+0000 7fc527e00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5080790c0 0x7fc50807b580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc51400bbe0 tx=0x7fc51400b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:48.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.652+0000 7fc524c00640 1 -- 172.21.15.71:0/2145749992 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc518061f10 con 0x7fc528073fb0 2024-09-16T07:27:48.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.836+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5280ae9f0 con 0x7fc528073fb0 2024-09-16T07:27:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.837+0000 7fc524c00640 1 -- 172.21.15.71:0/2145749992 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16750 (secure 0 0 0) 0x7fc518066940 con 0x7fc528073fb0 2024-09-16T07:27:48.837 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:48.837 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":61,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:27:47.864449+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:27:45.670244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T07:27:46.828046+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:27:48.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5080790c0 msgr2=0x7fc50807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:48.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5080790c0 0x7fc50807b580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc51400bbe0 tx=0x7fc51400b390 comp rx=0 tx=0).stop 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 msgr2=0x7fc5281f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 0x7fc5281f4890 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c3a0 tx=0x7fc51800c860 comp rx=0 tx=0).stop 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 shutdown_connections 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc5080790c0 0x7fc50807b580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc52815f5a0 0x7fc5280b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5280ccf00 0x7fc5280cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 --2- 172.21.15.71:0/2145749992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc528073fb0 0x7fc5281f4890 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 >> 172.21.15.71:0/2145749992 conn(0x7fc5280b1950 msgr2=0x7fc52809d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 shutdown_connections 2024-09-16T07:27:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:48.840+0000 7fc52fc00640 1 -- 172.21.15.71:0/2145749992 wait complete. 2024-09-16T07:27:48.856 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:27:49.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 -- 172.21.15.71:0/1429207100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c161520 msgr2=0x7f872c0d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 --2- 172.21.15.71:0/1429207100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c161520 0x7f872c0d4940 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f872c08e0a0 tx=0x7f872002a560 comp rx=0 tx=0).stop 2024-09-16T07:27:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 -- 172.21.15.71:0/1429207100 shutdown_connections 2024-09-16T07:27:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 --2- 172.21.15.71:0/1429207100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872c0d23e0 0x7f872c0d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 --2- 172.21.15.71:0/1429207100 >> v2:172.21.15.177:3300/0 conn(0x7f872c0cf440 0x7f872c0d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 --2- 172.21.15.71:0/1429207100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c161520 0x7f872c0d4940 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 -- 172.21.15.71:0/1429207100 >> 172.21.15.71:0/1429207100 conn(0x7f872c0d3e20 msgr2=0x7f872c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 -- 172.21.15.71:0/1429207100 shutdown_connections 2024-09-16T07:27:49.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.013+0000 7f8732e00640 1 -- 172.21.15.71:0/1429207100 wait complete. 2024-09-16T07:27:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.014+0000 7f8732e00640 1 Processor -- start 2024-09-16T07:27:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.015+0000 7f8732e00640 1 -- start start 2024-09-16T07:27:49.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.015+0000 7f8732e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c0cf440 0x7f872c1f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.015+0000 7f8732e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c0d23e0 0x7f872c15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.015+0000 7f8732e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872c161520 0x7f872c1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8732e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f872c0d1100 con 0x7f872c0d23e0 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8732e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f872c0d0f80 con 0x7f872c0cf440 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8732e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f872c0d1280 con 0x7f872c161520 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c0d23e0 0x7f872c15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c0cf440 0x7f872c1f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c0d23e0 0x7f872c15d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48360/0 (socket says 172.21.15.71:48360) 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731000640 1 -- 172.21.15.71:0/1066050112 learned_addr learned my addr 172.21.15.71:0/1066050112 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8732400640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872c161520 0x7f872c1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731a00640 1 -- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872c161520 msgr2=0x7f872c1f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:49.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731a00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872c161520 0x7f872c1f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731a00640 1 -- 172.21.15.71:0/1066050112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c0d23e0 msgr2=0x7f872c15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:49.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731a00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c0d23e0 0x7f872c15d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731a00640 1 -- 172.21.15.71:0/1066050112 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c1f7800 con 0x7f872c0cf440 2024-09-16T07:27:49.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.016+0000 7f8731000640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c0d23e0 0x7f872c15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:27:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.017+0000 7f8731a00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c0cf440 0x7f872c1f44b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f872c06a3b0 tx=0x7f87200038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.017+0000 7f8729600640 1 -- 172.21.15.71:0/1066050112 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872003e030 con 0x7f872c0cf440 2024-09-16T07:27:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.017+0000 7f8729600640 1 -- 172.21.15.71:0/1066050112 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8720003d90 con 0x7f872c0cf440 2024-09-16T07:27:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.017+0000 7f8729600640 1 -- 172.21.15.71:0/1066050112 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720035040 con 0x7f872c0cf440 2024-09-16T07:27:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.017+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872c0a02f0 con 0x7f872c0cf440 2024-09-16T07:27:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.017+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872c15ac70 con 0x7f872c0cf440 2024-09-16T07:27:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.018+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872c15b280 con 0x7f872c0cf440 2024-09-16T07:27:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.020+0000 7f8729600640 1 -- 172.21.15.71:0/1066050112 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f8720041030 con 0x7f872c0cf440 2024-09-16T07:27:49.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.020+0000 7f8729600640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f870c0790c0 0x7f870c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:49.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.021+0000 7f8729600640 1 -- 172.21.15.71:0/1066050112 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f87200bc7e0 con 0x7f872c0cf440 2024-09-16T07:27:49.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.021+0000 7f8731000640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f870c0790c0 0x7f870c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:49.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.022+0000 7f8731000640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f870c0790c0 0x7f870c07b580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f872c15d710 tx=0x7f871800b410 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:49.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.024+0000 7f8729600640 1 -- 172.21.15.71:0/1066050112 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f87200837c0 con 0x7f872c0cf440 2024-09-16T07:27:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.210+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f872c06f200 con 0x7f872c0cf440 2024-09-16T07:27:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.212+0000 7f8729600640 1 -- 172.21.15.71:0/1066050112 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16750 (secure 0 0 0) 0x7f87200881f0 con 0x7f872c0cf440 2024-09-16T07:27:49.211 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:49.212 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":61,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:27:47.864449+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:27:45.670244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T07:27:46.828046+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:27:49.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f870c0790c0 msgr2=0x7f870c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:49.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f870c0790c0 0x7f870c07b580 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f872c15d710 tx=0x7f871800b410 comp rx=0 tx=0).stop 2024-09-16T07:27:49.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c0cf440 msgr2=0x7f872c1f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c0cf440 0x7f872c1f44b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f872c06a3b0 tx=0x7f87200038a0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 shutdown_connections 2024-09-16T07:27:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f870c0790c0 0x7f870c07b580 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872c161520 0x7f872c1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f872c0d23e0 0x7f872c15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 --2- 172.21.15.71:0/1066050112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f872c0cf440 0x7f872c1f44b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.214+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 >> 172.21.15.71:0/1066050112 conn(0x7f872c0d3e20 msgr2=0x7f872c0b9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.215+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 shutdown_connections 2024-09-16T07:27:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.215+0000 7f8732e00640 1 -- 172.21.15.71:0/1066050112 wait complete. 2024-09-16T07:27:49.230 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:27:49.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.391+0000 7f53eae00640 1 -- 172.21.15.71:0/79769753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 msgr2=0x7f53cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:49.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.391+0000 7f53eae00640 1 --2- 172.21.15.71:0/79769753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 0x7f53cc004c80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f53e40d1ce0 tx=0x7f53cc005830 comp rx=0 tx=0).stop 2024-09-16T07:27:49.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.391+0000 7f53ea400640 1 -- 172.21.15.71:0/79769753 reap_dead start 2024-09-16T07:27:49.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.391+0000 7f53eae00640 1 -- 172.21.15.71:0/79769753 shutdown_connections 2024-09-16T07:27:49.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.391+0000 7f53eae00640 1 -- 172.21.15.71:0/79769753 >> 172.21.15.71:0/79769753 conn(0x7f53e409d0b0 msgr2=0x7f53e40d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:49.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.391+0000 7f53eae00640 1 -- 172.21.15.71:0/79769753 shutdown_connections 2024-09-16T07:27:49.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.391+0000 7f53eae00640 1 -- 172.21.15.71:0/79769753 wait complete. 2024-09-16T07:27:49.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.392+0000 7f53eae00640 1 Processor -- start 2024-09-16T07:27:49.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.393+0000 7f53eae00640 1 -- start start 2024-09-16T07:27:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.393+0000 7f53eae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 0x7f53e4157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.393+0000 7f53eae00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e415aa90 0x7f53e415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.393+0000 7f53eae00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e406d320 0x7f53e406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53eae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40b56f0 con 0x7f53cc004800 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53eae00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40b5150 con 0x7f53e406d320 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53eae00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f53e40b33b0 con 0x7f53e415aa90 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 0x7f53e4157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 0x7f53e4157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48372/0 (socket says 172.21.15.71:48372) 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9a00640 1 -- 172.21.15.71:0/871389977 learned_addr learned my addr 172.21.15.71:0/871389977 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53ea400640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e406d320 0x7f53e406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9000640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e415aa90 0x7f53e415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9a00640 1 -- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e415aa90 msgr2=0x7f53e415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9a00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e415aa90 0x7f53e415af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9a00640 1 -- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e406d320 msgr2=0x7f53e406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9a00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e406d320 0x7f53e406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9a00640 1 -- 172.21.15.71:0/871389977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e4158050 con 0x7f53cc004800 2024-09-16T07:27:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.394+0000 7f53e9000640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e415aa90 0x7f53e415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.395+0000 7f53e9a00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 0x7f53e4157250 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f53d8004e50 tx=0x7f53d8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.395+0000 7f53e1600640 1 -- 172.21.15.71:0/871389977 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8015020 con 0x7f53cc004800 2024-09-16T07:27:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.395+0000 7f53e1600640 1 -- 172.21.15.71:0/871389977 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53d8011960 con 0x7f53cc004800 2024-09-16T07:27:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.395+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e40b6e00 con 0x7f53cc004800 2024-09-16T07:27:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.395+0000 7f53e1600640 1 -- 172.21.15.71:0/871389977 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8011d70 con 0x7f53cc004800 2024-09-16T07:27:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.395+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e40b73c0 con 0x7f53cc004800 2024-09-16T07:27:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.396+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e40d22f0 con 0x7f53cc004800 2024-09-16T07:27:49.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.397+0000 7f53e1600640 1 -- 172.21.15.71:0/871389977 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f53d8018020 con 0x7f53cc004800 2024-09-16T07:27:49.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.398+0000 7f53e1600640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53c40790c0 0x7f53c407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.398+0000 7f53e1600640 1 -- 172.21.15.71:0/871389977 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f53d809c630 con 0x7f53cc004800 2024-09-16T07:27:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.398+0000 7f53e9000640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53c40790c0 0x7f53c407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.399+0000 7f53e9000640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53c40790c0 0x7f53c407b580 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53d0035000 tx=0x7f53d002d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.402+0000 7f53e1600640 1 -- 172.21.15.71:0/871389977 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f53d801f020 con 0x7f53cc004800 2024-09-16T07:27:49.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:49.626+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f53e40cdfd0 con 0x7f53cc004800 2024-09-16T07:27:50.426 INFO:tasks.ceph.mds.c.smithi071.stderr:2024-09-16T07:27:50.427+0000 7f042e400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:27:50.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.462+0000 7f53e1600640 1 -- 172.21.15.71:0/871389977 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) ==== 118+0+0 (secure 0 0 0) 0x7f53d8063c70 con 0x7f53cc004800 2024-09-16T07:27:50.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53c40790c0 msgr2=0x7f53c407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:50.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53c40790c0 0x7f53c407b580 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53d0035000 tx=0x7f53d002d040 comp rx=0 tx=0).stop 2024-09-16T07:27:50.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 msgr2=0x7f53e4157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:50.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 0x7f53e4157250 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f53d8004e50 tx=0x7f53d8001630 comp rx=0 tx=0).stop 2024-09-16T07:27:50.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 shutdown_connections 2024-09-16T07:27:50.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f53c40790c0 0x7f53c407b580 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:50.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e406d320 0x7f53e406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:50.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e415aa90 0x7f53e415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:50.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 --2- 172.21.15.71:0/871389977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53cc004800 0x7f53e4157250 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:50.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 >> 172.21.15.71:0/871389977 conn(0x7f53e409d0b0 msgr2=0x7f53e40b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:50.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 shutdown_connections 2024-09-16T07:27:50.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.465+0000 7f53eae00640 1 -- 172.21.15.71:0/871389977 wait complete. 2024-09-16T07:27:50.483 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:27:50.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:27:50.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.643+0000 7fe02d600640 1 -- 172.21.15.71:0/3918262720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c004d40 msgr2=0x7fe00c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:50.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.643+0000 7fe02d600640 1 --2- 172.21.15.71:0/3918262720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c004d40 0x7fe00c0051c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe02000c2c0 tx=0x7fe00c005830 comp rx=0 tx=0).stop 2024-09-16T07:27:50.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.643+0000 7fe02cc00640 1 -- 172.21.15.71:0/3918262720 reap_dead start 2024-09-16T07:27:50.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.643+0000 7fe02d600640 1 -- 172.21.15.71:0/3918262720 shutdown_connections 2024-09-16T07:27:50.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.643+0000 7fe02d600640 1 -- 172.21.15.71:0/3918262720 >> 172.21.15.71:0/3918262720 conn(0x7fe0280cf9f0 msgr2=0x7fe0280cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:50.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.643+0000 7fe02d600640 1 -- 172.21.15.71:0/3918262720 shutdown_connections 2024-09-16T07:27:50.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.643+0000 7fe02d600640 1 -- 172.21.15.71:0/3918262720 wait complete. 2024-09-16T07:27:50.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.645+0000 7fe02d600640 1 Processor -- start 2024-09-16T07:27:50.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.645+0000 7fe02d600640 1 -- start start 2024-09-16T07:27:50.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.645+0000 7fe02d600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe00c004d40 0x7fe028159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:50.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02d600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0280ce680 0x7fe0281f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe028152cd0 0x7fe0280c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe0280cb9a0 con 0x7fe028152cd0 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02d600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe02806ff40 con 0x7fe0280ce680 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02d600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe0280cbb20 con 0x7fe00c004d40 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe028152cd0 0x7fe0280c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe028152cd0 0x7fe0280c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48394/0 (socket says 172.21.15.71:48394) 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 -- 172.21.15.71:0/1713656253 learned_addr learned my addr 172.21.15.71:0/1713656253 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe027e00640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe00c004d40 0x7fe028159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe027400640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0280ce680 0x7fe0281f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 -- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe00c004d40 msgr2=0x7fe028159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe00c004d40 0x7fe028159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:50.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 -- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0280ce680 msgr2=0x7fe0281f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:50.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0280ce680 0x7fe0281f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:50.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 -- 172.21.15.71:0/1713656253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0280cec40 con 0x7fe028152cd0 2024-09-16T07:27:50.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.646+0000 7fe02cc00640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe028152cd0 0x7fe0280c83d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe02000c2c0 tx=0x7fe020006670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:50.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.647+0000 7fe01fe00640 1 -- 172.21.15.71:0/1713656253 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020031070 con 0x7fe028152cd0 2024-09-16T07:27:50.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.647+0000 7fe01fe00640 1 -- 172.21.15.71:0/1713656253 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe02002c690 con 0x7fe028152cd0 2024-09-16T07:27:50.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.647+0000 7fe01fe00640 1 -- 172.21.15.71:0/1713656253 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020006c50 con 0x7fe028152cd0 2024-09-16T07:27:50.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.647+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0280cef20 con 0x7fe028152cd0 2024-09-16T07:27:50.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.647+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0280cf4e0 con 0x7fe028152cd0 2024-09-16T07:27:50.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.648+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe028073ef0 con 0x7fe028152cd0 2024-09-16T07:27:50.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.649+0000 7fe01fe00640 1 -- 172.21.15.71:0/1713656253 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fe020034070 con 0x7fe028152cd0 2024-09-16T07:27:50.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.650+0000 7fe01fe00640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe0080790f0 0x7fe00807b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:50.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.650+0000 7fe01fe00640 1 -- 172.21.15.71:0/1713656253 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fe0200bb3c0 con 0x7fe028152cd0 2024-09-16T07:27:50.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.651+0000 7fe027e00640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe0080790f0 0x7fe00807b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:50.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.652+0000 7fe027e00640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe0080790f0 0x7fe00807b5b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe018005fd0 tx=0x7fe018009500 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:50.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.655+0000 7fe01fe00640 1 -- 172.21.15.71:0/1713656253 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe020082a00 con 0x7fe028152cd0 2024-09-16T07:27:50.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:50.879+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe02815e560 con 0x7fe028152cd0 2024-09-16T07:27:51.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.464+0000 7fe01fe00640 1 -- 172.21.15.71:0/1713656253 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7fe020082300 con 0x7fe028152cd0 2024-09-16T07:27:51.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe0080790f0 msgr2=0x7fe00807b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:51.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe0080790f0 0x7fe00807b5b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe018005fd0 tx=0x7fe018009500 comp rx=0 tx=0).stop 2024-09-16T07:27:51.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe028152cd0 msgr2=0x7fe0280c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:51.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe028152cd0 0x7fe0280c83d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe02000c2c0 tx=0x7fe020006670 comp rx=0 tx=0).stop 2024-09-16T07:27:51.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 shutdown_connections 2024-09-16T07:27:51.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe0080790f0 0x7fe00807b5b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe028152cd0 0x7fe0280c83d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0280ce680 0x7fe0281f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 --2- 172.21.15.71:0/1713656253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe00c004d40 0x7fe028159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 >> 172.21.15.71:0/1713656253 conn(0x7fe0280cf9f0 msgr2=0x7fe02815ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.467+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 shutdown_connections 2024-09-16T07:27:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.468+0000 7fe02d600640 1 -- 172.21.15.71:0/1713656253 wait complete. 2024-09-16T07:27:51.485 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:27:51.485 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:27:51.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.641+0000 7fbdcc600640 1 -- 172.21.15.71:0/1275365701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc415f5a0 msgr2=0x7fbdc40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:51.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.641+0000 7fbdcc600640 1 --2- 172.21.15.71:0/1275365701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc415f5a0 0x7fbdc40b2440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009830 tx=0x7fbdb802a480 comp rx=0 tx=0).stop 2024-09-16T07:27:51.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.641+0000 7fbdcc600640 1 -- 172.21.15.71:0/1275365701 shutdown_connections 2024-09-16T07:27:51.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.641+0000 7fbdcc600640 1 --2- 172.21.15.71:0/1275365701 >> v2:172.21.15.177:3300/0 conn(0x7fbdc4073fb0 0x7fbdc4074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.641+0000 7fbdcc600640 1 --2- 172.21.15.71:0/1275365701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc415f5a0 0x7fbdc40b2440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.641+0000 7fbdcc600640 1 -- 172.21.15.71:0/1275365701 >> 172.21.15.71:0/1275365701 conn(0x7fbdc40b1950 msgr2=0x7fbdc40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:51.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.642+0000 7fbdcc600640 1 -- 172.21.15.71:0/1275365701 shutdown_connections 2024-09-16T07:27:51.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.642+0000 7fbdcc600640 1 -- 172.21.15.71:0/1275365701 wait complete. 2024-09-16T07:27:51.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.643+0000 7fbdcc600640 1 Processor -- start 2024-09-16T07:27:51.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.643+0000 7fbdcc600640 1 -- start start 2024-09-16T07:27:51.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.644+0000 7fbdcc600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc4073fb0 0x7fbdc41525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:51.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.644+0000 7fbdcc600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40ccf00 0x7fbdc414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:51.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.644+0000 7fbdcc600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc415f5a0 0x7fbdc41f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:51.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.644+0000 7fbdcc600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc406f2e0 con 0x7fbdc4073fb0 2024-09-16T07:27:51.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.644+0000 7fbdcc600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc406f160 con 0x7fbdc415f5a0 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.644+0000 7fbdcc600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc406f460 con 0x7fbdc40ccf00 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc4073fb0 0x7fbdc41525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc4073fb0 0x7fbdc41525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48402/0 (socket says 172.21.15.71:48402) 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 -- 172.21.15.71:0/4100381162 learned_addr learned my addr 172.21.15.71:0/4100381162 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9200640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40ccf00 0x7fbdc414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdca600640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc415f5a0 0x7fbdc41f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 -- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40ccf00 msgr2=0x7fbdc414d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40ccf00 0x7fbdc414d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 -- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc415f5a0 msgr2=0x7fbdc41f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc415f5a0 0x7fbdc41f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 -- 172.21.15.71:0/4100381162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc41f3500 con 0x7fbdc4073fb0 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.645+0000 7fbdc9c00640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc4073fb0 0x7fbdc41525d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009950 tx=0x7fbdb8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:51.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.646+0000 7fbdc1600640 1 -- 172.21.15.71:0/4100381162 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8031d60 con 0x7fbdc4073fb0 2024-09-16T07:27:51.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.646+0000 7fbdc1600640 1 -- 172.21.15.71:0/4100381162 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdb8033e10 con 0x7fbdc4073fb0 2024-09-16T07:27:51.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.646+0000 7fbdc1600640 1 -- 172.21.15.71:0/4100381162 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80332b0 con 0x7fbdc4073fb0 2024-09-16T07:27:51.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.646+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41f37e0 con 0x7fbdc4073fb0 2024-09-16T07:27:51.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.646+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc4150f90 con 0x7fbdc4073fb0 2024-09-16T07:27:51.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.647+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc406f9f0 con 0x7fbdc4073fb0 2024-09-16T07:27:51.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.648+0000 7fbdc1600640 1 -- 172.21.15.71:0/4100381162 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fbdb8033450 con 0x7fbdc4073fb0 2024-09-16T07:27:51.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.649+0000 7fbdc1600640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda40790c0 0x7fbda407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:51.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.649+0000 7fbdc1600640 1 -- 172.21.15.71:0/4100381162 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fbdb80bcf30 con 0x7fbdc4073fb0 2024-09-16T07:27:51.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.649+0000 7fbdc9200640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda40790c0 0x7fbda407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:51.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.650+0000 7fbdc9200640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda40790c0 0x7fbda407b580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbdc414f460 tx=0x7fbdb0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:51.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.653+0000 7fbdc1600640 1 -- 172.21.15.71:0/4100381162 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbdb80c1070 con 0x7fbdc4073fb0 2024-09-16T07:27:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.874+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdc409d5a0 con 0x7fbdc4073fb0 2024-09-16T07:27:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.875+0000 7fbdc1600640 1 -- 172.21.15.71:0/4100381162 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7fbdb803f030 con 0x7fbdc4073fb0 2024-09-16T07:27:51.876 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:51.876 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":37,"btime":"2024-09-16T07:27:51:464479+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1090989192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1090989192},{"type":"v1","addr":"172.21.15.177:6836","nonce":1090989192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2044137966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2044137966},{"type":"v1","addr":"172.21.15.177:6837","nonce":2044137966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3911470669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3911470669},{"type":"v1","addr":"172.21.15.71:6835","nonce":3911470669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:27:47.864418+0000","modified":"2024-09-16T07:27:50.881337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6834/3242207389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3242207389},{"type":"v1","addr":"172.21.15.71:6834","nonce":3242207389}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5964,"qdb_cluster":[5964]},"id":4}]} 2024-09-16T07:27:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 37 2024-09-16T07:27:51.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda40790c0 msgr2=0x7fbda407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:51.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda40790c0 0x7fbda407b580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbdc414f460 tx=0x7fbdb0009040 comp rx=0 tx=0).stop 2024-09-16T07:27:51.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc4073fb0 msgr2=0x7fbdc41525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:51.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc4073fb0 0x7fbdc41525d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009950 tx=0x7fbdb8033c20 comp rx=0 tx=0).stop 2024-09-16T07:27:51.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 shutdown_connections 2024-09-16T07:27:51.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda40790c0 0x7fbda407b580 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc415f5a0 0x7fbdc41f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40ccf00 0x7fbdc414d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 --2- 172.21.15.71:0/4100381162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc4073fb0 0x7fbdc41525d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:51.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 >> 172.21.15.71:0/4100381162 conn(0x7fbdc40b1950 msgr2=0x7fbdc415e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:51.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 shutdown_connections 2024-09-16T07:27:51.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:51.877+0000 7fbdcc600640 1 -- 172.21.15.71:0/4100381162 wait complete. 2024-09-16T07:27:51.895 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:27:52.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.050+0000 7faffa000640 1 -- 172.21.15.71:0/2506630362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafd8003a50 msgr2=0x7fafd8003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.050+0000 7faffa000640 1 --2- 172.21.15.71:0/2506630362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafd8003a50 0x7fafd8003e90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c610 tx=0x7fafd8005830 comp rx=0 tx=0).stop 2024-09-16T07:27:52.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.051+0000 7faffa000640 1 -- 172.21.15.71:0/2506630362 shutdown_connections 2024-09-16T07:27:52.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.051+0000 7faffa000640 1 --2- 172.21.15.71:0/2506630362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafd8003a50 0x7fafd8003e90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.051+0000 7faffa000640 1 -- 172.21.15.71:0/2506630362 >> 172.21.15.71:0/2506630362 conn(0x7faff40d3df0 msgr2=0x7faff40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:52.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.051+0000 7faffa000640 1 -- 172.21.15.71:0/2506630362 shutdown_connections 2024-09-16T07:27:52.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.051+0000 7faffa000640 1 -- 172.21.15.71:0/2506630362 wait complete. 2024-09-16T07:27:52.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.052+0000 7faffa000640 1 Processor -- start 2024-09-16T07:27:52.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.052+0000 7faffa000640 1 -- start start 2024-09-16T07:27:52.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.053+0000 7faffa000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd8003a50 0x7faff41f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.053+0000 7faffa000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff415b350 0x7faff40b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.053+0000 7faffa000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faff4160080 0x7faff4160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.053+0000 7faffa000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faff40b3840 con 0x7faff4160080 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.053+0000 7faffa000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faff40b36c0 con 0x7faff415b350 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.053+0000 7faffa000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7faff40b39c0 con 0x7fafd8003a50 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faff4160080 0x7faff4160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faff4160080 0x7faff4160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48412/0 (socket says 172.21.15.71:48412) 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 -- 172.21.15.71:0/4110429262 learned_addr learned my addr 172.21.15.71:0/4110429262 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff3e00640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff415b350 0x7faff40b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff8c00640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd8003a50 0x7faff41f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 -- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd8003a50 msgr2=0x7faff41f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd8003a50 0x7faff41f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 -- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff415b350 msgr2=0x7faff40b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff415b350 0x7faff40b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 -- 172.21.15.71:0/4110429262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff40c7920 con 0x7faff4160080 2024-09-16T07:27:52.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.054+0000 7faff9600640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faff4160080 0x7faff4160500 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faff40698e0 tx=0x7fafec001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:52.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.055+0000 7faff0c00640 1 -- 172.21.15.71:0/4110429262 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec015020 con 0x7faff4160080 2024-09-16T07:27:52.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.055+0000 7faff0c00640 1 -- 172.21.15.71:0/4110429262 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafec011750 con 0x7faff4160080 2024-09-16T07:27:52.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.055+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff41f8dd0 con 0x7faff4160080 2024-09-16T07:27:52.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.055+0000 7faff0c00640 1 -- 172.21.15.71:0/4110429262 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec011ba0 con 0x7faff4160080 2024-09-16T07:27:52.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.055+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff40c7ea0 con 0x7faff4160080 2024-09-16T07:27:52.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.056+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff40d1d60 con 0x7faff4160080 2024-09-16T07:27:52.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.057+0000 7faff0c00640 1 -- 172.21.15.71:0/4110429262 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fafec018020 con 0x7faff4160080 2024-09-16T07:27:52.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.058+0000 7faff0c00640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fafd40790f0 0x7fafd407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.058+0000 7faff0c00640 1 -- 172.21.15.71:0/4110429262 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fafec099780 con 0x7faff4160080 2024-09-16T07:27:52.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.058+0000 7faff8c00640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fafd40790f0 0x7fafd407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.059+0000 7faff8c00640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fafd40790f0 0x7fafd407b5b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fafe4033000 tx=0x7faff40d4c10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:52.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.062+0000 7faff0c00640 1 -- 172.21.15.71:0/4110429262 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fafec01f020 con 0x7faff4160080 2024-09-16T07:27:52.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.248+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faff40b2570 con 0x7faff4160080 2024-09-16T07:27:52.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.249+0000 7faff0c00640 1 -- 172.21.15.71:0/4110429262 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16737 (secure 0 0 0) 0x7fafec063250 con 0x7faff4160080 2024-09-16T07:27:52.250 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:52.250 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":62,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:27:49.448783+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:27:45.670244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T07:27:46.828046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:27:52.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fafd40790f0 msgr2=0x7fafd407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fafd40790f0 0x7fafd407b5b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fafe4033000 tx=0x7faff40d4c10 comp rx=0 tx=0).stop 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faff4160080 msgr2=0x7faff4160500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faff4160080 0x7faff4160500 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faff40698e0 tx=0x7fafec001630 comp rx=0 tx=0).stop 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 shutdown_connections 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fafd40790f0 0x7fafd407b5b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faff4160080 0x7faff4160500 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff415b350 0x7faff40b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 --2- 172.21.15.71:0/4110429262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd8003a50 0x7faff41f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 >> 172.21.15.71:0/4110429262 conn(0x7faff40d3df0 msgr2=0x7faff40d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 shutdown_connections 2024-09-16T07:27:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.252+0000 7faffa000640 1 -- 172.21.15.71:0/4110429262 wait complete. 2024-09-16T07:27:52.269 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:27:52.269 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:27:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.424+0000 7f2313000640 1 -- 172.21.15.71:0/650205830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c074150 msgr2=0x7f230c072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.424+0000 7f2313000640 1 --2- 172.21.15.71:0/650205830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c074150 0x7f230c072ba0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f22f800a970 tx=0x7f22f802b7f0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.424+0000 7f2313000640 1 -- 172.21.15.71:0/650205830 shutdown_connections 2024-09-16T07:27:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.424+0000 7f2313000640 1 --2- 172.21.15.71:0/650205830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c074150 0x7f230c072ba0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.424+0000 7f2313000640 1 -- 172.21.15.71:0/650205830 >> 172.21.15.71:0/650205830 conn(0x7f230c0b1950 msgr2=0x7f230c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:52.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.424+0000 7f2313000640 1 -- 172.21.15.71:0/650205830 shutdown_connections 2024-09-16T07:27:52.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.424+0000 7f2313000640 1 -- 172.21.15.71:0/650205830 wait complete. 2024-09-16T07:27:52.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.426+0000 7f2313000640 1 Processor -- start 2024-09-16T07:27:52.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.426+0000 7f2313000640 1 -- start start 2024-09-16T07:27:52.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f2313000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230c074150 0x7f230c09d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f2313000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0746a0 0x7f230c1f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f2313000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f230c0b43f0 0x7f230c1f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f2313000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f230c073970 con 0x7f230c0746a0 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f2313000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f230c0737f0 con 0x7f230c0b43f0 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f2313000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f230c073af0 con 0x7f230c074150 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0746a0 0x7f230c1f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0746a0 0x7f230c1f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48426/0 (socket says 172.21.15.71:48426) 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230b400640 1 -- 172.21.15.71:0/2344546404 learned_addr learned my addr 172.21.15.71:0/2344546404 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230be00640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230c074150 0x7f230c09d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f2311000640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f230c0b43f0 0x7f230c1f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230b400640 1 -- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230c074150 msgr2=0x7f230c09d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230b400640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230c074150 0x7f230c09d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230b400640 1 -- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f230c0b43f0 msgr2=0x7f230c1f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230b400640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f230c0b43f0 0x7f230c1f85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.427+0000 7f230b400640 1 -- 172.21.15.71:0/2344546404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c1f9520 con 0x7f230c0746a0 2024-09-16T07:27:52.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.428+0000 7f230b400640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0746a0 0x7f230c1f5950 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f22f8000c00 tx=0x7f22f8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:52.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.428+0000 7f2303e00640 1 -- 172.21.15.71:0/2344546404 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8005220 con 0x7f230c0746a0 2024-09-16T07:27:52.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.428+0000 7f2303e00640 1 -- 172.21.15.71:0/2344546404 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22f80054d0 con 0x7f230c0746a0 2024-09-16T07:27:52.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.428+0000 7f2303e00640 1 -- 172.21.15.71:0/2344546404 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8003a60 con 0x7f230c0746a0 2024-09-16T07:27:52.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.428+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c1f8f60 con 0x7f230c0746a0 2024-09-16T07:27:52.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.428+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c1f5150 con 0x7f230c0746a0 2024-09-16T07:27:52.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.429+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230c06ac10 con 0x7f230c0746a0 2024-09-16T07:27:52.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.430+0000 7f2303e00640 1 -- 172.21.15.71:0/2344546404 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f22f8003c00 con 0x7f230c0746a0 2024-09-16T07:27:52.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.431+0000 7f2303e00640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f22ec0790c0 0x7f22ec07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.431+0000 7f2303e00640 1 -- 172.21.15.71:0/2344546404 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f22f80ba690 con 0x7f230c0746a0 2024-09-16T07:27:52.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.432+0000 7f230be00640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f22ec0790c0 0x7f22ec07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.432+0000 7f230be00640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f22ec0790c0 0x7f22ec07b580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0046b0 tx=0x7f22fc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:52.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.435+0000 7f2303e00640 1 -- 172.21.15.71:0/2344546404 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f230c06ac10 con 0x7f230c0746a0 2024-09-16T07:27:52.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.616+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f230c15c780 con 0x7f22ec0790c0 2024-09-16T07:27:52.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.622+0000 7f2303e00640 1 -- 172.21.15.71:0/2344546404 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249482 (secure 0 0 0) 0x7f230c15c780 con 0x7f22ec0790c0 2024-09-16T07:27:52.626 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:52.627 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f22ec0790c0 msgr2=0x7f22ec07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f22ec0790c0 0x7f22ec07b580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0046b0 tx=0x7f22fc009040 comp rx=0 tx=0).stop 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0746a0 msgr2=0x7f230c1f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0746a0 0x7f230c1f5950 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f22f8000c00 tx=0x7f22f8004360 comp rx=0 tx=0).stop 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 shutdown_connections 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f22ec0790c0 0x7f22ec07b580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f230c0b43f0 0x7f230c1f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0746a0 0x7f230c1f5950 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 --2- 172.21.15.71:0/2344546404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230c074150 0x7f230c09d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 >> 172.21.15.71:0/2344546404 conn(0x7f230c0b1950 msgr2=0x7f230c15a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 shutdown_connections 2024-09-16T07:27:52.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.626+0000 7f2313000640 1 -- 172.21.15.71:0/2344546404 wait complete. 2024-09-16T07:27:52.642 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-16T07:27:52.380524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":132,"num_write_kb":1634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":360328,"kb_used_data":6736,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754614392,"statfs":{"total":773094113280,"available":772725137408,"internally_reserved":0,"allocated":6897664,"data_stored":3551176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":361877040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.951511"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.453401+0000","last_change":"2024-09-16T07:27:47.453401+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.453401+0000","last_undegraded":"2024-09-16T07:27:47.453401+0000","last_fullsized":"2024-09-16T07:27:47.453401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444099+0000","last_change":"2024-09-16T07:27:47.444099+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444099+0000","last_undegraded":"2024-09-16T07:27:47.444099+0000","last_fullsized":"2024-09-16T07:27:47.444099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.463111+0000","last_change":"2024-09-16T07:27:48.453492+0000","last_active":"2024-09-16T07:27:48.463111+0000","last_peered":"2024-09-16T07:27:48.463111+0000","last_clean":"2024-09-16T07:27:48.463111+0000","last_became_active":"2024-09-16T07:27:48.453262+0000","last_became_peered":"2024-09-16T07:27:48.453262+0000","last_unstale":"2024-09-16T07:27:48.463111+0000","last_undegraded":"2024-09-16T07:27:48.463111+0000","last_fullsized":"2024-09-16T07:27:48.463111+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:50.447717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469342+0000","last_change":"2024-09-16T07:27:48.460353+0000","last_active":"2024-09-16T07:27:48.469342+0000","last_peered":"2024-09-16T07:27:48.469342+0000","last_clean":"2024-09-16T07:27:48.469342+0000","last_became_active":"2024-09-16T07:27:48.459644+0000","last_became_peered":"2024-09-16T07:27:48.459644+0000","last_unstale":"2024-09-16T07:27:48.469342+0000","last_undegraded":"2024-09-16T07:27:48.469342+0000","last_fullsized":"2024-09-16T07:27:48.469342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:00.444857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.451622+0000","last_change":"2024-09-16T07:27:47.451622+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.451622+0000","last_undegraded":"2024-09-16T07:27:47.451622+0000","last_fullsized":"2024-09-16T07:27:47.451622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.447931+0000","last_change":"2024-09-16T07:27:47.447931+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.447931+0000","last_undegraded":"2024-09-16T07:27:47.447931+0000","last_fullsized":"2024-09-16T07:27:47.447931+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.471918+0000","last_change":"2024-09-16T07:27:48.462460+0000","last_active":"2024-09-16T07:27:48.471918+0000","last_peered":"2024-09-16T07:27:48.471918+0000","last_clean":"2024-09-16T07:27:48.471918+0000","last_became_active":"2024-09-16T07:27:48.462193+0000","last_became_peered":"2024-09-16T07:27:48.462193+0000","last_unstale":"2024-09-16T07:27:48.471918+0000","last_undegraded":"2024-09-16T07:27:48.471918+0000","last_fullsized":"2024-09-16T07:27:48.471918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:03:02.929085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441520+0000","last_change":"2024-09-16T07:27:47.441520+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441520+0000","last_undegraded":"2024-09-16T07:27:47.441520+0000","last_fullsized":"2024-09-16T07:27:47.441520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469491+0000","last_change":"2024-09-16T07:27:48.461060+0000","last_active":"2024-09-16T07:27:48.469491+0000","last_peered":"2024-09-16T07:27:48.469491+0000","last_clean":"2024-09-16T07:27:48.469491+0000","last_became_active":"2024-09-16T07:27:48.460770+0000","last_became_peered":"2024-09-16T07:27:48.460770+0000","last_unstale":"2024-09-16T07:27:48.469491+0000","last_undegraded":"2024-09-16T07:27:48.469491+0000","last_fullsized":"2024-09-16T07:27:48.469491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:46.741293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.452048+0000","last_change":"2024-09-16T07:27:47.452048+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.452048+0000","last_undegraded":"2024-09-16T07:27:47.452048+0000","last_fullsized":"2024-09-16T07:27:47.452048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461558+0000","last_change":"2024-09-16T07:27:48.459130+0000","last_active":"2024-09-16T07:27:49.461558+0000","last_peered":"2024-09-16T07:27:49.461558+0000","last_clean":"2024-09-16T07:27:49.461558+0000","last_became_active":"2024-09-16T07:27:48.458360+0000","last_became_peered":"2024-09-16T07:27:48.458360+0000","last_unstale":"2024-09-16T07:27:49.461558+0000","last_undegraded":"2024-09-16T07:27:49.461558+0000","last_fullsized":"2024-09-16T07:27:49.461558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:42.756027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468989+0000","last_change":"2024-09-16T07:27:48.462533+0000","last_active":"2024-09-16T07:27:48.468989+0000","last_peered":"2024-09-16T07:27:48.468989+0000","last_clean":"2024-09-16T07:27:48.468989+0000","last_became_active":"2024-09-16T07:27:48.462296+0000","last_became_peered":"2024-09-16T07:27:48.462296+0000","last_unstale":"2024-09-16T07:27:48.468989+0000","last_undegraded":"2024-09-16T07:27:48.468989+0000","last_fullsized":"2024-09-16T07:27:48.468989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:25.012988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466007+0000","last_change":"2024-09-16T07:27:48.461185+0000","last_active":"2024-09-16T07:27:48.466007+0000","last_peered":"2024-09-16T07:27:48.466007+0000","last_clean":"2024-09-16T07:27:48.466007+0000","last_became_active":"2024-09-16T07:27:48.460889+0000","last_became_peered":"2024-09-16T07:27:48.460889+0000","last_unstale":"2024-09-16T07:27:48.466007+0000","last_undegraded":"2024-09-16T07:27:48.466007+0000","last_fullsized":"2024-09-16T07:27:48.466007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:32:32.356185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.464924+0000","last_change":"2024-09-16T07:27:48.458531+0000","last_active":"2024-09-16T07:27:48.464924+0000","last_peered":"2024-09-16T07:27:48.464924+0000","last_clean":"2024-09-16T07:27:48.464924+0000","last_became_active":"2024-09-16T07:27:48.458285+0000","last_became_peered":"2024-09-16T07:27:48.458285+0000","last_unstale":"2024-09-16T07:27:48.464924+0000","last_undegraded":"2024-09-16T07:27:48.464924+0000","last_fullsized":"2024-09-16T07:27:48.464924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:40.253906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469844+0000","last_change":"2024-09-16T07:27:48.466279+0000","last_active":"2024-09-16T07:27:48.469844+0000","last_peered":"2024-09-16T07:27:48.469844+0000","last_clean":"2024-09-16T07:27:48.469844+0000","last_became_active":"2024-09-16T07:27:48.466000+0000","last_became_peered":"2024-09-16T07:27:48.466000+0000","last_unstale":"2024-09-16T07:27:48.469844+0000","last_undegraded":"2024-09-16T07:27:48.469844+0000","last_fullsized":"2024-09-16T07:27:48.469844+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:28.032448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449957+0000","last_change":"2024-09-16T07:27:47.449957+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449957+0000","last_undegraded":"2024-09-16T07:27:47.449957+0000","last_fullsized":"2024-09-16T07:27:47.449957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449887+0000","last_change":"2024-09-16T07:27:47.449887+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449887+0000","last_undegraded":"2024-09-16T07:27:47.449887+0000","last_fullsized":"2024-09-16T07:27:47.449887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467286+0000","last_change":"2024-09-16T07:27:48.465698+0000","last_active":"2024-09-16T07:27:48.467286+0000","last_peered":"2024-09-16T07:27:48.467286+0000","last_clean":"2024-09-16T07:27:48.467286+0000","last_became_active":"2024-09-16T07:27:48.465355+0000","last_became_peered":"2024-09-16T07:27:48.465355+0000","last_unstale":"2024-09-16T07:27:48.467286+0000","last_undegraded":"2024-09-16T07:27:48.467286+0000","last_fullsized":"2024-09-16T07:27:48.467286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:59.562068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.447615+0000","last_change":"2024-09-16T07:27:47.447615+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.447615+0000","last_undegraded":"2024-09-16T07:27:47.447615+0000","last_fullsized":"2024-09-16T07:27:47.447615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461494+0000","last_change":"2024-09-16T07:27:48.460038+0000","last_active":"2024-09-16T07:27:49.461494+0000","last_peered":"2024-09-16T07:27:49.461494+0000","last_clean":"2024-09-16T07:27:49.461494+0000","last_became_active":"2024-09-16T07:27:48.459755+0000","last_became_peered":"2024-09-16T07:27:48.459755+0000","last_unstale":"2024-09-16T07:27:49.461494+0000","last_undegraded":"2024-09-16T07:27:49.461494+0000","last_fullsized":"2024-09-16T07:27:49.461494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:37.822664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467196+0000","last_change":"2024-09-16T07:27:48.464651+0000","last_active":"2024-09-16T07:27:48.467196+0000","last_peered":"2024-09-16T07:27:48.467196+0000","last_clean":"2024-09-16T07:27:48.467196+0000","last_became_active":"2024-09-16T07:27:48.464373+0000","last_became_peered":"2024-09-16T07:27:48.464373+0000","last_unstale":"2024-09-16T07:27:48.467196+0000","last_undegraded":"2024-09-16T07:27:48.467196+0000","last_fullsized":"2024-09-16T07:27:48.467196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:34.767968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461319+0000","last_change":"2024-09-16T07:27:48.461614+0000","last_active":"2024-09-16T07:27:49.461319+0000","last_peered":"2024-09-16T07:27:49.461319+0000","last_clean":"2024-09-16T07:27:49.461319+0000","last_became_active":"2024-09-16T07:27:48.461333+0000","last_became_peered":"2024-09-16T07:27:48.461333+0000","last_unstale":"2024-09-16T07:27:49.461319+0000","last_undegraded":"2024-09-16T07:27:49.461319+0000","last_fullsized":"2024-09-16T07:27:49.461319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:14.430788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.477576+0000","last_change":"2024-09-16T07:27:48.469453+0000","last_active":"2024-09-16T07:27:48.477576+0000","last_peered":"2024-09-16T07:27:48.477576+0000","last_clean":"2024-09-16T07:27:48.477576+0000","last_became_active":"2024-09-16T07:27:48.466391+0000","last_became_peered":"2024-09-16T07:27:48.466391+0000","last_unstale":"2024-09-16T07:27:48.477576+0000","last_undegraded":"2024-09-16T07:27:48.477576+0000","last_fullsized":"2024-09-16T07:27:48.477576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:26.749995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468874+0000","last_change":"2024-09-16T07:27:48.461655+0000","last_active":"2024-09-16T07:27:48.468874+0000","last_peered":"2024-09-16T07:27:48.468874+0000","last_clean":"2024-09-16T07:27:48.468874+0000","last_became_active":"2024-09-16T07:27:48.461371+0000","last_became_peered":"2024-09-16T07:27:48.461371+0000","last_unstale":"2024-09-16T07:27:48.468874+0000","last_undegraded":"2024-09-16T07:27:48.468874+0000","last_fullsized":"2024-09-16T07:27:48.468874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:14.719547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444900+0000","last_change":"2024-09-16T07:27:47.444900+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444900+0000","last_undegraded":"2024-09-16T07:27:47.444900+0000","last_fullsized":"2024-09-16T07:27:47.444900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.459564+0000","last_change":"2024-09-16T07:27:48.455880+0000","last_active":"2024-09-16T07:27:48.459564+0000","last_peered":"2024-09-16T07:27:48.459564+0000","last_clean":"2024-09-16T07:27:48.459564+0000","last_became_active":"2024-09-16T07:27:48.455633+0000","last_became_peered":"2024-09-16T07:27:48.455633+0000","last_unstale":"2024-09-16T07:27:48.459564+0000","last_undegraded":"2024-09-16T07:27:48.459564+0000","last_fullsized":"2024-09-16T07:27:48.459564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:56.366154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.446950+0000","last_change":"2024-09-16T07:27:47.446950+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.446950+0000","last_undegraded":"2024-09-16T07:27:47.446950+0000","last_fullsized":"2024-09-16T07:27:47.446950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441948+0000","last_change":"2024-09-16T07:27:47.441948+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441948+0000","last_undegraded":"2024-09-16T07:27:47.441948+0000","last_fullsized":"2024-09-16T07:27:47.441948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.439145+0000","last_change":"2024-09-16T07:27:47.439145+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.439145+0000","last_undegraded":"2024-09-16T07:27:47.439145+0000","last_fullsized":"2024-09-16T07:27:47.439145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.472115+0000","last_change":"2024-09-16T07:27:48.462181+0000","last_active":"2024-09-16T07:27:48.472115+0000","last_peered":"2024-09-16T07:27:48.472115+0000","last_clean":"2024-09-16T07:27:48.472115+0000","last_became_active":"2024-09-16T07:27:48.460601+0000","last_became_peered":"2024-09-16T07:27:48.460601+0000","last_unstale":"2024-09-16T07:27:48.472115+0000","last_undegraded":"2024-09-16T07:27:48.472115+0000","last_fullsized":"2024-09-16T07:27:48.472115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:18.404077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.443479+0000","last_change":"2024-09-16T07:27:47.443479+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.443479+0000","last_undegraded":"2024-09-16T07:27:47.443479+0000","last_fullsized":"2024-09-16T07:27:47.443479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461161+0000","last_change":"2024-09-16T07:27:48.460512+0000","last_active":"2024-09-16T07:27:49.461161+0000","last_peered":"2024-09-16T07:27:49.461161+0000","last_clean":"2024-09-16T07:27:49.461161+0000","last_became_active":"2024-09-16T07:27:48.460276+0000","last_became_peered":"2024-09-16T07:27:48.460276+0000","last_unstale":"2024-09-16T07:27:49.461161+0000","last_undegraded":"2024-09-16T07:27:49.461161+0000","last_fullsized":"2024-09-16T07:27:49.461161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.223196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.446917+0000","last_change":"2024-09-16T07:27:47.446917+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.446917+0000","last_undegraded":"2024-09-16T07:27:47.446917+0000","last_fullsized":"2024-09-16T07:27:47.446917+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461617+0000","last_change":"2024-09-16T07:27:47.450308+0000","last_active":"2024-09-16T07:27:49.461617+0000","last_peered":"2024-09-16T07:27:49.461617+0000","last_clean":"2024-09-16T07:27:49.461617+0000","last_became_active":"2024-09-16T07:27:47.448332+0000","last_became_peered":"2024-09-16T07:27:47.448332+0000","last_unstale":"2024-09-16T07:27:49.461617+0000","last_undegraded":"2024-09-16T07:27:49.461617+0000","last_fullsized":"2024-09-16T07:27:49.461617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:38.412742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449454+0000","last_change":"2024-09-16T07:27:47.449454+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449454+0000","last_undegraded":"2024-09-16T07:27:47.449454+0000","last_fullsized":"2024-09-16T07:27:47.449454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.454190+0000","last_change":"2024-09-16T07:27:47.452837+0000","last_active":"2024-09-16T07:27:47.454190+0000","last_peered":"2024-09-16T07:27:47.454190+0000","last_clean":"2024-09-16T07:27:47.454190+0000","last_became_active":"2024-09-16T07:27:47.452517+0000","last_became_peered":"2024-09-16T07:27:47.452517+0000","last_unstale":"2024-09-16T07:27:47.454190+0000","last_undegraded":"2024-09-16T07:27:47.454190+0000","last_fullsized":"2024-09-16T07:27:47.454190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:50.738735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441487+0000","last_change":"2024-09-16T07:27:47.441487+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441487+0000","last_undegraded":"2024-09-16T07:27:47.441487+0000","last_fullsized":"2024-09-16T07:27:47.441487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460327+0000","last_change":"2024-09-16T07:27:47.459254+0000","last_active":"2024-09-16T07:27:47.460327+0000","last_peered":"2024-09-16T07:27:47.460327+0000","last_clean":"2024-09-16T07:27:47.460327+0000","last_became_active":"2024-09-16T07:27:47.458912+0000","last_became_peered":"2024-09-16T07:27:47.458912+0000","last_unstale":"2024-09-16T07:27:47.460327+0000","last_undegraded":"2024-09-16T07:27:47.460327+0000","last_fullsized":"2024-09-16T07:27:47.460327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:07:13.520726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444836+0000","last_change":"2024-09-16T07:27:47.444836+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444836+0000","last_undegraded":"2024-09-16T07:27:47.444836+0000","last_fullsized":"2024-09-16T07:27:47.444836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.478412+0000","last_change":"2024-09-16T07:27:47.458877+0000","last_active":"2024-09-16T07:27:48.478412+0000","last_peered":"2024-09-16T07:27:48.478412+0000","last_clean":"2024-09-16T07:27:48.478412+0000","last_became_active":"2024-09-16T07:27:47.458615+0000","last_became_peered":"2024-09-16T07:27:47.458615+0000","last_unstale":"2024-09-16T07:27:48.478412+0000","last_undegraded":"2024-09-16T07:27:48.478412+0000","last_fullsized":"2024-09-16T07:27:48.478412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:00.779149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438546+0000","last_change":"2024-09-16T07:27:47.438546+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438546+0000","last_undegraded":"2024-09-16T07:27:47.438546+0000","last_fullsized":"2024-09-16T07:27:47.438546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.471392+0000","last_change":"2024-09-16T07:27:47.455983+0000","last_active":"2024-09-16T07:27:48.471392+0000","last_peered":"2024-09-16T07:27:48.471392+0000","last_clean":"2024-09-16T07:27:48.471392+0000","last_became_active":"2024-09-16T07:27:47.454631+0000","last_became_peered":"2024-09-16T07:27:47.454631+0000","last_unstale":"2024-09-16T07:27:48.471392+0000","last_undegraded":"2024-09-16T07:27:48.471392+0000","last_fullsized":"2024-09-16T07:27:48.471392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:37.670315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461455+0000","last_change":"2024-09-16T07:27:48.457180+0000","last_active":"2024-09-16T07:27:49.461455+0000","last_peered":"2024-09-16T07:27:49.461455+0000","last_clean":"2024-09-16T07:27:49.461455+0000","last_became_active":"2024-09-16T07:27:48.456834+0000","last_became_peered":"2024-09-16T07:27:48.456834+0000","last_unstale":"2024-09-16T07:27:49.461455+0000","last_undegraded":"2024-09-16T07:27:49.461455+0000","last_fullsized":"2024-09-16T07:27:49.461455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:04.346053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464365+0000","last_change":"2024-09-16T07:27:47.461641+0000","last_active":"2024-09-16T07:27:47.464365+0000","last_peered":"2024-09-16T07:27:47.464365+0000","last_clean":"2024-09-16T07:27:47.464365+0000","last_became_active":"2024-09-16T07:27:47.461318+0000","last_became_peered":"2024-09-16T07:27:47.461318+0000","last_unstale":"2024-09-16T07:27:47.464365+0000","last_undegraded":"2024-09-16T07:27:47.464365+0000","last_fullsized":"2024-09-16T07:27:47.464365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:10:51.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.465633+0000","last_change":"2024-09-16T07:27:48.455278+0000","last_active":"2024-09-16T07:27:48.465633+0000","last_peered":"2024-09-16T07:27:48.465633+0000","last_clean":"2024-09-16T07:27:48.465633+0000","last_became_active":"2024-09-16T07:27:48.454871+0000","last_became_peered":"2024-09-16T07:27:48.454871+0000","last_unstale":"2024-09-16T07:27:48.465633+0000","last_undegraded":"2024-09-16T07:27:48.465633+0000","last_fullsized":"2024-09-16T07:27:48.465633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:42.196208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443144+0000","last_change":"2024-09-16T07:27:47.452265+0000","last_active":"2024-09-16T07:27:48.443144+0000","last_peered":"2024-09-16T07:27:48.443144+0000","last_clean":"2024-09-16T07:27:48.443144+0000","last_became_active":"2024-09-16T07:27:47.452018+0000","last_became_peered":"2024-09-16T07:27:47.452018+0000","last_unstale":"2024-09-16T07:27:48.443144+0000","last_undegraded":"2024-09-16T07:27:48.443144+0000","last_fullsized":"2024-09-16T07:27:48.443144+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:35.000221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.448499+0000","last_change":"2024-09-16T07:27:47.448499+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.448499+0000","last_undegraded":"2024-09-16T07:27:47.448499+0000","last_fullsized":"2024-09-16T07:27:47.448499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.449811+0000","last_change":"2024-09-16T07:27:47.447783+0000","last_active":"2024-09-16T07:27:48.449811+0000","last_peered":"2024-09-16T07:27:48.449811+0000","last_clean":"2024-09-16T07:27:48.449811+0000","last_became_active":"2024-09-16T07:27:47.447510+0000","last_became_peered":"2024-09-16T07:27:47.447510+0000","last_unstale":"2024-09-16T07:27:48.449811+0000","last_undegraded":"2024-09-16T07:27:48.449811+0000","last_fullsized":"2024-09-16T07:27:48.449811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:14.370496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.459535+0000","last_change":"2024-09-16T07:27:48.454744+0000","last_active":"2024-09-16T07:27:48.459535+0000","last_peered":"2024-09-16T07:27:48.459535+0000","last_clean":"2024-09-16T07:27:48.459535+0000","last_became_active":"2024-09-16T07:27:48.454446+0000","last_became_peered":"2024-09-16T07:27:48.454446+0000","last_unstale":"2024-09-16T07:27:48.459535+0000","last_undegraded":"2024-09-16T07:27:48.459535+0000","last_fullsized":"2024-09-16T07:27:48.459535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:52.733528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443637+0000","last_change":"2024-09-16T07:27:47.450306+0000","last_active":"2024-09-16T07:27:48.443637+0000","last_peered":"2024-09-16T07:27:48.443637+0000","last_clean":"2024-09-16T07:27:48.443637+0000","last_became_active":"2024-09-16T07:27:47.449275+0000","last_became_peered":"2024-09-16T07:27:47.449275+0000","last_unstale":"2024-09-16T07:27:48.443637+0000","last_undegraded":"2024-09-16T07:27:48.443637+0000","last_fullsized":"2024-09-16T07:27:48.443637+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:00.479157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.470005+0000","last_change":"2024-09-16T07:27:48.468331+0000","last_active":"2024-09-16T07:27:48.470005+0000","last_peered":"2024-09-16T07:27:48.470005+0000","last_clean":"2024-09-16T07:27:48.470005+0000","last_became_active":"2024-09-16T07:27:48.468040+0000","last_became_peered":"2024-09-16T07:27:48.468040+0000","last_unstale":"2024-09-16T07:27:48.470005+0000","last_undegraded":"2024-09-16T07:27:48.470005+0000","last_fullsized":"2024-09-16T07:27:48.470005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:06.084517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453197+0000","last_change":"2024-09-16T07:27:47.450498+0000","last_active":"2024-09-16T07:27:47.453197+0000","last_peered":"2024-09-16T07:27:47.453197+0000","last_clean":"2024-09-16T07:27:47.453197+0000","last_became_active":"2024-09-16T07:27:47.449615+0000","last_became_peered":"2024-09-16T07:27:47.449615+0000","last_unstale":"2024-09-16T07:27:47.453197+0000","last_undegraded":"2024-09-16T07:27:47.453197+0000","last_fullsized":"2024-09-16T07:27:47.453197+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:23:34.664293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438573+0000","last_change":"2024-09-16T07:27:47.438573+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438573+0000","last_undegraded":"2024-09-16T07:27:47.438573+0000","last_fullsized":"2024-09-16T07:27:47.438573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.478154+0000","last_change":"2024-09-16T07:27:47.460337+0000","last_active":"2024-09-16T07:27:48.478154+0000","last_peered":"2024-09-16T07:27:48.478154+0000","last_clean":"2024-09-16T07:27:48.478154+0000","last_became_active":"2024-09-16T07:27:47.460023+0000","last_became_peered":"2024-09-16T07:27:47.460023+0000","last_unstale":"2024-09-16T07:27:48.478154+0000","last_undegraded":"2024-09-16T07:27:48.478154+0000","last_fullsized":"2024-09-16T07:27:48.478154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:52.763400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.456699+0000","last_change":"2024-09-16T07:27:48.449778+0000","last_active":"2024-09-16T07:27:48.456699+0000","last_peered":"2024-09-16T07:27:48.456699+0000","last_clean":"2024-09-16T07:27:48.456699+0000","last_became_active":"2024-09-16T07:27:48.449504+0000","last_became_peered":"2024-09-16T07:27:48.449504+0000","last_unstale":"2024-09-16T07:27:48.456699+0000","last_undegraded":"2024-09-16T07:27:48.456699+0000","last_fullsized":"2024-09-16T07:27:48.456699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:48.419517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.480574+0000","last_change":"2024-09-16T07:27:47.458310+0000","last_active":"2024-09-16T07:27:48.480574+0000","last_peered":"2024-09-16T07:27:48.480574+0000","last_clean":"2024-09-16T07:27:48.480574+0000","last_became_active":"2024-09-16T07:27:47.457920+0000","last_became_peered":"2024-09-16T07:27:47.457920+0000","last_unstale":"2024-09-16T07:27:48.480574+0000","last_undegraded":"2024-09-16T07:27:48.480574+0000","last_fullsized":"2024-09-16T07:27:48.480574+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.526953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469560+0000","last_change":"2024-09-16T07:27:48.467738+0000","last_active":"2024-09-16T07:27:48.469560+0000","last_peered":"2024-09-16T07:27:48.469560+0000","last_clean":"2024-09-16T07:27:48.469560+0000","last_became_active":"2024-09-16T07:27:48.467296+0000","last_became_peered":"2024-09-16T07:27:48.467296+0000","last_unstale":"2024-09-16T07:27:48.469560+0000","last_undegraded":"2024-09-16T07:27:48.469560+0000","last_fullsized":"2024-09-16T07:27:48.469560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:08.840299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453240+0000","last_change":"2024-09-16T07:27:47.449886+0000","last_active":"2024-09-16T07:27:47.453240+0000","last_peered":"2024-09-16T07:27:47.453240+0000","last_clean":"2024-09-16T07:27:47.453240+0000","last_became_active":"2024-09-16T07:27:47.449599+0000","last_became_peered":"2024-09-16T07:27:47.449599+0000","last_unstale":"2024-09-16T07:27:47.453240+0000","last_undegraded":"2024-09-16T07:27:47.453240+0000","last_fullsized":"2024-09-16T07:27:47.453240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:45:32.473190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466091+0000","last_change":"2024-09-16T07:27:48.453257+0000","last_active":"2024-09-16T07:27:48.466091+0000","last_peered":"2024-09-16T07:27:48.466091+0000","last_clean":"2024-09-16T07:27:48.466091+0000","last_became_active":"2024-09-16T07:27:48.453020+0000","last_became_peered":"2024-09-16T07:27:48.453020+0000","last_unstale":"2024-09-16T07:27:48.466091+0000","last_undegraded":"2024-09-16T07:27:48.466091+0000","last_fullsized":"2024-09-16T07:27:48.466091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:07.527458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.439529+0000","last_change":"2024-09-16T07:27:47.449870+0000","last_active":"2024-09-16T07:27:48.439529+0000","last_peered":"2024-09-16T07:27:48.439529+0000","last_clean":"2024-09-16T07:27:48.439529+0000","last_became_active":"2024-09-16T07:27:47.449587+0000","last_became_peered":"2024-09-16T07:27:47.449587+0000","last_unstale":"2024-09-16T07:27:48.439529+0000","last_undegraded":"2024-09-16T07:27:48.439529+0000","last_fullsized":"2024-09-16T07:27:48.439529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:56.658575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468654+0000","last_change":"2024-09-16T07:27:48.457526+0000","last_active":"2024-09-16T07:27:48.468654+0000","last_peered":"2024-09-16T07:27:48.468654+0000","last_clean":"2024-09-16T07:27:48.468654+0000","last_became_active":"2024-09-16T07:27:48.457292+0000","last_became_peered":"2024-09-16T07:27:48.457292+0000","last_unstale":"2024-09-16T07:27:48.468654+0000","last_undegraded":"2024-09-16T07:27:48.468654+0000","last_fullsized":"2024-09-16T07:27:48.468654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:18.042402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.439349+0000","last_change":"2024-09-16T07:27:47.450647+0000","last_active":"2024-09-16T07:27:48.439349+0000","last_peered":"2024-09-16T07:27:48.439349+0000","last_clean":"2024-09-16T07:27:48.439349+0000","last_became_active":"2024-09-16T07:27:47.450313+0000","last_became_peered":"2024-09-16T07:27:47.450313+0000","last_unstale":"2024-09-16T07:27:48.439349+0000","last_undegraded":"2024-09-16T07:27:48.439349+0000","last_fullsized":"2024-09-16T07:27:48.439349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:01.115128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467730+0000","last_change":"2024-09-16T07:27:48.459503+0000","last_active":"2024-09-16T07:27:48.467730+0000","last_peered":"2024-09-16T07:27:48.467730+0000","last_clean":"2024-09-16T07:27:48.467730+0000","last_became_active":"2024-09-16T07:27:48.459201+0000","last_became_peered":"2024-09-16T07:27:48.459201+0000","last_unstale":"2024-09-16T07:27:48.467730+0000","last_undegraded":"2024-09-16T07:27:48.467730+0000","last_fullsized":"2024-09-16T07:27:48.467730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:35.394756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.482800+0000","last_change":"2024-09-16T07:27:47.451210+0000","last_active":"2024-09-16T07:27:48.482800+0000","last_peered":"2024-09-16T07:27:48.482800+0000","last_clean":"2024-09-16T07:27:48.482800+0000","last_became_active":"2024-09-16T07:27:47.450925+0000","last_became_peered":"2024-09-16T07:27:47.450925+0000","last_unstale":"2024-09-16T07:27:48.482800+0000","last_undegraded":"2024-09-16T07:27:48.482800+0000","last_fullsized":"2024-09-16T07:27:48.482800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:03.229023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.464941+0000","last_change":"2024-09-16T07:27:48.457648+0000","last_active":"2024-09-16T07:27:48.464941+0000","last_peered":"2024-09-16T07:27:48.464941+0000","last_clean":"2024-09-16T07:27:48.464941+0000","last_became_active":"2024-09-16T07:27:48.457358+0000","last_became_peered":"2024-09-16T07:27:48.457358+0000","last_unstale":"2024-09-16T07:27:48.464941+0000","last_undegraded":"2024-09-16T07:27:48.464941+0000","last_fullsized":"2024-09-16T07:27:48.464941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:28.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460425+0000","last_change":"2024-09-16T07:27:47.458791+0000","last_active":"2024-09-16T07:27:47.460425+0000","last_peered":"2024-09-16T07:27:47.460425+0000","last_clean":"2024-09-16T07:27:47.460425+0000","last_became_active":"2024-09-16T07:27:47.458560+0000","last_became_peered":"2024-09-16T07:27:47.458560+0000","last_unstale":"2024-09-16T07:27:47.460425+0000","last_undegraded":"2024-09-16T07:27:47.460425+0000","last_fullsized":"2024-09-16T07:27:47.460425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:15.540429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468200+0000","last_change":"2024-09-16T07:27:48.460172+0000","last_active":"2024-09-16T07:27:48.468200+0000","last_peered":"2024-09-16T07:27:48.468200+0000","last_clean":"2024-09-16T07:27:48.468200+0000","last_became_active":"2024-09-16T07:27:48.459930+0000","last_became_peered":"2024-09-16T07:27:48.459930+0000","last_unstale":"2024-09-16T07:27:48.468200+0000","last_undegraded":"2024-09-16T07:27:48.468200+0000","last_fullsized":"2024-09-16T07:27:48.468200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:04.739035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453174+0000","last_change":"2024-09-16T07:27:47.450135+0000","last_active":"2024-09-16T07:27:47.453174+0000","last_peered":"2024-09-16T07:27:47.453174+0000","last_clean":"2024-09-16T07:27:47.453174+0000","last_became_active":"2024-09-16T07:27:47.449254+0000","last_became_peered":"2024-09-16T07:27:47.449254+0000","last_unstale":"2024-09-16T07:27:47.453174+0000","last_undegraded":"2024-09-16T07:27:47.453174+0000","last_fullsized":"2024-09-16T07:27:47.453174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:16.114777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444472+0000","last_change":"2024-09-16T07:27:47.444472+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444472+0000","last_undegraded":"2024-09-16T07:27:47.444472+0000","last_fullsized":"2024-09-16T07:27:47.444472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.476242+0000","last_change":"2024-09-16T07:27:47.455984+0000","last_active":"2024-09-16T07:27:48.476242+0000","last_peered":"2024-09-16T07:27:48.476242+0000","last_clean":"2024-09-16T07:27:48.476242+0000","last_became_active":"2024-09-16T07:27:47.454791+0000","last_became_peered":"2024-09-16T07:27:47.454791+0000","last_unstale":"2024-09-16T07:27:48.476242+0000","last_undegraded":"2024-09-16T07:27:48.476242+0000","last_fullsized":"2024-09-16T07:27:48.476242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:24.745832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.472184+0000","last_change":"2024-09-16T07:27:48.462451+0000","last_active":"2024-09-16T07:27:48.472184+0000","last_peered":"2024-09-16T07:27:48.472184+0000","last_clean":"2024-09-16T07:27:48.472184+0000","last_became_active":"2024-09-16T07:27:48.462145+0000","last_became_peered":"2024-09-16T07:27:48.462145+0000","last_unstale":"2024-09-16T07:27:48.472184+0000","last_undegraded":"2024-09-16T07:27:48.472184+0000","last_fullsized":"2024-09-16T07:27:48.472184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:25:02.985426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460913+0000","last_change":"2024-09-16T07:27:47.459943+0000","last_active":"2024-09-16T07:27:47.460913+0000","last_peered":"2024-09-16T07:27:47.460913+0000","last_clean":"2024-09-16T07:27:47.460913+0000","last_became_active":"2024-09-16T07:27:47.459680+0000","last_became_peered":"2024-09-16T07:27:47.459680+0000","last_unstale":"2024-09-16T07:27:47.460913+0000","last_undegraded":"2024-09-16T07:27:47.460913+0000","last_fullsized":"2024-09-16T07:27:47.460913+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:44.061784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468987+0000","last_change":"2024-09-16T07:27:48.461989+0000","last_active":"2024-09-16T07:27:48.468987+0000","last_peered":"2024-09-16T07:27:48.468987+0000","last_clean":"2024-09-16T07:27:48.468987+0000","last_became_active":"2024-09-16T07:27:48.461731+0000","last_became_peered":"2024-09-16T07:27:48.461731+0000","last_unstale":"2024-09-16T07:27:48.468987+0000","last_undegraded":"2024-09-16T07:27:48.468987+0000","last_fullsized":"2024-09-16T07:27:48.468987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:43.744251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461382+0000","last_change":"2024-09-16T07:27:47.452023+0000","last_active":"2024-09-16T07:27:49.461382+0000","last_peered":"2024-09-16T07:27:49.461382+0000","last_clean":"2024-09-16T07:27:49.461382+0000","last_became_active":"2024-09-16T07:27:47.451738+0000","last_became_peered":"2024-09-16T07:27:47.451738+0000","last_unstale":"2024-09-16T07:27:49.461382+0000","last_undegraded":"2024-09-16T07:27:49.461382+0000","last_fullsized":"2024-09-16T07:27:49.461382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:28.412285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438666+0000","last_change":"2024-09-16T07:27:47.438666+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438666+0000","last_undegraded":"2024-09-16T07:27:47.438666+0000","last_fullsized":"2024-09-16T07:27:47.438666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.457752+0000","last_change":"2024-09-16T07:27:47.452663+0000","last_active":"2024-09-16T07:27:47.457752+0000","last_peered":"2024-09-16T07:27:47.457752+0000","last_clean":"2024-09-16T07:27:47.457752+0000","last_became_active":"2024-09-16T07:27:47.451570+0000","last_became_peered":"2024-09-16T07:27:47.451570+0000","last_unstale":"2024-09-16T07:27:47.457752+0000","last_undegraded":"2024-09-16T07:27:47.457752+0000","last_fullsized":"2024-09-16T07:27:47.457752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:14.561916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.439442+0000","last_change":"2024-09-16T07:27:47.439442+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.439442+0000","last_undegraded":"2024-09-16T07:27:47.439442+0000","last_fullsized":"2024-09-16T07:27:47.439442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443403+0000","last_change":"2024-09-16T07:27:47.455318+0000","last_active":"2024-09-16T07:27:48.443403+0000","last_peered":"2024-09-16T07:27:48.443403+0000","last_clean":"2024-09-16T07:27:48.443403+0000","last_became_active":"2024-09-16T07:27:47.455047+0000","last_became_peered":"2024-09-16T07:27:47.455047+0000","last_unstale":"2024-09-16T07:27:48.443403+0000","last_undegraded":"2024-09-16T07:27:48.443403+0000","last_fullsized":"2024-09-16T07:27:48.443403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:08.028003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466860+0000","last_change":"2024-09-16T07:27:48.462125+0000","last_active":"2024-09-16T07:27:48.466860+0000","last_peered":"2024-09-16T07:27:48.466860+0000","last_clean":"2024-09-16T07:27:48.466860+0000","last_became_active":"2024-09-16T07:27:48.461779+0000","last_became_peered":"2024-09-16T07:27:48.461779+0000","last_unstale":"2024-09-16T07:27:48.466860+0000","last_undegraded":"2024-09-16T07:27:48.466860+0000","last_fullsized":"2024-09-16T07:27:48.466860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:37.463436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464161+0000","last_change":"2024-09-16T07:27:47.461596+0000","last_active":"2024-09-16T07:27:47.464161+0000","last_peered":"2024-09-16T07:27:47.464161+0000","last_clean":"2024-09-16T07:27:47.464161+0000","last_became_active":"2024-09-16T07:27:47.461292+0000","last_became_peered":"2024-09-16T07:27:47.461292+0000","last_unstale":"2024-09-16T07:27:47.464161+0000","last_undegraded":"2024-09-16T07:27:47.464161+0000","last_fullsized":"2024-09-16T07:27:47.464161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:22.241647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.463013+0000","last_change":"2024-09-16T07:27:48.452873+0000","last_active":"2024-09-16T07:27:48.463013+0000","last_peered":"2024-09-16T07:27:48.463013+0000","last_clean":"2024-09-16T07:27:48.463013+0000","last_became_active":"2024-09-16T07:27:48.450871+0000","last_became_peered":"2024-09-16T07:27:48.450871+0000","last_unstale":"2024-09-16T07:27:48.463013+0000","last_undegraded":"2024-09-16T07:27:48.463013+0000","last_fullsized":"2024-09-16T07:27:48.463013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:38:38.451975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464346+0000","last_change":"2024-09-16T07:27:47.461620+0000","last_active":"2024-09-16T07:27:47.464346+0000","last_peered":"2024-09-16T07:27:47.464346+0000","last_clean":"2024-09-16T07:27:47.464346+0000","last_became_active":"2024-09-16T07:27:47.461338+0000","last_became_peered":"2024-09-16T07:27:47.461338+0000","last_unstale":"2024-09-16T07:27:47.464346+0000","last_undegraded":"2024-09-16T07:27:47.464346+0000","last_fullsized":"2024-09-16T07:27:47.464346+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:28.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444027+0000","last_change":"2024-09-16T07:27:47.444027+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444027+0000","last_undegraded":"2024-09-16T07:27:47.444027+0000","last_fullsized":"2024-09-16T07:27:47.444027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443387+0000","last_change":"2024-09-16T07:27:47.458964+0000","last_active":"2024-09-16T07:27:48.443387+0000","last_peered":"2024-09-16T07:27:48.443387+0000","last_clean":"2024-09-16T07:27:48.443387+0000","last_became_active":"2024-09-16T07:27:47.458687+0000","last_became_peered":"2024-09-16T07:27:47.458687+0000","last_unstale":"2024-09-16T07:27:48.443387+0000","last_undegraded":"2024-09-16T07:27:48.443387+0000","last_fullsized":"2024-09-16T07:27:48.443387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:43.227646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438895+0000","last_change":"2024-09-16T07:27:47.438895+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438895+0000","last_undegraded":"2024-09-16T07:27:47.438895+0000","last_fullsized":"2024-09-16T07:27:47.438895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.448905+0000","last_change":"2024-09-16T07:27:47.456743+0000","last_active":"2024-09-16T07:27:48.448905+0000","last_peered":"2024-09-16T07:27:48.448905+0000","last_clean":"2024-09-16T07:27:48.448905+0000","last_became_active":"2024-09-16T07:27:47.455261+0000","last_became_peered":"2024-09-16T07:27:47.455261+0000","last_unstale":"2024-09-16T07:27:48.448905+0000","last_undegraded":"2024-09-16T07:27:48.448905+0000","last_fullsized":"2024-09-16T07:27:48.448905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:44.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444420+0000","last_change":"2024-09-16T07:27:47.444420+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444420+0000","last_undegraded":"2024-09-16T07:27:47.444420+0000","last_fullsized":"2024-09-16T07:27:47.444420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.459115+0000","last_change":"2024-09-16T07:27:47.458363+0000","last_active":"2024-09-16T07:27:47.459115+0000","last_peered":"2024-09-16T07:27:47.459115+0000","last_clean":"2024-09-16T07:27:47.459115+0000","last_became_active":"2024-09-16T07:27:47.455595+0000","last_became_peered":"2024-09-16T07:27:47.455595+0000","last_unstale":"2024-09-16T07:27:47.459115+0000","last_undegraded":"2024-09-16T07:27:47.459115+0000","last_fullsized":"2024-09-16T07:27:47.459115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:18.944163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.465977+0000","last_change":"2024-09-16T07:27:48.460471+0000","last_active":"2024-09-16T07:27:48.465977+0000","last_peered":"2024-09-16T07:27:48.465977+0000","last_clean":"2024-09-16T07:27:48.465977+0000","last_became_active":"2024-09-16T07:27:48.456813+0000","last_became_peered":"2024-09-16T07:27:48.456813+0000","last_unstale":"2024-09-16T07:27:48.465977+0000","last_undegraded":"2024-09-16T07:27:48.465977+0000","last_fullsized":"2024-09-16T07:27:48.465977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:45.085619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.449173+0000","last_change":"2024-09-16T07:27:47.452638+0000","last_active":"2024-09-16T07:27:48.449173+0000","last_peered":"2024-09-16T07:27:48.449173+0000","last_clean":"2024-09-16T07:27:48.449173+0000","last_became_active":"2024-09-16T07:27:47.452390+0000","last_became_peered":"2024-09-16T07:27:47.452390+0000","last_unstale":"2024-09-16T07:27:48.449173+0000","last_undegraded":"2024-09-16T07:27:48.449173+0000","last_fullsized":"2024-09-16T07:27:48.449173+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:04.058738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.440856+0000","last_change":"2024-09-16T07:27:47.440856+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.440856+0000","last_undegraded":"2024-09-16T07:27:47.440856+0000","last_fullsized":"2024-09-16T07:27:47.440856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.462516+0000","last_change":"2024-09-16T07:27:47.460442+0000","last_active":"2024-09-16T07:27:47.462516+0000","last_peered":"2024-09-16T07:27:47.462516+0000","last_clean":"2024-09-16T07:27:47.462516+0000","last_became_active":"2024-09-16T07:27:47.459037+0000","last_became_peered":"2024-09-16T07:27:47.459037+0000","last_unstale":"2024-09-16T07:27:47.462516+0000","last_undegraded":"2024-09-16T07:27:47.462516+0000","last_fullsized":"2024-09-16T07:27:47.462516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:25:58.753488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.442957+0000","last_change":"2024-09-16T07:27:47.442957+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.442957+0000","last_undegraded":"2024-09-16T07:27:47.442957+0000","last_fullsized":"2024-09-16T07:27:47.442957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460678+0000","last_change":"2024-09-16T07:27:47.453007+0000","last_active":"2024-09-16T07:27:47.460678+0000","last_peered":"2024-09-16T07:27:47.460678+0000","last_clean":"2024-09-16T07:27:47.460678+0000","last_became_active":"2024-09-16T07:27:47.452735+0000","last_became_peered":"2024-09-16T07:27:47.452735+0000","last_unstale":"2024-09-16T07:27:47.460678+0000","last_undegraded":"2024-09-16T07:27:47.460678+0000","last_fullsized":"2024-09-16T07:27:47.460678+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:47:52.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.440008+0000","last_change":"2024-09-16T07:27:47.440008+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.440008+0000","last_undegraded":"2024-09-16T07:27:47.440008+0000","last_fullsized":"2024-09-16T07:27:47.440008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.450452+0000","last_change":"2024-09-16T07:27:47.454175+0000","last_active":"2024-09-16T07:27:48.450452+0000","last_peered":"2024-09-16T07:27:48.450452+0000","last_clean":"2024-09-16T07:27:48.450452+0000","last_became_active":"2024-09-16T07:27:47.453769+0000","last_became_peered":"2024-09-16T07:27:47.453769+0000","last_unstale":"2024-09-16T07:27:48.450452+0000","last_undegraded":"2024-09-16T07:27:48.450452+0000","last_fullsized":"2024-09-16T07:27:48.450452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:17.667822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443464+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:27:48.443464+0000","last_peered":"2024-09-16T07:27:48.443464+0000","last_clean":"2024-09-16T07:27:48.443464+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:27:48.443464+0000","last_undegraded":"2024-09-16T07:27:48.443464+0000","last_fullsized":"2024-09-16T07:27:48.443464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476803,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":295094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476803,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62448,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309392,"statfs":{"total":96636764160,"available":96572817408,"internally_reserved":0,"allocated":741376,"data_stored":303641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476803,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40784,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331056,"statfs":{"total":96636764160,"available":96595001344,"internally_reserved":0,"allocated":1302528,"data_stored":885462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476802,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":295094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":288819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509506,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":288819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46352,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325488,"statfs":{"total":96636764160,"available":96589299712,"internally_reserved":0,"allocated":1302528,"data_stored":885462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49960,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321880,"statfs":{"total":96636764160,"available":96585605120,"internally_reserved":0,"allocated":741376,"data_stored":308785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:27:52.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:27:52.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.839+0000 7f5126c00640 1 -- 172.21.15.71:0/1547123609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51201438b0 msgr2=0x7f51200cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.839+0000 7f5126c00640 1 --2- 172.21.15.71:0/1547123609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51201438b0 0x7f51200cc5f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5110005fd0 tx=0x7f511002a920 comp rx=0 tx=0).stop 2024-09-16T07:27:52.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.839+0000 7f5126c00640 1 -- 172.21.15.71:0/1547123609 shutdown_connections 2024-09-16T07:27:52.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.839+0000 7f5126c00640 1 --2- 172.21.15.71:0/1547123609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51201438b0 0x7f51200cc5f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.839+0000 7f5126c00640 1 -- 172.21.15.71:0/1547123609 >> 172.21.15.71:0/1547123609 conn(0x7f51200b1950 msgr2=0x7f51200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:52.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.839+0000 7f5126c00640 1 -- 172.21.15.71:0/1547123609 shutdown_connections 2024-09-16T07:27:52.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.839+0000 7f5126c00640 1 -- 172.21.15.71:0/1547123609 wait complete. 2024-09-16T07:27:52.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.840+0000 7f5126c00640 1 Processor -- start 2024-09-16T07:27:52.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.840+0000 7f5126c00640 1 -- start start 2024-09-16T07:27:52.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.841+0000 7f5126c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512006ff60 0x7f512015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.841+0000 7f5126c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51200ccb30 0x7f51200b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.841+0000 7f5126c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51201438b0 0x7f51200b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.841+0000 7f5126c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51200b4610 con 0x7f51201438b0 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.841+0000 7f5126c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5120070b30 con 0x7f51200ccb30 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.841+0000 7f5126c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f51200b4790 con 0x7f512006ff60 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51201438b0 0x7f51200b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51201438b0 0x7f51200b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48436/0 (socket says 172.21.15.71:48436) 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 -- 172.21.15.71:0/496323030 learned_addr learned my addr 172.21.15.71:0/496323030 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f511fe00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512006ff60 0x7f512015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f511f400640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51200ccb30 0x7f51200b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 -- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512006ff60 msgr2=0x7f512015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512006ff60 0x7f512015ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 -- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51200ccb30 msgr2=0x7f51200b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:52.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51200ccb30 0x7f51200b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:52.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 -- 172.21.15.71:0/496323030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51200cbe00 con 0x7f51201438b0 2024-09-16T07:27:52.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f511f400640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51200ccb30 0x7f51200b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:52.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.842+0000 7f5124c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51201438b0 0x7f51200b09e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f511800c3f0 tx=0x7f511800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:52.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.843+0000 7f5117e00640 1 -- 172.21.15.71:0/496323030 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118017070 con 0x7f51201438b0 2024-09-16T07:27:52.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.843+0000 7f5117e00640 1 -- 172.21.15.71:0/496323030 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5118006d90 con 0x7f51201438b0 2024-09-16T07:27:52.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.843+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512009aba0 con 0x7f51201438b0 2024-09-16T07:27:52.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.843+0000 7f5117e00640 1 -- 172.21.15.71:0/496323030 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118007290 con 0x7f51201438b0 2024-09-16T07:27:52.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.843+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512009b160 con 0x7f51201438b0 2024-09-16T07:27:52.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.844+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512009b360 con 0x7f51201438b0 2024-09-16T07:27:52.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.845+0000 7f5117e00640 1 -- 172.21.15.71:0/496323030 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f511801a070 con 0x7f51201438b0 2024-09-16T07:27:52.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.846+0000 7f5117e00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f51000790c0 0x7f510007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:52.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.846+0000 7f5117e00640 1 -- 172.21.15.71:0/496323030 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f511809aeb0 con 0x7f51201438b0 2024-09-16T07:27:52.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.846+0000 7f511fe00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f51000790c0 0x7f510007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:52.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.847+0000 7f511fe00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f51000790c0 0x7f510007b580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5110033000 tx=0x7f5110004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:52.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:52.850+0000 7f5117e00640 1 -- 172.21.15.71:0/496323030 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f512009b360 con 0x7f51201438b0 2024-09-16T07:27:53.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.030+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f512015b380 con 0x7f51000790c0 2024-09-16T07:27:53.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.035+0000 7f5117e00640 1 -- 172.21.15.71:0/496323030 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249482 (secure 0 0 0) 0x7f512015b380 con 0x7f51000790c0 2024-09-16T07:27:53.038 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:53.039 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f51000790c0 msgr2=0x7f510007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f51000790c0 0x7f510007b580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5110033000 tx=0x7f5110004ab0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51201438b0 msgr2=0x7f51200b09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51201438b0 0x7f51200b09e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f511800c3f0 tx=0x7f511800c8b0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 shutdown_connections 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f51000790c0 0x7f510007b580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51201438b0 0x7f51200b09e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51200ccb30 0x7f51200b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 --2- 172.21.15.71:0/496323030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512006ff60 0x7f512015ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 >> 172.21.15.71:0/496323030 conn(0x7f51200b1950 msgr2=0x7f512015d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:53.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 shutdown_connections 2024-09-16T07:27:53.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.039+0000 7f5126c00640 1 -- 172.21.15.71:0/496323030 wait complete. 2024-09-16T07:27:53.056 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-16T07:27:52.380524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":132,"num_write_kb":1634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":360328,"kb_used_data":6736,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754614392,"statfs":{"total":773094113280,"available":772725137408,"internally_reserved":0,"allocated":6897664,"data_stored":3551176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":361877040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.951511"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.453401+0000","last_change":"2024-09-16T07:27:47.453401+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.453401+0000","last_undegraded":"2024-09-16T07:27:47.453401+0000","last_fullsized":"2024-09-16T07:27:47.453401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444099+0000","last_change":"2024-09-16T07:27:47.444099+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444099+0000","last_undegraded":"2024-09-16T07:27:47.444099+0000","last_fullsized":"2024-09-16T07:27:47.444099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.463111+0000","last_change":"2024-09-16T07:27:48.453492+0000","last_active":"2024-09-16T07:27:48.463111+0000","last_peered":"2024-09-16T07:27:48.463111+0000","last_clean":"2024-09-16T07:27:48.463111+0000","last_became_active":"2024-09-16T07:27:48.453262+0000","last_became_peered":"2024-09-16T07:27:48.453262+0000","last_unstale":"2024-09-16T07:27:48.463111+0000","last_undegraded":"2024-09-16T07:27:48.463111+0000","last_fullsized":"2024-09-16T07:27:48.463111+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:50.447717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469342+0000","last_change":"2024-09-16T07:27:48.460353+0000","last_active":"2024-09-16T07:27:48.469342+0000","last_peered":"2024-09-16T07:27:48.469342+0000","last_clean":"2024-09-16T07:27:48.469342+0000","last_became_active":"2024-09-16T07:27:48.459644+0000","last_became_peered":"2024-09-16T07:27:48.459644+0000","last_unstale":"2024-09-16T07:27:48.469342+0000","last_undegraded":"2024-09-16T07:27:48.469342+0000","last_fullsized":"2024-09-16T07:27:48.469342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:00.444857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.451622+0000","last_change":"2024-09-16T07:27:47.451622+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.451622+0000","last_undegraded":"2024-09-16T07:27:47.451622+0000","last_fullsized":"2024-09-16T07:27:47.451622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.447931+0000","last_change":"2024-09-16T07:27:47.447931+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.447931+0000","last_undegraded":"2024-09-16T07:27:47.447931+0000","last_fullsized":"2024-09-16T07:27:47.447931+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.471918+0000","last_change":"2024-09-16T07:27:48.462460+0000","last_active":"2024-09-16T07:27:48.471918+0000","last_peered":"2024-09-16T07:27:48.471918+0000","last_clean":"2024-09-16T07:27:48.471918+0000","last_became_active":"2024-09-16T07:27:48.462193+0000","last_became_peered":"2024-09-16T07:27:48.462193+0000","last_unstale":"2024-09-16T07:27:48.471918+0000","last_undegraded":"2024-09-16T07:27:48.471918+0000","last_fullsized":"2024-09-16T07:27:48.471918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:03:02.929085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441520+0000","last_change":"2024-09-16T07:27:47.441520+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441520+0000","last_undegraded":"2024-09-16T07:27:47.441520+0000","last_fullsized":"2024-09-16T07:27:47.441520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469491+0000","last_change":"2024-09-16T07:27:48.461060+0000","last_active":"2024-09-16T07:27:48.469491+0000","last_peered":"2024-09-16T07:27:48.469491+0000","last_clean":"2024-09-16T07:27:48.469491+0000","last_became_active":"2024-09-16T07:27:48.460770+0000","last_became_peered":"2024-09-16T07:27:48.460770+0000","last_unstale":"2024-09-16T07:27:48.469491+0000","last_undegraded":"2024-09-16T07:27:48.469491+0000","last_fullsized":"2024-09-16T07:27:48.469491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:46.741293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.452048+0000","last_change":"2024-09-16T07:27:47.452048+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.452048+0000","last_undegraded":"2024-09-16T07:27:47.452048+0000","last_fullsized":"2024-09-16T07:27:47.452048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461558+0000","last_change":"2024-09-16T07:27:48.459130+0000","last_active":"2024-09-16T07:27:49.461558+0000","last_peered":"2024-09-16T07:27:49.461558+0000","last_clean":"2024-09-16T07:27:49.461558+0000","last_became_active":"2024-09-16T07:27:48.458360+0000","last_became_peered":"2024-09-16T07:27:48.458360+0000","last_unstale":"2024-09-16T07:27:49.461558+0000","last_undegraded":"2024-09-16T07:27:49.461558+0000","last_fullsized":"2024-09-16T07:27:49.461558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:42.756027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468989+0000","last_change":"2024-09-16T07:27:48.462533+0000","last_active":"2024-09-16T07:27:48.468989+0000","last_peered":"2024-09-16T07:27:48.468989+0000","last_clean":"2024-09-16T07:27:48.468989+0000","last_became_active":"2024-09-16T07:27:48.462296+0000","last_became_peered":"2024-09-16T07:27:48.462296+0000","last_unstale":"2024-09-16T07:27:48.468989+0000","last_undegraded":"2024-09-16T07:27:48.468989+0000","last_fullsized":"2024-09-16T07:27:48.468989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:25.012988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466007+0000","last_change":"2024-09-16T07:27:48.461185+0000","last_active":"2024-09-16T07:27:48.466007+0000","last_peered":"2024-09-16T07:27:48.466007+0000","last_clean":"2024-09-16T07:27:48.466007+0000","last_became_active":"2024-09-16T07:27:48.460889+0000","last_became_peered":"2024-09-16T07:27:48.460889+0000","last_unstale":"2024-09-16T07:27:48.466007+0000","last_undegraded":"2024-09-16T07:27:48.466007+0000","last_fullsized":"2024-09-16T07:27:48.466007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:32:32.356185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.464924+0000","last_change":"2024-09-16T07:27:48.458531+0000","last_active":"2024-09-16T07:27:48.464924+0000","last_peered":"2024-09-16T07:27:48.464924+0000","last_clean":"2024-09-16T07:27:48.464924+0000","last_became_active":"2024-09-16T07:27:48.458285+0000","last_became_peered":"2024-09-16T07:27:48.458285+0000","last_unstale":"2024-09-16T07:27:48.464924+0000","last_undegraded":"2024-09-16T07:27:48.464924+0000","last_fullsized":"2024-09-16T07:27:48.464924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:40.253906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469844+0000","last_change":"2024-09-16T07:27:48.466279+0000","last_active":"2024-09-16T07:27:48.469844+0000","last_peered":"2024-09-16T07:27:48.469844+0000","last_clean":"2024-09-16T07:27:48.469844+0000","last_became_active":"2024-09-16T07:27:48.466000+0000","last_became_peered":"2024-09-16T07:27:48.466000+0000","last_unstale":"2024-09-16T07:27:48.469844+0000","last_undegraded":"2024-09-16T07:27:48.469844+0000","last_fullsized":"2024-09-16T07:27:48.469844+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:28.032448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449957+0000","last_change":"2024-09-16T07:27:47.449957+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449957+0000","last_undegraded":"2024-09-16T07:27:47.449957+0000","last_fullsized":"2024-09-16T07:27:47.449957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449887+0000","last_change":"2024-09-16T07:27:47.449887+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449887+0000","last_undegraded":"2024-09-16T07:27:47.449887+0000","last_fullsized":"2024-09-16T07:27:47.449887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467286+0000","last_change":"2024-09-16T07:27:48.465698+0000","last_active":"2024-09-16T07:27:48.467286+0000","last_peered":"2024-09-16T07:27:48.467286+0000","last_clean":"2024-09-16T07:27:48.467286+0000","last_became_active":"2024-09-16T07:27:48.465355+0000","last_became_peered":"2024-09-16T07:27:48.465355+0000","last_unstale":"2024-09-16T07:27:48.467286+0000","last_undegraded":"2024-09-16T07:27:48.467286+0000","last_fullsized":"2024-09-16T07:27:48.467286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:59.562068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.447615+0000","last_change":"2024-09-16T07:27:47.447615+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.447615+0000","last_undegraded":"2024-09-16T07:27:47.447615+0000","last_fullsized":"2024-09-16T07:27:47.447615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461494+0000","last_change":"2024-09-16T07:27:48.460038+0000","last_active":"2024-09-16T07:27:49.461494+0000","last_peered":"2024-09-16T07:27:49.461494+0000","last_clean":"2024-09-16T07:27:49.461494+0000","last_became_active":"2024-09-16T07:27:48.459755+0000","last_became_peered":"2024-09-16T07:27:48.459755+0000","last_unstale":"2024-09-16T07:27:49.461494+0000","last_undegraded":"2024-09-16T07:27:49.461494+0000","last_fullsized":"2024-09-16T07:27:49.461494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:37.822664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467196+0000","last_change":"2024-09-16T07:27:48.464651+0000","last_active":"2024-09-16T07:27:48.467196+0000","last_peered":"2024-09-16T07:27:48.467196+0000","last_clean":"2024-09-16T07:27:48.467196+0000","last_became_active":"2024-09-16T07:27:48.464373+0000","last_became_peered":"2024-09-16T07:27:48.464373+0000","last_unstale":"2024-09-16T07:27:48.467196+0000","last_undegraded":"2024-09-16T07:27:48.467196+0000","last_fullsized":"2024-09-16T07:27:48.467196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:34.767968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461319+0000","last_change":"2024-09-16T07:27:48.461614+0000","last_active":"2024-09-16T07:27:49.461319+0000","last_peered":"2024-09-16T07:27:49.461319+0000","last_clean":"2024-09-16T07:27:49.461319+0000","last_became_active":"2024-09-16T07:27:48.461333+0000","last_became_peered":"2024-09-16T07:27:48.461333+0000","last_unstale":"2024-09-16T07:27:49.461319+0000","last_undegraded":"2024-09-16T07:27:49.461319+0000","last_fullsized":"2024-09-16T07:27:49.461319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:14.430788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.477576+0000","last_change":"2024-09-16T07:27:48.469453+0000","last_active":"2024-09-16T07:27:48.477576+0000","last_peered":"2024-09-16T07:27:48.477576+0000","last_clean":"2024-09-16T07:27:48.477576+0000","last_became_active":"2024-09-16T07:27:48.466391+0000","last_became_peered":"2024-09-16T07:27:48.466391+0000","last_unstale":"2024-09-16T07:27:48.477576+0000","last_undegraded":"2024-09-16T07:27:48.477576+0000","last_fullsized":"2024-09-16T07:27:48.477576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:26.749995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468874+0000","last_change":"2024-09-16T07:27:48.461655+0000","last_active":"2024-09-16T07:27:48.468874+0000","last_peered":"2024-09-16T07:27:48.468874+0000","last_clean":"2024-09-16T07:27:48.468874+0000","last_became_active":"2024-09-16T07:27:48.461371+0000","last_became_peered":"2024-09-16T07:27:48.461371+0000","last_unstale":"2024-09-16T07:27:48.468874+0000","last_undegraded":"2024-09-16T07:27:48.468874+0000","last_fullsized":"2024-09-16T07:27:48.468874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:14.719547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444900+0000","last_change":"2024-09-16T07:27:47.444900+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444900+0000","last_undegraded":"2024-09-16T07:27:47.444900+0000","last_fullsized":"2024-09-16T07:27:47.444900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.459564+0000","last_change":"2024-09-16T07:27:48.455880+0000","last_active":"2024-09-16T07:27:48.459564+0000","last_peered":"2024-09-16T07:27:48.459564+0000","last_clean":"2024-09-16T07:27:48.459564+0000","last_became_active":"2024-09-16T07:27:48.455633+0000","last_became_peered":"2024-09-16T07:27:48.455633+0000","last_unstale":"2024-09-16T07:27:48.459564+0000","last_undegraded":"2024-09-16T07:27:48.459564+0000","last_fullsized":"2024-09-16T07:27:48.459564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:56.366154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.446950+0000","last_change":"2024-09-16T07:27:47.446950+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.446950+0000","last_undegraded":"2024-09-16T07:27:47.446950+0000","last_fullsized":"2024-09-16T07:27:47.446950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441948+0000","last_change":"2024-09-16T07:27:47.441948+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441948+0000","last_undegraded":"2024-09-16T07:27:47.441948+0000","last_fullsized":"2024-09-16T07:27:47.441948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.439145+0000","last_change":"2024-09-16T07:27:47.439145+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.439145+0000","last_undegraded":"2024-09-16T07:27:47.439145+0000","last_fullsized":"2024-09-16T07:27:47.439145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.472115+0000","last_change":"2024-09-16T07:27:48.462181+0000","last_active":"2024-09-16T07:27:48.472115+0000","last_peered":"2024-09-16T07:27:48.472115+0000","last_clean":"2024-09-16T07:27:48.472115+0000","last_became_active":"2024-09-16T07:27:48.460601+0000","last_became_peered":"2024-09-16T07:27:48.460601+0000","last_unstale":"2024-09-16T07:27:48.472115+0000","last_undegraded":"2024-09-16T07:27:48.472115+0000","last_fullsized":"2024-09-16T07:27:48.472115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:18.404077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.443479+0000","last_change":"2024-09-16T07:27:47.443479+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.443479+0000","last_undegraded":"2024-09-16T07:27:47.443479+0000","last_fullsized":"2024-09-16T07:27:47.443479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461161+0000","last_change":"2024-09-16T07:27:48.460512+0000","last_active":"2024-09-16T07:27:49.461161+0000","last_peered":"2024-09-16T07:27:49.461161+0000","last_clean":"2024-09-16T07:27:49.461161+0000","last_became_active":"2024-09-16T07:27:48.460276+0000","last_became_peered":"2024-09-16T07:27:48.460276+0000","last_unstale":"2024-09-16T07:27:49.461161+0000","last_undegraded":"2024-09-16T07:27:49.461161+0000","last_fullsized":"2024-09-16T07:27:49.461161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.223196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.446917+0000","last_change":"2024-09-16T07:27:47.446917+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.446917+0000","last_undegraded":"2024-09-16T07:27:47.446917+0000","last_fullsized":"2024-09-16T07:27:47.446917+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461617+0000","last_change":"2024-09-16T07:27:47.450308+0000","last_active":"2024-09-16T07:27:49.461617+0000","last_peered":"2024-09-16T07:27:49.461617+0000","last_clean":"2024-09-16T07:27:49.461617+0000","last_became_active":"2024-09-16T07:27:47.448332+0000","last_became_peered":"2024-09-16T07:27:47.448332+0000","last_unstale":"2024-09-16T07:27:49.461617+0000","last_undegraded":"2024-09-16T07:27:49.461617+0000","last_fullsized":"2024-09-16T07:27:49.461617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:38.412742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449454+0000","last_change":"2024-09-16T07:27:47.449454+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449454+0000","last_undegraded":"2024-09-16T07:27:47.449454+0000","last_fullsized":"2024-09-16T07:27:47.449454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.454190+0000","last_change":"2024-09-16T07:27:47.452837+0000","last_active":"2024-09-16T07:27:47.454190+0000","last_peered":"2024-09-16T07:27:47.454190+0000","last_clean":"2024-09-16T07:27:47.454190+0000","last_became_active":"2024-09-16T07:27:47.452517+0000","last_became_peered":"2024-09-16T07:27:47.452517+0000","last_unstale":"2024-09-16T07:27:47.454190+0000","last_undegraded":"2024-09-16T07:27:47.454190+0000","last_fullsized":"2024-09-16T07:27:47.454190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:50.738735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441487+0000","last_change":"2024-09-16T07:27:47.441487+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441487+0000","last_undegraded":"2024-09-16T07:27:47.441487+0000","last_fullsized":"2024-09-16T07:27:47.441487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460327+0000","last_change":"2024-09-16T07:27:47.459254+0000","last_active":"2024-09-16T07:27:47.460327+0000","last_peered":"2024-09-16T07:27:47.460327+0000","last_clean":"2024-09-16T07:27:47.460327+0000","last_became_active":"2024-09-16T07:27:47.458912+0000","last_became_peered":"2024-09-16T07:27:47.458912+0000","last_unstale":"2024-09-16T07:27:47.460327+0000","last_undegraded":"2024-09-16T07:27:47.460327+0000","last_fullsized":"2024-09-16T07:27:47.460327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:07:13.520726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444836+0000","last_change":"2024-09-16T07:27:47.444836+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444836+0000","last_undegraded":"2024-09-16T07:27:47.444836+0000","last_fullsized":"2024-09-16T07:27:47.444836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.478412+0000","last_change":"2024-09-16T07:27:47.458877+0000","last_active":"2024-09-16T07:27:48.478412+0000","last_peered":"2024-09-16T07:27:48.478412+0000","last_clean":"2024-09-16T07:27:48.478412+0000","last_became_active":"2024-09-16T07:27:47.458615+0000","last_became_peered":"2024-09-16T07:27:47.458615+0000","last_unstale":"2024-09-16T07:27:48.478412+0000","last_undegraded":"2024-09-16T07:27:48.478412+0000","last_fullsized":"2024-09-16T07:27:48.478412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:00.779149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438546+0000","last_change":"2024-09-16T07:27:47.438546+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438546+0000","last_undegraded":"2024-09-16T07:27:47.438546+0000","last_fullsized":"2024-09-16T07:27:47.438546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.471392+0000","last_change":"2024-09-16T07:27:47.455983+0000","last_active":"2024-09-16T07:27:48.471392+0000","last_peered":"2024-09-16T07:27:48.471392+0000","last_clean":"2024-09-16T07:27:48.471392+0000","last_became_active":"2024-09-16T07:27:47.454631+0000","last_became_peered":"2024-09-16T07:27:47.454631+0000","last_unstale":"2024-09-16T07:27:48.471392+0000","last_undegraded":"2024-09-16T07:27:48.471392+0000","last_fullsized":"2024-09-16T07:27:48.471392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:37.670315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461455+0000","last_change":"2024-09-16T07:27:48.457180+0000","last_active":"2024-09-16T07:27:49.461455+0000","last_peered":"2024-09-16T07:27:49.461455+0000","last_clean":"2024-09-16T07:27:49.461455+0000","last_became_active":"2024-09-16T07:27:48.456834+0000","last_became_peered":"2024-09-16T07:27:48.456834+0000","last_unstale":"2024-09-16T07:27:49.461455+0000","last_undegraded":"2024-09-16T07:27:49.461455+0000","last_fullsized":"2024-09-16T07:27:49.461455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:04.346053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464365+0000","last_change":"2024-09-16T07:27:47.461641+0000","last_active":"2024-09-16T07:27:47.464365+0000","last_peered":"2024-09-16T07:27:47.464365+0000","last_clean":"2024-09-16T07:27:47.464365+0000","last_became_active":"2024-09-16T07:27:47.461318+0000","last_became_peered":"2024-09-16T07:27:47.461318+0000","last_unstale":"2024-09-16T07:27:47.464365+0000","last_undegraded":"2024-09-16T07:27:47.464365+0000","last_fullsized":"2024-09-16T07:27:47.464365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:10:51.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.465633+0000","last_change":"2024-09-16T07:27:48.455278+0000","last_active":"2024-09-16T07:27:48.465633+0000","last_peered":"2024-09-16T07:27:48.465633+0000","last_clean":"2024-09-16T07:27:48.465633+0000","last_became_active":"2024-09-16T07:27:48.454871+0000","last_became_peered":"2024-09-16T07:27:48.454871+0000","last_unstale":"2024-09-16T07:27:48.465633+0000","last_undegraded":"2024-09-16T07:27:48.465633+0000","last_fullsized":"2024-09-16T07:27:48.465633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:42.196208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443144+0000","last_change":"2024-09-16T07:27:47.452265+0000","last_active":"2024-09-16T07:27:48.443144+0000","last_peered":"2024-09-16T07:27:48.443144+0000","last_clean":"2024-09-16T07:27:48.443144+0000","last_became_active":"2024-09-16T07:27:47.452018+0000","last_became_peered":"2024-09-16T07:27:47.452018+0000","last_unstale":"2024-09-16T07:27:48.443144+0000","last_undegraded":"2024-09-16T07:27:48.443144+0000","last_fullsized":"2024-09-16T07:27:48.443144+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:35.000221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.448499+0000","last_change":"2024-09-16T07:27:47.448499+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.448499+0000","last_undegraded":"2024-09-16T07:27:47.448499+0000","last_fullsized":"2024-09-16T07:27:47.448499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.449811+0000","last_change":"2024-09-16T07:27:47.447783+0000","last_active":"2024-09-16T07:27:48.449811+0000","last_peered":"2024-09-16T07:27:48.449811+0000","last_clean":"2024-09-16T07:27:48.449811+0000","last_became_active":"2024-09-16T07:27:47.447510+0000","last_became_peered":"2024-09-16T07:27:47.447510+0000","last_unstale":"2024-09-16T07:27:48.449811+0000","last_undegraded":"2024-09-16T07:27:48.449811+0000","last_fullsized":"2024-09-16T07:27:48.449811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:14.370496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.459535+0000","last_change":"2024-09-16T07:27:48.454744+0000","last_active":"2024-09-16T07:27:48.459535+0000","last_peered":"2024-09-16T07:27:48.459535+0000","last_clean":"2024-09-16T07:27:48.459535+0000","last_became_active":"2024-09-16T07:27:48.454446+0000","last_became_peered":"2024-09-16T07:27:48.454446+0000","last_unstale":"2024-09-16T07:27:48.459535+0000","last_undegraded":"2024-09-16T07:27:48.459535+0000","last_fullsized":"2024-09-16T07:27:48.459535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:52.733528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443637+0000","last_change":"2024-09-16T07:27:47.450306+0000","last_active":"2024-09-16T07:27:48.443637+0000","last_peered":"2024-09-16T07:27:48.443637+0000","last_clean":"2024-09-16T07:27:48.443637+0000","last_became_active":"2024-09-16T07:27:47.449275+0000","last_became_peered":"2024-09-16T07:27:47.449275+0000","last_unstale":"2024-09-16T07:27:48.443637+0000","last_undegraded":"2024-09-16T07:27:48.443637+0000","last_fullsized":"2024-09-16T07:27:48.443637+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:00.479157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.470005+0000","last_change":"2024-09-16T07:27:48.468331+0000","last_active":"2024-09-16T07:27:48.470005+0000","last_peered":"2024-09-16T07:27:48.470005+0000","last_clean":"2024-09-16T07:27:48.470005+0000","last_became_active":"2024-09-16T07:27:48.468040+0000","last_became_peered":"2024-09-16T07:27:48.468040+0000","last_unstale":"2024-09-16T07:27:48.470005+0000","last_undegraded":"2024-09-16T07:27:48.470005+0000","last_fullsized":"2024-09-16T07:27:48.470005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:06.084517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453197+0000","last_change":"2024-09-16T07:27:47.450498+0000","last_active":"2024-09-16T07:27:47.453197+0000","last_peered":"2024-09-16T07:27:47.453197+0000","last_clean":"2024-09-16T07:27:47.453197+0000","last_became_active":"2024-09-16T07:27:47.449615+0000","last_became_peered":"2024-09-16T07:27:47.449615+0000","last_unstale":"2024-09-16T07:27:47.453197+0000","last_undegraded":"2024-09-16T07:27:47.453197+0000","last_fullsized":"2024-09-16T07:27:47.453197+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:23:34.664293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438573+0000","last_change":"2024-09-16T07:27:47.438573+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438573+0000","last_undegraded":"2024-09-16T07:27:47.438573+0000","last_fullsized":"2024-09-16T07:27:47.438573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.478154+0000","last_change":"2024-09-16T07:27:47.460337+0000","last_active":"2024-09-16T07:27:48.478154+0000","last_peered":"2024-09-16T07:27:48.478154+0000","last_clean":"2024-09-16T07:27:48.478154+0000","last_became_active":"2024-09-16T07:27:47.460023+0000","last_became_peered":"2024-09-16T07:27:47.460023+0000","last_unstale":"2024-09-16T07:27:48.478154+0000","last_undegraded":"2024-09-16T07:27:48.478154+0000","last_fullsized":"2024-09-16T07:27:48.478154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:52.763400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.456699+0000","last_change":"2024-09-16T07:27:48.449778+0000","last_active":"2024-09-16T07:27:48.456699+0000","last_peered":"2024-09-16T07:27:48.456699+0000","last_clean":"2024-09-16T07:27:48.456699+0000","last_became_active":"2024-09-16T07:27:48.449504+0000","last_became_peered":"2024-09-16T07:27:48.449504+0000","last_unstale":"2024-09-16T07:27:48.456699+0000","last_undegraded":"2024-09-16T07:27:48.456699+0000","last_fullsized":"2024-09-16T07:27:48.456699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:48.419517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.480574+0000","last_change":"2024-09-16T07:27:47.458310+0000","last_active":"2024-09-16T07:27:48.480574+0000","last_peered":"2024-09-16T07:27:48.480574+0000","last_clean":"2024-09-16T07:27:48.480574+0000","last_became_active":"2024-09-16T07:27:47.457920+0000","last_became_peered":"2024-09-16T07:27:47.457920+0000","last_unstale":"2024-09-16T07:27:48.480574+0000","last_undegraded":"2024-09-16T07:27:48.480574+0000","last_fullsized":"2024-09-16T07:27:48.480574+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.526953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469560+0000","last_change":"2024-09-16T07:27:48.467738+0000","last_active":"2024-09-16T07:27:48.469560+0000","last_peered":"2024-09-16T07:27:48.469560+0000","last_clean":"2024-09-16T07:27:48.469560+0000","last_became_active":"2024-09-16T07:27:48.467296+0000","last_became_peered":"2024-09-16T07:27:48.467296+0000","last_unstale":"2024-09-16T07:27:48.469560+0000","last_undegraded":"2024-09-16T07:27:48.469560+0000","last_fullsized":"2024-09-16T07:27:48.469560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:08.840299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453240+0000","last_change":"2024-09-16T07:27:47.449886+0000","last_active":"2024-09-16T07:27:47.453240+0000","last_peered":"2024-09-16T07:27:47.453240+0000","last_clean":"2024-09-16T07:27:47.453240+0000","last_became_active":"2024-09-16T07:27:47.449599+0000","last_became_peered":"2024-09-16T07:27:47.449599+0000","last_unstale":"2024-09-16T07:27:47.453240+0000","last_undegraded":"2024-09-16T07:27:47.453240+0000","last_fullsized":"2024-09-16T07:27:47.453240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:45:32.473190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466091+0000","last_change":"2024-09-16T07:27:48.453257+0000","last_active":"2024-09-16T07:27:48.466091+0000","last_peered":"2024-09-16T07:27:48.466091+0000","last_clean":"2024-09-16T07:27:48.466091+0000","last_became_active":"2024-09-16T07:27:48.453020+0000","last_became_peered":"2024-09-16T07:27:48.453020+0000","last_unstale":"2024-09-16T07:27:48.466091+0000","last_undegraded":"2024-09-16T07:27:48.466091+0000","last_fullsized":"2024-09-16T07:27:48.466091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:07.527458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.439529+0000","last_change":"2024-09-16T07:27:47.449870+0000","last_active":"2024-09-16T07:27:48.439529+0000","last_peered":"2024-09-16T07:27:48.439529+0000","last_clean":"2024-09-16T07:27:48.439529+0000","last_became_active":"2024-09-16T07:27:47.449587+0000","last_became_peered":"2024-09-16T07:27:47.449587+0000","last_unstale":"2024-09-16T07:27:48.439529+0000","last_undegraded":"2024-09-16T07:27:48.439529+0000","last_fullsized":"2024-09-16T07:27:48.439529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:56.658575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468654+0000","last_change":"2024-09-16T07:27:48.457526+0000","last_active":"2024-09-16T07:27:48.468654+0000","last_peered":"2024-09-16T07:27:48.468654+0000","last_clean":"2024-09-16T07:27:48.468654+0000","last_became_active":"2024-09-16T07:27:48.457292+0000","last_became_peered":"2024-09-16T07:27:48.457292+0000","last_unstale":"2024-09-16T07:27:48.468654+0000","last_undegraded":"2024-09-16T07:27:48.468654+0000","last_fullsized":"2024-09-16T07:27:48.468654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:18.042402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.439349+0000","last_change":"2024-09-16T07:27:47.450647+0000","last_active":"2024-09-16T07:27:48.439349+0000","last_peered":"2024-09-16T07:27:48.439349+0000","last_clean":"2024-09-16T07:27:48.439349+0000","last_became_active":"2024-09-16T07:27:47.450313+0000","last_became_peered":"2024-09-16T07:27:47.450313+0000","last_unstale":"2024-09-16T07:27:48.439349+0000","last_undegraded":"2024-09-16T07:27:48.439349+0000","last_fullsized":"2024-09-16T07:27:48.439349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:01.115128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467730+0000","last_change":"2024-09-16T07:27:48.459503+0000","last_active":"2024-09-16T07:27:48.467730+0000","last_peered":"2024-09-16T07:27:48.467730+0000","last_clean":"2024-09-16T07:27:48.467730+0000","last_became_active":"2024-09-16T07:27:48.459201+0000","last_became_peered":"2024-09-16T07:27:48.459201+0000","last_unstale":"2024-09-16T07:27:48.467730+0000","last_undegraded":"2024-09-16T07:27:48.467730+0000","last_fullsized":"2024-09-16T07:27:48.467730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:35.394756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.482800+0000","last_change":"2024-09-16T07:27:47.451210+0000","last_active":"2024-09-16T07:27:48.482800+0000","last_peered":"2024-09-16T07:27:48.482800+0000","last_clean":"2024-09-16T07:27:48.482800+0000","last_became_active":"2024-09-16T07:27:47.450925+0000","last_became_peered":"2024-09-16T07:27:47.450925+0000","last_unstale":"2024-09-16T07:27:48.482800+0000","last_undegraded":"2024-09-16T07:27:48.482800+0000","last_fullsized":"2024-09-16T07:27:48.482800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:03.229023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.464941+0000","last_change":"2024-09-16T07:27:48.457648+0000","last_active":"2024-09-16T07:27:48.464941+0000","last_peered":"2024-09-16T07:27:48.464941+0000","last_clean":"2024-09-16T07:27:48.464941+0000","last_became_active":"2024-09-16T07:27:48.457358+0000","last_became_peered":"2024-09-16T07:27:48.457358+0000","last_unstale":"2024-09-16T07:27:48.464941+0000","last_undegraded":"2024-09-16T07:27:48.464941+0000","last_fullsized":"2024-09-16T07:27:48.464941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:28.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460425+0000","last_change":"2024-09-16T07:27:47.458791+0000","last_active":"2024-09-16T07:27:47.460425+0000","last_peered":"2024-09-16T07:27:47.460425+0000","last_clean":"2024-09-16T07:27:47.460425+0000","last_became_active":"2024-09-16T07:27:47.458560+0000","last_became_peered":"2024-09-16T07:27:47.458560+0000","last_unstale":"2024-09-16T07:27:47.460425+0000","last_undegraded":"2024-09-16T07:27:47.460425+0000","last_fullsized":"2024-09-16T07:27:47.460425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:15.540429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468200+0000","last_change":"2024-09-16T07:27:48.460172+0000","last_active":"2024-09-16T07:27:48.468200+0000","last_peered":"2024-09-16T07:27:48.468200+0000","last_clean":"2024-09-16T07:27:48.468200+0000","last_became_active":"2024-09-16T07:27:48.459930+0000","last_became_peered":"2024-09-16T07:27:48.459930+0000","last_unstale":"2024-09-16T07:27:48.468200+0000","last_undegraded":"2024-09-16T07:27:48.468200+0000","last_fullsized":"2024-09-16T07:27:48.468200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:04.739035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453174+0000","last_change":"2024-09-16T07:27:47.450135+0000","last_active":"2024-09-16T07:27:47.453174+0000","last_peered":"2024-09-16T07:27:47.453174+0000","last_clean":"2024-09-16T07:27:47.453174+0000","last_became_active":"2024-09-16T07:27:47.449254+0000","last_became_peered":"2024-09-16T07:27:47.449254+0000","last_unstale":"2024-09-16T07:27:47.453174+0000","last_undegraded":"2024-09-16T07:27:47.453174+0000","last_fullsized":"2024-09-16T07:27:47.453174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:16.114777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444472+0000","last_change":"2024-09-16T07:27:47.444472+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444472+0000","last_undegraded":"2024-09-16T07:27:47.444472+0000","last_fullsized":"2024-09-16T07:27:47.444472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.476242+0000","last_change":"2024-09-16T07:27:47.455984+0000","last_active":"2024-09-16T07:27:48.476242+0000","last_peered":"2024-09-16T07:27:48.476242+0000","last_clean":"2024-09-16T07:27:48.476242+0000","last_became_active":"2024-09-16T07:27:47.454791+0000","last_became_peered":"2024-09-16T07:27:47.454791+0000","last_unstale":"2024-09-16T07:27:48.476242+0000","last_undegraded":"2024-09-16T07:27:48.476242+0000","last_fullsized":"2024-09-16T07:27:48.476242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:24.745832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.472184+0000","last_change":"2024-09-16T07:27:48.462451+0000","last_active":"2024-09-16T07:27:48.472184+0000","last_peered":"2024-09-16T07:27:48.472184+0000","last_clean":"2024-09-16T07:27:48.472184+0000","last_became_active":"2024-09-16T07:27:48.462145+0000","last_became_peered":"2024-09-16T07:27:48.462145+0000","last_unstale":"2024-09-16T07:27:48.472184+0000","last_undegraded":"2024-09-16T07:27:48.472184+0000","last_fullsized":"2024-09-16T07:27:48.472184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:25:02.985426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460913+0000","last_change":"2024-09-16T07:27:47.459943+0000","last_active":"2024-09-16T07:27:47.460913+0000","last_peered":"2024-09-16T07:27:47.460913+0000","last_clean":"2024-09-16T07:27:47.460913+0000","last_became_active":"2024-09-16T07:27:47.459680+0000","last_became_peered":"2024-09-16T07:27:47.459680+0000","last_unstale":"2024-09-16T07:27:47.460913+0000","last_undegraded":"2024-09-16T07:27:47.460913+0000","last_fullsized":"2024-09-16T07:27:47.460913+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:44.061784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468987+0000","last_change":"2024-09-16T07:27:48.461989+0000","last_active":"2024-09-16T07:27:48.468987+0000","last_peered":"2024-09-16T07:27:48.468987+0000","last_clean":"2024-09-16T07:27:48.468987+0000","last_became_active":"2024-09-16T07:27:48.461731+0000","last_became_peered":"2024-09-16T07:27:48.461731+0000","last_unstale":"2024-09-16T07:27:48.468987+0000","last_undegraded":"2024-09-16T07:27:48.468987+0000","last_fullsized":"2024-09-16T07:27:48.468987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:43.744251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461382+0000","last_change":"2024-09-16T07:27:47.452023+0000","last_active":"2024-09-16T07:27:49.461382+0000","last_peered":"2024-09-16T07:27:49.461382+0000","last_clean":"2024-09-16T07:27:49.461382+0000","last_became_active":"2024-09-16T07:27:47.451738+0000","last_became_peered":"2024-09-16T07:27:47.451738+0000","last_unstale":"2024-09-16T07:27:49.461382+0000","last_undegraded":"2024-09-16T07:27:49.461382+0000","last_fullsized":"2024-09-16T07:27:49.461382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:28.412285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438666+0000","last_change":"2024-09-16T07:27:47.438666+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438666+0000","last_undegraded":"2024-09-16T07:27:47.438666+0000","last_fullsized":"2024-09-16T07:27:47.438666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.457752+0000","last_change":"2024-09-16T07:27:47.452663+0000","last_active":"2024-09-16T07:27:47.457752+0000","last_peered":"2024-09-16T07:27:47.457752+0000","last_clean":"2024-09-16T07:27:47.457752+0000","last_became_active":"2024-09-16T07:27:47.451570+0000","last_became_peered":"2024-09-16T07:27:47.451570+0000","last_unstale":"2024-09-16T07:27:47.457752+0000","last_undegraded":"2024-09-16T07:27:47.457752+0000","last_fullsized":"2024-09-16T07:27:47.457752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:14.561916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.439442+0000","last_change":"2024-09-16T07:27:47.439442+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.439442+0000","last_undegraded":"2024-09-16T07:27:47.439442+0000","last_fullsized":"2024-09-16T07:27:47.439442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443403+0000","last_change":"2024-09-16T07:27:47.455318+0000","last_active":"2024-09-16T07:27:48.443403+0000","last_peered":"2024-09-16T07:27:48.443403+0000","last_clean":"2024-09-16T07:27:48.443403+0000","last_became_active":"2024-09-16T07:27:47.455047+0000","last_became_peered":"2024-09-16T07:27:47.455047+0000","last_unstale":"2024-09-16T07:27:48.443403+0000","last_undegraded":"2024-09-16T07:27:48.443403+0000","last_fullsized":"2024-09-16T07:27:48.443403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:08.028003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466860+0000","last_change":"2024-09-16T07:27:48.462125+0000","last_active":"2024-09-16T07:27:48.466860+0000","last_peered":"2024-09-16T07:27:48.466860+0000","last_clean":"2024-09-16T07:27:48.466860+0000","last_became_active":"2024-09-16T07:27:48.461779+0000","last_became_peered":"2024-09-16T07:27:48.461779+0000","last_unstale":"2024-09-16T07:27:48.466860+0000","last_undegraded":"2024-09-16T07:27:48.466860+0000","last_fullsized":"2024-09-16T07:27:48.466860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:37.463436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464161+0000","last_change":"2024-09-16T07:27:47.461596+0000","last_active":"2024-09-16T07:27:47.464161+0000","last_peered":"2024-09-16T07:27:47.464161+0000","last_clean":"2024-09-16T07:27:47.464161+0000","last_became_active":"2024-09-16T07:27:47.461292+0000","last_became_peered":"2024-09-16T07:27:47.461292+0000","last_unstale":"2024-09-16T07:27:47.464161+0000","last_undegraded":"2024-09-16T07:27:47.464161+0000","last_fullsized":"2024-09-16T07:27:47.464161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:22.241647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.463013+0000","last_change":"2024-09-16T07:27:48.452873+0000","last_active":"2024-09-16T07:27:48.463013+0000","last_peered":"2024-09-16T07:27:48.463013+0000","last_clean":"2024-09-16T07:27:48.463013+0000","last_became_active":"2024-09-16T07:27:48.450871+0000","last_became_peered":"2024-09-16T07:27:48.450871+0000","last_unstale":"2024-09-16T07:27:48.463013+0000","last_undegraded":"2024-09-16T07:27:48.463013+0000","last_fullsized":"2024-09-16T07:27:48.463013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:38:38.451975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464346+0000","last_change":"2024-09-16T07:27:47.461620+0000","last_active":"2024-09-16T07:27:47.464346+0000","last_peered":"2024-09-16T07:27:47.464346+0000","last_clean":"2024-09-16T07:27:47.464346+0000","last_became_active":"2024-09-16T07:27:47.461338+0000","last_became_peered":"2024-09-16T07:27:47.461338+0000","last_unstale":"2024-09-16T07:27:47.464346+0000","last_undegraded":"2024-09-16T07:27:47.464346+0000","last_fullsized":"2024-09-16T07:27:47.464346+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:28.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444027+0000","last_change":"2024-09-16T07:27:47.444027+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444027+0000","last_undegraded":"2024-09-16T07:27:47.444027+0000","last_fullsized":"2024-09-16T07:27:47.444027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443387+0000","last_change":"2024-09-16T07:27:47.458964+0000","last_active":"2024-09-16T07:27:48.443387+0000","last_peered":"2024-09-16T07:27:48.443387+0000","last_clean":"2024-09-16T07:27:48.443387+0000","last_became_active":"2024-09-16T07:27:47.458687+0000","last_became_peered":"2024-09-16T07:27:47.458687+0000","last_unstale":"2024-09-16T07:27:48.443387+0000","last_undegraded":"2024-09-16T07:27:48.443387+0000","last_fullsized":"2024-09-16T07:27:48.443387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:43.227646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438895+0000","last_change":"2024-09-16T07:27:47.438895+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438895+0000","last_undegraded":"2024-09-16T07:27:47.438895+0000","last_fullsized":"2024-09-16T07:27:47.438895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.448905+0000","last_change":"2024-09-16T07:27:47.456743+0000","last_active":"2024-09-16T07:27:48.448905+0000","last_peered":"2024-09-16T07:27:48.448905+0000","last_clean":"2024-09-16T07:27:48.448905+0000","last_became_active":"2024-09-16T07:27:47.455261+0000","last_became_peered":"2024-09-16T07:27:47.455261+0000","last_unstale":"2024-09-16T07:27:48.448905+0000","last_undegraded":"2024-09-16T07:27:48.448905+0000","last_fullsized":"2024-09-16T07:27:48.448905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:44.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444420+0000","last_change":"2024-09-16T07:27:47.444420+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444420+0000","last_undegraded":"2024-09-16T07:27:47.444420+0000","last_fullsized":"2024-09-16T07:27:47.444420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.459115+0000","last_change":"2024-09-16T07:27:47.458363+0000","last_active":"2024-09-16T07:27:47.459115+0000","last_peered":"2024-09-16T07:27:47.459115+0000","last_clean":"2024-09-16T07:27:47.459115+0000","last_became_active":"2024-09-16T07:27:47.455595+0000","last_became_peered":"2024-09-16T07:27:47.455595+0000","last_unstale":"2024-09-16T07:27:47.459115+0000","last_undegraded":"2024-09-16T07:27:47.459115+0000","last_fullsized":"2024-09-16T07:27:47.459115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:18.944163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.465977+0000","last_change":"2024-09-16T07:27:48.460471+0000","last_active":"2024-09-16T07:27:48.465977+0000","last_peered":"2024-09-16T07:27:48.465977+0000","last_clean":"2024-09-16T07:27:48.465977+0000","last_became_active":"2024-09-16T07:27:48.456813+0000","last_became_peered":"2024-09-16T07:27:48.456813+0000","last_unstale":"2024-09-16T07:27:48.465977+0000","last_undegraded":"2024-09-16T07:27:48.465977+0000","last_fullsized":"2024-09-16T07:27:48.465977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:45.085619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.449173+0000","last_change":"2024-09-16T07:27:47.452638+0000","last_active":"2024-09-16T07:27:48.449173+0000","last_peered":"2024-09-16T07:27:48.449173+0000","last_clean":"2024-09-16T07:27:48.449173+0000","last_became_active":"2024-09-16T07:27:47.452390+0000","last_became_peered":"2024-09-16T07:27:47.452390+0000","last_unstale":"2024-09-16T07:27:48.449173+0000","last_undegraded":"2024-09-16T07:27:48.449173+0000","last_fullsized":"2024-09-16T07:27:48.449173+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:04.058738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.440856+0000","last_change":"2024-09-16T07:27:47.440856+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.440856+0000","last_undegraded":"2024-09-16T07:27:47.440856+0000","last_fullsized":"2024-09-16T07:27:47.440856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.462516+0000","last_change":"2024-09-16T07:27:47.460442+0000","last_active":"2024-09-16T07:27:47.462516+0000","last_peered":"2024-09-16T07:27:47.462516+0000","last_clean":"2024-09-16T07:27:47.462516+0000","last_became_active":"2024-09-16T07:27:47.459037+0000","last_became_peered":"2024-09-16T07:27:47.459037+0000","last_unstale":"2024-09-16T07:27:47.462516+0000","last_undegraded":"2024-09-16T07:27:47.462516+0000","last_fullsized":"2024-09-16T07:27:47.462516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:25:58.753488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.442957+0000","last_change":"2024-09-16T07:27:47.442957+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.442957+0000","last_undegraded":"2024-09-16T07:27:47.442957+0000","last_fullsized":"2024-09-16T07:27:47.442957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460678+0000","last_change":"2024-09-16T07:27:47.453007+0000","last_active":"2024-09-16T07:27:47.460678+0000","last_peered":"2024-09-16T07:27:47.460678+0000","last_clean":"2024-09-16T07:27:47.460678+0000","last_became_active":"2024-09-16T07:27:47.452735+0000","last_became_peered":"2024-09-16T07:27:47.452735+0000","last_unstale":"2024-09-16T07:27:47.460678+0000","last_undegraded":"2024-09-16T07:27:47.460678+0000","last_fullsized":"2024-09-16T07:27:47.460678+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:47:52.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.440008+0000","last_change":"2024-09-16T07:27:47.440008+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.440008+0000","last_undegraded":"2024-09-16T07:27:47.440008+0000","last_fullsized":"2024-09-16T07:27:47.440008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.450452+0000","last_change":"2024-09-16T07:27:47.454175+0000","last_active":"2024-09-16T07:27:48.450452+0000","last_peered":"2024-09-16T07:27:48.450452+0000","last_clean":"2024-09-16T07:27:48.450452+0000","last_became_active":"2024-09-16T07:27:47.453769+0000","last_became_peered":"2024-09-16T07:27:47.453769+0000","last_unstale":"2024-09-16T07:27:48.450452+0000","last_undegraded":"2024-09-16T07:27:48.450452+0000","last_fullsized":"2024-09-16T07:27:48.450452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:17.667822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443464+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:27:48.443464+0000","last_peered":"2024-09-16T07:27:48.443464+0000","last_clean":"2024-09-16T07:27:48.443464+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:27:48.443464+0000","last_undegraded":"2024-09-16T07:27:48.443464+0000","last_fullsized":"2024-09-16T07:27:48.443464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476803,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":295094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476803,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62448,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309392,"statfs":{"total":96636764160,"available":96572817408,"internally_reserved":0,"allocated":741376,"data_stored":303641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476803,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40784,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331056,"statfs":{"total":96636764160,"available":96595001344,"internally_reserved":0,"allocated":1302528,"data_stored":885462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476802,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":295094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":288819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509506,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":288819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46352,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325488,"statfs":{"total":96636764160,"available":96589299712,"internally_reserved":0,"allocated":1302528,"data_stored":885462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49960,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321880,"statfs":{"total":96636764160,"available":96585605120,"internally_reserved":0,"allocated":741376,"data_stored":308785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:27:53.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:27:53.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.253+0000 7fb61b200640 1 -- 172.21.15.71:0/3018579048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0725d0 msgr2=0x7fb61c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.253+0000 7fb61b200640 1 --2- 172.21.15.71:0/3018579048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0725d0 0x7fb61c0b28f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb61c074a80 tx=0x7fb60802a560 comp rx=0 tx=0).stop 2024-09-16T07:27:53.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.254+0000 7fb61b200640 1 -- 172.21.15.71:0/3018579048 shutdown_connections 2024-09-16T07:27:53.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.254+0000 7fb61b200640 1 --2- 172.21.15.71:0/3018579048 >> v2:172.21.15.177:3300/0 conn(0x7fb61c0d0540 0x7fb61c06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.254+0000 7fb61b200640 1 --2- 172.21.15.71:0/3018579048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0725d0 0x7fb61c0b28f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.254+0000 7fb61b200640 1 -- 172.21.15.71:0/3018579048 >> 172.21.15.71:0/3018579048 conn(0x7fb61c0cfa20 msgr2=0x7fb61c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:53.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.254+0000 7fb61b200640 1 -- 172.21.15.71:0/3018579048 shutdown_connections 2024-09-16T07:27:53.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.254+0000 7fb61b200640 1 -- 172.21.15.71:0/3018579048 wait complete. 2024-09-16T07:27:53.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.255+0000 7fb61b200640 1 Processor -- start 2024-09-16T07:27:53.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.255+0000 7fb61b200640 1 -- start start 2024-09-16T07:27:53.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.256+0000 7fb61b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0d0540 0x7fb61c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:53.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.256+0000 7fb61b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb61c15bc30 0x7fb61c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:53.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.256+0000 7fb61b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb61c157f00 0x7fb61c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:53.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.256+0000 7fb61b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c074660 con 0x7fb61c0d0540 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.256+0000 7fb61b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c06f860 con 0x7fb61c15bc30 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.256+0000 7fb61b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb61c1f4d70 con 0x7fb61c157f00 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.256+0000 7fb619e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0d0540 0x7fb61c0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0d0540 0x7fb61c0c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48454/0 (socket says 172.21.15.71:48454) 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619e00640 1 -- 172.21.15.71:0/1816726967 learned_addr learned my addr 172.21.15.71:0/1816726967 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619400640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb61c15bc30 0x7fb61c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb61a800640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb61c157f00 0x7fb61c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619e00640 1 -- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb61c157f00 msgr2=0x7fb61c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619e00640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb61c157f00 0x7fb61c0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619e00640 1 -- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb61c15bc30 msgr2=0x7fb61c1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619e00640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb61c15bc30 0x7fb61c1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619e00640 1 -- 172.21.15.71:0/1816726967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c1f5010 con 0x7fb61c0d0540 2024-09-16T07:27:53.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb619e00640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0d0540 0x7fb61c0c9320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00b3e0 tx=0x7fb60c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:53.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb612000640 1 -- 172.21.15.71:0/1816726967 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c015020 con 0x7fb61c0d0540 2024-09-16T07:27:53.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb612000640 1 -- 172.21.15.71:0/1816726967 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb60c011940 con 0x7fb61c0d0540 2024-09-16T07:27:53.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61c1f52f0 con 0x7fb61c0d0540 2024-09-16T07:27:53.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.257+0000 7fb612000640 1 -- 172.21.15.71:0/1816726967 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c011d90 con 0x7fb61c0d0540 2024-09-16T07:27:53.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.258+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61c1f58b0 con 0x7fb61c0d0540 2024-09-16T07:27:53.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.259+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb61c0c98c0 con 0x7fb61c0d0540 2024-09-16T07:27:53.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.260+0000 7fb612000640 1 -- 172.21.15.71:0/1816726967 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fb60c018020 con 0x7fb61c0d0540 2024-09-16T07:27:53.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.261+0000 7fb612000640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5fc0790c0 0x7fb5fc07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:53.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.261+0000 7fb612000640 1 -- 172.21.15.71:0/1816726967 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fb60c09b8c0 con 0x7fb61c0d0540 2024-09-16T07:27:53.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.261+0000 7fb619400640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5fc0790c0 0x7fb5fc07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:53.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.262+0000 7fb619400640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5fc0790c0 0x7fb5fc07b580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb608033000 tx=0x7fb608004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:53.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.265+0000 7fb612000640 1 -- 172.21.15.71:0/1816726967 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb60c01f030 con 0x7fb61c0d0540 2024-09-16T07:27:53.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.447+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb61c1f5af0 con 0x7fb5fc0790c0 2024-09-16T07:27:53.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.452+0000 7fb612000640 1 -- 172.21.15.71:0/1816726967 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249482 (secure 0 0 0) 0x7fb61c1f5af0 con 0x7fb5fc0790c0 2024-09-16T07:27:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:53.457 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:27:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5fc0790c0 msgr2=0x7fb5fc07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5fc0790c0 0x7fb5fc07b580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb608033000 tx=0x7fb608004730 comp rx=0 tx=0).stop 2024-09-16T07:27:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0d0540 msgr2=0x7fb61c0c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0d0540 0x7fb61c0c9320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00b3e0 tx=0x7fb60c00b8a0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 shutdown_connections 2024-09-16T07:27:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5fc0790c0 0x7fb5fc07b580 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb61c157f00 0x7fb61c0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb61c15bc30 0x7fb61c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 --2- 172.21.15.71:0/1816726967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb61c0d0540 0x7fb61c0c9320 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 >> 172.21.15.71:0/1816726967 conn(0x7fb61c0cfa20 msgr2=0x7fb61c0ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 shutdown_connections 2024-09-16T07:27:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.456+0000 7fb61b200640 1 -- 172.21.15.71:0/1816726967 wait complete. 2024-09-16T07:27:53.472 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-16T07:27:52.380524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":132,"num_write_kb":1634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":360328,"kb_used_data":6736,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754614392,"statfs":{"total":773094113280,"available":772725137408,"internally_reserved":0,"allocated":6897664,"data_stored":3551176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":361877040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.951511"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.453401+0000","last_change":"2024-09-16T07:27:47.453401+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.453401+0000","last_undegraded":"2024-09-16T07:27:47.453401+0000","last_fullsized":"2024-09-16T07:27:47.453401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444099+0000","last_change":"2024-09-16T07:27:47.444099+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444099+0000","last_undegraded":"2024-09-16T07:27:47.444099+0000","last_fullsized":"2024-09-16T07:27:47.444099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.463111+0000","last_change":"2024-09-16T07:27:48.453492+0000","last_active":"2024-09-16T07:27:48.463111+0000","last_peered":"2024-09-16T07:27:48.463111+0000","last_clean":"2024-09-16T07:27:48.463111+0000","last_became_active":"2024-09-16T07:27:48.453262+0000","last_became_peered":"2024-09-16T07:27:48.453262+0000","last_unstale":"2024-09-16T07:27:48.463111+0000","last_undegraded":"2024-09-16T07:27:48.463111+0000","last_fullsized":"2024-09-16T07:27:48.463111+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:50.447717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469342+0000","last_change":"2024-09-16T07:27:48.460353+0000","last_active":"2024-09-16T07:27:48.469342+0000","last_peered":"2024-09-16T07:27:48.469342+0000","last_clean":"2024-09-16T07:27:48.469342+0000","last_became_active":"2024-09-16T07:27:48.459644+0000","last_became_peered":"2024-09-16T07:27:48.459644+0000","last_unstale":"2024-09-16T07:27:48.469342+0000","last_undegraded":"2024-09-16T07:27:48.469342+0000","last_fullsized":"2024-09-16T07:27:48.469342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:00.444857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.451622+0000","last_change":"2024-09-16T07:27:47.451622+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.451622+0000","last_undegraded":"2024-09-16T07:27:47.451622+0000","last_fullsized":"2024-09-16T07:27:47.451622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.447931+0000","last_change":"2024-09-16T07:27:47.447931+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.447931+0000","last_undegraded":"2024-09-16T07:27:47.447931+0000","last_fullsized":"2024-09-16T07:27:47.447931+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.471918+0000","last_change":"2024-09-16T07:27:48.462460+0000","last_active":"2024-09-16T07:27:48.471918+0000","last_peered":"2024-09-16T07:27:48.471918+0000","last_clean":"2024-09-16T07:27:48.471918+0000","last_became_active":"2024-09-16T07:27:48.462193+0000","last_became_peered":"2024-09-16T07:27:48.462193+0000","last_unstale":"2024-09-16T07:27:48.471918+0000","last_undegraded":"2024-09-16T07:27:48.471918+0000","last_fullsized":"2024-09-16T07:27:48.471918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:03:02.929085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441520+0000","last_change":"2024-09-16T07:27:47.441520+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441520+0000","last_undegraded":"2024-09-16T07:27:47.441520+0000","last_fullsized":"2024-09-16T07:27:47.441520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469491+0000","last_change":"2024-09-16T07:27:48.461060+0000","last_active":"2024-09-16T07:27:48.469491+0000","last_peered":"2024-09-16T07:27:48.469491+0000","last_clean":"2024-09-16T07:27:48.469491+0000","last_became_active":"2024-09-16T07:27:48.460770+0000","last_became_peered":"2024-09-16T07:27:48.460770+0000","last_unstale":"2024-09-16T07:27:48.469491+0000","last_undegraded":"2024-09-16T07:27:48.469491+0000","last_fullsized":"2024-09-16T07:27:48.469491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:46.741293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.452048+0000","last_change":"2024-09-16T07:27:47.452048+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.452048+0000","last_undegraded":"2024-09-16T07:27:47.452048+0000","last_fullsized":"2024-09-16T07:27:47.452048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461558+0000","last_change":"2024-09-16T07:27:48.459130+0000","last_active":"2024-09-16T07:27:49.461558+0000","last_peered":"2024-09-16T07:27:49.461558+0000","last_clean":"2024-09-16T07:27:49.461558+0000","last_became_active":"2024-09-16T07:27:48.458360+0000","last_became_peered":"2024-09-16T07:27:48.458360+0000","last_unstale":"2024-09-16T07:27:49.461558+0000","last_undegraded":"2024-09-16T07:27:49.461558+0000","last_fullsized":"2024-09-16T07:27:49.461558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:42.756027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468989+0000","last_change":"2024-09-16T07:27:48.462533+0000","last_active":"2024-09-16T07:27:48.468989+0000","last_peered":"2024-09-16T07:27:48.468989+0000","last_clean":"2024-09-16T07:27:48.468989+0000","last_became_active":"2024-09-16T07:27:48.462296+0000","last_became_peered":"2024-09-16T07:27:48.462296+0000","last_unstale":"2024-09-16T07:27:48.468989+0000","last_undegraded":"2024-09-16T07:27:48.468989+0000","last_fullsized":"2024-09-16T07:27:48.468989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:25.012988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466007+0000","last_change":"2024-09-16T07:27:48.461185+0000","last_active":"2024-09-16T07:27:48.466007+0000","last_peered":"2024-09-16T07:27:48.466007+0000","last_clean":"2024-09-16T07:27:48.466007+0000","last_became_active":"2024-09-16T07:27:48.460889+0000","last_became_peered":"2024-09-16T07:27:48.460889+0000","last_unstale":"2024-09-16T07:27:48.466007+0000","last_undegraded":"2024-09-16T07:27:48.466007+0000","last_fullsized":"2024-09-16T07:27:48.466007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:32:32.356185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.464924+0000","last_change":"2024-09-16T07:27:48.458531+0000","last_active":"2024-09-16T07:27:48.464924+0000","last_peered":"2024-09-16T07:27:48.464924+0000","last_clean":"2024-09-16T07:27:48.464924+0000","last_became_active":"2024-09-16T07:27:48.458285+0000","last_became_peered":"2024-09-16T07:27:48.458285+0000","last_unstale":"2024-09-16T07:27:48.464924+0000","last_undegraded":"2024-09-16T07:27:48.464924+0000","last_fullsized":"2024-09-16T07:27:48.464924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:40.253906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469844+0000","last_change":"2024-09-16T07:27:48.466279+0000","last_active":"2024-09-16T07:27:48.469844+0000","last_peered":"2024-09-16T07:27:48.469844+0000","last_clean":"2024-09-16T07:27:48.469844+0000","last_became_active":"2024-09-16T07:27:48.466000+0000","last_became_peered":"2024-09-16T07:27:48.466000+0000","last_unstale":"2024-09-16T07:27:48.469844+0000","last_undegraded":"2024-09-16T07:27:48.469844+0000","last_fullsized":"2024-09-16T07:27:48.469844+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:28.032448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449957+0000","last_change":"2024-09-16T07:27:47.449957+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449957+0000","last_undegraded":"2024-09-16T07:27:47.449957+0000","last_fullsized":"2024-09-16T07:27:47.449957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449887+0000","last_change":"2024-09-16T07:27:47.449887+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449887+0000","last_undegraded":"2024-09-16T07:27:47.449887+0000","last_fullsized":"2024-09-16T07:27:47.449887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467286+0000","last_change":"2024-09-16T07:27:48.465698+0000","last_active":"2024-09-16T07:27:48.467286+0000","last_peered":"2024-09-16T07:27:48.467286+0000","last_clean":"2024-09-16T07:27:48.467286+0000","last_became_active":"2024-09-16T07:27:48.465355+0000","last_became_peered":"2024-09-16T07:27:48.465355+0000","last_unstale":"2024-09-16T07:27:48.467286+0000","last_undegraded":"2024-09-16T07:27:48.467286+0000","last_fullsized":"2024-09-16T07:27:48.467286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:59.562068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.447615+0000","last_change":"2024-09-16T07:27:47.447615+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.447615+0000","last_undegraded":"2024-09-16T07:27:47.447615+0000","last_fullsized":"2024-09-16T07:27:47.447615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461494+0000","last_change":"2024-09-16T07:27:48.460038+0000","last_active":"2024-09-16T07:27:49.461494+0000","last_peered":"2024-09-16T07:27:49.461494+0000","last_clean":"2024-09-16T07:27:49.461494+0000","last_became_active":"2024-09-16T07:27:48.459755+0000","last_became_peered":"2024-09-16T07:27:48.459755+0000","last_unstale":"2024-09-16T07:27:49.461494+0000","last_undegraded":"2024-09-16T07:27:49.461494+0000","last_fullsized":"2024-09-16T07:27:49.461494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:37.822664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467196+0000","last_change":"2024-09-16T07:27:48.464651+0000","last_active":"2024-09-16T07:27:48.467196+0000","last_peered":"2024-09-16T07:27:48.467196+0000","last_clean":"2024-09-16T07:27:48.467196+0000","last_became_active":"2024-09-16T07:27:48.464373+0000","last_became_peered":"2024-09-16T07:27:48.464373+0000","last_unstale":"2024-09-16T07:27:48.467196+0000","last_undegraded":"2024-09-16T07:27:48.467196+0000","last_fullsized":"2024-09-16T07:27:48.467196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:34.767968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461319+0000","last_change":"2024-09-16T07:27:48.461614+0000","last_active":"2024-09-16T07:27:49.461319+0000","last_peered":"2024-09-16T07:27:49.461319+0000","last_clean":"2024-09-16T07:27:49.461319+0000","last_became_active":"2024-09-16T07:27:48.461333+0000","last_became_peered":"2024-09-16T07:27:48.461333+0000","last_unstale":"2024-09-16T07:27:49.461319+0000","last_undegraded":"2024-09-16T07:27:49.461319+0000","last_fullsized":"2024-09-16T07:27:49.461319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:14.430788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.477576+0000","last_change":"2024-09-16T07:27:48.469453+0000","last_active":"2024-09-16T07:27:48.477576+0000","last_peered":"2024-09-16T07:27:48.477576+0000","last_clean":"2024-09-16T07:27:48.477576+0000","last_became_active":"2024-09-16T07:27:48.466391+0000","last_became_peered":"2024-09-16T07:27:48.466391+0000","last_unstale":"2024-09-16T07:27:48.477576+0000","last_undegraded":"2024-09-16T07:27:48.477576+0000","last_fullsized":"2024-09-16T07:27:48.477576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:26.749995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468874+0000","last_change":"2024-09-16T07:27:48.461655+0000","last_active":"2024-09-16T07:27:48.468874+0000","last_peered":"2024-09-16T07:27:48.468874+0000","last_clean":"2024-09-16T07:27:48.468874+0000","last_became_active":"2024-09-16T07:27:48.461371+0000","last_became_peered":"2024-09-16T07:27:48.461371+0000","last_unstale":"2024-09-16T07:27:48.468874+0000","last_undegraded":"2024-09-16T07:27:48.468874+0000","last_fullsized":"2024-09-16T07:27:48.468874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:14.719547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444900+0000","last_change":"2024-09-16T07:27:47.444900+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444900+0000","last_undegraded":"2024-09-16T07:27:47.444900+0000","last_fullsized":"2024-09-16T07:27:47.444900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.459564+0000","last_change":"2024-09-16T07:27:48.455880+0000","last_active":"2024-09-16T07:27:48.459564+0000","last_peered":"2024-09-16T07:27:48.459564+0000","last_clean":"2024-09-16T07:27:48.459564+0000","last_became_active":"2024-09-16T07:27:48.455633+0000","last_became_peered":"2024-09-16T07:27:48.455633+0000","last_unstale":"2024-09-16T07:27:48.459564+0000","last_undegraded":"2024-09-16T07:27:48.459564+0000","last_fullsized":"2024-09-16T07:27:48.459564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:56.366154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.446950+0000","last_change":"2024-09-16T07:27:47.446950+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.446950+0000","last_undegraded":"2024-09-16T07:27:47.446950+0000","last_fullsized":"2024-09-16T07:27:47.446950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441948+0000","last_change":"2024-09-16T07:27:47.441948+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441948+0000","last_undegraded":"2024-09-16T07:27:47.441948+0000","last_fullsized":"2024-09-16T07:27:47.441948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.439145+0000","last_change":"2024-09-16T07:27:47.439145+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.439145+0000","last_undegraded":"2024-09-16T07:27:47.439145+0000","last_fullsized":"2024-09-16T07:27:47.439145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.472115+0000","last_change":"2024-09-16T07:27:48.462181+0000","last_active":"2024-09-16T07:27:48.472115+0000","last_peered":"2024-09-16T07:27:48.472115+0000","last_clean":"2024-09-16T07:27:48.472115+0000","last_became_active":"2024-09-16T07:27:48.460601+0000","last_became_peered":"2024-09-16T07:27:48.460601+0000","last_unstale":"2024-09-16T07:27:48.472115+0000","last_undegraded":"2024-09-16T07:27:48.472115+0000","last_fullsized":"2024-09-16T07:27:48.472115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:18.404077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.443479+0000","last_change":"2024-09-16T07:27:47.443479+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.443479+0000","last_undegraded":"2024-09-16T07:27:47.443479+0000","last_fullsized":"2024-09-16T07:27:47.443479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461161+0000","last_change":"2024-09-16T07:27:48.460512+0000","last_active":"2024-09-16T07:27:49.461161+0000","last_peered":"2024-09-16T07:27:49.461161+0000","last_clean":"2024-09-16T07:27:49.461161+0000","last_became_active":"2024-09-16T07:27:48.460276+0000","last_became_peered":"2024-09-16T07:27:48.460276+0000","last_unstale":"2024-09-16T07:27:49.461161+0000","last_undegraded":"2024-09-16T07:27:49.461161+0000","last_fullsized":"2024-09-16T07:27:49.461161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.223196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.446917+0000","last_change":"2024-09-16T07:27:47.446917+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.446917+0000","last_undegraded":"2024-09-16T07:27:47.446917+0000","last_fullsized":"2024-09-16T07:27:47.446917+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461617+0000","last_change":"2024-09-16T07:27:47.450308+0000","last_active":"2024-09-16T07:27:49.461617+0000","last_peered":"2024-09-16T07:27:49.461617+0000","last_clean":"2024-09-16T07:27:49.461617+0000","last_became_active":"2024-09-16T07:27:47.448332+0000","last_became_peered":"2024-09-16T07:27:47.448332+0000","last_unstale":"2024-09-16T07:27:49.461617+0000","last_undegraded":"2024-09-16T07:27:49.461617+0000","last_fullsized":"2024-09-16T07:27:49.461617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:38.412742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449454+0000","last_change":"2024-09-16T07:27:47.449454+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449454+0000","last_undegraded":"2024-09-16T07:27:47.449454+0000","last_fullsized":"2024-09-16T07:27:47.449454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.454190+0000","last_change":"2024-09-16T07:27:47.452837+0000","last_active":"2024-09-16T07:27:47.454190+0000","last_peered":"2024-09-16T07:27:47.454190+0000","last_clean":"2024-09-16T07:27:47.454190+0000","last_became_active":"2024-09-16T07:27:47.452517+0000","last_became_peered":"2024-09-16T07:27:47.452517+0000","last_unstale":"2024-09-16T07:27:47.454190+0000","last_undegraded":"2024-09-16T07:27:47.454190+0000","last_fullsized":"2024-09-16T07:27:47.454190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:50.738735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441487+0000","last_change":"2024-09-16T07:27:47.441487+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441487+0000","last_undegraded":"2024-09-16T07:27:47.441487+0000","last_fullsized":"2024-09-16T07:27:47.441487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460327+0000","last_change":"2024-09-16T07:27:47.459254+0000","last_active":"2024-09-16T07:27:47.460327+0000","last_peered":"2024-09-16T07:27:47.460327+0000","last_clean":"2024-09-16T07:27:47.460327+0000","last_became_active":"2024-09-16T07:27:47.458912+0000","last_became_peered":"2024-09-16T07:27:47.458912+0000","last_unstale":"2024-09-16T07:27:47.460327+0000","last_undegraded":"2024-09-16T07:27:47.460327+0000","last_fullsized":"2024-09-16T07:27:47.460327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:07:13.520726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444836+0000","last_change":"2024-09-16T07:27:47.444836+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444836+0000","last_undegraded":"2024-09-16T07:27:47.444836+0000","last_fullsized":"2024-09-16T07:27:47.444836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.478412+0000","last_change":"2024-09-16T07:27:47.458877+0000","last_active":"2024-09-16T07:27:48.478412+0000","last_peered":"2024-09-16T07:27:48.478412+0000","last_clean":"2024-09-16T07:27:48.478412+0000","last_became_active":"2024-09-16T07:27:47.458615+0000","last_became_peered":"2024-09-16T07:27:47.458615+0000","last_unstale":"2024-09-16T07:27:48.478412+0000","last_undegraded":"2024-09-16T07:27:48.478412+0000","last_fullsized":"2024-09-16T07:27:48.478412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:00.779149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438546+0000","last_change":"2024-09-16T07:27:47.438546+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438546+0000","last_undegraded":"2024-09-16T07:27:47.438546+0000","last_fullsized":"2024-09-16T07:27:47.438546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.471392+0000","last_change":"2024-09-16T07:27:47.455983+0000","last_active":"2024-09-16T07:27:48.471392+0000","last_peered":"2024-09-16T07:27:48.471392+0000","last_clean":"2024-09-16T07:27:48.471392+0000","last_became_active":"2024-09-16T07:27:47.454631+0000","last_became_peered":"2024-09-16T07:27:47.454631+0000","last_unstale":"2024-09-16T07:27:48.471392+0000","last_undegraded":"2024-09-16T07:27:48.471392+0000","last_fullsized":"2024-09-16T07:27:48.471392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:37.670315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461455+0000","last_change":"2024-09-16T07:27:48.457180+0000","last_active":"2024-09-16T07:27:49.461455+0000","last_peered":"2024-09-16T07:27:49.461455+0000","last_clean":"2024-09-16T07:27:49.461455+0000","last_became_active":"2024-09-16T07:27:48.456834+0000","last_became_peered":"2024-09-16T07:27:48.456834+0000","last_unstale":"2024-09-16T07:27:49.461455+0000","last_undegraded":"2024-09-16T07:27:49.461455+0000","last_fullsized":"2024-09-16T07:27:49.461455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:04.346053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464365+0000","last_change":"2024-09-16T07:27:47.461641+0000","last_active":"2024-09-16T07:27:47.464365+0000","last_peered":"2024-09-16T07:27:47.464365+0000","last_clean":"2024-09-16T07:27:47.464365+0000","last_became_active":"2024-09-16T07:27:47.461318+0000","last_became_peered":"2024-09-16T07:27:47.461318+0000","last_unstale":"2024-09-16T07:27:47.464365+0000","last_undegraded":"2024-09-16T07:27:47.464365+0000","last_fullsized":"2024-09-16T07:27:47.464365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:10:51.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.465633+0000","last_change":"2024-09-16T07:27:48.455278+0000","last_active":"2024-09-16T07:27:48.465633+0000","last_peered":"2024-09-16T07:27:48.465633+0000","last_clean":"2024-09-16T07:27:48.465633+0000","last_became_active":"2024-09-16T07:27:48.454871+0000","last_became_peered":"2024-09-16T07:27:48.454871+0000","last_unstale":"2024-09-16T07:27:48.465633+0000","last_undegraded":"2024-09-16T07:27:48.465633+0000","last_fullsized":"2024-09-16T07:27:48.465633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:42.196208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443144+0000","last_change":"2024-09-16T07:27:47.452265+0000","last_active":"2024-09-16T07:27:48.443144+0000","last_peered":"2024-09-16T07:27:48.443144+0000","last_clean":"2024-09-16T07:27:48.443144+0000","last_became_active":"2024-09-16T07:27:47.452018+0000","last_became_peered":"2024-09-16T07:27:47.452018+0000","last_unstale":"2024-09-16T07:27:48.443144+0000","last_undegraded":"2024-09-16T07:27:48.443144+0000","last_fullsized":"2024-09-16T07:27:48.443144+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:35.000221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.448499+0000","last_change":"2024-09-16T07:27:47.448499+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.448499+0000","last_undegraded":"2024-09-16T07:27:47.448499+0000","last_fullsized":"2024-09-16T07:27:47.448499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.449811+0000","last_change":"2024-09-16T07:27:47.447783+0000","last_active":"2024-09-16T07:27:48.449811+0000","last_peered":"2024-09-16T07:27:48.449811+0000","last_clean":"2024-09-16T07:27:48.449811+0000","last_became_active":"2024-09-16T07:27:47.447510+0000","last_became_peered":"2024-09-16T07:27:47.447510+0000","last_unstale":"2024-09-16T07:27:48.449811+0000","last_undegraded":"2024-09-16T07:27:48.449811+0000","last_fullsized":"2024-09-16T07:27:48.449811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:14.370496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.459535+0000","last_change":"2024-09-16T07:27:48.454744+0000","last_active":"2024-09-16T07:27:48.459535+0000","last_peered":"2024-09-16T07:27:48.459535+0000","last_clean":"2024-09-16T07:27:48.459535+0000","last_became_active":"2024-09-16T07:27:48.454446+0000","last_became_peered":"2024-09-16T07:27:48.454446+0000","last_unstale":"2024-09-16T07:27:48.459535+0000","last_undegraded":"2024-09-16T07:27:48.459535+0000","last_fullsized":"2024-09-16T07:27:48.459535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:52.733528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443637+0000","last_change":"2024-09-16T07:27:47.450306+0000","last_active":"2024-09-16T07:27:48.443637+0000","last_peered":"2024-09-16T07:27:48.443637+0000","last_clean":"2024-09-16T07:27:48.443637+0000","last_became_active":"2024-09-16T07:27:47.449275+0000","last_became_peered":"2024-09-16T07:27:47.449275+0000","last_unstale":"2024-09-16T07:27:48.443637+0000","last_undegraded":"2024-09-16T07:27:48.443637+0000","last_fullsized":"2024-09-16T07:27:48.443637+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:00.479157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.470005+0000","last_change":"2024-09-16T07:27:48.468331+0000","last_active":"2024-09-16T07:27:48.470005+0000","last_peered":"2024-09-16T07:27:48.470005+0000","last_clean":"2024-09-16T07:27:48.470005+0000","last_became_active":"2024-09-16T07:27:48.468040+0000","last_became_peered":"2024-09-16T07:27:48.468040+0000","last_unstale":"2024-09-16T07:27:48.470005+0000","last_undegraded":"2024-09-16T07:27:48.470005+0000","last_fullsized":"2024-09-16T07:27:48.470005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:06.084517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453197+0000","last_change":"2024-09-16T07:27:47.450498+0000","last_active":"2024-09-16T07:27:47.453197+0000","last_peered":"2024-09-16T07:27:47.453197+0000","last_clean":"2024-09-16T07:27:47.453197+0000","last_became_active":"2024-09-16T07:27:47.449615+0000","last_became_peered":"2024-09-16T07:27:47.449615+0000","last_unstale":"2024-09-16T07:27:47.453197+0000","last_undegraded":"2024-09-16T07:27:47.453197+0000","last_fullsized":"2024-09-16T07:27:47.453197+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:23:34.664293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438573+0000","last_change":"2024-09-16T07:27:47.438573+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438573+0000","last_undegraded":"2024-09-16T07:27:47.438573+0000","last_fullsized":"2024-09-16T07:27:47.438573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.478154+0000","last_change":"2024-09-16T07:27:47.460337+0000","last_active":"2024-09-16T07:27:48.478154+0000","last_peered":"2024-09-16T07:27:48.478154+0000","last_clean":"2024-09-16T07:27:48.478154+0000","last_became_active":"2024-09-16T07:27:47.460023+0000","last_became_peered":"2024-09-16T07:27:47.460023+0000","last_unstale":"2024-09-16T07:27:48.478154+0000","last_undegraded":"2024-09-16T07:27:48.478154+0000","last_fullsized":"2024-09-16T07:27:48.478154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:52.763400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.456699+0000","last_change":"2024-09-16T07:27:48.449778+0000","last_active":"2024-09-16T07:27:48.456699+0000","last_peered":"2024-09-16T07:27:48.456699+0000","last_clean":"2024-09-16T07:27:48.456699+0000","last_became_active":"2024-09-16T07:27:48.449504+0000","last_became_peered":"2024-09-16T07:27:48.449504+0000","last_unstale":"2024-09-16T07:27:48.456699+0000","last_undegraded":"2024-09-16T07:27:48.456699+0000","last_fullsized":"2024-09-16T07:27:48.456699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:48.419517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.480574+0000","last_change":"2024-09-16T07:27:47.458310+0000","last_active":"2024-09-16T07:27:48.480574+0000","last_peered":"2024-09-16T07:27:48.480574+0000","last_clean":"2024-09-16T07:27:48.480574+0000","last_became_active":"2024-09-16T07:27:47.457920+0000","last_became_peered":"2024-09-16T07:27:47.457920+0000","last_unstale":"2024-09-16T07:27:48.480574+0000","last_undegraded":"2024-09-16T07:27:48.480574+0000","last_fullsized":"2024-09-16T07:27:48.480574+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.526953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469560+0000","last_change":"2024-09-16T07:27:48.467738+0000","last_active":"2024-09-16T07:27:48.469560+0000","last_peered":"2024-09-16T07:27:48.469560+0000","last_clean":"2024-09-16T07:27:48.469560+0000","last_became_active":"2024-09-16T07:27:48.467296+0000","last_became_peered":"2024-09-16T07:27:48.467296+0000","last_unstale":"2024-09-16T07:27:48.469560+0000","last_undegraded":"2024-09-16T07:27:48.469560+0000","last_fullsized":"2024-09-16T07:27:48.469560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:08.840299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453240+0000","last_change":"2024-09-16T07:27:47.449886+0000","last_active":"2024-09-16T07:27:47.453240+0000","last_peered":"2024-09-16T07:27:47.453240+0000","last_clean":"2024-09-16T07:27:47.453240+0000","last_became_active":"2024-09-16T07:27:47.449599+0000","last_became_peered":"2024-09-16T07:27:47.449599+0000","last_unstale":"2024-09-16T07:27:47.453240+0000","last_undegraded":"2024-09-16T07:27:47.453240+0000","last_fullsized":"2024-09-16T07:27:47.453240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:45:32.473190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466091+0000","last_change":"2024-09-16T07:27:48.453257+0000","last_active":"2024-09-16T07:27:48.466091+0000","last_peered":"2024-09-16T07:27:48.466091+0000","last_clean":"2024-09-16T07:27:48.466091+0000","last_became_active":"2024-09-16T07:27:48.453020+0000","last_became_peered":"2024-09-16T07:27:48.453020+0000","last_unstale":"2024-09-16T07:27:48.466091+0000","last_undegraded":"2024-09-16T07:27:48.466091+0000","last_fullsized":"2024-09-16T07:27:48.466091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:07.527458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.439529+0000","last_change":"2024-09-16T07:27:47.449870+0000","last_active":"2024-09-16T07:27:48.439529+0000","last_peered":"2024-09-16T07:27:48.439529+0000","last_clean":"2024-09-16T07:27:48.439529+0000","last_became_active":"2024-09-16T07:27:47.449587+0000","last_became_peered":"2024-09-16T07:27:47.449587+0000","last_unstale":"2024-09-16T07:27:48.439529+0000","last_undegraded":"2024-09-16T07:27:48.439529+0000","last_fullsized":"2024-09-16T07:27:48.439529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:56.658575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468654+0000","last_change":"2024-09-16T07:27:48.457526+0000","last_active":"2024-09-16T07:27:48.468654+0000","last_peered":"2024-09-16T07:27:48.468654+0000","last_clean":"2024-09-16T07:27:48.468654+0000","last_became_active":"2024-09-16T07:27:48.457292+0000","last_became_peered":"2024-09-16T07:27:48.457292+0000","last_unstale":"2024-09-16T07:27:48.468654+0000","last_undegraded":"2024-09-16T07:27:48.468654+0000","last_fullsized":"2024-09-16T07:27:48.468654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:18.042402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.439349+0000","last_change":"2024-09-16T07:27:47.450647+0000","last_active":"2024-09-16T07:27:48.439349+0000","last_peered":"2024-09-16T07:27:48.439349+0000","last_clean":"2024-09-16T07:27:48.439349+0000","last_became_active":"2024-09-16T07:27:47.450313+0000","last_became_peered":"2024-09-16T07:27:47.450313+0000","last_unstale":"2024-09-16T07:27:48.439349+0000","last_undegraded":"2024-09-16T07:27:48.439349+0000","last_fullsized":"2024-09-16T07:27:48.439349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:01.115128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467730+0000","last_change":"2024-09-16T07:27:48.459503+0000","last_active":"2024-09-16T07:27:48.467730+0000","last_peered":"2024-09-16T07:27:48.467730+0000","last_clean":"2024-09-16T07:27:48.467730+0000","last_became_active":"2024-09-16T07:27:48.459201+0000","last_became_peered":"2024-09-16T07:27:48.459201+0000","last_unstale":"2024-09-16T07:27:48.467730+0000","last_undegraded":"2024-09-16T07:27:48.467730+0000","last_fullsized":"2024-09-16T07:27:48.467730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:35.394756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.482800+0000","last_change":"2024-09-16T07:27:47.451210+0000","last_active":"2024-09-16T07:27:48.482800+0000","last_peered":"2024-09-16T07:27:48.482800+0000","last_clean":"2024-09-16T07:27:48.482800+0000","last_became_active":"2024-09-16T07:27:47.450925+0000","last_became_peered":"2024-09-16T07:27:47.450925+0000","last_unstale":"2024-09-16T07:27:48.482800+0000","last_undegraded":"2024-09-16T07:27:48.482800+0000","last_fullsized":"2024-09-16T07:27:48.482800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:03.229023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.464941+0000","last_change":"2024-09-16T07:27:48.457648+0000","last_active":"2024-09-16T07:27:48.464941+0000","last_peered":"2024-09-16T07:27:48.464941+0000","last_clean":"2024-09-16T07:27:48.464941+0000","last_became_active":"2024-09-16T07:27:48.457358+0000","last_became_peered":"2024-09-16T07:27:48.457358+0000","last_unstale":"2024-09-16T07:27:48.464941+0000","last_undegraded":"2024-09-16T07:27:48.464941+0000","last_fullsized":"2024-09-16T07:27:48.464941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:28.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460425+0000","last_change":"2024-09-16T07:27:47.458791+0000","last_active":"2024-09-16T07:27:47.460425+0000","last_peered":"2024-09-16T07:27:47.460425+0000","last_clean":"2024-09-16T07:27:47.460425+0000","last_became_active":"2024-09-16T07:27:47.458560+0000","last_became_peered":"2024-09-16T07:27:47.458560+0000","last_unstale":"2024-09-16T07:27:47.460425+0000","last_undegraded":"2024-09-16T07:27:47.460425+0000","last_fullsized":"2024-09-16T07:27:47.460425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:15.540429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468200+0000","last_change":"2024-09-16T07:27:48.460172+0000","last_active":"2024-09-16T07:27:48.468200+0000","last_peered":"2024-09-16T07:27:48.468200+0000","last_clean":"2024-09-16T07:27:48.468200+0000","last_became_active":"2024-09-16T07:27:48.459930+0000","last_became_peered":"2024-09-16T07:27:48.459930+0000","last_unstale":"2024-09-16T07:27:48.468200+0000","last_undegraded":"2024-09-16T07:27:48.468200+0000","last_fullsized":"2024-09-16T07:27:48.468200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:04.739035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453174+0000","last_change":"2024-09-16T07:27:47.450135+0000","last_active":"2024-09-16T07:27:47.453174+0000","last_peered":"2024-09-16T07:27:47.453174+0000","last_clean":"2024-09-16T07:27:47.453174+0000","last_became_active":"2024-09-16T07:27:47.449254+0000","last_became_peered":"2024-09-16T07:27:47.449254+0000","last_unstale":"2024-09-16T07:27:47.453174+0000","last_undegraded":"2024-09-16T07:27:47.453174+0000","last_fullsized":"2024-09-16T07:27:47.453174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:16.114777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444472+0000","last_change":"2024-09-16T07:27:47.444472+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444472+0000","last_undegraded":"2024-09-16T07:27:47.444472+0000","last_fullsized":"2024-09-16T07:27:47.444472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.476242+0000","last_change":"2024-09-16T07:27:47.455984+0000","last_active":"2024-09-16T07:27:48.476242+0000","last_peered":"2024-09-16T07:27:48.476242+0000","last_clean":"2024-09-16T07:27:48.476242+0000","last_became_active":"2024-09-16T07:27:47.454791+0000","last_became_peered":"2024-09-16T07:27:47.454791+0000","last_unstale":"2024-09-16T07:27:48.476242+0000","last_undegraded":"2024-09-16T07:27:48.476242+0000","last_fullsized":"2024-09-16T07:27:48.476242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:24.745832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.472184+0000","last_change":"2024-09-16T07:27:48.462451+0000","last_active":"2024-09-16T07:27:48.472184+0000","last_peered":"2024-09-16T07:27:48.472184+0000","last_clean":"2024-09-16T07:27:48.472184+0000","last_became_active":"2024-09-16T07:27:48.462145+0000","last_became_peered":"2024-09-16T07:27:48.462145+0000","last_unstale":"2024-09-16T07:27:48.472184+0000","last_undegraded":"2024-09-16T07:27:48.472184+0000","last_fullsized":"2024-09-16T07:27:48.472184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:25:02.985426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460913+0000","last_change":"2024-09-16T07:27:47.459943+0000","last_active":"2024-09-16T07:27:47.460913+0000","last_peered":"2024-09-16T07:27:47.460913+0000","last_clean":"2024-09-16T07:27:47.460913+0000","last_became_active":"2024-09-16T07:27:47.459680+0000","last_became_peered":"2024-09-16T07:27:47.459680+0000","last_unstale":"2024-09-16T07:27:47.460913+0000","last_undegraded":"2024-09-16T07:27:47.460913+0000","last_fullsized":"2024-09-16T07:27:47.460913+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:44.061784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468987+0000","last_change":"2024-09-16T07:27:48.461989+0000","last_active":"2024-09-16T07:27:48.468987+0000","last_peered":"2024-09-16T07:27:48.468987+0000","last_clean":"2024-09-16T07:27:48.468987+0000","last_became_active":"2024-09-16T07:27:48.461731+0000","last_became_peered":"2024-09-16T07:27:48.461731+0000","last_unstale":"2024-09-16T07:27:48.468987+0000","last_undegraded":"2024-09-16T07:27:48.468987+0000","last_fullsized":"2024-09-16T07:27:48.468987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:43.744251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461382+0000","last_change":"2024-09-16T07:27:47.452023+0000","last_active":"2024-09-16T07:27:49.461382+0000","last_peered":"2024-09-16T07:27:49.461382+0000","last_clean":"2024-09-16T07:27:49.461382+0000","last_became_active":"2024-09-16T07:27:47.451738+0000","last_became_peered":"2024-09-16T07:27:47.451738+0000","last_unstale":"2024-09-16T07:27:49.461382+0000","last_undegraded":"2024-09-16T07:27:49.461382+0000","last_fullsized":"2024-09-16T07:27:49.461382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:28.412285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438666+0000","last_change":"2024-09-16T07:27:47.438666+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438666+0000","last_undegraded":"2024-09-16T07:27:47.438666+0000","last_fullsized":"2024-09-16T07:27:47.438666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.457752+0000","last_change":"2024-09-16T07:27:47.452663+0000","last_active":"2024-09-16T07:27:47.457752+0000","last_peered":"2024-09-16T07:27:47.457752+0000","last_clean":"2024-09-16T07:27:47.457752+0000","last_became_active":"2024-09-16T07:27:47.451570+0000","last_became_peered":"2024-09-16T07:27:47.451570+0000","last_unstale":"2024-09-16T07:27:47.457752+0000","last_undegraded":"2024-09-16T07:27:47.457752+0000","last_fullsized":"2024-09-16T07:27:47.457752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:14.561916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.439442+0000","last_change":"2024-09-16T07:27:47.439442+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.439442+0000","last_undegraded":"2024-09-16T07:27:47.439442+0000","last_fullsized":"2024-09-16T07:27:47.439442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443403+0000","last_change":"2024-09-16T07:27:47.455318+0000","last_active":"2024-09-16T07:27:48.443403+0000","last_peered":"2024-09-16T07:27:48.443403+0000","last_clean":"2024-09-16T07:27:48.443403+0000","last_became_active":"2024-09-16T07:27:47.455047+0000","last_became_peered":"2024-09-16T07:27:47.455047+0000","last_unstale":"2024-09-16T07:27:48.443403+0000","last_undegraded":"2024-09-16T07:27:48.443403+0000","last_fullsized":"2024-09-16T07:27:48.443403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:08.028003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466860+0000","last_change":"2024-09-16T07:27:48.462125+0000","last_active":"2024-09-16T07:27:48.466860+0000","last_peered":"2024-09-16T07:27:48.466860+0000","last_clean":"2024-09-16T07:27:48.466860+0000","last_became_active":"2024-09-16T07:27:48.461779+0000","last_became_peered":"2024-09-16T07:27:48.461779+0000","last_unstale":"2024-09-16T07:27:48.466860+0000","last_undegraded":"2024-09-16T07:27:48.466860+0000","last_fullsized":"2024-09-16T07:27:48.466860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:37.463436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464161+0000","last_change":"2024-09-16T07:27:47.461596+0000","last_active":"2024-09-16T07:27:47.464161+0000","last_peered":"2024-09-16T07:27:47.464161+0000","last_clean":"2024-09-16T07:27:47.464161+0000","last_became_active":"2024-09-16T07:27:47.461292+0000","last_became_peered":"2024-09-16T07:27:47.461292+0000","last_unstale":"2024-09-16T07:27:47.464161+0000","last_undegraded":"2024-09-16T07:27:47.464161+0000","last_fullsized":"2024-09-16T07:27:47.464161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:22.241647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.463013+0000","last_change":"2024-09-16T07:27:48.452873+0000","last_active":"2024-09-16T07:27:48.463013+0000","last_peered":"2024-09-16T07:27:48.463013+0000","last_clean":"2024-09-16T07:27:48.463013+0000","last_became_active":"2024-09-16T07:27:48.450871+0000","last_became_peered":"2024-09-16T07:27:48.450871+0000","last_unstale":"2024-09-16T07:27:48.463013+0000","last_undegraded":"2024-09-16T07:27:48.463013+0000","last_fullsized":"2024-09-16T07:27:48.463013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:38:38.451975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464346+0000","last_change":"2024-09-16T07:27:47.461620+0000","last_active":"2024-09-16T07:27:47.464346+0000","last_peered":"2024-09-16T07:27:47.464346+0000","last_clean":"2024-09-16T07:27:47.464346+0000","last_became_active":"2024-09-16T07:27:47.461338+0000","last_became_peered":"2024-09-16T07:27:47.461338+0000","last_unstale":"2024-09-16T07:27:47.464346+0000","last_undegraded":"2024-09-16T07:27:47.464346+0000","last_fullsized":"2024-09-16T07:27:47.464346+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:28.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444027+0000","last_change":"2024-09-16T07:27:47.444027+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444027+0000","last_undegraded":"2024-09-16T07:27:47.444027+0000","last_fullsized":"2024-09-16T07:27:47.444027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443387+0000","last_change":"2024-09-16T07:27:47.458964+0000","last_active":"2024-09-16T07:27:48.443387+0000","last_peered":"2024-09-16T07:27:48.443387+0000","last_clean":"2024-09-16T07:27:48.443387+0000","last_became_active":"2024-09-16T07:27:47.458687+0000","last_became_peered":"2024-09-16T07:27:47.458687+0000","last_unstale":"2024-09-16T07:27:48.443387+0000","last_undegraded":"2024-09-16T07:27:48.443387+0000","last_fullsized":"2024-09-16T07:27:48.443387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:43.227646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438895+0000","last_change":"2024-09-16T07:27:47.438895+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438895+0000","last_undegraded":"2024-09-16T07:27:47.438895+0000","last_fullsized":"2024-09-16T07:27:47.438895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.448905+0000","last_change":"2024-09-16T07:27:47.456743+0000","last_active":"2024-09-16T07:27:48.448905+0000","last_peered":"2024-09-16T07:27:48.448905+0000","last_clean":"2024-09-16T07:27:48.448905+0000","last_became_active":"2024-09-16T07:27:47.455261+0000","last_became_peered":"2024-09-16T07:27:47.455261+0000","last_unstale":"2024-09-16T07:27:48.448905+0000","last_undegraded":"2024-09-16T07:27:48.448905+0000","last_fullsized":"2024-09-16T07:27:48.448905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:44.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444420+0000","last_change":"2024-09-16T07:27:47.444420+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444420+0000","last_undegraded":"2024-09-16T07:27:47.444420+0000","last_fullsized":"2024-09-16T07:27:47.444420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.459115+0000","last_change":"2024-09-16T07:27:47.458363+0000","last_active":"2024-09-16T07:27:47.459115+0000","last_peered":"2024-09-16T07:27:47.459115+0000","last_clean":"2024-09-16T07:27:47.459115+0000","last_became_active":"2024-09-16T07:27:47.455595+0000","last_became_peered":"2024-09-16T07:27:47.455595+0000","last_unstale":"2024-09-16T07:27:47.459115+0000","last_undegraded":"2024-09-16T07:27:47.459115+0000","last_fullsized":"2024-09-16T07:27:47.459115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:18.944163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.465977+0000","last_change":"2024-09-16T07:27:48.460471+0000","last_active":"2024-09-16T07:27:48.465977+0000","last_peered":"2024-09-16T07:27:48.465977+0000","last_clean":"2024-09-16T07:27:48.465977+0000","last_became_active":"2024-09-16T07:27:48.456813+0000","last_became_peered":"2024-09-16T07:27:48.456813+0000","last_unstale":"2024-09-16T07:27:48.465977+0000","last_undegraded":"2024-09-16T07:27:48.465977+0000","last_fullsized":"2024-09-16T07:27:48.465977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:45.085619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.449173+0000","last_change":"2024-09-16T07:27:47.452638+0000","last_active":"2024-09-16T07:27:48.449173+0000","last_peered":"2024-09-16T07:27:48.449173+0000","last_clean":"2024-09-16T07:27:48.449173+0000","last_became_active":"2024-09-16T07:27:47.452390+0000","last_became_peered":"2024-09-16T07:27:47.452390+0000","last_unstale":"2024-09-16T07:27:48.449173+0000","last_undegraded":"2024-09-16T07:27:48.449173+0000","last_fullsized":"2024-09-16T07:27:48.449173+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:04.058738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.440856+0000","last_change":"2024-09-16T07:27:47.440856+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.440856+0000","last_undegraded":"2024-09-16T07:27:47.440856+0000","last_fullsized":"2024-09-16T07:27:47.440856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.462516+0000","last_change":"2024-09-16T07:27:47.460442+0000","last_active":"2024-09-16T07:27:47.462516+0000","last_peered":"2024-09-16T07:27:47.462516+0000","last_clean":"2024-09-16T07:27:47.462516+0000","last_became_active":"2024-09-16T07:27:47.459037+0000","last_became_peered":"2024-09-16T07:27:47.459037+0000","last_unstale":"2024-09-16T07:27:47.462516+0000","last_undegraded":"2024-09-16T07:27:47.462516+0000","last_fullsized":"2024-09-16T07:27:47.462516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:25:58.753488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.442957+0000","last_change":"2024-09-16T07:27:47.442957+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.442957+0000","last_undegraded":"2024-09-16T07:27:47.442957+0000","last_fullsized":"2024-09-16T07:27:47.442957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460678+0000","last_change":"2024-09-16T07:27:47.453007+0000","last_active":"2024-09-16T07:27:47.460678+0000","last_peered":"2024-09-16T07:27:47.460678+0000","last_clean":"2024-09-16T07:27:47.460678+0000","last_became_active":"2024-09-16T07:27:47.452735+0000","last_became_peered":"2024-09-16T07:27:47.452735+0000","last_unstale":"2024-09-16T07:27:47.460678+0000","last_undegraded":"2024-09-16T07:27:47.460678+0000","last_fullsized":"2024-09-16T07:27:47.460678+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:47:52.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.440008+0000","last_change":"2024-09-16T07:27:47.440008+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.440008+0000","last_undegraded":"2024-09-16T07:27:47.440008+0000","last_fullsized":"2024-09-16T07:27:47.440008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.450452+0000","last_change":"2024-09-16T07:27:47.454175+0000","last_active":"2024-09-16T07:27:48.450452+0000","last_peered":"2024-09-16T07:27:48.450452+0000","last_clean":"2024-09-16T07:27:48.450452+0000","last_became_active":"2024-09-16T07:27:47.453769+0000","last_became_peered":"2024-09-16T07:27:47.453769+0000","last_unstale":"2024-09-16T07:27:48.450452+0000","last_undegraded":"2024-09-16T07:27:48.450452+0000","last_fullsized":"2024-09-16T07:27:48.450452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:17.667822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443464+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:27:48.443464+0000","last_peered":"2024-09-16T07:27:48.443464+0000","last_clean":"2024-09-16T07:27:48.443464+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:27:48.443464+0000","last_undegraded":"2024-09-16T07:27:48.443464+0000","last_fullsized":"2024-09-16T07:27:48.443464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476803,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":295094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476803,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62448,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309392,"statfs":{"total":96636764160,"available":96572817408,"internally_reserved":0,"allocated":741376,"data_stored":303641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476803,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40784,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331056,"statfs":{"total":96636764160,"available":96595001344,"internally_reserved":0,"allocated":1302528,"data_stored":885462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476802,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":295094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":288819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509506,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":288819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46352,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325488,"statfs":{"total":96636764160,"available":96589299712,"internally_reserved":0,"allocated":1302528,"data_stored":885462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49960,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321880,"statfs":{"total":96636764160,"available":96585605120,"internally_reserved":0,"allocated":741376,"data_stored":308785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:27:53.477 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3f is not active+clean 2024-09-16T07:27:53.477 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.453401+0000', 'last_change': '2024-09-16T07:27:47.453401+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.453401+0000', 'last_undegraded': '2024-09-16T07:27:47.453401+0000', 'last_fullsized': '2024-09-16T07:27:47.453401+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.477 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3e is not active+clean 2024-09-16T07:27:53.477 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.444099+0000', 'last_change': '2024-09-16T07:27:47.444099+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.444099+0000', 'last_undegraded': '2024-09-16T07:27:47.444099+0000', 'last_fullsized': '2024-09-16T07:27:47.444099+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:27:53.477 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3b is not active+clean 2024-09-16T07:27:53.477 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.451622+0000', 'last_change': '2024-09-16T07:27:47.451622+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.451622+0000', 'last_undegraded': '2024-09-16T07:27:47.451622+0000', 'last_fullsized': '2024-09-16T07:27:47.451622+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3a is not active+clean 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.447931+0000', 'last_change': '2024-09-16T07:27:47.447931+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.447931+0000', 'last_undegraded': '2024-09-16T07:27:47.447931+0000', 'last_fullsized': '2024-09-16T07:27:47.447931+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:PG 9.38 is not active+clean 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.441520+0000', 'last_change': '2024-09-16T07:27:47.441520+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.441520+0000', 'last_undegraded': '2024-09-16T07:27:47.441520+0000', 'last_fullsized': '2024-09-16T07:27:47.441520+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:PG 9.36 is not active+clean 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.452048+0000', 'last_change': '2024-09-16T07:27:47.452048+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.452048+0000', 'last_undegraded': '2024-09-16T07:27:47.452048+0000', 'last_fullsized': '2024-09-16T07:27:47.452048+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:PG 9.30 is not active+clean 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.449957+0000', 'last_change': '2024-09-16T07:27:47.449957+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.449957+0000', 'last_undegraded': '2024-09-16T07:27:47.449957+0000', 'last_fullsized': '2024-09-16T07:27:47.449957+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2f is not active+clean 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.449887+0000', 'last_change': '2024-09-16T07:27:47.449887+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.449887+0000', 'last_undegraded': '2024-09-16T07:27:47.449887+0000', 'last_fullsized': '2024-09-16T07:27:47.449887+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.478 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2d is not active+clean 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.447615+0000', 'last_change': '2024-09-16T07:27:47.447615+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.447615+0000', 'last_undegraded': '2024-09-16T07:27:47.447615+0000', 'last_fullsized': '2024-09-16T07:27:47.447615+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:PG 9.27 is not active+clean 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.444900+0000', 'last_change': '2024-09-16T07:27:47.444900+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.444900+0000', 'last_undegraded': '2024-09-16T07:27:47.444900+0000', 'last_fullsized': '2024-09-16T07:27:47.444900+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:PG 9.25 is not active+clean 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.446950+0000', 'last_change': '2024-09-16T07:27:47.446950+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.446950+0000', 'last_undegraded': '2024-09-16T07:27:47.446950+0000', 'last_fullsized': '2024-09-16T07:27:47.446950+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:PG 9.24 is not active+clean 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.441948+0000', 'last_change': '2024-09-16T07:27:47.441948+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.441948+0000', 'last_undegraded': '2024-09-16T07:27:47.441948+0000', 'last_fullsized': '2024-09-16T07:27:47.441948+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:PG 9.23 is not active+clean 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.439145+0000', 'last_change': '2024-09-16T07:27:47.439145+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.439145+0000', 'last_undegraded': '2024-09-16T07:27:47.439145+0000', 'last_fullsized': '2024-09-16T07:27:47.439145+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:PG 9.21 is not active+clean 2024-09-16T07:27:53.479 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.443479+0000', 'last_change': '2024-09-16T07:27:47.443479+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.443479+0000', 'last_undegraded': '2024-09-16T07:27:47.443479+0000', 'last_fullsized': '2024-09-16T07:27:47.443479+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1e is not active+clean 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.446917+0000', 'last_change': '2024-09-16T07:27:47.446917+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.446917+0000', 'last_undegraded': '2024-09-16T07:27:47.446917+0000', 'last_fullsized': '2024-09-16T07:27:47.446917+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1f is not active+clean 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.449454+0000', 'last_change': '2024-09-16T07:27:47.449454+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.449454+0000', 'last_undegraded': '2024-09-16T07:27:47.449454+0000', 'last_fullsized': '2024-09-16T07:27:47.449454+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1c is not active+clean 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.441487+0000', 'last_change': '2024-09-16T07:27:47.441487+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.441487+0000', 'last_undegraded': '2024-09-16T07:27:47.441487+0000', 'last_fullsized': '2024-09-16T07:27:47.441487+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1d is not active+clean 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.444836+0000', 'last_change': '2024-09-16T07:27:47.444836+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.444836+0000', 'last_undegraded': '2024-09-16T07:27:47.444836+0000', 'last_fullsized': '2024-09-16T07:27:47.444836+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1a is not active+clean 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.438546+0000', 'last_change': '2024-09-16T07:27:47.438546+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.438546+0000', 'last_undegraded': '2024-09-16T07:27:47.438546+0000', 'last_fullsized': '2024-09-16T07:27:47.438546+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.480 INFO:tasks.ceph.ceph_manager.ceph:PG 9.19 is not active+clean 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.448499+0000', 'last_change': '2024-09-16T07:27:47.448499+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.448499+0000', 'last_undegraded': '2024-09-16T07:27:47.448499+0000', 'last_fullsized': '2024-09-16T07:27:47.448499+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.438573+0000', 'last_change': '2024-09-16T07:27:47.438573+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.438573+0000', 'last_undegraded': '2024-09-16T07:27:47.438573+0000', 'last_fullsized': '2024-09-16T07:27:47.438573+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.444472+0000', 'last_change': '2024-09-16T07:27:47.444472+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.444472+0000', 'last_undegraded': '2024-09-16T07:27:47.444472+0000', 'last_fullsized': '2024-09-16T07:27:47.444472+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:PG 9.b is not active+clean 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.438666+0000', 'last_change': '2024-09-16T07:27:47.438666+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.438666+0000', 'last_undegraded': '2024-09-16T07:27:47.438666+0000', 'last_fullsized': '2024-09-16T07:27:47.438666+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:PG 9.9 is not active+clean 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.439442+0000', 'last_change': '2024-09-16T07:27:47.439442+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.439442+0000', 'last_undegraded': '2024-09-16T07:27:47.439442+0000', 'last_fullsized': '2024-09-16T07:27:47.439442+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-16T07:27:53.481 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.444027+0000', 'last_change': '2024-09-16T07:27:47.444027+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.444027+0000', 'last_undegraded': '2024-09-16T07:27:47.444027+0000', 'last_fullsized': '2024-09-16T07:27:47.444027+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:PG 9.5 is not active+clean 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.438895+0000', 'last_change': '2024-09-16T07:27:47.438895+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.438895+0000', 'last_undegraded': '2024-09-16T07:27:47.438895+0000', 'last_fullsized': '2024-09-16T07:27:47.438895+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2 is not active+clean 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.444420+0000', 'last_change': '2024-09-16T07:27:47.444420+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.444420+0000', 'last_undegraded': '2024-09-16T07:27:47.444420+0000', 'last_fullsized': '2024-09-16T07:27:47.444420+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.440856+0000', 'last_change': '2024-09-16T07:27:47.440856+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.440856+0000', 'last_undegraded': '2024-09-16T07:27:47.440856+0000', 'last_fullsized': '2024-09-16T07:27:47.440856+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.442957+0000', 'last_change': '2024-09-16T07:27:47.442957+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.442957+0000', 'last_undegraded': '2024-09-16T07:27:47.442957+0000', 'last_fullsized': '2024-09-16T07:27:47.442957+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:PG 9.8 is not active+clean 2024-09-16T07:27:53.482 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:27:47.440008+0000', 'last_change': '2024-09-16T07:27:47.440008+0000', 'last_active': '2024-09-16T07:27:47.428257+0000', 'last_peered': '2024-09-16T07:27:47.428257+0000', 'last_clean': '2024-09-16T07:27:47.428257+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:27:47.440008+0000', 'last_undegraded': '2024-09-16T07:27:47.440008+0000', 'last_fullsized': '2024-09-16T07:27:47.440008+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'last_clean_scrub_stamp': '2024-09-16T07:27:47.428257+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:27:53.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T07:27:53.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.668+0000 7f6fd0c00640 1 -- 172.21.15.71:0/2295800716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fcc073de0 msgr2=0x7f6fcc074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.668+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/2295800716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fcc073de0 0x7f6fcc074240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8005fd0 tx=0x7f6fb802a540 comp rx=0 tx=0).stop 2024-09-16T07:27:53.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.668+0000 7f6fd0c00640 1 -- 172.21.15.71:0/2295800716 shutdown_connections 2024-09-16T07:27:53.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.668+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/2295800716 >> v2:172.21.15.177:3300/0 conn(0x7f6fcc06e6f0 0x7f6fcc06eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.668+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/2295800716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fcc073de0 0x7f6fcc074240 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.668+0000 7f6fd0c00640 1 -- 172.21.15.71:0/2295800716 >> 172.21.15.71:0/2295800716 conn(0x7f6fcc0b18e0 msgr2=0x7f6fcc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:53.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.669+0000 7f6fd0c00640 1 -- 172.21.15.71:0/2295800716 shutdown_connections 2024-09-16T07:27:53.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.669+0000 7f6fd0c00640 1 -- 172.21.15.71:0/2295800716 wait complete. 2024-09-16T07:27:53.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.670+0000 7f6fd0c00640 1 Processor -- start 2024-09-16T07:27:53.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.670+0000 7f6fd0c00640 1 -- start start 2024-09-16T07:27:53.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.671+0000 7f6fd0c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fcc06e6f0 0x7f6fcc14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:53.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.671+0000 7f6fd0c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fcc073de0 0x7f6fcc1534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:53.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.671+0000 7f6fd0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fcc15f9a0 0x7f6fcc0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:53.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.671+0000 7f6fd0c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc06f0b0 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.671+0000 7f6fd0c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc0cd5e0 con 0x7f6fcc06e6f0 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.671+0000 7f6fd0c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc06f230 con 0x7f6fcc073de0 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.671+0000 7f6fca800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fcc15f9a0 0x7f6fcc0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fca800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fcc15f9a0 0x7f6fcc0b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48458/0 (socket says 172.21.15.71:48458) 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fca800640 1 -- 172.21.15.71:0/1403292015 learned_addr learned my addr 172.21.15.71:0/1403292015 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fc9400640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fcc073de0 0x7f6fcc1534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fc9e00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fcc06e6f0 0x7f6fcc14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fca800640 1 -- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fcc073de0 msgr2=0x7f6fcc1534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fca800640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fcc073de0 0x7f6fcc1534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fca800640 1 -- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fcc06e6f0 msgr2=0x7f6fcc14b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fca800640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fcc06e6f0 0x7f6fcc14b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fca800640 1 -- 172.21.15.71:0/1403292015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc153d00 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fca800640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fcc15f9a0 0x7f6fcc0b39a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400eb40 tx=0x7f6fc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:53.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.672+0000 7f6fc2000640 1 -- 172.21.15.71:0/1403292015 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4004b20 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.673+0000 7f6fc2000640 1 -- 172.21.15.71:0/1403292015 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fc4004ce0 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.673+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc153fe0 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.673+0000 7f6fc2000640 1 -- 172.21.15.71:0/1403292015 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4005280 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.673+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc1545a0 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.674+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc069360 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.675+0000 7f6fc2000640 1 -- 172.21.15.71:0/1403292015 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f6fc4005420 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.676+0000 7f6fc2000640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fac0790c0 0x7f6fac07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.676+0000 7f6fc2000640 1 -- 172.21.15.71:0/1403292015 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f6fc409c850 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.677+0000 7f6fc9e00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fac0790c0 0x7f6fac07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:53.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.678+0000 7f6fc9e00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fac0790c0 0x7f6fac07b580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc001720 tx=0x7f6fbc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:53.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.681+0000 7f6fc2000640 1 -- 172.21.15.71:0/1403292015 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6fc4063850 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.934+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6fcc06eb70 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.935+0000 7f6fc2000640 1 -- 172.21.15.71:0/1403292015 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1142 (secure 0 0 0) 0x7f6fc4068280 con 0x7f6fcc15f9a0 2024-09-16T07:27:53.935 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:53.935 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":330,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":8,"num_up_osds":8,"osd_up_since":1726471357,"num_in_osds":8,"osd_in_since":1726471347,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":67},{"state_name":"creating+peering","count":30}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":592722,"bytes_used":368975872,"bytes_avail":772725137408,"bytes_total":773094113280,"inactive_pgs_ratio":0.30927833914756775,"write_bytes_sec":1032,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":37,"btime":"2024-09-16T07:27:51:464479+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":5964}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-16T07:26:48.354224+0000","services":{}},"progress_events":{}} 2024-09-16T07:27:53.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fac0790c0 msgr2=0x7f6fac07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fac0790c0 0x7f6fac07b580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc001720 tx=0x7f6fbc009040 comp rx=0 tx=0).stop 2024-09-16T07:27:53.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fcc15f9a0 msgr2=0x7f6fcc0b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:53.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fcc15f9a0 0x7f6fcc0b39a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400eb40 tx=0x7f6fc400c2d0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 shutdown_connections 2024-09-16T07:27:53.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fac0790c0 0x7f6fac07b580 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fcc15f9a0 0x7f6fcc0b39a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fcc073de0 0x7f6fcc1534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 --2- 172.21.15.71:0/1403292015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fcc06e6f0 0x7f6fcc14b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 >> 172.21.15.71:0/1403292015 conn(0x7f6fcc0b18e0 msgr2=0x7f6fcc15b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 shutdown_connections 2024-09-16T07:27:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:53.937+0000 7f6fd0c00640 1 -- 172.21.15.71:0/1403292015 wait complete. 2024-09-16T07:27:53.953 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T07:27:53.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:27:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.110+0000 7f4ef5c00640 1 -- 172.21.15.71:0/4212698387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 msgr2=0x7f4ed4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.110+0000 7f4ef5c00640 1 --2- 172.21.15.71:0/4212698387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 0x7f4ed4003ec0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0001dc0 tx=0x7f4ed4005830 comp rx=0 tx=0).stop 2024-09-16T07:27:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.110+0000 7f4ef5c00640 1 -- 172.21.15.71:0/4212698387 shutdown_connections 2024-09-16T07:27:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.110+0000 7f4ef5c00640 1 --2- 172.21.15.71:0/4212698387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 0x7f4ed4003ec0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.110+0000 7f4ef5c00640 1 -- 172.21.15.71:0/4212698387 >> 172.21.15.71:0/4212698387 conn(0x7f4ef00b1950 msgr2=0x7f4ef00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.110+0000 7f4ef5c00640 1 -- 172.21.15.71:0/4212698387 shutdown_connections 2024-09-16T07:27:54.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.110+0000 7f4ef5c00640 1 -- 172.21.15.71:0/4212698387 wait complete. 2024-09-16T07:27:54.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.112+0000 7f4ef5c00640 1 Processor -- start 2024-09-16T07:27:54.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.112+0000 7f4ef5c00640 1 -- start start 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4ef5c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 0x7f4ef01f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4ef5c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef00cb9d0 0x7f4ef014d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4ef5c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ef00ca670 0x7f4ef00cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4ef5c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef0074420 con 0x7f4ed4003aa0 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4ef5c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef00b2700 con 0x7f4ef00cb9d0 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4ef5c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef00745a0 con 0x7f4ef00ca670 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4eeea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 0x7f4ef01f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4eeea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 0x7f4ef01f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48472/0 (socket says 172.21.15.71:48472) 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4eeea00640 1 -- 172.21.15.71:0/1616659642 learned_addr learned my addr 172.21.15.71:0/1616659642 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4eee000640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef00cb9d0 0x7f4ef014d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.113+0000 7f4eef400640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ef00ca670 0x7f4ef00cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:54.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4eeea00640 1 -- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ef00ca670 msgr2=0x7f4ef00cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4eeea00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ef00ca670 0x7f4ef00cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4eeea00640 1 -- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef00cb9d0 msgr2=0x7f4ef014d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4eeea00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef00cb9d0 0x7f4ef014d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4eeea00640 1 -- 172.21.15.71:0/1616659642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef00cbe50 con 0x7f4ed4003aa0 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4eeea00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 0x7f4ef01f53f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000f80 tx=0x7f4ee0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4ee6a00640 1 -- 172.21.15.71:0/1616659642 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0006460 con 0x7f4ed4003aa0 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4ee6a00640 1 -- 172.21.15.71:0/1616659642 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ee0006620 con 0x7f4ed4003aa0 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef014da40 con 0x7f4ed4003aa0 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4ee6a00640 1 -- 172.21.15.71:0/1616659642 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0033480 con 0x7f4ed4003aa0 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.114+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef014e000 con 0x7f4ed4003aa0 2024-09-16T07:27:54.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.115+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef00acd30 con 0x7f4ed4003aa0 2024-09-16T07:27:54.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.116+0000 7f4ee6a00640 1 -- 172.21.15.71:0/1616659642 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f4ee0033640 con 0x7f4ed4003aa0 2024-09-16T07:27:54.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.117+0000 7f4ee6a00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4ed00790f0 0x7f4ed007b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.117+0000 7f4ee6a00640 1 -- 172.21.15.71:0/1616659642 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f4ee00bb4e0 con 0x7f4ed4003aa0 2024-09-16T07:27:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.118+0000 7f4eee000640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4ed00790f0 0x7f4ed007b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:54.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.118+0000 7f4eee000640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4ed00790f0 0x7f4ed007b5b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ca5e0 tx=0x7f4edc005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:54.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.121+0000 7f4ee6a00640 1 -- 172.21.15.71:0/1616659642 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4ee0083080 con 0x7f4ed4003aa0 2024-09-16T07:27:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.303+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ef014d7e0 con 0x7f4ed00790f0 2024-09-16T07:27:54.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.308+0000 7f4ee6a00640 1 -- 172.21.15.71:0/1616659642 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249482 (secure 0 0 0) 0x7f4edc05dea0 con 0x7f4ed00790f0 2024-09-16T07:27:54.311 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:54.312 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:27:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.311+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4ed00790f0 msgr2=0x7f4ed007b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.311+0000 7f4ef5c00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4ed00790f0 0x7f4ed007b5b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00ca5e0 tx=0x7f4edc005f90 comp rx=0 tx=0).stop 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.311+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 msgr2=0x7f4ef01f53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.311+0000 7f4ef5c00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 0x7f4ef01f53f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000f80 tx=0x7f4ee0003d40 comp rx=0 tx=0).stop 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.312+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 shutdown_connections 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.312+0000 7f4ef5c00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4ed00790f0 0x7f4ed007b5b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.312+0000 7f4ef5c00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ef00ca670 0x7f4ef00cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.312+0000 7f4ef5c00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ef00cb9d0 0x7f4ef014d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.312+0000 7f4ef5c00640 1 --2- 172.21.15.71:0/1616659642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ed4003aa0 0x7f4ef01f53f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.312+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 >> 172.21.15.71:0/1616659642 conn(0x7f4ef00b1950 msgr2=0x7f4ef00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.312+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 shutdown_connections 2024-09-16T07:27:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:54.312+0000 7f4ef5c00640 1 -- 172.21.15.71:0/1616659642 wait complete. 2024-09-16T07:27:54.328 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-16T07:27:52.380524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":132,"num_write_kb":1634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":360328,"kb_used_data":6736,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754614392,"statfs":{"total":773094113280,"available":772725137408,"internally_reserved":0,"allocated":6897664,"data_stored":3551176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":361877040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.951511"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.453401+0000","last_change":"2024-09-16T07:27:47.453401+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.453401+0000","last_undegraded":"2024-09-16T07:27:47.453401+0000","last_fullsized":"2024-09-16T07:27:47.453401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444099+0000","last_change":"2024-09-16T07:27:47.444099+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444099+0000","last_undegraded":"2024-09-16T07:27:47.444099+0000","last_fullsized":"2024-09-16T07:27:47.444099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.463111+0000","last_change":"2024-09-16T07:27:48.453492+0000","last_active":"2024-09-16T07:27:48.463111+0000","last_peered":"2024-09-16T07:27:48.463111+0000","last_clean":"2024-09-16T07:27:48.463111+0000","last_became_active":"2024-09-16T07:27:48.453262+0000","last_became_peered":"2024-09-16T07:27:48.453262+0000","last_unstale":"2024-09-16T07:27:48.463111+0000","last_undegraded":"2024-09-16T07:27:48.463111+0000","last_fullsized":"2024-09-16T07:27:48.463111+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:50.447717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469342+0000","last_change":"2024-09-16T07:27:48.460353+0000","last_active":"2024-09-16T07:27:48.469342+0000","last_peered":"2024-09-16T07:27:48.469342+0000","last_clean":"2024-09-16T07:27:48.469342+0000","last_became_active":"2024-09-16T07:27:48.459644+0000","last_became_peered":"2024-09-16T07:27:48.459644+0000","last_unstale":"2024-09-16T07:27:48.469342+0000","last_undegraded":"2024-09-16T07:27:48.469342+0000","last_fullsized":"2024-09-16T07:27:48.469342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:00.444857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.451622+0000","last_change":"2024-09-16T07:27:47.451622+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.451622+0000","last_undegraded":"2024-09-16T07:27:47.451622+0000","last_fullsized":"2024-09-16T07:27:47.451622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.447931+0000","last_change":"2024-09-16T07:27:47.447931+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.447931+0000","last_undegraded":"2024-09-16T07:27:47.447931+0000","last_fullsized":"2024-09-16T07:27:47.447931+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.471918+0000","last_change":"2024-09-16T07:27:48.462460+0000","last_active":"2024-09-16T07:27:48.471918+0000","last_peered":"2024-09-16T07:27:48.471918+0000","last_clean":"2024-09-16T07:27:48.471918+0000","last_became_active":"2024-09-16T07:27:48.462193+0000","last_became_peered":"2024-09-16T07:27:48.462193+0000","last_unstale":"2024-09-16T07:27:48.471918+0000","last_undegraded":"2024-09-16T07:27:48.471918+0000","last_fullsized":"2024-09-16T07:27:48.471918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:03:02.929085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441520+0000","last_change":"2024-09-16T07:27:47.441520+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441520+0000","last_undegraded":"2024-09-16T07:27:47.441520+0000","last_fullsized":"2024-09-16T07:27:47.441520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469491+0000","last_change":"2024-09-16T07:27:48.461060+0000","last_active":"2024-09-16T07:27:48.469491+0000","last_peered":"2024-09-16T07:27:48.469491+0000","last_clean":"2024-09-16T07:27:48.469491+0000","last_became_active":"2024-09-16T07:27:48.460770+0000","last_became_peered":"2024-09-16T07:27:48.460770+0000","last_unstale":"2024-09-16T07:27:48.469491+0000","last_undegraded":"2024-09-16T07:27:48.469491+0000","last_fullsized":"2024-09-16T07:27:48.469491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:46.741293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.452048+0000","last_change":"2024-09-16T07:27:47.452048+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.452048+0000","last_undegraded":"2024-09-16T07:27:47.452048+0000","last_fullsized":"2024-09-16T07:27:47.452048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461558+0000","last_change":"2024-09-16T07:27:48.459130+0000","last_active":"2024-09-16T07:27:49.461558+0000","last_peered":"2024-09-16T07:27:49.461558+0000","last_clean":"2024-09-16T07:27:49.461558+0000","last_became_active":"2024-09-16T07:27:48.458360+0000","last_became_peered":"2024-09-16T07:27:48.458360+0000","last_unstale":"2024-09-16T07:27:49.461558+0000","last_undegraded":"2024-09-16T07:27:49.461558+0000","last_fullsized":"2024-09-16T07:27:49.461558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:42.756027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468989+0000","last_change":"2024-09-16T07:27:48.462533+0000","last_active":"2024-09-16T07:27:48.468989+0000","last_peered":"2024-09-16T07:27:48.468989+0000","last_clean":"2024-09-16T07:27:48.468989+0000","last_became_active":"2024-09-16T07:27:48.462296+0000","last_became_peered":"2024-09-16T07:27:48.462296+0000","last_unstale":"2024-09-16T07:27:48.468989+0000","last_undegraded":"2024-09-16T07:27:48.468989+0000","last_fullsized":"2024-09-16T07:27:48.468989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:25.012988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466007+0000","last_change":"2024-09-16T07:27:48.461185+0000","last_active":"2024-09-16T07:27:48.466007+0000","last_peered":"2024-09-16T07:27:48.466007+0000","last_clean":"2024-09-16T07:27:48.466007+0000","last_became_active":"2024-09-16T07:27:48.460889+0000","last_became_peered":"2024-09-16T07:27:48.460889+0000","last_unstale":"2024-09-16T07:27:48.466007+0000","last_undegraded":"2024-09-16T07:27:48.466007+0000","last_fullsized":"2024-09-16T07:27:48.466007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:32:32.356185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.464924+0000","last_change":"2024-09-16T07:27:48.458531+0000","last_active":"2024-09-16T07:27:48.464924+0000","last_peered":"2024-09-16T07:27:48.464924+0000","last_clean":"2024-09-16T07:27:48.464924+0000","last_became_active":"2024-09-16T07:27:48.458285+0000","last_became_peered":"2024-09-16T07:27:48.458285+0000","last_unstale":"2024-09-16T07:27:48.464924+0000","last_undegraded":"2024-09-16T07:27:48.464924+0000","last_fullsized":"2024-09-16T07:27:48.464924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:40.253906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469844+0000","last_change":"2024-09-16T07:27:48.466279+0000","last_active":"2024-09-16T07:27:48.469844+0000","last_peered":"2024-09-16T07:27:48.469844+0000","last_clean":"2024-09-16T07:27:48.469844+0000","last_became_active":"2024-09-16T07:27:48.466000+0000","last_became_peered":"2024-09-16T07:27:48.466000+0000","last_unstale":"2024-09-16T07:27:48.469844+0000","last_undegraded":"2024-09-16T07:27:48.469844+0000","last_fullsized":"2024-09-16T07:27:48.469844+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:28.032448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449957+0000","last_change":"2024-09-16T07:27:47.449957+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449957+0000","last_undegraded":"2024-09-16T07:27:47.449957+0000","last_fullsized":"2024-09-16T07:27:47.449957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449887+0000","last_change":"2024-09-16T07:27:47.449887+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449887+0000","last_undegraded":"2024-09-16T07:27:47.449887+0000","last_fullsized":"2024-09-16T07:27:47.449887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467286+0000","last_change":"2024-09-16T07:27:48.465698+0000","last_active":"2024-09-16T07:27:48.467286+0000","last_peered":"2024-09-16T07:27:48.467286+0000","last_clean":"2024-09-16T07:27:48.467286+0000","last_became_active":"2024-09-16T07:27:48.465355+0000","last_became_peered":"2024-09-16T07:27:48.465355+0000","last_unstale":"2024-09-16T07:27:48.467286+0000","last_undegraded":"2024-09-16T07:27:48.467286+0000","last_fullsized":"2024-09-16T07:27:48.467286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:59.562068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.447615+0000","last_change":"2024-09-16T07:27:47.447615+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.447615+0000","last_undegraded":"2024-09-16T07:27:47.447615+0000","last_fullsized":"2024-09-16T07:27:47.447615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461494+0000","last_change":"2024-09-16T07:27:48.460038+0000","last_active":"2024-09-16T07:27:49.461494+0000","last_peered":"2024-09-16T07:27:49.461494+0000","last_clean":"2024-09-16T07:27:49.461494+0000","last_became_active":"2024-09-16T07:27:48.459755+0000","last_became_peered":"2024-09-16T07:27:48.459755+0000","last_unstale":"2024-09-16T07:27:49.461494+0000","last_undegraded":"2024-09-16T07:27:49.461494+0000","last_fullsized":"2024-09-16T07:27:49.461494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:37.822664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467196+0000","last_change":"2024-09-16T07:27:48.464651+0000","last_active":"2024-09-16T07:27:48.467196+0000","last_peered":"2024-09-16T07:27:48.467196+0000","last_clean":"2024-09-16T07:27:48.467196+0000","last_became_active":"2024-09-16T07:27:48.464373+0000","last_became_peered":"2024-09-16T07:27:48.464373+0000","last_unstale":"2024-09-16T07:27:48.467196+0000","last_undegraded":"2024-09-16T07:27:48.467196+0000","last_fullsized":"2024-09-16T07:27:48.467196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:34.767968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461319+0000","last_change":"2024-09-16T07:27:48.461614+0000","last_active":"2024-09-16T07:27:49.461319+0000","last_peered":"2024-09-16T07:27:49.461319+0000","last_clean":"2024-09-16T07:27:49.461319+0000","last_became_active":"2024-09-16T07:27:48.461333+0000","last_became_peered":"2024-09-16T07:27:48.461333+0000","last_unstale":"2024-09-16T07:27:49.461319+0000","last_undegraded":"2024-09-16T07:27:49.461319+0000","last_fullsized":"2024-09-16T07:27:49.461319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:14.430788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.477576+0000","last_change":"2024-09-16T07:27:48.469453+0000","last_active":"2024-09-16T07:27:48.477576+0000","last_peered":"2024-09-16T07:27:48.477576+0000","last_clean":"2024-09-16T07:27:48.477576+0000","last_became_active":"2024-09-16T07:27:48.466391+0000","last_became_peered":"2024-09-16T07:27:48.466391+0000","last_unstale":"2024-09-16T07:27:48.477576+0000","last_undegraded":"2024-09-16T07:27:48.477576+0000","last_fullsized":"2024-09-16T07:27:48.477576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:26.749995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468874+0000","last_change":"2024-09-16T07:27:48.461655+0000","last_active":"2024-09-16T07:27:48.468874+0000","last_peered":"2024-09-16T07:27:48.468874+0000","last_clean":"2024-09-16T07:27:48.468874+0000","last_became_active":"2024-09-16T07:27:48.461371+0000","last_became_peered":"2024-09-16T07:27:48.461371+0000","last_unstale":"2024-09-16T07:27:48.468874+0000","last_undegraded":"2024-09-16T07:27:48.468874+0000","last_fullsized":"2024-09-16T07:27:48.468874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:14.719547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444900+0000","last_change":"2024-09-16T07:27:47.444900+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444900+0000","last_undegraded":"2024-09-16T07:27:47.444900+0000","last_fullsized":"2024-09-16T07:27:47.444900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.459564+0000","last_change":"2024-09-16T07:27:48.455880+0000","last_active":"2024-09-16T07:27:48.459564+0000","last_peered":"2024-09-16T07:27:48.459564+0000","last_clean":"2024-09-16T07:27:48.459564+0000","last_became_active":"2024-09-16T07:27:48.455633+0000","last_became_peered":"2024-09-16T07:27:48.455633+0000","last_unstale":"2024-09-16T07:27:48.459564+0000","last_undegraded":"2024-09-16T07:27:48.459564+0000","last_fullsized":"2024-09-16T07:27:48.459564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:56.366154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.446950+0000","last_change":"2024-09-16T07:27:47.446950+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.446950+0000","last_undegraded":"2024-09-16T07:27:47.446950+0000","last_fullsized":"2024-09-16T07:27:47.446950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441948+0000","last_change":"2024-09-16T07:27:47.441948+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441948+0000","last_undegraded":"2024-09-16T07:27:47.441948+0000","last_fullsized":"2024-09-16T07:27:47.441948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.439145+0000","last_change":"2024-09-16T07:27:47.439145+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.439145+0000","last_undegraded":"2024-09-16T07:27:47.439145+0000","last_fullsized":"2024-09-16T07:27:47.439145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.472115+0000","last_change":"2024-09-16T07:27:48.462181+0000","last_active":"2024-09-16T07:27:48.472115+0000","last_peered":"2024-09-16T07:27:48.472115+0000","last_clean":"2024-09-16T07:27:48.472115+0000","last_became_active":"2024-09-16T07:27:48.460601+0000","last_became_peered":"2024-09-16T07:27:48.460601+0000","last_unstale":"2024-09-16T07:27:48.472115+0000","last_undegraded":"2024-09-16T07:27:48.472115+0000","last_fullsized":"2024-09-16T07:27:48.472115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:18.404077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.443479+0000","last_change":"2024-09-16T07:27:47.443479+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.443479+0000","last_undegraded":"2024-09-16T07:27:47.443479+0000","last_fullsized":"2024-09-16T07:27:47.443479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461161+0000","last_change":"2024-09-16T07:27:48.460512+0000","last_active":"2024-09-16T07:27:49.461161+0000","last_peered":"2024-09-16T07:27:49.461161+0000","last_clean":"2024-09-16T07:27:49.461161+0000","last_became_active":"2024-09-16T07:27:48.460276+0000","last_became_peered":"2024-09-16T07:27:48.460276+0000","last_unstale":"2024-09-16T07:27:49.461161+0000","last_undegraded":"2024-09-16T07:27:49.461161+0000","last_fullsized":"2024-09-16T07:27:49.461161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.223196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.446917+0000","last_change":"2024-09-16T07:27:47.446917+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.446917+0000","last_undegraded":"2024-09-16T07:27:47.446917+0000","last_fullsized":"2024-09-16T07:27:47.446917+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461617+0000","last_change":"2024-09-16T07:27:47.450308+0000","last_active":"2024-09-16T07:27:49.461617+0000","last_peered":"2024-09-16T07:27:49.461617+0000","last_clean":"2024-09-16T07:27:49.461617+0000","last_became_active":"2024-09-16T07:27:47.448332+0000","last_became_peered":"2024-09-16T07:27:47.448332+0000","last_unstale":"2024-09-16T07:27:49.461617+0000","last_undegraded":"2024-09-16T07:27:49.461617+0000","last_fullsized":"2024-09-16T07:27:49.461617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:38.412742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.449454+0000","last_change":"2024-09-16T07:27:47.449454+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.449454+0000","last_undegraded":"2024-09-16T07:27:47.449454+0000","last_fullsized":"2024-09-16T07:27:47.449454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.454190+0000","last_change":"2024-09-16T07:27:47.452837+0000","last_active":"2024-09-16T07:27:47.454190+0000","last_peered":"2024-09-16T07:27:47.454190+0000","last_clean":"2024-09-16T07:27:47.454190+0000","last_became_active":"2024-09-16T07:27:47.452517+0000","last_became_peered":"2024-09-16T07:27:47.452517+0000","last_unstale":"2024-09-16T07:27:47.454190+0000","last_undegraded":"2024-09-16T07:27:47.454190+0000","last_fullsized":"2024-09-16T07:27:47.454190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:50.738735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.441487+0000","last_change":"2024-09-16T07:27:47.441487+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.441487+0000","last_undegraded":"2024-09-16T07:27:47.441487+0000","last_fullsized":"2024-09-16T07:27:47.441487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460327+0000","last_change":"2024-09-16T07:27:47.459254+0000","last_active":"2024-09-16T07:27:47.460327+0000","last_peered":"2024-09-16T07:27:47.460327+0000","last_clean":"2024-09-16T07:27:47.460327+0000","last_became_active":"2024-09-16T07:27:47.458912+0000","last_became_peered":"2024-09-16T07:27:47.458912+0000","last_unstale":"2024-09-16T07:27:47.460327+0000","last_undegraded":"2024-09-16T07:27:47.460327+0000","last_fullsized":"2024-09-16T07:27:47.460327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:07:13.520726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444836+0000","last_change":"2024-09-16T07:27:47.444836+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444836+0000","last_undegraded":"2024-09-16T07:27:47.444836+0000","last_fullsized":"2024-09-16T07:27:47.444836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.478412+0000","last_change":"2024-09-16T07:27:47.458877+0000","last_active":"2024-09-16T07:27:48.478412+0000","last_peered":"2024-09-16T07:27:48.478412+0000","last_clean":"2024-09-16T07:27:48.478412+0000","last_became_active":"2024-09-16T07:27:47.458615+0000","last_became_peered":"2024-09-16T07:27:47.458615+0000","last_unstale":"2024-09-16T07:27:48.478412+0000","last_undegraded":"2024-09-16T07:27:48.478412+0000","last_fullsized":"2024-09-16T07:27:48.478412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:00.779149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438546+0000","last_change":"2024-09-16T07:27:47.438546+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438546+0000","last_undegraded":"2024-09-16T07:27:47.438546+0000","last_fullsized":"2024-09-16T07:27:47.438546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.471392+0000","last_change":"2024-09-16T07:27:47.455983+0000","last_active":"2024-09-16T07:27:48.471392+0000","last_peered":"2024-09-16T07:27:48.471392+0000","last_clean":"2024-09-16T07:27:48.471392+0000","last_became_active":"2024-09-16T07:27:47.454631+0000","last_became_peered":"2024-09-16T07:27:47.454631+0000","last_unstale":"2024-09-16T07:27:48.471392+0000","last_undegraded":"2024-09-16T07:27:48.471392+0000","last_fullsized":"2024-09-16T07:27:48.471392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:37.670315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461455+0000","last_change":"2024-09-16T07:27:48.457180+0000","last_active":"2024-09-16T07:27:49.461455+0000","last_peered":"2024-09-16T07:27:49.461455+0000","last_clean":"2024-09-16T07:27:49.461455+0000","last_became_active":"2024-09-16T07:27:48.456834+0000","last_became_peered":"2024-09-16T07:27:48.456834+0000","last_unstale":"2024-09-16T07:27:49.461455+0000","last_undegraded":"2024-09-16T07:27:49.461455+0000","last_fullsized":"2024-09-16T07:27:49.461455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:04.346053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464365+0000","last_change":"2024-09-16T07:27:47.461641+0000","last_active":"2024-09-16T07:27:47.464365+0000","last_peered":"2024-09-16T07:27:47.464365+0000","last_clean":"2024-09-16T07:27:47.464365+0000","last_became_active":"2024-09-16T07:27:47.461318+0000","last_became_peered":"2024-09-16T07:27:47.461318+0000","last_unstale":"2024-09-16T07:27:47.464365+0000","last_undegraded":"2024-09-16T07:27:47.464365+0000","last_fullsized":"2024-09-16T07:27:47.464365+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:10:51.784199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.465633+0000","last_change":"2024-09-16T07:27:48.455278+0000","last_active":"2024-09-16T07:27:48.465633+0000","last_peered":"2024-09-16T07:27:48.465633+0000","last_clean":"2024-09-16T07:27:48.465633+0000","last_became_active":"2024-09-16T07:27:48.454871+0000","last_became_peered":"2024-09-16T07:27:48.454871+0000","last_unstale":"2024-09-16T07:27:48.465633+0000","last_undegraded":"2024-09-16T07:27:48.465633+0000","last_fullsized":"2024-09-16T07:27:48.465633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:42.196208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443144+0000","last_change":"2024-09-16T07:27:47.452265+0000","last_active":"2024-09-16T07:27:48.443144+0000","last_peered":"2024-09-16T07:27:48.443144+0000","last_clean":"2024-09-16T07:27:48.443144+0000","last_became_active":"2024-09-16T07:27:47.452018+0000","last_became_peered":"2024-09-16T07:27:47.452018+0000","last_unstale":"2024-09-16T07:27:48.443144+0000","last_undegraded":"2024-09-16T07:27:48.443144+0000","last_fullsized":"2024-09-16T07:27:48.443144+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:35.000221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.448499+0000","last_change":"2024-09-16T07:27:47.448499+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.448499+0000","last_undegraded":"2024-09-16T07:27:47.448499+0000","last_fullsized":"2024-09-16T07:27:47.448499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.449811+0000","last_change":"2024-09-16T07:27:47.447783+0000","last_active":"2024-09-16T07:27:48.449811+0000","last_peered":"2024-09-16T07:27:48.449811+0000","last_clean":"2024-09-16T07:27:48.449811+0000","last_became_active":"2024-09-16T07:27:47.447510+0000","last_became_peered":"2024-09-16T07:27:47.447510+0000","last_unstale":"2024-09-16T07:27:48.449811+0000","last_undegraded":"2024-09-16T07:27:48.449811+0000","last_fullsized":"2024-09-16T07:27:48.449811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:14.370496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.459535+0000","last_change":"2024-09-16T07:27:48.454744+0000","last_active":"2024-09-16T07:27:48.459535+0000","last_peered":"2024-09-16T07:27:48.459535+0000","last_clean":"2024-09-16T07:27:48.459535+0000","last_became_active":"2024-09-16T07:27:48.454446+0000","last_became_peered":"2024-09-16T07:27:48.454446+0000","last_unstale":"2024-09-16T07:27:48.459535+0000","last_undegraded":"2024-09-16T07:27:48.459535+0000","last_fullsized":"2024-09-16T07:27:48.459535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:52.733528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443637+0000","last_change":"2024-09-16T07:27:47.450306+0000","last_active":"2024-09-16T07:27:48.443637+0000","last_peered":"2024-09-16T07:27:48.443637+0000","last_clean":"2024-09-16T07:27:48.443637+0000","last_became_active":"2024-09-16T07:27:47.449275+0000","last_became_peered":"2024-09-16T07:27:47.449275+0000","last_unstale":"2024-09-16T07:27:48.443637+0000","last_undegraded":"2024-09-16T07:27:48.443637+0000","last_fullsized":"2024-09-16T07:27:48.443637+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:00.479157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.470005+0000","last_change":"2024-09-16T07:27:48.468331+0000","last_active":"2024-09-16T07:27:48.470005+0000","last_peered":"2024-09-16T07:27:48.470005+0000","last_clean":"2024-09-16T07:27:48.470005+0000","last_became_active":"2024-09-16T07:27:48.468040+0000","last_became_peered":"2024-09-16T07:27:48.468040+0000","last_unstale":"2024-09-16T07:27:48.470005+0000","last_undegraded":"2024-09-16T07:27:48.470005+0000","last_fullsized":"2024-09-16T07:27:48.470005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:06.084517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453197+0000","last_change":"2024-09-16T07:27:47.450498+0000","last_active":"2024-09-16T07:27:47.453197+0000","last_peered":"2024-09-16T07:27:47.453197+0000","last_clean":"2024-09-16T07:27:47.453197+0000","last_became_active":"2024-09-16T07:27:47.449615+0000","last_became_peered":"2024-09-16T07:27:47.449615+0000","last_unstale":"2024-09-16T07:27:47.453197+0000","last_undegraded":"2024-09-16T07:27:47.453197+0000","last_fullsized":"2024-09-16T07:27:47.453197+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:23:34.664293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438573+0000","last_change":"2024-09-16T07:27:47.438573+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438573+0000","last_undegraded":"2024-09-16T07:27:47.438573+0000","last_fullsized":"2024-09-16T07:27:47.438573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.478154+0000","last_change":"2024-09-16T07:27:47.460337+0000","last_active":"2024-09-16T07:27:48.478154+0000","last_peered":"2024-09-16T07:27:48.478154+0000","last_clean":"2024-09-16T07:27:48.478154+0000","last_became_active":"2024-09-16T07:27:47.460023+0000","last_became_peered":"2024-09-16T07:27:47.460023+0000","last_unstale":"2024-09-16T07:27:48.478154+0000","last_undegraded":"2024-09-16T07:27:48.478154+0000","last_fullsized":"2024-09-16T07:27:48.478154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:52.763400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.456699+0000","last_change":"2024-09-16T07:27:48.449778+0000","last_active":"2024-09-16T07:27:48.456699+0000","last_peered":"2024-09-16T07:27:48.456699+0000","last_clean":"2024-09-16T07:27:48.456699+0000","last_became_active":"2024-09-16T07:27:48.449504+0000","last_became_peered":"2024-09-16T07:27:48.449504+0000","last_unstale":"2024-09-16T07:27:48.456699+0000","last_undegraded":"2024-09-16T07:27:48.456699+0000","last_fullsized":"2024-09-16T07:27:48.456699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:48.419517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.480574+0000","last_change":"2024-09-16T07:27:47.458310+0000","last_active":"2024-09-16T07:27:48.480574+0000","last_peered":"2024-09-16T07:27:48.480574+0000","last_clean":"2024-09-16T07:27:48.480574+0000","last_became_active":"2024-09-16T07:27:47.457920+0000","last_became_peered":"2024-09-16T07:27:47.457920+0000","last_unstale":"2024-09-16T07:27:48.480574+0000","last_undegraded":"2024-09-16T07:27:48.480574+0000","last_fullsized":"2024-09-16T07:27:48.480574+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.526953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.469560+0000","last_change":"2024-09-16T07:27:48.467738+0000","last_active":"2024-09-16T07:27:48.469560+0000","last_peered":"2024-09-16T07:27:48.469560+0000","last_clean":"2024-09-16T07:27:48.469560+0000","last_became_active":"2024-09-16T07:27:48.467296+0000","last_became_peered":"2024-09-16T07:27:48.467296+0000","last_unstale":"2024-09-16T07:27:48.469560+0000","last_undegraded":"2024-09-16T07:27:48.469560+0000","last_fullsized":"2024-09-16T07:27:48.469560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:08.840299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453240+0000","last_change":"2024-09-16T07:27:47.449886+0000","last_active":"2024-09-16T07:27:47.453240+0000","last_peered":"2024-09-16T07:27:47.453240+0000","last_clean":"2024-09-16T07:27:47.453240+0000","last_became_active":"2024-09-16T07:27:47.449599+0000","last_became_peered":"2024-09-16T07:27:47.449599+0000","last_unstale":"2024-09-16T07:27:47.453240+0000","last_undegraded":"2024-09-16T07:27:47.453240+0000","last_fullsized":"2024-09-16T07:27:47.453240+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:45:32.473190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466091+0000","last_change":"2024-09-16T07:27:48.453257+0000","last_active":"2024-09-16T07:27:48.466091+0000","last_peered":"2024-09-16T07:27:48.466091+0000","last_clean":"2024-09-16T07:27:48.466091+0000","last_became_active":"2024-09-16T07:27:48.453020+0000","last_became_peered":"2024-09-16T07:27:48.453020+0000","last_unstale":"2024-09-16T07:27:48.466091+0000","last_undegraded":"2024-09-16T07:27:48.466091+0000","last_fullsized":"2024-09-16T07:27:48.466091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:07.527458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.439529+0000","last_change":"2024-09-16T07:27:47.449870+0000","last_active":"2024-09-16T07:27:48.439529+0000","last_peered":"2024-09-16T07:27:48.439529+0000","last_clean":"2024-09-16T07:27:48.439529+0000","last_became_active":"2024-09-16T07:27:47.449587+0000","last_became_peered":"2024-09-16T07:27:47.449587+0000","last_unstale":"2024-09-16T07:27:48.439529+0000","last_undegraded":"2024-09-16T07:27:48.439529+0000","last_fullsized":"2024-09-16T07:27:48.439529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:56.658575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468654+0000","last_change":"2024-09-16T07:27:48.457526+0000","last_active":"2024-09-16T07:27:48.468654+0000","last_peered":"2024-09-16T07:27:48.468654+0000","last_clean":"2024-09-16T07:27:48.468654+0000","last_became_active":"2024-09-16T07:27:48.457292+0000","last_became_peered":"2024-09-16T07:27:48.457292+0000","last_unstale":"2024-09-16T07:27:48.468654+0000","last_undegraded":"2024-09-16T07:27:48.468654+0000","last_fullsized":"2024-09-16T07:27:48.468654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:15:18.042402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.439349+0000","last_change":"2024-09-16T07:27:47.450647+0000","last_active":"2024-09-16T07:27:48.439349+0000","last_peered":"2024-09-16T07:27:48.439349+0000","last_clean":"2024-09-16T07:27:48.439349+0000","last_became_active":"2024-09-16T07:27:47.450313+0000","last_became_peered":"2024-09-16T07:27:47.450313+0000","last_unstale":"2024-09-16T07:27:48.439349+0000","last_undegraded":"2024-09-16T07:27:48.439349+0000","last_fullsized":"2024-09-16T07:27:48.439349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:01.115128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.467730+0000","last_change":"2024-09-16T07:27:48.459503+0000","last_active":"2024-09-16T07:27:48.467730+0000","last_peered":"2024-09-16T07:27:48.467730+0000","last_clean":"2024-09-16T07:27:48.467730+0000","last_became_active":"2024-09-16T07:27:48.459201+0000","last_became_peered":"2024-09-16T07:27:48.459201+0000","last_unstale":"2024-09-16T07:27:48.467730+0000","last_undegraded":"2024-09-16T07:27:48.467730+0000","last_fullsized":"2024-09-16T07:27:48.467730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:48:35.394756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.482800+0000","last_change":"2024-09-16T07:27:47.451210+0000","last_active":"2024-09-16T07:27:48.482800+0000","last_peered":"2024-09-16T07:27:48.482800+0000","last_clean":"2024-09-16T07:27:48.482800+0000","last_became_active":"2024-09-16T07:27:47.450925+0000","last_became_peered":"2024-09-16T07:27:47.450925+0000","last_unstale":"2024-09-16T07:27:48.482800+0000","last_undegraded":"2024-09-16T07:27:48.482800+0000","last_fullsized":"2024-09-16T07:27:48.482800+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:03.229023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.464941+0000","last_change":"2024-09-16T07:27:48.457648+0000","last_active":"2024-09-16T07:27:48.464941+0000","last_peered":"2024-09-16T07:27:48.464941+0000","last_clean":"2024-09-16T07:27:48.464941+0000","last_became_active":"2024-09-16T07:27:48.457358+0000","last_became_peered":"2024-09-16T07:27:48.457358+0000","last_unstale":"2024-09-16T07:27:48.464941+0000","last_undegraded":"2024-09-16T07:27:48.464941+0000","last_fullsized":"2024-09-16T07:27:48.464941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:28.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460425+0000","last_change":"2024-09-16T07:27:47.458791+0000","last_active":"2024-09-16T07:27:47.460425+0000","last_peered":"2024-09-16T07:27:47.460425+0000","last_clean":"2024-09-16T07:27:47.460425+0000","last_became_active":"2024-09-16T07:27:47.458560+0000","last_became_peered":"2024-09-16T07:27:47.458560+0000","last_unstale":"2024-09-16T07:27:47.460425+0000","last_undegraded":"2024-09-16T07:27:47.460425+0000","last_fullsized":"2024-09-16T07:27:47.460425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:15.540429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468200+0000","last_change":"2024-09-16T07:27:48.460172+0000","last_active":"2024-09-16T07:27:48.468200+0000","last_peered":"2024-09-16T07:27:48.468200+0000","last_clean":"2024-09-16T07:27:48.468200+0000","last_became_active":"2024-09-16T07:27:48.459930+0000","last_became_peered":"2024-09-16T07:27:48.459930+0000","last_unstale":"2024-09-16T07:27:48.468200+0000","last_undegraded":"2024-09-16T07:27:48.468200+0000","last_fullsized":"2024-09-16T07:27:48.468200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:04.739035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.453174+0000","last_change":"2024-09-16T07:27:47.450135+0000","last_active":"2024-09-16T07:27:47.453174+0000","last_peered":"2024-09-16T07:27:47.453174+0000","last_clean":"2024-09-16T07:27:47.453174+0000","last_became_active":"2024-09-16T07:27:47.449254+0000","last_became_peered":"2024-09-16T07:27:47.449254+0000","last_unstale":"2024-09-16T07:27:47.453174+0000","last_undegraded":"2024-09-16T07:27:47.453174+0000","last_fullsized":"2024-09-16T07:27:47.453174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:16.114777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444472+0000","last_change":"2024-09-16T07:27:47.444472+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444472+0000","last_undegraded":"2024-09-16T07:27:47.444472+0000","last_fullsized":"2024-09-16T07:27:47.444472+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.476242+0000","last_change":"2024-09-16T07:27:47.455984+0000","last_active":"2024-09-16T07:27:48.476242+0000","last_peered":"2024-09-16T07:27:48.476242+0000","last_clean":"2024-09-16T07:27:48.476242+0000","last_became_active":"2024-09-16T07:27:47.454791+0000","last_became_peered":"2024-09-16T07:27:47.454791+0000","last_unstale":"2024-09-16T07:27:48.476242+0000","last_undegraded":"2024-09-16T07:27:48.476242+0000","last_fullsized":"2024-09-16T07:27:48.476242+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:24.745832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.472184+0000","last_change":"2024-09-16T07:27:48.462451+0000","last_active":"2024-09-16T07:27:48.472184+0000","last_peered":"2024-09-16T07:27:48.472184+0000","last_clean":"2024-09-16T07:27:48.472184+0000","last_became_active":"2024-09-16T07:27:48.462145+0000","last_became_peered":"2024-09-16T07:27:48.462145+0000","last_unstale":"2024-09-16T07:27:48.472184+0000","last_undegraded":"2024-09-16T07:27:48.472184+0000","last_fullsized":"2024-09-16T07:27:48.472184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:25:02.985426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460913+0000","last_change":"2024-09-16T07:27:47.459943+0000","last_active":"2024-09-16T07:27:47.460913+0000","last_peered":"2024-09-16T07:27:47.460913+0000","last_clean":"2024-09-16T07:27:47.460913+0000","last_became_active":"2024-09-16T07:27:47.459680+0000","last_became_peered":"2024-09-16T07:27:47.459680+0000","last_unstale":"2024-09-16T07:27:47.460913+0000","last_undegraded":"2024-09-16T07:27:47.460913+0000","last_fullsized":"2024-09-16T07:27:47.460913+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:44.061784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.468987+0000","last_change":"2024-09-16T07:27:48.461989+0000","last_active":"2024-09-16T07:27:48.468987+0000","last_peered":"2024-09-16T07:27:48.468987+0000","last_clean":"2024-09-16T07:27:48.468987+0000","last_became_active":"2024-09-16T07:27:48.461731+0000","last_became_peered":"2024-09-16T07:27:48.461731+0000","last_unstale":"2024-09-16T07:27:48.468987+0000","last_undegraded":"2024-09-16T07:27:48.468987+0000","last_fullsized":"2024-09-16T07:27:48.468987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:33:43.744251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461382+0000","last_change":"2024-09-16T07:27:47.452023+0000","last_active":"2024-09-16T07:27:49.461382+0000","last_peered":"2024-09-16T07:27:49.461382+0000","last_clean":"2024-09-16T07:27:49.461382+0000","last_became_active":"2024-09-16T07:27:47.451738+0000","last_became_peered":"2024-09-16T07:27:47.451738+0000","last_unstale":"2024-09-16T07:27:49.461382+0000","last_undegraded":"2024-09-16T07:27:49.461382+0000","last_fullsized":"2024-09-16T07:27:49.461382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:28.412285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438666+0000","last_change":"2024-09-16T07:27:47.438666+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438666+0000","last_undegraded":"2024-09-16T07:27:47.438666+0000","last_fullsized":"2024-09-16T07:27:47.438666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.457752+0000","last_change":"2024-09-16T07:27:47.452663+0000","last_active":"2024-09-16T07:27:47.457752+0000","last_peered":"2024-09-16T07:27:47.457752+0000","last_clean":"2024-09-16T07:27:47.457752+0000","last_became_active":"2024-09-16T07:27:47.451570+0000","last_became_peered":"2024-09-16T07:27:47.451570+0000","last_unstale":"2024-09-16T07:27:47.457752+0000","last_undegraded":"2024-09-16T07:27:47.457752+0000","last_fullsized":"2024-09-16T07:27:47.457752+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:14.561916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.439442+0000","last_change":"2024-09-16T07:27:47.439442+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.439442+0000","last_undegraded":"2024-09-16T07:27:47.439442+0000","last_fullsized":"2024-09-16T07:27:47.439442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443403+0000","last_change":"2024-09-16T07:27:47.455318+0000","last_active":"2024-09-16T07:27:48.443403+0000","last_peered":"2024-09-16T07:27:48.443403+0000","last_clean":"2024-09-16T07:27:48.443403+0000","last_became_active":"2024-09-16T07:27:47.455047+0000","last_became_peered":"2024-09-16T07:27:47.455047+0000","last_unstale":"2024-09-16T07:27:48.443403+0000","last_undegraded":"2024-09-16T07:27:48.443403+0000","last_fullsized":"2024-09-16T07:27:48.443403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:08.028003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.466860+0000","last_change":"2024-09-16T07:27:48.462125+0000","last_active":"2024-09-16T07:27:48.466860+0000","last_peered":"2024-09-16T07:27:48.466860+0000","last_clean":"2024-09-16T07:27:48.466860+0000","last_became_active":"2024-09-16T07:27:48.461779+0000","last_became_peered":"2024-09-16T07:27:48.461779+0000","last_unstale":"2024-09-16T07:27:48.466860+0000","last_undegraded":"2024-09-16T07:27:48.466860+0000","last_fullsized":"2024-09-16T07:27:48.466860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:37.463436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464161+0000","last_change":"2024-09-16T07:27:47.461596+0000","last_active":"2024-09-16T07:27:47.464161+0000","last_peered":"2024-09-16T07:27:47.464161+0000","last_clean":"2024-09-16T07:27:47.464161+0000","last_became_active":"2024-09-16T07:27:47.461292+0000","last_became_peered":"2024-09-16T07:27:47.461292+0000","last_unstale":"2024-09-16T07:27:47.464161+0000","last_undegraded":"2024-09-16T07:27:47.464161+0000","last_fullsized":"2024-09-16T07:27:47.464161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:22.241647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.463013+0000","last_change":"2024-09-16T07:27:48.452873+0000","last_active":"2024-09-16T07:27:48.463013+0000","last_peered":"2024-09-16T07:27:48.463013+0000","last_clean":"2024-09-16T07:27:48.463013+0000","last_became_active":"2024-09-16T07:27:48.450871+0000","last_became_peered":"2024-09-16T07:27:48.450871+0000","last_unstale":"2024-09-16T07:27:48.463013+0000","last_undegraded":"2024-09-16T07:27:48.463013+0000","last_fullsized":"2024-09-16T07:27:48.463013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:38:38.451975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.464346+0000","last_change":"2024-09-16T07:27:47.461620+0000","last_active":"2024-09-16T07:27:47.464346+0000","last_peered":"2024-09-16T07:27:47.464346+0000","last_clean":"2024-09-16T07:27:47.464346+0000","last_became_active":"2024-09-16T07:27:47.461338+0000","last_became_peered":"2024-09-16T07:27:47.461338+0000","last_unstale":"2024-09-16T07:27:47.464346+0000","last_undegraded":"2024-09-16T07:27:47.464346+0000","last_fullsized":"2024-09-16T07:27:47.464346+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:28.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444027+0000","last_change":"2024-09-16T07:27:47.444027+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444027+0000","last_undegraded":"2024-09-16T07:27:47.444027+0000","last_fullsized":"2024-09-16T07:27:47.444027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443387+0000","last_change":"2024-09-16T07:27:47.458964+0000","last_active":"2024-09-16T07:27:48.443387+0000","last_peered":"2024-09-16T07:27:48.443387+0000","last_clean":"2024-09-16T07:27:48.443387+0000","last_became_active":"2024-09-16T07:27:47.458687+0000","last_became_peered":"2024-09-16T07:27:47.458687+0000","last_unstale":"2024-09-16T07:27:48.443387+0000","last_undegraded":"2024-09-16T07:27:48.443387+0000","last_fullsized":"2024-09-16T07:27:48.443387+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:43.227646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.438895+0000","last_change":"2024-09-16T07:27:47.438895+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.438895+0000","last_undegraded":"2024-09-16T07:27:47.438895+0000","last_fullsized":"2024-09-16T07:27:47.438895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.448905+0000","last_change":"2024-09-16T07:27:47.456743+0000","last_active":"2024-09-16T07:27:48.448905+0000","last_peered":"2024-09-16T07:27:48.448905+0000","last_clean":"2024-09-16T07:27:48.448905+0000","last_became_active":"2024-09-16T07:27:47.455261+0000","last_became_peered":"2024-09-16T07:27:47.455261+0000","last_unstale":"2024-09-16T07:27:48.448905+0000","last_undegraded":"2024-09-16T07:27:48.448905+0000","last_fullsized":"2024-09-16T07:27:48.448905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:44.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.444420+0000","last_change":"2024-09-16T07:27:47.444420+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.444420+0000","last_undegraded":"2024-09-16T07:27:47.444420+0000","last_fullsized":"2024-09-16T07:27:47.444420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.459115+0000","last_change":"2024-09-16T07:27:47.458363+0000","last_active":"2024-09-16T07:27:47.459115+0000","last_peered":"2024-09-16T07:27:47.459115+0000","last_clean":"2024-09-16T07:27:47.459115+0000","last_became_active":"2024-09-16T07:27:47.455595+0000","last_became_peered":"2024-09-16T07:27:47.455595+0000","last_unstale":"2024-09-16T07:27:47.459115+0000","last_undegraded":"2024-09-16T07:27:47.459115+0000","last_fullsized":"2024-09-16T07:27:47.459115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:18.944163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.465977+0000","last_change":"2024-09-16T07:27:48.460471+0000","last_active":"2024-09-16T07:27:48.465977+0000","last_peered":"2024-09-16T07:27:48.465977+0000","last_clean":"2024-09-16T07:27:48.465977+0000","last_became_active":"2024-09-16T07:27:48.456813+0000","last_became_peered":"2024-09-16T07:27:48.456813+0000","last_unstale":"2024-09-16T07:27:48.465977+0000","last_undegraded":"2024-09-16T07:27:48.465977+0000","last_fullsized":"2024-09-16T07:27:48.465977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:45.085619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.449173+0000","last_change":"2024-09-16T07:27:47.452638+0000","last_active":"2024-09-16T07:27:48.449173+0000","last_peered":"2024-09-16T07:27:48.449173+0000","last_clean":"2024-09-16T07:27:48.449173+0000","last_became_active":"2024-09-16T07:27:47.452390+0000","last_became_peered":"2024-09-16T07:27:47.452390+0000","last_unstale":"2024-09-16T07:27:48.449173+0000","last_undegraded":"2024-09-16T07:27:48.449173+0000","last_fullsized":"2024-09-16T07:27:48.449173+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:04.058738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.440856+0000","last_change":"2024-09-16T07:27:47.440856+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.440856+0000","last_undegraded":"2024-09-16T07:27:47.440856+0000","last_fullsized":"2024-09-16T07:27:47.440856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.462516+0000","last_change":"2024-09-16T07:27:47.460442+0000","last_active":"2024-09-16T07:27:47.462516+0000","last_peered":"2024-09-16T07:27:47.462516+0000","last_clean":"2024-09-16T07:27:47.462516+0000","last_became_active":"2024-09-16T07:27:47.459037+0000","last_became_peered":"2024-09-16T07:27:47.459037+0000","last_unstale":"2024-09-16T07:27:47.462516+0000","last_undegraded":"2024-09-16T07:27:47.462516+0000","last_fullsized":"2024-09-16T07:27:47.462516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:25:58.753488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.442957+0000","last_change":"2024-09-16T07:27:47.442957+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.442957+0000","last_undegraded":"2024-09-16T07:27:47.442957+0000","last_fullsized":"2024-09-16T07:27:47.442957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T07:27:47.460678+0000","last_change":"2024-09-16T07:27:47.453007+0000","last_active":"2024-09-16T07:27:47.460678+0000","last_peered":"2024-09-16T07:27:47.460678+0000","last_clean":"2024-09-16T07:27:47.460678+0000","last_became_active":"2024-09-16T07:27:47.452735+0000","last_became_peered":"2024-09-16T07:27:47.452735+0000","last_unstale":"2024-09-16T07:27:47.460678+0000","last_undegraded":"2024-09-16T07:27:47.460678+0000","last_fullsized":"2024-09-16T07:27:47.460678+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:47:52.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T07:27:47.440008+0000","last_change":"2024-09-16T07:27:47.440008+0000","last_active":"2024-09-16T07:27:47.428257+0000","last_peered":"2024-09-16T07:27:47.428257+0000","last_clean":"2024-09-16T07:27:47.428257+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:27:47.440008+0000","last_undegraded":"2024-09-16T07:27:47.440008+0000","last_fullsized":"2024-09-16T07:27:47.440008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.450452+0000","last_change":"2024-09-16T07:27:47.454175+0000","last_active":"2024-09-16T07:27:48.450452+0000","last_peered":"2024-09-16T07:27:48.450452+0000","last_clean":"2024-09-16T07:27:48.450452+0000","last_became_active":"2024-09-16T07:27:47.453769+0000","last_became_peered":"2024-09-16T07:27:47.453769+0000","last_unstale":"2024-09-16T07:27:48.450452+0000","last_undegraded":"2024-09-16T07:27:48.450452+0000","last_fullsized":"2024-09-16T07:27:48.450452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:17.667822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T07:27:48.443464+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:27:48.443464+0000","last_peered":"2024-09-16T07:27:48.443464+0000","last_clean":"2024-09-16T07:27:48.443464+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:27:48.443464+0000","last_undegraded":"2024-09-16T07:27:48.443464+0000","last_fullsized":"2024-09-16T07:27:48.443464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476803,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":295094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476803,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62448,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309392,"statfs":{"total":96636764160,"available":96572817408,"internally_reserved":0,"allocated":741376,"data_stored":303641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476803,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40784,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331056,"statfs":{"total":96636764160,"available":96595001344,"internally_reserved":0,"allocated":1302528,"data_stored":885462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476802,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":708608,"data_stored":295094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":288819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509506,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":288819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509506,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46352,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325488,"statfs":{"total":96636764160,"available":96589299712,"internally_reserved":0,"allocated":1302528,"data_stored":885462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49960,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321880,"statfs":{"total":96636764160,"available":96585605120,"internally_reserved":0,"allocated":741376,"data_stored":308785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:27:57.332 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:27:57.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.500+0000 7f65dbe00640 1 -- 172.21.15.71:0/15598260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 msgr2=0x7f65dc0cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:57.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.500+0000 7f65dbe00640 1 --2- 172.21.15.71:0/15598260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 0x7f65dc0cc860 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f65cc005fd0 tx=0x7f65cc02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.500+0000 7f65dbe00640 1 -- 172.21.15.71:0/15598260 shutdown_connections 2024-09-16T07:27:57.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.500+0000 7f65dbe00640 1 --2- 172.21.15.71:0/15598260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc0afde0 0x7f65dc0b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.500+0000 7f65dbe00640 1 --2- 172.21.15.71:0/15598260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 0x7f65dc0cc860 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.500+0000 7f65dbe00640 1 -- 172.21.15.71:0/15598260 >> 172.21.15.71:0/15598260 conn(0x7f65dc0b1e90 msgr2=0x7f65dc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:57.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.500+0000 7f65dbe00640 1 -- 172.21.15.71:0/15598260 shutdown_connections 2024-09-16T07:27:57.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.500+0000 7f65dbe00640 1 -- 172.21.15.71:0/15598260 wait complete. 2024-09-16T07:27:57.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.501+0000 7f65dbe00640 1 Processor -- start 2024-09-16T07:27:57.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.501+0000 7f65dbe00640 1 -- start start 2024-09-16T07:27:57.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.502+0000 7f65dbe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc074ca0 0x7f65dc0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:57.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.502+0000 7f65dbe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc0afde0 0x7f65dc158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.502+0000 7f65dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 0x7f65dc0c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.502+0000 7f65dbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc0b0d30 con 0x7f65dc0ce100 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.502+0000 7f65dbe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc0b0bb0 con 0x7f65dc0afde0 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.502+0000 7f65dbe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f65dc0b0eb0 con 0x7f65dc074ca0 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 0x7f65dc0c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 0x7f65dc0c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51650/0 (socket says 172.21.15.71:51650) 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 -- 172.21.15.71:0/3633613897 learned_addr learned my addr 172.21.15.71:0/3633613897 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65daa00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc074ca0 0x7f65dc0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65da000640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc0afde0 0x7f65dc158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 -- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc074ca0 msgr2=0x7f65dc0b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc074ca0 0x7f65dc0b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 -- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc0afde0 msgr2=0x7f65dc158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc0afde0 0x7f65dc158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 -- 172.21.15.71:0/3633613897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc1f2fe0 con 0x7f65dc0ce100 2024-09-16T07:27:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65daa00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc074ca0 0x7f65dc0b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65da000640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc0afde0 0x7f65dc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.503+0000 7f65db400640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 0x7f65dc0c61b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f65d400c970 tx=0x7f65d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.504+0000 7f65d2a00640 1 -- 172.21.15.71:0/3633613897 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4007bb0 con 0x7f65dc0ce100 2024-09-16T07:27:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.504+0000 7f65d2a00640 1 -- 172.21.15.71:0/3633613897 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65d400f450 con 0x7f65dc0ce100 2024-09-16T07:27:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.504+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc1f32c0 con 0x7f65dc0ce100 2024-09-16T07:27:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.504+0000 7f65d2a00640 1 -- 172.21.15.71:0/3633613897 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4007e60 con 0x7f65dc0ce100 2024-09-16T07:27:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.504+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1f3880 con 0x7f65dc0ce100 2024-09-16T07:27:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.505+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc0b0270 con 0x7f65dc0ce100 2024-09-16T07:27:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.506+0000 7f65d2a00640 1 -- 172.21.15.71:0/3633613897 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f65d400f8a0 con 0x7f65dc0ce100 2024-09-16T07:27:57.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.507+0000 7f65d2a00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc0790c0 0x7f65bc07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:57.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.507+0000 7f65d2a00640 1 -- 172.21.15.71:0/3633613897 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f65d409c4a0 con 0x7f65dc0ce100 2024-09-16T07:27:57.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.508+0000 7f65daa00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc0790c0 0x7f65bc07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:57.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.509+0000 7f65daa00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc0790c0 0x7f65bc07b580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65cc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:57.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.512+0000 7f65d2a00640 1 -- 172.21.15.71:0/3633613897 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f65d4014030 con 0x7f65dc0ce100 2024-09-16T07:27:57.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.694+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65dc0b3220 con 0x7f65bc0790c0 2024-09-16T07:27:57.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.699+0000 7f65d2a00640 1 -- 172.21.15.71:0/3633613897 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251924 (secure 0 0 0) 0x7f65cc004440 con 0x7f65bc0790c0 2024-09-16T07:27:57.702 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:57.704 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:27:57.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc0790c0 msgr2=0x7f65bc07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:57.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc0790c0 0x7f65bc07b580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65cc004730 comp rx=0 tx=0).stop 2024-09-16T07:27:57.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 msgr2=0x7f65dc0c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:57.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 0x7f65dc0c61b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f65d400c970 tx=0x7f65d400ce30 comp rx=0 tx=0).stop 2024-09-16T07:27:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 shutdown_connections 2024-09-16T07:27:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc0790c0 0x7f65bc07b580 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc0ce100 0x7f65dc0c61b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc0afde0 0x7f65dc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 --2- 172.21.15.71:0/3633613897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc074ca0 0x7f65dc0b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 >> 172.21.15.71:0/3633613897 conn(0x7f65dc0b1e90 msgr2=0x7f65dc0cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 shutdown_connections 2024-09-16T07:27:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.704+0000 7f65dbe00640 1 -- 172.21.15.71:0/3633613897 wait complete. 2024-09-16T07:27:57.719 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-16T07:27:56.381707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":360532,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":353395,"kb_avail":754614188,"statfs":{"total":773094113280,"available":772724928512,"internally_reserved":0,"allocated":7118848,"data_stored":3650908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":361877040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002181"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465806+0000","last_change":"2024-09-16T07:27:48.456727+0000","last_active":"2024-09-16T07:27:49.465806+0000","last_peered":"2024-09-16T07:27:49.465806+0000","last_clean":"2024-09-16T07:27:49.465806+0000","last_became_active":"2024-09-16T07:27:48.456367+0000","last_became_peered":"2024-09-16T07:27:48.456367+0000","last_unstale":"2024-09-16T07:27:49.465806+0000","last_undegraded":"2024-09-16T07:27:49.465806+0000","last_fullsized":"2024-09-16T07:27:49.465806+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:06.084517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625549+0000","last_change":"2024-09-16T07:27:48.468781+0000","last_active":"2024-09-16T07:27:49.625549+0000","last_peered":"2024-09-16T07:27:49.625549+0000","last_clean":"2024-09-16T07:27:49.625549+0000","last_became_active":"2024-09-16T07:27:48.466011+0000","last_became_peered":"2024-09-16T07:27:48.466011+0000","last_unstale":"2024-09-16T07:27:49.625549+0000","last_undegraded":"2024-09-16T07:27:49.625549+0000","last_fullsized":"2024-09-16T07:27:49.625549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:13.804863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.457893+0000","last_change":"2024-09-16T07:27:48.453492+0000","last_active":"2024-09-16T07:27:49.457893+0000","last_peered":"2024-09-16T07:27:49.457893+0000","last_clean":"2024-09-16T07:27:49.457893+0000","last_became_active":"2024-09-16T07:27:48.453262+0000","last_became_peered":"2024-09-16T07:27:48.453262+0000","last_unstale":"2024-09-16T07:27:49.457893+0000","last_undegraded":"2024-09-16T07:27:49.457893+0000","last_fullsized":"2024-09-16T07:27:49.457893+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:35:41.335551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130595+0000","last_change":"2024-09-16T07:27:48.460353+0000","last_active":"2024-09-16T07:27:50.130595+0000","last_peered":"2024-09-16T07:27:50.130595+0000","last_clean":"2024-09-16T07:27:50.130595+0000","last_became_active":"2024-09-16T07:27:48.459644+0000","last_became_peered":"2024-09-16T07:27:48.459644+0000","last_unstale":"2024-09-16T07:27:50.130595+0000","last_undegraded":"2024-09-16T07:27:50.130595+0000","last_fullsized":"2024-09-16T07:27:50.130595+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:04.484092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465215+0000","last_change":"2024-09-16T07:27:48.465389+0000","last_active":"2024-09-16T07:27:49.465215+0000","last_peered":"2024-09-16T07:27:49.465215+0000","last_clean":"2024-09-16T07:27:49.465215+0000","last_became_active":"2024-09-16T07:27:48.465120+0000","last_became_peered":"2024-09-16T07:27:48.465120+0000","last_unstale":"2024-09-16T07:27:49.465215+0000","last_undegraded":"2024-09-16T07:27:49.465215+0000","last_fullsized":"2024-09-16T07:27:49.465215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:07.527458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465393+0000","last_change":"2024-09-16T07:27:48.461012+0000","last_active":"2024-09-16T07:27:49.465393+0000","last_peered":"2024-09-16T07:27:49.465393+0000","last_clean":"2024-09-16T07:27:49.465393+0000","last_became_active":"2024-09-16T07:27:48.460664+0000","last_became_peered":"2024-09-16T07:27:48.460664+0000","last_unstale":"2024-09-16T07:27:49.465393+0000","last_undegraded":"2024-09-16T07:27:49.465393+0000","last_fullsized":"2024-09-16T07:27:49.465393+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:04.739035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129795+0000","last_change":"2024-09-16T07:27:48.462460+0000","last_active":"2024-09-16T07:27:50.129795+0000","last_peered":"2024-09-16T07:27:50.129795+0000","last_clean":"2024-09-16T07:27:50.129795+0000","last_became_active":"2024-09-16T07:27:48.462193+0000","last_became_peered":"2024-09-16T07:27:48.462193+0000","last_unstale":"2024-09-16T07:27:50.129795+0000","last_undegraded":"2024-09-16T07:27:50.129795+0000","last_fullsized":"2024-09-16T07:27:50.129795+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:15.719040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462241+0000","last_change":"2024-09-16T07:27:48.448607+0000","last_active":"2024-09-16T07:27:49.462241+0000","last_peered":"2024-09-16T07:27:49.462241+0000","last_clean":"2024-09-16T07:27:49.462241+0000","last_became_active":"2024-09-16T07:27:48.448142+0000","last_became_peered":"2024-09-16T07:27:48.448142+0000","last_unstale":"2024-09-16T07:27:49.462241+0000","last_undegraded":"2024-09-16T07:27:49.462241+0000","last_fullsized":"2024-09-16T07:27:49.462241+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:52.704159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130105+0000","last_change":"2024-09-16T07:27:48.461060+0000","last_active":"2024-09-16T07:27:50.130105+0000","last_peered":"2024-09-16T07:27:50.130105+0000","last_clean":"2024-09-16T07:27:50.130105+0000","last_became_active":"2024-09-16T07:27:48.460770+0000","last_became_peered":"2024-09-16T07:27:48.460770+0000","last_unstale":"2024-09-16T07:27:50.130105+0000","last_undegraded":"2024-09-16T07:27:50.130105+0000","last_fullsized":"2024-09-16T07:27:50.130105+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:15.376653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465758+0000","last_change":"2024-09-16T07:27:48.468251+0000","last_active":"2024-09-16T07:27:49.465758+0000","last_peered":"2024-09-16T07:27:49.465758+0000","last_clean":"2024-09-16T07:27:49.465758+0000","last_became_active":"2024-09-16T07:27:48.467079+0000","last_became_peered":"2024-09-16T07:27:48.467079+0000","last_unstale":"2024-09-16T07:27:49.465758+0000","last_undegraded":"2024-09-16T07:27:49.465758+0000","last_fullsized":"2024-09-16T07:27:49.465758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:59.562068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461558+0000","last_change":"2024-09-16T07:27:48.459130+0000","last_active":"2024-09-16T07:27:49.461558+0000","last_peered":"2024-09-16T07:27:49.461558+0000","last_clean":"2024-09-16T07:27:49.461558+0000","last_became_active":"2024-09-16T07:27:48.458360+0000","last_became_peered":"2024-09-16T07:27:48.458360+0000","last_unstale":"2024-09-16T07:27:49.461558+0000","last_undegraded":"2024-09-16T07:27:49.461558+0000","last_fullsized":"2024-09-16T07:27:49.461558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:42.756027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129274+0000","last_change":"2024-09-16T07:27:48.462533+0000","last_active":"2024-09-16T07:27:50.129274+0000","last_peered":"2024-09-16T07:27:50.129274+0000","last_clean":"2024-09-16T07:27:50.129274+0000","last_became_active":"2024-09-16T07:27:48.462296+0000","last_became_peered":"2024-09-16T07:27:48.462296+0000","last_unstale":"2024-09-16T07:27:50.129274+0000","last_undegraded":"2024-09-16T07:27:50.129274+0000","last_fullsized":"2024-09-16T07:27:50.129274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:51:50.978986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129196+0000","last_change":"2024-09-16T07:27:48.461185+0000","last_active":"2024-09-16T07:27:50.129196+0000","last_peered":"2024-09-16T07:27:50.129196+0000","last_clean":"2024-09-16T07:27:50.129196+0000","last_became_active":"2024-09-16T07:27:48.460889+0000","last_became_peered":"2024-09-16T07:27:48.460889+0000","last_unstale":"2024-09-16T07:27:50.129196+0000","last_undegraded":"2024-09-16T07:27:50.129196+0000","last_fullsized":"2024-09-16T07:27:50.129196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:28.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.457778+0000","last_change":"2024-09-16T07:27:48.458531+0000","last_active":"2024-09-16T07:27:49.457778+0000","last_peered":"2024-09-16T07:27:49.457778+0000","last_clean":"2024-09-16T07:27:49.457778+0000","last_became_active":"2024-09-16T07:27:48.458285+0000","last_became_peered":"2024-09-16T07:27:48.458285+0000","last_unstale":"2024-09-16T07:27:49.457778+0000","last_undegraded":"2024-09-16T07:27:49.457778+0000","last_fullsized":"2024-09-16T07:27:49.457778+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:17:10.966228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129457+0000","last_change":"2024-09-16T07:27:48.466279+0000","last_active":"2024-09-16T07:27:50.129457+0000","last_peered":"2024-09-16T07:27:50.129457+0000","last_clean":"2024-09-16T07:27:50.129457+0000","last_became_active":"2024-09-16T07:27:48.466000+0000","last_became_peered":"2024-09-16T07:27:48.466000+0000","last_unstale":"2024-09-16T07:27:50.129457+0000","last_undegraded":"2024-09-16T07:27:50.129457+0000","last_fullsized":"2024-09-16T07:27:50.129457+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:40.253906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465258+0000","last_change":"2024-09-16T07:27:48.469760+0000","last_active":"2024-09-16T07:27:49.465258+0000","last_peered":"2024-09-16T07:27:49.465258+0000","last_clean":"2024-09-16T07:27:49.465258+0000","last_became_active":"2024-09-16T07:27:48.469146+0000","last_became_peered":"2024-09-16T07:27:48.469146+0000","last_unstale":"2024-09-16T07:27:49.465258+0000","last_undegraded":"2024-09-16T07:27:49.465258+0000","last_fullsized":"2024-09-16T07:27:49.465258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:28.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465846+0000","last_change":"2024-09-16T07:27:48.471199+0000","last_active":"2024-09-16T07:27:49.465846+0000","last_peered":"2024-09-16T07:27:49.465846+0000","last_clean":"2024-09-16T07:27:49.465846+0000","last_became_active":"2024-09-16T07:27:48.470916+0000","last_became_peered":"2024-09-16T07:27:48.470916+0000","last_unstale":"2024-09-16T07:27:49.465846+0000","last_undegraded":"2024-09-16T07:27:49.465846+0000","last_fullsized":"2024-09-16T07:27:49.465846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:44.782087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.458148+0000","last_change":"2024-09-16T07:27:48.465698+0000","last_active":"2024-09-16T07:27:49.458148+0000","last_peered":"2024-09-16T07:27:49.458148+0000","last_clean":"2024-09-16T07:27:49.458148+0000","last_became_active":"2024-09-16T07:27:48.465355+0000","last_became_peered":"2024-09-16T07:27:48.465355+0000","last_unstale":"2024-09-16T07:27:49.458148+0000","last_undegraded":"2024-09-16T07:27:49.458148+0000","last_fullsized":"2024-09-16T07:27:49.458148+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:42.971725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625601+0000","last_change":"2024-09-16T07:27:48.462682+0000","last_active":"2024-09-16T07:27:49.625601+0000","last_peered":"2024-09-16T07:27:49.625601+0000","last_clean":"2024-09-16T07:27:49.625601+0000","last_became_active":"2024-09-16T07:27:48.462308+0000","last_became_peered":"2024-09-16T07:27:48.462308+0000","last_unstale":"2024-09-16T07:27:49.625601+0000","last_undegraded":"2024-09-16T07:27:49.625601+0000","last_fullsized":"2024-09-16T07:27:49.625601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:51.533110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461494+0000","last_change":"2024-09-16T07:27:48.460038+0000","last_active":"2024-09-16T07:27:49.461494+0000","last_peered":"2024-09-16T07:27:49.461494+0000","last_clean":"2024-09-16T07:27:49.461494+0000","last_became_active":"2024-09-16T07:27:48.459755+0000","last_became_peered":"2024-09-16T07:27:48.459755+0000","last_unstale":"2024-09-16T07:27:49.461494+0000","last_undegraded":"2024-09-16T07:27:49.461494+0000","last_fullsized":"2024-09-16T07:27:49.461494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:37.822664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129684+0000","last_change":"2024-09-16T07:27:48.464651+0000","last_active":"2024-09-16T07:27:50.129684+0000","last_peered":"2024-09-16T07:27:50.129684+0000","last_clean":"2024-09-16T07:27:50.129684+0000","last_became_active":"2024-09-16T07:27:48.464373+0000","last_became_peered":"2024-09-16T07:27:48.464373+0000","last_unstale":"2024-09-16T07:27:50.129684+0000","last_undegraded":"2024-09-16T07:27:50.129684+0000","last_fullsized":"2024-09-16T07:27:50.129684+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:37.463436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461319+0000","last_change":"2024-09-16T07:27:48.461614+0000","last_active":"2024-09-16T07:27:49.461319+0000","last_peered":"2024-09-16T07:27:49.461319+0000","last_clean":"2024-09-16T07:27:49.461319+0000","last_became_active":"2024-09-16T07:27:48.461333+0000","last_became_peered":"2024-09-16T07:27:48.461333+0000","last_unstale":"2024-09-16T07:27:49.461319+0000","last_undegraded":"2024-09-16T07:27:49.461319+0000","last_fullsized":"2024-09-16T07:27:49.461319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:14.430788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129993+0000","last_change":"2024-09-16T07:27:48.469453+0000","last_active":"2024-09-16T07:27:50.129993+0000","last_peered":"2024-09-16T07:27:50.129993+0000","last_clean":"2024-09-16T07:27:50.129993+0000","last_became_active":"2024-09-16T07:27:48.466391+0000","last_became_peered":"2024-09-16T07:27:48.466391+0000","last_unstale":"2024-09-16T07:27:50.129993+0000","last_undegraded":"2024-09-16T07:27:50.129993+0000","last_fullsized":"2024-09-16T07:27:50.129993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:51:50.368299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129055+0000","last_change":"2024-09-16T07:27:48.461655+0000","last_active":"2024-09-16T07:27:50.129055+0000","last_peered":"2024-09-16T07:27:50.129055+0000","last_clean":"2024-09-16T07:27:50.129055+0000","last_became_active":"2024-09-16T07:27:48.461371+0000","last_became_peered":"2024-09-16T07:27:48.461371+0000","last_unstale":"2024-09-16T07:27:50.129055+0000","last_undegraded":"2024-09-16T07:27:50.129055+0000","last_fullsized":"2024-09-16T07:27:50.129055+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:04:53.278846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625601+0000","last_change":"2024-09-16T07:27:48.462992+0000","last_active":"2024-09-16T07:27:49.625601+0000","last_peered":"2024-09-16T07:27:49.625601+0000","last_clean":"2024-09-16T07:27:49.625601+0000","last_became_active":"2024-09-16T07:27:48.462742+0000","last_became_peered":"2024-09-16T07:27:48.462742+0000","last_unstale":"2024-09-16T07:27:49.625601+0000","last_undegraded":"2024-09-16T07:27:49.625601+0000","last_fullsized":"2024-09-16T07:27:49.625601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:57.068076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129633+0000","last_change":"2024-09-16T07:27:48.455880+0000","last_active":"2024-09-16T07:27:50.129633+0000","last_peered":"2024-09-16T07:27:50.129633+0000","last_clean":"2024-09-16T07:27:50.129633+0000","last_became_active":"2024-09-16T07:27:48.455633+0000","last_became_peered":"2024-09-16T07:27:48.455633+0000","last_unstale":"2024-09-16T07:27:50.129633+0000","last_undegraded":"2024-09-16T07:27:50.129633+0000","last_fullsized":"2024-09-16T07:27:50.129633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:04.739035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625787+0000","last_change":"2024-09-16T07:27:48.469361+0000","last_active":"2024-09-16T07:27:49.625787+0000","last_peered":"2024-09-16T07:27:49.625787+0000","last_clean":"2024-09-16T07:27:49.625787+0000","last_became_active":"2024-09-16T07:27:48.465060+0000","last_became_peered":"2024-09-16T07:27:48.465060+0000","last_unstale":"2024-09-16T07:27:49.625787+0000","last_undegraded":"2024-09-16T07:27:49.625787+0000","last_fullsized":"2024-09-16T07:27:49.625787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:45.772584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462509+0000","last_change":"2024-09-16T07:27:48.452856+0000","last_active":"2024-09-16T07:27:49.462509+0000","last_peered":"2024-09-16T07:27:49.462509+0000","last_clean":"2024-09-16T07:27:49.462509+0000","last_became_active":"2024-09-16T07:27:48.452514+0000","last_became_peered":"2024-09-16T07:27:48.452514+0000","last_unstale":"2024-09-16T07:27:49.462509+0000","last_undegraded":"2024-09-16T07:27:49.462509+0000","last_fullsized":"2024-09-16T07:27:49.462509+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:55:49.386094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462607+0000","last_change":"2024-09-16T07:27:48.455170+0000","last_active":"2024-09-16T07:27:49.462607+0000","last_peered":"2024-09-16T07:27:49.462607+0000","last_clean":"2024-09-16T07:27:49.462607+0000","last_became_active":"2024-09-16T07:27:48.454503+0000","last_became_peered":"2024-09-16T07:27:48.454503+0000","last_unstale":"2024-09-16T07:27:49.462607+0000","last_undegraded":"2024-09-16T07:27:49.462607+0000","last_fullsized":"2024-09-16T07:27:49.462607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:33.983922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130453+0000","last_change":"2024-09-16T07:27:48.462181+0000","last_active":"2024-09-16T07:27:50.130453+0000","last_peered":"2024-09-16T07:27:50.130453+0000","last_clean":"2024-09-16T07:27:50.130453+0000","last_became_active":"2024-09-16T07:27:48.460601+0000","last_became_peered":"2024-09-16T07:27:48.460601+0000","last_unstale":"2024-09-16T07:27:50.130453+0000","last_undegraded":"2024-09-16T07:27:50.130453+0000","last_fullsized":"2024-09-16T07:27:50.130453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:49:49.791085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625190+0000","last_change":"2024-09-16T07:27:48.461062+0000","last_active":"2024-09-16T07:27:49.625190+0000","last_peered":"2024-09-16T07:27:49.625190+0000","last_clean":"2024-09-16T07:27:49.625190+0000","last_became_active":"2024-09-16T07:27:48.459636+0000","last_became_peered":"2024-09-16T07:27:48.459636+0000","last_unstale":"2024-09-16T07:27:49.625190+0000","last_undegraded":"2024-09-16T07:27:49.625190+0000","last_fullsized":"2024-09-16T07:27:49.625190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:26.798513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461161+0000","last_change":"2024-09-16T07:27:48.460512+0000","last_active":"2024-09-16T07:27:49.461161+0000","last_peered":"2024-09-16T07:27:49.461161+0000","last_clean":"2024-09-16T07:27:49.461161+0000","last_became_active":"2024-09-16T07:27:48.460276+0000","last_became_peered":"2024-09-16T07:27:48.460276+0000","last_unstale":"2024-09-16T07:27:49.461161+0000","last_undegraded":"2024-09-16T07:27:49.461161+0000","last_fullsized":"2024-09-16T07:27:49.461161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:09.223196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465892+0000","last_change":"2024-09-16T07:27:48.467939+0000","last_active":"2024-09-16T07:27:49.465892+0000","last_peered":"2024-09-16T07:27:49.465892+0000","last_clean":"2024-09-16T07:27:49.465892+0000","last_became_active":"2024-09-16T07:27:48.466728+0000","last_became_peered":"2024-09-16T07:27:48.466728+0000","last_unstale":"2024-09-16T07:27:49.465892+0000","last_undegraded":"2024-09-16T07:27:49.465892+0000","last_fullsized":"2024-09-16T07:27:49.465892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:15.724336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461617+0000","last_change":"2024-09-16T07:27:47.450308+0000","last_active":"2024-09-16T07:27:49.461617+0000","last_peered":"2024-09-16T07:27:49.461617+0000","last_clean":"2024-09-16T07:27:49.461617+0000","last_became_active":"2024-09-16T07:27:47.448332+0000","last_became_peered":"2024-09-16T07:27:47.448332+0000","last_unstale":"2024-09-16T07:27:49.461617+0000","last_undegraded":"2024-09-16T07:27:49.461617+0000","last_fullsized":"2024-09-16T07:27:49.461617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:38.412742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465957+0000","last_change":"2024-09-16T07:27:48.461361+0000","last_active":"2024-09-16T07:27:49.465957+0000","last_peered":"2024-09-16T07:27:49.465957+0000","last_clean":"2024-09-16T07:27:49.465957+0000","last_became_active":"2024-09-16T07:27:48.460680+0000","last_became_peered":"2024-09-16T07:27:48.460680+0000","last_unstale":"2024-09-16T07:27:49.465957+0000","last_undegraded":"2024-09-16T07:27:49.465957+0000","last_fullsized":"2024-09-16T07:27:49.465957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:46.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625548+0000","last_change":"2024-09-16T07:27:47.452837+0000","last_active":"2024-09-16T07:27:49.625548+0000","last_peered":"2024-09-16T07:27:49.625548+0000","last_clean":"2024-09-16T07:27:49.625548+0000","last_became_active":"2024-09-16T07:27:47.452517+0000","last_became_peered":"2024-09-16T07:27:47.452517+0000","last_unstale":"2024-09-16T07:27:49.625548+0000","last_undegraded":"2024-09-16T07:27:49.625548+0000","last_fullsized":"2024-09-16T07:27:49.625548+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:29.685314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462748+0000","last_change":"2024-09-16T07:27:48.464150+0000","last_active":"2024-09-16T07:27:49.462748+0000","last_peered":"2024-09-16T07:27:49.462748+0000","last_clean":"2024-09-16T07:27:49.462748+0000","last_became_active":"2024-09-16T07:27:48.463085+0000","last_became_peered":"2024-09-16T07:27:48.463085+0000","last_unstale":"2024-09-16T07:27:49.462748+0000","last_undegraded":"2024-09-16T07:27:49.462748+0000","last_fullsized":"2024-09-16T07:27:49.462748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:31.290839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465570+0000","last_change":"2024-09-16T07:27:47.459254+0000","last_active":"2024-09-16T07:27:49.465570+0000","last_peered":"2024-09-16T07:27:49.465570+0000","last_clean":"2024-09-16T07:27:49.465570+0000","last_became_active":"2024-09-16T07:27:47.458912+0000","last_became_peered":"2024-09-16T07:27:47.458912+0000","last_unstale":"2024-09-16T07:27:49.465570+0000","last_undegraded":"2024-09-16T07:27:49.465570+0000","last_fullsized":"2024-09-16T07:27:49.465570+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:23.067757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465480+0000","last_change":"2024-09-16T07:27:48.466871+0000","last_active":"2024-09-16T07:27:49.465480+0000","last_peered":"2024-09-16T07:27:49.465480+0000","last_clean":"2024-09-16T07:27:49.465480+0000","last_became_active":"2024-09-16T07:27:48.466594+0000","last_became_peered":"2024-09-16T07:27:48.466594+0000","last_unstale":"2024-09-16T07:27:49.465480+0000","last_undegraded":"2024-09-16T07:27:49.465480+0000","last_fullsized":"2024-09-16T07:27:49.465480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:37.463436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130599+0000","last_change":"2024-09-16T07:27:47.458877+0000","last_active":"2024-09-16T07:27:50.130599+0000","last_peered":"2024-09-16T07:27:50.130599+0000","last_clean":"2024-09-16T07:27:50.130599+0000","last_became_active":"2024-09-16T07:27:47.458615+0000","last_became_peered":"2024-09-16T07:27:47.458615+0000","last_unstale":"2024-09-16T07:27:50.130599+0000","last_undegraded":"2024-09-16T07:27:50.130599+0000","last_fullsized":"2024-09-16T07:27:50.130599+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:00.779149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462457+0000","last_change":"2024-09-16T07:27:48.451448+0000","last_active":"2024-09-16T07:27:49.462457+0000","last_peered":"2024-09-16T07:27:49.462457+0000","last_clean":"2024-09-16T07:27:49.462457+0000","last_became_active":"2024-09-16T07:27:48.451171+0000","last_became_peered":"2024-09-16T07:27:48.451171+0000","last_unstale":"2024-09-16T07:27:49.462457+0000","last_undegraded":"2024-09-16T07:27:49.462457+0000","last_fullsized":"2024-09-16T07:27:49.462457+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:24.587696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.458015+0000","last_change":"2024-09-16T07:27:47.455983+0000","last_active":"2024-09-16T07:27:49.458015+0000","last_peered":"2024-09-16T07:27:49.458015+0000","last_clean":"2024-09-16T07:27:49.458015+0000","last_became_active":"2024-09-16T07:27:47.454631+0000","last_became_peered":"2024-09-16T07:27:47.454631+0000","last_unstale":"2024-09-16T07:27:49.458015+0000","last_undegraded":"2024-09-16T07:27:49.458015+0000","last_fullsized":"2024-09-16T07:27:49.458015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:37.670315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461455+0000","last_change":"2024-09-16T07:27:48.457180+0000","last_active":"2024-09-16T07:27:49.461455+0000","last_peered":"2024-09-16T07:27:49.461455+0000","last_clean":"2024-09-16T07:27:49.461455+0000","last_became_active":"2024-09-16T07:27:48.456834+0000","last_became_peered":"2024-09-16T07:27:48.456834+0000","last_unstale":"2024-09-16T07:27:49.461455+0000","last_undegraded":"2024-09-16T07:27:49.461455+0000","last_fullsized":"2024-09-16T07:27:49.461455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:04.346053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465503+0000","last_change":"2024-09-16T07:27:47.461641+0000","last_active":"2024-09-16T07:27:49.465503+0000","last_peered":"2024-09-16T07:27:49.465503+0000","last_clean":"2024-09-16T07:27:49.465503+0000","last_became_active":"2024-09-16T07:27:47.461318+0000","last_became_peered":"2024-09-16T07:27:47.461318+0000","last_unstale":"2024-09-16T07:27:49.465503+0000","last_undegraded":"2024-09-16T07:27:49.465503+0000","last_fullsized":"2024-09-16T07:27:49.465503+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:38:19.181611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.457684+0000","last_change":"2024-09-16T07:27:48.455278+0000","last_active":"2024-09-16T07:27:49.457684+0000","last_peered":"2024-09-16T07:27:49.457684+0000","last_clean":"2024-09-16T07:27:49.457684+0000","last_became_active":"2024-09-16T07:27:48.454871+0000","last_became_peered":"2024-09-16T07:27:48.454871+0000","last_unstale":"2024-09-16T07:27:49.457684+0000","last_undegraded":"2024-09-16T07:27:49.457684+0000","last_fullsized":"2024-09-16T07:27:49.457684+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:44.782087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129822+0000","last_change":"2024-09-16T07:27:47.452265+0000","last_active":"2024-09-16T07:27:50.129822+0000","last_peered":"2024-09-16T07:27:50.129822+0000","last_clean":"2024-09-16T07:27:50.129822+0000","last_became_active":"2024-09-16T07:27:47.452018+0000","last_became_peered":"2024-09-16T07:27:47.452018+0000","last_unstale":"2024-09-16T07:27:50.129822+0000","last_undegraded":"2024-09-16T07:27:50.129822+0000","last_fullsized":"2024-09-16T07:27:50.129822+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:35.000221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465130+0000","last_change":"2024-09-16T07:27:48.469649+0000","last_active":"2024-09-16T07:27:49.465130+0000","last_peered":"2024-09-16T07:27:49.465130+0000","last_clean":"2024-09-16T07:27:49.465130+0000","last_became_active":"2024-09-16T07:27:48.469343+0000","last_became_peered":"2024-09-16T07:27:48.469343+0000","last_unstale":"2024-09-16T07:27:49.465130+0000","last_undegraded":"2024-09-16T07:27:49.465130+0000","last_fullsized":"2024-09-16T07:27:49.465130+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:42.196208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129640+0000","last_change":"2024-09-16T07:27:47.447783+0000","last_active":"2024-09-16T07:27:50.129640+0000","last_peered":"2024-09-16T07:27:50.129640+0000","last_clean":"2024-09-16T07:27:50.129640+0000","last_became_active":"2024-09-16T07:27:47.447510+0000","last_became_peered":"2024-09-16T07:27:47.447510+0000","last_unstale":"2024-09-16T07:27:50.129640+0000","last_undegraded":"2024-09-16T07:27:50.129640+0000","last_fullsized":"2024-09-16T07:27:50.129640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:14.370496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130064+0000","last_change":"2024-09-16T07:27:48.454744+0000","last_active":"2024-09-16T07:27:50.130064+0000","last_peered":"2024-09-16T07:27:50.130064+0000","last_clean":"2024-09-16T07:27:50.130064+0000","last_became_active":"2024-09-16T07:27:48.454446+0000","last_became_peered":"2024-09-16T07:27:48.454446+0000","last_unstale":"2024-09-16T07:27:50.130064+0000","last_undegraded":"2024-09-16T07:27:50.130064+0000","last_fullsized":"2024-09-16T07:27:50.130064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:06.084517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130181+0000","last_change":"2024-09-16T07:27:47.450306+0000","last_active":"2024-09-16T07:27:50.130181+0000","last_peered":"2024-09-16T07:27:50.130181+0000","last_clean":"2024-09-16T07:27:50.130181+0000","last_became_active":"2024-09-16T07:27:47.449275+0000","last_became_peered":"2024-09-16T07:27:47.449275+0000","last_unstale":"2024-09-16T07:27:50.130181+0000","last_undegraded":"2024-09-16T07:27:50.130181+0000","last_fullsized":"2024-09-16T07:27:50.130181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:00.479157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.458122+0000","last_change":"2024-09-16T07:27:48.468331+0000","last_active":"2024-09-16T07:27:49.458122+0000","last_peered":"2024-09-16T07:27:49.458122+0000","last_clean":"2024-09-16T07:27:49.458122+0000","last_became_active":"2024-09-16T07:27:48.468040+0000","last_became_peered":"2024-09-16T07:27:48.468040+0000","last_unstale":"2024-09-16T07:27:49.458122+0000","last_undegraded":"2024-09-16T07:27:49.458122+0000","last_fullsized":"2024-09-16T07:27:49.458122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:58.227836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462787+0000","last_change":"2024-09-16T07:27:47.450498+0000","last_active":"2024-09-16T07:27:49.462787+0000","last_peered":"2024-09-16T07:27:49.462787+0000","last_clean":"2024-09-16T07:27:49.462787+0000","last_became_active":"2024-09-16T07:27:47.449615+0000","last_became_peered":"2024-09-16T07:27:47.449615+0000","last_unstale":"2024-09-16T07:27:49.462787+0000","last_undegraded":"2024-09-16T07:27:49.462787+0000","last_fullsized":"2024-09-16T07:27:49.462787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:30.773040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462850+0000","last_change":"2024-09-16T07:27:48.462677+0000","last_active":"2024-09-16T07:27:49.462850+0000","last_peered":"2024-09-16T07:27:49.462850+0000","last_clean":"2024-09-16T07:27:49.462850+0000","last_became_active":"2024-09-16T07:27:48.462357+0000","last_became_peered":"2024-09-16T07:27:48.462357+0000","last_unstale":"2024-09-16T07:27:49.462850+0000","last_undegraded":"2024-09-16T07:27:49.462850+0000","last_fullsized":"2024-09-16T07:27:49.462850+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:03:44.308509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130726+0000","last_change":"2024-09-16T07:27:47.460337+0000","last_active":"2024-09-16T07:27:50.130726+0000","last_peered":"2024-09-16T07:27:50.130726+0000","last_clean":"2024-09-16T07:27:50.130726+0000","last_became_active":"2024-09-16T07:27:47.460023+0000","last_became_peered":"2024-09-16T07:27:47.460023+0000","last_unstale":"2024-09-16T07:27:50.130726+0000","last_undegraded":"2024-09-16T07:27:50.130726+0000","last_fullsized":"2024-09-16T07:27:50.130726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:52.763400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130626+0000","last_change":"2024-09-16T07:27:48.449778+0000","last_active":"2024-09-16T07:27:50.130626+0000","last_peered":"2024-09-16T07:27:50.130626+0000","last_clean":"2024-09-16T07:27:50.130626+0000","last_became_active":"2024-09-16T07:27:48.449504+0000","last_became_peered":"2024-09-16T07:27:48.449504+0000","last_unstale":"2024-09-16T07:27:50.130626+0000","last_undegraded":"2024-09-16T07:27:50.130626+0000","last_fullsized":"2024-09-16T07:27:50.130626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:38:09.143604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":21,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129787+0000","last_change":"2024-09-16T07:27:47.458310+0000","last_active":"2024-09-16T07:27:50.129787+0000","last_peered":"2024-09-16T07:27:50.129787+0000","last_clean":"2024-09-16T07:27:50.129787+0000","last_became_active":"2024-09-16T07:27:47.457920+0000","last_became_peered":"2024-09-16T07:27:47.457920+0000","last_unstale":"2024-09-16T07:27:50.129787+0000","last_undegraded":"2024-09-16T07:27:50.129787+0000","last_fullsized":"2024-09-16T07:27:50.129787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.526953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129058+0000","last_change":"2024-09-16T07:27:48.467738+0000","last_active":"2024-09-16T07:27:50.129058+0000","last_peered":"2024-09-16T07:27:50.129058+0000","last_clean":"2024-09-16T07:27:50.129058+0000","last_became_active":"2024-09-16T07:27:48.467296+0000","last_became_peered":"2024-09-16T07:27:48.467296+0000","last_unstale":"2024-09-16T07:27:50.129058+0000","last_undegraded":"2024-09-16T07:27:50.129058+0000","last_fullsized":"2024-09-16T07:27:50.129058+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:24.073914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.463010+0000","last_change":"2024-09-16T07:27:47.449886+0000","last_active":"2024-09-16T07:27:49.463010+0000","last_peered":"2024-09-16T07:27:49.463010+0000","last_clean":"2024-09-16T07:27:49.463010+0000","last_became_active":"2024-09-16T07:27:47.449599+0000","last_became_peered":"2024-09-16T07:27:47.449599+0000","last_unstale":"2024-09-16T07:27:49.463010+0000","last_undegraded":"2024-09-16T07:27:49.463010+0000","last_fullsized":"2024-09-16T07:27:49.463010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:19.707395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.457864+0000","last_change":"2024-09-16T07:27:48.453257+0000","last_active":"2024-09-16T07:27:49.457864+0000","last_peered":"2024-09-16T07:27:49.457864+0000","last_clean":"2024-09-16T07:27:49.457864+0000","last_became_active":"2024-09-16T07:27:48.453020+0000","last_became_peered":"2024-09-16T07:27:48.453020+0000","last_unstale":"2024-09-16T07:27:49.457864+0000","last_undegraded":"2024-09-16T07:27:49.457864+0000","last_fullsized":"2024-09-16T07:27:49.457864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:22:46.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130375+0000","last_change":"2024-09-16T07:27:47.449870+0000","last_active":"2024-09-16T07:27:50.130375+0000","last_peered":"2024-09-16T07:27:50.130375+0000","last_clean":"2024-09-16T07:27:50.130375+0000","last_became_active":"2024-09-16T07:27:47.449587+0000","last_became_peered":"2024-09-16T07:27:47.449587+0000","last_unstale":"2024-09-16T07:27:50.130375+0000","last_undegraded":"2024-09-16T07:27:50.130375+0000","last_fullsized":"2024-09-16T07:27:50.130375+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:56.658575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129290+0000","last_change":"2024-09-16T07:27:48.457526+0000","last_active":"2024-09-16T07:27:50.129290+0000","last_peered":"2024-09-16T07:27:50.129290+0000","last_clean":"2024-09-16T07:27:50.129290+0000","last_became_active":"2024-09-16T07:27:48.457292+0000","last_became_peered":"2024-09-16T07:27:48.457292+0000","last_unstale":"2024-09-16T07:27:50.129290+0000","last_undegraded":"2024-09-16T07:27:50.129290+0000","last_fullsized":"2024-09-16T07:27:50.129290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:57.568720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130342+0000","last_change":"2024-09-16T07:27:47.450647+0000","last_active":"2024-09-16T07:27:50.130342+0000","last_peered":"2024-09-16T07:27:50.130342+0000","last_clean":"2024-09-16T07:27:50.130342+0000","last_became_active":"2024-09-16T07:27:47.450313+0000","last_became_peered":"2024-09-16T07:27:47.450313+0000","last_unstale":"2024-09-16T07:27:50.130342+0000","last_undegraded":"2024-09-16T07:27:50.130342+0000","last_fullsized":"2024-09-16T07:27:50.130342+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:01.115128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130354+0000","last_change":"2024-09-16T07:27:48.459503+0000","last_active":"2024-09-16T07:27:50.130354+0000","last_peered":"2024-09-16T07:27:50.130354+0000","last_clean":"2024-09-16T07:27:50.130354+0000","last_became_active":"2024-09-16T07:27:48.459201+0000","last_became_peered":"2024-09-16T07:27:48.459201+0000","last_unstale":"2024-09-16T07:27:50.130354+0000","last_undegraded":"2024-09-16T07:27:50.130354+0000","last_fullsized":"2024-09-16T07:27:50.130354+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:48.451119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129496+0000","last_change":"2024-09-16T07:27:47.451210+0000","last_active":"2024-09-16T07:27:50.129496+0000","last_peered":"2024-09-16T07:27:50.129496+0000","last_clean":"2024-09-16T07:27:50.129496+0000","last_became_active":"2024-09-16T07:27:47.450925+0000","last_became_peered":"2024-09-16T07:27:47.450925+0000","last_unstale":"2024-09-16T07:27:50.129496+0000","last_undegraded":"2024-09-16T07:27:50.129496+0000","last_fullsized":"2024-09-16T07:27:50.129496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:03.229023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.458196+0000","last_change":"2024-09-16T07:27:48.457648+0000","last_active":"2024-09-16T07:27:49.458196+0000","last_peered":"2024-09-16T07:27:49.458196+0000","last_clean":"2024-09-16T07:27:49.458196+0000","last_became_active":"2024-09-16T07:27:48.457358+0000","last_became_peered":"2024-09-16T07:27:48.457358+0000","last_unstale":"2024-09-16T07:27:49.458196+0000","last_undegraded":"2024-09-16T07:27:49.458196+0000","last_fullsized":"2024-09-16T07:27:49.458196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:39:43.900096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625567+0000","last_change":"2024-09-16T07:27:47.458791+0000","last_active":"2024-09-16T07:27:49.625567+0000","last_peered":"2024-09-16T07:27:49.625567+0000","last_clean":"2024-09-16T07:27:49.625567+0000","last_became_active":"2024-09-16T07:27:47.458560+0000","last_became_peered":"2024-09-16T07:27:47.458560+0000","last_unstale":"2024-09-16T07:27:49.625567+0000","last_undegraded":"2024-09-16T07:27:49.625567+0000","last_fullsized":"2024-09-16T07:27:49.625567+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:09.721939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.458194+0000","last_change":"2024-09-16T07:27:48.460172+0000","last_active":"2024-09-16T07:27:49.458194+0000","last_peered":"2024-09-16T07:27:49.458194+0000","last_clean":"2024-09-16T07:27:49.458194+0000","last_became_active":"2024-09-16T07:27:48.459930+0000","last_became_peered":"2024-09-16T07:27:48.459930+0000","last_unstale":"2024-09-16T07:27:49.458194+0000","last_undegraded":"2024-09-16T07:27:49.458194+0000","last_fullsized":"2024-09-16T07:27:49.458194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:45.991164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462914+0000","last_change":"2024-09-16T07:27:47.450135+0000","last_active":"2024-09-16T07:27:49.462914+0000","last_peered":"2024-09-16T07:27:49.462914+0000","last_clean":"2024-09-16T07:27:49.462914+0000","last_became_active":"2024-09-16T07:27:47.449254+0000","last_became_peered":"2024-09-16T07:27:47.449254+0000","last_unstale":"2024-09-16T07:27:49.462914+0000","last_undegraded":"2024-09-16T07:27:49.462914+0000","last_fullsized":"2024-09-16T07:27:49.462914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:42.560778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625761+0000","last_change":"2024-09-16T07:27:48.467859+0000","last_active":"2024-09-16T07:27:49.625761+0000","last_peered":"2024-09-16T07:27:49.625761+0000","last_clean":"2024-09-16T07:27:49.625761+0000","last_became_active":"2024-09-16T07:27:48.467575+0000","last_became_peered":"2024-09-16T07:27:48.467575+0000","last_unstale":"2024-09-16T07:27:49.625761+0000","last_undegraded":"2024-09-16T07:27:49.625761+0000","last_fullsized":"2024-09-16T07:27:49.625761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.751989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.457996+0000","last_change":"2024-09-16T07:27:47.455984+0000","last_active":"2024-09-16T07:27:49.457996+0000","last_peered":"2024-09-16T07:27:49.457996+0000","last_clean":"2024-09-16T07:27:49.457996+0000","last_became_active":"2024-09-16T07:27:47.454791+0000","last_became_peered":"2024-09-16T07:27:47.454791+0000","last_unstale":"2024-09-16T07:27:49.457996+0000","last_undegraded":"2024-09-16T07:27:49.457996+0000","last_fullsized":"2024-09-16T07:27:49.457996+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:24.745832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.130648+0000","last_change":"2024-09-16T07:27:48.462451+0000","last_active":"2024-09-16T07:27:50.130648+0000","last_peered":"2024-09-16T07:27:50.130648+0000","last_clean":"2024-09-16T07:27:50.130648+0000","last_became_active":"2024-09-16T07:27:48.462145+0000","last_became_peered":"2024-09-16T07:27:48.462145+0000","last_unstale":"2024-09-16T07:27:50.130648+0000","last_undegraded":"2024-09-16T07:27:50.130648+0000","last_fullsized":"2024-09-16T07:27:50.130648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:08:14.720566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462931+0000","last_change":"2024-09-16T07:27:47.459943+0000","last_active":"2024-09-16T07:27:49.462931+0000","last_peered":"2024-09-16T07:27:49.462931+0000","last_clean":"2024-09-16T07:27:49.462931+0000","last_became_active":"2024-09-16T07:27:47.459680+0000","last_became_peered":"2024-09-16T07:27:47.459680+0000","last_unstale":"2024-09-16T07:27:49.462931+0000","last_undegraded":"2024-09-16T07:27:49.462931+0000","last_fullsized":"2024-09-16T07:27:49.462931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:14.702032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129495+0000","last_change":"2024-09-16T07:27:48.461989+0000","last_active":"2024-09-16T07:27:50.129495+0000","last_peered":"2024-09-16T07:27:50.129495+0000","last_clean":"2024-09-16T07:27:50.129495+0000","last_became_active":"2024-09-16T07:27:48.461731+0000","last_became_peered":"2024-09-16T07:27:48.461731+0000","last_unstale":"2024-09-16T07:27:50.129495+0000","last_undegraded":"2024-09-16T07:27:50.129495+0000","last_fullsized":"2024-09-16T07:27:50.129495+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:13.804863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.461382+0000","last_change":"2024-09-16T07:27:47.452023+0000","last_active":"2024-09-16T07:27:49.461382+0000","last_peered":"2024-09-16T07:27:49.461382+0000","last_clean":"2024-09-16T07:27:49.461382+0000","last_became_active":"2024-09-16T07:27:47.451738+0000","last_became_peered":"2024-09-16T07:27:47.451738+0000","last_unstale":"2024-09-16T07:27:49.461382+0000","last_undegraded":"2024-09-16T07:27:49.461382+0000","last_fullsized":"2024-09-16T07:27:49.461382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:28.412285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462841+0000","last_change":"2024-09-16T07:27:48.452502+0000","last_active":"2024-09-16T07:27:49.462841+0000","last_peered":"2024-09-16T07:27:49.462841+0000","last_clean":"2024-09-16T07:27:49.462841+0000","last_became_active":"2024-09-16T07:27:48.452178+0000","last_became_peered":"2024-09-16T07:27:48.452178+0000","last_unstale":"2024-09-16T07:27:49.462841+0000","last_undegraded":"2024-09-16T07:27:49.462841+0000","last_fullsized":"2024-09-16T07:27:49.462841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:14:05.565279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462647+0000","last_change":"2024-09-16T07:27:47.452663+0000","last_active":"2024-09-16T07:27:49.462647+0000","last_peered":"2024-09-16T07:27:49.462647+0000","last_clean":"2024-09-16T07:27:49.462647+0000","last_became_active":"2024-09-16T07:27:47.451570+0000","last_became_peered":"2024-09-16T07:27:47.451570+0000","last_unstale":"2024-09-16T07:27:49.462647+0000","last_undegraded":"2024-09-16T07:27:49.462647+0000","last_fullsized":"2024-09-16T07:27:49.462647+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:22.323136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462351+0000","last_change":"2024-09-16T07:27:48.464673+0000","last_active":"2024-09-16T07:27:49.462351+0000","last_peered":"2024-09-16T07:27:49.462351+0000","last_clean":"2024-09-16T07:27:49.462351+0000","last_became_active":"2024-09-16T07:27:48.464121+0000","last_became_peered":"2024-09-16T07:27:48.464121+0000","last_unstale":"2024-09-16T07:27:49.462351+0000","last_undegraded":"2024-09-16T07:27:49.462351+0000","last_fullsized":"2024-09-16T07:27:49.462351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:36.109200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129435+0000","last_change":"2024-09-16T07:27:47.455318+0000","last_active":"2024-09-16T07:27:50.129435+0000","last_peered":"2024-09-16T07:27:50.129435+0000","last_clean":"2024-09-16T07:27:50.129435+0000","last_became_active":"2024-09-16T07:27:47.455047+0000","last_became_peered":"2024-09-16T07:27:47.455047+0000","last_unstale":"2024-09-16T07:27:50.129435+0000","last_undegraded":"2024-09-16T07:27:50.129435+0000","last_fullsized":"2024-09-16T07:27:50.129435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:08.028003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.458364+0000","last_change":"2024-09-16T07:27:48.462125+0000","last_active":"2024-09-16T07:27:49.458364+0000","last_peered":"2024-09-16T07:27:49.458364+0000","last_clean":"2024-09-16T07:27:49.458364+0000","last_became_active":"2024-09-16T07:27:48.461779+0000","last_became_peered":"2024-09-16T07:27:48.461779+0000","last_unstale":"2024-09-16T07:27:49.458364+0000","last_undegraded":"2024-09-16T07:27:49.458364+0000","last_fullsized":"2024-09-16T07:27:49.458364+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:54.848039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625737+0000","last_change":"2024-09-16T07:27:47.461596+0000","last_active":"2024-09-16T07:27:49.625737+0000","last_peered":"2024-09-16T07:27:49.625737+0000","last_clean":"2024-09-16T07:27:49.625737+0000","last_became_active":"2024-09-16T07:27:47.461292+0000","last_became_peered":"2024-09-16T07:27:47.461292+0000","last_unstale":"2024-09-16T07:27:49.625737+0000","last_undegraded":"2024-09-16T07:27:49.625737+0000","last_fullsized":"2024-09-16T07:27:49.625737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:31:20.640980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.458333+0000","last_change":"2024-09-16T07:27:48.452873+0000","last_active":"2024-09-16T07:27:49.458333+0000","last_peered":"2024-09-16T07:27:49.458333+0000","last_clean":"2024-09-16T07:27:49.458333+0000","last_became_active":"2024-09-16T07:27:48.450871+0000","last_became_peered":"2024-09-16T07:27:48.450871+0000","last_unstale":"2024-09-16T07:27:49.458333+0000","last_undegraded":"2024-09-16T07:27:49.458333+0000","last_fullsized":"2024-09-16T07:27:49.458333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:50:26.595355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"62'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.466021+0000","last_change":"2024-09-16T07:27:47.461620+0000","last_active":"2024-09-16T07:27:49.466021+0000","last_peered":"2024-09-16T07:27:49.466021+0000","last_clean":"2024-09-16T07:27:49.466021+0000","last_became_active":"2024-09-16T07:27:47.461338+0000","last_became_peered":"2024-09-16T07:27:47.461338+0000","last_unstale":"2024-09-16T07:27:49.466021+0000","last_undegraded":"2024-09-16T07:27:49.466021+0000","last_fullsized":"2024-09-16T07:27:49.466021+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:23:47.203924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625415+0000","last_change":"2024-09-16T07:27:48.466615+0000","last_active":"2024-09-16T07:27:49.625415+0000","last_peered":"2024-09-16T07:27:49.625415+0000","last_clean":"2024-09-16T07:27:49.625415+0000","last_became_active":"2024-09-16T07:27:48.466324+0000","last_became_peered":"2024-09-16T07:27:48.466324+0000","last_unstale":"2024-09-16T07:27:49.625415+0000","last_undegraded":"2024-09-16T07:27:49.625415+0000","last_fullsized":"2024-09-16T07:27:49.625415+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:22.397174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129917+0000","last_change":"2024-09-16T07:27:47.458964+0000","last_active":"2024-09-16T07:27:50.129917+0000","last_peered":"2024-09-16T07:27:50.129917+0000","last_clean":"2024-09-16T07:27:50.129917+0000","last_became_active":"2024-09-16T07:27:47.458687+0000","last_became_peered":"2024-09-16T07:27:47.458687+0000","last_unstale":"2024-09-16T07:27:50.129917+0000","last_undegraded":"2024-09-16T07:27:50.129917+0000","last_fullsized":"2024-09-16T07:27:50.129917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:43.227646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.463127+0000","last_change":"2024-09-16T07:27:48.458637+0000","last_active":"2024-09-16T07:27:49.463127+0000","last_peered":"2024-09-16T07:27:49.463127+0000","last_clean":"2024-09-16T07:27:49.463127+0000","last_became_active":"2024-09-16T07:27:48.457050+0000","last_became_peered":"2024-09-16T07:27:48.457050+0000","last_unstale":"2024-09-16T07:27:49.463127+0000","last_undegraded":"2024-09-16T07:27:49.463127+0000","last_fullsized":"2024-09-16T07:27:49.463127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:05.328274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.457933+0000","last_change":"2024-09-16T07:27:47.456743+0000","last_active":"2024-09-16T07:27:49.457933+0000","last_peered":"2024-09-16T07:27:49.457933+0000","last_clean":"2024-09-16T07:27:49.457933+0000","last_became_active":"2024-09-16T07:27:47.455261+0000","last_became_peered":"2024-09-16T07:27:47.455261+0000","last_unstale":"2024-09-16T07:27:49.457933+0000","last_undegraded":"2024-09-16T07:27:49.457933+0000","last_fullsized":"2024-09-16T07:27:49.457933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:44.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465285+0000","last_change":"2024-09-16T07:27:48.460258+0000","last_active":"2024-09-16T07:27:49.465285+0000","last_peered":"2024-09-16T07:27:49.465285+0000","last_clean":"2024-09-16T07:27:49.465285+0000","last_became_active":"2024-09-16T07:27:48.459987+0000","last_became_peered":"2024-09-16T07:27:48.459987+0000","last_unstale":"2024-09-16T07:27:49.465285+0000","last_undegraded":"2024-09-16T07:27:49.465285+0000","last_fullsized":"2024-09-16T07:27:49.465285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:58.888447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625341+0000","last_change":"2024-09-16T07:27:47.458363+0000","last_active":"2024-09-16T07:27:49.625341+0000","last_peered":"2024-09-16T07:27:49.625341+0000","last_clean":"2024-09-16T07:27:49.625341+0000","last_became_active":"2024-09-16T07:27:47.455595+0000","last_became_peered":"2024-09-16T07:27:47.455595+0000","last_unstale":"2024-09-16T07:27:49.625341+0000","last_undegraded":"2024-09-16T07:27:49.625341+0000","last_fullsized":"2024-09-16T07:27:49.625341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:43.580966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129891+0000","last_change":"2024-09-16T07:27:48.460471+0000","last_active":"2024-09-16T07:27:50.129891+0000","last_peered":"2024-09-16T07:27:50.129891+0000","last_clean":"2024-09-16T07:27:50.129891+0000","last_became_active":"2024-09-16T07:27:48.456813+0000","last_became_peered":"2024-09-16T07:27:48.456813+0000","last_unstale":"2024-09-16T07:27:50.129891+0000","last_undegraded":"2024-09-16T07:27:50.129891+0000","last_fullsized":"2024-09-16T07:27:50.129891+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:59.562068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.457938+0000","last_change":"2024-09-16T07:27:47.452638+0000","last_active":"2024-09-16T07:27:49.457938+0000","last_peered":"2024-09-16T07:27:49.457938+0000","last_clean":"2024-09-16T07:27:49.457938+0000","last_became_active":"2024-09-16T07:27:47.452390+0000","last_became_peered":"2024-09-16T07:27:47.452390+0000","last_unstale":"2024-09-16T07:27:49.457938+0000","last_undegraded":"2024-09-16T07:27:49.457938+0000","last_fullsized":"2024-09-16T07:27:49.457938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:04.058738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462259+0000","last_change":"2024-09-16T07:27:48.453147+0000","last_active":"2024-09-16T07:27:49.462259+0000","last_peered":"2024-09-16T07:27:49.462259+0000","last_clean":"2024-09-16T07:27:49.462259+0000","last_became_active":"2024-09-16T07:27:48.452670+0000","last_became_peered":"2024-09-16T07:27:48.452670+0000","last_unstale":"2024-09-16T07:27:49.462259+0000","last_undegraded":"2024-09-16T07:27:49.462259+0000","last_fullsized":"2024-09-16T07:27:49.462259+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:13.020955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.465182+0000","last_change":"2024-09-16T07:27:47.460442+0000","last_active":"2024-09-16T07:27:49.465182+0000","last_peered":"2024-09-16T07:27:49.465182+0000","last_clean":"2024-09-16T07:27:49.465182+0000","last_became_active":"2024-09-16T07:27:47.459037+0000","last_became_peered":"2024-09-16T07:27:47.459037+0000","last_unstale":"2024-09-16T07:27:49.465182+0000","last_undegraded":"2024-09-16T07:27:49.465182+0000","last_fullsized":"2024-09-16T07:27:49.465182+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:56.262144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.625201+0000","last_change":"2024-09-16T07:27:48.462184+0000","last_active":"2024-09-16T07:27:49.625201+0000","last_peered":"2024-09-16T07:27:49.625201+0000","last_clean":"2024-09-16T07:27:49.625201+0000","last_became_active":"2024-09-16T07:27:48.461095+0000","last_became_peered":"2024-09-16T07:27:48.461095+0000","last_unstale":"2024-09-16T07:27:49.625201+0000","last_undegraded":"2024-09-16T07:27:49.625201+0000","last_fullsized":"2024-09-16T07:27:49.625201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:56.595977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462451+0000","last_change":"2024-09-16T07:27:47.453007+0000","last_active":"2024-09-16T07:27:49.462451+0000","last_peered":"2024-09-16T07:27:49.462451+0000","last_clean":"2024-09-16T07:27:49.462451+0000","last_became_active":"2024-09-16T07:27:47.452735+0000","last_became_peered":"2024-09-16T07:27:47.452735+0000","last_unstale":"2024-09-16T07:27:49.462451+0000","last_undegraded":"2024-09-16T07:27:49.462451+0000","last_fullsized":"2024-09-16T07:27:49.462451+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:00.396884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:49.462605+0000","last_change":"2024-09-16T07:27:48.457535+0000","last_active":"2024-09-16T07:27:49.462605+0000","last_peered":"2024-09-16T07:27:49.462605+0000","last_clean":"2024-09-16T07:27:49.462605+0000","last_became_active":"2024-09-16T07:27:48.457292+0000","last_became_peered":"2024-09-16T07:27:48.457292+0000","last_unstale":"2024-09-16T07:27:49.462605+0000","last_undegraded":"2024-09-16T07:27:49.462605+0000","last_fullsized":"2024-09-16T07:27:49.462605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:47.428257+0000","last_clean_scrub_stamp":"2024-09-16T07:27:47.428257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:16:35.015801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129610+0000","last_change":"2024-09-16T07:27:47.454175+0000","last_active":"2024-09-16T07:27:50.129610+0000","last_peered":"2024-09-16T07:27:50.129610+0000","last_clean":"2024-09-16T07:27:50.129610+0000","last_became_active":"2024-09-16T07:27:47.453769+0000","last_became_peered":"2024-09-16T07:27:47.453769+0000","last_unstale":"2024-09-16T07:27:50.129610+0000","last_undegraded":"2024-09-16T07:27:50.129610+0000","last_fullsized":"2024-09-16T07:27:50.129610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:27:46.422101+0000","last_clean_scrub_stamp":"2024-09-16T07:27:46.422101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:17.667822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":184,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T07:27:50.129527+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:27:50.129527+0000","last_peered":"2024-09-16T07:27:50.129527+0000","last_clean":"2024-09-16T07:27:50.129527+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:27:50.129527+0000","last_undegraded":"2024-09-16T07:27:50.129527+0000","last_fullsized":"2024-09-16T07:27:50.129527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40232,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331608,"statfs":{"total":96636764160,"available":96595566592,"internally_reserved":0,"allocated":737280,"data_stored":308259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476804,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62452,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309388,"statfs":{"total":96636764160,"available":96572813312,"internally_reserved":0,"allocated":753664,"data_stored":309941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476804,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40812,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331028,"statfs":{"total":96636764160,"available":96594972672,"internally_reserved":0,"allocated":1331200,"data_stored":899119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476803,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40236,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331604,"statfs":{"total":96636764160,"available":96595562496,"internally_reserved":0,"allocated":745472,"data_stored":309781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40232,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331608,"statfs":{"total":96636764160,"available":96595566592,"internally_reserved":0,"allocated":741376,"data_stored":308261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509507,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40232,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331608,"statfs":{"total":96636764160,"available":96595566592,"internally_reserved":0,"allocated":737280,"data_stored":308191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509507,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46376,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325464,"statfs":{"total":96636764160,"available":96589275136,"internally_reserved":0,"allocated":1331200,"data_stored":898571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49960,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321880,"statfs":{"total":96636764160,"available":96585605120,"internally_reserved":0,"allocated":741376,"data_stored":308785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:27:57.724 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:27:57.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:27:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.923+0000 7f319a600640 1 -- 172.21.15.71:0/4207143123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f319406efa0 msgr2=0x7f31940cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.923+0000 7f319a600640 1 --2- 172.21.15.71:0/4207143123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f319406efa0 0x7f31940cdbb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f318c005fd0 tx=0x7f318c02aa10 comp rx=0 tx=0).stop 2024-09-16T07:27:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.923+0000 7f319a600640 1 -- 172.21.15.71:0/4207143123 shutdown_connections 2024-09-16T07:27:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.923+0000 7f319a600640 1 --2- 172.21.15.71:0/4207143123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f319406efa0 0x7f31940cdbb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.923+0000 7f319a600640 1 --2- 172.21.15.71:0/4207143123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31940cccd0 0x7f3194073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.923+0000 7f319a600640 1 -- 172.21.15.71:0/4207143123 >> 172.21.15.71:0/4207143123 conn(0x7f31940b1840 msgr2=0x7f31940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.924+0000 7f319a600640 1 -- 172.21.15.71:0/4207143123 shutdown_connections 2024-09-16T07:27:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.924+0000 7f319a600640 1 -- 172.21.15.71:0/4207143123 wait complete. 2024-09-16T07:27:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.925+0000 7f319a600640 1 Processor -- start 2024-09-16T07:27:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.925+0000 7f319a600640 1 -- start start 2024-09-16T07:27:57.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f319a600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f319406efa0 0x7f319413dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:57.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f319a600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31940cccd0 0x7f31941e6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:57.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f319a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3194145750 0x7f31941f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:57.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f319a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f31940ce480 con 0x7f3194145750 2024-09-16T07:27:57.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f319a600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f31940ce300 con 0x7f319406efa0 2024-09-16T07:27:57.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f319a600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f31940ce600 con 0x7f31940cccd0 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f3193e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3194145750 0x7f31941f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f3193e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3194145750 0x7f31941f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51668/0 (socket says 172.21.15.71:51668) 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.926+0000 7f3193e00640 1 -- 172.21.15.71:0/3523920757 learned_addr learned my addr 172.21.15.71:0/3523920757 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3193400640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f319406efa0 0x7f319413dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3192a00640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31940cccd0 0x7f31941e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3193e00640 1 -- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31940cccd0 msgr2=0x7f31941e6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3193e00640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31940cccd0 0x7f31941e6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3193e00640 1 -- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f319406efa0 msgr2=0x7f319413dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3193e00640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f319406efa0 0x7f319413dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3193e00640 1 -- 172.21.15.71:0/3523920757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31941e4f60 con 0x7f3194145750 2024-09-16T07:27:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3192a00640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31940cccd0 0x7f31941e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f3193e00640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3194145750 0x7f31941f4cc0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f318c006010 tx=0x7f318c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f318b400640 1 -- 172.21.15.71:0/3523920757 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c047070 con 0x7f3194145750 2024-09-16T07:27:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f318b400640 1 -- 172.21.15.71:0/3523920757 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f318c004250 con 0x7f3194145750 2024-09-16T07:27:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.927+0000 7f318b400640 1 -- 172.21.15.71:0/3523920757 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c033050 con 0x7f3194145750 2024-09-16T07:27:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.928+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31941e5240 con 0x7f3194145750 2024-09-16T07:27:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.928+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31941e5800 con 0x7f3194145750 2024-09-16T07:27:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.929+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31940cdbb0 con 0x7f3194145750 2024-09-16T07:27:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.930+0000 7f318b400640 1 -- 172.21.15.71:0/3523920757 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f318c04a070 con 0x7f3194145750 2024-09-16T07:27:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.930+0000 7f318b400640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31740790c0 0x7f317407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.931+0000 7f318b400640 1 -- 172.21.15.71:0/3523920757 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f318c0bcdc0 con 0x7f3194145750 2024-09-16T07:27:57.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.931+0000 7f3193400640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31740790c0 0x7f317407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:57.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.932+0000 7f3193400640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31740790c0 0x7f317407b580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3184001720 tx=0x7f3184009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:57.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:57.935+0000 7f318b400640 1 -- 172.21.15.71:0/3523920757 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f318c083e40 con 0x7f3194145750 2024-09-16T07:27:58.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.187+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f31940b3710 con 0x7f3194145750 2024-09-16T07:27:58.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.191+0000 7f318b400640 1 -- 172.21.15.71:0/3523920757 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f318c088870 con 0x7f3194145750 2024-09-16T07:27:58.191 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:27:58.191 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:27:58.191 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:27:58.191 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:27:58.191 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:27:58.191 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:27:58.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31740790c0 msgr2=0x7f317407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:58.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31740790c0 0x7f317407b580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3184001720 tx=0x7f3184009040 comp rx=0 tx=0).stop 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3194145750 msgr2=0x7f31941f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3194145750 0x7f31941f4cc0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f318c006010 tx=0x7f318c003f10 comp rx=0 tx=0).stop 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 shutdown_connections 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f31740790c0 0x7f317407b580 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3194145750 0x7f31941f4cc0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31940cccd0 0x7f31941e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 --2- 172.21.15.71:0/3523920757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f319406efa0 0x7f319413dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 >> 172.21.15.71:0/3523920757 conn(0x7f31940b1840 msgr2=0x7f31940b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 shutdown_connections 2024-09-16T07:27:58.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.194+0000 7f319a600640 1 -- 172.21.15.71:0/3523920757 wait complete. 2024-09-16T07:27:58.210 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:27:58.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.367+0000 7f32fb800640 1 -- 172.21.15.71:0/2371056347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32f40ccf00 msgr2=0x7f32f40acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:58.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.367+0000 7f32fb800640 1 --2- 172.21.15.71:0/2371056347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32f40ccf00 0x7f32f40acd30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009830 tx=0x7f32e002a410 comp rx=0 tx=0).stop 2024-09-16T07:27:58.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.367+0000 7f32fb800640 1 -- 172.21.15.71:0/2371056347 shutdown_connections 2024-09-16T07:27:58.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.367+0000 7f32fb800640 1 --2- 172.21.15.71:0/2371056347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32f40ccf00 0x7f32f40acd30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.367+0000 7f32fb800640 1 --2- 172.21.15.71:0/2371056347 >> v2:172.21.15.177:3300/0 conn(0x7f32f415f5a0 0x7f32f40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.367+0000 7f32fb800640 1 -- 172.21.15.71:0/2371056347 >> 172.21.15.71:0/2371056347 conn(0x7f32f40b1950 msgr2=0x7f32f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:58.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.367+0000 7f32fb800640 1 -- 172.21.15.71:0/2371056347 shutdown_connections 2024-09-16T07:27:58.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.367+0000 7f32fb800640 1 -- 172.21.15.71:0/2371056347 wait complete. 2024-09-16T07:27:58.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.368+0000 7f32fb800640 1 Processor -- start 2024-09-16T07:27:58.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.369+0000 7f32fb800640 1 -- start start 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32fb800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32f40ad270 0x7f32f41f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32fb800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32f40ccf00 0x7f32f40cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32fb800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32f415f5a0 0x7f32f40d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32fb800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f32f406f2e0 con 0x7f32f40ccf00 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32fb800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32f406f160 con 0x7f32f415f5a0 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32fb800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f32f406f460 con 0x7f32f40ad270 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32f40ccf00 0x7f32f40cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32f40ccf00 0x7f32f40cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51676/0 (socket says 172.21.15.71:51676) 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f3e00640 1 -- 172.21.15.71:0/1515811995 learned_addr learned my addr 172.21.15.71:0/1515811995 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:58.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f8e00640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32f40ad270 0x7f32f41f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f3e00640 1 -- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32f40ad270 msgr2=0x7f32f41f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f3e00640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32f40ad270 0x7f32f41f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f9800640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32f415f5a0 0x7f32f40d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f3e00640 1 -- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32f415f5a0 msgr2=0x7f32f40d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f3e00640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32f415f5a0 0x7f32f40d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f3e00640 1 -- 172.21.15.71:0/1515811995 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f415c8b0 con 0x7f32f40ccf00 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.370+0000 7f32f9800640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32f415f5a0 0x7f32f40d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.371+0000 7f32f3e00640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32f40ccf00 0x7f32f40cfb30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009950 tx=0x7f32e0003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.371+0000 7f32f0c00640 1 -- 172.21.15.71:0/1515811995 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e003f030 con 0x7f32f40ccf00 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.371+0000 7f32f0c00640 1 -- 172.21.15.71:0/1515811995 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32e0004510 con 0x7f32f40ccf00 2024-09-16T07:27:58.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.371+0000 7f32f0c00640 1 -- 172.21.15.71:0/1515811995 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0037040 con 0x7f32f40ccf00 2024-09-16T07:27:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.371+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f40b4bf0 con 0x7f32f40ccf00 2024-09-16T07:27:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.371+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f40b3030 con 0x7f32f40ccf00 2024-09-16T07:27:58.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.372+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f406fa20 con 0x7f32f40ccf00 2024-09-16T07:27:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.373+0000 7f32f0c00640 1 -- 172.21.15.71:0/1515811995 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f32e0042070 con 0x7f32f40ccf00 2024-09-16T07:27:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.374+0000 7f32f0c00640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f32d40790c0 0x7f32d407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.374+0000 7f32f0c00640 1 -- 172.21.15.71:0/1515811995 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f32e00bb8b0 con 0x7f32f40ccf00 2024-09-16T07:27:58.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.375+0000 7f32f8e00640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f32d40790c0 0x7f32d407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:58.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.375+0000 7f32f8e00640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f32d40790c0 0x7f32d407b580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f32f409df90 tx=0x7f32e4009500 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:58.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.378+0000 7f32f0c00640 1 -- 172.21.15.71:0/1515811995 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f32e0049070 con 0x7f32f40ccf00 2024-09-16T07:27:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.599+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32f4155a80 con 0x7f32f40ccf00 2024-09-16T07:27:58.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.600+0000 7f32f0c00640 1 -- 172.21.15.71:0/1515811995 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7f32f415c8b0 con 0x7f32f40ccf00 2024-09-16T07:27:58.599 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:27:58.599 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":37,"btime":"2024-09-16T07:27:51:464479+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1090989192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1090989192},{"type":"v1","addr":"172.21.15.177:6836","nonce":1090989192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2044137966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2044137966},{"type":"v1","addr":"172.21.15.177:6837","nonce":2044137966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3911470669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3911470669},{"type":"v1","addr":"172.21.15.71:6835","nonce":3911470669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:27:47.864418+0000","modified":"2024-09-16T07:27:50.881337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6834/3242207389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3242207389},{"type":"v1","addr":"172.21.15.71:6834","nonce":3242207389}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5964,"qdb_cluster":[5964]},"id":4}]} 2024-09-16T07:27:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 37 2024-09-16T07:27:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f32d40790c0 msgr2=0x7f32d407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f32d40790c0 0x7f32d407b580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f32f409df90 tx=0x7f32e4009500 comp rx=0 tx=0).stop 2024-09-16T07:27:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32f40ccf00 msgr2=0x7f32f40cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32f40ccf00 0x7f32f40cfb30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009950 tx=0x7f32e0003b60 comp rx=0 tx=0).stop 2024-09-16T07:27:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 shutdown_connections 2024-09-16T07:27:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f32d40790c0 0x7f32d407b580 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32f415f5a0 0x7f32f40d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32f40ccf00 0x7f32f40cfb30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 --2- 172.21.15.71:0/1515811995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32f40ad270 0x7f32f41f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 >> 172.21.15.71:0/1515811995 conn(0x7f32f40b1950 msgr2=0x7f32f40ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 shutdown_connections 2024-09-16T07:27:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:58.602+0000 7f32fb800640 1 -- 172.21.15.71:0/1515811995 wait complete. 2024-09-16T07:27:59.020 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:27:47.864418+0000', 'modified': '2024-09-16T07:27:50.881337+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5964}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5964': {'gid': 5964, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6834/3242207389', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6833', 'nonce': 3242207389}, {'type': 'v1', 'addr': '172.21.15.71:6834', 'nonce': 3242207389}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5964, 'qdb_cluster': [5964]} 2024-09-16T07:27:59.020 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:27:59.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T07:27:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.180+0000 7fa0edc00640 1 -- 172.21.15.71:0/439420777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e80d0540 msgr2=0x7fa0e806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.180+0000 7fa0edc00640 1 --2- 172.21.15.71:0/439420777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e80d0540 0x7fa0e806f320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0005fd0 tx=0x7fa0e002a530 comp rx=0 tx=0).stop 2024-09-16T07:27:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.180+0000 7fa0edc00640 1 -- 172.21.15.71:0/439420777 shutdown_connections 2024-09-16T07:27:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.180+0000 7fa0edc00640 1 --2- 172.21.15.71:0/439420777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e80d0540 0x7fa0e806f320 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.180+0000 7fa0edc00640 1 --2- 172.21.15.71:0/439420777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0e815bc30 0x7fa0e80b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.180+0000 7fa0edc00640 1 -- 172.21.15.71:0/439420777 >> 172.21.15.71:0/439420777 conn(0x7fa0e80cfa20 msgr2=0x7fa0e80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:59.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.180+0000 7fa0edc00640 1 -- 172.21.15.71:0/439420777 shutdown_connections 2024-09-16T07:27:59.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.180+0000 7fa0edc00640 1 -- 172.21.15.71:0/439420777 wait complete. 2024-09-16T07:27:59.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.182+0000 7fa0edc00640 1 Processor -- start 2024-09-16T07:27:59.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.182+0000 7fa0edc00640 1 -- start start 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0edc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0e80d0540 0x7fa0e814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0edc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0e815bc30 0x7fa0e80c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0edc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e81f2380 0x7fa0e815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0edc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e8074660 con 0x7fa0e81f2380 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0edc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e815c200 con 0x7fa0e815bc30 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0edc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e80c7000 con 0x7fa0e80d0540 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0ed200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e81f2380 0x7fa0e815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0ed200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e81f2380 0x7fa0e815a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51702/0 (socket says 172.21.15.71:51702) 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0ed200640 1 -- 172.21.15.71:0/2353913732 learned_addr learned my addr 172.21.15.71:0/2353913732 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0e80d0540 0x7fa0e814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.183+0000 7fa0e7400640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0e815bc30 0x7fa0e80c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0ed200640 1 -- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0e80d0540 msgr2=0x7fa0e814f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0ed200640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0e80d0540 0x7fa0e814f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0ed200640 1 -- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0e815bc30 msgr2=0x7fa0e80c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0ed200640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0e815bc30 0x7fa0e80c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0ed200640 1 -- 172.21.15.71:0/2353913732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e814f680 con 0x7fa0e81f2380 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0ed200640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e81f2380 0x7fa0e815a9a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0006010 tx=0x7fa0e0003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0dfe00640 1 -- 172.21.15.71:0/2353913732 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e003f070 con 0x7fa0e81f2380 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0dfe00640 1 -- 172.21.15.71:0/2353913732 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0e0004890 con 0x7fa0e81f2380 2024-09-16T07:27:59.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0edc00640 1 -- 172.21.15.71:0/2353913732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e81f2dd0 con 0x7fa0e81f2380 2024-09-16T07:27:59.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0dfe00640 1 -- 172.21.15.71:0/2353913732 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0035030 con 0x7fa0e81f2380 2024-09-16T07:27:59.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.184+0000 7fa0edc00640 1 -- 172.21.15.71:0/2353913732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e80c6c90 con 0x7fa0e81f2380 2024-09-16T07:27:59.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.187+0000 7fa0dfe00640 1 -- 172.21.15.71:0/2353913732 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fa0e0042070 con 0x7fa0e81f2380 2024-09-16T07:27:59.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.188+0000 7fa0dfe00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0c80790c0 0x7fa0c807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.188+0000 7fa0dfe00640 1 -- 172.21.15.71:0/2353913732 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa0e00bcb30 con 0x7fa0e81f2380 2024-09-16T07:27:59.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.188+0000 7fa0f21f8740 1 Processor -- start 2024-09-16T07:27:59.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.188+0000 7fa0f21f8740 1 -- start start 2024-09-16T07:27:59.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.188+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0c80790c0 0x7fa0c807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55970119eaa0 0x55970119ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 0x55970117f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5597011531f0 0x55970117fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5597010dc430 con 0x55970119eaa0 2024-09-16T07:27:59.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5597010cc5d0 con 0x559701154110 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5597010bba00 con 0x5597011531f0 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0c80790c0 0x7fa0c807b580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0e81f34f0 tx=0x7fa0d800d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55970119eaa0 0x55970119ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55970119eaa0 0x55970119ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51706/0 (socket says 172.21.15.71:51706) 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 -- 172.21.15.71:0/3357951515 learned_addr learned my addr 172.21.15.71:0/3357951515 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7400640 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 0x55970117f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0ed200640 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5597011531f0 0x55970117fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 -- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5597011531f0 msgr2=0x55970117fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5597011531f0 0x55970117fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 -- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 msgr2=0x55970117f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 0x55970117f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0e7e00640 1 -- 172.21.15.71:0/3357951515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559701084520 con 0x55970119eaa0 2024-09-16T07:27:59.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.189+0000 7fa0ed200640 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5597011531f0 0x55970117fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.190+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55970119eaa0 0x55970119ee80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009900 tx=0x7fa0d8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea661c0e01e3089e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.190+0000 7fa0dcc00640 1 -- 172.21.15.71:0/3357951515 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8015a30 con 0x55970119eaa0 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.190+0000 7fa0dcc00640 1 -- 172.21.15.71:0/3357951515 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0d8025070 con 0x55970119eaa0 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.190+0000 7fa0dcc00640 1 -- 172.21.15.71:0/3357951515 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80203f0 con 0x55970119eaa0 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.190+0000 7fa0f21f8740 1 -- 172.21.15.71:0/3357951515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55970119eaa0 msgr2=0x55970119ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.190+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55970119eaa0 0x55970119ee80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009900 tx=0x7fa0d8011000 comp rx=0 tx=0).stop 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.191+0000 7fa0f21f8740 1 -- 172.21.15.71:0/3357951515 shutdown_connections 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.191+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5597011531f0 0x55970117fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.191+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 0x55970117f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.191+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/3357951515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55970119eaa0 0x55970119ee80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.191+0000 7fa0f21f8740 1 -- 172.21.15.71:0/3357951515 >> 172.21.15.71:0/3357951515 conn(0x5597011551e0 msgr2=0x559701158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:59.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.191+0000 7fa0f21f8740 1 -- 172.21.15.71:0/3357951515 shutdown_connections 2024-09-16T07:27:59.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.191+0000 7fa0f21f8740 1 -- 172.21.15.71:0/3357951515 wait complete. 2024-09-16T07:27:59.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.192+0000 7fa0f21f8740 1 -- start start 2024-09-16T07:27:59.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.192+0000 7fa0f21f8740 1 Processor -- start 2024-09-16T07:27:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x559701181ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55970119eaa0 0x559701180e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559701174860 0x559701183bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559701062330 con 0x559701154110 2024-09-16T07:27:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5597010bf810 con 0x55970119eaa0 2024-09-16T07:27:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x559700e2c620 con 0x559701174860 2024-09-16T07:27:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0f21f8740 10 client.0 _ll_register_callbacks cb 0x559701067420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:27:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x559701181ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x559701181ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51720/0 (socket says 172.21.15.71:51720) 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0e7e00640 1 -- 172.21.15.71:0/2303809986 learned_addr learned my addr 172.21.15.71:0/2303809986 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0e7400640 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55970119eaa0 0x559701180e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.193+0000 7fa0ed200640 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559701174860 0x559701183bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.194+0000 7fa0e7e00640 1 -- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559701174860 msgr2=0x559701183bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.194+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559701174860 0x559701183bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.194+0000 7fa0e7e00640 1 -- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55970119eaa0 msgr2=0x559701180e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.194+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55970119eaa0 0x559701180e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.194+0000 7fa0e7e00640 1 -- 172.21.15.71:0/2303809986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559700e1d850 con 0x559701154110 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.194+0000 7fa0ed200640 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559701174860 0x559701183bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.194+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x559701181ef0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8022040 tx=0x7fa0d8015d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51766dcba66c397b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.194+0000 7fa0d1600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0d1600640 1 -- 172.21.15.71:0/2303809986 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d802f070 con 0x559701154110 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0d1600640 1 -- 172.21.15.71:0/2303809986 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa0d8020b40 con 0x559701154110 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2303809986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d8025070 con 0x559701154110 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0d1600640 1 -- 172.21.15.71:0/2303809986 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa0d8016410 con 0x559701154110 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 10 client.6090 fetch_fsmap learned FSMap version 37 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2303809986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x559700e946b0 con 0x559701154110 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0d1600640 1 -- 172.21.15.71:0/2303809986 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0d8025070 con 0x559701154110 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0d1600640 1 -- 172.21.15.71:0/2303809986 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7fa0d8034440 con 0x559701154110 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 10 client.6090 fetch_fsmap finished waiting for FSMap version 37 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 10 client.6090 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T07:27:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 20 client.6090 populate_metadata read hostname 'smithi071' 2024-09-16T07:27:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] conn(0x5597011b5420 0x5597011b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 4 client.6090 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:27:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.195+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2303809986 --> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597010ce520 con 0x5597011b5420 2024-09-16T07:27:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.196+0000 7fa0e7400640 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] conn(0x5597011b5420 0x5597011b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.196+0000 7fa0e7400640 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] conn(0x5597011b5420 0x5597011b5800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eff8e2fb3a1bd5d7 server_cookie=7dc203be220159cf in_seq=0 out_seq=0 2024-09-16T07:27:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.196+0000 7fa0d1600640 10 client.6090 ms_handle_connect on v2:172.21.15.71:6833/3242207389 2024-09-16T07:27:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0d1600640 1 -- 172.21.15.71:0/2303809986 <== mds.0 v2:172.21.15.71:6833/3242207389 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5597010ce520 con 0x5597011b5420 2024-09-16T07:27:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0d1600640 10 client.6090 handle_command_reply: tid=0 2024-09-16T07:27:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 client.6090 shutdown 2024-09-16T07:27:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2303809986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 msgr2=0x559701181ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x559701181ef0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8022040 tx=0x7fa0d8015d90 comp rx=0 tx=0).stop 2024-09-16T07:27:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2303809986 shutdown_connections 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] conn(0x5597011b5420 0x5597011b5800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559701174860 0x559701183bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55970119eaa0 0x559701180e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2303809986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x559701181ef0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2303809986 >> 172.21.15.71:0/2303809986 conn(0x5597011531f0 msgr2=0x5597011586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0d1600640 0 client.6090 ms_handle_reset on v2:172.21.15.71:6833/3242207389 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.197+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2303809986 shutdown_connections 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.198+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2303809986 wait complete. 2024-09-16T07:27:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.198+0000 7fa0f21f8740 20 client.6090 trim_cache size 0 max 16384 2024-09-16T07:27:59.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.216+0000 7fa0f21f8740 1 Processor -- start 2024-09-16T07:27:59.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.217+0000 7fa0f21f8740 1 -- start start 2024-09-16T07:27:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.217+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x5597011bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.217+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701174860 0x5597011c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.217+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701185a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.217+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e80ceb60 con 0x559701154110 2024-09-16T07:27:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.217+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e80c8960 con 0x559701174860 2024-09-16T07:27:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.217+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e81f6680 con 0x55970119eaa0 2024-09-16T07:27:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x5597011bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x5597011bbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51736/0 (socket says 172.21.15.71:51736) 2024-09-16T07:27:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 -- 172.21.15.71:0/2510817433 learned_addr learned my addr 172.21.15.71:0/2510817433 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0ed200640 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701185a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 -- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 msgr2=0x559701185a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:27:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701185a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7400640 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701174860 0x5597011c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 -- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701174860 msgr2=0x5597011c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:27:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701174860 0x5597011c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 -- 172.21.15.71:0/2510817433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597011a0920 con 0x559701154110 2024-09-16T07:27:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0ed200640 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701185a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7400640 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701174860 0x5597011c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:27:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.218+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x5597011bbb00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80165d0 tx=0x7fa0d8009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cae0c5a5aa240bb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0d3e00640 1 -- 172.21.15.71:0/2510817433 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d802f040 con 0x559701154110 2024-09-16T07:27:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0d3e00640 1 -- 172.21.15.71:0/2510817433 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0d8025070 con 0x559701154110 2024-09-16T07:27:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0d3e00640 1 -- 172.21.15.71:0/2510817433 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d803a020 con 0x559701154110 2024-09-16T07:27:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2510817433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 msgr2=0x5597011bbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x5597011bbb00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80165d0 tx=0x7fa0d8009480 comp rx=0 tx=0).stop 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2510817433 shutdown_connections 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701185a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701174860 0x5597011c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/2510817433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701154110 0x5597011bbb00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2510817433 >> 172.21.15.71:0/2510817433 conn(0x5597011531f0 msgr2=0x5597011535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2510817433 shutdown_connections 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.219+0000 7fa0f21f8740 1 -- 172.21.15.71:0/2510817433 wait complete. 2024-09-16T07:27:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.220+0000 7fa0f21f8740 1 -- start start 2024-09-16T07:27:59.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.220+0000 7fa0f21f8740 1 Processor -- start 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 0x559701156f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701174860 0x5597011c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0f21f8740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701225920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5597011843f0 con 0x559701174860 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55970117c1a0 con 0x559701154110 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0f21f8740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x559701174cc0 con 0x55970119eaa0 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0f21f8740 10 client.0 _ll_register_callbacks cb 0x7fa0d8004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0e7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701174860 0x5597011c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0e7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701174860 0x5597011c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51750/0 (socket says 172.21.15.71:51750) 2024-09-16T07:27:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.221+0000 7fa0e7400640 1 -- 172.21.15.71:0/4131029710 learned_addr learned my addr 172.21.15.71:0/4131029710 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 0x559701156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0ed200640 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701225920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0e7400640 1 -- 172.21.15.71:0/4131029710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 msgr2=0x559701225920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0e7400640 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701225920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0e7400640 1 -- 172.21.15.71:0/4131029710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 msgr2=0x559701156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0e7400640 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 0x559701156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0e7400640 1 -- 172.21.15.71:0/4131029710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55970117b3d0 con 0x559701174860 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0e7400640 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701174860 0x5597011c48a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4000c00 tx=0x7fa0d4005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36475d84b57cd26c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0d1600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0d1600640 1 -- 172.21.15.71:0/4131029710 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4003cf0 con 0x559701174860 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0d1600640 1 -- 172.21.15.71:0/4131029710 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa0d4007030 con 0x559701174860 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.222+0000 7fa0f21f8740 1 -- 172.21.15.71:0/4131029710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d8025070 con 0x559701174860 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0d1600640 1 -- 172.21.15.71:0/4131029710 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa0d4007230 con 0x559701174860 2024-09-16T07:27:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0f21f8740 10 client.6096 fetch_fsmap learned FSMap version 37 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0f21f8740 1 -- 172.21.15.71:0/4131029710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x559701157490 con 0x559701174860 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0d1600640 1 -- 172.21.15.71:0/4131029710 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0d8025070 con 0x559701174860 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0d1600640 1 -- 172.21.15.71:0/4131029710 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7fa0d40073d0 con 0x559701174860 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0f21f8740 10 client.6096 fetch_fsmap finished waiting for FSMap version 37 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0f21f8740 10 client.6096 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0f21f8740 20 client.6096 populate_metadata read hostname 'smithi071' 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] conn(0x559701221e90 0x5597011b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0f21f8740 4 client.6096 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0f21f8740 1 -- 172.21.15.71:0/4131029710 --> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] -- command(tid 0: {"prefix": "status"}) -- 0x5597011a0530 con 0x559701221e90 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.223+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] conn(0x559701221e90 0x5597011b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0d1600640 10 client.6096 ms_handle_connect on v2:172.21.15.71:6833/3242207389 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0e7e00640 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] conn(0x559701221e90 0x5597011b4210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85feb7321361df71 server_cookie=a1f7d298a2699ce9 in_seq=0 out_seq=0 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0d1600640 1 -- 172.21.15.71:0/4131029710 <== mds.0 v2:172.21.15.71:6833/3242207389 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x5597011a0530 con 0x559701221e90 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0d1600640 10 client.6096 handle_command_reply: tid=0 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 client.6096 shutdown 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 -- 172.21.15.71:0/4131029710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701174860 msgr2=0x5597011c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701174860 0x5597011c48a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4000c00 tx=0x7fa0d4005420 comp rx=0 tx=0).stop 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 -- 172.21.15.71:0/4131029710 shutdown_connections 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.71:6833/3242207389,v1:172.21.15.71:6834/3242207389] conn(0x559701221e90 0x5597011b4210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55970119eaa0 0x559701225920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559701174860 0x5597011c48a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0d1600640 0 client.6096 ms_handle_reset on v2:172.21.15.71:6833/3242207389 2024-09-16T07:27:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 --2- 172.21.15.71:0/4131029710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559701154110 0x559701156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 -- 172.21.15.71:0/4131029710 >> 172.21.15.71:0/4131029710 conn(0x5597011531f0 msgr2=0x5597011535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 -- 172.21.15.71:0/4131029710 shutdown_connections 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.224+0000 7fa0f21f8740 1 -- 172.21.15.71:0/4131029710 wait complete. 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0f21f8740 20 client.6096 trim_cache size 0 max 16384 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5964, 2024-09-16T07:27:59.232 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:27:59.233 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:27:59.233 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:27:59.233 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 10.775358072, 2024-09-16T07:27:59.233 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 37, 2024-09-16T07:27:59.233 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 62, 2024-09-16T07:27:59.233 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 62, 2024-09-16T07:27:59.233 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 15.096948249 2024-09-16T07:27:59.233 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 -- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0c80790c0 msgr2=0x7fa0c807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0c80790c0 0x7fa0c807b580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0e81f34f0 tx=0x7fa0d800d040 comp rx=0 tx=0).stop 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 -- 172.21.15.71:0/2353913732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e81f2380 msgr2=0x7fa0e815a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e81f2380 0x7fa0e815a9a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0006010 tx=0x7fa0e0003960 comp rx=0 tx=0).stop 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 -- 172.21.15.71:0/2353913732 shutdown_connections 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0c80790c0 0x7fa0c807b580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0e81f2380 0x7fa0e815a9a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0e815bc30 0x7fa0e80c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 --2- 172.21.15.71:0/2353913732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0e80d0540 0x7fa0e814f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 -- 172.21.15.71:0/2353913732 >> 172.21.15.71:0/2353913732 conn(0x7fa0e80cfa20 msgr2=0x7fa0e8150ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:59.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 -- 172.21.15.71:0/2353913732 shutdown_connections 2024-09-16T07:27:59.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.225+0000 7fa0edc00640 1 -- 172.21.15.71:0/2353913732 wait complete. 2024-09-16T07:27:59.240 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:27:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.395+0000 7f665d800640 1 -- 172.21.15.71:0/3677598071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6658161440 msgr2=0x7f66580b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.395+0000 7f665d800640 1 --2- 172.21.15.71:0/3677598071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6658161440 0x7f66580b5370 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6648005fd0 tx=0x7f664802aa10 comp rx=0 tx=0).stop 2024-09-16T07:27:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.396+0000 7f665d800640 1 -- 172.21.15.71:0/3677598071 shutdown_connections 2024-09-16T07:27:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.396+0000 7f665d800640 1 --2- 172.21.15.71:0/3677598071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66580b4050 0x7f66580b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.396+0000 7f665d800640 1 --2- 172.21.15.71:0/3677598071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6658161440 0x7f66580b5370 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.396+0000 7f665d800640 1 -- 172.21.15.71:0/3677598071 >> 172.21.15.71:0/3677598071 conn(0x7f66580d3df0 msgr2=0x7f66580d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.396+0000 7f665d800640 1 -- 172.21.15.71:0/3677598071 shutdown_connections 2024-09-16T07:27:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.396+0000 7f665d800640 1 -- 172.21.15.71:0/3677598071 wait complete. 2024-09-16T07:27:59.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.397+0000 7f665d800640 1 Processor -- start 2024-09-16T07:27:59.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.397+0000 7f665d800640 1 -- start start 2024-09-16T07:27:59.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.398+0000 7f665d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580b34b0 0x7f66580b8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.398+0000 7f665d800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66580b4050 0x7f665814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.398+0000 7f665d800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6658161440 0x7f66580c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.398+0000 7f665d800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f66580b2040 con 0x7f66580b34b0 2024-09-16T07:27:59.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.398+0000 7f665d800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66580b1ec0 con 0x7f66580b4050 2024-09-16T07:27:59.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.398+0000 7f665d800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f66580b21c0 con 0x7f6658161440 2024-09-16T07:27:59.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.398+0000 7f6657e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580b34b0 0x7f66580b8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f6657e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580b34b0 0x7f66580b8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51780/0 (socket says 172.21.15.71:51780) 2024-09-16T07:27:59.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f6657e00640 1 -- 172.21.15.71:0/2448670893 learned_addr learned my addr 172.21.15.71:0/2448670893 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:27:59.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f665ce00640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6658161440 0x7f66580c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f6657e00640 1 -- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6658161440 msgr2=0x7f66580c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f6657e00640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6658161440 0x7f66580c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f6657e00640 1 -- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66580b4050 msgr2=0x7f665814e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f6657e00640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66580b4050 0x7f665814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f6657e00640 1 -- 172.21.15.71:0/2448670893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66580b96c0 con 0x7f66580b34b0 2024-09-16T07:27:59.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f665ce00640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6658161440 0x7f66580c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:27:59.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f6657e00640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580b34b0 0x7f66580b8290 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6648000f80 tx=0x7f6648003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:59.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f664fe00640 1 -- 172.21.15.71:0/2448670893 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648046070 con 0x7f66580b34b0 2024-09-16T07:27:59.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f664fe00640 1 -- 172.21.15.71:0/2448670893 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6648004540 con 0x7f66580b34b0 2024-09-16T07:27:59.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.399+0000 7f664fe00640 1 -- 172.21.15.71:0/2448670893 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648033070 con 0x7f66580b34b0 2024-09-16T07:27:59.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.400+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66580b77b0 con 0x7f66580b34b0 2024-09-16T07:27:59.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.400+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6658156600 con 0x7f66580b34b0 2024-09-16T07:27:59.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.401+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66580b12a0 con 0x7f66580b34b0 2024-09-16T07:27:59.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.402+0000 7f664fe00640 1 -- 172.21.15.71:0/2448670893 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f6648049070 con 0x7f66580b34b0 2024-09-16T07:27:59.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.403+0000 7f664fe00640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f66380790c0 0x7f663807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:27:59.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.403+0000 7f664fe00640 1 -- 172.21.15.71:0/2448670893 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f66480bcd10 con 0x7f66580b34b0 2024-09-16T07:27:59.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.403+0000 7f6657400640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f66380790c0 0x7f663807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:27:59.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.404+0000 7f6657400640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f66380790c0 0x7f663807b580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6644000f60 tx=0x7f66440079d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:27:59.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.407+0000 7f664fe00640 1 -- 172.21.15.71:0/2448670893 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66580b12a0 con 0x7f66580b34b0 2024-09-16T07:27:59.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.631+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f66580b1770 con 0x7f66580b34b0 2024-09-16T07:27:59.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.631+0000 7f664fe00640 1 -- 172.21.15.71:0/2448670893 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) ==== 53+0+73 (secure 0 0 0) 0x7f664803e070 con 0x7f66580b34b0 2024-09-16T07:27:59.631 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:27:59.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f66380790c0 msgr2=0x7f663807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f66380790c0 0x7f663807b580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6644000f60 tx=0x7f66440079d0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580b34b0 msgr2=0x7f66580b8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:27:59.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580b34b0 0x7f66580b8290 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6648000f80 tx=0x7f6648003f90 comp rx=0 tx=0).stop 2024-09-16T07:27:59.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 shutdown_connections 2024-09-16T07:27:59.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f66380790c0 0x7f663807b580 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6658161440 0x7f66580c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66580b4050 0x7f665814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 --2- 172.21.15.71:0/2448670893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580b34b0 0x7f66580b8290 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:27:59.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.633+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 >> 172.21.15.71:0/2448670893 conn(0x7f66580d3df0 msgr2=0x7f665815fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:27:59.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.634+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 shutdown_connections 2024-09-16T07:27:59.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59.634+0000 7f665d800640 1 -- 172.21.15.71:0/2448670893 wait complete. 2024-09-16T07:27:59.648 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:27:59.648 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:27:59.648 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:27:59.648 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:27:59.648 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:27:59.648 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:27:59.649 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:27:59.649 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:27:59.649 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:27:59.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:27:59.649 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:27:59.685 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42397sec preferred_lft 42397sec 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:27:59.686 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:27:59.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:27:59.688 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:27:59.778 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:27:59.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:27:59.781 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:27:59.817 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:27:59.817 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:27:59.817 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:27:59.818 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:27:59.818 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:27:59.818 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:27:59.818 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:27:59.818 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:27:59.818 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:27:59.818 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:27:59.818 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:27:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:27:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:28:00.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:28:00.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:00.014 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:28:00.049 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:28:00.049 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:28:00.049 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:28:00.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:00.050 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:28:00.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:28:00.147 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:28:00.181 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:28:00.182 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:28:00.182 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:28:00.257 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:28:00.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:00.257 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:28:00.385 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:28:00.386 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:28:00.412 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:28:00.412 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:28:00.412 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:28:00.412 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:28:00.412 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:28:00.414 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:28:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:28:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:28:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:28:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:28:00.417 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:28:00.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:00.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:28:00.474 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:28:00.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:28:00.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:28:00.570 INFO:teuthology.orchestra.run.smithi071.stdout:client6108 2024-09-16T07:28:00.570 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:00.570 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:00.605 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:28:00.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 -- 172.21.15.71:0/595444907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa62815f390 msgr2=0x7fa6280cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:00.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 --2- 172.21.15.71:0/595444907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa62815f390 0x7fa6280cfb90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa61c005fd0 tx=0x7fa61c02a580 comp rx=0 tx=0).stop 2024-09-16T07:28:00.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 -- 172.21.15.71:0/595444907 shutdown_connections 2024-09-16T07:28:00.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 --2- 172.21.15.71:0/595444907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6280b2c70 0x7fa6280d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:00.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 --2- 172.21.15.71:0/595444907 >> v2:172.21.15.177:3300/0 conn(0x7fa6280b2760 0x7fa6280d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:00.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 --2- 172.21.15.71:0/595444907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa62815f390 0x7fa6280cfb90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:00.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 -- 172.21.15.71:0/595444907 >> 172.21.15.71:0/595444907 conn(0x7fa6280d1db0 msgr2=0x7fa6280d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:00.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 -- 172.21.15.71:0/595444907 shutdown_connections 2024-09-16T07:28:00.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.806+0000 7fa630200640 1 -- 172.21.15.71:0/595444907 wait complete. 2024-09-16T07:28:00.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.807+0000 7fa630200640 1 Processor -- start 2024-09-16T07:28:00.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.807+0000 7fa630200640 1 -- start start 2024-09-16T07:28:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.808+0000 7fa630200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6280b2760 0x7fa6281f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.808+0000 7fa630200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6280b2c70 0x7fa62815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.808+0000 7fa630200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62815f390 0x7fa6281f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.808+0000 7fa630200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa6280aee10 con 0x7fa6280b2c70 2024-09-16T07:28:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.808+0000 7fa630200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa6280aec90 con 0x7fa6280b2760 2024-09-16T07:28:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.808+0000 7fa630200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa6280aef90 con 0x7fa62815f390 2024-09-16T07:28:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6280b2c70 0x7fa62815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:00.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6280b2c70 0x7fa62815a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51810/0 (socket says 172.21.15.71:51810) 2024-09-16T07:28:00.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 -- 172.21.15.71:0/727204300 learned_addr learned my addr 172.21.15.71:0/727204300 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:28:00.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62d800640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6280b2760 0x7fa6281f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:00.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62e200640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62815f390 0x7fa6281f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:00.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 -- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62815f390 msgr2=0x7fa6281f8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:00.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62815f390 0x7fa6281f8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:00.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 -- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6280b2760 msgr2=0x7fa6281f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:00.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6280b2760 0x7fa6281f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:00.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 -- 172.21.15.71:0/727204300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6281f8d70 con 0x7fa6280b2c70 2024-09-16T07:28:00.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.809+0000 7fa62ce00640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6280b2c70 0x7fa62815a500 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa614005940 tx=0x7fa61400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:00.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.810+0000 7fa625600640 1 -- 172.21.15.71:0/727204300 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614007bb0 con 0x7fa6280b2c70 2024-09-16T07:28:00.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.810+0000 7fa625600640 1 -- 172.21.15.71:0/727204300 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa614007e40 con 0x7fa6280b2c70 2024-09-16T07:28:00.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.810+0000 7fa625600640 1 -- 172.21.15.71:0/727204300 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61400f6b0 con 0x7fa6280b2c70 2024-09-16T07:28:00.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.810+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6281f9050 con 0x7fa6280b2c70 2024-09-16T07:28:00.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.810+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6281f5290 con 0x7fa6280b2c70 2024-09-16T07:28:00.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.811+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62806aca0 con 0x7fa6280b2c70 2024-09-16T07:28:00.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.812+0000 7fa625600640 1 -- 172.21.15.71:0/727204300 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fa61400f850 con 0x7fa6280b2c70 2024-09-16T07:28:00.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.813+0000 7fa625600640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6080790c0 0x7fa60807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:00.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.813+0000 7fa625600640 1 -- 172.21.15.71:0/727204300 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa61409c360 con 0x7fa6280b2c70 2024-09-16T07:28:00.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.813+0000 7fa62d800640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6080790c0 0x7fa60807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:00.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.814+0000 7fa62d800640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6080790c0 0x7fa60807b580 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa61c033000 tx=0x7fa61c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:00.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:00.817+0000 7fa625600640 1 -- 172.21.15.71:0/727204300 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa62806aca0 con 0x7fa6280b2c70 2024-09-16T07:28:01.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.042+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa6281f9220 con 0x7fa6280b2c70 2024-09-16T07:28:01.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.508+0000 7fa625600640 1 -- 172.21.15.71:0/727204300 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v38) ==== 136+0+0 (secure 0 0 0) 0x7fa614014030 con 0x7fa6280b2c70 2024-09-16T07:28:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.510+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6080790c0 msgr2=0x7fa60807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.510+0000 7fa630200640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6080790c0 0x7fa60807b580 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa61c033000 tx=0x7fa61c004730 comp rx=0 tx=0).stop 2024-09-16T07:28:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.510+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6280b2c70 msgr2=0x7fa62815a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6280b2c70 0x7fa62815a500 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa614005940 tx=0x7fa61400cd60 comp rx=0 tx=0).stop 2024-09-16T07:28:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 shutdown_connections 2024-09-16T07:28:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6080790c0 0x7fa60807b580 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62815f390 0x7fa6281f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6280b2c70 0x7fa62815a500 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 --2- 172.21.15.71:0/727204300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6280b2760 0x7fa6281f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 >> 172.21.15.71:0/727204300 conn(0x7fa6280d1db0 msgr2=0x7fa628072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 shutdown_connections 2024-09-16T07:28:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.511+0000 7fa630200640 1 -- 172.21.15.71:0/727204300 wait complete. 2024-09-16T07:28:01.529 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:28:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.688+0000 7f4d82600640 1 -- 172.21.15.71:0/1481783008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c15f5a0 msgr2=0x7f4d7c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.688+0000 7f4d82600640 1 --2- 172.21.15.71:0/1481783008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c15f5a0 0x7f4d7c0b2440 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0cfbb0 tx=0x7f4d6c02aa30 comp rx=0 tx=0).stop 2024-09-16T07:28:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.688+0000 7f4d82600640 1 -- 172.21.15.71:0/1481783008 shutdown_connections 2024-09-16T07:28:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.688+0000 7f4d82600640 1 --2- 172.21.15.71:0/1481783008 >> v2:172.21.15.177:3300/0 conn(0x7f4d7c0ccf00 0x7f4d7c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.688+0000 7f4d82600640 1 --2- 172.21.15.71:0/1481783008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c15f5a0 0x7f4d7c0b2440 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.688+0000 7f4d82600640 1 -- 172.21.15.71:0/1481783008 >> 172.21.15.71:0/1481783008 conn(0x7f4d7c0b1950 msgr2=0x7f4d7c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:01.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.688+0000 7f4d82600640 1 -- 172.21.15.71:0/1481783008 shutdown_connections 2024-09-16T07:28:01.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.688+0000 7f4d82600640 1 -- 172.21.15.71:0/1481783008 wait complete. 2024-09-16T07:28:01.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.690+0000 7f4d82600640 1 Processor -- start 2024-09-16T07:28:01.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.690+0000 7f4d82600640 1 -- start start 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d82600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c0ad270 0x7f4d7c15e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d82600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c0ccf00 0x7f4d7c14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d82600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c15f5a0 0x7f4d7c154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d82600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c06f160 con 0x7f4d7c0ad270 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d82600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c06f2e0 con 0x7f4d7c0ccf00 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d82600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7c06f460 con 0x7f4d7c15f5a0 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c0ad270 0x7f4d7c15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c0ad270 0x7f4d7c15e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51828/0 (socket says 172.21.15.71:51828) 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d7b400640 1 -- 172.21.15.71:0/3541916863 learned_addr learned my addr 172.21.15.71:0/3541916863 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d7aa00640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c0ccf00 0x7f4d7c14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d7b400640 1 -- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c15f5a0 msgr2=0x7f4d7c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.691+0000 7f4d7be00640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c15f5a0 0x7f4d7c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d7b400640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c15f5a0 0x7f4d7c154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d7b400640 1 -- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c0ccf00 msgr2=0x7f4d7c14e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d7b400640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c0ccf00 0x7f4d7c14e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d7b400640 1 -- 172.21.15.71:0/3541916863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c152fd0 con 0x7f4d7c0ad270 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d7aa00640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c0ccf00 0x7f4d7c14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d7be00640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c15f5a0 0x7f4d7c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d7b400640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c0ad270 0x7f4d7c15e590 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009bf0 tx=0x7f4d6c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d73400640 1 -- 172.21.15.71:0/3541916863 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c033050 con 0x7f4d7c0ad270 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d73400640 1 -- 172.21.15.71:0/3541916863 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d6c0044d0 con 0x7f4d7c0ad270 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c1532b0 con 0x7f4d7c0ad270 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d73400640 1 -- 172.21.15.71:0/3541916863 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c038450 con 0x7f4d7c0ad270 2024-09-16T07:28:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.692+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c153870 con 0x7f4d7c0ad270 2024-09-16T07:28:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.693+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7c06f9f0 con 0x7f4d7c0ad270 2024-09-16T07:28:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.695+0000 7f4d73400640 1 -- 172.21.15.71:0/3541916863 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f4d6c036070 con 0x7f4d7c0ad270 2024-09-16T07:28:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.695+0000 7f4d73400640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4d5c0790f0 0x7f4d5c07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.696+0000 7f4d73400640 1 -- 172.21.15.71:0/3541916863 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f4d6c0ba600 con 0x7f4d7c0ad270 2024-09-16T07:28:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.696+0000 7f4d7aa00640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4d5c0790f0 0x7f4d5c07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:01.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.697+0000 7f4d7aa00640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4d5c0790f0 0x7f4d5c07b5b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0b3b40 tx=0x7f4d68005e50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.700+0000 7f4d73400640 1 -- 172.21.15.71:0/3541916863 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4d6c084690 con 0x7f4d7c0ad270 2024-09-16T07:28:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.912+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f4d7c153c90 con 0x7f4d5c0790f0 2024-09-16T07:28:01.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.914+0000 7f4d73400640 1 -- 172.21.15.71:0/3541916863 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f4d7c153c90 con 0x7f4d5c0790f0 2024-09-16T07:28:01.914 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:28:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:28:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:28:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:28:01.914 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4d5c0790f0 msgr2=0x7f4d5c07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4d5c0790f0 0x7f4d5c07b5b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0b3b40 tx=0x7f4d68005e50 comp rx=0 tx=0).stop 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c0ad270 msgr2=0x7f4d7c15e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c0ad270 0x7f4d7c15e590 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009bf0 tx=0x7f4d6c003f10 comp rx=0 tx=0).stop 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 shutdown_connections 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4d5c0790f0 0x7f4d5c07b5b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c15f5a0 0x7f4d7c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c0ccf00 0x7f4d7c14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 --2- 172.21.15.71:0/3541916863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d7c0ad270 0x7f4d7c15e590 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 >> 172.21.15.71:0/3541916863 conn(0x7f4d7c0b1950 msgr2=0x7f4d7c0ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 shutdown_connections 2024-09-16T07:28:01.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:01.917+0000 7f4d82600640 1 -- 172.21.15.71:0/3541916863 wait complete. 2024-09-16T07:28:01.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:28:02.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.094+0000 7f498a000640 1 -- 172.21.15.71:0/1300023155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4984072560 msgr2=0x7f4984073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.094+0000 7f498a000640 1 --2- 172.21.15.71:0/1300023155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4984072560 0x7f4984073ef0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f49840c9ff0 tx=0x7f497002a540 comp rx=0 tx=0).stop 2024-09-16T07:28:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.094+0000 7f498a000640 1 -- 172.21.15.71:0/1300023155 shutdown_connections 2024-09-16T07:28:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.094+0000 7f498a000640 1 --2- 172.21.15.71:0/1300023155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4984072560 0x7f4984073ef0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.094+0000 7f498a000640 1 --2- 172.21.15.71:0/1300023155 >> v2:172.21.15.177:3300/0 conn(0x7f498415bec0 0x7f498406fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.094+0000 7f498a000640 1 -- 172.21.15.71:0/1300023155 >> 172.21.15.71:0/1300023155 conn(0x7f49840cf9f0 msgr2=0x7f49840cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.094+0000 7f498a000640 1 -- 172.21.15.71:0/1300023155 shutdown_connections 2024-09-16T07:28:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.094+0000 7f498a000640 1 -- 172.21.15.71:0/1300023155 wait complete. 2024-09-16T07:28:02.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.095+0000 7f498a000640 1 Processor -- start 2024-09-16T07:28:02.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.095+0000 7f498a000640 1 -- start start 2024-09-16T07:28:02.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.096+0000 7f498a000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4984072560 0x7f498414bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.096+0000 7f498a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49840cb4f0 0x7f49840c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.096+0000 7f498a000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498415bec0 0x7f49840c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.096+0000 7f498a000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f49840cbdf0 con 0x7f49840cb4f0 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.096+0000 7f498a000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49840cbc70 con 0x7f4984072560 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.096+0000 7f498a000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49840aebd0 con 0x7f498415bec0 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49840cb4f0 0x7f49840c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49840cb4f0 0x7f49840c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51850/0 (socket says 172.21.15.71:51850) 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 -- 172.21.15.71:0/2446356046 learned_addr learned my addr 172.21.15.71:0/2446356046 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 -- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498415bec0 msgr2=0x7f49840c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498415bec0 0x7f49840c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 -- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4984072560 msgr2=0x7f498414bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4984072560 0x7f498414bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 -- 172.21.15.71:0/2446356046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49841f3500 con 0x7f49840cb4f0 2024-09-16T07:28:02.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.097+0000 7f4983e00640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49840cb4f0 0x7f49840c9a50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4970009920 tx=0x7f4970032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:02.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.098+0000 7f4980c00640 1 -- 172.21.15.71:0/2446356046 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970040070 con 0x7f49840cb4f0 2024-09-16T07:28:02.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.098+0000 7f4980c00640 1 -- 172.21.15.71:0/2446356046 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4970005310 con 0x7f49840cb4f0 2024-09-16T07:28:02.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.098+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49841f37e0 con 0x7f49840cb4f0 2024-09-16T07:28:02.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.098+0000 7f4980c00640 1 -- 172.21.15.71:0/2446356046 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49700047a0 con 0x7f49840cb4f0 2024-09-16T07:28:02.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.098+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49841f3da0 con 0x7f49840cb4f0 2024-09-16T07:28:02.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.099+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49840af050 con 0x7f49840cb4f0 2024-09-16T07:28:02.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.100+0000 7f4980c00640 1 -- 172.21.15.71:0/2446356046 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f4970039040 con 0x7f49840cb4f0 2024-09-16T07:28:02.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.101+0000 7f4980c00640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49640790f0 0x7f496407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.101+0000 7f4980c00640 1 -- 172.21.15.71:0/2446356046 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f49700bcf60 con 0x7f49840cb4f0 2024-09-16T07:28:02.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.101+0000 7f4988c00640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49640790f0 0x7f496407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.102+0000 7f4988c00640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49640790f0 0x7f496407b5b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49780058c0 tx=0x7f4978005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:02.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.105+0000 7f4980c00640 1 -- 172.21.15.71:0/2446356046 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f49700845a0 con 0x7f49840cb4f0 2024-09-16T07:28:02.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.290+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f49840b2c90 con 0x7f49840cb4f0 2024-09-16T07:28:02.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.290+0000 7f4980c00640 1 -- 172.21.15.71:0/2446356046 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f4970083ea0 con 0x7f49840cb4f0 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49640790f0 msgr2=0x7f496407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49640790f0 0x7f496407b5b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49780058c0 tx=0x7f4978005850 comp rx=0 tx=0).stop 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49840cb4f0 msgr2=0x7f49840c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49840cb4f0 0x7f49840c9a50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4970009920 tx=0x7f4970032c10 comp rx=0 tx=0).stop 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 shutdown_connections 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49640790f0 0x7f496407b5b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498415bec0 0x7f49840c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49840cb4f0 0x7f49840c9a50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 --2- 172.21.15.71:0/2446356046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4984072560 0x7f498414bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 >> 172.21.15.71:0/2446356046 conn(0x7f49840cf9f0 msgr2=0x7f498415c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 shutdown_connections 2024-09-16T07:28:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.293+0000 7f498a000640 1 -- 172.21.15.71:0/2446356046 wait complete. 2024-09-16T07:28:02.309 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:28:02.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 -- 172.21.15.71:0/2445814881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4145750 msgr2=0x7efed4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 --2- 172.21.15.71:0/2445814881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4145750 0x7efed4073440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efed40af750 tx=0x7efec802a980 comp rx=0 tx=0).stop 2024-09-16T07:28:02.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 -- 172.21.15.71:0/2445814881 shutdown_connections 2024-09-16T07:28:02.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 --2- 172.21.15.71:0/2445814881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efed40ccb00 0x7efed415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 --2- 172.21.15.71:0/2445814881 >> v2:172.21.15.177:3300/0 conn(0x7efed406efa0 0x7efed40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 --2- 172.21.15.71:0/2445814881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4145750 0x7efed4073440 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 -- 172.21.15.71:0/2445814881 >> 172.21.15.71:0/2445814881 conn(0x7efed40b1840 msgr2=0x7efed40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:02.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 -- 172.21.15.71:0/2445814881 shutdown_connections 2024-09-16T07:28:02.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.468+0000 7efedc000640 1 -- 172.21.15.71:0/2445814881 wait complete. 2024-09-16T07:28:02.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.470+0000 7efedc000640 1 Processor -- start 2024-09-16T07:28:02.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.470+0000 7efedc000640 1 -- start start 2024-09-16T07:28:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efedc000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efed406efa0 0x7efed4141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efedc000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed40ccb00 0x7efed41f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efedc000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed4145750 0x7efed415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efedc000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efed41602c0 con 0x7efed40ccb00 2024-09-16T07:28:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efedc000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efed4160140 con 0x7efed4145750 2024-09-16T07:28:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efedc000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efed4160440 con 0x7efed406efa0 2024-09-16T07:28:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efed8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed40ccb00 0x7efed41f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efed8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed40ccb00 0x7efed41f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51876/0 (socket says 172.21.15.71:51876) 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efed8c00640 1 -- 172.21.15.71:0/1355832346 learned_addr learned my addr 172.21.15.71:0/1355832346 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.471+0000 7efed9600640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efed406efa0 0x7efed4141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efeda000640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed4145750 0x7efed415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed8c00640 1 -- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efed406efa0 msgr2=0x7efed4141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed8c00640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efed406efa0 0x7efed4141a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed8c00640 1 -- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed4145750 msgr2=0x7efed415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed8c00640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed4145750 0x7efed415ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed8c00640 1 -- 172.21.15.71:0/1355832346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed40b3560 con 0x7efed40ccb00 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efeda000640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed4145750 0x7efed415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed8c00640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed40ccb00 0x7efed41f8710 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efec000ea20 tx=0x7efec000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed1600640 1 -- 172.21.15.71:0/1355832346 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec000cd80 con 0x7efed40ccb00 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed1600640 1 -- 172.21.15.71:0/1355832346 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efec0010430 con 0x7efed40ccb00 2024-09-16T07:28:02.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.472+0000 7efed1600640 1 -- 172.21.15.71:0/1355832346 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0010840 con 0x7efed40ccb00 2024-09-16T07:28:02.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.473+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed40b3840 con 0x7efed40ccb00 2024-09-16T07:28:02.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.473+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed41f81f0 con 0x7efed40ccb00 2024-09-16T07:28:02.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.474+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed41442d0 con 0x7efed40ccb00 2024-09-16T07:28:02.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.475+0000 7efed1600640 1 -- 172.21.15.71:0/1355832346 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7efec00109e0 con 0x7efed40ccb00 2024-09-16T07:28:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.476+0000 7efed1600640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efeb40790f0 0x7efeb407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.477+0000 7efed1600640 1 -- 172.21.15.71:0/1355832346 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7efec009c6a0 con 0x7efed40ccb00 2024-09-16T07:28:02.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.477+0000 7efed9600640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efeb40790f0 0x7efeb407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.478+0000 7efed9600640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efeb40790f0 0x7efeb407b5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efec8035000 tx=0x7efec8004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:02.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.481+0000 7efed1600640 1 -- 172.21.15.71:0/1355832346 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efed41442d0 con 0x7efed40ccb00 2024-09-16T07:28:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.668+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7efed415f9f0 con 0x7efed40ccb00 2024-09-16T07:28:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.669+0000 7efed1600640 1 -- 172.21.15.71:0/1355832346 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v34) ==== 116+0+2 (secure 0 0 0) 0x7efec0063690 con 0x7efed40ccb00 2024-09-16T07:28:02.668 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:28:02.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efeb40790f0 msgr2=0x7efeb407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efeb40790f0 0x7efeb407b5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efec8035000 tx=0x7efec8004ab0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed40ccb00 msgr2=0x7efed41f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed40ccb00 0x7efed41f8710 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efec000ea20 tx=0x7efec000eee0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 shutdown_connections 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efeb40790f0 0x7efeb407b5b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed4145750 0x7efed415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed40ccb00 0x7efed41f8710 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 --2- 172.21.15.71:0/1355832346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efed406efa0 0x7efed4141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 >> 172.21.15.71:0/1355832346 conn(0x7efed40b1840 msgr2=0x7efed40cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 shutdown_connections 2024-09-16T07:28:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.671+0000 7efedc000640 1 -- 172.21.15.71:0/1355832346 wait complete. 2024-09-16T07:28:02.686 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:28:02.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 -- 172.21.15.71:0/1872486063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a415e010 msgr2=0x7f26a40d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1872486063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a415e010 0x7f26a40d03f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f26a40b69a0 tx=0x7f268c02a560 comp rx=0 tx=0).stop 2024-09-16T07:28:02.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 -- 172.21.15.71:0/1872486063 shutdown_connections 2024-09-16T07:28:02.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1872486063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26a40d4a00 0x7f26a40d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1872486063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a415e010 0x7f26a40d03f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1872486063 >> v2:172.21.15.177:3300/0 conn(0x7f26a415dc30 0x7f26a40d44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 -- 172.21.15.71:0/1872486063 >> 172.21.15.71:0/1872486063 conn(0x7f26a409d090 msgr2=0x7f26a40d3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:02.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 -- 172.21.15.71:0/1872486063 shutdown_connections 2024-09-16T07:28:02.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.847+0000 7f26a8a00640 1 -- 172.21.15.71:0/1872486063 wait complete. 2024-09-16T07:28:02.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.849+0000 7f26a8a00640 1 Processor -- start 2024-09-16T07:28:02.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.849+0000 7f26a8a00640 1 -- start start 2024-09-16T07:28:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a40d4a00 0x7f26a4157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a8a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a415dc30 0x7f26a41f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a8a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26a415e010 0x7f26a41f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f26a40cf1e0 con 0x7f26a40d4a00 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a8a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26a40cf060 con 0x7f26a415dc30 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a8a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26a40cf360 con 0x7f26a415e010 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a40d4a00 0x7f26a4157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a40d4a00 0x7f26a4157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51882/0 (socket says 172.21.15.71:51882) 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a2800640 1 -- 172.21.15.71:0/1394977531 learned_addr learned my addr 172.21.15.71:0/1394977531 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f269be00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a415dc30 0x7f26a41f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a3200640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26a415e010 0x7f26a41f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a2800640 1 -- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26a415e010 msgr2=0x7f26a41f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a2800640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26a415e010 0x7f26a41f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a2800640 1 -- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a415dc30 msgr2=0x7f26a41f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a2800640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a415dc30 0x7f26a41f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.850+0000 7f26a2800640 1 -- 172.21.15.71:0/1394977531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a41f5790 con 0x7f26a40d4a00 2024-09-16T07:28:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.851+0000 7f26a2800640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a40d4a00 0x7f26a4157310 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f269400e9f0 tx=0x7f269400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.851+0000 7f269aa00640 1 -- 172.21.15.71:0/1394977531 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269400cd80 con 0x7f26a40d4a00 2024-09-16T07:28:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.851+0000 7f269aa00640 1 -- 172.21.15.71:0/1394977531 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2694010430 con 0x7f26a40d4a00 2024-09-16T07:28:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.851+0000 7f269aa00640 1 -- 172.21.15.71:0/1394977531 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694010840 con 0x7f26a40d4a00 2024-09-16T07:28:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.851+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a41f5a70 con 0x7f26a40d4a00 2024-09-16T07:28:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.851+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a4157850 con 0x7f26a40d4a00 2024-09-16T07:28:02.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.852+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a40d44c0 con 0x7f26a40d4a00 2024-09-16T07:28:02.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.853+0000 7f269aa00640 1 -- 172.21.15.71:0/1394977531 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f26940109e0 con 0x7f26a40d4a00 2024-09-16T07:28:02.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.854+0000 7f269aa00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26800790c0 0x7f268007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:02.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.854+0000 7f269aa00640 1 -- 172.21.15.71:0/1394977531 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f269409c6a0 con 0x7f26a40d4a00 2024-09-16T07:28:02.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.855+0000 7f269be00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26800790c0 0x7f268007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:02.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.855+0000 7f269be00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26800790c0 0x7f268007b580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f268c033000 tx=0x7f268c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:02.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:02.858+0000 7f269aa00640 1 -- 172.21.15.71:0/1394977531 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2694063ee0 con 0x7f26a40d4a00 2024-09-16T07:28:03.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.043+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f26a40d05b0 con 0x7f26a40d4a00 2024-09-16T07:28:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.048+0000 7f269aa00640 1 -- 172.21.15.71:0/1394977531 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7f26940637e0 con 0x7f26a40d4a00 2024-09-16T07:28:03.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26800790c0 msgr2=0x7f268007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26800790c0 0x7f268007b580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f268c033000 tx=0x7f268c004730 comp rx=0 tx=0).stop 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a40d4a00 msgr2=0x7f26a4157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a40d4a00 0x7f26a4157310 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f269400e9f0 tx=0x7f269400eeb0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 shutdown_connections 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26800790c0 0x7f268007b580 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26a415e010 0x7f26a41f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26a415dc30 0x7f26a41f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 --2- 172.21.15.71:0/1394977531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26a40d4a00 0x7f26a4157310 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 >> 172.21.15.71:0/1394977531 conn(0x7f26a409d090 msgr2=0x7f26a40b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 shutdown_connections 2024-09-16T07:28:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.050+0000 7f26a8a00640 1 -- 172.21.15.71:0/1394977531 wait complete. 2024-09-16T07:28:03.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.066+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.066+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.066+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.066+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.066+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.066 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T07:28:03.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.229+0000 7f2689600640 1 -- 172.21.15.71:0/3928610998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2668005210 msgr2=0x7f266800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.229+0000 7f2689600640 1 --2- 172.21.15.71:0/3928610998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2668005210 0x7f266800b9c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f267c001f30 tx=0x7f2668005830 comp rx=0 tx=0).stop 2024-09-16T07:28:03.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.230+0000 7f2689600640 1 -- 172.21.15.71:0/3928610998 shutdown_connections 2024-09-16T07:28:03.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.230+0000 7f2689600640 1 --2- 172.21.15.71:0/3928610998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2668005210 0x7f266800b9c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.230+0000 7f2689600640 1 -- 172.21.15.71:0/3928610998 >> 172.21.15.71:0/3928610998 conn(0x7f26840b18e0 msgr2=0x7f26840b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:03.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.230+0000 7f2689600640 1 -- 172.21.15.71:0/3928610998 shutdown_connections 2024-09-16T07:28:03.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.230+0000 7f2689600640 1 -- 172.21.15.71:0/3928610998 wait complete. 2024-09-16T07:28:03.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.231+0000 7f2689600640 1 Processor -- start 2024-09-16T07:28:03.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.231+0000 7f2689600640 1 -- start start 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.232+0000 7f2689600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2668005210 0x7f2684161bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.232+0000 7f2689600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f268415c5a0 0x7f268415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.232+0000 7f2689600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26840c8f60 0x7f26840c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.232+0000 7f2689600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f268406f560 con 0x7f268415c5a0 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.232+0000 7f2689600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f268406f3e0 con 0x7f26840c8f60 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.232+0000 7f2689600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f268406f6e0 con 0x7f2668005210 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f268415c5a0 0x7f268415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f268415c5a0 0x7f268415ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51898/0 (socket says 172.21.15.71:51898) 2024-09-16T07:28:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 -- 172.21.15.71:0/3556072022 learned_addr learned my addr 172.21.15.71:0/3556072022 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2683400640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26840c8f60 0x7f26840c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682a00640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2668005210 0x7f2684161bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 -- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2668005210 msgr2=0x7f2684161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2668005210 0x7f2684161bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 -- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26840c8f60 msgr2=0x7f26840c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26840c8f60 0x7f26840c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 -- 172.21.15.71:0/3556072022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26840c9b40 con 0x7f268415c5a0 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2683400640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26840c8f60 0x7f26840c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682a00640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2668005210 0x7f2684161bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2682000640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f268415c5a0 0x7f268415ca20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2670011ab0 tx=0x7f2670011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f267aa00640 1 -- 172.21.15.71:0/3556072022 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267001e070 con 0x7f268415c5a0 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f267aa00640 1 -- 172.21.15.71:0/3556072022 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f267000fe70 con 0x7f268415c5a0 2024-09-16T07:28:03.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f267aa00640 1 -- 172.21.15.71:0/3556072022 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26700042b0 con 0x7f268415c5a0 2024-09-16T07:28:03.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.233+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26840c9e20 con 0x7f268415c5a0 2024-09-16T07:28:03.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.234+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26840ca3e0 con 0x7f268415c5a0 2024-09-16T07:28:03.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.235+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2684069360 con 0x7f268415c5a0 2024-09-16T07:28:03.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.236+0000 7f267aa00640 1 -- 172.21.15.71:0/3556072022 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7f2670021070 con 0x7f268415c5a0 2024-09-16T07:28:03.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.237+0000 7f267aa00640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26640790c0 0x7f266407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:03.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.237+0000 7f267aa00640 1 -- 172.21.15.71:0/3556072022 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f2670099b40 con 0x7f268415c5a0 2024-09-16T07:28:03.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.237+0000 7f2682a00640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26640790c0 0x7f266407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:03.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.238+0000 7f2682a00640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26640790c0 0x7f266407b580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2674001240 tx=0x7f2674001110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:03.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.241+0000 7f267aa00640 1 -- 172.21.15.71:0/3556072022 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2670063610 con 0x7f268415c5a0 2024-09-16T07:28:03.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.450+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f2684072a40 con 0x7f26640790c0 2024-09-16T07:28:03.460 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.462+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.461 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.462+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.461 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.462+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.461 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.462+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.461 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:28:03.462+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:28:03.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.508+0000 7f267aa00640 1 -- 172.21.15.71:0/3556072022 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2684072a40 con 0x7f26640790c0 2024-09-16T07:28:03.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26640790c0 msgr2=0x7f266407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26640790c0 0x7f266407b580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2674001240 tx=0x7f2674001110 comp rx=0 tx=0).stop 2024-09-16T07:28:03.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f268415c5a0 msgr2=0x7f268415ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f268415c5a0 0x7f268415ca20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2670011ab0 tx=0x7f2670011f70 comp rx=0 tx=0).stop 2024-09-16T07:28:03.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 shutdown_connections 2024-09-16T07:28:03.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26640790c0 0x7f266407b580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26840c8f60 0x7f26840c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f268415c5a0 0x7f268415ca20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 --2- 172.21.15.71:0/3556072022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2668005210 0x7f2684161bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 >> 172.21.15.71:0/3556072022 conn(0x7f26840b18e0 msgr2=0x7f26840cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 shutdown_connections 2024-09-16T07:28:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.510+0000 7f2689600640 1 -- 172.21.15.71:0/3556072022 wait complete. 2024-09-16T07:28:03.526 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:28:03.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.689+0000 7fb030a00640 1 -- 172.21.15.71:0/140630020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02c145750 msgr2=0x7fb02c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.689+0000 7fb030a00640 1 --2- 172.21.15.71:0/140630020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02c145750 0x7fb02c073440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb01c005fd0 tx=0x7fb01c02aa10 comp rx=0 tx=0).stop 2024-09-16T07:28:03.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.689+0000 7fb030a00640 1 -- 172.21.15.71:0/140630020 shutdown_connections 2024-09-16T07:28:03.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.689+0000 7fb030a00640 1 --2- 172.21.15.71:0/140630020 >> v2:172.21.15.177:3300/0 conn(0x7fb02c0cccd0 0x7fb02c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.689+0000 7fb030a00640 1 --2- 172.21.15.71:0/140630020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02c145750 0x7fb02c073440 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.689+0000 7fb030a00640 1 -- 172.21.15.71:0/140630020 >> 172.21.15.71:0/140630020 conn(0x7fb02c0b1840 msgr2=0x7fb02c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:03.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.689+0000 7fb030a00640 1 -- 172.21.15.71:0/140630020 shutdown_connections 2024-09-16T07:28:03.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.689+0000 7fb030a00640 1 -- 172.21.15.71:0/140630020 wait complete. 2024-09-16T07:28:03.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.690+0000 7fb030a00640 1 Processor -- start 2024-09-16T07:28:03.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.691+0000 7fb030a00640 1 -- start start 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.691+0000 7fb030a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb02c06efa0 0x7fb02c0ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.691+0000 7fb030a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb02c0cccd0 0x7fb02c13ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.691+0000 7fb030a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02c145750 0x7fb02c140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.691+0000 7fb030a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0ce480 con 0x7fb02c06efa0 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb030a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0ce300 con 0x7fb02c0cccd0 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb030a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c0ce600 con 0x7fb02c145750 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb02c06efa0 0x7fb02c0ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb02c06efa0 0x7fb02c0ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51902/0 (socket says 172.21.15.71:51902) 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 -- 172.21.15.71:0/4133789484 learned_addr learned my addr 172.21.15.71:0/4133789484 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:28:03.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb02a800640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02c145750 0x7fb02c140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:03.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029400640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb02c0cccd0 0x7fb02c13ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:03.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 -- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02c145750 msgr2=0x7fb02c140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02c145750 0x7fb02c140510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 -- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb02c0cccd0 msgr2=0x7fb02c13ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb02c0cccd0 0x7fb02c13ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 -- 172.21.15.71:0/4133789484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c1416d0 con 0x7fb02c06efa0 2024-09-16T07:28:03.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.692+0000 7fb029e00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb02c06efa0 0x7fb02c0ae3d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb01c000f80 tx=0x7fb01c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:03.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.693+0000 7fb022000640 1 -- 172.21.15.71:0/4133789484 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c046070 con 0x7fb02c06efa0 2024-09-16T07:28:03.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.693+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c140b50 con 0x7fb02c06efa0 2024-09-16T07:28:03.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.693+0000 7fb022000640 1 -- 172.21.15.71:0/4133789484 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb01c004540 con 0x7fb02c06efa0 2024-09-16T07:28:03.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.693+0000 7fb022000640 1 -- 172.21.15.71:0/4133789484 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c033070 con 0x7fb02c06efa0 2024-09-16T07:28:03.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.693+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c141110 con 0x7fb02c06efa0 2024-09-16T07:28:03.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.694+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02c0cdbb0 con 0x7fb02c06efa0 2024-09-16T07:28:03.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.695+0000 7fb022000640 1 -- 172.21.15.71:0/4133789484 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 102052+0+0 (secure 0 0 0) 0x7fb01c036070 con 0x7fb02c06efa0 2024-09-16T07:28:03.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.697+0000 7fb022000640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00c0790c0 0x7fb00c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:28:03.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.697+0000 7fb022000640 1 -- 172.21.15.71:0/4133789484 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fb01c0bc8a0 con 0x7fb02c06efa0 2024-09-16T07:28:03.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.697+0000 7fb029400640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00c0790c0 0x7fb00c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:28:03.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.698+0000 7fb029400640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00c0790c0 0x7fb00c07b580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb02c151600 tx=0x7fb018009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:28:03.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.701+0000 7fb022000640 1 -- 172.21.15.71:0/4133789484 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb01c03e070 con 0x7fb02c06efa0 2024-09-16T07:28:03.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.910+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb02c0cf020 con 0x7fb00c0790c0 2024-09-16T07:28:03.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.917+0000 7fb022000640 1 -- 172.21.15.71:0/4133789484 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb02c0cf020 con 0x7fb00c0790c0 2024-09-16T07:28:03.916 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279 2024-09-16T07:28:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00c0790c0 msgr2=0x7fb00c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00c0790c0 0x7fb00c07b580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb02c151600 tx=0x7fb018009040 comp rx=0 tx=0).stop 2024-09-16T07:28:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb02c06efa0 msgr2=0x7fb02c0ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:28:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb02c06efa0 0x7fb02c0ae3d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb01c000f80 tx=0x7fb01c003f90 comp rx=0 tx=0).stop 2024-09-16T07:28:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 shutdown_connections 2024-09-16T07:28:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00c0790c0 0x7fb00c07b580 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02c145750 0x7fb02c140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb02c0cccd0 0x7fb02c13ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 --2- 172.21.15.71:0/4133789484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb02c06efa0 0x7fb02c0ae3d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:28:03.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 >> 172.21.15.71:0/4133789484 conn(0x7fb02c0b1840 msgr2=0x7fb02c06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:28:03.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 shutdown_connections 2024-09-16T07:28:03.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:28:03.919+0000 7fb030a00640 1 -- 172.21.15.71:0/4133789484 wait complete. 2024-09-16T07:28:03.934 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279 2024-09-16T07:28:03.934 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:03.934 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:03.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:03.970 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:28:11.409 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:28:11.409 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:28:11.410 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.35617 s, 146 MB/s 2024-09-16T07:28:11.411 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:11.411 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:11.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:11.447 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:28:14.579 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:28:14.583 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:28:18.621 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:28:18.621 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:28:18.621 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.09756 s, 151 MB/s 2024-09-16T07:28:18.622 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:18.622 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:18.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:18.658 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:28:26.025 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:28:26.025 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:28:26.025 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.28981 s, 147 MB/s 2024-09-16T07:28:26.027 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:26.027 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:26.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:26.063 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:28:33.647 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:28:33.647 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:28:33.647 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.50731 s, 143 MB/s 2024-09-16T07:28:33.648 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:33.648 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:33.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:33.684 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:28:41.015 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:28:41.016 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:28:41.016 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.25524 s, 148 MB/s 2024-09-16T07:28:41.017 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:41.017 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:41.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:41.053 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:28:44.675 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:28:44.680 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:28:48.278 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:28:48.278 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:28:48.278 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.14893 s, 150 MB/s 2024-09-16T07:28:48.279 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:48.279 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:48.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:48.315 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:28:55.721 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:28:55.722 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:28:55.722 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.3304 s, 146 MB/s 2024-09-16T07:28:55.723 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:28:55.723 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:28:55.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:28:55.759 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:29:03.273 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:29:03.273 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:29:03.274 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.43689 s, 144 MB/s 2024-09-16T07:29:03.275 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:03.275 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:03.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:03.311 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:29:10.818 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:29:10.818 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:29:10.818 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.42989 s, 145 MB/s 2024-09-16T07:29:10.819 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:10.820 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:10.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:10.855 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:29:14.780 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:29:14.784 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:29:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:29:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:29:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.51178 s, 143 MB/s 2024-09-16T07:29:18.445 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T07:29:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.609+0000 7fbefae00640 1 -- 172.21.15.71:0/2266023660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef415cff0 msgr2=0x7fbef406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.609+0000 7fbefae00640 1 --2- 172.21.15.71:0/2266023660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef415cff0 0x7fbef406e6f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009830 tx=0x7fbee002a5b0 comp rx=0 tx=0).stop 2024-09-16T07:29:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.610+0000 7fbefae00640 1 -- 172.21.15.71:0/2266023660 shutdown_connections 2024-09-16T07:29:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.610+0000 7fbefae00640 1 --2- 172.21.15.71:0/2266023660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef415cff0 0x7fbef406e6f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.610+0000 7fbefae00640 1 -- 172.21.15.71:0/2266023660 >> 172.21.15.71:0/2266023660 conn(0x7fbef40b18e0 msgr2=0x7fbef40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.610+0000 7fbefae00640 1 -- 172.21.15.71:0/2266023660 shutdown_connections 2024-09-16T07:29:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.610+0000 7fbefae00640 1 -- 172.21.15.71:0/2266023660 wait complete. 2024-09-16T07:29:18.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.611+0000 7fbefae00640 1 Processor -- start 2024-09-16T07:29:18.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.611+0000 7fbefae00640 1 -- start start 2024-09-16T07:29:18.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.612+0000 7fbefae00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbef406f220 0x7fbef4148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:18.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.612+0000 7fbefae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef4159aa0 0x7fbef4154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:18.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.612+0000 7fbefae00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef415cff0 0x7fbef41546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:18.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.612+0000 7fbefae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbef40ccc20 con 0x7fbef4159aa0 2024-09-16T07:29:18.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.612+0000 7fbefae00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbef406f6b0 con 0x7fbef415cff0 2024-09-16T07:29:18.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.612+0000 7fbefae00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbef40cdf60 con 0x7fbef406f220 2024-09-16T07:29:18.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef4159aa0 0x7fbef4154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef4159aa0 0x7fbef4154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57938/0 (socket says 172.21.15.71:57938) 2024-09-16T07:29:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 -- 172.21.15.71:0/2538833545 learned_addr learned my addr 172.21.15.71:0/2538833545 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3e00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbef406f220 0x7fbef4148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef8e00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef415cff0 0x7fbef41546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 -- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbef406f220 msgr2=0x7fbef4148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbef406f220 0x7fbef4148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 -- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef415cff0 msgr2=0x7fbef41546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef415cff0 0x7fbef41546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 -- 172.21.15.71:0/2538833545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef40ce240 con 0x7fbef4159aa0 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3e00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbef406f220 0x7fbef4148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef8e00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef415cff0 0x7fbef41546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.613+0000 7fbef3400640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef4159aa0 0x7fbef4154180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbee0000c00 tx=0x7fbee0033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.614+0000 7fbeebe00640 1 -- 172.21.15.71:0/2538833545 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0031d60 con 0x7fbef4159aa0 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.614+0000 7fbeebe00640 1 -- 172.21.15.71:0/2538833545 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbee00330d0 con 0x7fbef4159aa0 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.614+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef40ce520 con 0x7fbef4159aa0 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.614+0000 7fbeebe00640 1 -- 172.21.15.71:0/2538833545 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0033520 con 0x7fbef4159aa0 2024-09-16T07:29:18.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.614+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef40ceae0 con 0x7fbef4159aa0 2024-09-16T07:29:18.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.615+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef4069360 con 0x7fbef4159aa0 2024-09-16T07:29:18.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.616+0000 7fbeebe00640 1 -- 172.21.15.71:0/2538833545 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 21) ==== 102154+0+0 (secure 0 0 0) 0x7fbee00035f0 con 0x7fbef4159aa0 2024-09-16T07:29:18.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.617+0000 7fbeebe00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbed40791e0 0x7fbed407b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:18.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.617+0000 7fbeebe00640 1 -- 172.21.15.71:0/2538833545 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7fbee00ba4f0 con 0x7fbef4159aa0 2024-09-16T07:29:18.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.617+0000 7fbef3e00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbed40791e0 0x7fbed407b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:18.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.618+0000 7fbef3e00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbed40791e0 0x7fbed407b6a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbee40046b0 tx=0x7fbee4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:18.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.621+0000 7fbeebe00640 1 -- 172.21.15.71:0/2538833545 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbee003f030 con 0x7fbef4159aa0 2024-09-16T07:29:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.839+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fbef4070980 con 0x7fbed40791e0 2024-09-16T07:29:18.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.853+0000 7fbeebe00640 1 -- 172.21.15.71:0/2538833545 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbef4070980 con 0x7fbed40791e0 2024-09-16T07:29:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.855+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbed40791e0 msgr2=0x7fbed407b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.855+0000 7fbefae00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbed40791e0 0x7fbed407b6a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbee40046b0 tx=0x7fbee4009040 comp rx=0 tx=0).stop 2024-09-16T07:29:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.855+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef4159aa0 msgr2=0x7fbef4154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.855+0000 7fbefae00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef4159aa0 0x7fbef4154180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbee0000c00 tx=0x7fbee0033d60 comp rx=0 tx=0).stop 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.856+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 shutdown_connections 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.856+0000 7fbefae00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbed40791e0 0x7fbed407b6a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.856+0000 7fbefae00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef415cff0 0x7fbef41546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.856+0000 7fbefae00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef4159aa0 0x7fbef4154180 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.856+0000 7fbefae00640 1 --2- 172.21.15.71:0/2538833545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbef406f220 0x7fbef4148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.856+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 >> 172.21.15.71:0/2538833545 conn(0x7fbef40b18e0 msgr2=0x7fbef415b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.856+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 shutdown_connections 2024-09-16T07:29:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:18.856+0000 7fbefae00640 1 -- 172.21.15.71:0/2538833545 wait complete. 2024-09-16T07:29:18.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:29:19.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 -- 172.21.15.71:0/1069633098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6578145750 msgr2=0x7f6578073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 --2- 172.21.15.71:0/1069633098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6578145750 0x7f6578073440 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f65780cc550 tx=0x7f656802a9a0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 -- 172.21.15.71:0/1069633098 shutdown_connections 2024-09-16T07:29:19.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 --2- 172.21.15.71:0/1069633098 >> v2:172.21.15.177:3300/0 conn(0x7f65780ccb00 0x7f657815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 --2- 172.21.15.71:0/1069633098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f657806efa0 0x7f65780cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 --2- 172.21.15.71:0/1069633098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6578145750 0x7f6578073440 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 -- 172.21.15.71:0/1069633098 >> 172.21.15.71:0/1069633098 conn(0x7f65780b1840 msgr2=0x7f65780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:19.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 -- 172.21.15.71:0/1069633098 shutdown_connections 2024-09-16T07:29:19.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.032+0000 7f657d400640 1 -- 172.21.15.71:0/1069633098 wait complete. 2024-09-16T07:29:19.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.033+0000 7f657d400640 1 Processor -- start 2024-09-16T07:29:19.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.034+0000 7f657d400640 1 -- start start 2024-09-16T07:29:19.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.034+0000 7f657d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f657806efa0 0x7f65781f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.034+0000 7f657d400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65780ccb00 0x7f657815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f657d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6578145750 0x7f6578142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f657d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f65781600b0 con 0x7f657806efa0 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f657d400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f657815ff30 con 0x7f6578145750 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f657d400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6578160230 con 0x7f65780ccb00 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f657806efa0 0x7f65781f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f657806efa0 0x7f65781f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57950/0 (socket says 172.21.15.71:57950) 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 -- 172.21.15.71:0/1861299125 learned_addr learned my addr 172.21.15.71:0/1861299125 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 -- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65780ccb00 msgr2=0x7f657815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65780ccb00 0x7f657815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 -- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6578145750 msgr2=0x7f6578142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6578145750 0x7f6578142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 -- 172.21.15.71:0/1861299125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65781f5680 con 0x7f657806efa0 2024-09-16T07:29:19.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.035+0000 7f6576800640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f657806efa0 0x7f65781f44d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f656802af90 tx=0x7f6568003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.036+0000 7f656ea00640 1 -- 172.21.15.71:0/1861299125 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568033050 con 0x7f657806efa0 2024-09-16T07:29:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.036+0000 7f656ea00640 1 -- 172.21.15.71:0/1861299125 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65680044f0 con 0x7f657806efa0 2024-09-16T07:29:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.036+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65781f4d80 con 0x7f657806efa0 2024-09-16T07:29:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.036+0000 7f656ea00640 1 -- 172.21.15.71:0/1861299125 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568038450 con 0x7f657806efa0 2024-09-16T07:29:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.036+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65781f5920 con 0x7f657806efa0 2024-09-16T07:29:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.037+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6578160710 con 0x7f657806efa0 2024-09-16T07:29:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.038+0000 7f656ea00640 1 -- 172.21.15.71:0/1861299125 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 21) ==== 102154+0+0 (secure 0 0 0) 0x7f6568036070 con 0x7f657806efa0 2024-09-16T07:29:19.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.039+0000 7f656ea00640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65580791e0 0x7f655807b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:19.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.039+0000 7f656ea00640 1 -- 172.21.15.71:0/1861299125 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7f65680ba280 con 0x7f657806efa0 2024-09-16T07:29:19.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.039+0000 7f6575e00640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65580791e0 0x7f655807b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:19.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.040+0000 7f6575e00640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65580791e0 0x7f655807b6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6578143100 tx=0x7f6564006c40 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:19.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.043+0000 7f656ea00640 1 -- 172.21.15.71:0/1861299125 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f65680844f0 con 0x7f657806efa0 2024-09-16T07:29:19.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.258+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f65781f5b20 con 0x7f65580791e0 2024-09-16T07:29:19.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.263+0000 7f656ea00640 1 -- 172.21.15.71:0/1861299125 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f65781f5b20 con 0x7f65580791e0 2024-09-16T07:29:19.262 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279 2024-09-16T07:29:19.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65580791e0 msgr2=0x7f655807b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65580791e0 0x7f655807b6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6578143100 tx=0x7f6564006c40 comp rx=0 tx=0).stop 2024-09-16T07:29:19.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f657806efa0 msgr2=0x7f65781f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f657806efa0 0x7f65781f44d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f656802af90 tx=0x7f6568003f90 comp rx=0 tx=0).stop 2024-09-16T07:29:19.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 shutdown_connections 2024-09-16T07:29:19.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65580791e0 0x7f655807b6a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6578145750 0x7f6578142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65780ccb00 0x7f657815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 --2- 172.21.15.71:0/1861299125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f657806efa0 0x7f65781f44d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 >> 172.21.15.71:0/1861299125 conn(0x7f65780b1840 msgr2=0x7f65780cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:19.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 shutdown_connections 2024-09-16T07:29:19.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.265+0000 7f657d400640 1 -- 172.21.15.71:0/1861299125 wait complete. 2024-09-16T07:29:19.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:19.280 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279 2024-09-16T07:29:19.280 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:29:19.315 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/17918c25-d3bb-49eb-ba36-c85b96ebf279 2024-09-16T07:29:19.316 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="10737418240" 2024-09-16T07:29:19.316 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:19.317 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-16T07:29:19.317 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-16T07:29:19.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.522+0000 7f1140200640 1 -- 172.21.15.71:0/3641351253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 msgr2=0x7f11380cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.522+0000 7f1140200640 1 --2- 172.21.15.71:0/3641351253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 0x7f11380cdbb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1130005fd0 tx=0x7f113002aa10 comp rx=0 tx=0).stop 2024-09-16T07:29:19.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.522+0000 7f1140200640 1 -- 172.21.15.71:0/3641351253 shutdown_connections 2024-09-16T07:29:19.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.522+0000 7f1140200640 1 --2- 172.21.15.71:0/3641351253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 0x7f11380cdbb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.522+0000 7f1140200640 1 --2- 172.21.15.71:0/3641351253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1138145750 0x7f1138073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.522+0000 7f1140200640 1 -- 172.21.15.71:0/3641351253 >> 172.21.15.71:0/3641351253 conn(0x7f11380b1840 msgr2=0x7f11380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.522+0000 7f1140200640 1 -- 172.21.15.71:0/3641351253 shutdown_connections 2024-09-16T07:29:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.522+0000 7f1140200640 1 -- 172.21.15.71:0/3641351253 wait complete. 2024-09-16T07:29:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.524+0000 7f1140200640 1 Processor -- start 2024-09-16T07:29:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.524+0000 7f1140200640 1 -- start start 2024-09-16T07:29:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f1140200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f113806efa0 0x7f113815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f1140200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 0x7f11380b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f1140200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1138145750 0x7f113812abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f1140200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11380ce480 con 0x7f11380cccd0 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f1140200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11380ce300 con 0x7f1138145750 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f1140200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11380ce600 con 0x7f113806efa0 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f113ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 0x7f11380b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f113ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 0x7f11380b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57970/0 (socket says 172.21.15.71:57970) 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f113ce00640 1 -- 172.21.15.71:0/2512294829 learned_addr learned my addr 172.21.15.71:0/2512294829 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f113d800640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f113806efa0 0x7f113815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.525+0000 7f113e200640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1138145750 0x7f113812abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f113ce00640 1 -- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f113806efa0 msgr2=0x7f113815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f113ce00640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f113806efa0 0x7f113815eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f113ce00640 1 -- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1138145750 msgr2=0x7f113812abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f113ce00640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1138145750 0x7f113812abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f113ce00640 1 -- 172.21.15.71:0/2512294829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11380b4830 con 0x7f11380cccd0 2024-09-16T07:29:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f113ce00640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 0x7f11380b3930 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f112400c450 tx=0x7f112400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f1135600640 1 -- 172.21.15.71:0/2512294829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240052d0 con 0x7f11380cccd0 2024-09-16T07:29:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f1135600640 1 -- 172.21.15.71:0/2512294829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1124005470 con 0x7f11380cccd0 2024-09-16T07:29:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11380b4b10 con 0x7f11380cccd0 2024-09-16T07:29:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f1135600640 1 -- 172.21.15.71:0/2512294829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124012870 con 0x7f11380cccd0 2024-09-16T07:29:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.526+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11380adff0 con 0x7f11380cccd0 2024-09-16T07:29:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.527+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11380cf910 con 0x7f11380cccd0 2024-09-16T07:29:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.528+0000 7f1135600640 1 -- 172.21.15.71:0/2512294829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 21) ==== 102154+0+0 (secure 0 0 0) 0x7f11240035f0 con 0x7f11380cccd0 2024-09-16T07:29:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.529+0000 7f1135600640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11180791e0 0x7f111807b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.529+0000 7f1135600640 1 -- 172.21.15.71:0/2512294829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7f1124098a90 con 0x7f11380cccd0 2024-09-16T07:29:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.530+0000 7f113d800640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11180791e0 0x7f111807b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.531+0000 7f113d800640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11180791e0 0x7f111807b6a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f11380b3620 tx=0x7f112c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:19.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.534+0000 7f1135600640 1 -- 172.21.15.71:0/2512294829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f11240626b0 con 0x7f11380cccd0 2024-09-16T07:29:19.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.754+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f11380cf630 con 0x7f11180791e0 2024-09-16T07:29:19.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.828+0000 7f1135600640 1 -- 172.21.15.71:0/2512294829 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f11380cf630 con 0x7f11180791e0 2024-09-16T07:29:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11180791e0 msgr2=0x7f111807b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11180791e0 0x7f111807b6a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f11380b3620 tx=0x7f112c005f90 comp rx=0 tx=0).stop 2024-09-16T07:29:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 msgr2=0x7f11380b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 0x7f11380b3930 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f112400c450 tx=0x7f112400c910 comp rx=0 tx=0).stop 2024-09-16T07:29:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 shutdown_connections 2024-09-16T07:29:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11180791e0 0x7f111807b6a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1138145750 0x7f113812abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11380cccd0 0x7f11380b3930 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 --2- 172.21.15.71:0/2512294829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f113806efa0 0x7f113815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.830+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 >> 172.21.15.71:0/2512294829 conn(0x7f11380b1840 msgr2=0x7f1138144160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.831+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 shutdown_connections 2024-09-16T07:29:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:19.831+0000 7f1140200640 1 -- 172.21.15.71:0/2512294829 wait complete. 2024-09-16T07:29:19.831 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.831+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.831 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.831+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.831 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.831+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.832 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.831+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.832 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.831+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.847 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:29:19.848 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.848+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.848 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.848+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.848 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.848+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.848 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.848+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:19.848 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:19.848+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:20.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.007+0000 7f5656200640 1 -- 172.21.15.71:0/3172655088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 msgr2=0x7f565006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:20.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.007+0000 7f5656200640 1 --2- 172.21.15.71:0/3172655088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 0x7f565006e6f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f56500b2680 tx=0x7f563c02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.007+0000 7f5656200640 1 -- 172.21.15.71:0/3172655088 shutdown_connections 2024-09-16T07:29:20.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.007+0000 7f5656200640 1 --2- 172.21.15.71:0/3172655088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 0x7f565006e6f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.007+0000 7f5656200640 1 --2- 172.21.15.71:0/3172655088 >> v2:172.21.15.177:3300/0 conn(0x7f5650159aa0 0x7f56500741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.007+0000 7f5656200640 1 -- 172.21.15.71:0/3172655088 >> 172.21.15.71:0/3172655088 conn(0x7f56500b18e0 msgr2=0x7f56500b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:20.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.007+0000 7f5656200640 1 -- 172.21.15.71:0/3172655088 shutdown_connections 2024-09-16T07:29:20.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.008+0000 7f5656200640 1 -- 172.21.15.71:0/3172655088 wait complete. 2024-09-16T07:29:20.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.009+0000 7f5656200640 1 Processor -- start 2024-09-16T07:29:20.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.009+0000 7f5656200640 1 -- start start 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f5656200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f565006f220 0x7f56500d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f5656200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5650159aa0 0x7f565014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f5656200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 0x7f56500ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f5656200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f56500ccc30 con 0x7f565015cff0 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f5656200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56500ccab0 con 0x7f565006f220 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f5656200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f56500ccdb0 con 0x7f5650159aa0 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f564fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 0x7f56500ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f564fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 0x7f56500ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57998/0 (socket says 172.21.15.71:57998) 2024-09-16T07:29:20.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.010+0000 7f564fe00640 1 -- 172.21.15.71:0/3683225519 learned_addr learned my addr 172.21.15.71:0/3683225519 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f564f400640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f565006f220 0x7f56500d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f564ea00640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5650159aa0 0x7f565014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f564fe00640 1 -- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5650159aa0 msgr2=0x7f565014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f564fe00640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5650159aa0 0x7f565014d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f564fe00640 1 -- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f565006f220 msgr2=0x7f56500d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f564fe00640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f565006f220 0x7f56500d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f564fe00640 1 -- 172.21.15.71:0/3683225519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56500b46e0 con 0x7f565015cff0 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f564fe00640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 0x7f56500ce2c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f564800c400 tx=0x7f564800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f5647400640 1 -- 172.21.15.71:0/3683225519 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648017070 con 0x7f565015cff0 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f5647400640 1 -- 172.21.15.71:0/3683225519 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56480053c0 con 0x7f565015cff0 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56500b3bc0 con 0x7f565015cff0 2024-09-16T07:29:20.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.011+0000 7f5647400640 1 -- 172.21.15.71:0/3683225519 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648003850 con 0x7f565015cff0 2024-09-16T07:29:20.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.012+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56500b4400 con 0x7f565015cff0 2024-09-16T07:29:20.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.013+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5650069360 con 0x7f565015cff0 2024-09-16T07:29:20.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.014+0000 7f5647400640 1 -- 172.21.15.71:0/3683225519 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 21) ==== 102154+0+0 (secure 0 0 0) 0x7f564801a070 con 0x7f565015cff0 2024-09-16T07:29:20.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.015+0000 7f5647400640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56300791e0 0x7f563007b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:20.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.015+0000 7f5647400640 1 -- 172.21.15.71:0/3683225519 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7f56480983f0 con 0x7f565015cff0 2024-09-16T07:29:20.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.015+0000 7f564f400640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56300791e0 0x7f563007b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:20.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.016+0000 7f564f400640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56300791e0 0x7f563007b6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5640005fd0 tx=0x7f5640009680 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:20.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.019+0000 7f5647400640 1 -- 172.21.15.71:0/3683225519 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f56500b3bc0 con 0x7f565015cff0 2024-09-16T07:29:20.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.275+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5650156ef0 con 0x7f565015cff0 2024-09-16T07:29:20.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.276+0000 7f5647400640 1 -- 172.21.15.71:0/3683225519 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1882 (secure 0 0 0) 0x7f5648062010 con 0x7f565015cff0 2024-09-16T07:29:20.275 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:20.275 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:29:20.276 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 417, 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:29:20.277 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 64, 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:29:20.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 2404, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 9979375449, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 20271788032, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 752822325248, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 131742809, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 0, 2024-09-16T07:29:20.279 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 34 2024-09-16T07:29:20.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:20.280 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:29:20.280 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 38, 2024-09-16T07:29:20.280 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:28:01:502092+0000", 2024-09-16T07:29:20.280 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4, 2024-09-16T07:29:20.281 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:29:20.281 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:29:20.281 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:29:20.281 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:29:20.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:20.281 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 4, 2024-09-16T07:29:20.282 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:29:20.282 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "c", 2024-09-16T07:29:20.282 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:29:20.282 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 5964 2024-09-16T07:29:20.282 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:20.283 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:20.283 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:29:20.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:20.283 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:29:20.283 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:29:20.283 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:29:20.284 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:29:20.284 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:29:20.284 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:29:20.284 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:29:20.284 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:20.285 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:29:20.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:20.285 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:29:20.285 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 12, 2024-09-16T07:29:20.285 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:28:56.398876+0000", 2024-09-16T07:29:20.285 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:29:20.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:20.286 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": {} 2024-09-16T07:29:20.286 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:29:20.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56300791e0 msgr2=0x7f563007b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:20.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56300791e0 0x7f563007b6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5640005fd0 tx=0x7f5640009680 comp rx=0 tx=0).stop 2024-09-16T07:29:20.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 msgr2=0x7f56500ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:20.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 0x7f56500ce2c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f564800c400 tx=0x7f564800c8c0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 shutdown_connections 2024-09-16T07:29:20.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56300791e0 0x7f563007b6a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565015cff0 0x7f56500ce2c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5650159aa0 0x7f565014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 --2- 172.21.15.71:0/3683225519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f565006f220 0x7f56500d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:20.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 >> 172.21.15.71:0/3683225519 conn(0x7f56500b18e0 msgr2=0x7f565015eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:20.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 shutdown_connections 2024-09-16T07:29:20.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:20.278+0000 7f5656200640 1 -- 172.21.15.71:0/3683225519 wait complete. 2024-09-16T07:29:20.841 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:20.841+0000 7fe734600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:20.841 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:20.841+0000 7fe734600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:20.841 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:20.841+0000 7fe734600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:20.841 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:20.841+0000 7fe734600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:20.841 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:20.841+0000 7fe734600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:21.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:29:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.497+0000 7f1f45400640 1 -- 172.21.15.71:0/3622214619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400af280 msgr2=0x7f1f400cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.497+0000 7f1f45400640 1 --2- 172.21.15.71:0/3622214619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400af280 0x7f1f400cca60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1f400af820 tx=0x7f1f2802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.497+0000 7f1f45400640 1 -- 172.21.15.71:0/3622214619 shutdown_connections 2024-09-16T07:29:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.497+0000 7f1f45400640 1 --2- 172.21.15.71:0/3622214619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400af280 0x7f1f400cca60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.497+0000 7f1f45400640 1 --2- 172.21.15.71:0/3622214619 >> v2:172.21.15.177:3300/0 conn(0x7f1f400cc680 0x7f1f400b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.497+0000 7f1f45400640 1 -- 172.21.15.71:0/3622214619 >> 172.21.15.71:0/3622214619 conn(0x7f1f400b18e0 msgr2=0x7f1f400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.497+0000 7f1f45400640 1 -- 172.21.15.71:0/3622214619 shutdown_connections 2024-09-16T07:29:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.497+0000 7f1f45400640 1 -- 172.21.15.71:0/3622214619 wait complete. 2024-09-16T07:29:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.499+0000 7f1f45400640 1 Processor -- start 2024-09-16T07:29:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.499+0000 7f1f45400640 1 -- start start 2024-09-16T07:29:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f45400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400acd30 0x7f1f40160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f45400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f400af280 0x7f1f401f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f45400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f400cc680 0x7f1f40151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f45400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1f400ad1c0 con 0x7f1f400acd30 2024-09-16T07:29:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f45400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4015e140 con 0x7f1f400cc680 2024-09-16T07:29:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f45400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4006fa60 con 0x7f1f400af280 2024-09-16T07:29:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f3fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400acd30 0x7f1f40160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f3fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400acd30 0x7f1f40160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58012/0 (socket says 172.21.15.71:58012) 2024-09-16T07:29:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f3fe00640 1 -- 172.21.15.71:0/4256824895 learned_addr learned my addr 172.21.15.71:0/4256824895 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f44a00640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f400cc680 0x7f1f40151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f3fe00640 1 -- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f400af280 msgr2=0x7f1f401f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f3fe00640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f400af280 0x7f1f401f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.500+0000 7f1f3fe00640 1 -- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f400cc680 msgr2=0x7f1f40151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f3fe00640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f400cc680 0x7f1f40151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f3fe00640 1 -- 172.21.15.71:0/4256824895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4015cb00 con 0x7f1f400acd30 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f44a00640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f400cc680 0x7f1f40151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f3fe00640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400acd30 0x7f1f40160b30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000b440 tx=0x7f1f3000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f3cc00640 1 -- 172.21.15.71:0/4256824895 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30015020 con 0x7f1f400acd30 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f3cc00640 1 -- 172.21.15.71:0/4256824895 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f30011980 con 0x7f1f400acd30 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f3cc00640 1 -- 172.21.15.71:0/4256824895 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30011dd0 con 0x7f1f400acd30 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4015cde0 con 0x7f1f400acd30 2024-09-16T07:29:21.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.501+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f401f8430 con 0x7f1f400acd30 2024-09-16T07:29:21.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.503+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f400cca60 con 0x7f1f400acd30 2024-09-16T07:29:21.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.505+0000 7f1f3cc00640 1 -- 172.21.15.71:0/4256824895 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 22) ==== 102256+0+0 (secure 0 0 0) 0x7f1f30018020 con 0x7f1f400acd30 2024-09-16T07:29:21.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.506+0000 7f1f3cc00640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1f1c079300 0x7f1f1c07b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:21.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.506+0000 7f1f3cc00640 1 -- 172.21.15.71:0/4256824895 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7f1f300993f0 con 0x7f1f400acd30 2024-09-16T07:29:21.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.507+0000 7f1f37e00640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1f1c079300 0x7f1f1c07b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:21.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.510+0000 7f1f37e00640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1f1c079300 0x7f1f1c07b7c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f28033000 tx=0x7f1f280046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:21.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.511+0000 7f1f3cc00640 1 -- 172.21.15.71:0/4256824895 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1f3001f020 con 0x7f1f400acd30 2024-09-16T07:29:21.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.897+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1f400cd850 con 0x7f1f400acd30 2024-09-16T07:29:21.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.898+0000 7f1f3cc00640 1 -- 172.21.15.71:0/4256824895 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1883 (secure 0 0 0) 0x7f1f3000d040 con 0x7f1f400acd30 2024-09-16T07:29:21.897 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:21.897 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:29:21.897 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:29:21.898 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 418, 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:21.899 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 64, 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:29:21.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:21.901 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:29:21.901 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:29:21.901 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:21.901 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:21.901 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:29:21.901 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:29:21.901 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 2525, 2024-09-16T07:29:21.901 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 10486899515, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 21060411392, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 752033701888, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 157949101, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 0, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 40 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 38, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:28:01:502092+0000", 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4, 2024-09-16T07:29:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 4, 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "c", 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 5964 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:29:21.903 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 12, 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:28:56.398876+0000", 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:21.904 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": {} 2024-09-16T07:29:21.905 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:29:21.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1f1c079300 msgr2=0x7f1f1c07b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1f1c079300 0x7f1f1c07b7c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f28033000 tx=0x7f1f280046c0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400acd30 msgr2=0x7f1f40160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400acd30 0x7f1f40160b30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000b440 tx=0x7f1f3000b900 comp rx=0 tx=0).stop 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 shutdown_connections 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1f1c079300 0x7f1f1c07b7c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f400cc680 0x7f1f40151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f400af280 0x7f1f401f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 --2- 172.21.15.71:0/4256824895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f400acd30 0x7f1f40160b30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 >> 172.21.15.71:0/4256824895 conn(0x7f1f400b18e0 msgr2=0x7f1f400ad340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.901+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 shutdown_connections 2024-09-16T07:29:21.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:21.902+0000 7f1f45400640 1 -- 172.21.15.71:0/4256824895 wait complete. 2024-09-16T07:29:22.920 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:29:23.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.126+0000 7fc0b8a00640 1 -- 172.21.15.71:0/2958907821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d25d0 msgr2=0x7fc0b40d4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:23.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.126+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/2958907821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d25d0 0x7fc0b40d4960 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac005fd0 tx=0x7fc0ac02a900 comp rx=0 tx=0).stop 2024-09-16T07:29:23.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.126+0000 7fc0b8a00640 1 -- 172.21.15.71:0/2958907821 shutdown_connections 2024-09-16T07:29:23.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.126+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/2958907821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d25d0 0x7fc0b40d4960 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.126+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/2958907821 >> v2:172.21.15.177:3300/0 conn(0x7fc0b415e050 0x7fc0b40cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.126+0000 7fc0b8a00640 1 -- 172.21.15.71:0/2958907821 >> 172.21.15.71:0/2958907821 conn(0x7fc0b40d3e40 msgr2=0x7fc0b40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:23.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.126+0000 7fc0b8a00640 1 -- 172.21.15.71:0/2958907821 shutdown_connections 2024-09-16T07:29:23.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.126+0000 7fc0b8a00640 1 -- 172.21.15.71:0/2958907821 wait complete. 2024-09-16T07:29:23.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.129+0000 7fc0b8a00640 1 Processor -- start 2024-09-16T07:29:23.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.129+0000 7fc0b8a00640 1 -- start start 2024-09-16T07:29:23.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d1260 0x7fc0b409e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b8a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0b40d25d0 0x7fc0b4150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b8a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0b415e050 0x7fc0b40b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b40b7120 con 0x7fc0b40d1260 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b8a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b40b6fa0 con 0x7fc0b40d25d0 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b8a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b40b72a0 con 0x7fc0b415e050 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d1260 0x7fc0b409e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d1260 0x7fc0b409e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58024/0 (socket says 172.21.15.71:58024) 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b2800640 1 -- 172.21.15.71:0/306814099 learned_addr learned my addr 172.21.15.71:0/306814099 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:23.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.131+0000 7fc0b1e00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0b40d25d0 0x7fc0b4150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0b2800640 1 -- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0b415e050 msgr2=0x7fc0b40b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0b2800640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0b415e050 0x7fc0b40b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0b2800640 1 -- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0b40d25d0 msgr2=0x7fc0b4150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0b2800640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0b40d25d0 0x7fc0b4150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0b2800640 1 -- 172.21.15.71:0/306814099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b40c9da0 con 0x7fc0b40d1260 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0b2800640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d1260 0x7fc0b409e780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b410 tx=0x7fc0a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0aaa00640 1 -- 172.21.15.71:0/306814099 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4015020 con 0x7fc0b40d1260 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0aaa00640 1 -- 172.21.15.71:0/306814099 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a4011940 con 0x7fc0b40d1260 2024-09-16T07:29:23.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0aaa00640 1 -- 172.21.15.71:0/306814099 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4011d90 con 0x7fc0b40d1260 2024-09-16T07:29:23.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.132+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b4151170 con 0x7fc0b40d1260 2024-09-16T07:29:23.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.133+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b409dca0 con 0x7fc0b40d1260 2024-09-16T07:29:23.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.134+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b40b77e0 con 0x7fc0b40d1260 2024-09-16T07:29:23.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.135+0000 7fc0aaa00640 1 -- 172.21.15.71:0/306814099 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7fc0a4018020 con 0x7fc0b40d1260 2024-09-16T07:29:23.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.137+0000 7fc0aaa00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc094079390 0x7fc09407b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:23.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.137+0000 7fc0aaa00640 1 -- 172.21.15.71:0/306814099 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7fc0a40991d0 con 0x7fc0b40d1260 2024-09-16T07:29:23.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.139+0000 7fc0b1e00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc094079390 0x7fc09407b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:23.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.141+0000 7fc0b1e00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc094079390 0x7fc09407b850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4156f80 tx=0x7fc0a0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:23.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.143+0000 7fc0aaa00640 1 -- 172.21.15.71:0/306814099 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc0a401f020 con 0x7fc0b40d1260 2024-09-16T07:29:23.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.512+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc0b406ea20 con 0x7fc0b40d1260 2024-09-16T07:29:23.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.513+0000 7fc0aaa00640 1 -- 172.21.15.71:0/306814099 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2181 (secure 0 0 0) 0x7fc0a40632f0 con 0x7fc0b40d1260 2024-09-16T07:29:23.512 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:23.513 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 420, 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:29:23.514 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 64, 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:23.515 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 2557, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 10621117243, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 21341429760, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 751752683520, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 2096575, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 166683455, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 0, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 41 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:23.516 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 38, 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:28:01:502092+0000", 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4, 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 4, 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "c", 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 5964 2024-09-16T07:29:23.517 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:29:23.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 12, 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:28:56.398876+0000", 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "1 ongoing clones - average progress is 4.766% (1s)\n [=...........................] (remaining: 21s)", 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.047660000622272491, 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:23.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:23.520 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:29:23.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc094079390 msgr2=0x7fc09407b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc094079390 0x7fc09407b850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4156f80 tx=0x7fc0a0009040 comp rx=0 tx=0).stop 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d1260 msgr2=0x7fc0b409e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d1260 0x7fc0b409e780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b410 tx=0x7fc0a400b8d0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 shutdown_connections 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc094079390 0x7fc09407b850 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0b415e050 0x7fc0b40b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0b40d25d0 0x7fc0b4150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 --2- 172.21.15.71:0/306814099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0b40d1260 0x7fc0b409e780 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 >> 172.21.15.71:0/306814099 conn(0x7fc0b40d3e40 msgr2=0x7fc0b40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.517+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 shutdown_connections 2024-09-16T07:29:23.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.518+0000 7fc0b8a00640 1 -- 172.21.15.71:0/306814099 wait complete. 2024-09-16T07:29:23.550 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-16T07:29:23.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.786+0000 7f5723c00640 1 -- 172.21.15.71:0/260163488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 msgr2=0x7f571c0af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:23.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.786+0000 7f5723c00640 1 --2- 172.21.15.71:0/260163488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 0x7f571c0af000 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f571c09b7d0 tx=0x7f571402a560 comp rx=0 tx=0).stop 2024-09-16T07:29:23.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.786+0000 7f5723c00640 1 -- 172.21.15.71:0/260163488 shutdown_connections 2024-09-16T07:29:23.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.786+0000 7f5723c00640 1 --2- 172.21.15.71:0/260163488 >> v2:172.21.15.177:3300/0 conn(0x7f571c1609c0 0x7f571c0aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.786+0000 7f5723c00640 1 --2- 172.21.15.71:0/260163488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 0x7f571c0af000 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.786+0000 7f5723c00640 1 -- 172.21.15.71:0/260163488 >> 172.21.15.71:0/260163488 conn(0x7f571c09b080 msgr2=0x7f571c0d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:23.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.787+0000 7f5723c00640 1 -- 172.21.15.71:0/260163488 shutdown_connections 2024-09-16T07:29:23.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.787+0000 7f5723c00640 1 -- 172.21.15.71:0/260163488 wait complete. 2024-09-16T07:29:23.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.789+0000 7f5723c00640 1 Processor -- start 2024-09-16T07:29:23.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.789+0000 7f5723c00640 1 -- start start 2024-09-16T07:29:23.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.790+0000 7f5723c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 0x7f571c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:23.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.790+0000 7f5723c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f571c1605e0 0x7f571c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:23.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.790+0000 7f5723c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f571c1609c0 0x7f571c14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:23.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.790+0000 7f5723c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f571c0cfec0 con 0x7f571c0d09a0 2024-09-16T07:29:23.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.790+0000 7f5723c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f571c0cfd40 con 0x7f571c1605e0 2024-09-16T07:29:23.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.790+0000 7f5723c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f571c0d0040 con 0x7f571c1609c0 2024-09-16T07:29:23.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 0x7f571c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:23.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 0x7f571c0b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58050/0 (socket says 172.21.15.71:58050) 2024-09-16T07:29:23.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721200640 1 -- 172.21.15.71:0/2900723150 learned_addr learned my addr 172.21.15.71:0/2900723150 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:23.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f571c1609c0 0x7f571c14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:23.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721200640 1 -- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f571c1609c0 msgr2=0x7f571c14dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:23.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f571be00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f571c1605e0 0x7f571c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721200640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f571c1609c0 0x7f571c14dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721200640 1 -- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f571c1605e0 msgr2=0x7f571c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721200640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f571c1605e0 0x7f571c0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721200640 1 -- 172.21.15.71:0/2900723150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c0b4460 con 0x7f571c0d09a0 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f5721c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f571c1609c0 0x7f571c14dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.791+0000 7f571be00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f571c1605e0 0x7f571c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.792+0000 7f5721200640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 0x7f571c0b2cc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f570c00c400 tx=0x7f570c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.792+0000 7f5718c00640 1 -- 172.21.15.71:0/2900723150 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c017070 con 0x7f571c0d09a0 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.792+0000 7f5718c00640 1 -- 172.21.15.71:0/2900723150 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f570c003640 con 0x7f571c0d09a0 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.792+0000 7f5718c00640 1 -- 172.21.15.71:0/2900723150 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c003a90 con 0x7f571c0d09a0 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.792+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571c14e160 con 0x7f571c0d09a0 2024-09-16T07:29:23.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.792+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571c0b36a0 con 0x7f571c0d09a0 2024-09-16T07:29:23.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.794+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571c0af8c0 con 0x7f571c0d09a0 2024-09-16T07:29:23.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.794+0000 7f5718c00640 1 -- 172.21.15.71:0/2900723150 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7f570c003c30 con 0x7f571c0d09a0 2024-09-16T07:29:23.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.795+0000 7f5718c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56fc079390 0x7f56fc07b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:23.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.796+0000 7f5718c00640 1 -- 172.21.15.71:0/2900723150 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7f570c0986f0 con 0x7f571c0d09a0 2024-09-16T07:29:23.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.796+0000 7f571be00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56fc079390 0x7f56fc07b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:23.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.797+0000 7f571be00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56fc079390 0x7f56fc07b850 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f571c1f8780 tx=0x7f57080073c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:23.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:23.801+0000 7f5718c00640 1 -- 172.21.15.71:0/2900723150 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f570c061f10 con 0x7f571c0d09a0 2024-09-16T07:29:24.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.129+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f571c0b7d00 con 0x7f56fc079390 2024-09-16T07:29:24.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.286+0000 7f5718c00640 1 -- 172.21.15.71:0/2900723150 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f571c0b7d00 con 0x7f56fc079390 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56fc079390 msgr2=0x7f56fc07b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56fc079390 0x7f56fc07b850 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f571c1f8780 tx=0x7f57080073c0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 msgr2=0x7f571c0b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 0x7f571c0b2cc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f570c00c400 tx=0x7f570c00c8c0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 shutdown_connections 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56fc079390 0x7f56fc07b850 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f571c1609c0 0x7f571c14dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f571c1605e0 0x7f571c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 --2- 172.21.15.71:0/2900723150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f571c0d09a0 0x7f571c0b2cc0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 >> 172.21.15.71:0/2900723150 conn(0x7f571c09b080 msgr2=0x7f571c06eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:24.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 shutdown_connections 2024-09-16T07:29:24.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.289+0000 7f5723c00640 1 -- 172.21.15.71:0/2900723150 wait complete. 2024-09-16T07:29:24.305 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:29:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 -- 172.21.15.71:0/1801665301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 msgr2=0x7fd848070c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 --2- 172.21.15.71:0/1801665301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 0x7fd848070c90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd848156570 tx=0x7fd83402a540 comp rx=0 tx=0).stop 2024-09-16T07:29:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 -- 172.21.15.71:0/1801665301 shutdown_connections 2024-09-16T07:29:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 --2- 172.21.15.71:0/1801665301 >> v2:172.21.15.177:3300/0 conn(0x7fd84815f5a0 0x7fd84815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 --2- 172.21.15.71:0/1801665301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 0x7fd848070c90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 --2- 172.21.15.71:0/1801665301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8481528d0 0x7fd848074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 -- 172.21.15.71:0/1801665301 >> 172.21.15.71:0/1801665301 conn(0x7fd8480b1b80 msgr2=0x7fd8480b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 -- 172.21.15.71:0/1801665301 shutdown_connections 2024-09-16T07:29:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.465+0000 7fd84da00640 1 -- 172.21.15.71:0/1801665301 wait complete. 2024-09-16T07:29:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.467+0000 7fd84da00640 1 Processor -- start 2024-09-16T07:29:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.467+0000 7fd84da00640 1 -- start start 2024-09-16T07:29:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.468+0000 7fd84da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 0x7fd8481f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.468+0000 7fd84da00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8481528d0 0x7fd8480af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.468+0000 7fd84da00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815f5a0 0x7fd8480cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.468+0000 7fd84da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd848152e30 con 0x7fd848070850 2024-09-16T07:29:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.468+0000 7fd84da00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd848152cb0 con 0x7fd8481528d0 2024-09-16T07:29:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.468+0000 7fd84da00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd84815ff10 con 0x7fd84815f5a0 2024-09-16T07:29:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 0x7fd8481f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8481528d0 0x7fd8480af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 0x7fd8481f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58056/0 (socket says 172.21.15.71:58056) 2024-09-16T07:29:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 -- 172.21.15.71:0/1537779954 learned_addr learned my addr 172.21.15.71:0/1537779954 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd84d000640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815f5a0 0x7fd8480cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 -- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815f5a0 msgr2=0x7fd8480cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815f5a0 0x7fd8480cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 -- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8481528d0 msgr2=0x7fd8480af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8481528d0 0x7fd8480af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 -- 172.21.15.71:0/1537779954 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8481f65b0 con 0x7fd848070850 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.469+0000 7fd847e00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 0x7fd8481f6070 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd83800e9f0 tx=0x7fd83800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.470+0000 7fd83fe00640 1 -- 172.21.15.71:0/1537779954 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83800cd80 con 0x7fd848070850 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.470+0000 7fd83fe00640 1 -- 172.21.15.71:0/1537779954 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd838005020 con 0x7fd848070850 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.470+0000 7fd83fe00640 1 -- 172.21.15.71:0/1537779954 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838005450 con 0x7fd848070850 2024-09-16T07:29:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.470+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd848145720 con 0x7fd848070850 2024-09-16T07:29:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.470+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd848145ce0 con 0x7fd848070850 2024-09-16T07:29:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.471+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd848160330 con 0x7fd848070850 2024-09-16T07:29:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.472+0000 7fd83fe00640 1 -- 172.21.15.71:0/1537779954 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7fd8380047a0 con 0x7fd848070850 2024-09-16T07:29:24.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.473+0000 7fd83fe00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828079390 0x7fd82807b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:24.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.473+0000 7fd83fe00640 1 -- 172.21.15.71:0/1537779954 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7fd838099ea0 con 0x7fd848070850 2024-09-16T07:29:24.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.473+0000 7fd847400640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828079390 0x7fd82807b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:24.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.474+0000 7fd847400640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828079390 0x7fd82807b850 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd834034000 tx=0x7fd834004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:24.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.477+0000 7fd83fe00640 1 -- 172.21.15.71:0/1537779954 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd838063780 con 0x7fd848070850 2024-09-16T07:29:24.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.686+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd84815f020 con 0x7fd828079390 2024-09-16T07:29:24.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.688+0000 7fd83fe00640 1 -- 172.21.15.71:0/1537779954 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd84815f020 con 0x7fd828079390 2024-09-16T07:29:24.688 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:29:24.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:24.688 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:29:24.688 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:24.688 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:29:24.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828079390 msgr2=0x7fd82807b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828079390 0x7fd82807b850 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd834034000 tx=0x7fd834004730 comp rx=0 tx=0).stop 2024-09-16T07:29:24.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 msgr2=0x7fd8481f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 0x7fd8481f6070 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd83800e9f0 tx=0x7fd83800edf0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 shutdown_connections 2024-09-16T07:29:24.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd828079390 0x7fd82807b850 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd84815f5a0 0x7fd8480cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8481528d0 0x7fd8480af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 --2- 172.21.15.71:0/1537779954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd848070850 0x7fd8481f6070 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 >> 172.21.15.71:0/1537779954 conn(0x7fd8480b1b80 msgr2=0x7fd8480afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 shutdown_connections 2024-09-16T07:29:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.691+0000 7fd84da00640 1 -- 172.21.15.71:0/1537779954 wait complete. 2024-09-16T07:29:24.706 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:29:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.865+0000 7fd681400640 1 -- 172.21.15.71:0/2088806256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67c06efa0 msgr2=0x7fd67c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.865+0000 7fd681400640 1 --2- 172.21.15.71:0/2088806256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67c06efa0 0x7fd67c0cdbb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd674009830 tx=0x7fd67402a480 comp rx=0 tx=0).stop 2024-09-16T07:29:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.865+0000 7fd681400640 1 -- 172.21.15.71:0/2088806256 shutdown_connections 2024-09-16T07:29:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.865+0000 7fd681400640 1 --2- 172.21.15.71:0/2088806256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67c06efa0 0x7fd67c0cdbb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.865+0000 7fd681400640 1 --2- 172.21.15.71:0/2088806256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd67c0cccd0 0x7fd67c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.865+0000 7fd681400640 1 -- 172.21.15.71:0/2088806256 >> 172.21.15.71:0/2088806256 conn(0x7fd67c0b1840 msgr2=0x7fd67c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.865+0000 7fd681400640 1 -- 172.21.15.71:0/2088806256 shutdown_connections 2024-09-16T07:29:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.865+0000 7fd681400640 1 -- 172.21.15.71:0/2088806256 wait complete. 2024-09-16T07:29:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.867+0000 7fd681400640 1 Processor -- start 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.867+0000 7fd681400640 1 -- start start 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd681400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd67c06efa0 0x7fd67c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd681400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd67c0cccd0 0x7fd67c0abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd681400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67c145750 0x7fd67c0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd681400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0ce480 con 0x7fd67c0cccd0 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd681400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0ce300 con 0x7fd67c145750 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd681400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c0ce600 con 0x7fd67c06efa0 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd679e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd67c0cccd0 0x7fd67c0abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd679e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd67c0cccd0 0x7fd67c0abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46876/0 (socket says 172.21.15.71:46876) 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd679e00640 1 -- 172.21.15.71:0/3723309788 learned_addr learned my addr 172.21.15.71:0/3723309788 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd67b200640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67c145750 0x7fd67c0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd67a800640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd67c06efa0 0x7fd67c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd679e00640 1 -- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd67c06efa0 msgr2=0x7fd67c156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd679e00640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd67c06efa0 0x7fd67c156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd679e00640 1 -- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67c145750 msgr2=0x7fd67c0ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd679e00640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67c145750 0x7fd67c0ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.869+0000 7fd679e00640 1 -- 172.21.15.71:0/3723309788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c0ab180 con 0x7fd67c0cccd0 2024-09-16T07:29:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.870+0000 7fd679e00640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd67c0cccd0 0x7fd67c0abc20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd66800e9f0 tx=0x7fd66800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.872+0000 7fd672a00640 1 -- 172.21.15.71:0/3723309788 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66800cde0 con 0x7fd67c0cccd0 2024-09-16T07:29:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.872+0000 7fd672a00640 1 -- 172.21.15.71:0/3723309788 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd668004020 con 0x7fd67c0cccd0 2024-09-16T07:29:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.872+0000 7fd672a00640 1 -- 172.21.15.71:0/3723309788 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668004450 con 0x7fd67c0cccd0 2024-09-16T07:29:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.872+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c0adaa0 con 0x7fd67c0cccd0 2024-09-16T07:29:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.872+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c0ab420 con 0x7fd67c0cccd0 2024-09-16T07:29:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.873+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c073740 con 0x7fd67c0cccd0 2024-09-16T07:29:24.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.874+0000 7fd672a00640 1 -- 172.21.15.71:0/3723309788 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7fd6680035f0 con 0x7fd67c0cccd0 2024-09-16T07:29:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.875+0000 7fd672a00640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd65c079390 0x7fd65c07b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.875+0000 7fd672a00640 1 -- 172.21.15.71:0/3723309788 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7fd668099e70 con 0x7fd67c0cccd0 2024-09-16T07:29:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.875+0000 7fd67a800640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd65c079390 0x7fd65c07b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:24.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.876+0000 7fd67a800640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd65c079390 0x7fd65c07b850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd66c001e20 tx=0x7fd66c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:24.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:24.879+0000 7fd672a00640 1 -- 172.21.15.71:0/3723309788 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd668063690 con 0x7fd67c0cccd0 2024-09-16T07:29:25.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.088+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd67c0a9500 con 0x7fd65c079390 2024-09-16T07:29:25.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.093+0000 7fd672a00640 1 -- 172.21.15.71:0/3723309788 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fd67c0a9500 con 0x7fd65c079390 2024-09-16T07:29:25.092 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:25.093 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:29:25.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:25.093 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sv1" 2024-09-16T07:29:25.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:25.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:25.093 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1clone1" 2024-09-16T07:29:25.093 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:25.093 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:29:25.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd65c079390 msgr2=0x7fd65c07b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd65c079390 0x7fd65c07b850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd66c001e20 tx=0x7fd66c009040 comp rx=0 tx=0).stop 2024-09-16T07:29:25.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd67c0cccd0 msgr2=0x7fd67c0abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd67c0cccd0 0x7fd67c0abc20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd66800e9f0 tx=0x7fd66800eeb0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 shutdown_connections 2024-09-16T07:29:25.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd65c079390 0x7fd65c07b850 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67c145750 0x7fd67c0ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd67c0cccd0 0x7fd67c0abc20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 --2- 172.21.15.71:0/3723309788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd67c06efa0 0x7fd67c156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 >> 172.21.15.71:0/3723309788 conn(0x7fd67c0b1840 msgr2=0x7fd67c0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:25.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 shutdown_connections 2024-09-16T07:29:25.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.095+0000 7fd681400640 1 -- 172.21.15.71:0/3723309788 wait complete. 2024-09-16T07:29:25.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T07:29:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.272+0000 7f3183200640 1 -- 172.21.15.71:0/3462112309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c0b3e20 msgr2=0x7f317c0cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.272+0000 7f3183200640 1 --2- 172.21.15.71:0/3462112309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c0b3e20 0x7f317c0cc340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3174005fd0 tx=0x7f317402aa00 comp rx=0 tx=0).stop 2024-09-16T07:29:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.272+0000 7f3183200640 1 -- 172.21.15.71:0/3462112309 shutdown_connections 2024-09-16T07:29:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.272+0000 7f3183200640 1 --2- 172.21.15.71:0/3462112309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317c0708b0 0x7f317c070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.272+0000 7f3183200640 1 --2- 172.21.15.71:0/3462112309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c0b3e20 0x7f317c0cc340 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.272+0000 7f3183200640 1 -- 172.21.15.71:0/3462112309 >> 172.21.15.71:0/3462112309 conn(0x7f317c0b18e0 msgr2=0x7f317c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.272+0000 7f3183200640 1 -- 172.21.15.71:0/3462112309 shutdown_connections 2024-09-16T07:29:25.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.272+0000 7f3183200640 1 -- 172.21.15.71:0/3462112309 wait complete. 2024-09-16T07:29:25.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.274+0000 7f3183200640 1 Processor -- start 2024-09-16T07:29:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.274+0000 7f3183200640 1 -- start start 2024-09-16T07:29:25.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3183200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317c0708b0 0x7f317c1516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3183200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f317c0b3e20 0x7f317c1f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3183200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c1512b0 0x7f317c14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3183200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f317c0cf130 con 0x7f317c1512b0 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3183200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f317c0cefb0 con 0x7f317c0b3e20 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3183200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f317c0cf2b0 con 0x7f317c0708b0 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3181200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c1512b0 0x7f317c14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3181200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c1512b0 0x7f317c14f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46892/0 (socket says 172.21.15.71:46892) 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3181200640 1 -- 172.21.15.71:0/3639800426 learned_addr learned my addr 172.21.15.71:0/3639800426 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f317be00640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317c0708b0 0x7f317c1516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:25.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f317b400640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f317c0b3e20 0x7f317c1f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3181200640 1 -- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317c0708b0 msgr2=0x7f317c1516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.275+0000 7f3181200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317c0708b0 0x7f317c1516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3181200640 1 -- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f317c0b3e20 msgr2=0x7f317c1f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3181200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f317c0b3e20 0x7f317c1f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3181200640 1 -- 172.21.15.71:0/3639800426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c09cfa0 con 0x7f317c1512b0 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3181200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c1512b0 0x7f317c14f7e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3174009d90 tx=0x7f3174003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3173e00640 1 -- 172.21.15.71:0/3639800426 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174047070 con 0x7f317c1512b0 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3173e00640 1 -- 172.21.15.71:0/3639800426 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3174004250 con 0x7f317c1512b0 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c151d80 con 0x7f317c1512b0 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3173e00640 1 -- 172.21.15.71:0/3639800426 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174033050 con 0x7f317c1512b0 2024-09-16T07:29:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.276+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c09d520 con 0x7f317c1512b0 2024-09-16T07:29:25.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.277+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c069360 con 0x7f317c1512b0 2024-09-16T07:29:25.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.278+0000 7f3173e00640 1 -- 172.21.15.71:0/3639800426 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7f317404a070 con 0x7f317c1512b0 2024-09-16T07:29:25.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.279+0000 7f3173e00640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f315c079390 0x7f315c07b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:25.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.279+0000 7f3173e00640 1 -- 172.21.15.71:0/3639800426 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7f31740ba3c0 con 0x7f317c1512b0 2024-09-16T07:29:25.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.280+0000 7f317be00640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f315c079390 0x7f315c07b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:25.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.280+0000 7f317be00640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f315c079390 0x7f315c07b850 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f317c1f3560 tx=0x7f316c009060 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:25.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.284+0000 7f3173e00640 1 -- 172.21.15.71:0/3639800426 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3174083db0 con 0x7f317c1512b0 2024-09-16T07:29:25.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.498+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f317c0cf790 con 0x7f315c079390 2024-09-16T07:29:25.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.515+0000 7f3173e00640 1 -- 172.21.15.71:0/3639800426 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f317c0cf790 con 0x7f315c079390 2024-09-16T07:29:25.515 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:25.515 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:29:25.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:25.515 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1" 2024-09-16T07:29:25.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:25.515 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:29:25.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f315c079390 msgr2=0x7f315c07b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f315c079390 0x7f315c07b850 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f317c1f3560 tx=0x7f316c009060 comp rx=0 tx=0).stop 2024-09-16T07:29:25.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c1512b0 msgr2=0x7f317c14f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c1512b0 0x7f317c14f7e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3174009d90 tx=0x7f3174003f30 comp rx=0 tx=0).stop 2024-09-16T07:29:25.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 shutdown_connections 2024-09-16T07:29:25.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f315c079390 0x7f315c07b850 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f317c1512b0 0x7f317c14f7e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f317c0b3e20 0x7f317c1f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 --2- 172.21.15.71:0/3639800426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317c0708b0 0x7f317c1516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 >> 172.21.15.71:0/3639800426 conn(0x7f317c0b18e0 msgr2=0x7f317c0ad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:25.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 shutdown_connections 2024-09-16T07:29:25.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.518+0000 7f3183200640 1 -- 172.21.15.71:0/3639800426 wait complete. 2024-09-16T07:29:25.533 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T07:29:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.695+0000 7fc0d9400640 1 -- 172.21.15.71:0/4104271081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 msgr2=0x7fc0d40b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.695+0000 7fc0d9400640 1 --2- 172.21.15.71:0/4104271081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 0x7fc0d40b2b40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005fd0 tx=0x7fc0c002a5c0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.695+0000 7fc0d9400640 1 -- 172.21.15.71:0/4104271081 shutdown_connections 2024-09-16T07:29:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.695+0000 7fc0d9400640 1 --2- 172.21.15.71:0/4104271081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0d40cec60 0x7fc0d40cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.695+0000 7fc0d9400640 1 --2- 172.21.15.71:0/4104271081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 0x7fc0d40b2b40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.695+0000 7fc0d9400640 1 -- 172.21.15.71:0/4104271081 >> 172.21.15.71:0/4104271081 conn(0x7fc0d40b1840 msgr2=0x7fc0d40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.695+0000 7fc0d9400640 1 -- 172.21.15.71:0/4104271081 shutdown_connections 2024-09-16T07:29:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.695+0000 7fc0d9400640 1 -- 172.21.15.71:0/4104271081 wait complete. 2024-09-16T07:29:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.697+0000 7fc0d9400640 1 Processor -- start 2024-09-16T07:29:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.697+0000 7fc0d9400640 1 -- start start 2024-09-16T07:29:25.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 0x7fc0d4147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:25.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d9400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0d40cec60 0x7fc0d41e7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:25.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d9400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0d4161760 0x7fc0d4147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:25.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d9400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4162310 con 0x7fc0d40af190 2024-09-16T07:29:25.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d9400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4162190 con 0x7fc0d4161760 2024-09-16T07:29:25.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d9400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d4162490 con 0x7fc0d40cec60 2024-09-16T07:29:25.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 0x7fc0d4147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:25.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 0x7fc0d4147fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46912/0 (socket says 172.21.15.71:46912) 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d2800640 1 -- 172.21.15.71:0/3103279586 learned_addr learned my addr 172.21.15.71:0/3103279586 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d1e00640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0d40cec60 0x7fc0d41e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.698+0000 7fc0d3200640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0d4161760 0x7fc0d4147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0d2800640 1 -- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0d40cec60 msgr2=0x7fc0d41e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0d2800640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0d40cec60 0x7fc0d41e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0d2800640 1 -- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0d4161760 msgr2=0x7fc0d4147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0d2800640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0d4161760 0x7fc0d4147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0d2800640 1 -- 172.21.15.71:0/3103279586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d4148640 con 0x7fc0d40af190 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0d2800640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 0x7fc0d4147fa0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400c3c0 tx=0x7fc0c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0caa00640 1 -- 172.21.15.71:0/3103279586 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4017070 con 0x7fc0d40af190 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0caa00640 1 -- 172.21.15.71:0/3103279586 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0c40035f0 con 0x7fc0d40af190 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0caa00640 1 -- 172.21.15.71:0/3103279586 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4003a80 con 0x7fc0d40af190 2024-09-16T07:29:25.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d4148920 con 0x7fc0d40af190 2024-09-16T07:29:25.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.699+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d4148ee0 con 0x7fc0d40af190 2024-09-16T07:29:25.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.701+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d4162940 con 0x7fc0d40af190 2024-09-16T07:29:25.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.702+0000 7fc0caa00640 1 -- 172.21.15.71:0/3103279586 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7fc0c401a070 con 0x7fc0d40af190 2024-09-16T07:29:25.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.703+0000 7fc0caa00640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0b4079390 0x7fc0b407b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:25.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.703+0000 7fc0caa00640 1 -- 172.21.15.71:0/3103279586 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7fc0c40986d0 con 0x7fc0d40af190 2024-09-16T07:29:25.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.703+0000 7fc0d1e00640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0b4079390 0x7fc0b407b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:25.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.704+0000 7fc0d1e00640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0b4079390 0x7fc0b407b850 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0033000 tx=0x7fc0c0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:25.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.707+0000 7fc0caa00640 1 -- 172.21.15.71:0/3103279586 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc0c4061f50 con 0x7fc0d40af190 2024-09-16T07:29:25.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.928+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0d4073db0 con 0x7fc0b4079390 2024-09-16T07:29:25.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.957+0000 7fc0caa00640 1 -- 172.21.15.71:0/3103279586 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc0d4073db0 con 0x7fc0b4079390 2024-09-16T07:29:25.956 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:25.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0b4079390 msgr2=0x7fc0b407b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0b4079390 0x7fc0b407b850 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0033000 tx=0x7fc0c0004730 comp rx=0 tx=0).stop 2024-09-16T07:29:25.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 msgr2=0x7fc0d4147fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:25.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 0x7fc0d4147fa0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400c3c0 tx=0x7fc0c400c880 comp rx=0 tx=0).stop 2024-09-16T07:29:25.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 shutdown_connections 2024-09-16T07:29:25.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0b4079390 0x7fc0b407b850 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0d4161760 0x7fc0d4147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0d40cec60 0x7fc0d41e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.959+0000 7fc0d9400640 1 --2- 172.21.15.71:0/3103279586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d40af190 0x7fc0d4147fa0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:25.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.960+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 >> 172.21.15.71:0/3103279586 conn(0x7fc0d40b1840 msgr2=0x7fc0d4072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:25.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.960+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 shutdown_connections 2024-09-16T07:29:25.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:25.960+0000 7fc0d9400640 1 -- 172.21.15.71:0/3103279586 wait complete. 2024-09-16T07:29:25.975 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T07:29:26.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.142+0000 7fb387200640 1 -- 172.21.15.71:0/3515587378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3880d2a00 msgr2=0x7fb3880b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:26.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.142+0000 7fb387200640 1 --2- 172.21.15.71:0/3515587378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3880d2a00 0x7fb3880b38f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb380009830 tx=0x7fb38002a480 comp rx=0 tx=0).stop 2024-09-16T07:29:26.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.142+0000 7fb387200640 1 -- 172.21.15.71:0/3515587378 shutdown_connections 2024-09-16T07:29:26.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.142+0000 7fb387200640 1 --2- 172.21.15.71:0/3515587378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3880d2a00 0x7fb3880b38f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.142+0000 7fb387200640 1 --2- 172.21.15.71:0/3515587378 >> v2:172.21.15.177:3300/0 conn(0x7fb3880cf830 0x7fb3880b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.142+0000 7fb387200640 1 -- 172.21.15.71:0/3515587378 >> 172.21.15.71:0/3515587378 conn(0x7fb38809d0b0 msgr2=0x7fb3880d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:26.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.142+0000 7fb387200640 1 -- 172.21.15.71:0/3515587378 shutdown_connections 2024-09-16T07:29:26.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.142+0000 7fb387200640 1 -- 172.21.15.71:0/3515587378 wait complete. 2024-09-16T07:29:26.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.144+0000 7fb387200640 1 Processor -- start 2024-09-16T07:29:26.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.144+0000 7fb387200640 1 -- start start 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb387200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3880cf830 0x7fb3880a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb387200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3880d2a00 0x7fb3880b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb387200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb388161460 0x7fb3880b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb387200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb3880b50c0 con 0x7fb388161460 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb387200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb3880b4f40 con 0x7fb3880cf830 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb387200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb3880b3e30 con 0x7fb3880d2a00 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb386800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb388161460 0x7fb3880b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb386800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb388161460 0x7fb3880b94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46920/0 (socket says 172.21.15.71:46920) 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb386800640 1 -- 172.21.15.71:0/1937951191 learned_addr learned my addr 172.21.15.71:0/1937951191 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb385e00640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3880cf830 0x7fb3880a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb37d400640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3880d2a00 0x7fb3880b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb386800640 1 -- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3880d2a00 msgr2=0x7fb3880b8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb386800640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3880d2a00 0x7fb3880b8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb386800640 1 -- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3880cf830 msgr2=0x7fb3880a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:26.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb386800640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3880cf830 0x7fb3880a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.145+0000 7fb386800640 1 -- 172.21.15.71:0/1937951191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3881f4760 con 0x7fb388161460 2024-09-16T07:29:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.146+0000 7fb386800640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb388161460 0x7fb3880b94c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb380009950 tx=0x7fb380032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.146+0000 7fb37ea00640 1 -- 172.21.15.71:0/1937951191 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380040070 con 0x7fb388161460 2024-09-16T07:29:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.146+0000 7fb37ea00640 1 -- 172.21.15.71:0/1937951191 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb380005310 con 0x7fb388161460 2024-09-16T07:29:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.146+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3881f8a40 con 0x7fb388161460 2024-09-16T07:29:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.146+0000 7fb37ea00640 1 -- 172.21.15.71:0/1937951191 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3800047a0 con 0x7fb388161460 2024-09-16T07:29:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.146+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3881f4fa0 con 0x7fb388161460 2024-09-16T07:29:26.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.147+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3880b42e0 con 0x7fb388161460 2024-09-16T07:29:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.148+0000 7fb37ea00640 1 -- 172.21.15.71:0/1937951191 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7fb380039040 con 0x7fb388161460 2024-09-16T07:29:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.149+0000 7fb37ea00640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb364079390 0x7fb36407b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.149+0000 7fb37ea00640 1 -- 172.21.15.71:0/1937951191 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7fb3800ba480 con 0x7fb388161460 2024-09-16T07:29:26.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.150+0000 7fb385e00640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb364079390 0x7fb36407b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:26.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.151+0000 7fb385e00640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb364079390 0x7fb36407b850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb378001e20 tx=0x7fb378009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.153+0000 7fb37ea00640 1 -- 172.21.15.71:0/1937951191 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb3881f4760 con 0x7fb388161460 2024-09-16T07:29:26.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.361+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb388069d80 con 0x7fb364079390 2024-09-16T07:29:26.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.381+0000 7fb37ea00640 1 -- 172.21.15.71:0/1937951191 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb37800f9e0 con 0x7fb364079390 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb364079390 msgr2=0x7fb36407b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb364079390 0x7fb36407b850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb378001e20 tx=0x7fb378009040 comp rx=0 tx=0).stop 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb388161460 msgr2=0x7fb3880b94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb388161460 0x7fb3880b94c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb380009950 tx=0x7fb380032b90 comp rx=0 tx=0).stop 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 shutdown_connections 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb364079390 0x7fb36407b850 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb388161460 0x7fb3880b94c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3880d2a00 0x7fb3880b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 --2- 172.21.15.71:0/1937951191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3880cf830 0x7fb3880a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 >> 172.21.15.71:0/1937951191 conn(0x7fb38809d0b0 msgr2=0x7fb38806eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:26.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 shutdown_connections 2024-09-16T07:29:26.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.384+0000 7fb387200640 1 -- 172.21.15.71:0/1937951191 wait complete. 2024-09-16T07:29:26.394 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:26.394+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:26.394 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:26.394+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:26.394 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:26.394+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:26.394 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:26.394+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:26.394 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:26.394+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:26.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-16T07:29:26.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.573+0000 7f55ebe00640 1 -- 172.21.15.71:0/480123307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec161580 msgr2=0x7f55ec0d48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:26.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.573+0000 7f55ebe00640 1 --2- 172.21.15.71:0/480123307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec161580 0x7f55ec0d48f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009830 tx=0x7f55d402a5c0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.573+0000 7f55ebe00640 1 -- 172.21.15.71:0/480123307 shutdown_connections 2024-09-16T07:29:26.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.573+0000 7f55ebe00640 1 --2- 172.21.15.71:0/480123307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec161580 0x7f55ec0d48f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.573+0000 7f55ebe00640 1 -- 172.21.15.71:0/480123307 >> 172.21.15.71:0/480123307 conn(0x7f55ec0d3dd0 msgr2=0x7f55ec0d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:26.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.573+0000 7f55ebe00640 1 -- 172.21.15.71:0/480123307 shutdown_connections 2024-09-16T07:29:26.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.573+0000 7f55ebe00640 1 -- 172.21.15.71:0/480123307 wait complete. 2024-09-16T07:29:26.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.574+0000 7f55ebe00640 1 Processor -- start 2024-09-16T07:29:26.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.574+0000 7f55ebe00640 1 -- start start 2024-09-16T07:29:26.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.575+0000 7f55ebe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec0cfa90 0x7f55ec1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.575+0000 7f55ebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55ec1611a0 0x7f55ec0ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.575+0000 7f55ebe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55ec161580 0x7f55ec0b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.575+0000 7f55ebe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec169140 con 0x7f55ec1611a0 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.575+0000 7f55ebe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec168fc0 con 0x7f55ec0cfa90 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.575+0000 7f55ebe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55ec1692c0 con 0x7f55ec161580 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55ec1611a0 0x7f55ec0ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55ec1611a0 0x7f55ec0ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46936/0 (socket says 172.21.15.71:46936) 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 -- 172.21.15.71:0/3318824145 learned_addr learned my addr 172.21.15.71:0/3318824145 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55eb400640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55ec161580 0x7f55ec0b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55eaa00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec0cfa90 0x7f55ec1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 -- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55ec161580 msgr2=0x7f55ec0b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:26.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55ec161580 0x7f55ec0b8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 -- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec0cfa90 msgr2=0x7f55ec1603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:26.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec0cfa90 0x7f55ec1603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:26.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 -- 172.21.15.71:0/3318824145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec0b3610 con 0x7f55ec1611a0 2024-09-16T07:29:26.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55eb400640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55ec161580 0x7f55ec0b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:26.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55eaa00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec0cfa90 0x7f55ec1603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:26.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.576+0000 7f55e3e00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55ec1611a0 0x7f55ec0ca4e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009950 tx=0x7f55d4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:26.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.577+0000 7f55e2a00640 1 -- 172.21.15.71:0/3318824145 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4031d60 con 0x7f55ec1611a0 2024-09-16T07:29:26.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.577+0000 7f55e2a00640 1 -- 172.21.15.71:0/3318824145 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55d4033e10 con 0x7f55ec1611a0 2024-09-16T07:29:26.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.577+0000 7f55e2a00640 1 -- 172.21.15.71:0/3318824145 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4033310 con 0x7f55ec1611a0 2024-09-16T07:29:26.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.577+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec0b38f0 con 0x7f55ec1611a0 2024-09-16T07:29:26.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.577+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec0b3eb0 con 0x7f55ec1611a0 2024-09-16T07:29:26.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.578+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ec0d48f0 con 0x7f55ec1611a0 2024-09-16T07:29:26.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.579+0000 7f55e2a00640 1 -- 172.21.15.71:0/3318824145 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7f55d40334b0 con 0x7f55ec1611a0 2024-09-16T07:29:26.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.580+0000 7f55e2a00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55c80793c0 0x7f55c807b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:26.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.580+0000 7f55e2a00640 1 -- 172.21.15.71:0/3318824145 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7f55d40ba870 con 0x7f55ec1611a0 2024-09-16T07:29:26.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.580+0000 7f55eaa00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55c80793c0 0x7f55c807b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:26.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.581+0000 7f55eaa00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55c80793c0 0x7f55c807b880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0046b0 tx=0x7f55dc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:26.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.584+0000 7f55e2a00640 1 -- 172.21.15.71:0/3318824145 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f55d403f030 con 0x7f55ec1611a0 2024-09-16T07:29:26.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.802+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f55ec0b2840 con 0x7f55c80793c0 2024-09-16T07:29:26.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:26.886+0000 7f55e2a00640 1 -- 172.21.15.71:0/3318824145 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f55d4080250 con 0x7f55ec1611a0 2024-09-16T07:29:28.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.332+0000 7f55e2a00640 1 -- 172.21.15.71:0/3318824145 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f55ec0b2840 con 0x7f55c80793c0 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55c80793c0 msgr2=0x7f55c807b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55c80793c0 0x7f55c807b880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0046b0 tx=0x7f55dc009040 comp rx=0 tx=0).stop 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55ec1611a0 msgr2=0x7f55ec0ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55ec1611a0 0x7f55ec0ca4e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009950 tx=0x7f55d4033c20 comp rx=0 tx=0).stop 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 shutdown_connections 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55c80793c0 0x7f55c807b880 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55ec161580 0x7f55ec0b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55ec1611a0 0x7f55ec0ca4e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:28.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 --2- 172.21.15.71:0/3318824145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55ec0cfa90 0x7f55ec1603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:28.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 >> 172.21.15.71:0/3318824145 conn(0x7f55ec0d3dd0 msgr2=0x7f55ec0cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:28.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 shutdown_connections 2024-09-16T07:29:28.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.335+0000 7f55ebe00640 1 -- 172.21.15.71:0/3318824145 wait complete. 2024-09-16T07:29:28.355 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:29:28.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:29:28.441 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:29:28.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:29:28.450 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:29:28.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.661+0000 7f4e3c800640 1 -- 172.21.15.71:0/2412783895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e340cccd0 msgr2=0x7f4e34073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:28.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.661+0000 7f4e3c800640 1 --2- 172.21.15.71:0/2412783895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e340cccd0 0x7f4e34073440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009830 tx=0x7f4e2402a480 comp rx=0 tx=0).stop 2024-09-16T07:29:28.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.661+0000 7f4e3c800640 1 -- 172.21.15.71:0/2412783895 shutdown_connections 2024-09-16T07:29:28.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.661+0000 7f4e3c800640 1 --2- 172.21.15.71:0/2412783895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e340cccd0 0x7f4e34073440 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:28.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.661+0000 7f4e3c800640 1 --2- 172.21.15.71:0/2412783895 >> v2:172.21.15.71:3300/0 conn(0x7f4e3406efa0 0x7f4e340cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:28.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.661+0000 7f4e3c800640 1 -- 172.21.15.71:0/2412783895 >> 172.21.15.71:0/2412783895 conn(0x7f4e340b1840 msgr2=0x7f4e340b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:28.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.662+0000 7f4e3c800640 1 -- 172.21.15.71:0/2412783895 shutdown_connections 2024-09-16T07:29:28.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.662+0000 7f4e3c800640 1 -- 172.21.15.71:0/2412783895 wait complete. 2024-09-16T07:29:28.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.664+0000 7f4e3c800640 1 Processor -- start 2024-09-16T07:29:28.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.665+0000 7f4e3c800640 1 -- start start 2024-09-16T07:29:28.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.666+0000 7f4e3c800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3406efa0 0x7f4e340cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:28.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.666+0000 7f4e3c800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e340cccd0 0x7f4e34144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:28.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.666+0000 7f4e3c800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e34145750 0x7f4e340aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:28.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.666+0000 7f4e3c800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340ce480 con 0x7f4e3406efa0 2024-09-16T07:29:28.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.666+0000 7f4e3c800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340ce300 con 0x7f4e34145750 2024-09-16T07:29:28.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.666+0000 7f4e3c800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4e340ce600 con 0x7f4e340cccd0 2024-09-16T07:29:28.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.667+0000 7f4e3a800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e34145750 0x7f4e340aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:28.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.667+0000 7f4e39e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3406efa0 0x7f4e340cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:28.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.667+0000 7f4e39e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3406efa0 0x7f4e340cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46954/0 (socket says 172.21.15.71:46954) 2024-09-16T07:29:28.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.667+0000 7f4e39e00640 1 -- 172.21.15.71:0/1696485688 learned_addr learned my addr 172.21.15.71:0/1696485688 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:28.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.667+0000 7f4e39400640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e340cccd0 0x7f4e34144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:28.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.668+0000 7f4e39e00640 1 -- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e340cccd0 msgr2=0x7f4e34144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:28.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.668+0000 7f4e39e00640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e340cccd0 0x7f4e34144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:28.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.668+0000 7f4e39e00640 1 -- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e34145750 msgr2=0x7f4e340aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:28.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.668+0000 7f4e39e00640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e34145750 0x7f4e340aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:28.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.668+0000 7f4e39e00640 1 -- 172.21.15.71:0/1696485688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e34138d00 con 0x7f4e3406efa0 2024-09-16T07:29:28.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.669+0000 7f4e3a800640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e34145750 0x7f4e340aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:28.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.669+0000 7f4e39e00640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3406efa0 0x7f4e340cbe00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800c430 tx=0x7f4e2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:28.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.669+0000 7f4e2e000640 1 -- 172.21.15.71:0/1696485688 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28011790 con 0x7f4e3406efa0 2024-09-16T07:29:28.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.669+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e34138f80 con 0x7f4e3406efa0 2024-09-16T07:29:28.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.670+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e34139540 con 0x7f4e3406efa0 2024-09-16T07:29:28.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.671+0000 7f4e2e000640 1 -- 172.21.15.71:0/1696485688 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e28017070 con 0x7f4e3406efa0 2024-09-16T07:29:28.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.671+0000 7f4e2e000640 1 -- 172.21.15.71:0/1696485688 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28011ce0 con 0x7f4e3406efa0 2024-09-16T07:29:28.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.672+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e10005290 con 0x7f4e3406efa0 2024-09-16T07:29:28.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.674+0000 7f4e2e000640 1 -- 172.21.15.71:0/1696485688 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f4e2801c460 con 0x7f4e3406efa0 2024-09-16T07:29:28.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.676+0000 7f4e2e000640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e18079420 0x7f4e1807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:28.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.676+0000 7f4e2e000640 1 -- 172.21.15.71:0/1696485688 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5767+0+0 (secure 0 0 0) 0x7f4e28099170 con 0x7f4e3406efa0 2024-09-16T07:29:28.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.677+0000 7f4e39400640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e18079420 0x7f4e1807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:28.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.678+0000 7f4e39400640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e18079420 0x7f4e1807b8e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4e24034000 tx=0x7f4e24004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:28.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:28.684+0000 7f4e2e000640 1 -- 172.21.15.71:0/1696485688 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e34138d00 con 0x7f4e3406efa0 2024-09-16T07:29:29.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.027+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f4e10005580 con 0x7f4e3406efa0 2024-09-16T07:29:29.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.028+0000 7f4e2e000640 1 -- 172.21.15.71:0/1696485688 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7f4e28062fd0 con 0x7f4e3406efa0 2024-09-16T07:29:29.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e18079420 msgr2=0x7f4e1807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:29.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e18079420 0x7f4e1807b8e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4e24034000 tx=0x7f4e24004730 comp rx=0 tx=0).stop 2024-09-16T07:29:29.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3406efa0 msgr2=0x7f4e340cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3406efa0 0x7f4e340cbe00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800c430 tx=0x7f4e2800c8f0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 shutdown_connections 2024-09-16T07:29:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e18079420 0x7f4e1807b8e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e34145750 0x7f4e340aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e340cccd0 0x7f4e34144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 --2- 172.21.15.71:0/1696485688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3406efa0 0x7f4e340cbe00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 >> 172.21.15.71:0/1696485688 conn(0x7f4e340b1840 msgr2=0x7f4e340b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.033+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 shutdown_connections 2024-09-16T07:29:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.034+0000 7f4e3c800640 1 -- 172.21.15.71:0/1696485688 wait complete. 2024-09-16T07:29:29.058 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:29:29.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:29:29.178 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:29:29.301 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:29:29.301 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:29:29.301 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:29.301 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:29.357 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:29.357 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:29.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:29:29.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.669+0000 7fdbc7200640 1 -- 172.21.15.71:0/559500682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc80b4100 msgr2=0x7fdbc80b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:29.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.669+0000 7fdbc7200640 1 --2- 172.21.15.71:0/559500682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc80b4100 0x7fdbc80b1770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80d4ea0 tx=0x7fdbc002a9f0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.669+0000 7fdbc7200640 1 -- 172.21.15.71:0/559500682 shutdown_connections 2024-09-16T07:29:29.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.669+0000 7fdbc7200640 1 --2- 172.21.15.71:0/559500682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc80b4100 0x7fdbc80b1770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.669+0000 7fdbc7200640 1 --2- 172.21.15.71:0/559500682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80b5370 0x7fdbc80b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.669+0000 7fdbc7200640 1 -- 172.21.15.71:0/559500682 >> 172.21.15.71:0/559500682 conn(0x7fdbc80d3df0 msgr2=0x7fdbc80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.670+0000 7fdbc7200640 1 -- 172.21.15.71:0/559500682 shutdown_connections 2024-09-16T07:29:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.670+0000 7fdbc7200640 1 -- 172.21.15.71:0/559500682 wait complete. 2024-09-16T07:29:29.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.673+0000 7fdbc7200640 1 Processor -- start 2024-09-16T07:29:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.673+0000 7fdbc7200640 1 -- start start 2024-09-16T07:29:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.675+0000 7fdbc7200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80b4100 0x7fdbc81e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.675+0000 7fdbc7200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc80b5370 0x7fdbc80cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.675+0000 7fdbc7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbc8161440 0x7fdbc806e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.675+0000 7fdbc7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc80b2040 con 0x7fdbc8161440 2024-09-16T07:29:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.675+0000 7fdbc7200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc80b1ec0 con 0x7fdbc80b5370 2024-09-16T07:29:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.675+0000 7fdbc7200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc80b21c0 con 0x7fdbc80b4100 2024-09-16T07:29:29.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.676+0000 7fdbc5400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc80b5370 0x7fdbc80cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:29.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.676+0000 7fdbc6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbc8161440 0x7fdbc806e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:29.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.676+0000 7fdbc6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbc8161440 0x7fdbc806e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46960/0 (socket says 172.21.15.71:46960) 2024-09-16T07:29:29.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.676+0000 7fdbc5e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80b4100 0x7fdbc81e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:29.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.676+0000 7fdbc6800640 1 -- 172.21.15.71:0/720149003 learned_addr learned my addr 172.21.15.71:0/720149003 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:29.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.676+0000 7fdbc6800640 1 -- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80b4100 msgr2=0x7fdbc81e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:29.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.676+0000 7fdbc6800640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80b4100 0x7fdbc81e9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.676+0000 7fdbc6800640 1 -- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc80b5370 msgr2=0x7fdbc80cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:29.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.677+0000 7fdbc6800640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc80b5370 0x7fdbc80cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.677+0000 7fdbc6800640 1 -- 172.21.15.71:0/720149003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc80cf5b0 con 0x7fdbc8161440 2024-09-16T07:29:29.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.677+0000 7fdbc6800640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbc8161440 0x7fdbc806e660 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009d60 tx=0x7fdbc0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:29.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.677+0000 7fdbbe000640 1 -- 172.21.15.71:0/720149003 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0047070 con 0x7fdbc8161440 2024-09-16T07:29:29.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.678+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc80cf890 con 0x7fdbc8161440 2024-09-16T07:29:29.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.678+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc80cf280 con 0x7fdbc8161440 2024-09-16T07:29:29.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.678+0000 7fdbbe000640 1 -- 172.21.15.71:0/720149003 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbc00098c0 con 0x7fdbc8161440 2024-09-16T07:29:29.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.678+0000 7fdbbe000640 1 -- 172.21.15.71:0/720149003 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0033050 con 0x7fdbc8161440 2024-09-16T07:29:29.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.682+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba0005290 con 0x7fdbc8161440 2024-09-16T07:29:29.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.683+0000 7fdbbe000640 1 -- 172.21.15.71:0/720149003 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fdbc0036070 con 0x7fdbc8161440 2024-09-16T07:29:29.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.684+0000 7fdbbe000640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdba8079420 0x7fdba807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:29.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.685+0000 7fdbbe000640 1 -- 172.21.15.71:0/720149003 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5823+0+0 (secure 0 0 0) 0x7fdbc00b9e40 con 0x7fdbc8161440 2024-09-16T07:29:29.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.685+0000 7fdbc5e00640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdba8079420 0x7fdba807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:29.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.686+0000 7fdbc5e00640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdba8079420 0x7fdba807b8e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdbc81609c0 tx=0x7fdbb8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:29.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.691+0000 7fdbbe000640 1 -- 172.21.15.71:0/720149003 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdbc0083890 con 0x7fdbc8161440 2024-09-16T07:29:29.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.914+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdba0005850 con 0x7fdbc8161440 2024-09-16T07:29:29.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.915+0000 7fdbbe000640 1 -- 172.21.15.71:0/720149003 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) ==== 80+0+0 (secure 0 0 0) 0x7fdbc00882c0 con 0x7fdbc8161440 2024-09-16T07:29:29.914 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:29:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdba8079420 msgr2=0x7fdba807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdba8079420 0x7fdba807b8e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdbc81609c0 tx=0x7fdbb8009040 comp rx=0 tx=0).stop 2024-09-16T07:29:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbc8161440 msgr2=0x7fdbc806e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbc8161440 0x7fdbc806e660 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009d60 tx=0x7fdbc0003f10 comp rx=0 tx=0).stop 2024-09-16T07:29:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 shutdown_connections 2024-09-16T07:29:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdba8079420 0x7fdba807b8e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbc8161440 0x7fdbc806e660 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc80b5370 0x7fdbc80cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 --2- 172.21.15.71:0/720149003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80b4100 0x7fdbc81e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:29.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 >> 172.21.15.71:0/720149003 conn(0x7fdbc80d3df0 msgr2=0x7fdbc80b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:29.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 shutdown_connections 2024-09-16T07:29:29.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:29.917+0000 7fdbc7200640 1 -- 172.21.15.71:0/720149003 wait complete. 2024-09-16T07:29:29.932 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:29:29.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:29.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:29:30.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:29:30.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:30.022 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:29:30.067 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:29:30.068 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:29:30.068 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:30.104 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:29:30.104 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:29:30.104 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:30.104 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:30.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:29:30.143 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:29:30.143 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:29:30.143 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:30.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:29:30.180 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:29:30.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 -- 172.21.15.71:0/2727680020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 msgr2=0x7fb7640b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:30.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 --2- 172.21.15.71:0/2727680020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 0x7fb7640b2440 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb7640cfbb0 tx=0x7fb75402a980 comp rx=0 tx=0).stop 2024-09-16T07:29:30.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 -- 172.21.15.71:0/2727680020 shutdown_connections 2024-09-16T07:29:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 --2- 172.21.15.71:0/2727680020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ad270 0x7fb764073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 --2- 172.21.15.71:0/2727680020 >> v2:172.21.15.177:3300/0 conn(0x7fb7640ccf00 0x7fb7640acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 --2- 172.21.15.71:0/2727680020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 0x7fb7640b2440 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 -- 172.21.15.71:0/2727680020 >> 172.21.15.71:0/2727680020 conn(0x7fb7640b1950 msgr2=0x7fb7640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 -- 172.21.15.71:0/2727680020 shutdown_connections 2024-09-16T07:29:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.340+0000 7fb76a600640 1 -- 172.21.15.71:0/2727680020 wait complete. 2024-09-16T07:29:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.342+0000 7fb76a600640 1 Processor -- start 2024-09-16T07:29:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.342+0000 7fb76a600640 1 -- start start 2024-09-16T07:29:30.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.343+0000 7fb76a600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ad270 0x7fb764150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:30.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.343+0000 7fb76a600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7640ccf00 0x7fb76414de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:30.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.343+0000 7fb76a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 0x7fb764150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:30.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.343+0000 7fb76a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb764074880 con 0x7fb76415f5a0 2024-09-16T07:29:30.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.343+0000 7fb76a600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb764074700 con 0x7fb7640ccf00 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.343+0000 7fb76a600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb76406f160 con 0x7fb7640ad270 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.343+0000 7fb763e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 0x7fb764150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.343+0000 7fb763e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 0x7fb764150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46970/0 (socket says 172.21.15.71:46970) 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb763e00640 1 -- 172.21.15.71:0/2341269816 learned_addr learned my addr 172.21.15.71:0/2341269816 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb762a00640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7640ccf00 0x7fb76414de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb763400640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ad270 0x7fb764150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb763e00640 1 -- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ad270 msgr2=0x7fb764150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb763e00640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ad270 0x7fb764150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb763e00640 1 -- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7640ccf00 msgr2=0x7fb76414de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:30.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb763e00640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7640ccf00 0x7fb76414de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb763e00640 1 -- 172.21.15.71:0/2341269816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76414cde0 con 0x7fb76415f5a0 2024-09-16T07:29:30.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb763e00640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 0x7fb764150990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00e970 tx=0x7fb75c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:30.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb75b400640 1 -- 172.21.15.71:0/2341269816 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00ccc0 con 0x7fb76415f5a0 2024-09-16T07:29:30.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.344+0000 7fb75b400640 1 -- 172.21.15.71:0/2341269816 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb75c00ce60 con 0x7fb76415f5a0 2024-09-16T07:29:30.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.345+0000 7fb75b400640 1 -- 172.21.15.71:0/2341269816 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c005280 con 0x7fb76415f5a0 2024-09-16T07:29:30.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.345+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7640cf750 con 0x7fb76415f5a0 2024-09-16T07:29:30.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.345+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7640cfd10 con 0x7fb76415f5a0 2024-09-16T07:29:30.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.346+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb764073fb0 con 0x7fb76415f5a0 2024-09-16T07:29:30.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.347+0000 7fb75b400640 1 -- 172.21.15.71:0/2341269816 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fb75c005420 con 0x7fb76415f5a0 2024-09-16T07:29:30.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.348+0000 7fb75b400640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb744079420 0x7fb74407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:30.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.348+0000 7fb75b400640 1 -- 172.21.15.71:0/2341269816 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5823+0+0 (secure 0 0 0) 0x7fb75c099af0 con 0x7fb76415f5a0 2024-09-16T07:29:30.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.348+0000 7fb763400640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb744079420 0x7fb74407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:30.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.349+0000 7fb763400640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb744079420 0x7fb74407b8e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb754035000 tx=0x7fb754004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:30.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.352+0000 7fb75b400640 1 -- 172.21.15.71:0/2341269816 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb76414cde0 con 0x7fb76415f5a0 2024-09-16T07:29:30.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.614+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7640afb80 con 0x7fb76415f5a0 2024-09-16T07:29:30.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.615+0000 7fb75b400640 1 -- 172.21.15.71:0/2341269816 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5229 (secure 0 0 0) 0x7fb75c0635c0 con 0x7fb76415f5a0 2024-09-16T07:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:30.614 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"btime":"2024-09-16T07:28:01:502092+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1090989192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1090989192},{"type":"v1","addr":"172.21.15.177:6836","nonce":1090989192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2044137966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2044137966},{"type":"v1","addr":"172.21.15.177:6837","nonce":2044137966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3911470669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3911470669},{"type":"v1","addr":"172.21.15.71:6835","nonce":3911470669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:27:47.864418+0000","modified":"2024-09-16T07:27:50.881337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6834/3242207389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3242207389},{"type":"v1","addr":"172.21.15.71:6834","nonce":3242207389}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5964,"qdb_cluster":[5964]},"id":4}]} 2024-09-16T07:29:30.614 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2024-09-16T07:29:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb744079420 msgr2=0x7fb74407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb744079420 0x7fb74407b8e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb754035000 tx=0x7fb754004ab0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 msgr2=0x7fb764150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 0x7fb764150990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00e970 tx=0x7fb75c00ee30 comp rx=0 tx=0).stop 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 shutdown_connections 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb744079420 0x7fb74407b8e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb76415f5a0 0x7fb764150990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7640ccf00 0x7fb76414de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 --2- 172.21.15.71:0/2341269816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ad270 0x7fb764150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 >> 172.21.15.71:0/2341269816 conn(0x7fb7640b1950 msgr2=0x7fb76409c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 shutdown_connections 2024-09-16T07:29:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.618+0000 7fb76a600640 1 -- 172.21.15.71:0/2341269816 wait complete. 2024-09-16T07:29:30.642 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:29:30.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.833+0000 7fdca0200640 1 -- 172.21.15.71:0/2603056589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc80004800 msgr2=0x7fdc80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:30.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.833+0000 7fdca0200640 1 --2- 172.21.15.71:0/2603056589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc80004800 0x7fdc80004c80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009b50 tx=0x7fdc80005830 comp rx=0 tx=0).stop 2024-09-16T07:29:30.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.833+0000 7fdc9e200640 1 -- 172.21.15.71:0/2603056589 reap_dead start 2024-09-16T07:29:30.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.833+0000 7fdca0200640 1 -- 172.21.15.71:0/2603056589 shutdown_connections 2024-09-16T07:29:30.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.833+0000 7fdca0200640 1 -- 172.21.15.71:0/2603056589 >> 172.21.15.71:0/2603056589 conn(0x7fdc980b1950 msgr2=0x7fdc980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:30.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.833+0000 7fdca0200640 1 -- 172.21.15.71:0/2603056589 shutdown_connections 2024-09-16T07:29:30.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.833+0000 7fdca0200640 1 -- 172.21.15.71:0/2603056589 wait complete. 2024-09-16T07:29:30.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.835+0000 7fdca0200640 1 Processor -- start 2024-09-16T07:29:30.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.835+0000 7fdca0200640 1 -- start start 2024-09-16T07:29:30.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdca0200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc80004800 0x7fdc98154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:30.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdca0200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc98155a10 0x7fdc9815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:30.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdca0200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc980b0990 0x7fdc980b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:30.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdca0200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdc980aceb0 con 0x7fdc980b0990 2024-09-16T07:29:30.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdca0200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdc980acd30 con 0x7fdc98155a10 2024-09-16T07:29:30.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdca0200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdc980ad030 con 0x7fdc80004800 2024-09-16T07:29:30.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdc9e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc980b0990 0x7fdc980b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:30.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdc9e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc980b0990 0x7fdc980b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46990/0 (socket says 172.21.15.71:46990) 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdc9e200640 1 -- 172.21.15.71:0/2388084591 learned_addr learned my addr 172.21.15.71:0/2388084591 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdc9d800640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc80004800 0x7fdc98154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.836+0000 7fdc9ce00640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc98155a10 0x7fdc9815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc9e200640 1 -- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc80004800 msgr2=0x7fdc98154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc9e200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc80004800 0x7fdc98154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc9e200640 1 -- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc98155a10 msgr2=0x7fdc9815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc9e200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc98155a10 0x7fdc9815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc9e200640 1 -- 172.21.15.71:0/2388084591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9815a8b0 con 0x7fdc980b0990 2024-09-16T07:29:30.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc9e200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc980b0990 0x7fdc980b0e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fdc9806a510 tx=0x7fdc90006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:30.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc95600640 1 -- 172.21.15.71:0/2388084591 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000fa80 con 0x7fdc980b0990 2024-09-16T07:29:30.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc95600640 1 -- 172.21.15.71:0/2388084591 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc90018070 con 0x7fdc980b0990 2024-09-16T07:29:30.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9815ab90 con 0x7fdc980b0990 2024-09-16T07:29:30.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdc95600640 1 -- 172.21.15.71:0/2388084591 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90011430 con 0x7fdc980b0990 2024-09-16T07:29:30.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.837+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9815b150 con 0x7fdc980b0990 2024-09-16T07:29:30.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.838+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc980b1350 con 0x7fdc980b0990 2024-09-16T07:29:30.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.839+0000 7fdc95600640 1 -- 172.21.15.71:0/2388084591 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fdc900115d0 con 0x7fdc980b0990 2024-09-16T07:29:30.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.841+0000 7fdc95600640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc78079450 0x7fdc7807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:30.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.841+0000 7fdc95600640 1 -- 172.21.15.71:0/2388084591 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5823+0+0 (secure 0 0 0) 0x7fdc90099c70 con 0x7fdc980b0990 2024-09-16T07:29:30.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.841+0000 7fdc9d800640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc78079450 0x7fdc7807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:30.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.842+0000 7fdc9d800640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc78079450 0x7fdc7807b910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009c50 tx=0x7fdc8c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:30.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:30.845+0000 7fdc95600640 1 -- 172.21.15.71:0/2388084591 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdc90063ad0 con 0x7fdc980b0990 2024-09-16T07:29:31.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.065+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc9809b650 con 0x7fdc980b0990 2024-09-16T07:29:31.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.066+0000 7fdc95600640 1 -- 172.21.15.71:0/2388084591 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5229 (secure 0 0 0) 0x7fdc900633d0 con 0x7fdc980b0990 2024-09-16T07:29:31.066 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:31.066 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"btime":"2024-09-16T07:28:01:502092+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1090989192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1090989192},{"type":"v1","addr":"172.21.15.177:6836","nonce":1090989192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2044137966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2044137966},{"type":"v1","addr":"172.21.15.177:6837","nonce":2044137966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3911470669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3911470669},{"type":"v1","addr":"172.21.15.71:6835","nonce":3911470669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:27:47.864418+0000","modified":"2024-09-16T07:27:50.881337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6834/3242207389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3242207389},{"type":"v1","addr":"172.21.15.71:6834","nonce":3242207389}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5964,"qdb_cluster":[5964]},"id":4}]} 2024-09-16T07:29:31.066 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2024-09-16T07:29:31.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc78079450 msgr2=0x7fdc7807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc78079450 0x7fdc7807b910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009c50 tx=0x7fdc8c005f90 comp rx=0 tx=0).stop 2024-09-16T07:29:31.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc980b0990 msgr2=0x7fdc980b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc980b0990 0x7fdc980b0e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fdc9806a510 tx=0x7fdc90006660 comp rx=0 tx=0).stop 2024-09-16T07:29:31.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 shutdown_connections 2024-09-16T07:29:31.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc78079450 0x7fdc7807b910 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc980b0990 0x7fdc980b0e10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc98155a10 0x7fdc9815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 --2- 172.21.15.71:0/2388084591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc80004800 0x7fdc98154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.068+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 >> 172.21.15.71:0/2388084591 conn(0x7fdc980b1950 msgr2=0x7fdc98073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:31.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.069+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 shutdown_connections 2024-09-16T07:29:31.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.069+0000 7fdca0200640 1 -- 172.21.15.71:0/2388084591 wait complete. 2024-09-16T07:29:31.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:29:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 -- 172.21.15.71:0/4063939373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92541612d0 msgr2=0x7f92540b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 --2- 172.21.15.71:0/4063939373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92541612d0 0x7f92540b4e00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f92540c7390 tx=0x7f924402a560 comp rx=0 tx=0).stop 2024-09-16T07:29:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 -- 172.21.15.71:0/4063939373 shutdown_connections 2024-09-16T07:29:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 --2- 172.21.15.71:0/4063939373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92541616b0 0x7f92540d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 --2- 172.21.15.71:0/4063939373 >> v2:172.21.15.177:3300/0 conn(0x7f92540d4980 0x7f92540d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 --2- 172.21.15.71:0/4063939373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92541612d0 0x7f92540b4e00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 -- 172.21.15.71:0/4063939373 >> 172.21.15.71:0/4063939373 conn(0x7f92540d3df0 msgr2=0x7f92540d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 -- 172.21.15.71:0/4063939373 shutdown_connections 2024-09-16T07:29:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.242+0000 7f9258a00640 1 -- 172.21.15.71:0/4063939373 wait complete. 2024-09-16T07:29:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.244+0000 7f9258a00640 1 Processor -- start 2024-09-16T07:29:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.244+0000 7f9258a00640 1 -- start start 2024-09-16T07:29:31.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9258a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92540d4980 0x7f92540c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9258a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92541612d0 0x7f92540b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9258a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92541616b0 0x7f9254157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9258a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f92540b3a20 con 0x7f92540d4980 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9258a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92540b38a0 con 0x7f92541612d0 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9258a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f92540b3ba0 con 0x7f92541616b0 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92540d4980 0x7f92540c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92540d4980 0x7f92540c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47010/0 (socket says 172.21.15.71:47010) 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253400640 1 -- 172.21.15.71:0/2635093782 learned_addr learned my addr 172.21.15.71:0/2635093782 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253e00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92541616b0 0x7f9254157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f924be00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92541612d0 0x7f92540b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253400640 1 -- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92541616b0 msgr2=0x7f9254157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253400640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92541616b0 0x7f9254157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253400640 1 -- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92541612d0 msgr2=0x7f92540b8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253400640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92541612d0 0x7f92540b8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.245+0000 7f9253400640 1 -- 172.21.15.71:0/2635093782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92540b9660 con 0x7f92540d4980 2024-09-16T07:29:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.246+0000 7f9253400640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92540d4980 0x7f92540c9510 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9244009850 tx=0x7f9244003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.246+0000 7f924b400640 1 -- 172.21.15.71:0/2635093782 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924403f070 con 0x7f92540d4980 2024-09-16T07:29:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.246+0000 7f924b400640 1 -- 172.21.15.71:0/2635093782 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92440047a0 con 0x7f92540d4980 2024-09-16T07:29:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.246+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92541581e0 con 0x7f92540d4980 2024-09-16T07:29:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.246+0000 7f924b400640 1 -- 172.21.15.71:0/2635093782 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244035030 con 0x7f92540d4980 2024-09-16T07:29:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.246+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92541f8a20 con 0x7f92540d4980 2024-09-16T07:29:31.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.247+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9254157fd0 con 0x7f92540d4980 2024-09-16T07:29:31.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.248+0000 7f924b400640 1 -- 172.21.15.71:0/2635093782 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f9244042070 con 0x7f92540d4980 2024-09-16T07:29:31.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.250+0000 7f924b400640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9230079420 0x7f923007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:31.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.250+0000 7f924b400640 1 -- 172.21.15.71:0/2635093782 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5823+0+0 (secure 0 0 0) 0x7f92440ba420 con 0x7f92540d4980 2024-09-16T07:29:31.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.250+0000 7f924be00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9230079420 0x7f923007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:31.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.251+0000 7f924be00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9230079420 0x7f923007b8e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92540c9270 tx=0x7f923c00b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:31.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.254+0000 7f924b400640 1 -- 172.21.15.71:0/2635093782 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9244083c30 con 0x7f92540d4980 2024-09-16T07:29:31.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.441+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92540b4100 con 0x7f92540d4980 2024-09-16T07:29:31.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.442+0000 7f924b400640 1 -- 172.21.15.71:0/2635093782 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16940 (secure 0 0 0) 0x7f9244088660 con 0x7f92540d4980 2024-09-16T07:29:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:31.442 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":65,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:29:29.361529+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:27:45.670244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T07:27:46.828046+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":6,"to":4}]},{"pgid":"9.1c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:29:31.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9230079420 msgr2=0x7f923007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9230079420 0x7f923007b8e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f92540c9270 tx=0x7f923c00b2e0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92540d4980 msgr2=0x7f92540c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92540d4980 0x7f92540c9510 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9244009850 tx=0x7f9244003910 comp rx=0 tx=0).stop 2024-09-16T07:29:31.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 shutdown_connections 2024-09-16T07:29:31.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9230079420 0x7f923007b8e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92541616b0 0x7f9254157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92541612d0 0x7f92540b8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 --2- 172.21.15.71:0/2635093782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92540d4980 0x7f92540c9510 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.444+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 >> 172.21.15.71:0/2635093782 conn(0x7f92540d3df0 msgr2=0x7f92540cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.445+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 shutdown_connections 2024-09-16T07:29:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.445+0000 7f9258a00640 1 -- 172.21.15.71:0/2635093782 wait complete. 2024-09-16T07:29:31.461 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:29:31.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:29:31.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.619+0000 7f7c5ce00640 1 -- 172.21.15.71:0/953067072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c581445d0 msgr2=0x7f7c580cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.619+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/953067072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c581445d0 0x7f7c580cebf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c48005fd0 tx=0x7f7c4802ab30 comp rx=0 tx=0).stop 2024-09-16T07:29:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.619+0000 7f7c5ce00640 1 -- 172.21.15.71:0/953067072 shutdown_connections 2024-09-16T07:29:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.620+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/953067072 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c580ccfd0 0x7f7c580cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.620+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/953067072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c581445d0 0x7f7c580cebf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.620+0000 7f7c5ce00640 1 -- 172.21.15.71:0/953067072 >> 172.21.15.71:0/953067072 conn(0x7f7c580b1950 msgr2=0x7f7c580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.620+0000 7f7c5ce00640 1 -- 172.21.15.71:0/953067072 shutdown_connections 2024-09-16T07:29:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.620+0000 7f7c5ce00640 1 -- 172.21.15.71:0/953067072 wait complete. 2024-09-16T07:29:31.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.621+0000 7f7c5ce00640 1 Processor -- start 2024-09-16T07:29:31.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.621+0000 7f7c5ce00640 1 -- start start 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.622+0000 7f7c5ce00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c580b4ca0 0x7f7c5809bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.622+0000 7f7c5ce00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c580ccfd0 0x7f7c581f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.622+0000 7f7c5ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c581445d0 0x7f7c580cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.622+0000 7f7c5ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5815f080 con 0x7f7c581445d0 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.622+0000 7f7c5ce00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58073bc0 con 0x7f7c580ccfd0 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.622+0000 7f7c5ce00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5815f200 con 0x7f7c580b4ca0 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.622+0000 7f7c56800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c581445d0 0x7f7c580cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c56800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c581445d0 0x7f7c580cb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47016/0 (socket says 172.21.15.71:47016) 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c56800640 1 -- 172.21.15.71:0/502533336 learned_addr learned my addr 172.21.15.71:0/502533336 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c55e00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c580b4ca0 0x7f7c5809bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:31.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c55400640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c580ccfd0 0x7f7c581f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:31.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c56800640 1 -- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c580b4ca0 msgr2=0x7f7c5809bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c56800640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c580b4ca0 0x7f7c5809bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c56800640 1 -- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c580ccfd0 msgr2=0x7f7c581f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c56800640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c580ccfd0 0x7f7c581f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c56800640 1 -- 172.21.15.71:0/502533336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c580c9220 con 0x7f7c581445d0 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c56800640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c581445d0 0x7f7c580cb140 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c430 tx=0x7f7c5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.623+0000 7f7c4e000640 1 -- 172.21.15.71:0/502533336 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50017070 con 0x7f7c581445d0 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.624+0000 7f7c4e000640 1 -- 172.21.15.71:0/502533336 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c500053c0 con 0x7f7c581445d0 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.624+0000 7f7c4e000640 1 -- 172.21.15.71:0/502533336 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50003850 con 0x7f7c581445d0 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.624+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c580c9500 con 0x7f7c581445d0 2024-09-16T07:29:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.624+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c580c9ac0 con 0x7f7c581445d0 2024-09-16T07:29:31.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.625+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5815f6b0 con 0x7f7c581445d0 2024-09-16T07:29:31.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.626+0000 7f7c4e000640 1 -- 172.21.15.71:0/502533336 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f7c5001a070 con 0x7f7c581445d0 2024-09-16T07:29:31.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.627+0000 7f7c4e000640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7c38079420 0x7f7c3807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:31.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.627+0000 7f7c4e000640 1 -- 172.21.15.71:0/502533336 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5823+0+0 (secure 0 0 0) 0x7f7c500988b0 con 0x7f7c581445d0 2024-09-16T07:29:31.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.628+0000 7f7c55e00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7c38079420 0x7f7c3807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:31.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.628+0000 7f7c55e00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7c38079420 0x7f7c3807b8e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7c48035000 tx=0x7f7c48038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:31.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.632+0000 7f7c4e000640 1 -- 172.21.15.71:0/502533336 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7c50062080 con 0x7f7c581445d0 2024-09-16T07:29:31.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.853+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c58070a20 con 0x7f7c581445d0 2024-09-16T07:29:31.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.854+0000 7f7c4e000640 1 -- 172.21.15.71:0/502533336 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5229 (secure 0 0 0) 0x7f7c50066ab0 con 0x7f7c581445d0 2024-09-16T07:29:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:31.854 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"btime":"2024-09-16T07:28:01:502092+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1090989192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1090989192},{"type":"v1","addr":"172.21.15.177:6836","nonce":1090989192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2044137966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2044137966},{"type":"v1","addr":"172.21.15.177:6837","nonce":2044137966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3911470669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3911470669},{"type":"v1","addr":"172.21.15.71:6835","nonce":3911470669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:27:47.864418+0000","modified":"2024-09-16T07:27:50.881337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6834/3242207389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3242207389},{"type":"v1","addr":"172.21.15.71:6834","nonce":3242207389}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5964,"qdb_cluster":[5964]},"id":4}]} 2024-09-16T07:29:31.854 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2024-09-16T07:29:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.856+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7c38079420 msgr2=0x7f7c3807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.856+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7c38079420 0x7f7c3807b8e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7c48035000 tx=0x7f7c48038040 comp rx=0 tx=0).stop 2024-09-16T07:29:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.856+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c581445d0 msgr2=0x7f7c580cb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.856+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c581445d0 0x7f7c580cb140 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000c430 tx=0x7f7c5000c8f0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.856+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 shutdown_connections 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.856+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7c38079420 0x7f7c3807b8e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.856+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c581445d0 0x7f7c580cb140 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.857+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c580ccfd0 0x7f7c581f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.857+0000 7f7c5ce00640 1 --2- 172.21.15.71:0/502533336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c580b4ca0 0x7f7c5809bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.857+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 >> 172.21.15.71:0/502533336 conn(0x7f7c580b1950 msgr2=0x7f7c580cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.857+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 shutdown_connections 2024-09-16T07:29:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:31.857+0000 7f7c5ce00640 1 -- 172.21.15.71:0/502533336 wait complete. 2024-09-16T07:29:31.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:29:32.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.028+0000 7f3687200640 1 -- 172.21.15.71:0/370015574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36880d0540 msgr2=0x7f3688074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.028+0000 7f3687200640 1 --2- 172.21.15.71:0/370015574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36880d0540 0x7f3688074280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f36880cd570 tx=0x7f368002aa10 comp rx=0 tx=0).stop 2024-09-16T07:29:32.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.028+0000 7f3687200640 1 -- 172.21.15.71:0/370015574 shutdown_connections 2024-09-16T07:29:32.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.028+0000 7f3687200640 1 --2- 172.21.15.71:0/370015574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36880d0540 0x7f3688074280 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.028+0000 7f3687200640 1 --2- 172.21.15.71:0/370015574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36880725d0 0x7f3688073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.028+0000 7f3687200640 1 --2- 172.21.15.71:0/370015574 >> v2:172.21.15.177:3300/0 conn(0x7f368815bc30 0x7f36880b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.028+0000 7f3687200640 1 -- 172.21.15.71:0/370015574 >> 172.21.15.71:0/370015574 conn(0x7f36880cfa20 msgr2=0x7f36880cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:32.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.028+0000 7f3687200640 1 -- 172.21.15.71:0/370015574 shutdown_connections 2024-09-16T07:29:32.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.029+0000 7f3687200640 1 -- 172.21.15.71:0/370015574 wait complete. 2024-09-16T07:29:32.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.030+0000 7f3687200640 1 Processor -- start 2024-09-16T07:29:32.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.030+0000 7f3687200640 1 -- start start 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3687200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36880d0540 0x7f36880c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3687200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f368815bc30 0x7f3688151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3687200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881f3000 0x7f36880c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3687200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f368806f320 con 0x7f36881f3000 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3687200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f368806f4a0 con 0x7f36880d0540 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3687200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f368806f620 con 0x7f368815bc30 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3686800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881f3000 0x7f36880c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3686800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881f3000 0x7f36880c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47036/0 (socket says 172.21.15.71:47036) 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3686800640 1 -- 172.21.15.71:0/3217444873 learned_addr learned my addr 172.21.15.71:0/3217444873 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3685400640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f368815bc30 0x7f3688151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.031+0000 7f3685e00640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36880d0540 0x7f36880c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3686800640 1 -- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f368815bc30 msgr2=0x7f3688151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3686800640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f368815bc30 0x7f3688151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3686800640 1 -- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36880d0540 msgr2=0x7f36880c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3686800640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36880d0540 0x7f36880c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3686800640 1 -- 172.21.15.71:0/3217444873 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36881f3490 con 0x7f36881f3000 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3685e00640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36880d0540 0x7f36880c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3686800640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881f3000 0x7f36880c7c70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3680009d60 tx=0x7f3680003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f367e000640 1 -- 172.21.15.71:0/3217444873 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680047070 con 0x7f36881f3000 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f367e000640 1 -- 172.21.15.71:0/3217444873 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3680004250 con 0x7f36881f3000 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f367e000640 1 -- 172.21.15.71:0/3217444873 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680033050 con 0x7f36881f3000 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3688153600 con 0x7f36881f3000 2024-09-16T07:29:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.032+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3688151d50 con 0x7f36881f3000 2024-09-16T07:29:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.034+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3688153dd0 con 0x7f36881f3000 2024-09-16T07:29:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.035+0000 7f367e000640 1 -- 172.21.15.71:0/3217444873 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f368004a070 con 0x7f36881f3000 2024-09-16T07:29:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.035+0000 7f367e000640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3668079420 0x7f366807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.036+0000 7f367e000640 1 -- 172.21.15.71:0/3217444873 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5823+0+0 (secure 0 0 0) 0x7f36800ba4e0 con 0x7f36881f3000 2024-09-16T07:29:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.036+0000 7f3685e00640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3668079420 0x7f366807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.036+0000 7f3685e00640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3668079420 0x7f366807b8e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f368815c740 tx=0x7f3678005f10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.040+0000 7f367e000640 1 -- 172.21.15.71:0/3217444873 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3680083f50 con 0x7f36881f3000 2024-09-16T07:29:32.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.262+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36880b28f0 con 0x7f36881f3000 2024-09-16T07:29:32.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.262+0000 7f367e000640 1 -- 172.21.15.71:0/3217444873 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5229 (secure 0 0 0) 0x7f3680088980 con 0x7f36881f3000 2024-09-16T07:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:32.262 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"btime":"2024-09-16T07:28:01:502092+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1090989192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1090989192},{"type":"v1","addr":"172.21.15.177:6836","nonce":1090989192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2044137966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2044137966},{"type":"v1","addr":"172.21.15.177:6837","nonce":2044137966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3911470669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3911470669},{"type":"v1","addr":"172.21.15.71:6835","nonce":3911470669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:27:47.864418+0000","modified":"2024-09-16T07:27:50.881337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6834/3242207389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":3242207389},{"type":"v1","addr":"172.21.15.71:6834","nonce":3242207389}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5964,"qdb_cluster":[5964]},"id":4}]} 2024-09-16T07:29:32.262 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3668079420 msgr2=0x7f366807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3668079420 0x7f366807b8e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f368815c740 tx=0x7f3678005f10 comp rx=0 tx=0).stop 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881f3000 msgr2=0x7f36880c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881f3000 0x7f36880c7c70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3680009d60 tx=0x7f3680003f10 comp rx=0 tx=0).stop 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 shutdown_connections 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3668079420 0x7f366807b8e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36881f3000 0x7f36880c7c70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f368815bc30 0x7f3688151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 --2- 172.21.15.71:0/3217444873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36880d0540 0x7f36880c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 >> 172.21.15.71:0/3217444873 conn(0x7f36880cfa20 msgr2=0x7f36880ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 shutdown_connections 2024-09-16T07:29:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.265+0000 7f3687200640 1 -- 172.21.15.71:0/3217444873 wait complete. 2024-09-16T07:29:32.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:29:32.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.439+0000 7f9fc7000640 1 -- 172.21.15.71:0/3010741244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa80052e0 msgr2=0x7f9fa800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.439+0000 7f9fc7000640 1 --2- 172.21.15.71:0/3010741244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa80052e0 0x7f9fa800b9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8001eb0 tx=0x7f9fa8005830 comp rx=0 tx=0).stop 2024-09-16T07:29:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.440+0000 7f9fc7000640 1 -- 172.21.15.71:0/3010741244 shutdown_connections 2024-09-16T07:29:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.440+0000 7f9fc7000640 1 --2- 172.21.15.71:0/3010741244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa80052e0 0x7f9fa800b9c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.440+0000 7f9fc7000640 1 -- 172.21.15.71:0/3010741244 >> 172.21.15.71:0/3010741244 conn(0x7f9fc00cfa20 msgr2=0x7f9fc00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.440+0000 7f9fc7000640 1 -- 172.21.15.71:0/3010741244 shutdown_connections 2024-09-16T07:29:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.440+0000 7f9fc7000640 1 -- 172.21.15.71:0/3010741244 wait complete. 2024-09-16T07:29:32.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.441+0000 7f9fc7000640 1 Processor -- start 2024-09-16T07:29:32.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.441+0000 7f9fc7000640 1 -- start start 2024-09-16T07:29:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.442+0000 7f9fc7000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa80052e0 0x7f9fc014df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.442+0000 7f9fc7000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc014e460 0x7f9fc00c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.442+0000 7f9fc7000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fc00c9310 0x7f9fc01f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.442+0000 7f9fc7000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00b2c80 con 0x7f9fc00c9310 2024-09-16T07:29:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.442+0000 7f9fc7000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc00b2b00 con 0x7f9fc014e460 2024-09-16T07:29:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.442+0000 7f9fc7000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc0073d40 con 0x7f9fa80052e0 2024-09-16T07:29:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fc00c9310 0x7f9fc01f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fc00c9310 0x7f9fc01f79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47054/0 (socket says 172.21.15.71:47054) 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 -- 172.21.15.71:0/3021158619 learned_addr learned my addr 172.21.15.71:0/3021158619 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc5200640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc014e460 0x7f9fc00c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc5c00640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa80052e0 0x7f9fc014df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 -- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa80052e0 msgr2=0x7f9fc014df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa80052e0 0x7f9fc014df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 -- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc014e460 msgr2=0x7f9fc00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc014e460 0x7f9fc00c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 -- 172.21.15.71:0/3021158619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc01f43f0 con 0x7f9fc00c9310 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc5c00640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa80052e0 0x7f9fc014df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fc6600640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fc00c9310 0x7f9fc01f79e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800c3d0 tx=0x7f9fb8006650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fbd600640 1 -- 172.21.15.71:0/3021158619 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8031050 con 0x7f9fc00c9310 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.443+0000 7f9fbd600640 1 -- 172.21.15.71:0/3021158619 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fb802c680 con 0x7f9fc00c9310 2024-09-16T07:29:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.444+0000 7f9fbd600640 1 -- 172.21.15.71:0/3021158619 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb802cab0 con 0x7f9fc00c9310 2024-09-16T07:29:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.444+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc01f86d0 con 0x7f9fc00c9310 2024-09-16T07:29:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.444+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc01f4950 con 0x7f9fc00c9310 2024-09-16T07:29:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.445+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc00c9780 con 0x7f9fc00c9310 2024-09-16T07:29:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.446+0000 7f9fbd600640 1 -- 172.21.15.71:0/3021158619 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f9fb8034070 con 0x7f9fc00c9310 2024-09-16T07:29:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.447+0000 7f9fbd600640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9fa0079420 0x7f9fa007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.447+0000 7f9fbd600640 1 -- 172.21.15.71:0/3021158619 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5823+0+0 (secure 0 0 0) 0x7f9fb80b8b90 con 0x7f9fc00c9310 2024-09-16T07:29:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.447+0000 7f9fc5c00640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9fa0079420 0x7f9fa007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.448+0000 7f9fc5c00640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9fa0079420 0x7f9fa007b8e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000f80 tx=0x7f9fb4006ef0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:32.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.451+0000 7f9fbd600640 1 -- 172.21.15.71:0/3021158619 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9fc00c9780 con 0x7f9fc00c9310 2024-09-16T07:29:32.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.644+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fc00742a0 con 0x7f9fc00c9310 2024-09-16T07:29:32.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.645+0000 7f9fbd600640 1 -- 172.21.15.71:0/3021158619 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16940 (secure 0 0 0) 0x7f9fb8045030 con 0x7f9fc00c9310 2024-09-16T07:29:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:32.645 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":65,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:29:29.361529+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:27:45.670244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T07:27:46.828046+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":6,"to":4}]},{"pgid":"9.1c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:29:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.647+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9fa0079420 msgr2=0x7f9fa007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.647+0000 7f9fc7000640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9fa0079420 0x7f9fa007b8e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000f80 tx=0x7f9fb4006ef0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.647+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fc00c9310 msgr2=0x7f9fc01f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.647+0000 7f9fc7000640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fc00c9310 0x7f9fc01f79e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800c3d0 tx=0x7f9fb8006650 comp rx=0 tx=0).stop 2024-09-16T07:29:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.648+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 shutdown_connections 2024-09-16T07:29:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.648+0000 7f9fc7000640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9fa0079420 0x7f9fa007b8e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.648+0000 7f9fc7000640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fc00c9310 0x7f9fc01f79e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.648+0000 7f9fc7000640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc014e460 0x7f9fc00c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.648+0000 7f9fc7000640 1 --2- 172.21.15.71:0/3021158619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa80052e0 0x7f9fc014df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.648+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 >> 172.21.15.71:0/3021158619 conn(0x7f9fc00cfa20 msgr2=0x7f9fc00b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.648+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 shutdown_connections 2024-09-16T07:29:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.648+0000 7f9fc7000640 1 -- 172.21.15.71:0/3021158619 wait complete. 2024-09-16T07:29:32.663 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:29:32.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.822+0000 7f555f200640 1 -- 172.21.15.71:0/3395052520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55581612d0 msgr2=0x7f55580b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.822+0000 7f555f200640 1 --2- 172.21.15.71:0/3395052520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55581612d0 0x7f55580b4e00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f554c009830 tx=0x7f554c02a490 comp rx=0 tx=0).stop 2024-09-16T07:29:32.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.822+0000 7f555f200640 1 -- 172.21.15.71:0/3395052520 shutdown_connections 2024-09-16T07:29:32.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.822+0000 7f555f200640 1 --2- 172.21.15.71:0/3395052520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55580d4980 0x7f55580d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.822+0000 7f555f200640 1 --2- 172.21.15.71:0/3395052520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55581612d0 0x7f55580b4e00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.822+0000 7f555f200640 1 -- 172.21.15.71:0/3395052520 >> 172.21.15.71:0/3395052520 conn(0x7f55580d3df0 msgr2=0x7f55580d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:32.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.822+0000 7f555f200640 1 -- 172.21.15.71:0/3395052520 shutdown_connections 2024-09-16T07:29:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.822+0000 7f555f200640 1 -- 172.21.15.71:0/3395052520 wait complete. 2024-09-16T07:29:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.824+0000 7f555f200640 1 Processor -- start 2024-09-16T07:29:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.824+0000 7f555f200640 1 -- start start 2024-09-16T07:29:32.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55580d4980 0x7f55581f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555f200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55581612d0 0x7f55581f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55581616b0 0x7f55580b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55580b1c00 con 0x7f55580d4980 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55580b1a80 con 0x7f55581616b0 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555f200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55580b1d80 con 0x7f55581612d0 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55580d4980 0x7f55581f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55580d4980 0x7f55581f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47070/0 (socket says 172.21.15.71:47070) 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555de00640 1 -- 172.21.15.71:0/4100791096 learned_addr learned my addr 172.21.15.71:0/4100791096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555e800640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55581616b0 0x7f55580b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.825+0000 7f555d400640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55581612d0 0x7f55581f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f555de00640 1 -- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55581612d0 msgr2=0x7f55581f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f555de00640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55581612d0 0x7f55581f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f555de00640 1 -- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55581616b0 msgr2=0x7f55580b8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f555de00640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55581616b0 0x7f55580b8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f555de00640 1 -- 172.21.15.71:0/4100791096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55580c7a90 con 0x7f55580d4980 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f555de00640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55580d4980 0x7f55581f2070 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f554c000c00 tx=0x7f554c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f5552000640 1 -- 172.21.15.71:0/4100791096 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c040070 con 0x7f55580d4980 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f5552000640 1 -- 172.21.15.71:0/4100791096 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f554c0038e0 con 0x7f55580d4980 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f5552000640 1 -- 172.21.15.71:0/4100791096 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c005c90 con 0x7f55580d4980 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55580c7d70 con 0x7f55580d4980 2024-09-16T07:29:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.826+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55580c8330 con 0x7f55580d4980 2024-09-16T07:29:32.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.827+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55580b4e00 con 0x7f55580d4980 2024-09-16T07:29:32.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.828+0000 7f5552000640 1 -- 172.21.15.71:0/4100791096 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f554c039040 con 0x7f55580d4980 2024-09-16T07:29:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.829+0000 7f5552000640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f553c079420 0x7f553c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.830+0000 7f5552000640 1 -- 172.21.15.71:0/4100791096 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5823+0+0 (secure 0 0 0) 0x7f554c0ba8b0 con 0x7f55580d4980 2024-09-16T07:29:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.830+0000 7f555d400640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f553c079420 0x7f553c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.831+0000 7f555d400640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f553c079420 0x7f553c07b8e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f555808c500 tx=0x7f5548005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:32.833+0000 7f5552000640 1 -- 172.21.15.71:0/4100791096 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f554c0840c0 con 0x7f55580d4980 2024-09-16T07:29:33.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.055+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f55580d1310 con 0x7f55580d4980 2024-09-16T07:29:33.055 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:29:33.056+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:29:33.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.058+0000 7f5552000640 1 -- 172.21.15.71:0/4100791096 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7f554c088af0 con 0x7f55580d4980 2024-09-16T07:29:33.058 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f553c079420 msgr2=0x7f553c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f553c079420 0x7f553c07b8e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f555808c500 tx=0x7f5548005f90 comp rx=0 tx=0).stop 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55580d4980 msgr2=0x7f55581f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55580d4980 0x7f55581f2070 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f554c000c00 tx=0x7f554c032c10 comp rx=0 tx=0).stop 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 shutdown_connections 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f553c079420 0x7f553c07b8e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55581616b0 0x7f55580b8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55581612d0 0x7f55581f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 --2- 172.21.15.71:0/4100791096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55580d4980 0x7f55581f2070 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 >> 172.21.15.71:0/4100791096 conn(0x7f55580d3df0 msgr2=0x7f55580ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:33.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 shutdown_connections 2024-09-16T07:29:33.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.062+0000 7f555f200640 1 -- 172.21.15.71:0/4100791096 wait complete. 2024-09-16T07:29:33.079 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:29:33.129 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:29:33.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.241+0000 7fa6ce000640 1 -- 172.21.15.71:0/2703994294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c806efa0 msgr2=0x7fa6c80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:33.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.241+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2703994294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c806efa0 0x7fa6c80cdbb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005fd0 tx=0x7fa6c002aa10 comp rx=0 tx=0).stop 2024-09-16T07:29:33.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.241+0000 7fa6ce000640 1 -- 172.21.15.71:0/2703994294 shutdown_connections 2024-09-16T07:29:33.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.241+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2703994294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c806efa0 0x7fa6c80cdbb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:33.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.241+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2703994294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c80cccd0 0x7fa6c8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:33.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.241+0000 7fa6ce000640 1 -- 172.21.15.71:0/2703994294 >> 172.21.15.71:0/2703994294 conn(0x7fa6c80b1840 msgr2=0x7fa6c80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:33.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.241+0000 7fa6ce000640 1 -- 172.21.15.71:0/2703994294 shutdown_connections 2024-09-16T07:29:33.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.241+0000 7fa6ce000640 1 -- 172.21.15.71:0/2703994294 wait complete. 2024-09-16T07:29:33.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.243+0000 7fa6ce000640 1 Processor -- start 2024-09-16T07:29:33.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.243+0000 7fa6ce000640 1 -- start start 2024-09-16T07:29:33.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c806efa0 0x7fa6c81f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:33.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c80cccd0 0x7fa6c8141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:33.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6ce000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c8145750 0x7fa6c8140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:33.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6ce000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c80ce480 con 0x7fa6c8145750 2024-09-16T07:29:33.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6ce000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c80ce300 con 0x7fa6c80cccd0 2024-09-16T07:29:33.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6ce000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c80ce600 con 0x7fa6c806efa0 2024-09-16T07:29:33.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c8145750 0x7fa6c8140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:33.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c8145750 0x7fa6c8140b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47118/0 (socket says 172.21.15.71:47118) 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7e00640 1 -- 172.21.15.71:0/2431400944 learned_addr learned my addr 172.21.15.71:0/2431400944 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c6a00640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c80cccd0 0x7fa6c8141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7400640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c806efa0 0x7fa6c81f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7e00640 1 -- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c806efa0 msgr2=0x7fa6c81f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7e00640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c806efa0 0x7fa6c81f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7e00640 1 -- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c80cccd0 msgr2=0x7fa6c8141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7e00640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c80cccd0 0x7fa6c8141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7e00640 1 -- 172.21.15.71:0/2431400944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c8141d90 con 0x7fa6c8145750 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c7400640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c806efa0 0x7fa6c81f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.244+0000 7fa6c6a00640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c80cccd0 0x7fa6c8141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:33.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.245+0000 7fa6c7e00640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c8145750 0x7fa6c8140b30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0006010 tx=0x7fa6c0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:33.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.245+0000 7fa6bf400640 1 -- 172.21.15.71:0/2431400944 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0047070 con 0x7fa6c8145750 2024-09-16T07:29:33.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.245+0000 7fa6bf400640 1 -- 172.21.15.71:0/2431400944 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6c0004250 con 0x7fa6c8145750 2024-09-16T07:29:33.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.245+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c8142c90 con 0x7fa6c8145750 2024-09-16T07:29:33.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.245+0000 7fa6bf400640 1 -- 172.21.15.71:0/2431400944 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0033050 con 0x7fa6c8145750 2024-09-16T07:29:33.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.245+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c81f4780 con 0x7fa6c8145750 2024-09-16T07:29:33.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.246+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c80cdb90 con 0x7fa6c8145750 2024-09-16T07:29:33.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.247+0000 7fa6bf400640 1 -- 172.21.15.71:0/2431400944 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fa6c004a070 con 0x7fa6c8145750 2024-09-16T07:29:33.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.248+0000 7fa6bf400640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6a8079450 0x7fa6a807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:33.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.248+0000 7fa6bf400640 1 -- 172.21.15.71:0/2431400944 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5881+0+0 (secure 0 0 0) 0x7fa6c00ba4a0 con 0x7fa6c8145750 2024-09-16T07:29:33.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.249+0000 7fa6c7400640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6a8079450 0x7fa6a807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:33.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.249+0000 7fa6c7400640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6a8079450 0x7fa6a807b910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8000f60 tx=0x7fa6b8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:33.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.252+0000 7fa6bf400640 1 -- 172.21.15.71:0/2431400944 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa6c0084560 con 0x7fa6c8145750 2024-09-16T07:29:33.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:33.474+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa6c8144cb0 con 0x7fa6c8145750 2024-09-16T07:29:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.060+0000 7fa6bf400640 1 -- 172.21.15.71:0/2431400944 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7fa6c0083e60 con 0x7fa6c8145750 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6a8079450 msgr2=0x7fa6a807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6a8079450 0x7fa6a807b910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8000f60 tx=0x7fa6b8009040 comp rx=0 tx=0).stop 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c8145750 msgr2=0x7fa6c8140b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c8145750 0x7fa6c8140b30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0006010 tx=0x7fa6c0003f10 comp rx=0 tx=0).stop 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 shutdown_connections 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa6a8079450 0x7fa6a807b910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c8145750 0x7fa6c8140b30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c80cccd0 0x7fa6c8141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 --2- 172.21.15.71:0/2431400944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c806efa0 0x7fa6c81f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 >> 172.21.15.71:0/2431400944 conn(0x7fa6c80b1840 msgr2=0x7fa6c80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:34.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 shutdown_connections 2024-09-16T07:29:34.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.063+0000 7fa6ce000640 1 -- 172.21.15.71:0/2431400944 wait complete. 2024-09-16T07:29:34.080 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:29:34.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.244+0000 7f3ea7200640 1 -- 172.21.15.71:0/3106751866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 msgr2=0x7f3e84003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:34.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.244+0000 7f3ea7200640 1 --2- 172.21.15.71:0/3106751866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 0x7f3e84003e70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e3d0 tx=0x7f3e84005830 comp rx=0 tx=0).stop 2024-09-16T07:29:34.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.244+0000 7f3ea5200640 1 -- 172.21.15.71:0/3106751866 reap_dead start 2024-09-16T07:29:34.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.244+0000 7f3ea7200640 1 -- 172.21.15.71:0/3106751866 shutdown_connections 2024-09-16T07:29:34.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.244+0000 7f3ea7200640 1 -- 172.21.15.71:0/3106751866 >> 172.21.15.71:0/3106751866 conn(0x7f3ea00b18e0 msgr2=0x7f3ea00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:34.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.244+0000 7f3ea7200640 1 -- 172.21.15.71:0/3106751866 shutdown_connections 2024-09-16T07:29:34.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.244+0000 7f3ea7200640 1 -- 172.21.15.71:0/3106751866 wait complete. 2024-09-16T07:29:34.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.245+0000 7f3ea7200640 1 Processor -- start 2024-09-16T07:29:34.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.245+0000 7f3ea7200640 1 -- start start 2024-09-16T07:29:34.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.246+0000 7f3ea7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 0x7f3ea015d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:34.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.246+0000 7f3ea7200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ea0154b90 0x7f3ea01543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.246+0000 7f3ea7200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ea009c420 0x7f3ea009c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.246+0000 7f3ea7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea00ada60 con 0x7f3e84003a50 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.246+0000 7f3ea7200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea00ad8e0 con 0x7f3ea009c420 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.246+0000 7f3ea7200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea00adbe0 con 0x7f3ea0154b90 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 0x7f3ea015d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 0x7f3ea015d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47130/0 (socket says 172.21.15.71:47130) 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 -- 172.21.15.71:0/2384729452 learned_addr learned my addr 172.21.15.71:0/2384729452 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9f400640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ea0154b90 0x7f3ea01543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3ea5200640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ea009c420 0x7f3ea009c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 -- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ea0154b90 msgr2=0x7f3ea01543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ea0154b90 0x7f3ea01543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 -- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ea009c420 msgr2=0x7f3ea009c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:34.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ea009c420 0x7f3ea009c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:34.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 -- 172.21.15.71:0/2384729452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01f6070 con 0x7f3e84003a50 2024-09-16T07:29:34.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3ea5200640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ea009c420 0x7f3ea009c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:34.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9f400640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ea0154b90 0x7f3ea01543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:34.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e9fe00640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 0x7f3ea015d390 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e4f0 tx=0x7f3e90009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:34.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.247+0000 7f3e97e00640 1 -- 172.21.15.71:0/2384729452 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90040070 con 0x7f3e84003a50 2024-09-16T07:29:34.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.248+0000 7f3e97e00640 1 -- 172.21.15.71:0/2384729452 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e90004d20 con 0x7f3e84003a50 2024-09-16T07:29:34.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.248+0000 7f3e97e00640 1 -- 172.21.15.71:0/2384729452 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9003c860 con 0x7f3e84003a50 2024-09-16T07:29:34.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.248+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea015d8d0 con 0x7f3e84003a50 2024-09-16T07:29:34.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.248+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea01f5590 con 0x7f3e84003a50 2024-09-16T07:29:34.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.249+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea0069360 con 0x7f3e84003a50 2024-09-16T07:29:34.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.250+0000 7f3e97e00640 1 -- 172.21.15.71:0/2384729452 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f3e900324c0 con 0x7f3e84003a50 2024-09-16T07:29:34.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.251+0000 7f3e97e00640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e80079450 0x7f3e8007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:34.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.251+0000 7f3e97e00640 1 -- 172.21.15.71:0/2384729452 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5881+0+0 (secure 0 0 0) 0x7f3e900bb0a0 con 0x7f3e84003a50 2024-09-16T07:29:34.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.251+0000 7f3e9f400640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e80079450 0x7f3e8007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:34.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.252+0000 7f3e9f400640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e80079450 0x7f3e8007b910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3ea01576a0 tx=0x7f3e8c006720 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:34.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.255+0000 7f3e97e00640 1 -- 172.21.15.71:0/2384729452 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3e90084f60 con 0x7f3e84003a50 2024-09-16T07:29:34.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:34.443+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ea006f300 con 0x7f3e84003a50 2024-09-16T07:29:35.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.392+0000 7f3e97e00640 1 -- 172.21.15.71:0/2384729452 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) ==== 181+0+0 (secure 0 0 0) 0x7f3e90084860 con 0x7f3e84003a50 2024-09-16T07:29:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:29:35.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.395+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e80079450 msgr2=0x7f3e8007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:35.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.395+0000 7f3ea7200640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e80079450 0x7f3e8007b910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3ea01576a0 tx=0x7f3e8c006720 comp rx=0 tx=0).stop 2024-09-16T07:29:35.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.395+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 msgr2=0x7f3ea015d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:35.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.395+0000 7f3ea7200640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 0x7f3ea015d390 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e4f0 tx=0x7f3e90009d00 comp rx=0 tx=0).stop 2024-09-16T07:29:35.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.396+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 shutdown_connections 2024-09-16T07:29:35.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.396+0000 7f3ea7200640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3e80079450 0x7f3e8007b910 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:35.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.396+0000 7f3ea7200640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ea009c420 0x7f3ea009c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:35.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.396+0000 7f3ea7200640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ea0154b90 0x7f3ea01543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:35.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.396+0000 7f3ea7200640 1 --2- 172.21.15.71:0/2384729452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e84003a50 0x7f3ea015d390 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:35.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.396+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 >> 172.21.15.71:0/2384729452 conn(0x7f3ea00b18e0 msgr2=0x7f3ea00af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:35.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.396+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 shutdown_connections 2024-09-16T07:29:35.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.396+0000 7f3ea7200640 1 -- 172.21.15.71:0/2384729452 wait complete. 2024-09-16T07:29:35.416 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:29:35.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.582+0000 7fad1fc00640 1 -- 172.21.15.71:0/1851484694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facfc003a00 msgr2=0x7facfc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:35.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.582+0000 7fad1fc00640 1 --2- 172.21.15.71:0/1851484694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facfc003a00 0x7facfc003e20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fad08009b50 tx=0x7facfc005830 comp rx=0 tx=0).stop 2024-09-16T07:29:35.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.582+0000 7fad1dc00640 1 -- 172.21.15.71:0/1851484694 reap_dead start 2024-09-16T07:29:35.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.582+0000 7fad1fc00640 1 -- 172.21.15.71:0/1851484694 shutdown_connections 2024-09-16T07:29:35.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.582+0000 7fad1fc00640 1 -- 172.21.15.71:0/1851484694 >> 172.21.15.71:0/1851484694 conn(0x7fad180b18e0 msgr2=0x7fad180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:35.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.582+0000 7fad1fc00640 1 -- 172.21.15.71:0/1851484694 shutdown_connections 2024-09-16T07:29:35.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.583+0000 7fad1fc00640 1 -- 172.21.15.71:0/1851484694 wait complete. 2024-09-16T07:29:35.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.584+0000 7fad1fc00640 1 Processor -- start 2024-09-16T07:29:35.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.584+0000 7fad1fc00640 1 -- start start 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad1fc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facfc003a00 0x7fad181f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad1fc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad1813bcc0 0x7fad18156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad1fc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad18156b80 0x7fad180cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad1fc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fad180cdc50 con 0x7fad1813bcc0 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad1fc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad180cdad0 con 0x7fad18156b80 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad1fc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fad180cddd0 con 0x7facfc003a00 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad17e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad1813bcc0 0x7fad18156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad17e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad1813bcc0 0x7fad18156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48360/0 (socket says 172.21.15.71:48360) 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.585+0000 7fad17e00640 1 -- 172.21.15.71:0/2610074330 learned_addr learned my addr 172.21.15.71:0/2610074330 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad1dc00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad18156b80 0x7fad180cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:35.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad1d200640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facfc003a00 0x7fad181f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad17e00640 1 -- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facfc003a00 msgr2=0x7fad181f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad17e00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facfc003a00 0x7fad181f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad17e00640 1 -- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad18156b80 msgr2=0x7fad180cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad17e00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad18156b80 0x7fad180cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad17e00640 1 -- 172.21.15.71:0/2610074330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad18157000 con 0x7fad1813bcc0 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad17e00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad1813bcc0 0x7fad18156640 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fad04006ec0 tx=0x7fad040049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad14c00640 1 -- 172.21.15.71:0/2610074330 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04015020 con 0x7fad1813bcc0 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad14c00640 1 -- 172.21.15.71:0/2610074330 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad04010980 con 0x7fad1813bcc0 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad181572e0 con 0x7fad1813bcc0 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad14c00640 1 -- 172.21.15.71:0/2610074330 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04010d90 con 0x7fad1813bcc0 2024-09-16T07:29:35.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.586+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad181f67d0 con 0x7fad1813bcc0 2024-09-16T07:29:35.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.587+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad18069360 con 0x7fad1813bcc0 2024-09-16T07:29:35.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.589+0000 7fad14c00640 1 -- 172.21.15.71:0/2610074330 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fad04018020 con 0x7fad1813bcc0 2024-09-16T07:29:35.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.589+0000 7fad14c00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facf8079420 0x7facf807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:35.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.590+0000 7fad14c00640 1 -- 172.21.15.71:0/2610074330 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5443+0+0 (secure 0 0 0) 0x7fad0409a2b0 con 0x7fad1813bcc0 2024-09-16T07:29:35.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.590+0000 7fad1d200640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facf8079420 0x7facf807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:35.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.591+0000 7fad1d200640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facf8079420 0x7facf807b8e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad08035000 tx=0x7fad0802d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:35.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.593+0000 7fad14c00640 1 -- 172.21.15.71:0/2610074330 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fad0401f020 con 0x7fad1813bcc0 2024-09-16T07:29:35.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:35.780+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fad180cc5b0 con 0x7fad1813bcc0 2024-09-16T07:29:36.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.394+0000 7fad14c00640 1 -- 172.21.15.71:0/2610074330 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) ==== 169+0+0 (secure 0 0 0) 0x7fad1815c140 con 0x7fad1813bcc0 2024-09-16T07:29:36.394 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:29:36.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facf8079420 msgr2=0x7facf807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facf8079420 0x7facf807b8e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad08035000 tx=0x7fad0802d040 comp rx=0 tx=0).stop 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad1813bcc0 msgr2=0x7fad18156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad1813bcc0 0x7fad18156640 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fad04006ec0 tx=0x7fad040049d0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 shutdown_connections 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facf8079420 0x7facf807b8e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad18156b80 0x7fad180cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad1813bcc0 0x7fad18156640 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 --2- 172.21.15.71:0/2610074330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facfc003a00 0x7fad181f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.398+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 >> 172.21.15.71:0/2610074330 conn(0x7fad180b18e0 msgr2=0x7fad180b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:36.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.399+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 shutdown_connections 2024-09-16T07:29:36.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.399+0000 7fad1fc00640 1 -- 172.21.15.71:0/2610074330 wait complete. 2024-09-16T07:29:36.431 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:29:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.590+0000 7fa234400640 1 -- 172.21.15.71:0/1293352981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 msgr2=0x7fa22c0cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.590+0000 7fa234400640 1 --2- 172.21.15.71:0/1293352981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 0x7fa22c0cf130 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa224005fd0 tx=0x7fa22402a6c0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.590+0000 7fa234400640 1 -- 172.21.15.71:0/1293352981 shutdown_connections 2024-09-16T07:29:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.590+0000 7fa234400640 1 --2- 172.21.15.71:0/1293352981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 0x7fa22c0cf130 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.590+0000 7fa234400640 1 --2- 172.21.15.71:0/1293352981 >> v2:172.21.15.177:3300/0 conn(0x7fa22c1445d0 0x7fa22c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.590+0000 7fa234400640 1 -- 172.21.15.71:0/1293352981 >> 172.21.15.71:0/1293352981 conn(0x7fa22c0b1950 msgr2=0x7fa22c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.590+0000 7fa234400640 1 -- 172.21.15.71:0/1293352981 shutdown_connections 2024-09-16T07:29:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.590+0000 7fa234400640 1 -- 172.21.15.71:0/1293352981 wait complete. 2024-09-16T07:29:36.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.592+0000 7fa234400640 1 Processor -- start 2024-09-16T07:29:36.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.592+0000 7fa234400640 1 -- start start 2024-09-16T07:29:36.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa234400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 0x7fa22c1413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:36.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa234400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa22c0ccfd0 0x7fa22c1e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:36.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa234400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa22c1445d0 0x7fa22c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:36.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa234400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c15f080 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa234400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c073bc0 con 0x7fa22c0ccfd0 2024-09-16T07:29:36.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa234400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa22c15f200 con 0x7fa22c1445d0 2024-09-16T07:29:36.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 0x7fa22c1413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 0x7fa22c1413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48390/0 (socket says 172.21.15.71:48390) 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231a00640 1 -- 172.21.15.71:0/734304506 learned_addr learned my addr 172.21.15.71:0/734304506 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231000640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa22c0ccfd0 0x7fa22c1e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa232400640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa22c1445d0 0x7fa22c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231a00640 1 -- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa22c1445d0 msgr2=0x7fa22c06fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231a00640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa22c1445d0 0x7fa22c06fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231a00640 1 -- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa22c0ccfd0 msgr2=0x7fa22c1e8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231a00640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa22c0ccfd0 0x7fa22c1e8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.593+0000 7fa231a00640 1 -- 172.21.15.71:0/734304506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c15cc00 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.594+0000 7fa231a00640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 0x7fa22c1413f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa22000e9f0 tx=0x7fa22000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:36.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.594+0000 7fa229600640 1 -- 172.21.15.71:0/734304506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22000cd80 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.594+0000 7fa229600640 1 -- 172.21.15.71:0/734304506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa220004c10 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.594+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c141150 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.594+0000 7fa229600640 1 -- 172.21.15.71:0/734304506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220005020 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.594+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c15d160 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.595+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22c15f720 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.596+0000 7fa229600640 1 -- 172.21.15.71:0/734304506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fa2200051c0 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.597+0000 7fa229600640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa20c079420 0x7fa20c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:36.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.597+0000 7fa229600640 1 -- 172.21.15.71:0/734304506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4971+0+0 (secure 0 0 0) 0x7fa220099f70 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.598+0000 7fa231000640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa20c079420 0x7fa20c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:36.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.598+0000 7fa231000640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa20c079420 0x7fa20c07b8e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa22c1f80e0 tx=0x7fa218009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:36.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.601+0000 7fa229600640 1 -- 172.21.15.71:0/734304506 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa22c15cc00 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.788+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa22c0b26e0 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.792+0000 7fa229600640 1 -- 172.21.15.71:0/734304506 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v36) ==== 71+0+0 (secure 0 0 0) 0x7fa220016030 con 0x7fa22c0b4ca0 2024-09-16T07:29:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa20c079420 msgr2=0x7fa20c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa20c079420 0x7fa20c07b8e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa22c1f80e0 tx=0x7fa218009040 comp rx=0 tx=0).stop 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 msgr2=0x7fa22c1413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 0x7fa22c1413f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa22000e9f0 tx=0x7fa22000eeb0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 shutdown_connections 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa20c079420 0x7fa20c07b8e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa22c1445d0 0x7fa22c06fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa22c0ccfd0 0x7fa22c1e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 --2- 172.21.15.71:0/734304506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c0b4ca0 0x7fa22c1413f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 >> 172.21.15.71:0/734304506 conn(0x7fa22c0b1950 msgr2=0x7fa22c0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 shutdown_connections 2024-09-16T07:29:36.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.795+0000 7fa234400640 1 -- 172.21.15.71:0/734304506 wait complete. 2024-09-16T07:29:36.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-16T07:29:36.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.968+0000 7f0ed7000640 1 -- 172.21.15.71:0/1946529532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 msgr2=0x7f0ed00b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.968+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1946529532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 0x7f0ed00b28f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4005fd0 tx=0x7f0ec402aa90 comp rx=0 tx=0).stop 2024-09-16T07:29:36.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.968+0000 7f0ed7000640 1 -- 172.21.15.71:0/1946529532 shutdown_connections 2024-09-16T07:29:36.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.968+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1946529532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed00725d0 0x7f0ed0073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.968+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1946529532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 0x7f0ed00b28f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.968+0000 7f0ed7000640 1 -- 172.21.15.71:0/1946529532 >> 172.21.15.71:0/1946529532 conn(0x7f0ed00cfa20 msgr2=0x7f0ed00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:36.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.968+0000 7f0ed7000640 1 -- 172.21.15.71:0/1946529532 shutdown_connections 2024-09-16T07:29:36.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.968+0000 7f0ed7000640 1 -- 172.21.15.71:0/1946529532 wait complete. 2024-09-16T07:29:36.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.970+0000 7f0ed7000640 1 Processor -- start 2024-09-16T07:29:36.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.970+0000 7f0ed7000640 1 -- start start 2024-09-16T07:29:36.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed7000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed00d0540 0x7f0ed00c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:36.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed7000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 0x7f0ed01544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed7000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed015d070 0x7f0ed0154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed7000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0074610 con 0x7f0ed015bc30 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed7000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0074490 con 0x7f0ed00d0540 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed7000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed0074790 con 0x7f0ed015d070 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 0x7f0ed01544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 0x7f0ed01544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48406/0 (socket says 172.21.15.71:48406) 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5200640 1 -- 172.21.15.71:0/1816988751 learned_addr learned my addr 172.21.15.71:0/1816988751 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5c00640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed00d0540 0x7f0ed00c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed6600640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed015d070 0x7f0ed0154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5200640 1 -- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed015d070 msgr2=0x7f0ed0154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5200640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed015d070 0x7f0ed0154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5200640 1 -- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed00d0540 msgr2=0x7f0ed00c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5200640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed00d0540 0x7f0ed00c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed5200640 1 -- 172.21.15.71:0/1816988751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed01f2fd0 con 0x7f0ed015bc30 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.971+0000 7f0ed6600640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed015d070 0x7f0ed0154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.972+0000 7f0ed5200640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 0x7f0ed01544c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00d770 tx=0x7f0ebc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.972+0000 7f0ecd600640 1 -- 172.21.15.71:0/1816988751 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc017070 con 0x7f0ed015bc30 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.972+0000 7f0ecd600640 1 -- 172.21.15.71:0/1816988751 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ebc009d20 con 0x7f0ed015bc30 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.972+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed015c590 con 0x7f0ed015bc30 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.972+0000 7f0ecd600640 1 -- 172.21.15.71:0/1816988751 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc005280 con 0x7f0ed015bc30 2024-09-16T07:29:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.972+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed015cb50 con 0x7f0ed015bc30 2024-09-16T07:29:36.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.973+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed006fcb0 con 0x7f0ed015bc30 2024-09-16T07:29:36.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.974+0000 7f0ecd600640 1 -- 172.21.15.71:0/1816988751 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f0ebc005420 con 0x7f0ed015bc30 2024-09-16T07:29:36.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.975+0000 7f0ecd600640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0eb0079420 0x7f0eb007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:36.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.975+0000 7f0ecd600640 1 -- 172.21.15.71:0/1816988751 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4971+0+0 (secure 0 0 0) 0x7f0ebc099720 con 0x7f0ed015bc30 2024-09-16T07:29:36.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.976+0000 7f0ed5c00640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0eb0079420 0x7f0eb007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.976+0000 7f0ed5c00640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0eb0079420 0x7f0eb007b8e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4035000 tx=0x7f0ec4038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:36.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:36.979+0000 7f0ecd600640 1 -- 172.21.15.71:0/1816988751 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0ebc063580 con 0x7f0ed015bc30 2024-09-16T07:29:37.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.232+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7f0ed015cd20 con 0x7f0ed015bc30 2024-09-16T07:29:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.401+0000 7f0ecd600640 1 -- 172.21.15.71:0/1816988751 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v314) ==== 167+0+0 (secure 0 0 0) 0x7f0ebc09c0d0 con 0x7f0ed015bc30 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0eb0079420 msgr2=0x7f0eb007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0eb0079420 0x7f0eb007b8e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4035000 tx=0x7f0ec4038040 comp rx=0 tx=0).stop 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 msgr2=0x7f0ed01544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 0x7f0ed01544c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00d770 tx=0x7f0ebc00dc30 comp rx=0 tx=0).stop 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 shutdown_connections 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0eb0079420 0x7f0eb007b8e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed015d070 0x7f0ed0154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ed015bc30 0x7f0ed01544c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 --2- 172.21.15.71:0/1816988751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed00d0540 0x7f0ed00c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 >> 172.21.15.71:0/1816988751 conn(0x7f0ed00cfa20 msgr2=0x7f0ed00c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 shutdown_connections 2024-09-16T07:29:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.404+0000 7f0ed7000640 1 -- 172.21.15.71:0/1816988751 wait complete. 2024-09-16T07:29:37.420 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-16T07:29:37.421 INFO:tasks.cephfs_test_runner:Starting test: test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed in output of "ceph status" 2024-09-16T07:29:37.421 INFO:tasks.cephfs_test_runner:test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:29:37.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.578+0000 7f88b7e00640 1 -- 172.21.15.71:0/2726190606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b015f5a0 msgr2=0x7f88b00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.578+0000 7f88b7e00640 1 --2- 172.21.15.71:0/2726190606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b015f5a0 0x7f88b00b2440 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f88b0076720 tx=0x7f88a402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.578+0000 7f88b7e00640 1 -- 172.21.15.71:0/2726190606 shutdown_connections 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.578+0000 7f88b7e00640 1 --2- 172.21.15.71:0/2726190606 >> v2:172.21.15.177:3300/0 conn(0x7f88b00ad270 0x7f88b0073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.578+0000 7f88b7e00640 1 --2- 172.21.15.71:0/2726190606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88b00ccf00 0x7f88b00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.578+0000 7f88b7e00640 1 --2- 172.21.15.71:0/2726190606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b015f5a0 0x7f88b00b2440 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.578+0000 7f88b7e00640 1 -- 172.21.15.71:0/2726190606 >> 172.21.15.71:0/2726190606 conn(0x7f88b00b1950 msgr2=0x7f88b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.579+0000 7f88b7e00640 1 -- 172.21.15.71:0/2726190606 shutdown_connections 2024-09-16T07:29:37.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.579+0000 7f88b7e00640 1 -- 172.21.15.71:0/2726190606 wait complete. 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.580+0000 7f88b7e00640 1 Processor -- start 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.580+0000 7f88b7e00640 1 -- start start 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b7e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88b00ad270 0x7f88b01f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b00ccf00 0x7f88b014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b015f5a0 0x7f88b0151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88b006f160 con 0x7f88b00ccf00 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b7e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88b0074780 con 0x7f88b015f5a0 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b7e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f88b006f2e0 con 0x7f88b00ad270 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b00ccf00 0x7f88b014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b00ccf00 0x7f88b014c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48432/0 (socket says 172.21.15.71:48432) 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.581+0000 7f88b4a00640 1 -- 172.21.15.71:0/3507331237 learned_addr learned my addr 172.21.15.71:0/3507331237 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b5400640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88b00ad270 0x7f88b01f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:37.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b4a00640 1 -- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88b00ad270 msgr2=0x7f88b01f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b4a00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88b00ad270 0x7f88b01f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b5e00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b015f5a0 0x7f88b0151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b4a00640 1 -- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b015f5a0 msgr2=0x7f88b0151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b4a00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b015f5a0 0x7f88b0151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b4a00640 1 -- 172.21.15.71:0/3507331237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b0152760 con 0x7f88b00ccf00 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b5400640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88b00ad270 0x7f88b01f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b5e00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b015f5a0 0x7f88b0151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b4a00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b00ccf00 0x7f88b014c7c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b4d0 tx=0x7f889c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88ad600640 1 -- 172.21.15.71:0/3507331237 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c014070 con 0x7f88b00ccf00 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88ad600640 1 -- 172.21.15.71:0/3507331237 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f889c010960 con 0x7f88b00ccf00 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88ad600640 1 -- 172.21.15.71:0/3507331237 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c010db0 con 0x7f88b00ccf00 2024-09-16T07:29:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.582+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b0152a40 con 0x7f88b00ccf00 2024-09-16T07:29:37.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.583+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b0153000 con 0x7f88b00ccf00 2024-09-16T07:29:37.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.583+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b0153330 con 0x7f88b00ccf00 2024-09-16T07:29:37.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.585+0000 7f88ad600640 1 -- 172.21.15.71:0/3507331237 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f889c017070 con 0x7f88b00ccf00 2024-09-16T07:29:37.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.586+0000 7f88ad600640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8890079420 0x7f889007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:37.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.586+0000 7f88ad600640 1 -- 172.21.15.71:0/3507331237 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4971+0+0 (secure 0 0 0) 0x7f889c099140 con 0x7f88b00ccf00 2024-09-16T07:29:37.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.586+0000 7f88b5400640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8890079420 0x7f889007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:37.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.587+0000 7f88b5400640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8890079420 0x7f889007b8e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f88a4033000 tx=0x7f88a4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:37.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.590+0000 7f88ad600640 1 -- 172.21.15.71:0/3507331237 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f889c063200 con 0x7f88b00ccf00 2024-09-16T07:29:37.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.775+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88b009d6a0 con 0x7f88b00ccf00 2024-09-16T07:29:37.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.775+0000 7f88ad600640 1 -- 172.21.15.71:0/3507331237 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+13087 (secure 0 0 0) 0x7f889c07ce20 con 0x7f88b00ccf00 2024-09-16T07:29:37.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:37.775 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":68,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:29:36.393198+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6834/3242207389":"2024-09-17T07:29:33.057156+0000","172.21.15.71:6833/3242207389":"2024-09-17T07:29:33.057156+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:29:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8890079420 msgr2=0x7f889007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8890079420 0x7f889007b8e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f88a4033000 tx=0x7f88a4004730 comp rx=0 tx=0).stop 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b00ccf00 msgr2=0x7f88b014c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b00ccf00 0x7f88b014c7c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b4d0 tx=0x7f889c00b990 comp rx=0 tx=0).stop 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 shutdown_connections 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8890079420 0x7f889007b8e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b015f5a0 0x7f88b0151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b00ccf00 0x7f88b014c7c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 --2- 172.21.15.71:0/3507331237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88b00ad270 0x7f88b01f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 >> 172.21.15.71:0/3507331237 conn(0x7f88b00b1950 msgr2=0x7f88b009bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 shutdown_connections 2024-09-16T07:29:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.778+0000 7f88b7e00640 1 -- 172.21.15.71:0/3507331237 wait complete. 2024-09-16T07:29:37.793 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:29:37.793 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:29:37.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.955+0000 7f16ca400640 1 -- 172.21.15.71:0/3213025130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a8003fd0 msgr2=0x7f16a8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.955+0000 7f16ca400640 1 --2- 172.21.15.71:0/3213025130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a8003fd0 0x7f16a8004800 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009b20 tx=0x7f16a8005830 comp rx=0 tx=0).stop 2024-09-16T07:29:37.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.955+0000 7f16c9a00640 1 -- 172.21.15.71:0/3213025130 reap_dead start 2024-09-16T07:29:37.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.955+0000 7f16ca400640 1 -- 172.21.15.71:0/3213025130 shutdown_connections 2024-09-16T07:29:37.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.955+0000 7f16ca400640 1 -- 172.21.15.71:0/3213025130 >> 172.21.15.71:0/3213025130 conn(0x7f16c40d3e20 msgr2=0x7f16c40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:37.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.955+0000 7f16ca400640 1 -- 172.21.15.71:0/3213025130 shutdown_connections 2024-09-16T07:29:37.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.955+0000 7f16ca400640 1 -- 172.21.15.71:0/3213025130 wait complete. 2024-09-16T07:29:37.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.956+0000 7f16ca400640 1 Processor -- start 2024-09-16T07:29:37.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.956+0000 7f16ca400640 1 -- start start 2024-09-16T07:29:37.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.957+0000 7f16ca400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a8003fd0 0x7f16c40ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:37.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.957+0000 7f16ca400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c40c87b0 0x7f16c415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:37.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.957+0000 7f16ca400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16c415e740 0x7f16c40b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:37.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.957+0000 7f16ca400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4162230 con 0x7f16c40c87b0 2024-09-16T07:29:37.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.957+0000 7f16ca400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41620b0 con 0x7f16c415e740 2024-09-16T07:29:37.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.957+0000 7f16ca400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f16c40b1680 con 0x7f16a8003fd0 2024-09-16T07:29:37.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c40c87b0 0x7f16c415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c40c87b0 0x7f16c415e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48458/0 (socket says 172.21.15.71:48458) 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 -- 172.21.15.71:0/1185340605 learned_addr learned my addr 172.21.15.71:0/1185340605 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c9000640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a8003fd0 0x7f16c40ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c9a00640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16c415e740 0x7f16c40b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 -- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a8003fd0 msgr2=0x7f16c40ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a8003fd0 0x7f16c40ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 -- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16c415e740 msgr2=0x7f16c40b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16c415e740 0x7f16c40b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 -- 172.21.15.71:0/1185340605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c41f7a80 con 0x7f16c40c87b0 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c9000640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a8003fd0 0x7f16c40ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c9a00640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16c415e740 0x7f16c40b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c3e00640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c40c87b0 0x7f16c415e200 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f16b0000c00 tx=0x7f16b0033cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c0c00640 1 -- 172.21.15.71:0/1185340605 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0038040 con 0x7f16c40c87b0 2024-09-16T07:29:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c0c00640 1 -- 172.21.15.71:0/1185340605 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16b0005020 con 0x7f16c40c87b0 2024-09-16T07:29:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16c0c00640 1 -- 172.21.15.71:0/1185340605 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0005470 con 0x7f16c40c87b0 2024-09-16T07:29:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.958+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c41f7d60 con 0x7f16c40c87b0 2024-09-16T07:29:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.959+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c41f8320 con 0x7f16c40c87b0 2024-09-16T07:29:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.960+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c40d23e0 con 0x7f16c40c87b0 2024-09-16T07:29:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.961+0000 7f16c0c00640 1 -- 172.21.15.71:0/1185340605 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f16b0043020 con 0x7f16c40c87b0 2024-09-16T07:29:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.961+0000 7f16c0c00640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f16a4079450 0x7f16a407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.962+0000 7f16c0c00640 1 -- 172.21.15.71:0/1185340605 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4971+0+0 (secure 0 0 0) 0x7f16b00b9c00 con 0x7f16c40c87b0 2024-09-16T07:29:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.962+0000 7f16c9000640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f16a4079450 0x7f16a407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:37.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.963+0000 7f16c9000640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f16a4079450 0x7f16a407b910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f16c4155e60 tx=0x7f16b4005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:37.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:37.966+0000 7f16c0c00640 1 -- 172.21.15.71:0/1185340605 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f16b0083a60 con 0x7f16c40c87b0 2024-09-16T07:29:38.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.150+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16c415eb20 con 0x7f16c40c87b0 2024-09-16T07:29:38.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.150+0000 7f16c0c00640 1 -- 172.21.15.71:0/1185340605 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f16b0083360 con 0x7f16c40c87b0 2024-09-16T07:29:38.150 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f16a4079450 msgr2=0x7f16a407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f16a4079450 0x7f16a407b910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f16c4155e60 tx=0x7f16b4005f90 comp rx=0 tx=0).stop 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c40c87b0 msgr2=0x7f16c415e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c40c87b0 0x7f16c415e200 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f16b0000c00 tx=0x7f16b0033cb0 comp rx=0 tx=0).stop 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 shutdown_connections 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f16a4079450 0x7f16a407b910 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16c415e740 0x7f16c40b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c40c87b0 0x7f16c415e200 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 --2- 172.21.15.71:0/1185340605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a8003fd0 0x7f16c40ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 >> 172.21.15.71:0/1185340605 conn(0x7f16c40d3e20 msgr2=0x7f16c4160480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:38.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 shutdown_connections 2024-09-16T07:29:38.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.153+0000 7f16ca400640 1 -- 172.21.15.71:0/1185340605 wait complete. 2024-09-16T07:29:38.169 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-16T07:29:38.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.327+0000 7fc000800640 1 -- 172.21.15.71:0/773931342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0004d40 msgr2=0x7fbfe00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:38.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.327+0000 7fc000800640 1 --2- 172.21.15.71:0/773931342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0004d40 0x7fbfe00051c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009ed0 tx=0x7fbfe0005830 comp rx=0 tx=0).stop 2024-09-16T07:29:38.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.327+0000 7fbffe800640 1 -- 172.21.15.71:0/773931342 reap_dead start 2024-09-16T07:29:38.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.327+0000 7fc000800640 1 -- 172.21.15.71:0/773931342 shutdown_connections 2024-09-16T07:29:38.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.327+0000 7fc000800640 1 -- 172.21.15.71:0/773931342 >> 172.21.15.71:0/773931342 conn(0x7fbff8069560 msgr2=0x7fbff806cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:38.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.327+0000 7fc000800640 1 -- 172.21.15.71:0/773931342 shutdown_connections 2024-09-16T07:29:38.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.327+0000 7fc000800640 1 -- 172.21.15.71:0/773931342 wait complete. 2024-09-16T07:29:38.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.329+0000 7fc000800640 1 Processor -- start 2024-09-16T07:29:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.329+0000 7fc000800640 1 -- start start 2024-09-16T07:29:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fc000800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfe0004d40 0x7fbff8159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fc000800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff81f4610 0x7fbff8151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fc000800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbff80b0760 0x7fbff80b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fc000800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8070c50 con 0x7fbff81f4610 2024-09-16T07:29:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fc000800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8070ad0 con 0x7fbff80b0760 2024-09-16T07:29:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fc000800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbff8070dd0 con 0x7fbfe0004d40 2024-09-16T07:29:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff81f4610 0x7fbff8151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff81f4610 0x7fbff8151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48480/0 (socket says 172.21.15.71:48480) 2024-09-16T07:29:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffd400640 1 -- 172.21.15.71:0/3162580335 learned_addr learned my addr 172.21.15.71:0/3162580335 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffe800640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbff80b0760 0x7fbff80b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffd400640 1 -- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfe0004d40 msgr2=0x7fbff8159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffd400640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfe0004d40 0x7fbff8159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffd400640 1 -- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbff80b0760 msgr2=0x7fbff80b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffd400640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbff80b0760 0x7fbff80b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:38.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.330+0000 7fbffd400640 1 -- 172.21.15.71:0/3162580335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff81f4e60 con 0x7fbff81f4610 2024-09-16T07:29:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.331+0000 7fbffd400640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff81f4610 0x7fbff8151180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8001a50 tx=0x7fbfe80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.331+0000 7fbff2000640 1 -- 172.21.15.71:0/3162580335 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8015070 con 0x7fbff81f4610 2024-09-16T07:29:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.331+0000 7fbff2000640 1 -- 172.21.15.71:0/3162580335 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfe8011900 con 0x7fbff81f4610 2024-09-16T07:29:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.331+0000 7fbff2000640 1 -- 172.21.15.71:0/3162580335 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8011d30 con 0x7fbff81f4610 2024-09-16T07:29:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.331+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff81f5140 con 0x7fbff81f4610 2024-09-16T07:29:38.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.331+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff81516c0 con 0x7fbff81f4610 2024-09-16T07:29:38.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.332+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff8073010 con 0x7fbff81f4610 2024-09-16T07:29:38.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.333+0000 7fbff2000640 1 -- 172.21.15.71:0/3162580335 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fbfe8018070 con 0x7fbff81f4610 2024-09-16T07:29:38.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.334+0000 7fbff2000640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbfdc079450 0x7fbfdc07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.334+0000 7fbff2000640 1 -- 172.21.15.71:0/3162580335 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4971+0+0 (secure 0 0 0) 0x7fbfe809a320 con 0x7fbff81f4610 2024-09-16T07:29:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.334+0000 7fbffde00640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbfdc079450 0x7fbfdc07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.335+0000 7fbffde00640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbfdc079450 0x7fbfdc07b910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbff8159a20 tx=0x7fbfec009660 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.339+0000 7fbff2000640 1 -- 172.21.15.71:0/3162580335 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbff8156740 con 0x7fbff81f4610 2024-09-16T07:29:38.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:38.591+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7fbff80cd310 con 0x7fbff81f4610 2024-09-16T07:29:39.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.410+0000 7fbff2000640 1 -- 172.21.15.71:0/3162580335 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v316) ==== 170+0+0 (secure 0 0 0) 0x7fbfe8064160 con 0x7fbff81f4610 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbfdc079450 msgr2=0x7fbfdc07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbfdc079450 0x7fbfdc07b910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbff8159a20 tx=0x7fbfec009660 comp rx=0 tx=0).stop 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff81f4610 msgr2=0x7fbff8151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff81f4610 0x7fbff8151180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8001a50 tx=0x7fbfe80039c0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 shutdown_connections 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbfdc079450 0x7fbfdc07b910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbff80b0760 0x7fbff80b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff81f4610 0x7fbff8151180 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 --2- 172.21.15.71:0/3162580335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfe0004d40 0x7fbff8159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 >> 172.21.15.71:0/3162580335 conn(0x7fbff8069560 msgr2=0x7fbff80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:39.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 shutdown_connections 2024-09-16T07:29:39.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.413+0000 7fc000800640 1 -- 172.21.15.71:0/3162580335 wait complete. 2024-09-16T07:29:39.431 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:29:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.589+0000 7f2de1c00640 1 -- 172.21.15.71:0/1360563251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc161550 msgr2=0x7f2ddc06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.589+0000 7f2de1c00640 1 --2- 172.21.15.71:0/1360563251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc161550 0x7f2ddc06ef50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc005fd0 tx=0x7f2dcc02aa10 comp rx=0 tx=0).stop 2024-09-16T07:29:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.589+0000 7f2de1c00640 1 -- 172.21.15.71:0/1360563251 shutdown_connections 2024-09-16T07:29:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.589+0000 7f2de1c00640 1 --2- 172.21.15.71:0/1360563251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ddc0af3d0 0x7f2ddc06f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.589+0000 7f2de1c00640 1 --2- 172.21.15.71:0/1360563251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc161550 0x7f2ddc06ef50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.589+0000 7f2de1c00640 1 -- 172.21.15.71:0/1360563251 >> 172.21.15.71:0/1360563251 conn(0x7f2ddc0b1950 msgr2=0x7f2ddc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.590+0000 7f2de1c00640 1 -- 172.21.15.71:0/1360563251 shutdown_connections 2024-09-16T07:29:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.590+0000 7f2de1c00640 1 -- 172.21.15.71:0/1360563251 wait complete. 2024-09-16T07:29:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.591+0000 7f2de1c00640 1 Processor -- start 2024-09-16T07:29:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.591+0000 7f2de1c00640 1 -- start start 2024-09-16T07:29:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2de1c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ddc06f9d0 0x7f2ddc0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2de1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc0af3d0 0x7f2ddc1f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2de1c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ddc161550 0x7f2ddc1f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2de1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc1623a0 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2de1c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc074010 con 0x7f2ddc161550 2024-09-16T07:29:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2de1c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2ddc162520 con 0x7f2ddc06f9d0 2024-09-16T07:29:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2dda000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc0af3d0 0x7f2ddc1f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2dda000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc0af3d0 0x7f2ddc1f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48502/0 (socket says 172.21.15.71:48502) 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.592+0000 7f2dda000640 1 -- 172.21.15.71:0/2324701833 learned_addr learned my addr 172.21.15.71:0/2324701833 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2ddaa00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ddc06f9d0 0x7f2ddc0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2ddb400640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ddc161550 0x7f2ddc1f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dda000640 1 -- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ddc06f9d0 msgr2=0x7f2ddc0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dda000640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ddc06f9d0 0x7f2ddc0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dda000640 1 -- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ddc161550 msgr2=0x7f2ddc1f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dda000640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ddc161550 0x7f2ddc1f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dda000640 1 -- 172.21.15.71:0/2324701833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc1f4b30 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2ddb400640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ddc161550 0x7f2ddc1f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dda000640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc0af3d0 0x7f2ddc1f8120 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800b450 tx=0x7f2dc800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dd2a00640 1 -- 172.21.15.71:0/2324701833 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8015020 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dd2a00640 1 -- 172.21.15.71:0/2324701833 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dc8011920 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2dd2a00640 1 -- 172.21.15.71:0/2324701833 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8011d70 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.593+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ddc1f8660 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.594+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ddc1f8be0 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.595+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ddc0760b0 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.596+0000 7f2dd2a00640 1 -- 172.21.15.71:0/2324701833 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f2dc8018020 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.596+0000 7f2dd2a00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2dbc079420 0x7f2dbc07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.597+0000 7f2dd2a00640 1 -- 172.21.15.71:0/2324701833 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4971+0+0 (secure 0 0 0) 0x7f2dc8099580 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.597+0000 7f2ddaa00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2dbc079420 0x7f2dbc07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:39.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.598+0000 7f2ddaa00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2dbc079420 0x7f2dbc07b8e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc035000 tx=0x7f2dcc004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:39.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.601+0000 7f2dd2a00640 1 -- 172.21.15.71:0/2324701833 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2dc801f020 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.785+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2ddc162ab0 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.786+0000 7f2dd2a00640 1 -- 172.21.15.71:0/2324701833 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) ==== 59+0+1998 (secure 0 0 0) 0x7f2ddc1f4b30 con 0x7f2ddc0af3d0 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:29:39.786 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:29:39.787 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:29:39.788 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:29:39.788 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:29:39.788 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:29:39.788 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2dbc079420 msgr2=0x7f2dbc07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2dbc079420 0x7f2dbc07b8e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc035000 tx=0x7f2dcc004ab0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc0af3d0 msgr2=0x7f2ddc1f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc0af3d0 0x7f2ddc1f8120 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800b450 tx=0x7f2dc800b910 comp rx=0 tx=0).stop 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 shutdown_connections 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2dbc079420 0x7f2dbc07b8e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ddc161550 0x7f2ddc1f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ddc0af3d0 0x7f2ddc1f8120 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 --2- 172.21.15.71:0/2324701833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ddc06f9d0 0x7f2ddc0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 >> 172.21.15.71:0/2324701833 conn(0x7f2ddc0b1950 msgr2=0x7f2ddc0b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:39.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.788+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 shutdown_connections 2024-09-16T07:29:39.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.789+0000 7f2de1c00640 1 -- 172.21.15.71:0/2324701833 wait complete. 2024-09-16T07:29:39.804 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:29:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.964+0000 7fa8d6a00640 1 -- 172.21.15.71:0/2183854163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d0145750 msgr2=0x7fa8d0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.964+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/2183854163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d0145750 0x7fa8d0073440 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0005fd0 tx=0x7fa8c002aa10 comp rx=0 tx=0).stop 2024-09-16T07:29:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.964+0000 7fa8d6a00640 1 -- 172.21.15.71:0/2183854163 shutdown_connections 2024-09-16T07:29:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.964+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/2183854163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8d00cccd0 0x7fa8d00cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.964+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/2183854163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d0145750 0x7fa8d0073440 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.964+0000 7fa8d6a00640 1 -- 172.21.15.71:0/2183854163 >> 172.21.15.71:0/2183854163 conn(0x7fa8d00b1840 msgr2=0x7fa8d00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.964+0000 7fa8d6a00640 1 -- 172.21.15.71:0/2183854163 shutdown_connections 2024-09-16T07:29:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.964+0000 7fa8d6a00640 1 -- 172.21.15.71:0/2183854163 wait complete. 2024-09-16T07:29:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.966+0000 7fa8d6a00640 1 Processor -- start 2024-09-16T07:29:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.966+0000 7fa8d6a00640 1 -- start start 2024-09-16T07:29:39.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8d6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d006efa0 0x7fa8d01f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8d6a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8d00cccd0 0x7fa8d00c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8d6a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8d0145750 0x7fa8d01f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8d6a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d00ce480 con 0x7fa8d006efa0 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8d6a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d00ce300 con 0x7fa8d0145750 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8d6a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d00ce600 con 0x7fa8d00cccd0 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d006efa0 0x7fa8d01f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d006efa0 0x7fa8d01f78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48520/0 (socket says 172.21.15.71:48520) 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.967+0000 7fa8cfe00640 1 -- 172.21.15.71:0/271656214 learned_addr learned my addr 172.21.15.71:0/271656214 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8d4a00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8d0145750 0x7fa8d01f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8cf400640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8d00cccd0 0x7fa8d00c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8cfe00640 1 -- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8d00cccd0 msgr2=0x7fa8d00c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8cfe00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8d00cccd0 0x7fa8d00c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8cfe00640 1 -- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8d0145750 msgr2=0x7fa8d01f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:39.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8cfe00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8d0145750 0x7fa8d01f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:39.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8cfe00640 1 -- 172.21.15.71:0/271656214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d00b3bc0 con 0x7fa8d006efa0 2024-09-16T07:29:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8cfe00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d006efa0 0x7fa8d01f78f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009bf0 tx=0x7fa8c0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8c7e00640 1 -- 172.21.15.71:0/271656214 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0033050 con 0x7fa8d006efa0 2024-09-16T07:29:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8c7e00640 1 -- 172.21.15.71:0/271656214 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8c0004450 con 0x7fa8d006efa0 2024-09-16T07:29:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.968+0000 7fa8c7e00640 1 -- 172.21.15.71:0/271656214 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004840 con 0x7fa8d006efa0 2024-09-16T07:29:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.969+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d00b3ea0 con 0x7fa8d006efa0 2024-09-16T07:29:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.969+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d00b4460 con 0x7fa8d006efa0 2024-09-16T07:29:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.970+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d015ef70 con 0x7fa8d006efa0 2024-09-16T07:29:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.971+0000 7fa8c7e00640 1 -- 172.21.15.71:0/271656214 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fa8c0036070 con 0x7fa8d006efa0 2024-09-16T07:29:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.972+0000 7fa8c7e00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8b0079420 0x7fa8b007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.972+0000 7fa8c7e00640 1 -- 172.21.15.71:0/271656214 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4971+0+0 (secure 0 0 0) 0x7fa8c00ba530 con 0x7fa8d006efa0 2024-09-16T07:29:39.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.972+0000 7fa8cf400640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8b0079420 0x7fa8b007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:39.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.973+0000 7fa8cf400640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8b0079420 0x7fa8b007b8e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0144ab0 tx=0x7fa8bc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:39.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:39.976+0000 7fa8c7e00640 1 -- 172.21.15.71:0/271656214 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8c0083fa0 con 0x7fa8d006efa0 2024-09-16T07:29:40.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.160+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa8d01438d0 con 0x7fa8d006efa0 2024-09-16T07:29:40.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.164+0000 7fa8c7e00640 1 -- 172.21.15.71:0/271656214 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7fa8c00889d0 con 0x7fa8d006efa0 2024-09-16T07:29:40.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8b0079420 msgr2=0x7fa8b007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:40.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8b0079420 0x7fa8b007b8e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0144ab0 tx=0x7fa8bc009040 comp rx=0 tx=0).stop 2024-09-16T07:29:40.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d006efa0 msgr2=0x7fa8d01f78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:40.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d006efa0 0x7fa8d01f78f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009bf0 tx=0x7fa8c0003f90 comp rx=0 tx=0).stop 2024-09-16T07:29:40.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 shutdown_connections 2024-09-16T07:29:40.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8b0079420 0x7fa8b007b8e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8d0145750 0x7fa8d01f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8d00cccd0 0x7fa8d00c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 --2- 172.21.15.71:0/271656214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8d006efa0 0x7fa8d01f78f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 >> 172.21.15.71:0/271656214 conn(0x7fa8d00b1840 msgr2=0x7fa8d00a94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:40.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 shutdown_connections 2024-09-16T07:29:40.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.166+0000 7fa8d6a00640 1 -- 172.21.15.71:0/271656214 wait complete. 2024-09-16T07:29:40.181 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:29:40.181 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:29:40.267 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:29:40.369 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:29:40.370 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:40.370 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:40.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:29:40.405 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:29:40.406 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:40.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:29:40.444 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:40.444 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:40.481 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:29:40.481 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:29:40.481 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:40.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:29:40.521 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:29:40.521 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:29:40.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.680+0000 7fd67b200640 1 -- 172.21.15.71:0/4274515912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd658004800 msgr2=0x7fd658004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:40.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.680+0000 7fd67b200640 1 --2- 172.21.15.71:0/4274515912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd658004800 0x7fd658004c80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd65c001740 tx=0x7fd658005830 comp rx=0 tx=0).stop 2024-09-16T07:29:40.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.680+0000 7fd67b200640 1 -- 172.21.15.71:0/4274515912 shutdown_connections 2024-09-16T07:29:40.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.680+0000 7fd67b200640 1 --2- 172.21.15.71:0/4274515912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd658004800 0x7fd658004c80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.680+0000 7fd67b200640 1 -- 172.21.15.71:0/4274515912 >> 172.21.15.71:0/4274515912 conn(0x7fd6740b1950 msgr2=0x7fd6740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:40.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.681+0000 7fd67b200640 1 -- 172.21.15.71:0/4274515912 shutdown_connections 2024-09-16T07:29:40.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.681+0000 7fd67b200640 1 -- 172.21.15.71:0/4274515912 wait complete. 2024-09-16T07:29:40.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.682+0000 7fd67b200640 1 Processor -- start 2024-09-16T07:29:40.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.682+0000 7fd67b200640 1 -- start start 2024-09-16T07:29:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd67b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd658004800 0x7fd67413a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd67b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67415c7c0 0x7fd6741f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd67b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674140610 0x7fd67406f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd67b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740cebf0 con 0x7fd674140610 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd67b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740b4eb0 con 0x7fd67415c7c0 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd67b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd6740ced70 con 0x7fd658004800 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd679200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674140610 0x7fd67406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd679200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674140610 0x7fd67406f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48538/0 (socket says 172.21.15.71:48538) 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.683+0000 7fd679200640 1 -- 172.21.15.71:0/3538495417 learned_addr learned my addr 172.21.15.71:0/3538495417 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd673e00640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd658004800 0x7fd67413a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd66be00640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67415c7c0 0x7fd6741f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd679200640 1 -- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd658004800 msgr2=0x7fd67413a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd679200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd658004800 0x7fd67413a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd679200640 1 -- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67415c7c0 msgr2=0x7fd6741f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:40.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd679200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67415c7c0 0x7fd6741f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd679200640 1 -- 172.21.15.71:0/3538495417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67409bc90 con 0x7fd674140610 2024-09-16T07:29:40.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd679200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674140610 0x7fd67406f570 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd66c011bb0 tx=0x7fd66c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:40.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd670c00640 1 -- 172.21.15.71:0/3538495417 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c003d80 con 0x7fd674140610 2024-09-16T07:29:40.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd670c00640 1 -- 172.21.15.71:0/3538495417 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd66c004b20 con 0x7fd674140610 2024-09-16T07:29:40.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd670c00640 1 -- 172.21.15.71:0/3538495417 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004050 con 0x7fd674140610 2024-09-16T07:29:40.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.684+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67409bf70 con 0x7fd674140610 2024-09-16T07:29:40.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.685+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67409c530 con 0x7fd674140610 2024-09-16T07:29:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.686+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67415e970 con 0x7fd674140610 2024-09-16T07:29:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.687+0000 7fd670c00640 1 -- 172.21.15.71:0/3538495417 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fd66c0041f0 con 0x7fd674140610 2024-09-16T07:29:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.688+0000 7fd670c00640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd650079420 0x7fd65007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.688+0000 7fd670c00640 1 -- 172.21.15.71:0/3538495417 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4971+0+0 (secure 0 0 0) 0x7fd66c09a070 con 0x7fd674140610 2024-09-16T07:29:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.688+0000 7fd673e00640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd650079420 0x7fd65007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.689+0000 7fd673e00640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd650079420 0x7fd65007b8e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd664001050 tx=0x7fd664001110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:40.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.692+0000 7fd670c00640 1 -- 172.21.15.71:0/3538495417 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd66c063f30 con 0x7fd674140610 2024-09-16T07:29:40.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.911+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd67406f050 con 0x7fd674140610 2024-09-16T07:29:40.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.912+0000 7fd670c00640 1 -- 172.21.15.71:0/3538495417 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+3917 (secure 0 0 0) 0x7fd66c063830 con 0x7fd674140610 2024-09-16T07:29:40.912 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:40.912 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":40,"btime":"2024-09-16T07:29:34:059753+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5363,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1090989192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1090989192},{"type":"v1","addr":"172.21.15.177:6836","nonce":1090989192}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2044137966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2044137966},{"type":"v1","addr":"172.21.15.177:6837","nonce":2044137966}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3911470669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3911470669},{"type":"v1","addr":"172.21.15.71:6835","nonce":3911470669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":6324,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/2112293077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":2112293077},{"type":"v1","addr":"172.21.15.71:6834","nonce":2112293077}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[]} 2024-09-16T07:29:40.912 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 40 2024-09-16T07:29:40.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd650079420 msgr2=0x7fd65007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:40.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd650079420 0x7fd65007b8e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd664001050 tx=0x7fd664001110 comp rx=0 tx=0).stop 2024-09-16T07:29:40.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674140610 msgr2=0x7fd67406f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:40.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674140610 0x7fd67406f570 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd66c011bb0 tx=0x7fd66c00f790 comp rx=0 tx=0).stop 2024-09-16T07:29:40.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 shutdown_connections 2024-09-16T07:29:40.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd650079420 0x7fd65007b8e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674140610 0x7fd67406f570 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd67415c7c0 0x7fd6741f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 --2- 172.21.15.71:0/3538495417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd658004800 0x7fd67413a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:40.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.914+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 >> 172.21.15.71:0/3538495417 conn(0x7fd6740b1950 msgr2=0x7fd6740cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:40.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.915+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 shutdown_connections 2024-09-16T07:29:40.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:40.915+0000 7fd67b200640 1 -- 172.21.15.71:0/3538495417 wait complete. 2024-09-16T07:29:40.931 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:29:40.931 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:29:40.931 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:29:40.931 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:29:40.931 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:29:40.931 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:29:40.932 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:29:40.932 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:29:40.932 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:29:40.932 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:29:40.932 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:29:40.932 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:29:40.933 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:29:40.933 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:29:40.933 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:29:40.933 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:29:40.948 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:29:40.948 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:29:40.949 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:29:40.950 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:29:40.951 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:29:40.951 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:29:40.952 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:29:40.952 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:29:40.993 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:29:40.994 DEBUG:teuthology.parallel:result is None 2024-09-16T07:29:40.997 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:29:40.997 DEBUG:teuthology.parallel:result is None 2024-09-16T07:29:40.997 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:29:40.998 DEBUG:teuthology.parallel:result is None 2024-09-16T07:29:41.001 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:29:41.001 DEBUG:teuthology.parallel:result is None 2024-09-16T07:29:41.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:29:41.126 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:29:41.126 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:29:41.135 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:29:41.136 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:29:41.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.171+0000 7f52a9e00640 1 -- 172.21.15.71:0/3158440943 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4030e20 con 0x7f52ac06f9d0 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.171+0000 7f52b1400640 1 -- 172.21.15.71:0/3158440943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 msgr2=0x7f52ac161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.171+0000 7f52b1400640 1 --2- 172.21.15.71:0/3158440943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 0x7f52ac161c90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0723f0 tx=0x7f52a402a980 comp rx=0 tx=0).stop 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.171+0000 7f52b1400640 1 -- 172.21.15.71:0/3158440943 shutdown_connections 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.171+0000 7f52b1400640 1 --2- 172.21.15.71:0/3158440943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 0x7f52ac161c90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.171+0000 7f52b1400640 1 --2- 172.21.15.71:0/3158440943 >> v2:172.21.15.177:3300/0 conn(0x7f52ac0af3d0 0x7f52ac06f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.171+0000 7f52b1400640 1 --2- 172.21.15.71:0/3158440943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52ac161550 0x7f52ac06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.172+0000 7f52b1400640 1 -- 172.21.15.71:0/3158440943 >> 172.21.15.71:0/3158440943 conn(0x7f52ac0b1950 msgr2=0x7f52ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.172+0000 7f52b1400640 1 -- 172.21.15.71:0/3158440943 shutdown_connections 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.172+0000 7f52b1400640 1 -- 172.21.15.71:0/3158440943 wait complete. 2024-09-16T07:29:41.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.173+0000 7f52b1400640 1 Processor -- start 2024-09-16T07:29:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.174+0000 7f52b1400640 1 -- start start 2024-09-16T07:29:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52b1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 0x7f52ac1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52b1400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52ac0af3d0 0x7f52ac14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52b1400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52ac161550 0x7f52ac155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52b1400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac162560 con 0x7f52ac06f9d0 2024-09-16T07:29:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52b1400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1623e0 con 0x7f52ac161550 2024-09-16T07:29:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52b1400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f52ac1626e0 con 0x7f52ac0af3d0 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52aa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 0x7f52ac1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52aa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 0x7f52ac1f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48564/0 (socket says 172.21.15.71:48564) 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52aa800640 1 -- 172.21.15.71:0/1795012096 learned_addr learned my addr 172.21.15.71:0/1795012096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52a1e00640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52ac0af3d0 0x7f52ac14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.175+0000 7f52ab200640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52ac161550 0x7f52ac155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.176+0000 7f52aa800640 1 -- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52ac0af3d0 msgr2=0x7f52ac14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.176+0000 7f52aa800640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52ac0af3d0 0x7f52ac14fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.176+0000 7f52aa800640 1 -- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52ac161550 msgr2=0x7f52ac155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.176+0000 7f52aa800640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52ac161550 0x7f52ac155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.176+0000 7f52aa800640 1 -- 172.21.15.71:0/1795012096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac156320 con 0x7f52ac06f9d0 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.176+0000 7f52a1e00640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52ac0af3d0 0x7f52ac14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:29:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.176+0000 7f52ab200640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52ac161550 0x7f52ac155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:29:41.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.176+0000 7f52aa800640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 0x7f52ac1f8350 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f529c00e7c0 tx=0x7f529c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:41.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.178+0000 7f52a3400640 1 -- 172.21.15.71:0/1795012096 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c00ccc0 con 0x7f52ac06f9d0 2024-09-16T07:29:41.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.178+0000 7f52a3400640 1 -- 172.21.15.71:0/1795012096 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f529c00ce80 con 0x7f52ac06f9d0 2024-09-16T07:29:41.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.178+0000 7f52a3400640 1 -- 172.21.15.71:0/1795012096 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c005280 con 0x7f52ac06f9d0 2024-09-16T07:29:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.178+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac1f89e0 con 0x7f52ac06f9d0 2024-09-16T07:29:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.178+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac1f8fa0 con 0x7f52ac06f9d0 2024-09-16T07:29:41.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.179+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ac15f420 con 0x7f52ac06f9d0 2024-09-16T07:29:41.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.180+0000 7f52a3400640 1 -- 172.21.15.71:0/1795012096 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f529c005420 con 0x7f52ac06f9d0 2024-09-16T07:29:41.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.181+0000 7f52a3400640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5288079470 0x7f528807b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:41.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.181+0000 7f52a3400640 1 -- 172.21.15.71:0/1795012096 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4971+0+0 (secure 0 0 0) 0x7f529c099b50 con 0x7f52ac06f9d0 2024-09-16T07:29:41.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.182+0000 7f52a1e00640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5288079470 0x7f528807b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:41.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.182+0000 7f52a1e00640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5288079470 0x7f528807b930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f52ac09d0f0 tx=0x7f5294005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:41.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.185+0000 7f52a3400640 1 -- 172.21.15.71:0/1795012096 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f52ac15f420 con 0x7f52ac06f9d0 2024-09-16T07:29:41.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:41.380+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f52ac161c90 con 0x7f52ac06f9d0 2024-09-16T07:29:42.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.177+0000 7f52a3400640 1 -- 172.21.15.71:0/1795012096 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) ==== 97+0+0 (secure 0 0 0) 0x7f529c0635c0 con 0x7f52ac06f9d0 2024-09-16T07:29:42.177 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:29:42.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.180+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5288079470 msgr2=0x7f528807b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.180+0000 7f52b1400640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5288079470 0x7f528807b930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f52ac09d0f0 tx=0x7f5294005f90 comp rx=0 tx=0).stop 2024-09-16T07:29:42.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.180+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 msgr2=0x7f52ac1f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.180+0000 7f52b1400640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 0x7f52ac1f8350 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f529c00e7c0 tx=0x7f529c00ec80 comp rx=0 tx=0).stop 2024-09-16T07:29:42.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.181+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 shutdown_connections 2024-09-16T07:29:42.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.181+0000 7f52b1400640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5288079470 0x7f528807b930 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.181+0000 7f52b1400640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52ac161550 0x7f52ac155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.181+0000 7f52b1400640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52ac0af3d0 0x7f52ac14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.181+0000 7f52b1400640 1 --2- 172.21.15.71:0/1795012096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52ac06f9d0 0x7f52ac1f8350 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.181+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 >> 172.21.15.71:0/1795012096 conn(0x7f52ac0b1950 msgr2=0x7f52ac0b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:42.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.181+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 shutdown_connections 2024-09-16T07:29:42.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.181+0000 7f52b1400640 1 -- 172.21.15.71:0/1795012096 wait complete. 2024-09-16T07:29:42.199 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:29:42.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.356+0000 7f1fd7e00640 1 -- 172.21.15.71:0/4015407207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd80725d0 msgr2=0x7f1fd80b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.356+0000 7f1fd7e00640 1 --2- 172.21.15.71:0/4015407207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd80725d0 0x7f1fd80b28f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005fd0 tx=0x7f1fc402a5d0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.357+0000 7f1fd7e00640 1 -- 172.21.15.71:0/4015407207 shutdown_connections 2024-09-16T07:29:42.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.357+0000 7f1fd7e00640 1 --2- 172.21.15.71:0/4015407207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd80725d0 0x7f1fd80b28f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.357+0000 7f1fd7e00640 1 -- 172.21.15.71:0/4015407207 >> 172.21.15.71:0/4015407207 conn(0x7f1fd80cfa20 msgr2=0x7f1fd80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:42.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.357+0000 7f1fd7e00640 1 -- 172.21.15.71:0/4015407207 shutdown_connections 2024-09-16T07:29:42.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.357+0000 7f1fd7e00640 1 -- 172.21.15.71:0/4015407207 wait complete. 2024-09-16T07:29:42.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.358+0000 7f1fd7e00640 1 Processor -- start 2024-09-16T07:29:42.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.358+0000 7f1fd7e00640 1 -- start start 2024-09-16T07:29:42.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.359+0000 7f1fd7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd80d0540 0x7f1fd80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:42.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.359+0000 7f1fd7e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd815bc30 0x7f1fd81512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:42.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.359+0000 7f1fd7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd8151a30 0x7f1fd815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:42.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.359+0000 7f1fd7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8074660 con 0x7f1fd8151a30 2024-09-16T07:29:42.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.359+0000 7f1fd7e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd806fa30 con 0x7f1fd80d0540 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.359+0000 7f1fd7e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd81522d0 con 0x7f1fd815bc30 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.359+0000 7f1fd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd8151a30 0x7f1fd815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd8151a30 0x7f1fd815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48588/0 (socket says 172.21.15.71:48588) 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd7400640 1 -- 172.21.15.71:0/691982839 learned_addr learned my addr 172.21.15.71:0/691982839 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd6a00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd80d0540 0x7f1fd80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd6000640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd815bc30 0x7f1fd81512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd7400640 1 -- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd815bc30 msgr2=0x7f1fd81512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd7400640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd815bc30 0x7f1fd81512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd7400640 1 -- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd80d0540 msgr2=0x7f1fd80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd7400640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd80d0540 0x7f1fd80cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd7400640 1 -- 172.21.15.71:0/691982839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd8151830 con 0x7f1fd8151a30 2024-09-16T07:29:42.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd6000640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd815bc30 0x7f1fd81512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:42.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fd7400640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd8151a30 0x7f1fd815efd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c780 tx=0x7f1fd000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:42.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fcea00640 1 -- 172.21.15.71:0/691982839 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0004b60 con 0x7f1fd8151a30 2024-09-16T07:29:42.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.360+0000 7f1fcea00640 1 -- 172.21.15.71:0/691982839 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fd0004df0 con 0x7f1fd8151a30 2024-09-16T07:29:42.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.361+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd815f510 con 0x7f1fd8151a30 2024-09-16T07:29:42.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.361+0000 7f1fcea00640 1 -- 172.21.15.71:0/691982839 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000aa10 con 0x7f1fd8151a30 2024-09-16T07:29:42.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.361+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd815d300 con 0x7f1fd8151a30 2024-09-16T07:29:42.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.362+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd806a180 con 0x7f1fd8151a30 2024-09-16T07:29:42.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.363+0000 7f1fcea00640 1 -- 172.21.15.71:0/691982839 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f1fd000abb0 con 0x7f1fd8151a30 2024-09-16T07:29:42.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.364+0000 7f1fcea00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1fb8079450 0x7f1fb807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:42.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.364+0000 7f1fcea00640 1 -- 172.21.15.71:0/691982839 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4885+0+0 (secure 0 0 0) 0x7f1fd00995e0 con 0x7f1fd8151a30 2024-09-16T07:29:42.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.364+0000 7f1fd6a00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1fb8079450 0x7f1fb807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:42.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.365+0000 7f1fd6a00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1fb8079450 0x7f1fb807b910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8001720 tx=0x7f1fc8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.368+0000 7f1fcea00640 1 -- 172.21.15.71:0/691982839 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1fd806a180 con 0x7f1fd8151a30 2024-09-16T07:29:42.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.617+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1fd80b28f0 con 0x7f1fd8151a30 2024-09-16T07:29:42.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.618+0000 7f1fcea00640 1 -- 172.21.15.71:0/691982839 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f1fd0020070 con 0x7f1fd8151a30 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:29:42.619 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:29:42.620 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:29:42.622 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:29:42.624 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:29:42.625 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:29:42.626 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:29:42.627 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:29:42.629 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:29:42.631 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:29:42.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1fb8079450 msgr2=0x7f1fb807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1fb8079450 0x7f1fb807b910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8001720 tx=0x7f1fc8009040 comp rx=0 tx=0).stop 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd8151a30 msgr2=0x7f1fd815efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd8151a30 0x7f1fd815efd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000c780 tx=0x7f1fd000cc40 comp rx=0 tx=0).stop 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 shutdown_connections 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1fb8079450 0x7f1fb807b910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd8151a30 0x7f1fd815efd0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd815bc30 0x7f1fd81512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 --2- 172.21.15.71:0/691982839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd80d0540 0x7f1fd80cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.620+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 >> 172.21.15.71:0/691982839 conn(0x7f1fd80cfa20 msgr2=0x7f1fd80b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.621+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 shutdown_connections 2024-09-16T07:29:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.621+0000 7f1fd7e00640 1 -- 172.21.15.71:0/691982839 wait complete. 2024-09-16T07:29:42.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:29:42.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.793+0000 7f0912600640 1 -- 172.21.15.71:0/4026057676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08f4004d40 msgr2=0x7f08f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.793+0000 7f0912600640 1 --2- 172.21.15.71:0/4026057676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08f4004d40 0x7f08f40051c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0904009ed0 tx=0x7f08f4005830 comp rx=0 tx=0).stop 2024-09-16T07:29:42.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.793+0000 7f0911c00640 1 -- 172.21.15.71:0/4026057676 reap_dead start 2024-09-16T07:29:42.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.794+0000 7f0912600640 1 -- 172.21.15.71:0/4026057676 shutdown_connections 2024-09-16T07:29:42.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.794+0000 7f0912600640 1 -- 172.21.15.71:0/4026057676 >> 172.21.15.71:0/4026057676 conn(0x7f090c0d3de0 msgr2=0x7f090c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:42.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.794+0000 7f0912600640 1 -- 172.21.15.71:0/4026057676 shutdown_connections 2024-09-16T07:29:42.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.794+0000 7f0912600640 1 -- 172.21.15.71:0/4026057676 wait complete. 2024-09-16T07:29:42.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.795+0000 7f0912600640 1 Processor -- start 2024-09-16T07:29:42.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.795+0000 7f0912600640 1 -- start start 2024-09-16T07:29:42.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.796+0000 7f0912600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08f4004d40 0x7f090c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:42.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.796+0000 7f0912600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090c0c9640 0x7f090c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:42.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.796+0000 7f0912600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f090c1f8050 0x7f090c1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:42.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.796+0000 7f0912600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f090c0d2630 con 0x7f090c1f8050 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.796+0000 7f0912600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f090c0d24b0 con 0x7f08f4004d40 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.796+0000 7f0912600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f090c162e90 con 0x7f090c0c9640 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f090c1f8050 0x7f090c1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f090c1f8050 0x7f090c1f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48602/0 (socket says 172.21.15.71:48602) 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 -- 172.21.15.71:0/4114868066 learned_addr learned my addr 172.21.15.71:0/4114868066 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f090be00640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090c0c9640 0x7f090c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911200640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08f4004d40 0x7f090c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 -- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090c0c9640 msgr2=0x7f090c15c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090c0c9640 0x7f090c15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 -- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08f4004d40 msgr2=0x7f090c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08f4004d40 0x7f090c154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 -- 172.21.15.71:0/4114868066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090c15d4f0 con 0x7f090c1f8050 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911200640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08f4004d40 0x7f090c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:42.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.797+0000 7f0911c00640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f090c1f8050 0x7f090c1f84d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f090402af20 tx=0x7f0904033da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:42.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.798+0000 7f0908c00640 1 -- 172.21.15.71:0/4114868066 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904042070 con 0x7f090c1f8050 2024-09-16T07:29:42.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.798+0000 7f0908c00640 1 -- 172.21.15.71:0/4114868066 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0904003b70 con 0x7f090c1f8050 2024-09-16T07:29:42.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.798+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090c15cf30 con 0x7f090c1f8050 2024-09-16T07:29:42.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.798+0000 7f0908c00640 1 -- 172.21.15.71:0/4114868066 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904038040 con 0x7f090c1f8050 2024-09-16T07:29:42.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.798+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090c15c8c0 con 0x7f090c1f8050 2024-09-16T07:29:42.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.799+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f090c162020 con 0x7f090c1f8050 2024-09-16T07:29:42.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.800+0000 7f0908c00640 1 -- 172.21.15.71:0/4114868066 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f0904004030 con 0x7f090c1f8050 2024-09-16T07:29:42.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.801+0000 7f0908c00640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f08ec079420 0x7f08ec07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:42.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.801+0000 7f0908c00640 1 -- 172.21.15.71:0/4114868066 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4885+0+0 (secure 0 0 0) 0x7f09040b9910 con 0x7f090c1f8050 2024-09-16T07:29:42.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.801+0000 7f0911200640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f08ec079420 0x7f08ec07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:42.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.802+0000 7f0911200640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f08ec079420 0x7f08ec07b8e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0900001860 tx=0x7f09000067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:42.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.805+0000 7f0908c00640 1 -- 172.21.15.71:0/4114868066 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0904083060 con 0x7f090c1f8050 2024-09-16T07:29:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.991+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f090c0c9a20 con 0x7f090c1f8050 2024-09-16T07:29:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.991+0000 7f0908c00640 1 -- 172.21.15.71:0/4114868066 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f0904087a90 con 0x7f090c1f8050 2024-09-16T07:29:42.991 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:42.991 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:29:42.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:42.991 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:29:42.991 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:29:42.991 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:42.991 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:29:42.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f08ec079420 msgr2=0x7f08ec07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f08ec079420 0x7f08ec07b8e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0900001860 tx=0x7f09000067a0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f090c1f8050 msgr2=0x7f090c1f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f090c1f8050 0x7f090c1f84d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f090402af20 tx=0x7f0904033da0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 shutdown_connections 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f08ec079420 0x7f08ec07b8e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f090c1f8050 0x7f090c1f84d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090c0c9640 0x7f090c15c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 --2- 172.21.15.71:0/4114868066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08f4004d40 0x7f090c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 >> 172.21.15.71:0/4114868066 conn(0x7f090c0d3de0 msgr2=0x7f090c157460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 shutdown_connections 2024-09-16T07:29:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:42.994+0000 7f0912600640 1 -- 172.21.15.71:0/4114868066 wait complete. 2024-09-16T07:29:43.009 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:29:43.010 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:29:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.171+0000 7f65e1600640 1 -- 172.21.15.71:0/3199978536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15f9a0 msgr2=0x7f65dc15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.171+0000 7f65e1600640 1 --2- 172.21.15.71:0/3199978536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15f9a0 0x7f65dc15fd80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0702a0 tx=0x7f65cc02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:29:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.172+0000 7f65e1600640 1 -- 172.21.15.71:0/3199978536 shutdown_connections 2024-09-16T07:29:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.172+0000 7f65e1600640 1 --2- 172.21.15.71:0/3199978536 >> v2:172.21.15.177:3300/0 conn(0x7f65dc15c3f0 0x7f65dc15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.172+0000 7f65e1600640 1 --2- 172.21.15.71:0/3199978536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc073de0 0x7f65dc074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.172+0000 7f65e1600640 1 --2- 172.21.15.71:0/3199978536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15f9a0 0x7f65dc15fd80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.172+0000 7f65e1600640 1 -- 172.21.15.71:0/3199978536 >> 172.21.15.71:0/3199978536 conn(0x7f65dc0b18e0 msgr2=0x7f65dc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.172+0000 7f65e1600640 1 -- 172.21.15.71:0/3199978536 shutdown_connections 2024-09-16T07:29:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.172+0000 7f65e1600640 1 -- 172.21.15.71:0/3199978536 wait complete. 2024-09-16T07:29:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.173+0000 7f65e1600640 1 Processor -- start 2024-09-16T07:29:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.173+0000 7f65e1600640 1 -- start start 2024-09-16T07:29:43.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.174+0000 7f65e1600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc073de0 0x7f65dc1f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:43.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.174+0000 7f65e1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15c3f0 0x7f65dc152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:43.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.174+0000 7f65e1600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc15f9a0 0x7f65dc0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.174+0000 7f65e1600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc0cd3d0 con 0x7f65dc15c3f0 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.174+0000 7f65e1600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc0cc970 con 0x7f65dc073de0 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.174+0000 7f65e1600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f65dc0cd550 con 0x7f65dc15f9a0 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15c3f0 0x7f65dc152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15c3f0 0x7f65dc152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48620/0 (socket says 172.21.15.71:48620) 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 -- 172.21.15.71:0/906739489 learned_addr learned my addr 172.21.15.71:0/906739489 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65daa00640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc073de0 0x7f65dc1f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65db400640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc15f9a0 0x7f65dc0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 -- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc15f9a0 msgr2=0x7f65dc0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc15f9a0 0x7f65dc0c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 -- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc073de0 msgr2=0x7f65dc1f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc073de0 0x7f65dc1f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 -- 172.21.15.71:0/906739489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc157c60 con 0x7f65dc15c3f0 2024-09-16T07:29:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65db400640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc15f9a0 0x7f65dc0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:43.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.175+0000 7f65da000640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15c3f0 0x7f65dc152230 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f65c800d720 tx=0x7f65c800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:43.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.176+0000 7f65d2a00640 1 -- 172.21.15.71:0/906739489 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8017070 con 0x7f65dc15c3f0 2024-09-16T07:29:43.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.176+0000 7f65d2a00640 1 -- 172.21.15.71:0/906739489 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65c8009d20 con 0x7f65dc15c3f0 2024-09-16T07:29:43.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.176+0000 7f65d2a00640 1 -- 172.21.15.71:0/906739489 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c80106c0 con 0x7f65dc15c3f0 2024-09-16T07:29:43.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.176+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc157f40 con 0x7f65dc15c3f0 2024-09-16T07:29:43.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.176+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc158500 con 0x7f65dc15c3f0 2024-09-16T07:29:43.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.177+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc069360 con 0x7f65dc15c3f0 2024-09-16T07:29:43.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.178+0000 7f65d2a00640 1 -- 172.21.15.71:0/906739489 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f65c801a070 con 0x7f65dc15c3f0 2024-09-16T07:29:43.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.179+0000 7f65d2a00640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc079450 0x7f65bc07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:43.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.179+0000 7f65d2a00640 1 -- 172.21.15.71:0/906739489 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4885+0+0 (secure 0 0 0) 0x7f65c8099730 con 0x7f65dc15c3f0 2024-09-16T07:29:43.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.179+0000 7f65daa00640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc079450 0x7f65bc07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:43.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.180+0000 7f65daa00640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc079450 0x7f65bc07b910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65cc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:43.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.183+0000 7f65d2a00640 1 -- 172.21.15.71:0/906739489 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f65c8063590 con 0x7f65dc15c3f0 2024-09-16T07:29:43.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:43.370+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f65dc1586d0 con 0x7f65dc15c3f0 2024-09-16T07:29:44.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.186+0000 7f65d2a00640 1 -- 172.21.15.71:0/906739489 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f65c8062e90 con 0x7f65dc15c3f0 2024-09-16T07:29:44.186 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:29:44.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc079450 msgr2=0x7f65bc07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:44.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc079450 0x7f65bc07b910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65cc004730 comp rx=0 tx=0).stop 2024-09-16T07:29:44.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15c3f0 msgr2=0x7f65dc152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:44.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15c3f0 0x7f65dc152230 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f65c800d720 tx=0x7f65c800dbe0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 shutdown_connections 2024-09-16T07:29:44.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65bc079450 0x7f65bc07b910 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65dc15f9a0 0x7f65dc0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65dc15c3f0 0x7f65dc152230 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 --2- 172.21.15.71:0/906739489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65dc073de0 0x7f65dc1f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.190+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 >> 172.21.15.71:0/906739489 conn(0x7f65dc0b18e0 msgr2=0x7f65dc15b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:44.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.191+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 shutdown_connections 2024-09-16T07:29:44.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.191+0000 7f65e1600640 1 -- 172.21.15.71:0/906739489 wait complete. 2024-09-16T07:29:44.212 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:29:44.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.380+0000 7efc86a00640 1 -- 172.21.15.71:0/115636737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc80145750 msgr2=0x7efc80073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:44.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.380+0000 7efc86a00640 1 --2- 172.21.15.71:0/115636737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc80145750 0x7efc80073440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efc70009830 tx=0x7efc7002a480 comp rx=0 tx=0).stop 2024-09-16T07:29:44.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.380+0000 7efc86a00640 1 -- 172.21.15.71:0/115636737 shutdown_connections 2024-09-16T07:29:44.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.380+0000 7efc86a00640 1 --2- 172.21.15.71:0/115636737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc800cccd0 0x7efc800cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.380+0000 7efc86a00640 1 --2- 172.21.15.71:0/115636737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc80145750 0x7efc80073440 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.380+0000 7efc86a00640 1 -- 172.21.15.71:0/115636737 >> 172.21.15.71:0/115636737 conn(0x7efc800b1840 msgr2=0x7efc800b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:44.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.380+0000 7efc86a00640 1 -- 172.21.15.71:0/115636737 shutdown_connections 2024-09-16T07:29:44.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.380+0000 7efc86a00640 1 -- 172.21.15.71:0/115636737 wait complete. 2024-09-16T07:29:44.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.382+0000 7efc86a00640 1 Processor -- start 2024-09-16T07:29:44.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.382+0000 7efc86a00640 1 -- start start 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc86a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc8006efa0 0x7efc801f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc86a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc800cccd0 0x7efc801428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc86a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc80145750 0x7efc800ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc86a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc800ce480 con 0x7efc800cccd0 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc86a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efc800ce300 con 0x7efc80145750 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc86a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efc800ce600 con 0x7efc8006efa0 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc7f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc800cccd0 0x7efc801428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc7f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc800cccd0 0x7efc801428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48636/0 (socket says 172.21.15.71:48636) 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc7f400640 1 -- 172.21.15.71:0/253444785 learned_addr learned my addr 172.21.15.71:0/253444785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc7fe00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc8006efa0 0x7efc801f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.383+0000 7efc84a00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc80145750 0x7efc800ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc7f400640 1 -- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc8006efa0 msgr2=0x7efc801f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc7f400640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc8006efa0 0x7efc801f2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc7f400640 1 -- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc80145750 msgr2=0x7efc800ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc7f400640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc80145750 0x7efc800ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc7f400640 1 -- 172.21.15.71:0/253444785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc800aa8a0 con 0x7efc800cccd0 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc7f400640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc800cccd0 0x7efc801428a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c450 tx=0x7efc6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc77e00640 1 -- 172.21.15.71:0/253444785 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c016020 con 0x7efc800cccd0 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc77e00640 1 -- 172.21.15.71:0/253444785 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc6c012920 con 0x7efc800cccd0 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc800aab40 con 0x7efc800cccd0 2024-09-16T07:29:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc77e00640 1 -- 172.21.15.71:0/253444785 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c012d70 con 0x7efc800cccd0 2024-09-16T07:29:44.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.384+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc800ab100 con 0x7efc800cccd0 2024-09-16T07:29:44.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.386+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc800cdbb0 con 0x7efc800cccd0 2024-09-16T07:29:44.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.387+0000 7efc77e00640 1 -- 172.21.15.71:0/253444785 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7efc6c019020 con 0x7efc800cccd0 2024-09-16T07:29:44.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.388+0000 7efc77e00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc60079420 0x7efc6007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:44.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.388+0000 7efc77e00640 1 -- 172.21.15.71:0/253444785 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5255+0+0 (secure 0 0 0) 0x7efc6c099630 con 0x7efc800cccd0 2024-09-16T07:29:44.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.388+0000 7efc7fe00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc60079420 0x7efc6007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:44.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.389+0000 7efc7fe00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc60079420 0x7efc6007b8e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efc70034000 tx=0x7efc70004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:44.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.392+0000 7efc77e00640 1 -- 172.21.15.71:0/253444785 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc6c020030 con 0x7efc800cccd0 2024-09-16T07:29:44.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:44.578+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efc800cf740 con 0x7efc800cccd0 2024-09-16T07:29:44.901 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:29:44.905 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:29:45.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.192+0000 7efc77e00640 1 -- 172.21.15.71:0/253444785 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7efc6c062e40 con 0x7efc800cccd0 2024-09-16T07:29:45.192 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:29:45.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc60079420 msgr2=0x7efc6007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:45.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc60079420 0x7efc6007b8e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efc70034000 tx=0x7efc70004730 comp rx=0 tx=0).stop 2024-09-16T07:29:45.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc800cccd0 msgr2=0x7efc801428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:45.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc800cccd0 0x7efc801428a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c450 tx=0x7efc6c00c910 comp rx=0 tx=0).stop 2024-09-16T07:29:45.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 shutdown_connections 2024-09-16T07:29:45.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc60079420 0x7efc6007b8e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:45.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc80145750 0x7efc800ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:45.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc800cccd0 0x7efc801428a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:45.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 --2- 172.21.15.71:0/253444785 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc8006efa0 0x7efc801f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:45.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.195+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 >> 172.21.15.71:0/253444785 conn(0x7efc800b1840 msgr2=0x7efc800a9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:45.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.196+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 shutdown_connections 2024-09-16T07:29:45.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.196+0000 7efc86a00640 1 -- 172.21.15.71:0/253444785 wait complete. 2024-09-16T07:29:45.227 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:29:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.385+0000 7fde9f200640 1 -- 172.21.15.71:0/1776365303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980d2a00 msgr2=0x7fde980b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.385+0000 7fde9f200640 1 --2- 172.21.15.71:0/1776365303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980d2a00 0x7fde980b38f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fde980d1ce0 tx=0x7fde9402a820 comp rx=0 tx=0).stop 2024-09-16T07:29:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.386+0000 7fde9f200640 1 -- 172.21.15.71:0/1776365303 shutdown_connections 2024-09-16T07:29:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.386+0000 7fde9f200640 1 --2- 172.21.15.71:0/1776365303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980d2a00 0x7fde980b38f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.386+0000 7fde9f200640 1 --2- 172.21.15.71:0/1776365303 >> v2:172.21.15.177:3300/0 conn(0x7fde980cf830 0x7fde980b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.386+0000 7fde9f200640 1 -- 172.21.15.71:0/1776365303 >> 172.21.15.71:0/1776365303 conn(0x7fde9809d0b0 msgr2=0x7fde980d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.386+0000 7fde9f200640 1 -- 172.21.15.71:0/1776365303 shutdown_connections 2024-09-16T07:29:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.386+0000 7fde9f200640 1 -- 172.21.15.71:0/1776365303 wait complete. 2024-09-16T07:29:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.387+0000 7fde9f200640 1 Processor -- start 2024-09-16T07:29:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.387+0000 7fde9f200640 1 -- start start 2024-09-16T07:29:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980cf830 0x7fde9815d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde980d2a00 0x7fde980b9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9f200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde98161460 0x7fde980ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde980b50c0 con 0x7fde980cf830 2024-09-16T07:29:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde980b4f40 con 0x7fde980d2a00 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9f200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fde980b3e30 con 0x7fde98161460 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980cf830 0x7fde9815d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980cf830 0x7fde9815d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33050/0 (socket says 172.21.15.71:33050) 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.388+0000 7fde9de00640 1 -- 172.21.15.71:0/1188287757 learned_addr learned my addr 172.21.15.71:0/1188287757 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9e800640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde98161460 0x7fde980ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9d400640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde980d2a00 0x7fde980b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9de00640 1 -- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde98161460 msgr2=0x7fde980ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9de00640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde98161460 0x7fde980ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9de00640 1 -- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde980d2a00 msgr2=0x7fde980b9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9de00640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde980d2a00 0x7fde980b9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9de00640 1 -- 172.21.15.71:0/1188287757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde980b8170 con 0x7fde980cf830 2024-09-16T07:29:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9e800640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde98161460 0x7fde980ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:45.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9de00640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980cf830 0x7fde9815d7e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c3a0 tx=0x7fde8c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:45.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde92000640 1 -- 172.21.15.71:0/1188287757 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c011790 con 0x7fde980cf830 2024-09-16T07:29:45.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde92000640 1 -- 172.21.15.71:0/1188287757 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde8c017070 con 0x7fde980cf830 2024-09-16T07:29:45.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde92000640 1 -- 172.21.15.71:0/1188287757 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c011cc0 con 0x7fde980cf830 2024-09-16T07:29:45.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.389+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde980b8450 con 0x7fde980cf830 2024-09-16T07:29:45.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.390+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde980b8a10 con 0x7fde980cf830 2024-09-16T07:29:45.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.390+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde980d19a0 con 0x7fde980cf830 2024-09-16T07:29:45.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.392+0000 7fde92000640 1 -- 172.21.15.71:0/1188287757 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fde8c01c460 con 0x7fde980cf830 2024-09-16T07:29:45.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.393+0000 7fde92000640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde7c079420 0x7fde7c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:45.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.393+0000 7fde92000640 1 -- 172.21.15.71:0/1188287757 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5637+0+0 (secure 0 0 0) 0x7fde8c099d30 con 0x7fde980cf830 2024-09-16T07:29:45.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.393+0000 7fde9d400640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde7c079420 0x7fde7c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:45.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.394+0000 7fde9d400640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde7c079420 0x7fde7c07b8e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fde981f7b30 tx=0x7fde8800b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:45.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.396+0000 7fde92000640 1 -- 172.21.15.71:0/1188287757 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fde980d19a0 con 0x7fde980cf830 2024-09-16T07:29:45.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:45.617+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fde980b42e0 con 0x7fde980cf830 2024-09-16T07:29:46.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.193+0000 7fde92000640 1 -- 172.21.15.71:0/1188287757 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7fde8c063b10 con 0x7fde980cf830 2024-09-16T07:29:46.194 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 10 and data pool 11 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde7c079420 msgr2=0x7fde7c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde7c079420 0x7fde7c07b8e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fde981f7b30 tx=0x7fde8800b390 comp rx=0 tx=0).stop 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980cf830 msgr2=0x7fde9815d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980cf830 0x7fde9815d7e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c3a0 tx=0x7fde8c00c860 comp rx=0 tx=0).stop 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 shutdown_connections 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde7c079420 0x7fde7c07b8e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde98161460 0x7fde980ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde980d2a00 0x7fde980b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 --2- 172.21.15.71:0/1188287757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde980cf830 0x7fde9815d7e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 >> 172.21.15.71:0/1188287757 conn(0x7fde9809d0b0 msgr2=0x7fde980b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:46.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.196+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 shutdown_connections 2024-09-16T07:29:46.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.197+0000 7fde9f200640 1 -- 172.21.15.71:0/1188287757 wait complete. 2024-09-16T07:29:46.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:29:46.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.395+0000 7f82b5200640 1 -- 172.21.15.71:0/4075252929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8294003a00 msgr2=0x7f8294003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.395+0000 7f82b5200640 1 --2- 172.21.15.71:0/4075252929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8294003a00 0x7f8294003e20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f82a000c260 tx=0x7f8294005830 comp rx=0 tx=0).stop 2024-09-16T07:29:46.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.395+0000 7f82afe00640 1 -- 172.21.15.71:0/4075252929 reap_dead start 2024-09-16T07:29:46.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.396+0000 7f82b5200640 1 -- 172.21.15.71:0/4075252929 shutdown_connections 2024-09-16T07:29:46.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.396+0000 7f82b5200640 1 -- 172.21.15.71:0/4075252929 >> 172.21.15.71:0/4075252929 conn(0x7f82b00d3df0 msgr2=0x7f82b00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:46.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.396+0000 7f82b5200640 1 -- 172.21.15.71:0/4075252929 shutdown_connections 2024-09-16T07:29:46.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.396+0000 7f82b5200640 1 -- 172.21.15.71:0/4075252929 wait complete. 2024-09-16T07:29:46.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.397+0000 7f82b5200640 1 Processor -- start 2024-09-16T07:29:46.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.397+0000 7f82b5200640 1 -- start start 2024-09-16T07:29:46.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.398+0000 7f82b5200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8294003a00 0x7f82b0152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.398+0000 7f82b5200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82b00d08e0 0x7f82b00b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.398+0000 7f82b5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82b00cc3c0 0x7f82b00b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.398+0000 7f82b5200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00b1b00 con 0x7f82b00cc3c0 2024-09-16T07:29:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.398+0000 7f82b5200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00b1980 con 0x7f82b00d08e0 2024-09-16T07:29:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.398+0000 7f82b5200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f82b00b1c80 con 0x7f8294003a00 2024-09-16T07:29:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82b00cc3c0 0x7f82b00b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82b00cc3c0 0x7f82b00b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33074/0 (socket says 172.21.15.71:33074) 2024-09-16T07:29:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 -- 172.21.15.71:0/855676223 learned_addr learned my addr 172.21.15.71:0/855676223 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:46.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82aea00640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82b00d08e0 0x7f82b00b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:46.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82af400640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8294003a00 0x7f82b0152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:46.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 -- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8294003a00 msgr2=0x7f82b0152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8294003a00 0x7f82b0152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 -- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82b00d08e0 msgr2=0x7f82b00b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82b00d08e0 0x7f82b00b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 -- 172.21.15.71:0/855676223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b00cc870 con 0x7f82b00cc3c0 2024-09-16T07:29:46.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.399+0000 7f82afe00640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82b00cc3c0 0x7f82b00b3880 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f82a8005fd0 tx=0x7f82a8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:46.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.400+0000 7f82a7400640 1 -- 172.21.15.71:0/855676223 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a800fcb0 con 0x7f82b00cc3c0 2024-09-16T07:29:46.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.400+0000 7f82a7400640 1 -- 172.21.15.71:0/855676223 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a800fe50 con 0x7f82b00cc3c0 2024-09-16T07:29:46.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.400+0000 7f82a7400640 1 -- 172.21.15.71:0/855676223 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8004d60 con 0x7f82b00cc3c0 2024-09-16T07:29:46.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.400+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b00ccb50 con 0x7f82b00cc3c0 2024-09-16T07:29:46.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.400+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b00cd110 con 0x7f82b00cc3c0 2024-09-16T07:29:46.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.401+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b00b2210 con 0x7f82b00cc3c0 2024-09-16T07:29:46.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.402+0000 7f82a7400640 1 -- 172.21.15.71:0/855676223 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f82a8020020 con 0x7f82b00cc3c0 2024-09-16T07:29:46.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.403+0000 7f82a7400640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8290079420 0x7f829007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:46.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.403+0000 7f82a7400640 1 -- 172.21.15.71:0/855676223 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5737+0+0 (secure 0 0 0) 0x7f82a8099c30 con 0x7f82b00cc3c0 2024-09-16T07:29:46.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.403+0000 7f82af400640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8290079420 0x7f829007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:46.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.404+0000 7f82af400640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8290079420 0x7f829007b8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f82a0033000 tx=0x7f82a0038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:46.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.407+0000 7f82a7400640 1 -- 172.21.15.71:0/855676223 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f82a8063a90 con 0x7f82b00cc3c0 2024-09-16T07:29:46.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.592+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82b01570b0 con 0x7f82b00cc3c0 2024-09-16T07:29:46.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.593+0000 7f82a7400640 1 -- 172.21.15.71:0/855676223 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16736 (secure 0 0 0) 0x7f82a8063390 con 0x7f82b00cc3c0 2024-09-16T07:29:46.594 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:46.594 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:29:45.618504+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:29:43.380127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T07:29:44.588310+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:29:46.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8290079420 msgr2=0x7f829007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8290079420 0x7f829007b8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f82a0033000 tx=0x7f82a0038040 comp rx=0 tx=0).stop 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82b00cc3c0 msgr2=0x7f82b00b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82b00cc3c0 0x7f82b00b3880 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f82a8005fd0 tx=0x7f82a8011ed0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 shutdown_connections 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8290079420 0x7f829007b8e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82b00cc3c0 0x7f82b00b3880 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82b00d08e0 0x7f82b00b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 --2- 172.21.15.71:0/855676223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8294003a00 0x7f82b0152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 >> 172.21.15.71:0/855676223 conn(0x7f82b00d3df0 msgr2=0x7f82b0160a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 shutdown_connections 2024-09-16T07:29:46.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.596+0000 7f82b5200640 1 -- 172.21.15.71:0/855676223 wait complete. 2024-09-16T07:29:46.612 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:29:46.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.769+0000 7fe1a3200640 1 -- 172.21.15.71:0/1091861804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a40cd5f0 msgr2=0x7fe1a40cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.769+0000 7fe1a3200640 1 --2- 172.21.15.71:0/1091861804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a40cd5f0 0x7fe1a40cda80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe19c005fd0 tx=0x7fe19c02a8b0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.769+0000 7fe1a3200640 1 -- 172.21.15.71:0/1091861804 shutdown_connections 2024-09-16T07:29:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.769+0000 7fe1a3200640 1 --2- 172.21.15.71:0/1091861804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a40cd5f0 0x7fe1a40cda80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.769+0000 7fe1a3200640 1 --2- 172.21.15.71:0/1091861804 >> v2:172.21.15.177:3300/0 conn(0x7fe1a40ad880 0x7fe1a40adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.769+0000 7fe1a3200640 1 -- 172.21.15.71:0/1091861804 >> 172.21.15.71:0/1091861804 conn(0x7fe1a40cfa20 msgr2=0x7fe1a40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.770+0000 7fe1a3200640 1 -- 172.21.15.71:0/1091861804 shutdown_connections 2024-09-16T07:29:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.770+0000 7fe1a3200640 1 -- 172.21.15.71:0/1091861804 wait complete. 2024-09-16T07:29:46.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.771+0000 7fe1a3200640 1 Processor -- start 2024-09-16T07:29:46.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.771+0000 7fe1a3200640 1 -- start start 2024-09-16T07:29:46.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe1a3200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a406fb90 0x7fe1a40c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:46.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe1a3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1a40ad880 0x7fe1a40b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:46.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe1a3200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1a40cd5f0 0x7fe1a41f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:46.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe1a3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a415edf0 con 0x7fe1a40ad880 2024-09-16T07:29:46.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe1a3200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a415ec70 con 0x7fe1a40cd5f0 2024-09-16T07:29:46.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe1a3200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a415ef70 con 0x7fe1a406fb90 2024-09-16T07:29:46.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe199400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1a40ad880 0x7fe1a40b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe199400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1a40ad880 0x7fe1a40b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33078/0 (socket says 172.21.15.71:33078) 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.772+0000 7fe199400640 1 -- 172.21.15.71:0/3586809421 learned_addr learned my addr 172.21.15.71:0/3586809421 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe1a1e00640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a406fb90 0x7fe1a40c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe1a2800640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1a40cd5f0 0x7fe1a41f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe199400640 1 -- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a406fb90 msgr2=0x7fe1a40c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe199400640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a406fb90 0x7fe1a40c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe199400640 1 -- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1a40cd5f0 msgr2=0x7fe1a41f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe199400640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1a40cd5f0 0x7fe1a41f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe199400640 1 -- 172.21.15.71:0/3586809421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a40b2fe0 con 0x7fe1a40ad880 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe1a1e00640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a406fb90 0x7fe1a40c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe1a2800640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1a40cd5f0 0x7fe1a41f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe199400640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1a40ad880 0x7fe1a40b0770 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00ea90 tx=0x7fe18c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:46.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.773+0000 7fe19aa00640 1 -- 172.21.15.71:0/3586809421 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00ce40 con 0x7fe1a40ad880 2024-09-16T07:29:46.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.774+0000 7fe19aa00640 1 -- 172.21.15.71:0/3586809421 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe18c004020 con 0x7fe1a40ad880 2024-09-16T07:29:46.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.774+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a41f4120 con 0x7fe1a40ad880 2024-09-16T07:29:46.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.774+0000 7fe19aa00640 1 -- 172.21.15.71:0/3586809421 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c004470 con 0x7fe1a40ad880 2024-09-16T07:29:46.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.774+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a40b2d00 con 0x7fe1a40ad880 2024-09-16T07:29:46.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.775+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a41f50f0 con 0x7fe1a40ad880 2024-09-16T07:29:46.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.776+0000 7fe19aa00640 1 -- 172.21.15.71:0/3586809421 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fe18c0035f0 con 0x7fe1a40ad880 2024-09-16T07:29:46.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.777+0000 7fe19aa00640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe180079450 0x7fe18007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:46.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.777+0000 7fe19aa00640 1 -- 172.21.15.71:0/3586809421 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5737+0+0 (secure 0 0 0) 0x7fe18c099da0 con 0x7fe1a40ad880 2024-09-16T07:29:46.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.777+0000 7fe1a1e00640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe180079450 0x7fe18007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:46.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.778+0000 7fe1a1e00640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe180079450 0x7fe18007b910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe194007a50 tx=0x7fe194009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:46.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.781+0000 7fe19aa00640 1 -- 172.21.15.71:0/3586809421 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe18c063e60 con 0x7fe1a40ad880 2024-09-16T07:29:46.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.967+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1a40adcd0 con 0x7fe1a40ad880 2024-09-16T07:29:46.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.968+0000 7fe19aa00640 1 -- 172.21.15.71:0/3586809421 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16736 (secure 0 0 0) 0x7fe18c063760 con 0x7fe1a40ad880 2024-09-16T07:29:46.969 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:46.969 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:29:45.618504+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:29:43.380127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T07:29:44.588310+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:29:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.970+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe180079450 msgr2=0x7fe18007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.970+0000 7fe1a3200640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe180079450 0x7fe18007b910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe194007a50 tx=0x7fe194009040 comp rx=0 tx=0).stop 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.970+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1a40ad880 msgr2=0x7fe1a40b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.970+0000 7fe1a3200640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1a40ad880 0x7fe1a40b0770 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00ea90 tx=0x7fe18c00ef50 comp rx=0 tx=0).stop 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.971+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 shutdown_connections 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.971+0000 7fe1a3200640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe180079450 0x7fe18007b910 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.971+0000 7fe1a3200640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1a40cd5f0 0x7fe1a41f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.971+0000 7fe1a3200640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1a40ad880 0x7fe1a40b0770 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.971+0000 7fe1a3200640 1 --2- 172.21.15.71:0/3586809421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1a406fb90 0x7fe1a40c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.971+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 >> 172.21.15.71:0/3586809421 conn(0x7fe1a40cfa20 msgr2=0x7fe1a40700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.971+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 shutdown_connections 2024-09-16T07:29:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:46.971+0000 7fe1a3200640 1 -- 172.21.15.71:0/3586809421 wait complete. 2024-09-16T07:29:46.988 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:29:47.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.147+0000 7f4987000640 1 -- 172.21.15.71:0/2714850865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498006efa0 msgr2=0x7f4980073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:47.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.147+0000 7f4987000640 1 --2- 172.21.15.71:0/2714850865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498006efa0 0x7f4980073440 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4978009830 tx=0x7f497802a480 comp rx=0 tx=0).stop 2024-09-16T07:29:47.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.147+0000 7f4987000640 1 -- 172.21.15.71:0/2714850865 shutdown_connections 2024-09-16T07:29:47.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.147+0000 7f4987000640 1 --2- 172.21.15.71:0/2714850865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498006efa0 0x7f4980073440 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:47.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.147+0000 7f4987000640 1 -- 172.21.15.71:0/2714850865 >> 172.21.15.71:0/2714850865 conn(0x7f49800b1840 msgr2=0x7f49800b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:47.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.147+0000 7f4987000640 1 -- 172.21.15.71:0/2714850865 shutdown_connections 2024-09-16T07:29:47.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.147+0000 7f4987000640 1 -- 172.21.15.71:0/2714850865 wait complete. 2024-09-16T07:29:47.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.149+0000 7f4987000640 1 Processor -- start 2024-09-16T07:29:47.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.149+0000 7f4987000640 1 -- start start 2024-09-16T07:29:47.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f4987000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498006efa0 0x7f49801f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:47.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f4987000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49800cccd0 0x7f498013bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:47.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f4987000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4980145750 0x7f49801406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:47.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f4987000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f49800ce6a0 con 0x7f49800cccd0 2024-09-16T07:29:47.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f4987000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49800ce520 con 0x7f4980145750 2024-09-16T07:29:47.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f4987000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49800ce820 con 0x7f498006efa0 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49800cccd0 0x7f498013bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49800cccd0 0x7f498013bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33094/0 (socket says 172.21.15.71:33094) 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497f400640 1 -- 172.21.15.71:0/1105840442 learned_addr learned my addr 172.21.15.71:0/1105840442 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497fe00640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498006efa0 0x7f49801f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497f400640 1 -- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498006efa0 msgr2=0x7f49801f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497f400640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498006efa0 0x7f49801f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497f400640 1 -- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4980145750 msgr2=0x7f49801406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497f400640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4980145750 0x7f49801406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.150+0000 7f497f400640 1 -- 172.21.15.71:0/1105840442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4980141160 con 0x7f49800cccd0 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.151+0000 7f497f400640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49800cccd0 0x7f498013bcd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f496c00c460 tx=0x7f496c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:47.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.151+0000 7f4977e00640 1 -- 172.21.15.71:0/1105840442 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c011790 con 0x7f49800cccd0 2024-09-16T07:29:47.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.151+0000 7f4977e00640 1 -- 172.21.15.71:0/1105840442 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f496c017070 con 0x7f49800cccd0 2024-09-16T07:29:47.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.151+0000 7f4977e00640 1 -- 172.21.15.71:0/1105840442 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c011d00 con 0x7f49800cccd0 2024-09-16T07:29:47.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.151+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4980141720 con 0x7f49800cccd0 2024-09-16T07:29:47.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.151+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4980141ce0 con 0x7f49800cccd0 2024-09-16T07:29:47.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.152+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49800cecd0 con 0x7f49800cccd0 2024-09-16T07:29:47.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.153+0000 7f4977e00640 1 -- 172.21.15.71:0/1105840442 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f496c01c460 con 0x7f49800cccd0 2024-09-16T07:29:47.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.155+0000 7f4977e00640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4960079420 0x7f496007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:47.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.155+0000 7f4977e00640 1 -- 172.21.15.71:0/1105840442 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5737+0+0 (secure 0 0 0) 0x7f496c099410 con 0x7f49800cccd0 2024-09-16T07:29:47.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.155+0000 7f497fe00640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4960079420 0x7f496007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:47.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.156+0000 7f497fe00640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4960079420 0x7f496007b8e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f49700020b0 tx=0x7f4970009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:47.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.159+0000 7f4977e00640 1 -- 172.21.15.71:0/1105840442 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f496c063270 con 0x7f49800cccd0 2024-09-16T07:29:47.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:47.387+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f498013fab0 con 0x7f49800cccd0 2024-09-16T07:29:48.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.224+0000 7f4977e00640 1 -- 172.21.15.71:0/1105840442 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f496c062b70 con 0x7f49800cccd0 2024-09-16T07:29:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4960079420 msgr2=0x7f496007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4960079420 0x7f496007b8e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f49700020b0 tx=0x7f4970009040 comp rx=0 tx=0).stop 2024-09-16T07:29:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49800cccd0 msgr2=0x7f498013bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49800cccd0 0x7f498013bcd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f496c00c460 tx=0x7f496c00c920 comp rx=0 tx=0).stop 2024-09-16T07:29:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 shutdown_connections 2024-09-16T07:29:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4960079420 0x7f496007b8e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4980145750 0x7f49801406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:48.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49800cccd0 0x7f498013bcd0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:48.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 --2- 172.21.15.71:0/1105840442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498006efa0 0x7f49801f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:48.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 >> 172.21.15.71:0/1105840442 conn(0x7f49800b1840 msgr2=0x7f49800b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:48.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 shutdown_connections 2024-09-16T07:29:48.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.227+0000 7f4987000640 1 -- 172.21.15.71:0/1105840442 wait complete. 2024-09-16T07:29:48.244 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:29:48.244 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:29:48.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.405+0000 7f1a07200640 1 -- 172.21.15.71:0/3545572143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a0815bec0 msgr2=0x7f1a0806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:48.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.405+0000 7f1a07200640 1 --2- 172.21.15.71:0/3545572143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a0815bec0 0x7f1a0806fce0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1a0806f130 tx=0x7f19f802a510 comp rx=0 tx=0).stop 2024-09-16T07:29:48.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.405+0000 7f1a07200640 1 -- 172.21.15.71:0/3545572143 shutdown_connections 2024-09-16T07:29:48.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.405+0000 7f1a07200640 1 --2- 172.21.15.71:0/3545572143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a0815bec0 0x7f1a0806fce0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:48.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.405+0000 7f1a07200640 1 -- 172.21.15.71:0/3545572143 >> 172.21.15.71:0/3545572143 conn(0x7f1a080cf9f0 msgr2=0x7f1a080cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:48.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.405+0000 7f1a07200640 1 -- 172.21.15.71:0/3545572143 shutdown_connections 2024-09-16T07:29:48.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.405+0000 7f1a07200640 1 -- 172.21.15.71:0/3545572143 wait complete. 2024-09-16T07:29:48.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.406+0000 7f1a07200640 1 Processor -- start 2024-09-16T07:29:48.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.407+0000 7f1a07200640 1 -- start start 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.407+0000 7f1a07200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a08072560 0x7f1a08156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.407+0000 7f1a07200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a080cbc00 0x7f1a081e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.407+0000 7f1a07200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a0815bec0 0x7f1a081e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.407+0000 7f1a07200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1a080aebd0 con 0x7f1a080cbc00 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.407+0000 7f1a07200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1a08074400 con 0x7f1a0815bec0 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.407+0000 7f1a07200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1a080aed50 con 0x7f1a08072560 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a080cbc00 0x7f1a081e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a08072560 0x7f1a08156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a080cbc00 0x7f1a081e89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33116/0 (socket says 172.21.15.71:33116) 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05400640 1 -- 172.21.15.71:0/755594693 learned_addr learned my addr 172.21.15.71:0/755594693 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a06800640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a0815bec0 0x7f1a081e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05400640 1 -- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a08072560 msgr2=0x7f1a08156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05400640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a08072560 0x7f1a08156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05400640 1 -- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a0815bec0 msgr2=0x7f1a081e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05400640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a0815bec0 0x7f1a081e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.408+0000 7f1a05400640 1 -- 172.21.15.71:0/755594693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a08156a80 con 0x7f1a080cbc00 2024-09-16T07:29:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.409+0000 7f1a05400640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a080cbc00 0x7f1a081e89a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f19f400c460 tx=0x7f19f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.409+0000 7f19fe000640 1 -- 172.21.15.71:0/755594693 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4016020 con 0x7f1a080cbc00 2024-09-16T07:29:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.409+0000 7f19fe000640 1 -- 172.21.15.71:0/755594693 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19f4012960 con 0x7f1a080cbc00 2024-09-16T07:29:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.409+0000 7f19fe000640 1 -- 172.21.15.71:0/755594693 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4012db0 con 0x7f1a080cbc00 2024-09-16T07:29:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.409+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a080c96c0 con 0x7f1a080cbc00 2024-09-16T07:29:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.409+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a081f8610 con 0x7f1a080cbc00 2024-09-16T07:29:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.410+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0806fcc0 con 0x7f1a080cbc00 2024-09-16T07:29:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.411+0000 7f19fe000640 1 -- 172.21.15.71:0/755594693 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f19f4019020 con 0x7f1a080cbc00 2024-09-16T07:29:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.412+0000 7f19fe000640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19e8079420 0x7f19e807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.412+0000 7f19fe000640 1 -- 172.21.15.71:0/755594693 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f19f40991a0 con 0x7f1a080cbc00 2024-09-16T07:29:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.413+0000 7f1a05e00640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19e8079420 0x7f19e807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.413+0000 7f1a05e00640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19e8079420 0x7f19e807b8e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f19f8034000 tx=0x7f19f8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.416+0000 7f19fe000640 1 -- 172.21.15.71:0/755594693 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f19f4020030 con 0x7f1a080cbc00 2024-09-16T07:29:48.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:48.638+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1a080b2db0 con 0x7f1a080cbc00 2024-09-16T07:29:49.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.233+0000 7f19fe000640 1 -- 172.21.15.71:0/755594693 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f1a080c96c0 con 0x7f1a080cbc00 2024-09-16T07:29:49.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19e8079420 msgr2=0x7f19e807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19e8079420 0x7f19e807b8e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f19f8034000 tx=0x7f19f8004730 comp rx=0 tx=0).stop 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a080cbc00 msgr2=0x7f1a081e89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a080cbc00 0x7f1a081e89a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f19f400c460 tx=0x7f19f400c920 comp rx=0 tx=0).stop 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 shutdown_connections 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19e8079420 0x7f19e807b8e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a0815bec0 0x7f1a081e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a080cbc00 0x7f1a081e89a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 --2- 172.21.15.71:0/755594693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a08072560 0x7f1a08156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 >> 172.21.15.71:0/755594693 conn(0x7f1a080cf9f0 msgr2=0x7f1a080b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 shutdown_connections 2024-09-16T07:29:49.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.236+0000 7f1a07200640 1 -- 172.21.15.71:0/755594693 wait complete. 2024-09-16T07:29:49.254 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:29:49.256 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:29:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.412+0000 7f0fd3800640 1 -- 172.21.15.71:0/3673566007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 msgr2=0x7f0fcc074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.412+0000 7f0fd3800640 1 --2- 172.21.15.71:0/3673566007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 0x7f0fcc074440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4005fd0 tx=0x7f0fc402aa10 comp rx=0 tx=0).stop 2024-09-16T07:29:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.413+0000 7f0fd3800640 1 -- 172.21.15.71:0/3673566007 shutdown_connections 2024-09-16T07:29:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.413+0000 7f0fd3800640 1 --2- 172.21.15.71:0/3673566007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 0x7f0fcc074440 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.413+0000 7f0fd3800640 1 --2- 172.21.15.71:0/3673566007 >> v2:172.21.15.177:3300/0 conn(0x7f0fcc15f5a0 0x7f0fcc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.413+0000 7f0fd3800640 1 -- 172.21.15.71:0/3673566007 >> 172.21.15.71:0/3673566007 conn(0x7f0fcc0b1950 msgr2=0x7f0fcc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.413+0000 7f0fd3800640 1 -- 172.21.15.71:0/3673566007 shutdown_connections 2024-09-16T07:29:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.413+0000 7f0fd3800640 1 -- 172.21.15.71:0/3673566007 wait complete. 2024-09-16T07:29:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.414+0000 7f0fd3800640 1 Processor -- start 2024-09-16T07:29:49.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.414+0000 7f0fd3800640 1 -- start start 2024-09-16T07:29:49.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.415+0000 7f0fd3800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 0x7f0fcc156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:49.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.415+0000 7f0fd3800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0ccf00 0x7f0fcc153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:49.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.415+0000 7f0fd3800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc15f5a0 0x7f0fcc156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:49.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.415+0000 7f0fd3800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc06f2e0 con 0x7f0fcc0ccf00 2024-09-16T07:29:49.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.415+0000 7f0fd3800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc06f160 con 0x7f0fcc15f5a0 2024-09-16T07:29:49.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.415+0000 7f0fd3800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0fcc06f460 con 0x7f0fcc073fb0 2024-09-16T07:29:49.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fcbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0ccf00 0x7f0fcc153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:49.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 0x7f0fcc156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:49.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 0x7f0fcc156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:54408/0 (socket says 172.21.15.71:54408) 2024-09-16T07:29:49.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 -- 172.21.15.71:0/122708766 learned_addr learned my addr 172.21.15.71:0/122708766 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:49.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd1800640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc15f5a0 0x7f0fcc156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:49.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 -- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc15f5a0 msgr2=0x7f0fcc156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc15f5a0 0x7f0fcc156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 -- 172.21.15.71:0/122708766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0ccf00 msgr2=0x7f0fcc153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0ccf00 0x7f0fcc153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 -- 172.21.15.71:0/122708766 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc1f4470 con 0x7f0fcc073fb0 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fcbe00640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0ccf00 0x7f0fcc153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.416+0000 7f0fd0e00640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 0x7f0fcc156090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00e790 tx=0x7f0fbc00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.417+0000 7f0fc8c00640 1 -- 172.21.15.71:0/122708766 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00eea0 con 0x7f0fcc073fb0 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.417+0000 7f0fc8c00640 1 -- 172.21.15.71:0/122708766 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fbc00cdb0 con 0x7f0fcc073fb0 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.417+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fcc154580 con 0x7f0fcc073fb0 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.417+0000 7f0fc8c00640 1 -- 172.21.15.71:0/122708766 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc010690 con 0x7f0fcc073fb0 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.417+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fcc1f46f0 con 0x7f0fcc073fb0 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.418+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fcc06f980 con 0x7f0fcc073fb0 2024-09-16T07:29:49.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.419+0000 7f0fc8c00640 1 -- 172.21.15.71:0/122708766 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f0fbc010830 con 0x7f0fcc073fb0 2024-09-16T07:29:49.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.420+0000 7f0fc8c00640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fac079420 0x7f0fac07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:49.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.420+0000 7f0fc8c00640 1 -- 172.21.15.71:0/122708766 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f0fbc09a4b0 con 0x7f0fcc073fb0 2024-09-16T07:29:49.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.421+0000 7f0fcbe00640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fac079420 0x7f0fac07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:49.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.421+0000 7f0fcbe00640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fac079420 0x7f0fac07b8e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc157650 tx=0x7f0fb8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:49.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.424+0000 7f0fc8c00640 1 -- 172.21.15.71:0/122708766 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0fcc1f4470 con 0x7f0fcc073fb0 2024-09-16T07:29:49.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.645+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fcc1571b0 con 0x7f0fcc073fb0 2024-09-16T07:29:49.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.646+0000 7f0fc8c00640 1 -- 172.21.15.71:0/122708766 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5231 (secure 0 0 0) 0x7f0fbc063f20 con 0x7f0fcc073fb0 2024-09-16T07:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:49.647 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":46,"btime":"2024-09-16T07:29:49:226790+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/4181257111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":4181257111},{"type":"v1","addr":"172.21.15.177:6836","nonce":4181257111}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2868012570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2868012570},{"type":"v1","addr":"172.21.15.177:6837","nonce":2868012570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6414,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3075140097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3075140097},{"type":"v1","addr":"172.21.15.71:6834","nonce":3075140097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:29:45.618471+0000","modified":"2024-09-16T07:29:48.640333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6417},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6417":{"gid":6417,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1371001821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":1371001821},{"type":"v1","addr":"172.21.15.71:6835","nonce":1371001821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6417,"qdb_cluster":[6417]},"id":5}]} 2024-09-16T07:29:49.647 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 46 2024-09-16T07:29:49.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.648+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fac079420 msgr2=0x7f0fac07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.648+0000 7f0fd3800640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fac079420 0x7f0fac07b8e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc157650 tx=0x7f0fb8009040 comp rx=0 tx=0).stop 2024-09-16T07:29:49.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.648+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 msgr2=0x7f0fcc156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.648+0000 7f0fd3800640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 0x7f0fcc156090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00e790 tx=0x7f0fbc00ec50 comp rx=0 tx=0).stop 2024-09-16T07:29:49.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.649+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 shutdown_connections 2024-09-16T07:29:49.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.649+0000 7f0fd3800640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0fac079420 0x7f0fac07b8e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.649+0000 7f0fd3800640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc15f5a0 0x7f0fcc156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.649+0000 7f0fd3800640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0ccf00 0x7f0fcc153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.649+0000 7f0fd3800640 1 --2- 172.21.15.71:0/122708766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc073fb0 0x7f0fcc156090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.649+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 >> 172.21.15.71:0/122708766 conn(0x7f0fcc0b1950 msgr2=0x7f0fcc0af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:49.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.649+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 shutdown_connections 2024-09-16T07:29:49.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.649+0000 7f0fd3800640 1 -- 172.21.15.71:0/122708766 wait complete. 2024-09-16T07:29:49.665 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:29:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.822+0000 7f10fa800640 1 -- 172.21.15.71:0/3346526554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 msgr2=0x7f10d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.822+0000 7f10fa800640 1 --2- 172.21.15.71:0/3346526554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 0x7f10d800b9c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0016c0 tx=0x7f10d8005830 comp rx=0 tx=0).stop 2024-09-16T07:29:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.822+0000 7f10fa800640 1 -- 172.21.15.71:0/3346526554 shutdown_connections 2024-09-16T07:29:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.822+0000 7f10fa800640 1 --2- 172.21.15.71:0/3346526554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 0x7f10d800b9c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.822+0000 7f10fa800640 1 -- 172.21.15.71:0/3346526554 >> 172.21.15.71:0/3346526554 conn(0x7f10f40b1950 msgr2=0x7f10f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:49.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.822+0000 7f10fa800640 1 -- 172.21.15.71:0/3346526554 shutdown_connections 2024-09-16T07:29:49.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.822+0000 7f10fa800640 1 -- 172.21.15.71:0/3346526554 wait complete. 2024-09-16T07:29:49.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.824+0000 7f10fa800640 1 Processor -- start 2024-09-16T07:29:49.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.824+0000 7f10fa800640 1 -- start start 2024-09-16T07:29:49.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 0x7f10f40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:49.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10fa800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10f40b4800 0x7f10f41509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:49.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10fa800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10f406f2f0 0x7f10f41f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:49.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10fa800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4074420 con 0x7f10d80052e0 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10fa800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f10f40b2700 con 0x7f10f40b4800 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10fa800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10f40745a0 con 0x7f10f406f2f0 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10f3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 0x7f10f40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10f3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 0x7f10f40b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33144/0 (socket says 172.21.15.71:33144) 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10f3400640 1 -- 172.21.15.71:0/3201747266 learned_addr learned my addr 172.21.15.71:0/3201747266 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.825+0000 7f10f3e00640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10f406f2f0 0x7f10f41f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10f2a00640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10f40b4800 0x7f10f41509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10f3400640 1 -- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10f406f2f0 msgr2=0x7f10f41f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10f3400640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10f406f2f0 0x7f10f41f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10f3400640 1 -- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10f40b4800 msgr2=0x7f10f41509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10f3400640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10f40b4800 0x7f10f41509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10f3400640 1 -- 172.21.15.71:0/3201747266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f41506c0 con 0x7f10d80052e0 2024-09-16T07:29:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10f3400640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 0x7f10f40b2d50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e930 tx=0x7f10e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10eb400640 1 -- 172.21.15.71:0/3201747266 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4015070 con 0x7f10d80052e0 2024-09-16T07:29:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10eb400640 1 -- 172.21.15.71:0/3201747266 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10e400cd40 con 0x7f10d80052e0 2024-09-16T07:29:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10eb400640 1 -- 172.21.15.71:0/3201747266 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e40116c0 con 0x7f10d80052e0 2024-09-16T07:29:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.826+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f414fb40 con 0x7f10d80052e0 2024-09-16T07:29:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.827+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f4150100 con 0x7f10d80052e0 2024-09-16T07:29:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.828+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f40760b0 con 0x7f10d80052e0 2024-09-16T07:29:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.829+0000 7f10eb400640 1 -- 172.21.15.71:0/3201747266 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f10e4018070 con 0x7f10d80052e0 2024-09-16T07:29:49.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.830+0000 7f10eb400640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f10d4079420 0x7f10d407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:49.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.830+0000 7f10eb400640 1 -- 172.21.15.71:0/3201747266 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f10e4099b80 con 0x7f10d80052e0 2024-09-16T07:29:49.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.830+0000 7f10f2a00640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f10d4079420 0x7f10d407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:49.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.831+0000 7f10f2a00640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f10d4079420 0x7f10d407b8e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f10f40b4130 tx=0x7f10e0005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:49.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:49.834+0000 7f10eb400640 1 -- 172.21.15.71:0/3201747266 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f10f40760b0 con 0x7f10d80052e0 2024-09-16T07:29:50.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.018+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10f4154f60 con 0x7f10d80052e0 2024-09-16T07:29:50.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.019+0000 7f10eb400640 1 -- 172.21.15.71:0/3201747266 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16723 (secure 0 0 0) 0x7f10e4063390 con 0x7f10d80052e0 2024-09-16T07:29:50.020 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:50.020 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":77,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:29:47.195438+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:29:43.380127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T07:29:44.588310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:29:50.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f10d4079420 msgr2=0x7f10d407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f10d4079420 0x7f10d407b8e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f10f40b4130 tx=0x7f10e0005f90 comp rx=0 tx=0).stop 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 msgr2=0x7f10f40b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 0x7f10f40b2d50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e930 tx=0x7f10e400edf0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 shutdown_connections 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f10d4079420 0x7f10d407b8e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10f406f2f0 0x7f10f41f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10f40b4800 0x7f10f41509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 --2- 172.21.15.71:0/3201747266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d80052e0 0x7f10f40b2d50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 >> 172.21.15.71:0/3201747266 conn(0x7f10f40b1950 msgr2=0x7f10f40b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 shutdown_connections 2024-09-16T07:29:50.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.022+0000 7f10fa800640 1 -- 172.21.15.71:0/3201747266 wait complete. 2024-09-16T07:29:50.039 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:29:50.039 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:29:50.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 -- 172.21.15.71:0/2321349842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 msgr2=0x7fcd800acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 --2- 172.21.15.71:0/2321349842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 0x7fcd800acd30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcd80075130 tx=0x7fcd6c02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 -- 172.21.15.71:0/2321349842 shutdown_connections 2024-09-16T07:29:50.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 --2- 172.21.15.71:0/2321349842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd800ad270 0x7fcd80073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 --2- 172.21.15.71:0/2321349842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 0x7fcd800acd30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 --2- 172.21.15.71:0/2321349842 >> v2:172.21.15.177:3300/0 conn(0x7fcd8015f5a0 0x7fcd800b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 -- 172.21.15.71:0/2321349842 >> 172.21.15.71:0/2321349842 conn(0x7fcd800b1950 msgr2=0x7fcd800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:50.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 -- 172.21.15.71:0/2321349842 shutdown_connections 2024-09-16T07:29:50.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.195+0000 7fcd87600640 1 -- 172.21.15.71:0/2321349842 wait complete. 2024-09-16T07:29:50.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.197+0000 7fcd87600640 1 Processor -- start 2024-09-16T07:29:50.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.197+0000 7fcd87600640 1 -- start start 2024-09-16T07:29:50.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd87600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd800ad270 0x7fcd80154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:50.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd87600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 0x7fcd8009ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:50.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd87600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd8015f5a0 0x7fcd8009d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:50.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd87600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80074880 con 0x7fcd800ccf00 2024-09-16T07:29:50.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd87600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80074700 con 0x7fcd8015f5a0 2024-09-16T07:29:50.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd87600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8006f160 con 0x7fcd800ad270 2024-09-16T07:29:50.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd7fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 0x7fcd8009ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:50.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd7fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 0x7fcd8009ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33162/0 (socket says 172.21.15.71:33162) 2024-09-16T07:29:50.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd7fe00640 1 -- 172.21.15.71:0/3727486705 learned_addr learned my addr 172.21.15.71:0/3727486705 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:50.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd84c00640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd800ad270 0x7fcd80154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:50.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.198+0000 7fcd85600640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd8015f5a0 0x7fcd8009d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:50.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7fe00640 1 -- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd800ad270 msgr2=0x7fcd80154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7fe00640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd800ad270 0x7fcd80154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7fe00640 1 -- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd8015f5a0 msgr2=0x7fcd8009d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7fe00640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd8015f5a0 0x7fcd8009d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7fe00640 1 -- 172.21.15.71:0/3727486705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd801f4770 con 0x7fcd800ccf00 2024-09-16T07:29:50.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7fe00640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 0x7fcd8009ce60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009850 tx=0x7fcd6c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7cc00640 1 -- 172.21.15.71:0/3727486705 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c03f070 con 0x7fcd800ccf00 2024-09-16T07:29:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7cc00640 1 -- 172.21.15.71:0/3727486705 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd6c0047a0 con 0x7fcd800ccf00 2024-09-16T07:29:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd801f49e0 con 0x7fcd800ccf00 2024-09-16T07:29:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd7cc00640 1 -- 172.21.15.71:0/3727486705 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c035030 con 0x7fcd800ccf00 2024-09-16T07:29:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.199+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd80154d80 con 0x7fcd800ccf00 2024-09-16T07:29:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.201+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd80073fb0 con 0x7fcd800ccf00 2024-09-16T07:29:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.202+0000 7fcd7cc00640 1 -- 172.21.15.71:0/3727486705 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fcd6c042070 con 0x7fcd800ccf00 2024-09-16T07:29:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.203+0000 7fcd7cc00640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd60079420 0x7fcd6007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.203+0000 7fcd7cc00640 1 -- 172.21.15.71:0/3727486705 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fcd6c0ba340 con 0x7fcd800ccf00 2024-09-16T07:29:50.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.203+0000 7fcd84c00640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd60079420 0x7fcd6007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:50.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.204+0000 7fcd84c00640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd60079420 0x7fcd6007b8e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000bb50 tx=0x7fcd7000b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.207+0000 7fcd7cc00640 1 -- 172.21.15.71:0/3727486705 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcd6c083bb0 con 0x7fcd800ccf00 2024-09-16T07:29:50.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.388+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd80069ac0 con 0x7fcd60079420 2024-09-16T07:29:50.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.394+0000 7fcd7cc00640 1 -- 172.21.15.71:0/3727486705 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243754 (secure 0 0 0) 0x7fcd80069ac0 con 0x7fcd60079420 2024-09-16T07:29:50.397 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:50.398 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:29:50.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd60079420 msgr2=0x7fcd6007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd60079420 0x7fcd6007b8e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000bb50 tx=0x7fcd7000b2e0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 msgr2=0x7fcd8009ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 0x7fcd8009ce60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009850 tx=0x7fcd6c003910 comp rx=0 tx=0).stop 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 shutdown_connections 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd60079420 0x7fcd6007b8e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd8015f5a0 0x7fcd8009d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd800ccf00 0x7fcd8009ce60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 --2- 172.21.15.71:0/3727486705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd800ad270 0x7fcd80154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 >> 172.21.15.71:0/3727486705 conn(0x7fcd800b1950 msgr2=0x7fcd800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 shutdown_connections 2024-09-16T07:29:50.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.397+0000 7fcd87600640 1 -- 172.21.15.71:0/3727486705 wait complete. 2024-09-16T07:29:50.415 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-16T07:29:48.412117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592104,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":114,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":406248,"kb_used_data":8184,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754568472,"statfs":{"total":773094113280,"available":772678115328,"internally_reserved":0,"allocated":8380416,"data_stored":4222109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":407424560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1736,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.222663"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.221689+0000","last_change":"2024-09-16T07:29:46.218341+0000","last_active":"2024-09-16T07:29:46.221689+0000","last_peered":"2024-09-16T07:29:46.221689+0000","last_clean":"2024-09-16T07:29:46.221689+0000","last_became_active":"2024-09-16T07:29:46.218075+0000","last_became_peered":"2024-09-16T07:29:46.218075+0000","last_unstale":"2024-09-16T07:29:46.221689+0000","last_undegraded":"2024-09-16T07:29:46.221689+0000","last_fullsized":"2024-09-16T07:29:46.221689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:51.529176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.224896+0000","last_change":"2024-09-16T07:29:46.220749+0000","last_active":"2024-09-16T07:29:46.224896+0000","last_peered":"2024-09-16T07:29:46.224896+0000","last_clean":"2024-09-16T07:29:46.224896+0000","last_became_active":"2024-09-16T07:29:46.219839+0000","last_became_peered":"2024-09-16T07:29:46.219839+0000","last_unstale":"2024-09-16T07:29:46.224896+0000","last_undegraded":"2024-09-16T07:29:46.224896+0000","last_fullsized":"2024-09-16T07:29:46.224896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:57:55.424351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202505+0000","last_change":"2024-09-16T07:29:46.239904+0000","last_active":"2024-09-16T07:29:47.202505+0000","last_peered":"2024-09-16T07:29:47.202505+0000","last_clean":"2024-09-16T07:29:47.202505+0000","last_became_active":"2024-09-16T07:29:46.239612+0000","last_became_peered":"2024-09-16T07:29:46.239612+0000","last_unstale":"2024-09-16T07:29:47.202505+0000","last_undegraded":"2024-09-16T07:29:47.202505+0000","last_fullsized":"2024-09-16T07:29:47.202505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:34.011862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.221707+0000","last_change":"2024-09-16T07:29:46.217117+0000","last_active":"2024-09-16T07:29:46.221707+0000","last_peered":"2024-09-16T07:29:46.221707+0000","last_clean":"2024-09-16T07:29:46.221707+0000","last_became_active":"2024-09-16T07:29:46.216757+0000","last_became_peered":"2024-09-16T07:29:46.216757+0000","last_unstale":"2024-09-16T07:29:46.221707+0000","last_undegraded":"2024-09-16T07:29:46.221707+0000","last_fullsized":"2024-09-16T07:29:46.221707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:55.192855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.222655+0000","last_change":"2024-09-16T07:29:46.217736+0000","last_active":"2024-09-16T07:29:46.222655+0000","last_peered":"2024-09-16T07:29:46.222655+0000","last_clean":"2024-09-16T07:29:46.222655+0000","last_became_active":"2024-09-16T07:29:46.217370+0000","last_became_peered":"2024-09-16T07:29:46.217370+0000","last_unstale":"2024-09-16T07:29:46.222655+0000","last_undegraded":"2024-09-16T07:29:46.222655+0000","last_fullsized":"2024-09-16T07:29:46.222655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:43.567797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.223716+0000","last_change":"2024-09-16T07:29:46.215135+0000","last_active":"2024-09-16T07:29:46.223716+0000","last_peered":"2024-09-16T07:29:46.223716+0000","last_clean":"2024-09-16T07:29:46.223716+0000","last_became_active":"2024-09-16T07:29:46.214886+0000","last_became_peered":"2024-09-16T07:29:46.214886+0000","last_unstale":"2024-09-16T07:29:46.223716+0000","last_undegraded":"2024-09-16T07:29:46.223716+0000","last_fullsized":"2024-09-16T07:29:46.223716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:51:01.994799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.227641+0000","last_change":"2024-09-16T07:29:46.225187+0000","last_active":"2024-09-16T07:29:46.227641+0000","last_peered":"2024-09-16T07:29:46.227641+0000","last_clean":"2024-09-16T07:29:46.227641+0000","last_became_active":"2024-09-16T07:29:46.219710+0000","last_became_peered":"2024-09-16T07:29:46.219710+0000","last_unstale":"2024-09-16T07:29:46.227641+0000","last_undegraded":"2024-09-16T07:29:46.227641+0000","last_fullsized":"2024-09-16T07:29:46.227641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:33.498202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.222541+0000","last_change":"2024-09-16T07:29:46.216660+0000","last_active":"2024-09-16T07:29:46.222541+0000","last_peered":"2024-09-16T07:29:46.222541+0000","last_clean":"2024-09-16T07:29:46.222541+0000","last_became_active":"2024-09-16T07:29:46.216305+0000","last_became_peered":"2024-09-16T07:29:46.216305+0000","last_unstale":"2024-09-16T07:29:46.222541+0000","last_undegraded":"2024-09-16T07:29:46.222541+0000","last_fullsized":"2024-09-16T07:29:46.222541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:51:08.428781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202989+0000","last_change":"2024-09-16T07:29:46.218519+0000","last_active":"2024-09-16T07:29:47.202989+0000","last_peered":"2024-09-16T07:29:47.202989+0000","last_clean":"2024-09-16T07:29:47.202989+0000","last_became_active":"2024-09-16T07:29:46.218285+0000","last_became_peered":"2024-09-16T07:29:46.218285+0000","last_unstale":"2024-09-16T07:29:47.202989+0000","last_undegraded":"2024-09-16T07:29:47.202989+0000","last_fullsized":"2024-09-16T07:29:47.202989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:36:33.184903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202948+0000","last_change":"2024-09-16T07:29:46.217947+0000","last_active":"2024-09-16T07:29:47.202948+0000","last_peered":"2024-09-16T07:29:47.202948+0000","last_clean":"2024-09-16T07:29:47.202948+0000","last_became_active":"2024-09-16T07:29:46.217697+0000","last_became_peered":"2024-09-16T07:29:46.217697+0000","last_unstale":"2024-09-16T07:29:47.202948+0000","last_undegraded":"2024-09-16T07:29:47.202948+0000","last_fullsized":"2024-09-16T07:29:47.202948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:10.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202860+0000","last_change":"2024-09-16T07:29:46.221180+0000","last_active":"2024-09-16T07:29:47.202860+0000","last_peered":"2024-09-16T07:29:47.202860+0000","last_clean":"2024-09-16T07:29:47.202860+0000","last_became_active":"2024-09-16T07:29:46.220876+0000","last_became_peered":"2024-09-16T07:29:46.220876+0000","last_unstale":"2024-09-16T07:29:47.202860+0000","last_undegraded":"2024-09-16T07:29:47.202860+0000","last_fullsized":"2024-09-16T07:29:47.202860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:58.436343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.203178+0000","last_change":"2024-09-16T07:29:46.227354+0000","last_active":"2024-09-16T07:29:47.203178+0000","last_peered":"2024-09-16T07:29:47.203178+0000","last_clean":"2024-09-16T07:29:47.203178+0000","last_became_active":"2024-09-16T07:29:46.226740+0000","last_became_peered":"2024-09-16T07:29:46.226740+0000","last_unstale":"2024-09-16T07:29:47.203178+0000","last_undegraded":"2024-09-16T07:29:47.203178+0000","last_fullsized":"2024-09-16T07:29:47.203178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:22.242185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.226254+0000","last_change":"2024-09-16T07:29:46.222660+0000","last_active":"2024-09-16T07:29:46.226254+0000","last_peered":"2024-09-16T07:29:46.226254+0000","last_clean":"2024-09-16T07:29:46.226254+0000","last_became_active":"2024-09-16T07:29:46.222410+0000","last_became_peered":"2024-09-16T07:29:46.222410+0000","last_unstale":"2024-09-16T07:29:46.226254+0000","last_undegraded":"2024-09-16T07:29:46.226254+0000","last_fullsized":"2024-09-16T07:29:46.226254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:46.179136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.216951+0000","last_change":"2024-09-16T07:29:46.209844+0000","last_active":"2024-09-16T07:29:46.216951+0000","last_peered":"2024-09-16T07:29:46.216951+0000","last_clean":"2024-09-16T07:29:46.216951+0000","last_became_active":"2024-09-16T07:29:46.209571+0000","last_became_peered":"2024-09-16T07:29:46.209571+0000","last_unstale":"2024-09-16T07:29:46.216951+0000","last_undegraded":"2024-09-16T07:29:46.216951+0000","last_fullsized":"2024-09-16T07:29:46.216951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:34.156942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.203060+0000","last_change":"2024-09-16T07:29:45.210810+0000","last_active":"2024-09-16T07:29:47.203060+0000","last_peered":"2024-09-16T07:29:47.203060+0000","last_clean":"2024-09-16T07:29:47.203060+0000","last_became_active":"2024-09-16T07:29:45.210530+0000","last_became_peered":"2024-09-16T07:29:45.210530+0000","last_unstale":"2024-09-16T07:29:47.203060+0000","last_undegraded":"2024-09-16T07:29:47.203060+0000","last_fullsized":"2024-09-16T07:29:47.203060+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:37.897764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"76'1","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.230855+0000","last_change":"2024-09-16T07:29:45.212253+0000","last_active":"2024-09-16T07:29:46.230855+0000","last_peered":"2024-09-16T07:29:46.230855+0000","last_clean":"2024-09-16T07:29:46.230855+0000","last_became_active":"2024-09-16T07:29:45.210152+0000","last_became_peered":"2024-09-16T07:29:45.210152+0000","last_unstale":"2024-09-16T07:29:46.230855+0000","last_undegraded":"2024-09-16T07:29:46.230855+0000","last_fullsized":"2024-09-16T07:29:46.230855+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:49:17.268570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.216848+0000","last_change":"2024-09-16T07:29:46.210584+0000","last_active":"2024-09-16T07:29:46.216848+0000","last_peered":"2024-09-16T07:29:46.216848+0000","last_clean":"2024-09-16T07:29:46.216848+0000","last_became_active":"2024-09-16T07:29:46.210276+0000","last_became_peered":"2024-09-16T07:29:46.210276+0000","last_unstale":"2024-09-16T07:29:46.216848+0000","last_undegraded":"2024-09-16T07:29:46.216848+0000","last_fullsized":"2024-09-16T07:29:46.216848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:18.494034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.203605+0000","last_change":"2024-09-16T07:29:45.215117+0000","last_active":"2024-09-16T07:29:46.203605+0000","last_peered":"2024-09-16T07:29:46.203605+0000","last_clean":"2024-09-16T07:29:46.203605+0000","last_became_active":"2024-09-16T07:29:45.214120+0000","last_became_peered":"2024-09-16T07:29:45.214120+0000","last_unstale":"2024-09-16T07:29:46.203605+0000","last_undegraded":"2024-09-16T07:29:46.203605+0000","last_fullsized":"2024-09-16T07:29:46.203605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:06.245988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.225068+0000","last_change":"2024-09-16T07:29:46.218571+0000","last_active":"2024-09-16T07:29:46.225068+0000","last_peered":"2024-09-16T07:29:46.225068+0000","last_clean":"2024-09-16T07:29:46.225068+0000","last_became_active":"2024-09-16T07:29:46.214625+0000","last_became_peered":"2024-09-16T07:29:46.214625+0000","last_unstale":"2024-09-16T07:29:46.225068+0000","last_undegraded":"2024-09-16T07:29:46.225068+0000","last_fullsized":"2024-09-16T07:29:46.225068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:45.363499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.209485+0000","last_change":"2024-09-16T07:29:45.214541+0000","last_active":"2024-09-16T07:29:46.209485+0000","last_peered":"2024-09-16T07:29:46.209485+0000","last_clean":"2024-09-16T07:29:46.209485+0000","last_became_active":"2024-09-16T07:29:45.213012+0000","last_became_peered":"2024-09-16T07:29:45.213012+0000","last_unstale":"2024-09-16T07:29:46.209485+0000","last_undegraded":"2024-09-16T07:29:46.209485+0000","last_fullsized":"2024-09-16T07:29:46.209485+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:18.876183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"76'3","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.234424+0000","last_change":"2024-09-16T07:29:45.208752+0000","last_active":"2024-09-16T07:29:46.234424+0000","last_peered":"2024-09-16T07:29:46.234424+0000","last_clean":"2024-09-16T07:29:46.234424+0000","last_became_active":"2024-09-16T07:29:45.208420+0000","last_became_peered":"2024-09-16T07:29:45.208420+0000","last_unstale":"2024-09-16T07:29:46.234424+0000","last_undegraded":"2024-09-16T07:29:46.234424+0000","last_fullsized":"2024-09-16T07:29:46.234424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:03.501445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202994+0000","last_change":"2024-09-16T07:29:46.226982+0000","last_active":"2024-09-16T07:29:47.202994+0000","last_peered":"2024-09-16T07:29:47.202994+0000","last_clean":"2024-09-16T07:29:47.202994+0000","last_became_active":"2024-09-16T07:29:46.226377+0000","last_became_peered":"2024-09-16T07:29:46.226377+0000","last_unstale":"2024-09-16T07:29:47.202994+0000","last_undegraded":"2024-09-16T07:29:47.202994+0000","last_fullsized":"2024-09-16T07:29:47.202994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.148341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"76'6","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.234174+0000","last_change":"2024-09-16T07:29:45.215376+0000","last_active":"2024-09-16T07:29:46.234174+0000","last_peered":"2024-09-16T07:29:46.234174+0000","last_clean":"2024-09-16T07:29:46.234174+0000","last_became_active":"2024-09-16T07:29:45.214835+0000","last_became_peered":"2024-09-16T07:29:45.214835+0000","last_unstale":"2024-09-16T07:29:46.234174+0000","last_undegraded":"2024-09-16T07:29:46.234174+0000","last_fullsized":"2024-09-16T07:29:46.234174+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:20.276225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"76'2","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.232940+0000","last_change":"2024-09-16T07:29:45.218660+0000","last_active":"2024-09-16T07:29:46.232940+0000","last_peered":"2024-09-16T07:29:46.232940+0000","last_clean":"2024-09-16T07:29:46.232940+0000","last_became_active":"2024-09-16T07:29:45.218368+0000","last_became_peered":"2024-09-16T07:29:45.218368+0000","last_unstale":"2024-09-16T07:29:46.232940+0000","last_undegraded":"2024-09-16T07:29:46.232940+0000","last_fullsized":"2024-09-16T07:29:46.232940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:23.785072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.201677+0000","last_change":"2024-09-16T07:29:46.214720+0000","last_active":"2024-09-16T07:29:47.201677+0000","last_peered":"2024-09-16T07:29:47.201677+0000","last_clean":"2024-09-16T07:29:47.201677+0000","last_became_active":"2024-09-16T07:29:46.214391+0000","last_became_peered":"2024-09-16T07:29:46.214391+0000","last_unstale":"2024-09-16T07:29:47.201677+0000","last_undegraded":"2024-09-16T07:29:47.201677+0000","last_fullsized":"2024-09-16T07:29:47.201677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:10:13.303867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.216852+0000","last_change":"2024-09-16T07:29:46.210110+0000","last_active":"2024-09-16T07:29:46.216852+0000","last_peered":"2024-09-16T07:29:46.216852+0000","last_clean":"2024-09-16T07:29:46.216852+0000","last_became_active":"2024-09-16T07:29:46.207997+0000","last_became_peered":"2024-09-16T07:29:46.207997+0000","last_unstale":"2024-09-16T07:29:46.216852+0000","last_undegraded":"2024-09-16T07:29:46.216852+0000","last_fullsized":"2024-09-16T07:29:46.216852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:09.019859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.222716+0000","last_change":"2024-09-16T07:29:46.215016+0000","last_active":"2024-09-16T07:29:46.222716+0000","last_peered":"2024-09-16T07:29:46.222716+0000","last_clean":"2024-09-16T07:29:46.222716+0000","last_became_active":"2024-09-16T07:29:46.214705+0000","last_became_peered":"2024-09-16T07:29:46.214705+0000","last_unstale":"2024-09-16T07:29:46.222716+0000","last_undegraded":"2024-09-16T07:29:46.222716+0000","last_fullsized":"2024-09-16T07:29:46.222716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:42.531575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.226147+0000","last_change":"2024-09-16T07:29:46.221969+0000","last_active":"2024-09-16T07:29:46.226147+0000","last_peered":"2024-09-16T07:29:46.226147+0000","last_clean":"2024-09-16T07:29:46.226147+0000","last_became_active":"2024-09-16T07:29:46.220296+0000","last_became_peered":"2024-09-16T07:29:46.220296+0000","last_unstale":"2024-09-16T07:29:46.226147+0000","last_undegraded":"2024-09-16T07:29:46.226147+0000","last_fullsized":"2024-09-16T07:29:46.226147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:59.544926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.212762+0000","last_change":"2024-09-16T07:29:46.206959+0000","last_active":"2024-09-16T07:29:46.212762+0000","last_peered":"2024-09-16T07:29:46.212762+0000","last_clean":"2024-09-16T07:29:46.212762+0000","last_became_active":"2024-09-16T07:29:46.206704+0000","last_became_peered":"2024-09-16T07:29:46.206704+0000","last_unstale":"2024-09-16T07:29:46.212762+0000","last_undegraded":"2024-09-16T07:29:46.212762+0000","last_fullsized":"2024-09-16T07:29:46.212762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:01:44.742300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.225272+0000","last_change":"2024-09-16T07:29:46.219188+0000","last_active":"2024-09-16T07:29:46.225272+0000","last_peered":"2024-09-16T07:29:46.225272+0000","last_clean":"2024-09-16T07:29:46.225272+0000","last_became_active":"2024-09-16T07:29:46.217110+0000","last_became_peered":"2024-09-16T07:29:46.217110+0000","last_unstale":"2024-09-16T07:29:46.225272+0000","last_undegraded":"2024-09-16T07:29:46.225272+0000","last_fullsized":"2024-09-16T07:29:46.225272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:01:17.765673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.218785+0000","last_change":"2024-09-16T07:29:46.215064+0000","last_active":"2024-09-16T07:29:46.218785+0000","last_peered":"2024-09-16T07:29:46.218785+0000","last_clean":"2024-09-16T07:29:46.218785+0000","last_became_active":"2024-09-16T07:29:46.212423+0000","last_became_peered":"2024-09-16T07:29:46.212423+0000","last_unstale":"2024-09-16T07:29:46.218785+0000","last_undegraded":"2024-09-16T07:29:46.218785+0000","last_fullsized":"2024-09-16T07:29:46.218785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:11.028118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.201451+0000","last_change":"2024-09-16T07:29:45.206969+0000","last_active":"2024-09-16T07:29:47.201451+0000","last_peered":"2024-09-16T07:29:47.201451+0000","last_clean":"2024-09-16T07:29:47.201451+0000","last_became_active":"2024-09-16T07:29:45.206652+0000","last_became_peered":"2024-09-16T07:29:45.206652+0000","last_unstale":"2024-09-16T07:29:47.201451+0000","last_undegraded":"2024-09-16T07:29:47.201451+0000","last_fullsized":"2024-09-16T07:29:47.201451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:13.712685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.203110+0000","last_change":"2024-09-16T07:29:46.223922+0000","last_active":"2024-09-16T07:29:47.203110+0000","last_peered":"2024-09-16T07:29:47.203110+0000","last_clean":"2024-09-16T07:29:47.203110+0000","last_became_active":"2024-09-16T07:29:46.223638+0000","last_became_peered":"2024-09-16T07:29:46.223638+0000","last_unstale":"2024-09-16T07:29:47.203110+0000","last_undegraded":"2024-09-16T07:29:47.203110+0000","last_fullsized":"2024-09-16T07:29:47.203110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:13.554430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.211726+0000","last_change":"2024-09-16T07:29:45.206950+0000","last_active":"2024-09-16T07:29:46.211726+0000","last_peered":"2024-09-16T07:29:46.211726+0000","last_clean":"2024-09-16T07:29:46.211726+0000","last_became_active":"2024-09-16T07:29:45.206663+0000","last_became_peered":"2024-09-16T07:29:45.206663+0000","last_unstale":"2024-09-16T07:29:46.211726+0000","last_undegraded":"2024-09-16T07:29:46.211726+0000","last_fullsized":"2024-09-16T07:29:46.211726+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:27:31.965496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.211621+0000","last_change":"2024-09-16T07:29:45.208847+0000","last_active":"2024-09-16T07:29:46.211621+0000","last_peered":"2024-09-16T07:29:46.211621+0000","last_clean":"2024-09-16T07:29:46.211621+0000","last_became_active":"2024-09-16T07:29:45.208554+0000","last_became_peered":"2024-09-16T07:29:45.208554+0000","last_unstale":"2024-09-16T07:29:46.211621+0000","last_undegraded":"2024-09-16T07:29:46.211621+0000","last_fullsized":"2024-09-16T07:29:46.211621+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:36.140097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202304+0000","last_change":"2024-09-16T07:29:46.221265+0000","last_active":"2024-09-16T07:29:47.202304+0000","last_peered":"2024-09-16T07:29:47.202304+0000","last_clean":"2024-09-16T07:29:47.202304+0000","last_became_active":"2024-09-16T07:29:46.220946+0000","last_became_peered":"2024-09-16T07:29:46.220946+0000","last_unstale":"2024-09-16T07:29:47.202304+0000","last_undegraded":"2024-09-16T07:29:47.202304+0000","last_fullsized":"2024-09-16T07:29:47.202304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:18.644653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.211640+0000","last_change":"2024-09-16T07:29:45.210343+0000","last_active":"2024-09-16T07:29:46.211640+0000","last_peered":"2024-09-16T07:29:46.211640+0000","last_clean":"2024-09-16T07:29:46.211640+0000","last_became_active":"2024-09-16T07:29:45.209867+0000","last_became_peered":"2024-09-16T07:29:45.209867+0000","last_unstale":"2024-09-16T07:29:46.211640+0000","last_undegraded":"2024-09-16T07:29:46.211640+0000","last_fullsized":"2024-09-16T07:29:46.211640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:49.721218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202371+0000","last_change":"2024-09-16T07:29:45.212711+0000","last_active":"2024-09-16T07:29:47.202371+0000","last_peered":"2024-09-16T07:29:47.202371+0000","last_clean":"2024-09-16T07:29:47.202371+0000","last_became_active":"2024-09-16T07:29:45.212443+0000","last_became_peered":"2024-09-16T07:29:45.212443+0000","last_unstale":"2024-09-16T07:29:47.202371+0000","last_undegraded":"2024-09-16T07:29:47.202371+0000","last_fullsized":"2024-09-16T07:29:47.202371+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:53.383357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.229558+0000","last_change":"2024-09-16T07:29:46.228412+0000","last_active":"2024-09-16T07:29:46.229558+0000","last_peered":"2024-09-16T07:29:46.229558+0000","last_clean":"2024-09-16T07:29:46.229558+0000","last_became_active":"2024-09-16T07:29:46.228084+0000","last_became_peered":"2024-09-16T07:29:46.228084+0000","last_unstale":"2024-09-16T07:29:46.229558+0000","last_undegraded":"2024-09-16T07:29:46.229558+0000","last_fullsized":"2024-09-16T07:29:46.229558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:32:11.020949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:45.187877+0000","last_change":"2024-09-16T07:29:45.187877+0000","last_active":"2024-09-16T07:29:45.187877+0000","last_peered":"2024-09-16T07:29:45.187877+0000","last_clean":"2024-09-16T07:29:45.187877+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:45.187877+0000","last_undegraded":"2024-09-16T07:29:45.187877+0000","last_fullsized":"2024-09-16T07:29:45.187877+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202113+0000","last_change":"2024-09-16T07:29:46.221500+0000","last_active":"2024-09-16T07:29:47.202113+0000","last_peered":"2024-09-16T07:29:47.202113+0000","last_clean":"2024-09-16T07:29:47.202113+0000","last_became_active":"2024-09-16T07:29:46.221202+0000","last_became_peered":"2024-09-16T07:29:46.221202+0000","last_unstale":"2024-09-16T07:29:47.202113+0000","last_undegraded":"2024-09-16T07:29:47.202113+0000","last_fullsized":"2024-09-16T07:29:47.202113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:53.293800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:29:44.184515+0000","last_change":"2024-09-16T07:29:44.184515+0000","last_active":"2024-09-16T07:29:44.184515+0000","last_peered":"2024-09-16T07:29:44.184515+0000","last_clean":"2024-09-16T07:29:44.184515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:29:44.184515+0000","last_undegraded":"2024-09-16T07:29:44.184515+0000","last_fullsized":"2024-09-16T07:29:44.184515+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":204,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T07:29:42.180503+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:29:42.180503+0000","last_peered":"2024-09-16T07:29:42.180503+0000","last_clean":"2024-09-16T07:29:42.180503+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:29:42.180503+0000","last_undegraded":"2024-09-16T07:29:42.180503+0000","last_fullsized":"2024-09-16T07:29:42.180503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":56,"acting":56,"num_store_stats":4},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1736,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476826,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50092,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":876544,"data_stored":370341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476826,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50092,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":876544,"data_stored":370341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476826,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46516,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325324,"statfs":{"total":96636764160,"available":96589131776,"internally_reserved":0,"allocated":1470464,"data_stored":960709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476825,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50092,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":876544,"data_stored":370341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50148,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321692,"statfs":{"total":96636764160,"available":96585412608,"internally_reserved":0,"allocated":925696,"data_stored":390896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50144,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321696,"statfs":{"total":96636764160,"available":96585416704,"internally_reserved":0,"allocated":929792,"data_stored":390930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509530,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54884,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316956,"statfs":{"total":96636764160,"available":96580562944,"internally_reserved":0,"allocated":1523712,"data_stored":986070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542232,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54280,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317560,"statfs":{"total":96636764160,"available":96581181440,"internally_reserved":0,"allocated":901120,"data_stored":382481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:29:50.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:29:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.611+0000 7f0500800640 1 -- 172.21.15.71:0/1311025738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f80af3d0 msgr2=0x7f04f806f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.611+0000 7f0500800640 1 --2- 172.21.15.71:0/1311025738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f80af3d0 0x7f04f806f490 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f04f806d9e0 tx=0x7f04e802a4a0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.611+0000 7f0500800640 1 -- 172.21.15.71:0/1311025738 shutdown_connections 2024-09-16T07:29:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.611+0000 7f0500800640 1 --2- 172.21.15.71:0/1311025738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f80af3d0 0x7f04f806f490 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.611+0000 7f0500800640 1 --2- 172.21.15.71:0/1311025738 >> v2:172.21.15.177:3300/0 conn(0x7f04f8161550 0x7f04f806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.611+0000 7f0500800640 1 -- 172.21.15.71:0/1311025738 >> 172.21.15.71:0/1311025738 conn(0x7f04f80b1950 msgr2=0x7f04f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.611+0000 7f0500800640 1 -- 172.21.15.71:0/1311025738 shutdown_connections 2024-09-16T07:29:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.612+0000 7f0500800640 1 -- 172.21.15.71:0/1311025738 wait complete. 2024-09-16T07:29:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.613+0000 7f0500800640 1 Processor -- start 2024-09-16T07:29:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.613+0000 7f0500800640 1 -- start start 2024-09-16T07:29:50.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f0500800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04f806f9d0 0x7f04f81f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:50.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f0500800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04f80af3d0 0x7f04f81f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:50.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f0500800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f8161550 0x7f04f809c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:50.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f0500800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8073f80 con 0x7f04f8161550 2024-09-16T07:29:50.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f0500800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8073e00 con 0x7f04f806f9d0 2024-09-16T07:29:50.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f0500800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f04f8074100 con 0x7f04f80af3d0 2024-09-16T07:29:50.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f04fe800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f8161550 0x7f04f809c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f04fe800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f8161550 0x7f04f809c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33186/0 (socket says 172.21.15.71:33186) 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f04fe800640 1 -- 172.21.15.71:0/3412344453 learned_addr learned my addr 172.21.15.71:0/3412344453 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f04fde00640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04f806f9d0 0x7f04f81f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.614+0000 7f04fd400640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04f80af3d0 0x7f04f81f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04fe800640 1 -- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04f80af3d0 msgr2=0x7f04f81f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04fe800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04f80af3d0 0x7f04f81f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04fe800640 1 -- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04f806f9d0 msgr2=0x7f04f81f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04fe800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04f806f9d0 0x7f04f81f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04fe800640 1 -- 172.21.15.71:0/3412344453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81f2550 con 0x7f04f8161550 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04fde00640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04f806f9d0 0x7f04f81f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04fd400640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04f80af3d0 0x7f04f81f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:50.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04fe800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f8161550 0x7f04f809c810 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f04f400c770 tx=0x7f04f400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:50.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04f2000640 1 -- 172.21.15.71:0/3412344453 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4003da0 con 0x7f04f8161550 2024-09-16T07:29:50.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04f2000640 1 -- 172.21.15.71:0/3412344453 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f4005070 con 0x7f04f8161550 2024-09-16T07:29:50.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f04f2000640 1 -- 172.21.15.71:0/3412344453 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4005500 con 0x7f04f8161550 2024-09-16T07:29:50.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f809aa00 con 0x7f04f8161550 2024-09-16T07:29:50.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.615+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f809afa0 con 0x7f04f8161550 2024-09-16T07:29:50.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.616+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f80b3030 con 0x7f04f8161550 2024-09-16T07:29:50.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.618+0000 7f04f2000640 1 -- 172.21.15.71:0/3412344453 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f04f40056a0 con 0x7f04f8161550 2024-09-16T07:29:50.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.618+0000 7f04f2000640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f04dc079420 0x7f04dc07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:50.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.619+0000 7f04f2000640 1 -- 172.21.15.71:0/3412344453 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f04f4099400 con 0x7f04f8161550 2024-09-16T07:29:50.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.619+0000 7f04fde00640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f04dc079420 0x7f04dc07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:50.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.620+0000 7f04fde00640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f04dc079420 0x7f04dc07b8e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00ab40 tx=0x7f04ec00a520 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:50.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.623+0000 7f04f2000640 1 -- 172.21.15.71:0/3412344453 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f04f80b3030 con 0x7f04f8161550 2024-09-16T07:29:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.802+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04f81626a0 con 0x7f04dc079420 2024-09-16T07:29:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.807+0000 7f04f2000640 1 -- 172.21.15.71:0/3412344453 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7f04f81626a0 con 0x7f04dc079420 2024-09-16T07:29:50.811 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:50.812 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:29:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f04dc079420 msgr2=0x7f04dc07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f04dc079420 0x7f04dc07b8e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00ab40 tx=0x7f04ec00a520 comp rx=0 tx=0).stop 2024-09-16T07:29:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f8161550 msgr2=0x7f04f809c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f8161550 0x7f04f809c810 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f04f400c770 tx=0x7f04f400cc30 comp rx=0 tx=0).stop 2024-09-16T07:29:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 shutdown_connections 2024-09-16T07:29:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f04dc079420 0x7f04dc07b8e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04f8161550 0x7f04f809c810 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04f80af3d0 0x7f04f81f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 --2- 172.21.15.71:0/3412344453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04f806f9d0 0x7f04f81f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 >> 172.21.15.71:0/3412344453 conn(0x7f04f80b1950 msgr2=0x7f04f815fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 shutdown_connections 2024-09-16T07:29:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:50.811+0000 7f0500800640 1 -- 172.21.15.71:0/3412344453 wait complete. 2024-09-16T07:29:50.828 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-16T07:29:50.412573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406484,"kb_used_data":8396,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754568236,"statfs":{"total":773094113280,"available":772677873664,"internally_reserved":0,"allocated":8597504,"data_stored":4323971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":407424560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001117"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850831+0000","last_change":"2024-09-16T07:29:46.217192+0000","last_active":"2024-09-16T07:29:47.850831+0000","last_peered":"2024-09-16T07:29:47.850831+0000","last_clean":"2024-09-16T07:29:47.850831+0000","last_became_active":"2024-09-16T07:29:46.216931+0000","last_became_peered":"2024-09-16T07:29:46.216931+0000","last_unstale":"2024-09-16T07:29:47.850831+0000","last_undegraded":"2024-09-16T07:29:47.850831+0000","last_fullsized":"2024-09-16T07:29:47.850831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:56.527321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199814+0000","last_change":"2024-09-16T07:29:46.233334+0000","last_active":"2024-09-16T07:29:47.199814+0000","last_peered":"2024-09-16T07:29:47.199814+0000","last_clean":"2024-09-16T07:29:47.199814+0000","last_became_active":"2024-09-16T07:29:46.232991+0000","last_became_peered":"2024-09-16T07:29:46.232991+0000","last_unstale":"2024-09-16T07:29:47.199814+0000","last_undegraded":"2024-09-16T07:29:47.199814+0000","last_fullsized":"2024-09-16T07:29:47.199814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:35:14.449871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.221689+0000","last_change":"2024-09-16T07:29:46.218341+0000","last_active":"2024-09-16T07:29:46.221689+0000","last_peered":"2024-09-16T07:29:46.221689+0000","last_clean":"2024-09-16T07:29:46.221689+0000","last_became_active":"2024-09-16T07:29:46.218075+0000","last_became_peered":"2024-09-16T07:29:46.218075+0000","last_unstale":"2024-09-16T07:29:46.221689+0000","last_undegraded":"2024-09-16T07:29:46.221689+0000","last_fullsized":"2024-09-16T07:29:46.221689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:51.529176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.224896+0000","last_change":"2024-09-16T07:29:46.220749+0000","last_active":"2024-09-16T07:29:46.224896+0000","last_peered":"2024-09-16T07:29:46.224896+0000","last_clean":"2024-09-16T07:29:46.224896+0000","last_became_active":"2024-09-16T07:29:46.219839+0000","last_became_peered":"2024-09-16T07:29:46.219839+0000","last_unstale":"2024-09-16T07:29:46.224896+0000","last_undegraded":"2024-09-16T07:29:46.224896+0000","last_fullsized":"2024-09-16T07:29:46.224896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:57:55.424351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202505+0000","last_change":"2024-09-16T07:29:46.239904+0000","last_active":"2024-09-16T07:29:47.202505+0000","last_peered":"2024-09-16T07:29:47.202505+0000","last_clean":"2024-09-16T07:29:47.202505+0000","last_became_active":"2024-09-16T07:29:46.239612+0000","last_became_peered":"2024-09-16T07:29:46.239612+0000","last_unstale":"2024-09-16T07:29:47.202505+0000","last_undegraded":"2024-09-16T07:29:47.202505+0000","last_fullsized":"2024-09-16T07:29:47.202505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:34.011862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850563+0000","last_change":"2024-09-16T07:29:46.222650+0000","last_active":"2024-09-16T07:29:47.850563+0000","last_peered":"2024-09-16T07:29:47.850563+0000","last_clean":"2024-09-16T07:29:47.850563+0000","last_became_active":"2024-09-16T07:29:46.222372+0000","last_became_peered":"2024-09-16T07:29:46.222372+0000","last_unstale":"2024-09-16T07:29:47.850563+0000","last_undegraded":"2024-09-16T07:29:47.850563+0000","last_fullsized":"2024-09-16T07:29:47.850563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:38.648366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850764+0000","last_change":"2024-09-16T07:29:46.214923+0000","last_active":"2024-09-16T07:29:47.850764+0000","last_peered":"2024-09-16T07:29:47.850764+0000","last_clean":"2024-09-16T07:29:47.850764+0000","last_became_active":"2024-09-16T07:29:46.214598+0000","last_became_peered":"2024-09-16T07:29:46.214598+0000","last_unstale":"2024-09-16T07:29:47.850764+0000","last_undegraded":"2024-09-16T07:29:47.850764+0000","last_fullsized":"2024-09-16T07:29:47.850764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:57.817764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.849883+0000","last_change":"2024-09-16T07:29:46.211227+0000","last_active":"2024-09-16T07:29:47.849883+0000","last_peered":"2024-09-16T07:29:47.849883+0000","last_clean":"2024-09-16T07:29:47.849883+0000","last_became_active":"2024-09-16T07:29:46.210983+0000","last_became_peered":"2024-09-16T07:29:46.210983+0000","last_unstale":"2024-09-16T07:29:47.849883+0000","last_undegraded":"2024-09-16T07:29:47.849883+0000","last_fullsized":"2024-09-16T07:29:47.849883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:07:09.675298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.221707+0000","last_change":"2024-09-16T07:29:46.217117+0000","last_active":"2024-09-16T07:29:46.221707+0000","last_peered":"2024-09-16T07:29:46.221707+0000","last_clean":"2024-09-16T07:29:46.221707+0000","last_became_active":"2024-09-16T07:29:46.216757+0000","last_became_peered":"2024-09-16T07:29:46.216757+0000","last_unstale":"2024-09-16T07:29:46.221707+0000","last_undegraded":"2024-09-16T07:29:46.221707+0000","last_fullsized":"2024-09-16T07:29:46.221707+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:55.192855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.852002+0000","last_change":"2024-09-16T07:29:46.220487+0000","last_active":"2024-09-16T07:29:47.852002+0000","last_peered":"2024-09-16T07:29:47.852002+0000","last_clean":"2024-09-16T07:29:47.852002+0000","last_became_active":"2024-09-16T07:29:46.220181+0000","last_became_peered":"2024-09-16T07:29:46.220181+0000","last_unstale":"2024-09-16T07:29:47.852002+0000","last_undegraded":"2024-09-16T07:29:47.852002+0000","last_fullsized":"2024-09-16T07:29:47.852002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:40:06.903224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851288+0000","last_change":"2024-09-16T07:29:46.220580+0000","last_active":"2024-09-16T07:29:47.851288+0000","last_peered":"2024-09-16T07:29:47.851288+0000","last_clean":"2024-09-16T07:29:47.851288+0000","last_became_active":"2024-09-16T07:29:46.220079+0000","last_became_peered":"2024-09-16T07:29:46.220079+0000","last_unstale":"2024-09-16T07:29:47.851288+0000","last_undegraded":"2024-09-16T07:29:47.851288+0000","last_fullsized":"2024-09-16T07:29:47.851288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:28.546376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.222655+0000","last_change":"2024-09-16T07:29:46.217736+0000","last_active":"2024-09-16T07:29:46.222655+0000","last_peered":"2024-09-16T07:29:46.222655+0000","last_clean":"2024-09-16T07:29:46.222655+0000","last_became_active":"2024-09-16T07:29:46.217370+0000","last_became_peered":"2024-09-16T07:29:46.217370+0000","last_unstale":"2024-09-16T07:29:46.222655+0000","last_undegraded":"2024-09-16T07:29:46.222655+0000","last_fullsized":"2024-09-16T07:29:46.222655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:43.567797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.223716+0000","last_change":"2024-09-16T07:29:46.215135+0000","last_active":"2024-09-16T07:29:46.223716+0000","last_peered":"2024-09-16T07:29:46.223716+0000","last_clean":"2024-09-16T07:29:46.223716+0000","last_became_active":"2024-09-16T07:29:46.214886+0000","last_became_peered":"2024-09-16T07:29:46.214886+0000","last_unstale":"2024-09-16T07:29:46.223716+0000","last_undegraded":"2024-09-16T07:29:46.223716+0000","last_fullsized":"2024-09-16T07:29:46.223716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:51:01.994799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.227641+0000","last_change":"2024-09-16T07:29:46.225187+0000","last_active":"2024-09-16T07:29:46.227641+0000","last_peered":"2024-09-16T07:29:46.227641+0000","last_clean":"2024-09-16T07:29:46.227641+0000","last_became_active":"2024-09-16T07:29:46.219710+0000","last_became_peered":"2024-09-16T07:29:46.219710+0000","last_unstale":"2024-09-16T07:29:46.227641+0000","last_undegraded":"2024-09-16T07:29:46.227641+0000","last_fullsized":"2024-09-16T07:29:46.227641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:33.498202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.222541+0000","last_change":"2024-09-16T07:29:46.216660+0000","last_active":"2024-09-16T07:29:46.222541+0000","last_peered":"2024-09-16T07:29:46.222541+0000","last_clean":"2024-09-16T07:29:46.222541+0000","last_became_active":"2024-09-16T07:29:46.216305+0000","last_became_peered":"2024-09-16T07:29:46.216305+0000","last_unstale":"2024-09-16T07:29:46.222541+0000","last_undegraded":"2024-09-16T07:29:46.222541+0000","last_fullsized":"2024-09-16T07:29:46.222541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:51:08.428781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850074+0000","last_change":"2024-09-16T07:29:46.233240+0000","last_active":"2024-09-16T07:29:47.850074+0000","last_peered":"2024-09-16T07:29:47.850074+0000","last_clean":"2024-09-16T07:29:47.850074+0000","last_became_active":"2024-09-16T07:29:46.232485+0000","last_became_peered":"2024-09-16T07:29:46.232485+0000","last_unstale":"2024-09-16T07:29:47.850074+0000","last_undegraded":"2024-09-16T07:29:47.850074+0000","last_fullsized":"2024-09-16T07:29:47.850074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:08.406128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202989+0000","last_change":"2024-09-16T07:29:46.218519+0000","last_active":"2024-09-16T07:29:47.202989+0000","last_peered":"2024-09-16T07:29:47.202989+0000","last_clean":"2024-09-16T07:29:47.202989+0000","last_became_active":"2024-09-16T07:29:46.218285+0000","last_became_peered":"2024-09-16T07:29:46.218285+0000","last_unstale":"2024-09-16T07:29:47.202989+0000","last_undegraded":"2024-09-16T07:29:47.202989+0000","last_fullsized":"2024-09-16T07:29:47.202989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:36:33.184903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202948+0000","last_change":"2024-09-16T07:29:46.217947+0000","last_active":"2024-09-16T07:29:47.202948+0000","last_peered":"2024-09-16T07:29:47.202948+0000","last_clean":"2024-09-16T07:29:47.202948+0000","last_became_active":"2024-09-16T07:29:46.217697+0000","last_became_peered":"2024-09-16T07:29:46.217697+0000","last_unstale":"2024-09-16T07:29:47.202948+0000","last_undegraded":"2024-09-16T07:29:47.202948+0000","last_fullsized":"2024-09-16T07:29:47.202948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:10.304153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.849972+0000","last_change":"2024-09-16T07:29:46.208795+0000","last_active":"2024-09-16T07:29:47.849972+0000","last_peered":"2024-09-16T07:29:47.849972+0000","last_clean":"2024-09-16T07:29:47.849972+0000","last_became_active":"2024-09-16T07:29:46.208565+0000","last_became_peered":"2024-09-16T07:29:46.208565+0000","last_unstale":"2024-09-16T07:29:47.849972+0000","last_undegraded":"2024-09-16T07:29:47.849972+0000","last_fullsized":"2024-09-16T07:29:47.849972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:04:21.269083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202860+0000","last_change":"2024-09-16T07:29:46.221180+0000","last_active":"2024-09-16T07:29:47.202860+0000","last_peered":"2024-09-16T07:29:47.202860+0000","last_clean":"2024-09-16T07:29:47.202860+0000","last_became_active":"2024-09-16T07:29:46.220876+0000","last_became_peered":"2024-09-16T07:29:46.220876+0000","last_unstale":"2024-09-16T07:29:47.202860+0000","last_undegraded":"2024-09-16T07:29:47.202860+0000","last_fullsized":"2024-09-16T07:29:47.202860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:58.436343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199618+0000","last_change":"2024-09-16T07:29:46.236750+0000","last_active":"2024-09-16T07:29:47.199618+0000","last_peered":"2024-09-16T07:29:47.199618+0000","last_clean":"2024-09-16T07:29:47.199618+0000","last_became_active":"2024-09-16T07:29:46.236425+0000","last_became_peered":"2024-09-16T07:29:46.236425+0000","last_unstale":"2024-09-16T07:29:47.199618+0000","last_undegraded":"2024-09-16T07:29:47.199618+0000","last_fullsized":"2024-09-16T07:29:47.199618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:38.358181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850627+0000","last_change":"2024-09-16T07:29:46.223746+0000","last_active":"2024-09-16T07:29:47.850627+0000","last_peered":"2024-09-16T07:29:47.850627+0000","last_clean":"2024-09-16T07:29:47.850627+0000","last_became_active":"2024-09-16T07:29:46.223453+0000","last_became_peered":"2024-09-16T07:29:46.223453+0000","last_unstale":"2024-09-16T07:29:47.850627+0000","last_undegraded":"2024-09-16T07:29:47.850627+0000","last_fullsized":"2024-09-16T07:29:47.850627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:56.672827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850968+0000","last_change":"2024-09-16T07:29:46.232542+0000","last_active":"2024-09-16T07:29:47.850968+0000","last_peered":"2024-09-16T07:29:47.850968+0000","last_clean":"2024-09-16T07:29:47.850968+0000","last_became_active":"2024-09-16T07:29:46.231417+0000","last_became_peered":"2024-09-16T07:29:46.231417+0000","last_unstale":"2024-09-16T07:29:47.850968+0000","last_undegraded":"2024-09-16T07:29:47.850968+0000","last_fullsized":"2024-09-16T07:29:47.850968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:15.061294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850304+0000","last_change":"2024-09-16T07:29:46.215268+0000","last_active":"2024-09-16T07:29:47.850304+0000","last_peered":"2024-09-16T07:29:47.850304+0000","last_clean":"2024-09-16T07:29:47.850304+0000","last_became_active":"2024-09-16T07:29:46.215004+0000","last_became_peered":"2024-09-16T07:29:46.215004+0000","last_unstale":"2024-09-16T07:29:47.850304+0000","last_undegraded":"2024-09-16T07:29:47.850304+0000","last_fullsized":"2024-09-16T07:29:47.850304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:53.540121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.852050+0000","last_change":"2024-09-16T07:29:46.234791+0000","last_active":"2024-09-16T07:29:47.852050+0000","last_peered":"2024-09-16T07:29:47.852050+0000","last_clean":"2024-09-16T07:29:47.852050+0000","last_became_active":"2024-09-16T07:29:46.234620+0000","last_became_peered":"2024-09-16T07:29:46.234620+0000","last_unstale":"2024-09-16T07:29:47.852050+0000","last_undegraded":"2024-09-16T07:29:47.852050+0000","last_fullsized":"2024-09-16T07:29:47.852050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:55.421097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850499+0000","last_change":"2024-09-16T07:29:46.215243+0000","last_active":"2024-09-16T07:29:47.850499+0000","last_peered":"2024-09-16T07:29:47.850499+0000","last_clean":"2024-09-16T07:29:47.850499+0000","last_became_active":"2024-09-16T07:29:46.214960+0000","last_became_peered":"2024-09-16T07:29:46.214960+0000","last_unstale":"2024-09-16T07:29:47.850499+0000","last_undegraded":"2024-09-16T07:29:47.850499+0000","last_fullsized":"2024-09-16T07:29:47.850499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:43.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.203178+0000","last_change":"2024-09-16T07:29:46.227354+0000","last_active":"2024-09-16T07:29:47.203178+0000","last_peered":"2024-09-16T07:29:47.203178+0000","last_clean":"2024-09-16T07:29:47.203178+0000","last_became_active":"2024-09-16T07:29:46.226740+0000","last_became_peered":"2024-09-16T07:29:46.226740+0000","last_unstale":"2024-09-16T07:29:47.203178+0000","last_undegraded":"2024-09-16T07:29:47.203178+0000","last_fullsized":"2024-09-16T07:29:47.203178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:22.242185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199373+0000","last_change":"2024-09-16T07:29:46.231959+0000","last_active":"2024-09-16T07:29:47.199373+0000","last_peered":"2024-09-16T07:29:47.199373+0000","last_clean":"2024-09-16T07:29:47.199373+0000","last_became_active":"2024-09-16T07:29:46.231693+0000","last_became_peered":"2024-09-16T07:29:46.231693+0000","last_unstale":"2024-09-16T07:29:47.199373+0000","last_undegraded":"2024-09-16T07:29:47.199373+0000","last_fullsized":"2024-09-16T07:29:47.199373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:28.000809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.226254+0000","last_change":"2024-09-16T07:29:46.222660+0000","last_active":"2024-09-16T07:29:46.226254+0000","last_peered":"2024-09-16T07:29:46.226254+0000","last_clean":"2024-09-16T07:29:46.226254+0000","last_became_active":"2024-09-16T07:29:46.222410+0000","last_became_peered":"2024-09-16T07:29:46.222410+0000","last_unstale":"2024-09-16T07:29:46.226254+0000","last_undegraded":"2024-09-16T07:29:46.226254+0000","last_fullsized":"2024-09-16T07:29:46.226254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:46.179136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850068+0000","last_change":"2024-09-16T07:29:46.205815+0000","last_active":"2024-09-16T07:29:47.850068+0000","last_peered":"2024-09-16T07:29:47.850068+0000","last_clean":"2024-09-16T07:29:47.850068+0000","last_became_active":"2024-09-16T07:29:46.205537+0000","last_became_peered":"2024-09-16T07:29:46.205537+0000","last_unstale":"2024-09-16T07:29:47.850068+0000","last_undegraded":"2024-09-16T07:29:47.850068+0000","last_fullsized":"2024-09-16T07:29:47.850068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:28:27.067463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199347+0000","last_change":"2024-09-16T07:29:46.227465+0000","last_active":"2024-09-16T07:29:47.199347+0000","last_peered":"2024-09-16T07:29:47.199347+0000","last_clean":"2024-09-16T07:29:47.199347+0000","last_became_active":"2024-09-16T07:29:46.227199+0000","last_became_peered":"2024-09-16T07:29:46.227199+0000","last_unstale":"2024-09-16T07:29:47.199347+0000","last_undegraded":"2024-09-16T07:29:47.199347+0000","last_fullsized":"2024-09-16T07:29:47.199347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:06.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199595+0000","last_change":"2024-09-16T07:29:46.227162+0000","last_active":"2024-09-16T07:29:47.199595+0000","last_peered":"2024-09-16T07:29:47.199595+0000","last_clean":"2024-09-16T07:29:47.199595+0000","last_became_active":"2024-09-16T07:29:46.226540+0000","last_became_peered":"2024-09-16T07:29:46.226540+0000","last_unstale":"2024-09-16T07:29:47.199595+0000","last_undegraded":"2024-09-16T07:29:47.199595+0000","last_fullsized":"2024-09-16T07:29:47.199595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:11.558641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850230+0000","last_change":"2024-09-16T07:29:46.234753+0000","last_active":"2024-09-16T07:29:47.850230+0000","last_peered":"2024-09-16T07:29:47.850230+0000","last_clean":"2024-09-16T07:29:47.850230+0000","last_became_active":"2024-09-16T07:29:46.234510+0000","last_became_peered":"2024-09-16T07:29:46.234510+0000","last_unstale":"2024-09-16T07:29:47.850230+0000","last_undegraded":"2024-09-16T07:29:47.850230+0000","last_fullsized":"2024-09-16T07:29:47.850230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:51.522503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"76'4","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850698+0000","last_change":"2024-09-16T07:29:45.215630+0000","last_active":"2024-09-16T07:29:47.850698+0000","last_peered":"2024-09-16T07:29:47.850698+0000","last_clean":"2024-09-16T07:29:47.850698+0000","last_became_active":"2024-09-16T07:29:45.215364+0000","last_became_peered":"2024-09-16T07:29:45.215364+0000","last_unstale":"2024-09-16T07:29:47.850698+0000","last_undegraded":"2024-09-16T07:29:47.850698+0000","last_fullsized":"2024-09-16T07:29:47.850698+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:40:24.057406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.216951+0000","last_change":"2024-09-16T07:29:46.209844+0000","last_active":"2024-09-16T07:29:46.216951+0000","last_peered":"2024-09-16T07:29:46.216951+0000","last_clean":"2024-09-16T07:29:46.216951+0000","last_became_active":"2024-09-16T07:29:46.209571+0000","last_became_peered":"2024-09-16T07:29:46.209571+0000","last_unstale":"2024-09-16T07:29:46.216951+0000","last_undegraded":"2024-09-16T07:29:46.216951+0000","last_fullsized":"2024-09-16T07:29:46.216951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:34.156942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.203060+0000","last_change":"2024-09-16T07:29:45.210810+0000","last_active":"2024-09-16T07:29:47.203060+0000","last_peered":"2024-09-16T07:29:47.203060+0000","last_clean":"2024-09-16T07:29:47.203060+0000","last_became_active":"2024-09-16T07:29:45.210530+0000","last_became_peered":"2024-09-16T07:29:45.210530+0000","last_unstale":"2024-09-16T07:29:47.203060+0000","last_undegraded":"2024-09-16T07:29:47.203060+0000","last_fullsized":"2024-09-16T07:29:47.203060+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:37.897764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851047+0000","last_change":"2024-09-16T07:29:46.226495+0000","last_active":"2024-09-16T07:29:47.851047+0000","last_peered":"2024-09-16T07:29:47.851047+0000","last_clean":"2024-09-16T07:29:47.851047+0000","last_became_active":"2024-09-16T07:29:46.226193+0000","last_became_peered":"2024-09-16T07:29:46.226193+0000","last_unstale":"2024-09-16T07:29:47.851047+0000","last_undegraded":"2024-09-16T07:29:47.851047+0000","last_fullsized":"2024-09-16T07:29:47.851047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:50.773792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851178+0000","last_change":"2024-09-16T07:29:45.217049+0000","last_active":"2024-09-16T07:29:47.851178+0000","last_peered":"2024-09-16T07:29:47.851178+0000","last_clean":"2024-09-16T07:29:47.851178+0000","last_became_active":"2024-09-16T07:29:45.216682+0000","last_became_peered":"2024-09-16T07:29:45.216682+0000","last_unstale":"2024-09-16T07:29:47.851178+0000","last_undegraded":"2024-09-16T07:29:47.851178+0000","last_fullsized":"2024-09-16T07:29:47.851178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:39:34.394215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851809+0000","last_change":"2024-09-16T07:29:46.216920+0000","last_active":"2024-09-16T07:29:47.851809+0000","last_peered":"2024-09-16T07:29:47.851809+0000","last_clean":"2024-09-16T07:29:47.851809+0000","last_became_active":"2024-09-16T07:29:46.216552+0000","last_became_peered":"2024-09-16T07:29:46.216552+0000","last_unstale":"2024-09-16T07:29:47.851809+0000","last_undegraded":"2024-09-16T07:29:47.851809+0000","last_fullsized":"2024-09-16T07:29:47.851809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:48.127919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850775+0000","last_change":"2024-09-16T07:29:45.212479+0000","last_active":"2024-09-16T07:29:47.850775+0000","last_peered":"2024-09-16T07:29:47.850775+0000","last_clean":"2024-09-16T07:29:47.850775+0000","last_became_active":"2024-09-16T07:29:45.212003+0000","last_became_peered":"2024-09-16T07:29:45.212003+0000","last_unstale":"2024-09-16T07:29:47.850775+0000","last_undegraded":"2024-09-16T07:29:47.850775+0000","last_fullsized":"2024-09-16T07:29:47.850775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:36.797141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850294+0000","last_change":"2024-09-16T07:29:46.218959+0000","last_active":"2024-09-16T07:29:47.850294+0000","last_peered":"2024-09-16T07:29:47.850294+0000","last_clean":"2024-09-16T07:29:47.850294+0000","last_became_active":"2024-09-16T07:29:46.218665+0000","last_became_peered":"2024-09-16T07:29:46.218665+0000","last_unstale":"2024-09-16T07:29:47.850294+0000","last_undegraded":"2024-09-16T07:29:47.850294+0000","last_fullsized":"2024-09-16T07:29:47.850294+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:02:01.400238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"76'1","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.230855+0000","last_change":"2024-09-16T07:29:45.212253+0000","last_active":"2024-09-16T07:29:46.230855+0000","last_peered":"2024-09-16T07:29:46.230855+0000","last_clean":"2024-09-16T07:29:46.230855+0000","last_became_active":"2024-09-16T07:29:45.210152+0000","last_became_peered":"2024-09-16T07:29:45.210152+0000","last_unstale":"2024-09-16T07:29:46.230855+0000","last_undegraded":"2024-09-16T07:29:46.230855+0000","last_fullsized":"2024-09-16T07:29:46.230855+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:49:17.268570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851179+0000","last_change":"2024-09-16T07:29:46.224318+0000","last_active":"2024-09-16T07:29:47.851179+0000","last_peered":"2024-09-16T07:29:47.851179+0000","last_clean":"2024-09-16T07:29:47.851179+0000","last_became_active":"2024-09-16T07:29:46.221813+0000","last_became_peered":"2024-09-16T07:29:46.221813+0000","last_unstale":"2024-09-16T07:29:47.851179+0000","last_undegraded":"2024-09-16T07:29:47.851179+0000","last_fullsized":"2024-09-16T07:29:47.851179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:34.260941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851248+0000","last_change":"2024-09-16T07:29:45.216366+0000","last_active":"2024-09-16T07:29:47.851248+0000","last_peered":"2024-09-16T07:29:47.851248+0000","last_clean":"2024-09-16T07:29:47.851248+0000","last_became_active":"2024-09-16T07:29:45.215950+0000","last_became_peered":"2024-09-16T07:29:45.215950+0000","last_unstale":"2024-09-16T07:29:47.851248+0000","last_undegraded":"2024-09-16T07:29:47.851248+0000","last_fullsized":"2024-09-16T07:29:47.851248+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:10.662687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.216848+0000","last_change":"2024-09-16T07:29:46.210584+0000","last_active":"2024-09-16T07:29:46.216848+0000","last_peered":"2024-09-16T07:29:46.216848+0000","last_clean":"2024-09-16T07:29:46.216848+0000","last_became_active":"2024-09-16T07:29:46.210276+0000","last_became_peered":"2024-09-16T07:29:46.210276+0000","last_unstale":"2024-09-16T07:29:46.216848+0000","last_undegraded":"2024-09-16T07:29:46.216848+0000","last_fullsized":"2024-09-16T07:29:46.216848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:18.494034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.203605+0000","last_change":"2024-09-16T07:29:45.215117+0000","last_active":"2024-09-16T07:29:46.203605+0000","last_peered":"2024-09-16T07:29:46.203605+0000","last_clean":"2024-09-16T07:29:46.203605+0000","last_became_active":"2024-09-16T07:29:45.214120+0000","last_became_peered":"2024-09-16T07:29:45.214120+0000","last_unstale":"2024-09-16T07:29:46.203605+0000","last_undegraded":"2024-09-16T07:29:46.203605+0000","last_fullsized":"2024-09-16T07:29:46.203605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:06.245988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.225068+0000","last_change":"2024-09-16T07:29:46.218571+0000","last_active":"2024-09-16T07:29:46.225068+0000","last_peered":"2024-09-16T07:29:46.225068+0000","last_clean":"2024-09-16T07:29:46.225068+0000","last_became_active":"2024-09-16T07:29:46.214625+0000","last_became_peered":"2024-09-16T07:29:46.214625+0000","last_unstale":"2024-09-16T07:29:46.225068+0000","last_undegraded":"2024-09-16T07:29:46.225068+0000","last_fullsized":"2024-09-16T07:29:46.225068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:45.363499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.209485+0000","last_change":"2024-09-16T07:29:45.214541+0000","last_active":"2024-09-16T07:29:46.209485+0000","last_peered":"2024-09-16T07:29:46.209485+0000","last_clean":"2024-09-16T07:29:46.209485+0000","last_became_active":"2024-09-16T07:29:45.213012+0000","last_became_peered":"2024-09-16T07:29:45.213012+0000","last_unstale":"2024-09-16T07:29:46.209485+0000","last_undegraded":"2024-09-16T07:29:46.209485+0000","last_fullsized":"2024-09-16T07:29:46.209485+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:18.876183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851951+0000","last_change":"2024-09-16T07:29:46.232098+0000","last_active":"2024-09-16T07:29:47.851951+0000","last_peered":"2024-09-16T07:29:47.851951+0000","last_clean":"2024-09-16T07:29:47.851951+0000","last_became_active":"2024-09-16T07:29:46.231786+0000","last_became_peered":"2024-09-16T07:29:46.231786+0000","last_unstale":"2024-09-16T07:29:47.851951+0000","last_undegraded":"2024-09-16T07:29:47.851951+0000","last_fullsized":"2024-09-16T07:29:47.851951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:47.550704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850255+0000","last_change":"2024-09-16T07:29:45.214536+0000","last_active":"2024-09-16T07:29:47.850255+0000","last_peered":"2024-09-16T07:29:47.850255+0000","last_clean":"2024-09-16T07:29:47.850255+0000","last_became_active":"2024-09-16T07:29:45.211407+0000","last_became_peered":"2024-09-16T07:29:45.211407+0000","last_unstale":"2024-09-16T07:29:47.850255+0000","last_undegraded":"2024-09-16T07:29:47.850255+0000","last_fullsized":"2024-09-16T07:29:47.850255+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:32.720751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199476+0000","last_change":"2024-09-16T07:29:46.231824+0000","last_active":"2024-09-16T07:29:47.199476+0000","last_peered":"2024-09-16T07:29:47.199476+0000","last_clean":"2024-09-16T07:29:47.199476+0000","last_became_active":"2024-09-16T07:29:46.231553+0000","last_became_peered":"2024-09-16T07:29:46.231553+0000","last_unstale":"2024-09-16T07:29:47.199476+0000","last_undegraded":"2024-09-16T07:29:47.199476+0000","last_fullsized":"2024-09-16T07:29:47.199476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:01.097180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199905+0000","last_change":"2024-09-16T07:29:45.207401+0000","last_active":"2024-09-16T07:29:47.199905+0000","last_peered":"2024-09-16T07:29:47.199905+0000","last_clean":"2024-09-16T07:29:47.199905+0000","last_became_active":"2024-09-16T07:29:45.207093+0000","last_became_peered":"2024-09-16T07:29:45.207093+0000","last_unstale":"2024-09-16T07:29:47.199905+0000","last_undegraded":"2024-09-16T07:29:47.199905+0000","last_fullsized":"2024-09-16T07:29:47.199905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:59.558582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850356+0000","last_change":"2024-09-16T07:29:46.218073+0000","last_active":"2024-09-16T07:29:47.850356+0000","last_peered":"2024-09-16T07:29:47.850356+0000","last_clean":"2024-09-16T07:29:47.850356+0000","last_became_active":"2024-09-16T07:29:46.217808+0000","last_became_peered":"2024-09-16T07:29:46.217808+0000","last_unstale":"2024-09-16T07:29:47.850356+0000","last_undegraded":"2024-09-16T07:29:47.850356+0000","last_fullsized":"2024-09-16T07:29:47.850356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:24.174496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"76'3","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.234424+0000","last_change":"2024-09-16T07:29:45.208752+0000","last_active":"2024-09-16T07:29:46.234424+0000","last_peered":"2024-09-16T07:29:46.234424+0000","last_clean":"2024-09-16T07:29:46.234424+0000","last_became_active":"2024-09-16T07:29:45.208420+0000","last_became_peered":"2024-09-16T07:29:45.208420+0000","last_unstale":"2024-09-16T07:29:46.234424+0000","last_undegraded":"2024-09-16T07:29:46.234424+0000","last_fullsized":"2024-09-16T07:29:46.234424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:03.501445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202994+0000","last_change":"2024-09-16T07:29:46.226982+0000","last_active":"2024-09-16T07:29:47.202994+0000","last_peered":"2024-09-16T07:29:47.202994+0000","last_clean":"2024-09-16T07:29:47.202994+0000","last_became_active":"2024-09-16T07:29:46.226377+0000","last_became_peered":"2024-09-16T07:29:46.226377+0000","last_unstale":"2024-09-16T07:29:47.202994+0000","last_undegraded":"2024-09-16T07:29:47.202994+0000","last_fullsized":"2024-09-16T07:29:47.202994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:15.148341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"76'6","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.234174+0000","last_change":"2024-09-16T07:29:45.215376+0000","last_active":"2024-09-16T07:29:46.234174+0000","last_peered":"2024-09-16T07:29:46.234174+0000","last_clean":"2024-09-16T07:29:46.234174+0000","last_became_active":"2024-09-16T07:29:45.214835+0000","last_became_peered":"2024-09-16T07:29:45.214835+0000","last_unstale":"2024-09-16T07:29:46.234174+0000","last_undegraded":"2024-09-16T07:29:46.234174+0000","last_fullsized":"2024-09-16T07:29:46.234174+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:20.276225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850852+0000","last_change":"2024-09-16T07:29:46.217791+0000","last_active":"2024-09-16T07:29:47.850852+0000","last_peered":"2024-09-16T07:29:47.850852+0000","last_clean":"2024-09-16T07:29:47.850852+0000","last_became_active":"2024-09-16T07:29:46.217559+0000","last_became_peered":"2024-09-16T07:29:46.217559+0000","last_unstale":"2024-09-16T07:29:47.850852+0000","last_undegraded":"2024-09-16T07:29:47.850852+0000","last_fullsized":"2024-09-16T07:29:47.850852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:34:38.508059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"76'2","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.232940+0000","last_change":"2024-09-16T07:29:45.218660+0000","last_active":"2024-09-16T07:29:46.232940+0000","last_peered":"2024-09-16T07:29:46.232940+0000","last_clean":"2024-09-16T07:29:46.232940+0000","last_became_active":"2024-09-16T07:29:45.218368+0000","last_became_peered":"2024-09-16T07:29:45.218368+0000","last_unstale":"2024-09-16T07:29:46.232940+0000","last_undegraded":"2024-09-16T07:29:46.232940+0000","last_fullsized":"2024-09-16T07:29:46.232940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:23.785072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851502+0000","last_change":"2024-09-16T07:29:46.224902+0000","last_active":"2024-09-16T07:29:47.851502+0000","last_peered":"2024-09-16T07:29:47.851502+0000","last_clean":"2024-09-16T07:29:47.851502+0000","last_became_active":"2024-09-16T07:29:46.224645+0000","last_became_peered":"2024-09-16T07:29:46.224645+0000","last_unstale":"2024-09-16T07:29:47.851502+0000","last_undegraded":"2024-09-16T07:29:47.851502+0000","last_fullsized":"2024-09-16T07:29:47.851502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:13.478659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851334+0000","last_change":"2024-09-16T07:29:45.213669+0000","last_active":"2024-09-16T07:29:47.851334+0000","last_peered":"2024-09-16T07:29:47.851334+0000","last_clean":"2024-09-16T07:29:47.851334+0000","last_became_active":"2024-09-16T07:29:45.212779+0000","last_became_peered":"2024-09-16T07:29:45.212779+0000","last_unstale":"2024-09-16T07:29:47.851334+0000","last_undegraded":"2024-09-16T07:29:47.851334+0000","last_fullsized":"2024-09-16T07:29:47.851334+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:56:01.449658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.201677+0000","last_change":"2024-09-16T07:29:46.214720+0000","last_active":"2024-09-16T07:29:47.201677+0000","last_peered":"2024-09-16T07:29:47.201677+0000","last_clean":"2024-09-16T07:29:47.201677+0000","last_became_active":"2024-09-16T07:29:46.214391+0000","last_became_peered":"2024-09-16T07:29:46.214391+0000","last_unstale":"2024-09-16T07:29:47.201677+0000","last_undegraded":"2024-09-16T07:29:47.201677+0000","last_fullsized":"2024-09-16T07:29:47.201677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:10:13.303867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851017+0000","last_change":"2024-09-16T07:29:45.217377+0000","last_active":"2024-09-16T07:29:47.851017+0000","last_peered":"2024-09-16T07:29:47.851017+0000","last_clean":"2024-09-16T07:29:47.851017+0000","last_became_active":"2024-09-16T07:29:45.217140+0000","last_became_peered":"2024-09-16T07:29:45.217140+0000","last_unstale":"2024-09-16T07:29:47.851017+0000","last_undegraded":"2024-09-16T07:29:47.851017+0000","last_fullsized":"2024-09-16T07:29:47.851017+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:17.990262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.216852+0000","last_change":"2024-09-16T07:29:46.210110+0000","last_active":"2024-09-16T07:29:46.216852+0000","last_peered":"2024-09-16T07:29:46.216852+0000","last_clean":"2024-09-16T07:29:46.216852+0000","last_became_active":"2024-09-16T07:29:46.207997+0000","last_became_peered":"2024-09-16T07:29:46.207997+0000","last_unstale":"2024-09-16T07:29:46.216852+0000","last_undegraded":"2024-09-16T07:29:46.216852+0000","last_fullsized":"2024-09-16T07:29:46.216852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:09.019859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"76'8","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.849752+0000","last_change":"2024-09-16T07:29:45.210634+0000","last_active":"2024-09-16T07:29:47.849752+0000","last_peered":"2024-09-16T07:29:47.849752+0000","last_clean":"2024-09-16T07:29:47.849752+0000","last_became_active":"2024-09-16T07:29:45.210309+0000","last_became_peered":"2024-09-16T07:29:45.210309+0000","last_unstale":"2024-09-16T07:29:47.849752+0000","last_undegraded":"2024-09-16T07:29:47.849752+0000","last_fullsized":"2024-09-16T07:29:47.849752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.618520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199810+0000","last_change":"2024-09-16T07:29:46.231140+0000","last_active":"2024-09-16T07:29:47.199810+0000","last_peered":"2024-09-16T07:29:47.199810+0000","last_clean":"2024-09-16T07:29:47.199810+0000","last_became_active":"2024-09-16T07:29:46.230832+0000","last_became_peered":"2024-09-16T07:29:46.230832+0000","last_unstale":"2024-09-16T07:29:47.199810+0000","last_undegraded":"2024-09-16T07:29:47.199810+0000","last_fullsized":"2024-09-16T07:29:47.199810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:56.672827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850353+0000","last_change":"2024-09-16T07:29:45.210912+0000","last_active":"2024-09-16T07:29:47.850353+0000","last_peered":"2024-09-16T07:29:47.850353+0000","last_clean":"2024-09-16T07:29:47.850353+0000","last_became_active":"2024-09-16T07:29:45.210626+0000","last_became_peered":"2024-09-16T07:29:45.210626+0000","last_unstale":"2024-09-16T07:29:47.850353+0000","last_undegraded":"2024-09-16T07:29:47.850353+0000","last_fullsized":"2024-09-16T07:29:47.850353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:15:21.365918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199538+0000","last_change":"2024-09-16T07:29:46.232689+0000","last_active":"2024-09-16T07:29:47.199538+0000","last_peered":"2024-09-16T07:29:47.199538+0000","last_clean":"2024-09-16T07:29:47.199538+0000","last_became_active":"2024-09-16T07:29:46.232163+0000","last_became_peered":"2024-09-16T07:29:46.232163+0000","last_unstale":"2024-09-16T07:29:47.199538+0000","last_undegraded":"2024-09-16T07:29:47.199538+0000","last_fullsized":"2024-09-16T07:29:47.199538+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:54.010018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199928+0000","last_change":"2024-09-16T07:29:45.214545+0000","last_active":"2024-09-16T07:29:47.199928+0000","last_peered":"2024-09-16T07:29:47.199928+0000","last_clean":"2024-09-16T07:29:47.199928+0000","last_became_active":"2024-09-16T07:29:45.214138+0000","last_became_peered":"2024-09-16T07:29:45.214138+0000","last_unstale":"2024-09-16T07:29:47.199928+0000","last_undegraded":"2024-09-16T07:29:47.199928+0000","last_fullsized":"2024-09-16T07:29:47.199928+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:44.412756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.222716+0000","last_change":"2024-09-16T07:29:46.215016+0000","last_active":"2024-09-16T07:29:46.222716+0000","last_peered":"2024-09-16T07:29:46.222716+0000","last_clean":"2024-09-16T07:29:46.222716+0000","last_became_active":"2024-09-16T07:29:46.214705+0000","last_became_peered":"2024-09-16T07:29:46.214705+0000","last_unstale":"2024-09-16T07:29:46.222716+0000","last_undegraded":"2024-09-16T07:29:46.222716+0000","last_fullsized":"2024-09-16T07:29:46.222716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:42.531575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"76'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199317+0000","last_change":"2024-09-16T07:29:45.217958+0000","last_active":"2024-09-16T07:29:47.199317+0000","last_peered":"2024-09-16T07:29:47.199317+0000","last_clean":"2024-09-16T07:29:47.199317+0000","last_became_active":"2024-09-16T07:29:45.212831+0000","last_became_peered":"2024-09-16T07:29:45.212831+0000","last_unstale":"2024-09-16T07:29:47.199317+0000","last_undegraded":"2024-09-16T07:29:47.199317+0000","last_fullsized":"2024-09-16T07:29:47.199317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:23.292574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.226147+0000","last_change":"2024-09-16T07:29:46.221969+0000","last_active":"2024-09-16T07:29:46.226147+0000","last_peered":"2024-09-16T07:29:46.226147+0000","last_clean":"2024-09-16T07:29:46.226147+0000","last_became_active":"2024-09-16T07:29:46.220296+0000","last_became_peered":"2024-09-16T07:29:46.220296+0000","last_unstale":"2024-09-16T07:29:46.226147+0000","last_undegraded":"2024-09-16T07:29:46.226147+0000","last_fullsized":"2024-09-16T07:29:46.226147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:59.544926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850891+0000","last_change":"2024-09-16T07:29:45.220225+0000","last_active":"2024-09-16T07:29:47.850891+0000","last_peered":"2024-09-16T07:29:47.850891+0000","last_clean":"2024-09-16T07:29:47.850891+0000","last_became_active":"2024-09-16T07:29:45.219946+0000","last_became_peered":"2024-09-16T07:29:45.219946+0000","last_unstale":"2024-09-16T07:29:47.850891+0000","last_undegraded":"2024-09-16T07:29:47.850891+0000","last_fullsized":"2024-09-16T07:29:47.850891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:40:59.070975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.212762+0000","last_change":"2024-09-16T07:29:46.206959+0000","last_active":"2024-09-16T07:29:46.212762+0000","last_peered":"2024-09-16T07:29:46.212762+0000","last_clean":"2024-09-16T07:29:46.212762+0000","last_became_active":"2024-09-16T07:29:46.206704+0000","last_became_peered":"2024-09-16T07:29:46.206704+0000","last_unstale":"2024-09-16T07:29:46.212762+0000","last_undegraded":"2024-09-16T07:29:46.212762+0000","last_fullsized":"2024-09-16T07:29:46.212762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:01:44.742300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850982+0000","last_change":"2024-09-16T07:29:45.216924+0000","last_active":"2024-09-16T07:29:47.850982+0000","last_peered":"2024-09-16T07:29:47.850982+0000","last_clean":"2024-09-16T07:29:47.850982+0000","last_became_active":"2024-09-16T07:29:45.216362+0000","last_became_peered":"2024-09-16T07:29:45.216362+0000","last_unstale":"2024-09-16T07:29:47.850982+0000","last_undegraded":"2024-09-16T07:29:47.850982+0000","last_fullsized":"2024-09-16T07:29:47.850982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:32.319459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.225272+0000","last_change":"2024-09-16T07:29:46.219188+0000","last_active":"2024-09-16T07:29:46.225272+0000","last_peered":"2024-09-16T07:29:46.225272+0000","last_clean":"2024-09-16T07:29:46.225272+0000","last_became_active":"2024-09-16T07:29:46.217110+0000","last_became_peered":"2024-09-16T07:29:46.217110+0000","last_unstale":"2024-09-16T07:29:46.225272+0000","last_undegraded":"2024-09-16T07:29:46.225272+0000","last_fullsized":"2024-09-16T07:29:46.225272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:01:17.765673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199459+0000","last_change":"2024-09-16T07:29:45.203058+0000","last_active":"2024-09-16T07:29:47.199459+0000","last_peered":"2024-09-16T07:29:47.199459+0000","last_clean":"2024-09-16T07:29:47.199459+0000","last_became_active":"2024-09-16T07:29:45.202771+0000","last_became_peered":"2024-09-16T07:29:45.202771+0000","last_unstale":"2024-09-16T07:29:47.199459+0000","last_undegraded":"2024-09-16T07:29:47.199459+0000","last_fullsized":"2024-09-16T07:29:47.199459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:47.863659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.218785+0000","last_change":"2024-09-16T07:29:46.215064+0000","last_active":"2024-09-16T07:29:46.218785+0000","last_peered":"2024-09-16T07:29:46.218785+0000","last_clean":"2024-09-16T07:29:46.218785+0000","last_became_active":"2024-09-16T07:29:46.212423+0000","last_became_peered":"2024-09-16T07:29:46.212423+0000","last_unstale":"2024-09-16T07:29:46.218785+0000","last_undegraded":"2024-09-16T07:29:46.218785+0000","last_fullsized":"2024-09-16T07:29:46.218785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:11.028118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.201451+0000","last_change":"2024-09-16T07:29:45.206969+0000","last_active":"2024-09-16T07:29:47.201451+0000","last_peered":"2024-09-16T07:29:47.201451+0000","last_clean":"2024-09-16T07:29:47.201451+0000","last_became_active":"2024-09-16T07:29:45.206652+0000","last_became_peered":"2024-09-16T07:29:45.206652+0000","last_unstale":"2024-09-16T07:29:47.201451+0000","last_undegraded":"2024-09-16T07:29:47.201451+0000","last_fullsized":"2024-09-16T07:29:47.201451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:13.712685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.203110+0000","last_change":"2024-09-16T07:29:46.223922+0000","last_active":"2024-09-16T07:29:47.203110+0000","last_peered":"2024-09-16T07:29:47.203110+0000","last_clean":"2024-09-16T07:29:47.203110+0000","last_became_active":"2024-09-16T07:29:46.223638+0000","last_became_peered":"2024-09-16T07:29:46.223638+0000","last_unstale":"2024-09-16T07:29:47.203110+0000","last_undegraded":"2024-09-16T07:29:47.203110+0000","last_fullsized":"2024-09-16T07:29:47.203110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:13.554430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850457+0000","last_change":"2024-09-16T07:29:45.216141+0000","last_active":"2024-09-16T07:29:47.850457+0000","last_peered":"2024-09-16T07:29:47.850457+0000","last_clean":"2024-09-16T07:29:47.850457+0000","last_became_active":"2024-09-16T07:29:45.214870+0000","last_became_peered":"2024-09-16T07:29:45.214870+0000","last_unstale":"2024-09-16T07:29:47.850457+0000","last_undegraded":"2024-09-16T07:29:47.850457+0000","last_fullsized":"2024-09-16T07:29:47.850457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:56.976391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.852084+0000","last_change":"2024-09-16T07:29:46.221561+0000","last_active":"2024-09-16T07:29:47.852084+0000","last_peered":"2024-09-16T07:29:47.852084+0000","last_clean":"2024-09-16T07:29:47.852084+0000","last_became_active":"2024-09-16T07:29:46.221320+0000","last_became_peered":"2024-09-16T07:29:46.221320+0000","last_unstale":"2024-09-16T07:29:47.852084+0000","last_undegraded":"2024-09-16T07:29:47.852084+0000","last_fullsized":"2024-09-16T07:29:47.852084+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:12.480186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"77'1","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:48.201780+0000","last_change":"2024-09-16T07:29:45.215831+0000","last_active":"2024-09-16T07:29:48.201780+0000","last_peered":"2024-09-16T07:29:48.201780+0000","last_clean":"2024-09-16T07:29:48.201780+0000","last_became_active":"2024-09-16T07:29:45.214188+0000","last_became_peered":"2024-09-16T07:29:45.214188+0000","last_unstale":"2024-09-16T07:29:48.201780+0000","last_undegraded":"2024-09-16T07:29:48.201780+0000","last_fullsized":"2024-09-16T07:29:48.201780+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:31.321559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199564+0000","last_change":"2024-09-16T07:29:46.228875+0000","last_active":"2024-09-16T07:29:47.199564+0000","last_peered":"2024-09-16T07:29:47.199564+0000","last_clean":"2024-09-16T07:29:47.199564+0000","last_became_active":"2024-09-16T07:29:46.226387+0000","last_became_peered":"2024-09-16T07:29:46.226387+0000","last_unstale":"2024-09-16T07:29:47.199564+0000","last_undegraded":"2024-09-16T07:29:47.199564+0000","last_fullsized":"2024-09-16T07:29:47.199564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:05.090204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.211726+0000","last_change":"2024-09-16T07:29:45.206950+0000","last_active":"2024-09-16T07:29:46.211726+0000","last_peered":"2024-09-16T07:29:46.211726+0000","last_clean":"2024-09-16T07:29:46.211726+0000","last_became_active":"2024-09-16T07:29:45.206663+0000","last_became_peered":"2024-09-16T07:29:45.206663+0000","last_unstale":"2024-09-16T07:29:46.211726+0000","last_undegraded":"2024-09-16T07:29:46.211726+0000","last_fullsized":"2024-09-16T07:29:46.211726+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:27:31.965496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851894+0000","last_change":"2024-09-16T07:29:46.214221+0000","last_active":"2024-09-16T07:29:47.851894+0000","last_peered":"2024-09-16T07:29:47.851894+0000","last_clean":"2024-09-16T07:29:47.851894+0000","last_became_active":"2024-09-16T07:29:46.213906+0000","last_became_peered":"2024-09-16T07:29:46.213906+0000","last_unstale":"2024-09-16T07:29:47.851894+0000","last_undegraded":"2024-09-16T07:29:47.851894+0000","last_fullsized":"2024-09-16T07:29:47.851894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:46.210739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.211621+0000","last_change":"2024-09-16T07:29:45.208847+0000","last_active":"2024-09-16T07:29:46.211621+0000","last_peered":"2024-09-16T07:29:46.211621+0000","last_clean":"2024-09-16T07:29:46.211621+0000","last_became_active":"2024-09-16T07:29:45.208554+0000","last_became_peered":"2024-09-16T07:29:45.208554+0000","last_unstale":"2024-09-16T07:29:46.211621+0000","last_undegraded":"2024-09-16T07:29:46.211621+0000","last_fullsized":"2024-09-16T07:29:46.211621+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:36.140097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202304+0000","last_change":"2024-09-16T07:29:46.221265+0000","last_active":"2024-09-16T07:29:47.202304+0000","last_peered":"2024-09-16T07:29:47.202304+0000","last_clean":"2024-09-16T07:29:47.202304+0000","last_became_active":"2024-09-16T07:29:46.220946+0000","last_became_peered":"2024-09-16T07:29:46.220946+0000","last_unstale":"2024-09-16T07:29:47.202304+0000","last_undegraded":"2024-09-16T07:29:47.202304+0000","last_fullsized":"2024-09-16T07:29:47.202304+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:18.644653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.211640+0000","last_change":"2024-09-16T07:29:45.210343+0000","last_active":"2024-09-16T07:29:46.211640+0000","last_peered":"2024-09-16T07:29:46.211640+0000","last_clean":"2024-09-16T07:29:46.211640+0000","last_became_active":"2024-09-16T07:29:45.209867+0000","last_became_peered":"2024-09-16T07:29:45.209867+0000","last_unstale":"2024-09-16T07:29:46.211640+0000","last_undegraded":"2024-09-16T07:29:46.211640+0000","last_fullsized":"2024-09-16T07:29:46.211640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:49.721218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.850183+0000","last_change":"2024-09-16T07:29:46.238314+0000","last_active":"2024-09-16T07:29:47.850183+0000","last_peered":"2024-09-16T07:29:47.850183+0000","last_clean":"2024-09-16T07:29:47.850183+0000","last_became_active":"2024-09-16T07:29:46.238063+0000","last_became_peered":"2024-09-16T07:29:46.238063+0000","last_unstale":"2024-09-16T07:29:47.850183+0000","last_undegraded":"2024-09-16T07:29:47.850183+0000","last_fullsized":"2024-09-16T07:29:47.850183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:26.962159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202371+0000","last_change":"2024-09-16T07:29:45.212711+0000","last_active":"2024-09-16T07:29:47.202371+0000","last_peered":"2024-09-16T07:29:47.202371+0000","last_clean":"2024-09-16T07:29:47.202371+0000","last_became_active":"2024-09-16T07:29:45.212443+0000","last_became_peered":"2024-09-16T07:29:45.212443+0000","last_unstale":"2024-09-16T07:29:47.202371+0000","last_undegraded":"2024-09-16T07:29:47.202371+0000","last_fullsized":"2024-09-16T07:29:47.202371+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:53.383357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T07:29:46.229558+0000","last_change":"2024-09-16T07:29:46.228412+0000","last_active":"2024-09-16T07:29:46.229558+0000","last_peered":"2024-09-16T07:29:46.229558+0000","last_clean":"2024-09-16T07:29:46.229558+0000","last_became_active":"2024-09-16T07:29:46.228084+0000","last_became_peered":"2024-09-16T07:29:46.228084+0000","last_unstale":"2024-09-16T07:29:46.229558+0000","last_undegraded":"2024-09-16T07:29:46.229558+0000","last_fullsized":"2024-09-16T07:29:46.229558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:32:11.020949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199551+0000","last_change":"2024-09-16T07:29:45.208685+0000","last_active":"2024-09-16T07:29:47.199551+0000","last_peered":"2024-09-16T07:29:47.199551+0000","last_clean":"2024-09-16T07:29:47.199551+0000","last_became_active":"2024-09-16T07:29:45.207127+0000","last_became_peered":"2024-09-16T07:29:45.207127+0000","last_unstale":"2024-09-16T07:29:47.199551+0000","last_undegraded":"2024-09-16T07:29:47.199551+0000","last_fullsized":"2024-09-16T07:29:47.199551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:30:29.740440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.199620+0000","last_change":"2024-09-16T07:29:46.232291+0000","last_active":"2024-09-16T07:29:47.199620+0000","last_peered":"2024-09-16T07:29:47.199620+0000","last_clean":"2024-09-16T07:29:47.199620+0000","last_became_active":"2024-09-16T07:29:46.231899+0000","last_became_peered":"2024-09-16T07:29:46.231899+0000","last_unstale":"2024-09-16T07:29:47.199620+0000","last_undegraded":"2024-09-16T07:29:47.199620+0000","last_fullsized":"2024-09-16T07:29:47.199620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:40:25.060768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.849716+0000","last_change":"2024-09-16T07:29:45.210160+0000","last_active":"2024-09-16T07:29:47.849716+0000","last_peered":"2024-09-16T07:29:47.849716+0000","last_clean":"2024-09-16T07:29:47.849716+0000","last_became_active":"2024-09-16T07:29:45.209905+0000","last_became_peered":"2024-09-16T07:29:45.209905+0000","last_unstale":"2024-09-16T07:29:47.849716+0000","last_undegraded":"2024-09-16T07:29:47.849716+0000","last_fullsized":"2024-09-16T07:29:47.849716+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:07:25.518394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.202113+0000","last_change":"2024-09-16T07:29:46.221500+0000","last_active":"2024-09-16T07:29:47.202113+0000","last_peered":"2024-09-16T07:29:47.202113+0000","last_clean":"2024-09-16T07:29:47.202113+0000","last_became_active":"2024-09-16T07:29:46.221202+0000","last_became_peered":"2024-09-16T07:29:46.221202+0000","last_unstale":"2024-09-16T07:29:47.202113+0000","last_undegraded":"2024-09-16T07:29:47.202113+0000","last_fullsized":"2024-09-16T07:29:47.202113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:45.187877+0000","last_clean_scrub_stamp":"2024-09-16T07:29:45.187877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:53.293800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"76'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.851562+0000","last_change":"2024-09-16T07:29:45.219328+0000","last_active":"2024-09-16T07:29:47.851562+0000","last_peered":"2024-09-16T07:29:47.851562+0000","last_clean":"2024-09-16T07:29:47.851562+0000","last_became_active":"2024-09-16T07:29:45.219066+0000","last_became_peered":"2024-09-16T07:29:45.219066+0000","last_unstale":"2024-09-16T07:29:47.851562+0000","last_undegraded":"2024-09-16T07:29:47.851562+0000","last_fullsized":"2024-09-16T07:29:47.851562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:29:44.184515+0000","last_clean_scrub_stamp":"2024-09-16T07:29:44.184515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:43.389915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":212,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T07:29:47.849764+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:29:47.849764+0000","last_peered":"2024-09-16T07:29:47.849764+0000","last_clean":"2024-09-16T07:29:47.849764+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:29:47.849764+0000","last_undegraded":"2024-09-16T07:29:47.849764+0000","last_fullsized":"2024-09-16T07:29:47.849764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476827,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50160,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321680,"statfs":{"total":96636764160,"available":96585400320,"internally_reserved":0,"allocated":937984,"data_stored":395738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476827,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50152,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321688,"statfs":{"total":96636764160,"available":96585408512,"internally_reserved":0,"allocated":929792,"data_stored":396194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476827,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46564,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325276,"statfs":{"total":96636764160,"available":96589082624,"internally_reserved":0,"allocated":1519616,"data_stored":985980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476826,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50152,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321688,"statfs":{"total":96636764160,"available":96585408512,"internally_reserved":0,"allocated":929792,"data_stored":395682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50148,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321692,"statfs":{"total":96636764160,"available":96585412608,"internally_reserved":0,"allocated":925696,"data_stored":390896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50144,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321696,"statfs":{"total":96636764160,"available":96585416704,"internally_reserved":0,"allocated":929792,"data_stored":390930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509530,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54884,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316956,"statfs":{"total":96636764160,"available":96580562944,"internally_reserved":0,"allocated":1523712,"data_stored":986070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542232,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54280,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317560,"statfs":{"total":96636764160,"available":96581181440,"internally_reserved":0,"allocated":901120,"data_stored":382481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:29:50.832 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:29:50.832 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:29:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.032+0000 7f55f7200640 1 -- 172.21.15.71:0/2392860032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f00734a0 msgr2=0x7f55f006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.032+0000 7f55f7200640 1 --2- 172.21.15.71:0/2392860032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f00734a0 0x7f55f006f960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009c10 tx=0x7f55e802a940 comp rx=0 tx=0).stop 2024-09-16T07:29:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.032+0000 7f55f7200640 1 -- 172.21.15.71:0/2392860032 shutdown_connections 2024-09-16T07:29:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.032+0000 7f55f7200640 1 --2- 172.21.15.71:0/2392860032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f00734a0 0x7f55f006f960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.032+0000 7f55f7200640 1 -- 172.21.15.71:0/2392860032 >> 172.21.15.71:0/2392860032 conn(0x7f55f00b2130 msgr2=0x7f55f00b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.032+0000 7f55f7200640 1 -- 172.21.15.71:0/2392860032 shutdown_connections 2024-09-16T07:29:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.032+0000 7f55f7200640 1 -- 172.21.15.71:0/2392860032 wait complete. 2024-09-16T07:29:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.034+0000 7f55f7200640 1 Processor -- start 2024-09-16T07:29:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.034+0000 7f55f7200640 1 -- start start 2024-09-16T07:29:51.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f7200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f0073030 0x7f55f00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:51.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f7200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f00734a0 0x7f55f01f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:51.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f0151cb0 0x7f55f014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55f006fea0 con 0x7f55f0151cb0 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f7200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0074280 con 0x7f55f00734a0 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f7200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55f0070020 con 0x7f55f0073030 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f0151cb0 0x7f55f014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f0151cb0 0x7f55f014ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33200/0 (socket says 172.21.15.71:33200) 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55f5200640 1 -- 172.21.15.71:0/2389929590 learned_addr learned my addr 172.21.15.71:0/2389929590 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55ef400640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f00734a0 0x7f55f01f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.035+0000 7f55efe00640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f0073030 0x7f55f00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:51.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55f5200640 1 -- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f0073030 msgr2=0x7f55f00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55f5200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f0073030 0x7f55f00cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55f5200640 1 -- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f00734a0 msgr2=0x7f55f01f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55f5200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f00734a0 0x7f55f01f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55f5200640 1 -- 172.21.15.71:0/2389929590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f00b07f0 con 0x7f55f0151cb0 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55ef400640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f00734a0 0x7f55f01f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55efe00640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f0073030 0x7f55f00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55f5200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f0151cb0 0x7f55f014ff50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009c10 tx=0x7f55e8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55e7e00640 1 -- 172.21.15.71:0/2389929590 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8004040 con 0x7f55f0151cb0 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55e7e00640 1 -- 172.21.15.71:0/2389929590 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e80041e0 con 0x7f55f0151cb0 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55e7e00640 1 -- 172.21.15.71:0/2389929590 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e80045f0 con 0x7f55f0151cb0 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f00b0ad0 con 0x7f55f0151cb0 2024-09-16T07:29:51.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.036+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f00cc330 con 0x7f55f0151cb0 2024-09-16T07:29:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.037+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f00b37d0 con 0x7f55f0151cb0 2024-09-16T07:29:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.039+0000 7f55e7e00640 1 -- 172.21.15.71:0/2389929590 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f55e803a450 con 0x7f55f0151cb0 2024-09-16T07:29:51.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.039+0000 7f55e7e00640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55d0079420 0x7f55d007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:51.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.040+0000 7f55e7e00640 1 -- 172.21.15.71:0/2389929590 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f55e80bd6a0 con 0x7f55f0151cb0 2024-09-16T07:29:51.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.040+0000 7f55efe00640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55d0079420 0x7f55d007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:51.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.041+0000 7f55efe00640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55d0079420 0x7f55d007b8e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f55e0000f60 tx=0x7f55e0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:51.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.043+0000 7f55e7e00640 1 -- 172.21.15.71:0/2389929590 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f55e8084e20 con 0x7f55f0151cb0 2024-09-16T07:29:51.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.299+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f55f006d900 con 0x7f55f0151cb0 2024-09-16T07:29:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.304+0000 7f55e7e00640 1 -- 172.21.15.71:0/2389929590 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f55e8084720 con 0x7f55f0151cb0 2024-09-16T07:29:51.304 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:29:51.304 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:29:51.304 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:29:51.304 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:29:51.304 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:29:51.304 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:29:51.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55d0079420 msgr2=0x7f55d007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55d0079420 0x7f55d007b8e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f55e0000f60 tx=0x7f55e0009040 comp rx=0 tx=0).stop 2024-09-16T07:29:51.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f0151cb0 msgr2=0x7f55f014ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f0151cb0 0x7f55f014ff50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009c10 tx=0x7f55e8003ba0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 shutdown_connections 2024-09-16T07:29:51.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f55d0079420 0x7f55d007b8e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f0151cb0 0x7f55f014ff50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f00734a0 0x7f55f01f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 --2- 172.21.15.71:0/2389929590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f0073030 0x7f55f00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 >> 172.21.15.71:0/2389929590 conn(0x7f55f00b2130 msgr2=0x7f55f00b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:51.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 shutdown_connections 2024-09-16T07:29:51.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.306+0000 7f55f7200640 1 -- 172.21.15.71:0/2389929590 wait complete. 2024-09-16T07:29:51.322 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:29:51.423 INFO:tasks.ceph.mds.c.smithi071.stderr:2024-09-16T07:29:51.423+0000 7f46a6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:29:51.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.479+0000 7fb0ff400640 1 -- 172.21.15.71:0/506414009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80ccf00 msgr2=0x7fb0f80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.479+0000 7fb0ff400640 1 --2- 172.21.15.71:0/506414009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80ccf00 0x7fb0f80b2440 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4005fd0 tx=0x7fb0e402a540 comp rx=0 tx=0).stop 2024-09-16T07:29:51.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.479+0000 7fb0ff400640 1 -- 172.21.15.71:0/506414009 shutdown_connections 2024-09-16T07:29:51.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.479+0000 7fb0ff400640 1 --2- 172.21.15.71:0/506414009 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f8073fb0 0x7fb0f8074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.479+0000 7fb0ff400640 1 --2- 172.21.15.71:0/506414009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80ccf00 0x7fb0f80b2440 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.479+0000 7fb0ff400640 1 -- 172.21.15.71:0/506414009 >> 172.21.15.71:0/506414009 conn(0x7fb0f80b1950 msgr2=0x7fb0f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:51.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.479+0000 7fb0ff400640 1 -- 172.21.15.71:0/506414009 shutdown_connections 2024-09-16T07:29:51.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.479+0000 7fb0ff400640 1 -- 172.21.15.71:0/506414009 wait complete. 2024-09-16T07:29:51.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.481+0000 7fb0ff400640 1 Processor -- start 2024-09-16T07:29:51.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.481+0000 7fb0ff400640 1 -- start start 2024-09-16T07:29:51.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0ff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f8073fb0 0x7fb0f8154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:51.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0ff400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f80ccf00 0x7fb0f80cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:51.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0ff400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f815f5a0 0x7fb0f814a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:51.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0ff400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f806f2e0 con 0x7fb0f8073fb0 2024-09-16T07:29:51.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0ff400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f806f160 con 0x7fb0f815f5a0 2024-09-16T07:29:51.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0ff400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f806f460 con 0x7fb0f80ccf00 2024-09-16T07:29:51.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f8073fb0 0x7fb0f8154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:51.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f8073fb0 0x7fb0f8154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33220/0 (socket says 172.21.15.71:33220) 2024-09-16T07:29:51.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fca00640 1 -- 172.21.15.71:0/1982799175 learned_addr learned my addr 172.21.15.71:0/1982799175 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:51.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0f7e00640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f80ccf00 0x7fb0f80cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:51.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fd400640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f815f5a0 0x7fb0f814a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:51.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fca00640 1 -- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f80ccf00 msgr2=0x7fb0f80cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fca00640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f80ccf00 0x7fb0f80cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fca00640 1 -- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f815f5a0 msgr2=0x7fb0f814a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fca00640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f815f5a0 0x7fb0f814a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fca00640 1 -- 172.21.15.71:0/1982799175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f80cba60 con 0x7fb0f8073fb0 2024-09-16T07:29:51.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.482+0000 7fb0fd400640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f815f5a0 0x7fb0f814a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:51.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.483+0000 7fb0fca00640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f8073fb0 0x7fb0f8154d30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c3a0 tx=0x7fb0e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:51.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.483+0000 7fb0f4c00640 1 -- 172.21.15.71:0/1982799175 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8017070 con 0x7fb0f8073fb0 2024-09-16T07:29:51.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.483+0000 7fb0f4c00640 1 -- 172.21.15.71:0/1982799175 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0e80053c0 con 0x7fb0f8073fb0 2024-09-16T07:29:51.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.483+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f814aab0 con 0x7fb0f8073fb0 2024-09-16T07:29:51.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.483+0000 7fb0f4c00640 1 -- 172.21.15.71:0/1982799175 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8003850 con 0x7fb0f8073fb0 2024-09-16T07:29:51.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.483+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f81e4910 con 0x7fb0f8073fb0 2024-09-16T07:29:51.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.484+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f806f980 con 0x7fb0f8073fb0 2024-09-16T07:29:51.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.485+0000 7fb0f4c00640 1 -- 172.21.15.71:0/1982799175 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fb0e801a070 con 0x7fb0f8073fb0 2024-09-16T07:29:51.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.487+0000 7fb0f4c00640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d8079420 0x7fb0d807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:51.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.487+0000 7fb0f4c00640 1 -- 172.21.15.71:0/1982799175 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fb0e8098900 con 0x7fb0f8073fb0 2024-09-16T07:29:51.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.487+0000 7fb0f7e00640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d8079420 0x7fb0d807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:51.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.488+0000 7fb0f7e00640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d8079420 0x7fb0d807b8e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4033000 tx=0x7fb0e4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:51.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.491+0000 7fb0f4c00640 1 -- 172.21.15.71:0/1982799175 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb0e80620b0 con 0x7fb0f8073fb0 2024-09-16T07:29:51.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.709+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0f80acff0 con 0x7fb0f8073fb0 2024-09-16T07:29:51.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.710+0000 7fb0f4c00640 1 -- 172.21.15.71:0/1982799175 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5231 (secure 0 0 0) 0x7fb0e8066ae0 con 0x7fb0f8073fb0 2024-09-16T07:29:51.711 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:29:51.711 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":46,"btime":"2024-09-16T07:29:49:226790+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/4181257111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":4181257111},{"type":"v1","addr":"172.21.15.177:6836","nonce":4181257111}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2868012570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2868012570},{"type":"v1","addr":"172.21.15.177:6837","nonce":2868012570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6414,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3075140097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3075140097},{"type":"v1","addr":"172.21.15.71:6834","nonce":3075140097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:29:45.618471+0000","modified":"2024-09-16T07:29:48.640333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6417},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6417":{"gid":6417,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1371001821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":1371001821},{"type":"v1","addr":"172.21.15.71:6835","nonce":1371001821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6417,"qdb_cluster":[6417]},"id":5}]} 2024-09-16T07:29:51.711 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 46 2024-09-16T07:29:51.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d8079420 msgr2=0x7fb0d807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d8079420 0x7fb0d807b8e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4033000 tx=0x7fb0e4004730 comp rx=0 tx=0).stop 2024-09-16T07:29:51.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f8073fb0 msgr2=0x7fb0f8154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:51.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f8073fb0 0x7fb0f8154d30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c3a0 tx=0x7fb0e800c860 comp rx=0 tx=0).stop 2024-09-16T07:29:51.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 shutdown_connections 2024-09-16T07:29:51.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d8079420 0x7fb0d807b8e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f815f5a0 0x7fb0f814a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f80ccf00 0x7fb0f80cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 --2- 172.21.15.71:0/1982799175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f8073fb0 0x7fb0f8154d30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:51.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 >> 172.21.15.71:0/1982799175 conn(0x7fb0f80b1950 msgr2=0x7fb0f809ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:51.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 shutdown_connections 2024-09-16T07:29:51.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:51.712+0000 7fb0ff400640 1 -- 172.21.15.71:0/1982799175 wait complete. 2024-09-16T07:29:52.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:29:45.618471+0000', 'modified': '2024-09-16T07:29:48.640333+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6417}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6417': {'gid': 6417, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/1371001821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6833', 'nonce': 1371001821}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 1371001821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6417, 'qdb_cluster': [6417]} 2024-09-16T07:29:52.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:29:52.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T07:29:52.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.289+0000 7fd11e600640 1 -- 172.21.15.71:0/2576169674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b4020 msgr2=0x7fd1180b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.289+0000 7fd11e600640 1 --2- 172.21.15.71:0/2576169674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b4020 0x7fd1180b1770 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd1180d4ea0 tx=0x7fd11002a9f0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.290+0000 7fd11e600640 1 -- 172.21.15.71:0/2576169674 shutdown_connections 2024-09-16T07:29:52.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.290+0000 7fd11e600640 1 --2- 172.21.15.71:0/2576169674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b4020 0x7fd1180b1770 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.290+0000 7fd11e600640 1 --2- 172.21.15.71:0/2576169674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd118161440 0x7fd1180b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.290+0000 7fd11e600640 1 -- 172.21.15.71:0/2576169674 >> 172.21.15.71:0/2576169674 conn(0x7fd1180d3df0 msgr2=0x7fd1180d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:52.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.290+0000 7fd11e600640 1 -- 172.21.15.71:0/2576169674 shutdown_connections 2024-09-16T07:29:52.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.290+0000 7fd11e600640 1 -- 172.21.15.71:0/2576169674 wait complete. 2024-09-16T07:29:52.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.291+0000 7fd11e600640 1 Processor -- start 2024-09-16T07:29:52.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.291+0000 7fd11e600640 1 -- start start 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b34b0 0x7fd1180caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11e600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1180b4020 0x7fd11806e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11e600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd118161440 0x7fd1181f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1180b2040 con 0x7fd1180b34b0 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11e600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1180b1ec0 con 0x7fd1180b4020 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11e600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1180b21c0 con 0x7fd118161440 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b34b0 0x7fd1180caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b34b0 0x7fd1180caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33240/0 (socket says 172.21.15.71:33240) 2024-09-16T07:29:52.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.292+0000 7fd11d200640 1 -- 172.21.15.71:0/2615128031 learned_addr learned my addr 172.21.15.71:0/2615128031 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd11d200640 1 -- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd118161440 msgr2=0x7fd1181f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd11dc00640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd118161440 0x7fd1181f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd117e00640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1180b4020 0x7fd11806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd11d200640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd118161440 0x7fd1181f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd11d200640 1 -- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1180b4020 msgr2=0x7fd11806e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd11d200640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1180b4020 0x7fd11806e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd11d200640 1 -- 172.21.15.71:0/2615128031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1180ccaa0 con 0x7fd1180b34b0 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd117e00640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1180b4020 0x7fd11806e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd11d200640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b34b0 0x7fd1180caec0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00d6d0 tx=0x7fd10c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd114c00640 1 -- 172.21.15.71:0/2615128031 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c017070 con 0x7fd1180b34b0 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd114c00640 1 -- 172.21.15.71:0/2615128031 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd10c005330 con 0x7fd1180b34b0 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd114c00640 1 -- 172.21.15.71:0/2615128031 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c009820 con 0x7fd1180b34b0 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.293+0000 7fd11e600640 1 -- 172.21.15.71:0/2615128031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1180b72f0 con 0x7fd1180b34b0 2024-09-16T07:29:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.294+0000 7fd11e600640 1 -- 172.21.15.71:0/2615128031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1180ccd20 con 0x7fd1180b34b0 2024-09-16T07:29:52.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.296+0000 7fd114c00640 1 -- 172.21.15.71:0/2615128031 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fd10c009bd0 con 0x7fd1180b34b0 2024-09-16T07:29:52.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.297+0000 7fd114c00640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0f8079420 0x7fd0f807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.297+0000 7fd114c00640 1 -- 172.21.15.71:0/2615128031 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fd10c099900 con 0x7fd1180b34b0 2024-09-16T07:29:52.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.297+0000 7fd122c5e740 1 Processor -- start 2024-09-16T07:29:52.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.297+0000 7fd122c5e740 1 -- start start 2024-09-16T07:29:52.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.297+0000 7fd117e00640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0f8079420 0x7fd0f807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c593e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54ec10 0x561d2c592580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c592ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd122c5e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561d2c4d1430 con 0x561d2c54ec10 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd122c5e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561d2c4c15d0 con 0x561d2c54f190 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd122c5e740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561d2c4b0a00 con 0x561d2c593aa0 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd117e00640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0f8079420 0x7fd0f807b8e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd11806da20 tx=0x7fd104009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd117e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54ec10 0x561d2c592580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd117e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54ec10 0x561d2c592580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33256/0 (socket says 172.21.15.71:33256) 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd117e00640 1 -- 172.21.15.71:0/2827667217 learned_addr learned my addr 172.21.15.71:0/2827667217 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd11d200640 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c593e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.298+0000 7fd11dc00640 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c592ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd117e00640 1 -- 172.21.15.71:0/2827667217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 msgr2=0x561d2c593e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:29:52.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd117e00640 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c593e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd117e00640 1 -- 172.21.15.71:0/2827667217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 msgr2=0x561d2c592ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:29:52.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd117e00640 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c592ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd117e00640 1 -- 172.21.15.71:0/2827667217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2c479520 con 0x561d2c54ec10 2024-09-16T07:29:52.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd117e00640 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54ec10 0x561d2c592580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd10400d9e0 tx=0x7fd104010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5305de1e009347f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:52.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd109600640 1 -- 172.21.15.71:0/2827667217 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104024070 con 0x561d2c54ec10 2024-09-16T07:29:52.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd109600640 1 -- 172.21.15.71:0/2827667217 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd104014db0 con 0x561d2c54ec10 2024-09-16T07:29:52.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.299+0000 7fd109600640 1 -- 172.21.15.71:0/2827667217 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10400dea0 con 0x561d2c54ec10 2024-09-16T07:29:52.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 -- 172.21.15.71:0/2827667217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54ec10 msgr2=0x561d2c592580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54ec10 0x561d2c592580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd10400d9e0 tx=0x7fd104010000 comp rx=0 tx=0).stop 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 -- 172.21.15.71:0/2827667217 shutdown_connections 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c592ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54ec10 0x561d2c592580 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2827667217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c593e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 -- 172.21.15.71:0/2827667217 >> 172.21.15.71:0/2827667217 conn(0x561d2c54a1e0 msgr2=0x561d2c54dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 -- 172.21.15.71:0/2827667217 shutdown_connections 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.300+0000 7fd122c5e740 1 -- 172.21.15.71:0/2827667217 wait complete. 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.301+0000 7fd122c5e740 1 -- start start 2024-09-16T07:29:52.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.301+0000 7fd122c5e740 1 Processor -- start 2024-09-16T07:29:52.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 0x561d2c590560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c54da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 0x561d2c5a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd122c5e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561d2c457330 con 0x561d2c593aa0 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd122c5e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561d2c4b4810 con 0x561d2c54df90 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd122c5e740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561d2c221620 con 0x561d2c54f190 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd122c5e740 10 client.0 _ll_register_callbacks cb 0x561d2c45c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd117e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c54da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd117e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c54da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33262/0 (socket says 172.21.15.71:33262) 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd117e00640 1 -- 172.21.15.71:0/2732416721 learned_addr learned my addr 172.21.15.71:0/2732416721 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd11d200640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 0x561d2c590560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.302+0000 7fd11dc00640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 0x561d2c5a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd117e00640 1 -- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 msgr2=0x561d2c590560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd117e00640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 0x561d2c590560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd117e00640 1 -- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 msgr2=0x561d2c5a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd117e00640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 0x561d2c5a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd117e00640 1 -- 172.21.15.71:0/2732416721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2c212850 con 0x561d2c593aa0 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd11d200640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 0x561d2c590560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd11dc00640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 0x561d2c5a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd117e00640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c54da50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd104022040 tx=0x7fd10401f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d811a5ed7f7f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd0fe000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd0fe000640 1 -- 172.21.15.71:0/2732416721 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104024040 con 0x561d2c593aa0 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd0fe000640 1 -- 172.21.15.71:0/2732416721 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fd104007030 con 0x561d2c593aa0 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd122c5e740 1 -- 172.21.15.71:0/2732416721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd104014db0 con 0x561d2c593aa0 2024-09-16T07:29:52.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd0fe000640 1 -- 172.21.15.71:0/2732416721 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd104036050 con 0x561d2c593aa0 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd122c5e740 10 client.6540 fetch_fsmap learned FSMap version 46 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.303+0000 7fd122c5e740 1 -- 172.21.15.71:0/2732416721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x561d2c2896b0 con 0x561d2c593aa0 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd0fe000640 1 -- 172.21.15.71:0/2732416721 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd104014db0 con 0x561d2c593aa0 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd0fe000640 1 -- 172.21.15.71:0/2732416721 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7fd104039020 con 0x561d2c593aa0 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd122c5e740 10 client.6540 fetch_fsmap finished waiting for FSMap version 46 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd122c5e740 10 client.6540 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd122c5e740 20 client.6540 populate_metadata read hostname 'smithi071' 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] conn(0x561d2c592290 0x561d2c592670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd122c5e740 4 client.6540 mds_command: new command op to 6417 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd122c5e740 1 -- 172.21.15.71:0/2732416721 --> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d2c4c3520 con 0x561d2c592290 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.304+0000 7fd11d200640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] conn(0x561d2c592290 0x561d2c592670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.305+0000 7fd11d200640 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] conn(0x561d2c592290 0x561d2c592670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8b6a514db5d48f server_cookie=dbb97644eb24f0d1 in_seq=0 out_seq=0 2024-09-16T07:29:52.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.305+0000 7fd0fe000640 10 client.6540 ms_handle_connect on v2:172.21.15.71:6833/1371001821 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.305+0000 7fd0fe000640 1 -- 172.21.15.71:0/2732416721 <== mds.0 v2:172.21.15.71:6833/1371001821 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d2c4c3520 con 0x561d2c592290 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.305+0000 7fd0fe000640 10 client.6540 handle_command_reply: tid=0 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.305+0000 7fd122c5e740 1 client.6540 shutdown 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 -- 172.21.15.71:0/2732416721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 msgr2=0x561d2c54da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c54da50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd104022040 tx=0x7fd10401f630 comp rx=0 tx=0).stop 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 -- 172.21.15.71:0/2732416721 shutdown_connections 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] conn(0x561d2c592290 0x561d2c592670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 0x561d2c5a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c54da50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 --2- 172.21.15.71:0/2732416721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 0x561d2c590560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd0fe000640 0 client.6540 ms_handle_reset on v2:172.21.15.71:6833/1371001821 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 -- 172.21.15.71:0/2732416721 >> 172.21.15.71:0/2732416721 conn(0x561d2c54ec10 msgr2=0x561d2c58b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 -- 172.21.15.71:0/2732416721 shutdown_connections 2024-09-16T07:29:52.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 1 -- 172.21.15.71:0/2732416721 wait complete. 2024-09-16T07:29:52.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.306+0000 7fd122c5e740 20 client.6540 trim_cache size 0 max 16384 2024-09-16T07:29:52.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.325+0000 7fd122c5e740 1 Processor -- start 2024-09-16T07:29:52.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.325+0000 7fd122c5e740 1 -- start start 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 0x561d2c60df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 0x561d2c5470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd122c5e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1182001a0 con 0x561d2c593aa0 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd122c5e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1180c8300 con 0x561d2c54df90 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd122c5e740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1181ffd00 con 0x561d2c54f190 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd11dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd11dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33278/0 (socket says 172.21.15.71:33278) 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd11dc00640 1 -- 172.21.15.71:0/1091397116 learned_addr learned my addr 172.21.15.71:0/1091397116 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd11dc00640 1 -- 172.21.15.71:0/1091397116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 msgr2=0x561d2c5470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:29:52.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd11dc00640 1 --2- 172.21.15.71:0/1091397116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 0x561d2c5470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd11dc00640 1 -- 172.21.15.71:0/1091397116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 msgr2=0x561d2c60df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:29:52.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd11dc00640 1 --2- 172.21.15.71:0/1091397116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 0x561d2c60df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.326+0000 7fd11dc00640 1 -- 172.21.15.71:0/1091397116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2c589260 con 0x561d2c593aa0 2024-09-16T07:29:52.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd11dc00640 1 --2- 172.21.15.71:0/1091397116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c547630 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd110009d60 tx=0x7fd110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4547475980534c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:52.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd108c00640 1 -- 172.21.15.71:0/1091397116 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110048070 con 0x561d2c593aa0 2024-09-16T07:29:52.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd108c00640 1 -- 172.21.15.71:0/1091397116 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd110003ae0 con 0x561d2c593aa0 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd108c00640 1 -- 172.21.15.71:0/1091397116 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110033040 con 0x561d2c593aa0 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd122c5e740 1 -- 172.21.15.71:0/1091397116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 msgr2=0x561d2c547630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd122c5e740 1 --2- 172.21.15.71:0/1091397116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c547630 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd110009d60 tx=0x7fd110004a40 comp rx=0 tx=0).stop 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd122c5e740 1 -- 172.21.15.71:0/1091397116 shutdown_connections 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd122c5e740 1 --2- 172.21.15.71:0/1091397116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd122c5e740 1 --2- 172.21.15.71:0/1091397116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c54f190 0x561d2c5470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd122c5e740 1 --2- 172.21.15.71:0/1091397116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54df90 0x561d2c60df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.327+0000 7fd122c5e740 1 -- 172.21.15.71:0/1091397116 >> 172.21.15.71:0/1091397116 conn(0x561d2c54ec10 msgr2=0x561d2c58f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:52.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.328+0000 7fd122c5e740 1 -- 172.21.15.71:0/1091397116 shutdown_connections 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.328+0000 7fd122c5e740 1 -- 172.21.15.71:0/1091397116 wait complete. 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.328+0000 7fd122c5e740 1 -- start start 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.328+0000 7fd122c5e740 1 Processor -- start 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.329+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54df90 0x561d2c5b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.329+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c5470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.329+0000 7fd122c5e740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.329+0000 7fd122c5e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561d2c565600 con 0x561d2c54df90 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.329+0000 7fd122c5e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561d2c5a38a0 con 0x561d2c54f190 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.329+0000 7fd122c5e740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561d2c5629d0 con 0x561d2c593aa0 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.329+0000 7fd122c5e740 10 client.0 _ll_register_callbacks cb 0x7fd10c05f5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54df90 0x561d2c5b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54df90 0x561d2c5b6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33294/0 (socket says 172.21.15.71:33294) 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 -- 172.21.15.71:0/3554173205 learned_addr learned my addr 172.21.15.71:0/3554173205 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11dc00640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd117e00640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c5470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 -- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 msgr2=0x561d2c547630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 -- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 msgr2=0x561d2c5470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c5470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 -- 172.21.15.71:0/3554173205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2c5902f0 con 0x561d2c54df90 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11dc00640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd117e00640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c5470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:29:52.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd11d200640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54df90 0x561d2c5b6530 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd10c060210 tx=0x7fd10c062710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b176927a793dca22 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd0fe000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd0fe000640 1 -- 172.21.15.71:0/3554173205 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c07c020 con 0x561d2c54df90 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd0fe000640 1 -- 172.21.15.71:0/3554173205 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fd10c09c450 con 0x561d2c54df90 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.330+0000 7fd122c5e740 1 -- 172.21.15.71:0/3554173205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd110003ae0 con 0x561d2c54df90 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd0fe000640 1 -- 172.21.15.71:0/3554173205 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd10c09b040 con 0x561d2c54df90 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd122c5e740 10 client.6546 fetch_fsmap learned FSMap version 46 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd122c5e740 1 -- 172.21.15.71:0/3554173205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x561d2c5b6ed0 con 0x561d2c54df90 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd0fe000640 1 -- 172.21.15.71:0/3554173205 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd110003ae0 con 0x561d2c54df90 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd0fe000640 1 -- 172.21.15.71:0/3554173205 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7fd10c06cb80 con 0x561d2c54df90 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd122c5e740 10 client.6546 fetch_fsmap finished waiting for FSMap version 46 2024-09-16T07:29:52.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd122c5e740 10 client.6546 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T07:29:52.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd122c5e740 20 client.6546 populate_metadata read hostname 'smithi071' 2024-09-16T07:29:52.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd122c5e740 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] conn(0x561d2c573f00 0x561d2c5742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd122c5e740 4 client.6546 mds_command: new command op to 6417 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:29:52.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd122c5e740 1 -- 172.21.15.71:0/3554173205 --> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] -- command(tid 0: {"prefix": "status"}) -- 0x561d2c4befd0 con 0x561d2c573f00 2024-09-16T07:29:52.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.331+0000 7fd117e00640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] conn(0x561d2c573f00 0x561d2c5742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd117e00640 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] conn(0x561d2c573f00 0x561d2c5742e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9fc84117a95ed86 server_cookie=28f6d66ca7142e2e in_seq=0 out_seq=0 2024-09-16T07:29:52.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd0fe000640 10 client.6546 ms_handle_connect on v2:172.21.15.71:6833/1371001821 2024-09-16T07:29:52.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd0fe000640 1 -- 172.21.15.71:0/3554173205 <== mds.0 v2:172.21.15.71:6833/1371001821 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x561d2c4befd0 con 0x561d2c573f00 2024-09-16T07:29:52.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd0fe000640 10 client.6546 handle_command_reply: tid=0 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 client.6546 shutdown 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 -- 172.21.15.71:0/3554173205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54df90 msgr2=0x561d2c5b6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54df90 0x561d2c5b6530 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd10c060210 tx=0x7fd10c062710 comp rx=0 tx=0).stop 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 -- 172.21.15.71:0/3554173205 shutdown_connections 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.71:6833/1371001821,v1:172.21.15.71:6835/1371001821] conn(0x561d2c573f00 0x561d2c5742e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d2c593aa0 0x561d2c547630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d2c54f190 0x561d2c5470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd0fe000640 0 client.6546 ms_handle_reset on v2:172.21.15.71:6833/1371001821 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 --2- 172.21.15.71:0/3554173205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2c54df90 0x561d2c5b6530 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 -- 172.21.15.71:0/3554173205 >> 172.21.15.71:0/3554173205 conn(0x561d2c54ec10 msgr2=0x561d2c58f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 -- 172.21.15.71:0/3554173205 shutdown_connections 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 1 -- 172.21.15.71:0/3554173205 wait complete. 2024-09-16T07:29:52.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.332+0000 7fd122c5e740 20 client.6546 trim_cache size 0 max 16384 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6417, 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 6.1378734110000002, 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 46, 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 77, 2024-09-16T07:29:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 77, 2024-09-16T07:29:52.340 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 11.200557492 2024-09-16T07:29:52.340 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:29:52.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 -- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0f8079420 msgr2=0x7fd0f807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0f8079420 0x7fd0f807b8e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd11806da20 tx=0x7fd104009040 comp rx=0 tx=0).stop 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 -- 172.21.15.71:0/2615128031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b34b0 msgr2=0x7fd1180caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b34b0 0x7fd1180caec0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00d6d0 tx=0x7fd10c00db90 comp rx=0 tx=0).stop 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 -- 172.21.15.71:0/2615128031 shutdown_connections 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0f8079420 0x7fd0f807b8e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd118161440 0x7fd1181f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1180b4020 0x7fd11806e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 --2- 172.21.15.71:0/2615128031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1180b34b0 0x7fd1180caec0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 -- 172.21.15.71:0/2615128031 >> 172.21.15.71:0/2615128031 conn(0x7fd1180d3df0 msgr2=0x7fd1180b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 -- 172.21.15.71:0/2615128031 shutdown_connections 2024-09-16T07:29:52.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.333+0000 7fd11e600640 1 -- 172.21.15.71:0/2615128031 wait complete. 2024-09-16T07:29:52.349 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:29:52.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.505+0000 7ff383000640 1 -- 172.21.15.71:0/2140923614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c15f5a0 msgr2=0x7ff37c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.505+0000 7ff383000640 1 --2- 172.21.15.71:0/2140923614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c15f5a0 0x7ff37c0b2440 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff36c005fd0 tx=0x7ff36c02aa10 comp rx=0 tx=0).stop 2024-09-16T07:29:52.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.506+0000 7ff383000640 1 -- 172.21.15.71:0/2140923614 shutdown_connections 2024-09-16T07:29:52.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.506+0000 7ff383000640 1 --2- 172.21.15.71:0/2140923614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c15f5a0 0x7ff37c0b2440 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.506+0000 7ff383000640 1 -- 172.21.15.71:0/2140923614 >> 172.21.15.71:0/2140923614 conn(0x7ff37c0b1950 msgr2=0x7ff37c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:52.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.506+0000 7ff383000640 1 -- 172.21.15.71:0/2140923614 shutdown_connections 2024-09-16T07:29:52.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.506+0000 7ff383000640 1 -- 172.21.15.71:0/2140923614 wait complete. 2024-09-16T07:29:52.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.507+0000 7ff383000640 1 Processor -- start 2024-09-16T07:29:52.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.507+0000 7ff383000640 1 -- start start 2024-09-16T07:29:52.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.508+0000 7ff383000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c073fb0 0x7ff37c14cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.508+0000 7ff383000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff37c0ccf00 0x7ff37c146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.508+0000 7ff383000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff37c15f5a0 0x7ff37c0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.508+0000 7ff383000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c06f9f0 con 0x7ff37c073fb0 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.508+0000 7ff383000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c06f870 con 0x7ff37c0ccf00 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.508+0000 7ff383000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff37c074420 con 0x7ff37c15f5a0 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c073fb0 0x7ff37c14cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c073fb0 0x7ff37c14cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33312/0 (socket says 172.21.15.71:33312) 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 -- 172.21.15.71:0/1851533985 learned_addr learned my addr 172.21.15.71:0/1851533985 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37b400640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff37c0ccf00 0x7ff37c146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff381000640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff37c15f5a0 0x7ff37c0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 -- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff37c15f5a0 msgr2=0x7ff37c0cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff37c15f5a0 0x7ff37c0cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 -- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff37c0ccf00 msgr2=0x7ff37c146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff37c0ccf00 0x7ff37c146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 -- 172.21.15.71:0/1851533985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c0b3d20 con 0x7ff37c073fb0 2024-09-16T07:29:52.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.509+0000 7ff37be00640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c073fb0 0x7ff37c14cb90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff36c000f80 tx=0x7ff36c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:52.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.510+0000 7ff373e00640 1 -- 172.21.15.71:0/1851533985 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c046070 con 0x7ff37c073fb0 2024-09-16T07:29:52.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.510+0000 7ff373e00640 1 -- 172.21.15.71:0/1851533985 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff36c0046c0 con 0x7ff37c073fb0 2024-09-16T07:29:52.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.510+0000 7ff373e00640 1 -- 172.21.15.71:0/1851533985 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c033070 con 0x7ff37c073fb0 2024-09-16T07:29:52.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.510+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37c0b4000 con 0x7ff37c073fb0 2024-09-16T07:29:52.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.510+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37c0b45c0 con 0x7ff37c073fb0 2024-09-16T07:29:52.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.511+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37c0b2440 con 0x7ff37c073fb0 2024-09-16T07:29:52.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.512+0000 7ff373e00640 1 -- 172.21.15.71:0/1851533985 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7ff36c036070 con 0x7ff37c073fb0 2024-09-16T07:29:52.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.513+0000 7ff373e00640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff35c079420 0x7ff35c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:52.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.513+0000 7ff373e00640 1 -- 172.21.15.71:0/1851533985 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7ff36c0ba1a0 con 0x7ff37c073fb0 2024-09-16T07:29:52.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.514+0000 7ff37b400640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff35c079420 0x7ff35c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:52.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.515+0000 7ff37b400640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff35c079420 0x7ff35c07b8e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0c9580 tx=0x7ff368009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:52.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.518+0000 7ff373e00640 1 -- 172.21.15.71:0/1851533985 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff36c03e070 con 0x7ff37c073fb0 2024-09-16T07:29:52.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.738+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff37c155200 con 0x7ff37c073fb0 2024-09-16T07:29:52.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.739+0000 7ff373e00640 1 -- 172.21.15.71:0/1851533985 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7ff36c084260 con 0x7ff37c073fb0 2024-09-16T07:29:52.741 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:29:52.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff35c079420 msgr2=0x7ff35c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff35c079420 0x7ff35c07b8e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0c9580 tx=0x7ff368009040 comp rx=0 tx=0).stop 2024-09-16T07:29:52.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c073fb0 msgr2=0x7ff37c14cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:52.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c073fb0 0x7ff37c14cb90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff36c000f80 tx=0x7ff36c003f90 comp rx=0 tx=0).stop 2024-09-16T07:29:52.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 shutdown_connections 2024-09-16T07:29:52.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff35c079420 0x7ff35c07b8e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff37c15f5a0 0x7ff37c0cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff37c0ccf00 0x7ff37c146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 --2- 172.21.15.71:0/1851533985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff37c073fb0 0x7ff37c14cb90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:52.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 >> 172.21.15.71:0/1851533985 conn(0x7ff37c0b1950 msgr2=0x7ff37c154af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:52.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 shutdown_connections 2024-09-16T07:29:52.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:52.741+0000 7ff383000640 1 -- 172.21.15.71:0/1851533985 wait complete. 2024-09-16T07:29:52.758 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:29:52.758 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:29:52.758 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:29:52.758 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:29:52.759 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:29:52.759 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:29:52.759 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:29:52.759 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:29:52.759 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:29:52.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:52.759 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:29:52.795 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42284sec preferred_lft 42284sec 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:29:52.796 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:29:52.797 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:29:52.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:52.798 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:29:52.888 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:29:52.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:52.892 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:29:52.928 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:29:52.928 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:29:52.928 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:29:52.928 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:29:52.929 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:29:52.929 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:29:52.929 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:29:52.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:29:52.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:29:52.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:29:52.929 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:29:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:29:53.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:29:53.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:53.124 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:29:53.159 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:29:53.159 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:29:53.160 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:29:53.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:53.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:29:53.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:29:53.257 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:29:53.292 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:29:53.292 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:29:53.292 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:29:53.367 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:29:53.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:53.367 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:29:53.495 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:29:53.524 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:29:53.524 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:29:53.524 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:29:53.524 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:29:53.524 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:29:53.527 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:29:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:29:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:29:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:29:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:29:53.530 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:29:53.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:53.530 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:29:53.586 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:29:53.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:29:53.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:29:53.682 INFO:teuthology.orchestra.run.smithi071.stdout:client6220 2024-09-16T07:29:53.683 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:53.683 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:53.718 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:29:53.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.922+0000 7f40dbe00640 1 -- 172.21.15.71:0/790737579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40c0004d40 msgr2=0x7f40c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.922+0000 7f40dbe00640 1 --2- 172.21.15.71:0/790737579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40c0004d40 0x7f40c00051c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009b50 tx=0x7f40c0005830 comp rx=0 tx=0).stop 2024-09-16T07:29:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.922+0000 7f40db400640 1 -- 172.21.15.71:0/790737579 reap_dead start 2024-09-16T07:29:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.922+0000 7f40dbe00640 1 -- 172.21.15.71:0/790737579 shutdown_connections 2024-09-16T07:29:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.922+0000 7f40dbe00640 1 -- 172.21.15.71:0/790737579 >> 172.21.15.71:0/790737579 conn(0x7f40dc0b1510 msgr2=0x7f40dc0b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.922+0000 7f40dbe00640 1 -- 172.21.15.71:0/790737579 shutdown_connections 2024-09-16T07:29:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.922+0000 7f40dbe00640 1 -- 172.21.15.71:0/790737579 wait complete. 2024-09-16T07:29:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.924+0000 7f40dbe00640 1 Processor -- start 2024-09-16T07:29:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.924+0000 7f40dbe00640 1 -- start start 2024-09-16T07:29:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40dbe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c0004d40 0x7f40dc14b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40dbe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40dc1e8b80 0x7f40dc1f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40dc1e9000 0x7f40dc0c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40dbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc06eee0 con 0x7f40dc1e9000 2024-09-16T07:29:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40dbe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc06ed60 con 0x7f40c0004d40 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40dbe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f40dc06f060 con 0x7f40dc1e8b80 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40db400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40dc1e9000 0x7f40dc0c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40db400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40dc1e9000 0x7f40dc0c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33320/0 (socket says 172.21.15.71:33320) 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40db400640 1 -- 172.21.15.71:0/754985769 learned_addr learned my addr 172.21.15.71:0/754985769 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40da000640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40dc1e8b80 0x7f40dc1f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40daa00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c0004d40 0x7f40dc14b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40db400640 1 -- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40dc1e8b80 msgr2=0x7f40dc1f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40db400640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40dc1e8b80 0x7f40dc1f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40db400640 1 -- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c0004d40 msgr2=0x7f40dc14b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40db400640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c0004d40 0x7f40dc14b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.925+0000 7f40db400640 1 -- 172.21.15.71:0/754985769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc1f5290 con 0x7f40dc1e9000 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.926+0000 7f40db400640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40dc1e9000 0x7f40dc0c6a30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f40d4036040 tx=0x7f40d4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.926+0000 7f40d2a00640 1 -- 172.21.15.71:0/754985769 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4040030 con 0x7f40dc1e9000 2024-09-16T07:29:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.926+0000 7f40d2a00640 1 -- 172.21.15.71:0/754985769 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40d4004020 con 0x7f40dc1e9000 2024-09-16T07:29:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.926+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc1f5570 con 0x7f40dc1e9000 2024-09-16T07:29:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.926+0000 7f40d2a00640 1 -- 172.21.15.71:0/754985769 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4038040 con 0x7f40dc1e9000 2024-09-16T07:29:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.926+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc15a340 con 0x7f40dc1e9000 2024-09-16T07:29:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.927+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc06f5f0 con 0x7f40dc1e9000 2024-09-16T07:29:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.928+0000 7f40d2a00640 1 -- 172.21.15.71:0/754985769 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f40d4043070 con 0x7f40dc1e9000 2024-09-16T07:29:53.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.929+0000 7f40d2a00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f40bc079420 0x7f40bc07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:53.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.929+0000 7f40d2a00640 1 -- 172.21.15.71:0/754985769 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f40d40b9460 con 0x7f40dc1e9000 2024-09-16T07:29:53.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.930+0000 7f40daa00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f40bc079420 0x7f40bc07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:53.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.931+0000 7f40daa00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f40bc079420 0x7f40bc07b8e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0062b0 tx=0x7f40cc004980 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:53.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:53.934+0000 7f40d2a00640 1 -- 172.21.15.71:0/754985769 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f40d4082c70 con 0x7f40dc1e9000 2024-09-16T07:29:54.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.153+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f40dc069d20 con 0x7f40dc1e9000 2024-09-16T07:29:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.247+0000 7f40d2a00640 1 -- 172.21.15.71:0/754985769 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f40d40bc110 con 0x7f40dc1e9000 2024-09-16T07:29:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f40bc079420 msgr2=0x7f40bc07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f40bc079420 0x7f40bc07b8e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0062b0 tx=0x7f40cc004980 comp rx=0 tx=0).stop 2024-09-16T07:29:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40dc1e9000 msgr2=0x7f40dc0c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40dc1e9000 0x7f40dc0c6a30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f40d4036040 tx=0x7f40d4033670 comp rx=0 tx=0).stop 2024-09-16T07:29:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 shutdown_connections 2024-09-16T07:29:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f40bc079420 0x7f40bc07b8e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40dc1e9000 0x7f40dc0c6a30 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40dc1e8b80 0x7f40dc1f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 --2- 172.21.15.71:0/754985769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c0004d40 0x7f40dc14b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.249+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 >> 172.21.15.71:0/754985769 conn(0x7f40dc0b1510 msgr2=0x7f40dc0b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:54.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.250+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 shutdown_connections 2024-09-16T07:29:54.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.250+0000 7f40dbe00640 1 -- 172.21.15.71:0/754985769 wait complete. 2024-09-16T07:29:54.267 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:29:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.427+0000 7f3e05800640 1 -- 172.21.15.71:0/4011630069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 msgr2=0x7f3e000b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.427+0000 7f3e05800640 1 --2- 172.21.15.71:0/4011630069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 0x7f3e000b2440 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3df0005fd0 tx=0x7f3df002aa10 comp rx=0 tx=0).stop 2024-09-16T07:29:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.427+0000 7f3e05800640 1 -- 172.21.15.71:0/4011630069 shutdown_connections 2024-09-16T07:29:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.427+0000 7f3e05800640 1 --2- 172.21.15.71:0/4011630069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e000ccf00 0x7f3e000acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.427+0000 7f3e05800640 1 --2- 172.21.15.71:0/4011630069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 0x7f3e000b2440 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.427+0000 7f3e05800640 1 -- 172.21.15.71:0/4011630069 >> 172.21.15.71:0/4011630069 conn(0x7f3e000b1950 msgr2=0x7f3e000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.427+0000 7f3e05800640 1 -- 172.21.15.71:0/4011630069 shutdown_connections 2024-09-16T07:29:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.427+0000 7f3e05800640 1 -- 172.21.15.71:0/4011630069 wait complete. 2024-09-16T07:29:54.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.429+0000 7f3e05800640 1 Processor -- start 2024-09-16T07:29:54.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.429+0000 7f3e05800640 1 -- start start 2024-09-16T07:29:54.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3e05800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e000ad270 0x7f3e000cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:54.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3e05800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e000ccf00 0x7f3e000cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:54.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3e05800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 0x7f3e00149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3e05800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0006f2e0 con 0x7f3e0015f5a0 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3e05800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0006f160 con 0x7f3e000ad270 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3e05800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0006f460 con 0x7f3e000ccf00 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 0x7f3e00149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 0x7f3e00149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33338/0 (socket says 172.21.15.71:33338) 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dff400640 1 -- 172.21.15.71:0/1126980361 learned_addr learned my addr 172.21.15.71:0/1126980361 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dfe000640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e000ccf00 0x7f3e000cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dfea00640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e000ad270 0x7f3e000cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dff400640 1 -- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e000ccf00 msgr2=0x7f3e000cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dff400640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e000ccf00 0x7f3e000cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dff400640 1 -- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e000ad270 msgr2=0x7f3e000cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dff400640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e000ad270 0x7f3e000cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.430+0000 7f3dff400640 1 -- 172.21.15.71:0/1126980361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0014fe70 con 0x7f3e0015f5a0 2024-09-16T07:29:54.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.431+0000 7f3dff400640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 0x7f3e00149280 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c3f0 tx=0x7f3df800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:54.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.431+0000 7f3df6a00640 1 -- 172.21.15.71:0/1126980361 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8017070 con 0x7f3e0015f5a0 2024-09-16T07:29:54.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.431+0000 7f3df6a00640 1 -- 172.21.15.71:0/1126980361 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3df80053c0 con 0x7f3e0015f5a0 2024-09-16T07:29:54.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.431+0000 7f3df6a00640 1 -- 172.21.15.71:0/1126980361 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8003850 con 0x7f3e0015f5a0 2024-09-16T07:29:54.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.431+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e000ce080 con 0x7f3e0015f5a0 2024-09-16T07:29:54.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.431+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e000ce640 con 0x7f3e0015f5a0 2024-09-16T07:29:54.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.432+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0006f9f0 con 0x7f3e0015f5a0 2024-09-16T07:29:54.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.433+0000 7f3df6a00640 1 -- 172.21.15.71:0/1126980361 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f3df801a070 con 0x7f3e0015f5a0 2024-09-16T07:29:54.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.434+0000 7f3df6a00640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3de0079420 0x7f3de007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:54.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.434+0000 7f3df6a00640 1 -- 172.21.15.71:0/1126980361 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f3df8098820 con 0x7f3e0015f5a0 2024-09-16T07:29:54.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.435+0000 7f3dfea00640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3de0079420 0x7f3de007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:54.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.435+0000 7f3dfea00640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3de0079420 0x7f3de007b8e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3df0035000 tx=0x7f3df003b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.438+0000 7f3df6a00640 1 -- 172.21.15.71:0/1126980361 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3df8062030 con 0x7f3e0015f5a0 2024-09-16T07:29:54.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.647+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3e0015ed50 con 0x7f3de0079420 2024-09-16T07:29:54.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.649+0000 7f3df6a00640 1 -- 172.21.15.71:0/1126980361 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3e0015ed50 con 0x7f3de0079420 2024-09-16T07:29:54.648 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:29:54.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:29:54.649 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:29:54.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:29:54.649 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:29:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3de0079420 msgr2=0x7f3de007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3de0079420 0x7f3de007b8e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3df0035000 tx=0x7f3df003b040 comp rx=0 tx=0).stop 2024-09-16T07:29:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 msgr2=0x7f3e00149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 0x7f3e00149280 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c3f0 tx=0x7f3df800c8b0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 shutdown_connections 2024-09-16T07:29:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3de0079420 0x7f3de007b8e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e0015f5a0 0x7f3e00149280 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e000ccf00 0x7f3e000cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 --2- 172.21.15.71:0/1126980361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e000ad270 0x7f3e000cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.651+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 >> 172.21.15.71:0/1126980361 conn(0x7f3e000b1950 msgr2=0x7f3e0009d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:54.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.652+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 shutdown_connections 2024-09-16T07:29:54.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.652+0000 7f3e05800640 1 -- 172.21.15.71:0/1126980361 wait complete. 2024-09-16T07:29:54.667 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:29:54.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.826+0000 7fb63f200640 1 -- 172.21.15.71:0/4049574789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb638145750 msgr2=0x7fb638073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.826+0000 7fb63f200640 1 --2- 172.21.15.71:0/4049574789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb638145750 0x7fb638073440 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb6380cfb70 tx=0x7fb62802a930 comp rx=0 tx=0).stop 2024-09-16T07:29:54.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.826+0000 7fb63f200640 1 -- 172.21.15.71:0/4049574789 shutdown_connections 2024-09-16T07:29:54.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.826+0000 7fb63f200640 1 --2- 172.21.15.71:0/4049574789 >> v2:172.21.15.177:3300/0 conn(0x7fb6380cccd0 0x7fb6380cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.826+0000 7fb63f200640 1 --2- 172.21.15.71:0/4049574789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb638145750 0x7fb638073440 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.826+0000 7fb63f200640 1 -- 172.21.15.71:0/4049574789 >> 172.21.15.71:0/4049574789 conn(0x7fb6380b1840 msgr2=0x7fb6380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:54.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.826+0000 7fb63f200640 1 -- 172.21.15.71:0/4049574789 shutdown_connections 2024-09-16T07:29:54.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.826+0000 7fb63f200640 1 -- 172.21.15.71:0/4049574789 wait complete. 2024-09-16T07:29:54.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.828+0000 7fb63f200640 1 Processor -- start 2024-09-16T07:29:54.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.828+0000 7fb63f200640 1 -- start start 2024-09-16T07:29:54.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb63f200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb63806efa0 0x7fb63815e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:54.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb63f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6380cccd0 0x7fb6381f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:54.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb63f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb638145750 0x7fb6380ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:54.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb63f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb6380ce480 con 0x7fb6380cccd0 2024-09-16T07:29:54.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb63f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb6380ce300 con 0x7fb638145750 2024-09-16T07:29:54.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb63f200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb6380ce600 con 0x7fb63806efa0 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb637400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6380cccd0 0x7fb6381f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb637400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6380cccd0 0x7fb6381f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41470/0 (socket says 172.21.15.71:41470) 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb637400640 1 -- 172.21.15.71:0/1945601810 learned_addr learned my addr 172.21.15.71:0/1945601810 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb63d200640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb638145750 0x7fb6380ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.829+0000 7fb637e00640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb63806efa0 0x7fb63815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb637400640 1 -- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb63806efa0 msgr2=0x7fb63815e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb637400640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb63806efa0 0x7fb63815e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb637400640 1 -- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb638145750 msgr2=0x7fb6380ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb637400640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb638145750 0x7fb6380ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb637400640 1 -- 172.21.15.71:0/1945601810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6381f2870 con 0x7fb6380cccd0 2024-09-16T07:29:54.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb637400640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6380cccd0 0x7fb6381f2330 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c760 tx=0x7fb62400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:54.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb62fe00640 1 -- 172.21.15.71:0/1945601810 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624017070 con 0x7fb6380cccd0 2024-09-16T07:29:54.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb62fe00640 1 -- 172.21.15.71:0/1945601810 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb624005380 con 0x7fb6380cccd0 2024-09-16T07:29:54.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb62fe00640 1 -- 172.21.15.71:0/1945601810 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62400a8a0 con 0x7fb6380cccd0 2024-09-16T07:29:54.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63813fce0 con 0x7fb6380cccd0 2024-09-16T07:29:54.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.830+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6381f2df0 con 0x7fb6380cccd0 2024-09-16T07:29:54.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.832+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63813feb0 con 0x7fb6380cccd0 2024-09-16T07:29:54.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.833+0000 7fb62fe00640 1 -- 172.21.15.71:0/1945601810 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fb62400aa40 con 0x7fb6380cccd0 2024-09-16T07:29:54.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.834+0000 7fb62fe00640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb618079450 0x7fb61807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:54.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.834+0000 7fb62fe00640 1 -- 172.21.15.71:0/1945601810 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fb6240987b0 con 0x7fb6380cccd0 2024-09-16T07:29:54.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.834+0000 7fb637e00640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb618079450 0x7fb61807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:54.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.835+0000 7fb637e00640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb618079450 0x7fb61807b910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb628035000 tx=0x7fb628004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:54.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:54.838+0000 7fb62fe00640 1 -- 172.21.15.71:0/1945601810 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb624061fc0 con 0x7fb6380cccd0 2024-09-16T07:29:55.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.025+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb6380cdbb0 con 0x7fb6380cccd0 2024-09-16T07:29:55.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.026+0000 7fb62fe00640 1 -- 172.21.15.71:0/1945601810 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7fb6240669f0 con 0x7fb6380cccd0 2024-09-16T07:29:55.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb618079450 msgr2=0x7fb61807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb618079450 0x7fb61807b910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb628035000 tx=0x7fb628004ab0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6380cccd0 msgr2=0x7fb6381f2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6380cccd0 0x7fb6381f2330 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c760 tx=0x7fb62400cc20 comp rx=0 tx=0).stop 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 shutdown_connections 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb618079450 0x7fb61807b910 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb638145750 0x7fb6380ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6380cccd0 0x7fb6381f2330 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 --2- 172.21.15.71:0/1945601810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb63806efa0 0x7fb63815e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 >> 172.21.15.71:0/1945601810 conn(0x7fb6380b1840 msgr2=0x7fb6380cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 shutdown_connections 2024-09-16T07:29:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.028+0000 7fb63f200640 1 -- 172.21.15.71:0/1945601810 wait complete. 2024-09-16T07:29:55.043 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:29:55.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.203+0000 7f0dbcc00640 1 -- 172.21.15.71:0/155035896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db806efa0 msgr2=0x7f0db80cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.203+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/155035896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db806efa0 0x7f0db80cc5c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0db80d0cd0 tx=0x7f0da402a560 comp rx=0 tx=0).stop 2024-09-16T07:29:55.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.203+0000 7f0dbcc00640 1 -- 172.21.15.71:0/155035896 shutdown_connections 2024-09-16T07:29:55.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.203+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/155035896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db806efa0 0x7f0db80cc5c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.203+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/155035896 >> v2:172.21.15.177:3300/0 conn(0x7f0db8145750 0x7f0db8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.203+0000 7f0dbcc00640 1 -- 172.21.15.71:0/155035896 >> 172.21.15.71:0/155035896 conn(0x7f0db80b1840 msgr2=0x7f0db80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:55.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.203+0000 7f0dbcc00640 1 -- 172.21.15.71:0/155035896 shutdown_connections 2024-09-16T07:29:55.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.203+0000 7f0dbcc00640 1 -- 172.21.15.71:0/155035896 wait complete. 2024-09-16T07:29:55.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.205+0000 7f0dbcc00640 1 Processor -- start 2024-09-16T07:29:55.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.205+0000 7f0dbcc00640 1 -- start start 2024-09-16T07:29:55.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0dbcc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db806efa0 0x7f0db813e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0dbcc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db80ccb00 0x7f0db80ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0dbcc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db8145750 0x7f0db80ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0dbcc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80ced40 con 0x7f0db8145750 2024-09-16T07:29:55.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0dbcc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80cebc0 con 0x7f0db806efa0 2024-09-16T07:29:55.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0dbcc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0db80ceec0 con 0x7f0db80ccb00 2024-09-16T07:29:55.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db8145750 0x7f0db80ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db8145750 0x7f0db80ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41488/0 (socket says 172.21.15.71:41488) 2024-09-16T07:29:55.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db6800640 1 -- 172.21.15.71:0/748242926 learned_addr learned my addr 172.21.15.71:0/748242926 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:55.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db5400640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db80ccb00 0x7f0db80ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db5e00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db806efa0 0x7f0db813e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db6800640 1 -- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db80ccb00 msgr2=0x7f0db80ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db6800640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db80ccb00 0x7f0db80ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db6800640 1 -- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db806efa0 msgr2=0x7f0db813e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db6800640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db806efa0 0x7f0db813e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.206+0000 7f0db6800640 1 -- 172.21.15.71:0/748242926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db81f8110 con 0x7f0db8145750 2024-09-16T07:29:55.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.207+0000 7f0db6800640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db8145750 0x7f0db80ae750 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c7a0 tx=0x7f0db000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:55.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.207+0000 7f0dae000640 1 -- 172.21.15.71:0/748242926 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0003b70 con 0x7f0db8145750 2024-09-16T07:29:55.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.207+0000 7f0dae000640 1 -- 172.21.15.71:0/748242926 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0db0003d10 con 0x7f0db8145750 2024-09-16T07:29:55.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.207+0000 7f0dae000640 1 -- 172.21.15.71:0/748242926 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db000aab0 con 0x7f0db8145750 2024-09-16T07:29:55.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.207+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db81f83d0 con 0x7f0db8145750 2024-09-16T07:29:55.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.207+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81f8990 con 0x7f0db8145750 2024-09-16T07:29:55.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.208+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db80cc5c0 con 0x7f0db8145750 2024-09-16T07:29:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.209+0000 7f0dae000640 1 -- 172.21.15.71:0/748242926 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f0db000ac50 con 0x7f0db8145750 2024-09-16T07:29:55.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.210+0000 7f0dae000640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d98079420 0x7f0d9807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.210+0000 7f0dae000640 1 -- 172.21.15.71:0/748242926 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f0db0099280 con 0x7f0db8145750 2024-09-16T07:29:55.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.211+0000 7f0db5e00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d98079420 0x7f0d9807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.212+0000 7f0db5e00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d98079420 0x7f0d9807b8e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0da8004630 tx=0x7f0da800b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:55.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.215+0000 7f0dae000640 1 -- 172.21.15.71:0/748242926 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0db0062a90 con 0x7f0db8145750 2024-09-16T07:29:55.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.400+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f0db806fd90 con 0x7f0db8145750 2024-09-16T07:29:55.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.400+0000 7f0dae000640 1 -- 172.21.15.71:0/748242926 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v37) ==== 116+0+2 (secure 0 0 0) 0x7f0db00674c0 con 0x7f0db8145750 2024-09-16T07:29:55.401 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:29:55.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.402+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d98079420 msgr2=0x7f0d9807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.402+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d98079420 0x7f0d9807b8e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0da8004630 tx=0x7f0da800b2e0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.402+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db8145750 msgr2=0x7f0db80ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.402+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db8145750 0x7f0db80ae750 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c7a0 tx=0x7f0db000cc60 comp rx=0 tx=0).stop 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.402+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 shutdown_connections 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.402+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0d98079420 0x7f0d9807b8e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.403+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0db8145750 0x7f0db80ae750 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.403+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db80ccb00 0x7f0db80ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.403+0000 7f0dbcc00640 1 --2- 172.21.15.71:0/748242926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db806efa0 0x7f0db813e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.403+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 >> 172.21.15.71:0/748242926 conn(0x7f0db80b1840 msgr2=0x7f0db80b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.403+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 shutdown_connections 2024-09-16T07:29:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.403+0000 7f0dbcc00640 1 -- 172.21.15.71:0/748242926 wait complete. 2024-09-16T07:29:55.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:29:55.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.578+0000 7ffae2a00640 1 -- 172.21.15.71:0/2900337035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffac0003c60 msgr2=0x7ffac0004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.578+0000 7ffae2a00640 1 --2- 172.21.15.71:0/2900337035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffac0003c60 0x7ffac0004060 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffacc001f30 tx=0x7ffac0005830 comp rx=0 tx=0).stop 2024-09-16T07:29:55.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.578+0000 7ffae0a00640 1 -- 172.21.15.71:0/2900337035 reap_dead start 2024-09-16T07:29:55.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.578+0000 7ffae2a00640 1 -- 172.21.15.71:0/2900337035 shutdown_connections 2024-09-16T07:29:55.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.578+0000 7ffae2a00640 1 -- 172.21.15.71:0/2900337035 >> 172.21.15.71:0/2900337035 conn(0x7ffadc0b1840 msgr2=0x7ffadc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:55.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.578+0000 7ffae2a00640 1 -- 172.21.15.71:0/2900337035 shutdown_connections 2024-09-16T07:29:55.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.578+0000 7ffae2a00640 1 -- 172.21.15.71:0/2900337035 wait complete. 2024-09-16T07:29:55.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.580+0000 7ffae2a00640 1 Processor -- start 2024-09-16T07:29:55.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.580+0000 7ffae2a00640 1 -- start start 2024-09-16T07:29:55.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffae2a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffac0003c60 0x7ffadc15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffae2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc1419e0 0x7ffadc141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffae2a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc13ff60 0x7ffadc1403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffae2a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc0ce2c0 con 0x7ffadc1419e0 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffae2a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc0736a0 con 0x7ffadc13ff60 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffae2a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffadc0ce440 con 0x7ffac0003c60 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffadb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc1419e0 0x7ffadc141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffadb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc1419e0 0x7ffadc141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41504/0 (socket says 172.21.15.71:41504) 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffadb400640 1 -- 172.21.15.71:0/1156519583 learned_addr learned my addr 172.21.15.71:0/1156519583 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffadbe00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffac0003c60 0x7ffadc15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.581+0000 7ffae0a00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc13ff60 0x7ffadc1403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffadb400640 1 -- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffac0003c60 msgr2=0x7ffadc15e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffadb400640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffac0003c60 0x7ffadc15e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffadb400640 1 -- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc13ff60 msgr2=0x7ffadc1403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffadb400640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc13ff60 0x7ffadc1403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffadb400640 1 -- 172.21.15.71:0/1156519583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc1e7a60 con 0x7ffadc1419e0 2024-09-16T07:29:55.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffadb400640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc1419e0 0x7ffadc141e40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffac8011ab0 tx=0x7ffac8011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:55.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffad3e00640 1 -- 172.21.15.71:0/1156519583 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac800fd50 con 0x7ffadc1419e0 2024-09-16T07:29:55.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffad3e00640 1 -- 172.21.15.71:0/1156519583 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffac8016070 con 0x7ffadc1419e0 2024-09-16T07:29:55.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc1e7d00 con 0x7ffadc1419e0 2024-09-16T07:29:55.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffad3e00640 1 -- 172.21.15.71:0/1156519583 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8012690 con 0x7ffadc1419e0 2024-09-16T07:29:55.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.582+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc1e82c0 con 0x7ffadc1419e0 2024-09-16T07:29:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.583+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc0ce9d0 con 0x7ffadc1419e0 2024-09-16T07:29:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.584+0000 7ffad3e00640 1 -- 172.21.15.71:0/1156519583 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7ffac8018030 con 0x7ffadc1419e0 2024-09-16T07:29:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.585+0000 7ffad3e00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc079420 0x7ffabc07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.586+0000 7ffad3e00640 1 -- 172.21.15.71:0/1156519583 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7ffac809c6b0 con 0x7ffadc1419e0 2024-09-16T07:29:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.586+0000 7ffadbe00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc079420 0x7ffabc07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.587+0000 7ffadbe00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc079420 0x7ffabc07b8e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffacc033000 tx=0x7ffacc0016e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:55.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.589+0000 7ffad3e00640 1 -- 172.21.15.71:0/1156519583 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffac8063580 con 0x7ffadc1419e0 2024-09-16T07:29:55.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.774+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7ffadc1e75c0 con 0x7ffadc1419e0 2024-09-16T07:29:55.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.778+0000 7ffad3e00640 1 -- 172.21.15.71:0/1156519583 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v38)=0 v38) ==== 131+0+0 (secure 0 0 0) 0x7ffac8067fb0 con 0x7ffadc1419e0 2024-09-16T07:29:55.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.780+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc079420 msgr2=0x7ffabc07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.780+0000 7ffae2a00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc079420 0x7ffabc07b8e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffacc033000 tx=0x7ffacc0016e0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.780+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc1419e0 msgr2=0x7ffadc141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.780+0000 7ffae2a00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc1419e0 0x7ffadc141e40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffac8011ab0 tx=0x7ffac8011f70 comp rx=0 tx=0).stop 2024-09-16T07:29:55.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.781+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 shutdown_connections 2024-09-16T07:29:55.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.781+0000 7ffae2a00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc079420 0x7ffabc07b8e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.781+0000 7ffae2a00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc13ff60 0x7ffadc1403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.781+0000 7ffae2a00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc1419e0 0x7ffadc141e40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.781+0000 7ffae2a00640 1 --2- 172.21.15.71:0/1156519583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffac0003c60 0x7ffadc15e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.781+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 >> 172.21.15.71:0/1156519583 conn(0x7ffadc0b1840 msgr2=0x7ffadc1ea4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:55.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.781+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 shutdown_connections 2024-09-16T07:29:55.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.781+0000 7ffae2a00640 1 -- 172.21.15.71:0/1156519583 wait complete. 2024-09-16T07:29:55.795 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:55.796+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:55.795 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:55.796+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:55.795 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:55.796+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:55.795 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:55.796+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:55.796 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:55.796+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:55.797 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-16T07:29:55.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.957+0000 7fa867a00640 1 -- 172.21.15.71:0/3899026840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600d0250 msgr2=0x7fa8600b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.957+0000 7fa867a00640 1 --2- 172.21.15.71:0/3899026840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600d0250 0x7fa8600b1740 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa8600d1850 tx=0x7fa85802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.957+0000 7fa867a00640 1 -- 172.21.15.71:0/3899026840 shutdown_connections 2024-09-16T07:29:55.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.957+0000 7fa867a00640 1 --2- 172.21.15.71:0/3899026840 >> v2:172.21.15.177:3300/0 conn(0x7fa8600b0810 0x7fa860160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.957+0000 7fa867a00640 1 --2- 172.21.15.71:0/3899026840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600d0250 0x7fa8600b1740 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.958+0000 7fa867a00640 1 -- 172.21.15.71:0/3899026840 >> 172.21.15.71:0/3899026840 conn(0x7fa8600d1e20 msgr2=0x7fa8600d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:55.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.958+0000 7fa867a00640 1 -- 172.21.15.71:0/3899026840 shutdown_connections 2024-09-16T07:29:55.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.958+0000 7fa867a00640 1 -- 172.21.15.71:0/3899026840 wait complete. 2024-09-16T07:29:55.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.959+0000 7fa867a00640 1 Processor -- start 2024-09-16T07:29:55.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.959+0000 7fa867a00640 1 -- start start 2024-09-16T07:29:55.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.960+0000 7fa867a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8600b0810 0x7fa8600cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.960+0000 7fa867a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600cfe70 0x7fa8601f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.960+0000 7fa867a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8600d0250 0x7fa8600ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.960+0000 7fa867a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8600b1c80 con 0x7fa8600cfe70 2024-09-16T07:29:55.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.960+0000 7fa867a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8600b2ea0 con 0x7fa8600d0250 2024-09-16T07:29:55.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.960+0000 7fa867a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8600b1e00 con 0x7fa8600b0810 2024-09-16T07:29:55.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.960+0000 7fa85fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600cfe70 0x7fa8601f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600cfe70 0x7fa8601f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41530/0 (socket says 172.21.15.71:41530) 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85fe00640 1 -- 172.21.15.71:0/2949834498 learned_addr learned my addr 172.21.15.71:0/2949834498 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa865000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8600b0810 0x7fa8600cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa865a00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8600d0250 0x7fa8600ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85fe00640 1 -- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8600b0810 msgr2=0x7fa8600cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85fe00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8600b0810 0x7fa8600cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85fe00640 1 -- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8600d0250 msgr2=0x7fa8600ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85fe00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8600d0250 0x7fa8600ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85fe00640 1 -- 172.21.15.71:0/2949834498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8601f4c00 con 0x7fa8600cfe70 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85fe00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600cfe70 0x7fa8601f7730 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00b4a0 tx=0x7fa84c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.961+0000 7fa85cc00640 1 -- 172.21.15.71:0/2949834498 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c014070 con 0x7fa8600cfe70 2024-09-16T07:29:55.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.962+0000 7fa85cc00640 1 -- 172.21.15.71:0/2949834498 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa84c010700 con 0x7fa8600cfe70 2024-09-16T07:29:55.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.962+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8601f4ee0 con 0x7fa8600cfe70 2024-09-16T07:29:55.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.962+0000 7fa85cc00640 1 -- 172.21.15.71:0/2949834498 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c010b50 con 0x7fa8600cfe70 2024-09-16T07:29:55.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.962+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8601f54a0 con 0x7fa8600cfe70 2024-09-16T07:29:55.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.963+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8600b1410 con 0x7fa8600cfe70 2024-09-16T07:29:55.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.964+0000 7fa85cc00640 1 -- 172.21.15.71:0/2949834498 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7fa84c017070 con 0x7fa8600cfe70 2024-09-16T07:29:55.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.965+0000 7fa85cc00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa840079420 0x7fa84007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:55.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.965+0000 7fa85cc00640 1 -- 172.21.15.71:0/2949834498 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fa84c098f70 con 0x7fa8600cfe70 2024-09-16T07:29:55.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.965+0000 7fa865000640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa840079420 0x7fa84007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:55.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.966+0000 7fa865000640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa840079420 0x7fa84007b8e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa8601f8ef0 tx=0x7fa850003d90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:55.969+0000 7fa85cc00640 1 -- 172.21.15.71:0/2949834498 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8600b1410 con 0x7fa8600cfe70 2024-09-16T07:29:56.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.154+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7fa8600b1740 con 0x7fa8600cfe70 2024-09-16T07:29:56.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.158+0000 7fa85cc00640 1 -- 172.21.15.71:0/2949834498 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v39)=0 v39) ==== 136+0+0 (secure 0 0 0) 0x7fa84c062780 con 0x7fa8600cfe70 2024-09-16T07:29:56.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa840079420 msgr2=0x7fa84007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa840079420 0x7fa84007b8e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa8601f8ef0 tx=0x7fa850003d90 comp rx=0 tx=0).stop 2024-09-16T07:29:56.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600cfe70 msgr2=0x7fa8601f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600cfe70 0x7fa8601f7730 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00b4a0 tx=0x7fa84c00b960 comp rx=0 tx=0).stop 2024-09-16T07:29:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 shutdown_connections 2024-09-16T07:29:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa840079420 0x7fa84007b8e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8600d0250 0x7fa8600ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8600cfe70 0x7fa8601f7730 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 --2- 172.21.15.71:0/2949834498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8600b0810 0x7fa8600cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.160+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 >> 172.21.15.71:0/2949834498 conn(0x7fa8600d1e20 msgr2=0x7fa86006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.161+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 shutdown_connections 2024-09-16T07:29:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.161+0000 7fa867a00640 1 -- 172.21.15.71:0/2949834498 wait complete. 2024-09-16T07:29:56.175 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.337+0000 7f8a88e00640 1 -- 172.21.15.71:0/1581625665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a8415b700 msgr2=0x7f8a840ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.337+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1581625665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a8415b700 0x7f8a840ccca0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8a8406aa00 tx=0x7f8a7002a530 comp rx=0 tx=0).stop 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.337+0000 7f8a88e00640 1 -- 172.21.15.71:0/1581625665 shutdown_connections 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.337+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1581625665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a8415b700 0x7f8a840ccca0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.337+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1581625665 >> v2:172.21.15.177:3300/0 conn(0x7f8a84158900 0x7f8a8415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.337+0000 7f8a88e00640 1 -- 172.21.15.71:0/1581625665 >> 172.21.15.71:0/1581625665 conn(0x7f8a840b18e0 msgr2=0x7f8a840b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.337+0000 7f8a88e00640 1 -- 172.21.15.71:0/1581625665 shutdown_connections 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.337+0000 7f8a88e00640 1 -- 172.21.15.71:0/1581625665 wait complete. 2024-09-16T07:29:56.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.339+0000 7f8a88e00640 1 Processor -- start 2024-09-16T07:29:56.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.339+0000 7f8a88e00640 1 -- start start 2024-09-16T07:29:56.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.340+0000 7f8a88e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a840cd1e0 0x7f8a8409ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.340+0000 7f8a88e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a84158900 0x7f8a841576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.340+0000 7f8a88e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8415b700 0x7f8a8409d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.340+0000 7f8a88e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a840ced60 con 0x7f8a84158900 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.340+0000 7f8a88e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8a840cebe0 con 0x7f8a840cd1e0 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.340+0000 7f8a88e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8a840ceee0 con 0x7f8a8415b700 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a84158900 0x7f8a841576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a84158900 0x7f8a841576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41554/0 (socket says 172.21.15.71:41554) 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 -- 172.21.15.71:0/1943518740 learned_addr learned my addr 172.21.15.71:0/1943518740 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a840cd1e0 0x7f8a8409ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a82800640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8415b700 0x7f8a8409d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 -- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8415b700 msgr2=0x7f8a8409d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8415b700 0x7f8a8409d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 -- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a840cd1e0 msgr2=0x7f8a8409ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a840cd1e0 0x7f8a8409ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 -- 172.21.15.71:0/1943518740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8409bf90 con 0x7f8a84158900 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.341+0000 7f8a81400640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a84158900 0x7f8a841576b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009920 tx=0x7f8a70033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.342+0000 7f8a7a000640 1 -- 172.21.15.71:0/1943518740 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70031d60 con 0x7f8a84158900 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.342+0000 7f8a7a000640 1 -- 172.21.15.71:0/1943518740 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a70033e10 con 0x7f8a84158900 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.342+0000 7f8a7a000640 1 -- 172.21.15.71:0/1943518740 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70005260 con 0x7f8a84158900 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.342+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a841f8be0 con 0x7f8a84158900 2024-09-16T07:29:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.342+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8415a370 con 0x7f8a84158900 2024-09-16T07:29:56.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.343+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a84069360 con 0x7f8a84158900 2024-09-16T07:29:56.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.344+0000 7f8a7a000640 1 -- 172.21.15.71:0/1943518740 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f8a70005400 con 0x7f8a84158900 2024-09-16T07:29:56.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.345+0000 7f8a7a000640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8a64079420 0x7f8a6407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:56.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.345+0000 7f8a7a000640 1 -- 172.21.15.71:0/1943518740 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f8a700ba580 con 0x7f8a84158900 2024-09-16T07:29:56.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.345+0000 7f8a81e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8a64079420 0x7f8a6407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:56.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.346+0000 7f8a81e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8a64079420 0x7f8a6407b8e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400bbe0 tx=0x7f8a7400b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:56.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.349+0000 7f8a7a000640 1 -- 172.21.15.71:0/1943518740 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8a7003f030 con 0x7f8a84158900 2024-09-16T07:29:56.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.558+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f8a840cf470 con 0x7f8a64079420 2024-09-16T07:29:56.568 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:56.568+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:56.568 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:56.568+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:56.568 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:56.568+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:56.568 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:56.568+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:56.568 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:29:56.568+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:29:56.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.613+0000 7f8a7a000640 1 -- 172.21.15.71:0/1943518740 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a840cf470 con 0x7f8a64079420 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8a64079420 msgr2=0x7f8a6407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8a64079420 0x7f8a6407b8e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400bbe0 tx=0x7f8a7400b390 comp rx=0 tx=0).stop 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a84158900 msgr2=0x7f8a841576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a84158900 0x7f8a841576b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009920 tx=0x7f8a70033c20 comp rx=0 tx=0).stop 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 shutdown_connections 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8a64079420 0x7f8a6407b8e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8415b700 0x7f8a8409d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a84158900 0x7f8a841576b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 --2- 172.21.15.71:0/1943518740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a840cd1e0 0x7f8a8409ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 >> 172.21.15.71:0/1943518740 conn(0x7f8a840b18e0 msgr2=0x7f8a84158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:56.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 shutdown_connections 2024-09-16T07:29:56.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.616+0000 7f8a88e00640 1 -- 172.21.15.71:0/1943518740 wait complete. 2024-09-16T07:29:56.633 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:29:56.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.796+0000 7f4333600640 1 -- 172.21.15.71:0/1855982163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4314005910 msgr2=0x7f4314005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.796+0000 7f4333600640 1 --2- 172.21.15.71:0/1855982163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4314005910 0x7f4314005d90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4318001830 tx=0x7f4314006940 comp rx=0 tx=0).stop 2024-09-16T07:29:56.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.796+0000 7f4331600640 1 -- 172.21.15.71:0/1855982163 reap_dead start 2024-09-16T07:29:56.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.796+0000 7f4333600640 1 -- 172.21.15.71:0/1855982163 shutdown_connections 2024-09-16T07:29:56.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.796+0000 7f4333600640 1 -- 172.21.15.71:0/1855982163 >> 172.21.15.71:0/1855982163 conn(0x7f432c0b1840 msgr2=0x7f432c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:56.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.796+0000 7f4333600640 1 -- 172.21.15.71:0/1855982163 shutdown_connections 2024-09-16T07:29:56.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.796+0000 7f4333600640 1 -- 172.21.15.71:0/1855982163 wait complete. 2024-09-16T07:29:56.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.798+0000 7f4333600640 1 Processor -- start 2024-09-16T07:29:56.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.798+0000 7f4333600640 1 -- start start 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4333600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4314005910 0x7f432c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4333600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f432c06ef60 0x7f432c1f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4333600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f432c1f9170 0x7f432c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4333600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f432c0cc860 con 0x7f432c06ef60 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4333600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f432c074500 con 0x7f432c1f9170 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4333600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f432c0cc9e0 con 0x7f4314005910 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f432be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f432c06ef60 0x7f432c1f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f432be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f432c06ef60 0x7f432c1f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41560/0 (socket says 172.21.15.71:41560) 2024-09-16T07:29:56.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f432be00640 1 -- 172.21.15.71:0/2088669273 learned_addr learned my addr 172.21.15.71:0/2088669273 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4330c00640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4314005910 0x7f432c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4331600640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f432c1f9170 0x7f432c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f432be00640 1 -- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4314005910 msgr2=0x7f432c15a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f432be00640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4314005910 0x7f432c15a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f432be00640 1 -- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f432c1f9170 msgr2=0x7f432c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f432be00640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f432c1f9170 0x7f432c0cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f432be00640 1 -- 172.21.15.71:0/2088669273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c06f520 con 0x7f432c06ef60 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.799+0000 7f4331600640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f432c1f9170 0x7f432c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.800+0000 7f432be00640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f432c06ef60 0x7f432c1f4b30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f431802aa50 tx=0x7f43180069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.800+0000 7f4328c00640 1 -- 172.21.15.71:0/2088669273 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318043070 con 0x7f432c06ef60 2024-09-16T07:29:56.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.800+0000 7f4328c00640 1 -- 172.21.15.71:0/2088669273 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43180038e0 con 0x7f432c06ef60 2024-09-16T07:29:56.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.800+0000 7f4328c00640 1 -- 172.21.15.71:0/2088669273 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318003d10 con 0x7f432c06ef60 2024-09-16T07:29:56.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.800+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c06f800 con 0x7f432c06ef60 2024-09-16T07:29:56.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.800+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c06fdc0 con 0x7f432c06ef60 2024-09-16T07:29:56.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.801+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c0b3a40 con 0x7f432c06ef60 2024-09-16T07:29:56.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.802+0000 7f4328c00640 1 -- 172.21.15.71:0/2088669273 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 102358+0+0 (secure 0 0 0) 0x7f4318033480 con 0x7f432c06ef60 2024-09-16T07:29:56.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.803+0000 7f4328c00640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f430c079450 0x7f430c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:29:56.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.803+0000 7f4328c00640 1 -- 172.21.15.71:0/2088669273 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f43180b9340 con 0x7f432c06ef60 2024-09-16T07:29:56.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.804+0000 7f4330c00640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f430c079450 0x7f430c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:29:56.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.804+0000 7f4330c00640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f430c079450 0x7f430c07b910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4320001860 tx=0x7f4320001110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:29:56.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:56.808+0000 7f4328c00640 1 -- 172.21.15.71:0/2088669273 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f432c0b3a40 con 0x7f432c06ef60 2024-09-16T07:29:57.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.016+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f432c1f5070 con 0x7f430c079450 2024-09-16T07:29:57.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.022+0000 7f4328c00640 1 -- 172.21.15.71:0/2088669273 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f432c1f5070 con 0x7f430c079450 2024-09-16T07:29:57.021 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95 2024-09-16T07:29:57.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f430c079450 msgr2=0x7f430c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:57.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f430c079450 0x7f430c07b910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4320001860 tx=0x7f4320001110 comp rx=0 tx=0).stop 2024-09-16T07:29:57.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f432c06ef60 msgr2=0x7f432c1f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:29:57.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f432c06ef60 0x7f432c1f4b30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f431802aa50 tx=0x7f43180069d0 comp rx=0 tx=0).stop 2024-09-16T07:29:57.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 shutdown_connections 2024-09-16T07:29:57.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f430c079450 0x7f430c07b910 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:57.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f432c1f9170 0x7f432c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:57.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f432c06ef60 0x7f432c1f4b30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:57.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 --2- 172.21.15.71:0/2088669273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4314005910 0x7f432c15a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:29:57.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 >> 172.21.15.71:0/2088669273 conn(0x7f432c0b1840 msgr2=0x7f432c157400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:29:57.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 shutdown_connections 2024-09-16T07:29:57.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:29:57.024+0000 7f4333600640 1 -- 172.21.15.71:0/2088669273 wait complete. 2024-09-16T07:29:57.040 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95 2024-09-16T07:29:57.040 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:29:57.040 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:29:57.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:29:57.076 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:30:04.579 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:30:04.579 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:30:04.579 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.4191 s, 145 MB/s 2024-09-16T07:30:04.580 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:30:04.580 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:30:04.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:30:04.616 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:30:12.044 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:30:12.045 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:30:12.045 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.35239 s, 146 MB/s 2024-09-16T07:30:12.046 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:30:12.046 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:30:12.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:30:12.082 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:30:15.006 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:30:15.011 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:30:19.661 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:30:19.662 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:30:19.662 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.50212 s, 143 MB/s 2024-09-16T07:30:19.663 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T07:30:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.825+0000 7fe49cc00640 1 -- 172.21.15.71:0/931540189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806f220 msgr2=0x7fe49806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.825+0000 7fe49cc00640 1 --2- 172.21.15.71:0/931540189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806f220 0x7fe49806f6d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe490009860 tx=0x7fe49002a540 comp rx=0 tx=0).stop 2024-09-16T07:30:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.826+0000 7fe49cc00640 1 -- 172.21.15.71:0/931540189 shutdown_connections 2024-09-16T07:30:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.826+0000 7fe49cc00640 1 --2- 172.21.15.71:0/931540189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806f220 0x7fe49806f6d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.826+0000 7fe49cc00640 1 --2- 172.21.15.71:0/931540189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe49815cff0 0x7fe49806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.826+0000 7fe49cc00640 1 -- 172.21.15.71:0/931540189 >> 172.21.15.71:0/931540189 conn(0x7fe4980b18e0 msgr2=0x7fe4980b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.826+0000 7fe49cc00640 1 -- 172.21.15.71:0/931540189 shutdown_connections 2024-09-16T07:30:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.826+0000 7fe49cc00640 1 -- 172.21.15.71:0/931540189 wait complete. 2024-09-16T07:30:19.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.827+0000 7fe49cc00640 1 Processor -- start 2024-09-16T07:30:19.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.827+0000 7fe49cc00640 1 -- start start 2024-09-16T07:30:19.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.828+0000 7fe49cc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806f220 0x7fe49809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:19.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.828+0000 7fe49cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498159aa0 0x7fe4981f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:19.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.828+0000 7fe49cc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe49815cff0 0x7fe4980adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:19.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.828+0000 7fe49cc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980cc520 con 0x7fe498159aa0 2024-09-16T07:30:19.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.828+0000 7fe49cc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980cc3a0 con 0x7fe49806f220 2024-09-16T07:30:19.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.828+0000 7fe49cc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe4980cc6a0 con 0x7fe49815cff0 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.828+0000 7fe495400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498159aa0 0x7fe4981f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498159aa0 0x7fe4981f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60536/0 (socket says 172.21.15.71:60536) 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495400640 1 -- 172.21.15.71:0/2075340036 learned_addr learned my addr 172.21.15.71:0/2075340036 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe496800640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe49815cff0 0x7fe4980adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495e00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806f220 0x7fe49809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495400640 1 -- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe49815cff0 msgr2=0x7fe4980adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495400640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe49815cff0 0x7fe4980adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495400640 1 -- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806f220 msgr2=0x7fe49809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495400640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806f220 0x7fe49809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495400640 1 -- 172.21.15.71:0/2075340036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49809b4b0 con 0x7fe498159aa0 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.829+0000 7fe495400640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498159aa0 0x7fe4981f2fb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe48400e910 tx=0x7fe48400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.830+0000 7fe48e000640 1 -- 172.21.15.71:0/2075340036 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48400cd80 con 0x7fe498159aa0 2024-09-16T07:30:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.830+0000 7fe48e000640 1 -- 172.21.15.71:0/2075340036 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe484010430 con 0x7fe498159aa0 2024-09-16T07:30:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.830+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49809b750 con 0x7fe498159aa0 2024-09-16T07:30:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.830+0000 7fe48e000640 1 -- 172.21.15.71:0/2075340036 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484010860 con 0x7fe498159aa0 2024-09-16T07:30:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.830+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49809af40 con 0x7fe498159aa0 2024-09-16T07:30:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.831+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4980ccbc0 con 0x7fe498159aa0 2024-09-16T07:30:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.832+0000 7fe48e000640 1 -- 172.21.15.71:0/2075340036 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 102460+0+0 (secure 0 0 0) 0x7fe48401f020 con 0x7fe498159aa0 2024-09-16T07:30:19.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.833+0000 7fe48e000640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe478079540 0x7fe47807ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:19.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.833+0000 7fe48e000640 1 -- 172.21.15.71:0/2075340036 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fe4840999f0 con 0x7fe498159aa0 2024-09-16T07:30:19.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.833+0000 7fe495e00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe478079540 0x7fe47807ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:19.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.834+0000 7fe495e00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe478079540 0x7fe47807ba00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe488001840 tx=0x7fe4880017d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:19.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:19.837+0000 7fe48e000640 1 -- 172.21.15.71:0/2075340036 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe484063440 con 0x7fe498159aa0 2024-09-16T07:30:20.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.051+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fe498070980 con 0x7fe478079540 2024-09-16T07:30:20.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.066+0000 7fe48e000640 1 -- 172.21.15.71:0/2075340036 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe498070980 con 0x7fe478079540 2024-09-16T07:30:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe478079540 msgr2=0x7fe47807ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe478079540 0x7fe47807ba00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe488001840 tx=0x7fe4880017d0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498159aa0 msgr2=0x7fe4981f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498159aa0 0x7fe4981f2fb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe48400e910 tx=0x7fe48400edd0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 shutdown_connections 2024-09-16T07:30:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe478079540 0x7fe47807ba00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe49815cff0 0x7fe4980adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498159aa0 0x7fe4981f2fb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 --2- 172.21.15.71:0/2075340036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806f220 0x7fe49809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 >> 172.21.15.71:0/2075340036 conn(0x7fe4980b18e0 msgr2=0x7fe49815d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:20.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 shutdown_connections 2024-09-16T07:30:20.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.068+0000 7fe49cc00640 1 -- 172.21.15.71:0/2075340036 wait complete. 2024-09-16T07:30:20.083 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:30:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.244+0000 7f738f800640 1 -- 172.21.15.71:0/3651526586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73881517a0 msgr2=0x7f73880cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.244+0000 7f738f800640 1 --2- 172.21.15.71:0/3651526586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73881517a0 0x7f73880cdad0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7388072fb0 tx=0x7f737802a5a0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.244+0000 7f738f800640 1 -- 172.21.15.71:0/3651526586 shutdown_connections 2024-09-16T07:30:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.244+0000 7f738f800640 1 --2- 172.21.15.71:0/3651526586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73881517a0 0x7f73880cdad0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.244+0000 7f738f800640 1 -- 172.21.15.71:0/3651526586 >> 172.21.15.71:0/3651526586 conn(0x7f73880b1840 msgr2=0x7f73880b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.244+0000 7f738f800640 1 -- 172.21.15.71:0/3651526586 shutdown_connections 2024-09-16T07:30:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.244+0000 7f738f800640 1 -- 172.21.15.71:0/3651526586 wait complete. 2024-09-16T07:30:20.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.246+0000 7f738f800640 1 Processor -- start 2024-09-16T07:30:20.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.246+0000 7f738f800640 1 -- start start 2024-09-16T07:30:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.247+0000 7f738f800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73880739d0 0x7f738814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.247+0000 7f738f800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73880cd490 0x7f73880ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.247+0000 7f738f800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73881517a0 0x7f7388159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.247+0000 7f738f800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f73880cc7d0 con 0x7f73880739d0 2024-09-16T07:30:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.247+0000 7f738f800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f73880cc650 con 0x7f73880cd490 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.247+0000 7f738f800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f73880cc950 con 0x7f73881517a0 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.247+0000 7f738ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73880739d0 0x7f738814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73880739d0 0x7f738814f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60546/0 (socket says 172.21.15.71:60546) 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738ce00640 1 -- 172.21.15.71:0/968506202 learned_addr learned my addr 172.21.15.71:0/968506202 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f7387e00640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73880cd490 0x7f73880ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738d800640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73881517a0 0x7f7388159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738ce00640 1 -- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73881517a0 msgr2=0x7f7388159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738ce00640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73881517a0 0x7f7388159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738ce00640 1 -- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73880cd490 msgr2=0x7f73880ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738ce00640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73880cd490 0x7f73880ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738ce00640 1 -- 172.21.15.71:0/968506202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738815cce0 con 0x7f73880739d0 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738d800640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73881517a0 0x7f7388159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:30:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f738ce00640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73880739d0 0x7f738814f2a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7378009920 tx=0x7f7378033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.248+0000 7f7384c00640 1 -- 172.21.15.71:0/968506202 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378031d60 con 0x7f73880739d0 2024-09-16T07:30:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.249+0000 7f7384c00640 1 -- 172.21.15.71:0/968506202 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7378033e10 con 0x7f73880739d0 2024-09-16T07:30:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.249+0000 7f7384c00640 1 -- 172.21.15.71:0/968506202 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378005260 con 0x7f73880739d0 2024-09-16T07:30:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.249+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738815cfc0 con 0x7f73880739d0 2024-09-16T07:30:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.249+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738815d580 con 0x7f73880739d0 2024-09-16T07:30:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.250+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73880cdad0 con 0x7f73880739d0 2024-09-16T07:30:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.251+0000 7f7384c00640 1 -- 172.21.15.71:0/968506202 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 102460+0+0 (secure 0 0 0) 0x7f7378005400 con 0x7f73880739d0 2024-09-16T07:30:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.252+0000 7f7384c00640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7368079540 0x7f736807ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.252+0000 7f7384c00640 1 -- 172.21.15.71:0/968506202 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f73780ba380 con 0x7f73880739d0 2024-09-16T07:30:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.252+0000 7f7387e00640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7368079540 0x7f736807ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:20.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.253+0000 7f7387e00640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7368079540 0x7f736807ba00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73880b30a0 tx=0x7f7374007cc0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:20.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.256+0000 7f7384c00640 1 -- 172.21.15.71:0/968506202 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f737803f030 con 0x7f73880739d0 2024-09-16T07:30:20.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.469+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f73880cd190 con 0x7f7368079540 2024-09-16T07:30:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.474+0000 7f7384c00640 1 -- 172.21.15.71:0/968506202 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f73880cd190 con 0x7f7368079540 2024-09-16T07:30:20.474 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95 2024-09-16T07:30:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.476+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7368079540 msgr2=0x7f736807ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.476+0000 7f738f800640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7368079540 0x7f736807ba00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73880b30a0 tx=0x7f7374007cc0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.476+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73880739d0 msgr2=0x7f738814f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.476+0000 7f738f800640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73880739d0 0x7f738814f2a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7378009920 tx=0x7f7378033c20 comp rx=0 tx=0).stop 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.477+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 shutdown_connections 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.477+0000 7f738f800640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7368079540 0x7f736807ba00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.477+0000 7f738f800640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73881517a0 0x7f7388159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.477+0000 7f738f800640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73880cd490 0x7f73880ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.477+0000 7f738f800640 1 --2- 172.21.15.71:0/968506202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73880739d0 0x7f738814f2a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.477+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 >> 172.21.15.71:0/968506202 conn(0x7f73880b1840 msgr2=0x7f73881e68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.477+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 shutdown_connections 2024-09-16T07:30:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:20.477+0000 7f738f800640 1 -- 172.21.15.71:0/968506202 wait complete. 2024-09-16T07:30:20.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:30:20.492 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95 2024-09-16T07:30:20.492 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:30:20.528 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95 2024-09-16T07:30:20.529 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="3055190016" 2024-09-16T07:30:20.529 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:20.530 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-16T07:30:22.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:30:22.531 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95 2024-09-16T07:30:22.531 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:30:22.572 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/4758f611-c533-4608-bae0-021a87395d95 2024-09-16T07:30:22.572 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T07:30:22.572 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:22.573 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T07:30:22.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone82630 2024-09-16T07:30:22.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.783+0000 7f5f1e800640 1 -- 172.21.15.71:0/2821701994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f1815f670 msgr2=0x7f5f180c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:22.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.783+0000 7f5f1e800640 1 --2- 172.21.15.71:0/2821701994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f1815f670 0x7f5f180c6430 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009830 tx=0x7f5f0c02a510 comp rx=0 tx=0).stop 2024-09-16T07:30:22.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.783+0000 7f5f1e800640 1 -- 172.21.15.71:0/2821701994 shutdown_connections 2024-09-16T07:30:22.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.783+0000 7f5f1e800640 1 --2- 172.21.15.71:0/2821701994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f1815f670 0x7f5f180c6430 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:22.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.783+0000 7f5f1e800640 1 -- 172.21.15.71:0/2821701994 >> 172.21.15.71:0/2821701994 conn(0x7f5f180cc490 msgr2=0x7f5f180cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:22.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.783+0000 7f5f1e800640 1 -- 172.21.15.71:0/2821701994 shutdown_connections 2024-09-16T07:30:22.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.783+0000 7f5f1e800640 1 -- 172.21.15.71:0/2821701994 wait complete. 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.785+0000 7f5f1e800640 1 Processor -- start 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.785+0000 7f5f1e800640 1 -- start start 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f1806e660 0x7f5f18159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f180c5730 0x7f5f181f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1e800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f1815f670 0x7f5f181f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1e800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180ca4f0 con 0x7f5f180c5730 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1e800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180ca370 con 0x7f5f1806e660 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1e800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1806f1d0 con 0x7f5f1815f670 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f180c5730 0x7f5f181f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:22.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f180c5730 0x7f5f181f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60568/0 (socket says 172.21.15.71:60568) 2024-09-16T07:30:22.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1ca00640 1 -- 172.21.15.71:0/764616142 learned_addr learned my addr 172.21.15.71:0/764616142 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:22.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1d400640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f1806e660 0x7f5f18159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:22.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1de00640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f1815f670 0x7f5f181f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:22.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1ca00640 1 -- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f1815f670 msgr2=0x7f5f181f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:22.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1ca00640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f1815f670 0x7f5f181f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:22.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1ca00640 1 -- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f1806e660 msgr2=0x7f5f18159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:22.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1ca00640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f1806e660 0x7f5f18159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1ca00640 1 -- 172.21.15.71:0/764616142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181f3670 con 0x7f5f180c5730 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.786+0000 7f5f1d400640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f1806e660 0x7f5f18159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.787+0000 7f5f1ca00640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f180c5730 0x7f5f181f3130 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c440 tx=0x7f5f0400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.787+0000 7f5f15600640 1 -- 172.21.15.71:0/764616142 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04005480 con 0x7f5f180c5730 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.787+0000 7f5f15600640 1 -- 172.21.15.71:0/764616142 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f040035f0 con 0x7f5f180c5730 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.787+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f18158580 con 0x7f5f180c5730 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.787+0000 7f5f15600640 1 -- 172.21.15.71:0/764616142 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04003a20 con 0x7f5f180c5730 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.787+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f18158b40 con 0x7f5f180c5730 2024-09-16T07:30:22.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.788+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f18142cf0 con 0x7f5f180c5730 2024-09-16T07:30:22.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.789+0000 7f5f15600640 1 -- 172.21.15.71:0/764616142 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 102460+0+0 (secure 0 0 0) 0x7f5f04003bc0 con 0x7f5f180c5730 2024-09-16T07:30:22.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.790+0000 7f5f15600640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5ef8079710 0x7f5ef807bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:22.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.790+0000 7f5f15600640 1 -- 172.21.15.71:0/764616142 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f5f04098ec0 con 0x7f5f180c5730 2024-09-16T07:30:22.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.791+0000 7f5f1d400640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5ef8079710 0x7f5ef807bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:22.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.791+0000 7f5f1d400640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5ef8079710 0x7f5ef807bbd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c034000 tx=0x7f5f0c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:22.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:22.794+0000 7f5f15600640 1 -- 172.21.15.71:0/764616142 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5f18142cf0 con 0x7f5f180c5730 2024-09-16T07:30:23.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.011+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone82630", "target": ["mon-mgr", ""]}) -- 0x7f5f180ccbf0 con 0x7f5ef8079710 2024-09-16T07:30:23.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.088+0000 7f5f15600640 1 -- 172.21.15.71:0/764616142 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5f180ccbf0 con 0x7f5ef8079710 2024-09-16T07:30:23.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5ef8079710 msgr2=0x7f5ef807bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:23.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5ef8079710 0x7f5ef807bbd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c034000 tx=0x7f5f0c004730 comp rx=0 tx=0).stop 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f180c5730 msgr2=0x7f5f181f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f180c5730 0x7f5f181f3130 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c440 tx=0x7f5f0400c900 comp rx=0 tx=0).stop 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 shutdown_connections 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5ef8079710 0x7f5ef807bbd0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f1815f670 0x7f5f181f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f180c5730 0x7f5f181f3130 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 --2- 172.21.15.71:0/764616142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f1806e660 0x7f5f18159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 >> 172.21.15.71:0/764616142 conn(0x7f5f180cc490 msgr2=0x7f5f180af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 shutdown_connections 2024-09-16T07:30:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.091+0000 7f5f1e800640 1 -- 172.21.15.71:0/764616142 wait complete. 2024-09-16T07:30:23.095 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.095+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.095 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.095+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.095 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.095+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.095 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.095+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.095 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.095+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone84781 2024-09-16T07:30:23.112 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.112+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.112 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.112+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.112 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.112+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.112 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.112+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.112 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:23.112+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:23.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.276+0000 7fd861000640 1 -- 172.21.15.71:0/3700862818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0c5730 msgr2=0x7fd85c0c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.276+0000 7fd861000640 1 --2- 172.21.15.71:0/3700862818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0c5730 0x7fd85c0c6430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd848009830 tx=0x7fd84802a510 comp rx=0 tx=0).stop 2024-09-16T07:30:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.276+0000 7fd861000640 1 -- 172.21.15.71:0/3700862818 shutdown_connections 2024-09-16T07:30:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.276+0000 7fd861000640 1 --2- 172.21.15.71:0/3700862818 >> v2:172.21.15.177:3300/0 conn(0x7fd85c06e660 0x7fd85c06eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.276+0000 7fd861000640 1 --2- 172.21.15.71:0/3700862818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0c5730 0x7fd85c0c6430 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.276+0000 7fd861000640 1 -- 172.21.15.71:0/3700862818 >> 172.21.15.71:0/3700862818 conn(0x7fd85c0cc490 msgr2=0x7fd85c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.276+0000 7fd861000640 1 -- 172.21.15.71:0/3700862818 shutdown_connections 2024-09-16T07:30:23.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.276+0000 7fd861000640 1 -- 172.21.15.71:0/3700862818 wait complete. 2024-09-16T07:30:23.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.277+0000 7fd861000640 1 Processor -- start 2024-09-16T07:30:23.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.278+0000 7fd861000640 1 -- start start 2024-09-16T07:30:23.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.278+0000 7fd861000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c06e660 0x7fd85c151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.278+0000 7fd861000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0c5730 0x7fd85c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.278+0000 7fd861000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f670 0x7fd85c15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.278+0000 7fd861000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c0ca4f0 con 0x7fd85c15f670 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.278+0000 7fd861000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c0ca370 con 0x7fd85c06e660 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.278+0000 7fd861000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd85c06f050 con 0x7fd85c0c5730 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f670 0x7fd85c15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f670 0x7fd85c15e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60584/0 (socket says 172.21.15.71:60584) 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 -- 172.21.15.71:0/3662042374 learned_addr learned my addr 172.21.15.71:0/3662042374 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85b400640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c06e660 0x7fd85c151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85aa00640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0c5730 0x7fd85c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:23.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 -- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0c5730 msgr2=0x7fd85c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:23.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0c5730 0x7fd85c15b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:23.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 -- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c06e660 msgr2=0x7fd85c151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:23.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c06e660 0x7fd85c151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:23.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 -- 172.21.15.71:0/3662042374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c151960 con 0x7fd85c15f670 2024-09-16T07:30:23.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.279+0000 7fd85be00640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f670 0x7fd85c15e0b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd85400eb10 tx=0x7fd85400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:23.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.280+0000 7fd853400640 1 -- 172.21.15.71:0/3662042374 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854004b20 con 0x7fd85c15f670 2024-09-16T07:30:23.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.280+0000 7fd853400640 1 -- 172.21.15.71:0/3662042374 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd854004ce0 con 0x7fd85c15f670 2024-09-16T07:30:23.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.280+0000 7fd853400640 1 -- 172.21.15.71:0/3662042374 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854005280 con 0x7fd85c15f670 2024-09-16T07:30:23.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.280+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c0b48c0 con 0x7fd85c15f670 2024-09-16T07:30:23.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.280+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c0b4e80 con 0x7fd85c15f670 2024-09-16T07:30:23.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.281+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c06eb10 con 0x7fd85c15f670 2024-09-16T07:30:23.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.282+0000 7fd853400640 1 -- 172.21.15.71:0/3662042374 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 102460+0+0 (secure 0 0 0) 0x7fd854005420 con 0x7fd85c15f670 2024-09-16T07:30:23.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.283+0000 7fd853400640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd83c079540 0x7fd83c07ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:23.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.283+0000 7fd853400640 1 -- 172.21.15.71:0/3662042374 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fd854099f90 con 0x7fd85c15f670 2024-09-16T07:30:23.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.283+0000 7fd85b400640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd83c079540 0x7fd83c07ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:23.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.284+0000 7fd85b400640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd83c079540 0x7fd83c07ba00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd84c007fd0 tx=0x7fd84c007e50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:23.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.287+0000 7fd853400640 1 -- 172.21.15.71:0/3662042374 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8540a2020 con 0x7fd85c15f670 2024-09-16T07:30:23.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:23.504+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone84781", "target": ["mon-mgr", ""]}) -- 0x7fd85c0b0b50 con 0x7fd83c079540 2024-09-16T07:30:24.093 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:24.094+0000 7fe745e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:24.093 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:24.094+0000 7fe745e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:24.093 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:24.094+0000 7fe745e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:24.094 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:24.094+0000 7fe745e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:24.094 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:24.094+0000 7fe745e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:24.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.184+0000 7fd853400640 1 -- 172.21.15.71:0/3662042374 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd85c0b0b50 con 0x7fd83c079540 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd83c079540 msgr2=0x7fd83c07ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd83c079540 0x7fd83c07ba00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd84c007fd0 tx=0x7fd84c007e50 comp rx=0 tx=0).stop 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f670 msgr2=0x7fd85c15e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f670 0x7fd85c15e0b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd85400eb10 tx=0x7fd85400c2d0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 shutdown_connections 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd83c079540 0x7fd83c07ba00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd85c15f670 0x7fd85c15e0b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd85c0c5730 0x7fd85c15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 --2- 172.21.15.71:0/3662042374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd85c06e660 0x7fd85c151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.187+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 >> 172.21.15.71:0/3662042374 conn(0x7fd85c0cc490 msgr2=0x7fd85c0af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:24.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.188+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 shutdown_connections 2024-09-16T07:30:24.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.188+0000 7fd861000640 1 -- 172.21.15.71:0/3662042374 wait complete. 2024-09-16T07:30:24.209 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone19152 2024-09-16T07:30:24.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.463+0000 7f1543200640 1 -- 172.21.15.71:0/1172845118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440b56e0 msgr2=0x7f15440b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.463+0000 7f1543200640 1 --2- 172.21.15.71:0/1172845118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440b56e0 0x7f15440b5b70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f153c005fd0 tx=0x7f153c02a5c0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.464+0000 7f1543200640 1 -- 172.21.15.71:0/1172845118 shutdown_connections 2024-09-16T07:30:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.464+0000 7f1543200640 1 --2- 172.21.15.71:0/1172845118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440b56e0 0x7f15440b5b70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.464+0000 7f1543200640 1 --2- 172.21.15.71:0/1172845118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15440ce640 0x7f15440cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.464+0000 7f1543200640 1 -- 172.21.15.71:0/1172845118 >> 172.21.15.71:0/1172845118 conn(0x7f15440d1c30 msgr2=0x7f15440d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.464+0000 7f1543200640 1 -- 172.21.15.71:0/1172845118 shutdown_connections 2024-09-16T07:30:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.464+0000 7f1543200640 1 -- 172.21.15.71:0/1172845118 wait complete. 2024-09-16T07:30:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.466+0000 7f1543200640 1 Processor -- start 2024-09-16T07:30:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.466+0000 7f1543200640 1 -- start start 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.467+0000 7f1543200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15440b56e0 0x7f1544150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.467+0000 7f1543200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440ce640 0x7f15441e8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.467+0000 7f1543200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15440d2750 0x7f15441e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.467+0000 7f1543200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f15440b6440 con 0x7f15440ce640 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.467+0000 7f1543200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15440b62c0 con 0x7f15440b56e0 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.467+0000 7f1543200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f15440b65c0 con 0x7f15440d2750 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.467+0000 7f1541400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440ce640 0x7f15441e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1541400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440ce640 0x7f15441e8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60612/0 (socket says 172.21.15.71:60612) 2024-09-16T07:30:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1541400640 1 -- 172.21.15.71:0/3779681006 learned_addr learned my addr 172.21.15.71:0/3779681006 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1541e00640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15440b56e0 0x7f1544150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1542800640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15440d2750 0x7f15441e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1541400640 1 -- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15440d2750 msgr2=0x7f15441e8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1541400640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15440d2750 0x7f15441e8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1541400640 1 -- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15440b56e0 msgr2=0x7f1544150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1541400640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15440b56e0 0x7f1544150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1541400640 1 -- 172.21.15.71:0/3779681006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154406ddb0 con 0x7f15440ce640 2024-09-16T07:30:24.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.468+0000 7f1542800640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15440d2750 0x7f15441e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:30:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.469+0000 7f1541400640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440ce640 0x7f15441e8940 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b3f0 tx=0x7f152c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.469+0000 7f153a000640 1 -- 172.21.15.71:0/3779681006 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c015020 con 0x7f15440ce640 2024-09-16T07:30:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.469+0000 7f153a000640 1 -- 172.21.15.71:0/3779681006 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f152c011920 con 0x7f15440ce640 2024-09-16T07:30:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.469+0000 7f153a000640 1 -- 172.21.15.71:0/3779681006 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c011d70 con 0x7f15440ce640 2024-09-16T07:30:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.469+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154406e090 con 0x7f15440ce640 2024-09-16T07:30:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.469+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1544150560 con 0x7f15440ce640 2024-09-16T07:30:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.472+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15440d0b20 con 0x7f15440ce640 2024-09-16T07:30:24.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.474+0000 7f153a000640 1 -- 172.21.15.71:0/3779681006 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 102460+0+0 (secure 0 0 0) 0x7f152c018020 con 0x7f15440ce640 2024-09-16T07:30:24.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.475+0000 7f153a000640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1520079590 0x7f152007ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:24.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.476+0000 7f153a000640 1 -- 172.21.15.71:0/3779681006 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f152c098ff0 con 0x7f15440ce640 2024-09-16T07:30:24.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.478+0000 7f1541e00640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1520079590 0x7f152007ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:24.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.483+0000 7f153a000640 1 -- 172.21.15.71:0/3779681006 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f15440d0b20 con 0x7f15440ce640 2024-09-16T07:30:24.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.485+0000 7f1541e00640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1520079590 0x7f152007ba50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1534009f20 tx=0x7f15340094f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:24.871+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone19152", "target": ["mon-mgr", ""]}) -- 0x7f15440c7d70 con 0x7f1520079590 2024-09-16T07:30:25.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.270+0000 7f153a000640 1 -- 172.21.15.71:0/3779681006 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15440c7d70 con 0x7f1520079590 2024-09-16T07:30:25.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1520079590 msgr2=0x7f152007ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:25.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1520079590 0x7f152007ba50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1534009f20 tx=0x7f15340094f0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440ce640 msgr2=0x7f15441e8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440ce640 0x7f15441e8940 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b3f0 tx=0x7f152c00b8b0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 shutdown_connections 2024-09-16T07:30:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1520079590 0x7f152007ba50 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1534009f20 tx=0x7f15340094f0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15440d2750 0x7f15441e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15440ce640 0x7f15441e8940 secure :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b3f0 tx=0x7f152c00b8b0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 --2- 172.21.15.71:0/3779681006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15440b56e0 0x7f1544150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.274+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 >> 172.21.15.71:0/3779681006 conn(0x7f15440d1c30 msgr2=0x7f15440b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:25.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.275+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 shutdown_connections 2024-09-16T07:30:25.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.275+0000 7f1543200640 1 -- 172.21.15.71:0/3779681006 wait complete. 2024-09-16T07:30:25.287 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.288+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.287 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.288+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.287 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.288+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.287 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.288+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.287 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.288+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.314 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone22153 2024-09-16T07:30:25.395 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.397+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.395 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.397+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.395 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.397+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.396 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.397+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.396 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:25.397+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.597+0000 7f9f64800640 1 -- 172.21.15.71:0/2565568391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c161590 msgr2=0x7f9f5c0b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.597+0000 7f9f64800640 1 --2- 172.21.15.71:0/2565568391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c161590 0x7f9f5c0b6e00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0cf2a0 tx=0x7f9f5002a510 comp rx=0 tx=0).stop 2024-09-16T07:30:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.597+0000 7f9f64800640 1 -- 172.21.15.71:0/2565568391 shutdown_connections 2024-09-16T07:30:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.597+0000 7f9f64800640 1 --2- 172.21.15.71:0/2565568391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c161590 0x7f9f5c0b6e00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.597+0000 7f9f64800640 1 -- 172.21.15.71:0/2565568391 >> 172.21.15.71:0/2565568391 conn(0x7f9f5c0d3fb0 msgr2=0x7f9f5c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.597+0000 7f9f64800640 1 -- 172.21.15.71:0/2565568391 shutdown_connections 2024-09-16T07:30:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.597+0000 7f9f64800640 1 -- 172.21.15.71:0/2565568391 wait complete. 2024-09-16T07:30:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.600+0000 7f9f64800640 1 Processor -- start 2024-09-16T07:30:25.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.600+0000 7f9f64800640 1 -- start start 2024-09-16T07:30:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.602+0000 7f9f64800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c0b5630 0x7f9f5c1fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.602+0000 7f9f64800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c0b7340 0x7f9f5c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.602+0000 7f9f64800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f5c161590 0x7f9f5c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.602+0000 7f9f64800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c0d2740 con 0x7f9f5c0b7340 2024-09-16T07:30:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.602+0000 7f9f64800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c0cfed0 con 0x7f9f5c0b5630 2024-09-16T07:30:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.602+0000 7f9f64800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5c0d28c0 con 0x7f9f5c161590 2024-09-16T07:30:25.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c0b7340 0x7f9f5c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:25.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c0b7340 0x7f9f5c154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46540/0 (socket says 172.21.15.71:46540) 2024-09-16T07:30:25.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61400640 1 -- 172.21.15.71:0/774958458 learned_addr learned my addr 172.21.15.71:0/774958458 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:25.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61e00640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c0b5630 0x7f9f5c1fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61400640 1 -- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f5c161590 msgr2=0x7f9f5c06dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61400640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f5c161590 0x7f9f5c06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61400640 1 -- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c0b5630 msgr2=0x7f9f5c1fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61400640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c0b5630 0x7f9f5c1fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.603+0000 7f9f61400640 1 -- 172.21.15.71:0/774958458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c1f8fd0 con 0x7f9f5c0b7340 2024-09-16T07:30:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.604+0000 7f9f61400640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c0b7340 0x7f9f5c154320 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c500 tx=0x7f9f4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:25.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.604+0000 7f9f56000640 1 -- 172.21.15.71:0/774958458 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c017070 con 0x7f9f5c0b7340 2024-09-16T07:30:25.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.604+0000 7f9f56000640 1 -- 172.21.15.71:0/774958458 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f4c0035f0 con 0x7f9f5c0b7340 2024-09-16T07:30:25.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.604+0000 7f9f56000640 1 -- 172.21.15.71:0/774958458 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c003a40 con 0x7f9f5c0b7340 2024-09-16T07:30:25.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.604+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c1f92b0 con 0x7f9f5c0b7340 2024-09-16T07:30:25.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.604+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1f9870 con 0x7f9f5c0b7340 2024-09-16T07:30:25.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.606+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c0d2d60 con 0x7f9f5c0b7340 2024-09-16T07:30:25.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.608+0000 7f9f56000640 1 -- 172.21.15.71:0/774958458 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) ==== 102613+0+0 (secure 0 0 0) 0x7f9f4c01a070 con 0x7f9f5c0b7340 2024-09-16T07:30:25.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.609+0000 7f9f56000640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f400796f0 0x7f9f4007bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:25.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.609+0000 7f9f56000640 1 -- 172.21.15.71:0/774958458 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7f9f4c098b70 con 0x7f9f5c0b7340 2024-09-16T07:30:25.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.617+0000 7f9f56000640 1 -- 172.21.15.71:0/774958458 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9f4c0629b0 con 0x7f9f5c0b7340 2024-09-16T07:30:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.627+0000 7f9f61e00640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f400796f0 0x7f9f4007bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:25.671+0000 7f9f61e00640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f400796f0 0x7f9f4007bbb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9f50034000 tx=0x7f9f50004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:26.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:26.069+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone22153", "target": ["mon-mgr", ""]}) -- 0x7f9f5c1fd300 con 0x7f9f400796f0 2024-09-16T07:30:26.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:26.597+0000 7f9f56000640 1 -- 172.21.15.71:0/774958458 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7f9f4c05e520 con 0x7f9f5c0b7340 2024-09-16T07:30:27.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.291+0000 7f9f56000640 1 -- 172.21.15.71:0/774958458 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f5c1fd300 con 0x7f9f400796f0 2024-09-16T07:30:27.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f400796f0 msgr2=0x7f9f4007bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:27.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f400796f0 0x7f9f4007bbb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9f50034000 tx=0x7f9f50004730 comp rx=0 tx=0).stop 2024-09-16T07:30:27.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c0b7340 msgr2=0x7f9f5c154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:27.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c0b7340 0x7f9f5c154320 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c500 tx=0x7f9f4c00c9c0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 shutdown_connections 2024-09-16T07:30:27.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f400796f0 0x7f9f4007bbb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f5c161590 0x7f9f5c06dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f5c0b7340 0x7f9f5c154320 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 --2- 172.21.15.71:0/774958458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c0b5630 0x7f9f5c1fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.296+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 >> 172.21.15.71:0/774958458 conn(0x7f9f5c0d3fb0 msgr2=0x7f9f5c0d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:27.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.297+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 shutdown_connections 2024-09-16T07:30:27.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.297+0000 7f9f64800640 1 -- 172.21.15.71:0/774958458 wait complete. 2024-09-16T07:30:27.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone31984 2024-09-16T07:30:27.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.584+0000 7fd079000640 1 -- 172.21.15.71:0/1550670952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0740d0710 msgr2=0x7fd0740b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:27.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.584+0000 7fd079000640 1 --2- 172.21.15.71:0/1550670952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0740d0710 0x7fd0740b4c70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd0740cc400 tx=0x7fd06c02a7c0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.586+0000 7fd079000640 1 -- 172.21.15.71:0/1550670952 shutdown_connections 2024-09-16T07:30:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.586+0000 7fd079000640 1 --2- 172.21.15.71:0/1550670952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0740d0710 0x7fd0740b4c70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.586+0000 7fd079000640 1 --2- 172.21.15.71:0/1550670952 >> v2:172.21.15.177:3300/0 conn(0x7fd0740cd6f0 0x7fd0740d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.586+0000 7fd079000640 1 -- 172.21.15.71:0/1550670952 >> 172.21.15.71:0/1550670952 conn(0x7fd0740d1e20 msgr2=0x7fd0740d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.586+0000 7fd079000640 1 -- 172.21.15.71:0/1550670952 shutdown_connections 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.586+0000 7fd079000640 1 -- 172.21.15.71:0/1550670952 wait complete. 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.588+0000 7fd079000640 1 Processor -- start 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.589+0000 7fd079000640 1 -- start start 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.590+0000 7fd079000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0740cd6f0 0x7fd0740cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.591+0000 7fd079000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0740d0710 0x7fd07406f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.591+0000 7fd079000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd07415bf10 0x7fd07406f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.591+0000 7fd079000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740cfb70 con 0x7fd0740cd6f0 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.591+0000 7fd079000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd0740cf9f0 con 0x7fd0740d0710 2024-09-16T07:30:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.591+0000 7fd079000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd0740b51b0 con 0x7fd07415bf10 2024-09-16T07:30:27.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.592+0000 7fd071e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0740cd6f0 0x7fd0740cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:27.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.592+0000 7fd071e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0740cd6f0 0x7fd0740cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46550/0 (socket says 172.21.15.71:46550) 2024-09-16T07:30:27.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.592+0000 7fd071e00640 1 -- 172.21.15.71:0/1082312454 learned_addr learned my addr 172.21.15.71:0/1082312454 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:27.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.593+0000 7fd072800640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd07415bf10 0x7fd07406f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:27.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.594+0000 7fd071e00640 1 -- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd07415bf10 msgr2=0x7fd07406f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:27.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.594+0000 7fd071e00640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd07415bf10 0x7fd07406f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.594+0000 7fd071e00640 1 -- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0740d0710 msgr2=0x7fd07406f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:27.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.594+0000 7fd071e00640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0740d0710 0x7fd07406f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:27.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.594+0000 7fd071e00640 1 -- 172.21.15.71:0/1082312454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0741f4f10 con 0x7fd0740cd6f0 2024-09-16T07:30:27.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.595+0000 7fd071e00640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0740cd6f0 0x7fd0740cb330 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b3f0 tx=0x7fd06400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:27.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.596+0000 7fd06a000640 1 -- 172.21.15.71:0/1082312454 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064015020 con 0x7fd0740cd6f0 2024-09-16T07:30:27.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.596+0000 7fd06a000640 1 -- 172.21.15.71:0/1082312454 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd064011920 con 0x7fd0740cd6f0 2024-09-16T07:30:27.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.596+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0741f51f0 con 0x7fd0740cd6f0 2024-09-16T07:30:27.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.596+0000 7fd06a000640 1 -- 172.21.15.71:0/1082312454 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064011d70 con 0x7fd0740cd6f0 2024-09-16T07:30:27.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.597+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0741f57b0 con 0x7fd0740cd6f0 2024-09-16T07:30:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.598+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0740b5680 con 0x7fd0740cd6f0 2024-09-16T07:30:27.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.601+0000 7fd06a000640 1 -- 172.21.15.71:0/1082312454 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7fd064018020 con 0x7fd0740cd6f0 2024-09-16T07:30:27.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.602+0000 7fd06a000640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd054079890 0x7fd05407bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:27.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.602+0000 7fd06a000640 1 -- 172.21.15.71:0/1082312454 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7fd064099410 con 0x7fd0740cd6f0 2024-09-16T07:30:27.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.609+0000 7fd06a000640 1 -- 172.21.15.71:0/1082312454 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd06401f020 con 0x7fd0740cd6f0 2024-09-16T07:30:27.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.627+0000 7fd071400640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd054079890 0x7fd05407bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:27.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.641+0000 7fd071400640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd054079890 0x7fd05407bd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd060005fd0 tx=0x7fd060005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:27.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:27.965+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone31984", "target": ["mon-mgr", ""]}) -- 0x7fd0741f4cc0 con 0x7fd054079890 2024-09-16T07:30:29.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.245+0000 7fd06a000640 1 -- 172.21.15.71:0/1082312454 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd0741f4cc0 con 0x7fd054079890 2024-09-16T07:30:29.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.247+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd054079890 msgr2=0x7fd05407bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:29.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.247+0000 7fd079000640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd054079890 0x7fd05407bd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd060005fd0 tx=0x7fd060005850 comp rx=0 tx=0).stop 2024-09-16T07:30:29.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.247+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0740cd6f0 msgr2=0x7fd0740cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:29.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.247+0000 7fd079000640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0740cd6f0 0x7fd0740cb330 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b3f0 tx=0x7fd06400b8b0 comp rx=0 tx=0).stop 2024-09-16T07:30:29.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.248+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 shutdown_connections 2024-09-16T07:30:29.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.248+0000 7fd079000640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd054079890 0x7fd05407bd50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:29.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.248+0000 7fd079000640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd07415bf10 0x7fd07406f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:29.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.248+0000 7fd079000640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0740d0710 0x7fd07406f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:29.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.248+0000 7fd079000640 1 --2- 172.21.15.71:0/1082312454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0740cd6f0 0x7fd0740cb330 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:29.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.248+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 >> 172.21.15.71:0/1082312454 conn(0x7fd0740d1e20 msgr2=0x7fd0740d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:29.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.248+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 shutdown_connections 2024-09-16T07:30:29.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.248+0000 7fd079000640 1 -- 172.21.15.71:0/1082312454 wait complete. 2024-09-16T07:30:29.272 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20415 2024-09-16T07:30:29.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.572+0000 7ff651a00640 1 -- 172.21.15.71:0/2032040402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff630004800 msgr2=0x7ff630004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:29.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.572+0000 7ff651a00640 1 --2- 172.21.15.71:0/2032040402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff630004800 0x7ff630004c80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff63800c6f0 tx=0x7ff630005830 comp rx=0 tx=0).stop 2024-09-16T07:30:29.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.572+0000 7ff651a00640 1 -- 172.21.15.71:0/2032040402 shutdown_connections 2024-09-16T07:30:29.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.572+0000 7ff651a00640 1 --2- 172.21.15.71:0/2032040402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff630004800 0x7ff630004c80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:29.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.572+0000 7ff651a00640 1 -- 172.21.15.71:0/2032040402 >> 172.21.15.71:0/2032040402 conn(0x7ff64c0b1840 msgr2=0x7ff64c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:29.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.572+0000 7ff651a00640 1 -- 172.21.15.71:0/2032040402 shutdown_connections 2024-09-16T07:30:29.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.572+0000 7ff651a00640 1 -- 172.21.15.71:0/2032040402 wait complete. 2024-09-16T07:30:29.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.577+0000 7ff651a00640 1 Processor -- start 2024-09-16T07:30:29.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.577+0000 7ff651a00640 1 -- start start 2024-09-16T07:30:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.580+0000 7ff651a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff630004800 0x7ff64c1e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:29.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.580+0000 7ff651a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff64c1e7970 0x7ff64c0ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:29.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.580+0000 7ff651a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff64c0ac730 0x7ff64c138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:29.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.580+0000 7ff651a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c0737d0 con 0x7ff64c1e7970 2024-09-16T07:30:29.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.580+0000 7ff651a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c073650 con 0x7ff630004800 2024-09-16T07:30:29.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.580+0000 7ff651a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff64c0cc5c0 con 0x7ff64c0ac730 2024-09-16T07:30:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.581+0000 7ff64a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff64c1e7970 0x7ff64c0ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.581+0000 7ff64a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff64c1e7970 0x7ff64c0ac1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46576/0 (socket says 172.21.15.71:46576) 2024-09-16T07:30:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.582+0000 7ff64a000640 1 -- 172.21.15.71:0/858389752 learned_addr learned my addr 172.21.15.71:0/858389752 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.582+0000 7ff64b400640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff64c0ac730 0x7ff64c138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.583+0000 7ff64a000640 1 -- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff64c0ac730 msgr2=0x7ff64c138490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.583+0000 7ff64a000640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff64c0ac730 0x7ff64c138490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.583+0000 7ff64a000640 1 -- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff630004800 msgr2=0x7ff64c1e8ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:30:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.583+0000 7ff64a000640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff630004800 0x7ff64c1e8ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.583+0000 7ff64a000640 1 -- 172.21.15.71:0/858389752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c1e7e50 con 0x7ff64c1e7970 2024-09-16T07:30:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.585+0000 7ff64a000640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff64c1e7970 0x7ff64c0ac1f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff63800c7e0 tx=0x7ff638006150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:29.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.587+0000 7ff642a00640 1 -- 172.21.15.71:0/858389752 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638003810 con 0x7ff64c1e7970 2024-09-16T07:30:29.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.587+0000 7ff642a00640 1 -- 172.21.15.71:0/858389752 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6380039b0 con 0x7ff64c1e7970 2024-09-16T07:30:29.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.587+0000 7ff642a00640 1 -- 172.21.15.71:0/858389752 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638003de0 con 0x7ff64c1e7970 2024-09-16T07:30:29.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.587+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c1e8130 con 0x7ff64c1e7970 2024-09-16T07:30:29.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.588+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c1e86f0 con 0x7ff64c1e7970 2024-09-16T07:30:29.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.593+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c06a370 con 0x7ff64c1e7970 2024-09-16T07:30:29.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.595+0000 7ff642a00640 1 -- 172.21.15.71:0/858389752 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7ff63803e430 con 0x7ff64c1e7970 2024-09-16T07:30:29.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.598+0000 7ff642a00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff62c079810 0x7ff62c07bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:29.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.598+0000 7ff642a00640 1 -- 172.21.15.71:0/858389752 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5737+0+0 (secure 0 0 0) 0x7ff6380b8f10 con 0x7ff64c1e7970 2024-09-16T07:30:29.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.603+0000 7ff64aa00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff62c079810 0x7ff62c07bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:29.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.608+0000 7ff642a00640 1 -- 172.21.15.71:0/858389752 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff64c06a370 con 0x7ff64c1e7970 2024-09-16T07:30:29.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.615+0000 7ff64aa00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff62c079810 0x7ff62c07bcd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff644009ec0 tx=0x7ff644009880 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:29.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:29.989+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20415", "target": ["mon-mgr", ""]}) -- 0x7ff64c15e880 con 0x7ff62c079810 2024-09-16T07:30:31.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.342+0000 7ff642a00640 1 -- 172.21.15.71:0/858389752 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff644007e50 con 0x7ff62c079810 2024-09-16T07:30:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.350+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff62c079810 msgr2=0x7ff62c07bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.350+0000 7ff651a00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff62c079810 0x7ff62c07bcd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff644009ec0 tx=0x7ff644009880 comp rx=0 tx=0).stop 2024-09-16T07:30:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.350+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff64c1e7970 msgr2=0x7ff64c0ac1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.350+0000 7ff651a00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff64c1e7970 0x7ff64c0ac1f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff63800c7e0 tx=0x7ff638006150 comp rx=0 tx=0).stop 2024-09-16T07:30:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.352+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 shutdown_connections 2024-09-16T07:30:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.352+0000 7ff651a00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff62c079810 0x7ff62c07bcd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.352+0000 7ff651a00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff64c0ac730 0x7ff64c138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.352+0000 7ff651a00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff64c1e7970 0x7ff64c0ac1f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.352+0000 7ff651a00640 1 --2- 172.21.15.71:0/858389752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff630004800 0x7ff64c1e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.352+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 >> 172.21.15.71:0/858389752 conn(0x7ff64c0b1840 msgr2=0x7ff64c0b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.354+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 shutdown_connections 2024-09-16T07:30:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.355+0000 7ff651a00640 1 -- 172.21.15.71:0/858389752 wait complete. 2024-09-16T07:30:31.400 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone52566 2024-09-16T07:30:31.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.704+0000 7f708d200640 1 -- 172.21.15.71:0/1162419195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f706c004800 msgr2=0x7f706c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:31.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.704+0000 7f708d200640 1 --2- 172.21.15.71:0/1162419195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f706c004800 0x7f706c004c80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7078006fc0 tx=0x7f706c005830 comp rx=0 tx=0).stop 2024-09-16T07:30:31.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.706+0000 7f708d200640 1 -- 172.21.15.71:0/1162419195 shutdown_connections 2024-09-16T07:30:31.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.706+0000 7f708d200640 1 --2- 172.21.15.71:0/1162419195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f706c004800 0x7f706c004c80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:31.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.706+0000 7f708d200640 1 -- 172.21.15.71:0/1162419195 >> 172.21.15.71:0/1162419195 conn(0x7f70880d0d40 msgr2=0x7f70880d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:31.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.706+0000 7f708d200640 1 -- 172.21.15.71:0/1162419195 shutdown_connections 2024-09-16T07:30:31.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.706+0000 7f708d200640 1 -- 172.21.15.71:0/1162419195 wait complete. 2024-09-16T07:30:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.708+0000 7f708d200640 1 Processor -- start 2024-09-16T07:30:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.708+0000 7f708d200640 1 -- start start 2024-09-16T07:30:31.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.709+0000 7f708d200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f706c004800 0x7f70880af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.709+0000 7f708d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088149d60 0x7f708814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.709+0000 7f708d200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7088160060 0x7f70881604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.709+0000 7f708d200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70880cf9a0 con 0x7f7088149d60 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.709+0000 7f708d200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f708806d3d0 con 0x7f706c004800 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.709+0000 7f708d200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f70880cfb20 con 0x7f7088160060 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.710+0000 7f7085400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088149d60 0x7f708814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.710+0000 7f7085e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f706c004800 0x7f70880af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.710+0000 7f7085e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f706c004800 0x7f70880af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:58286/0 (socket says 172.21.15.71:58286) 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.710+0000 7f7085e00640 1 -- 172.21.15.71:0/82515352 learned_addr learned my addr 172.21.15.71:0/82515352 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:31.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.710+0000 7f7085400640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088149d60 0x7f708814a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46598/0 (socket says 172.21.15.71:46598) 2024-09-16T07:30:31.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.710+0000 7f7086800640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7088160060 0x7f70881604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:31.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.711+0000 7f7085e00640 1 -- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7088160060 msgr2=0x7f70881604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:31.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.711+0000 7f7085e00640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7088160060 0x7f70881604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:31.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.711+0000 7f7085e00640 1 -- 172.21.15.71:0/82515352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088149d60 msgr2=0x7f708814a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:31.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.711+0000 7f7085e00640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088149d60 0x7f708814a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:31.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.711+0000 7f7085e00640 1 -- 172.21.15.71:0/82515352 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7088160dd0 con 0x7f706c004800 2024-09-16T07:30:31.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.711+0000 7f7085e00640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f706c004800 0x7f70880af160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f707400e970 tx=0x7f707400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:31.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.712+0000 7f707e000640 1 -- 172.21.15.71:0/82515352 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707400ccc0 con 0x7f706c004800 2024-09-16T07:30:31.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.712+0000 7f707e000640 1 -- 172.21.15.71:0/82515352 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f707400ce60 con 0x7f706c004800 2024-09-16T07:30:31.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.712+0000 7f707e000640 1 -- 172.21.15.71:0/82515352 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074005280 con 0x7f706c004800 2024-09-16T07:30:31.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.712+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70881610b0 con 0x7f706c004800 2024-09-16T07:30:31.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.712+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70880b3c10 con 0x7f706c004800 2024-09-16T07:30:31.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.714+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70880cb390 con 0x7f706c004800 2024-09-16T07:30:31.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.716+0000 7f707e000640 1 -- 172.21.15.71:0/82515352 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7f7074005420 con 0x7f706c004800 2024-09-16T07:30:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.718+0000 7f707e000640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7068079810 0x7f706807bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.718+0000 7f707e000640 1 -- 172.21.15.71:0/82515352 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f707409a090 con 0x7f706c004800 2024-09-16T07:30:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.724+0000 7f7085400640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7068079810 0x7f706807bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.725+0000 7f7085400640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7068079810 0x7f706807bcd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7078033000 tx=0x7f707803a040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:31.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:31.729+0000 7f707e000640 1 -- 172.21.15.71:0/82515352 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7074063850 con 0x7f706c004800 2024-09-16T07:30:32.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:32.091+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone52566", "target": ["mon-mgr", ""]}) -- 0x7f70880b0e20 con 0x7f7068079810 2024-09-16T07:30:33.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.554+0000 7f707e000640 1 -- 172.21.15.71:0/82515352 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f70880b0e20 con 0x7f7068079810 2024-09-16T07:30:33.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.560+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7068079810 msgr2=0x7f706807bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:33.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.560+0000 7f708d200640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7068079810 0x7f706807bcd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7078033000 tx=0x7f707803a040 comp rx=0 tx=0).stop 2024-09-16T07:30:33.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.560+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f706c004800 msgr2=0x7f70880af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:33.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.560+0000 7f708d200640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f706c004800 0x7f70880af160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f707400e970 tx=0x7f707400ed70 comp rx=0 tx=0).stop 2024-09-16T07:30:33.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.561+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 shutdown_connections 2024-09-16T07:30:33.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.561+0000 7f708d200640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7068079810 0x7f706807bcd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.561+0000 7f708d200640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7088160060 0x7f70881604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.561+0000 7f708d200640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088149d60 0x7f708814a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.561+0000 7f708d200640 1 --2- 172.21.15.71:0/82515352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f706c004800 0x7f70880af160 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.561+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 >> 172.21.15.71:0/82515352 conn(0x7f70880d0d40 msgr2=0x7f708815d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:33.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.562+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 shutdown_connections 2024-09-16T07:30:33.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.562+0000 7f708d200640 1 -- 172.21.15.71:0/82515352 wait complete. 2024-09-16T07:30:33.591 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:30:33.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.883+0000 7f56e3000640 1 -- 172.21.15.71:0/1172074899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56dc161440 msgr2=0x7f56dc0b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.883+0000 7f56e3000640 1 --2- 172.21.15.71:0/1172074899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56dc161440 0x7f56dc0b3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009900 tx=0x7f56d002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.883+0000 7f56e3000640 1 -- 172.21.15.71:0/1172074899 shutdown_connections 2024-09-16T07:30:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.883+0000 7f56e3000640 1 --2- 172.21.15.71:0/1172074899 >> v2:172.21.15.177:3300/0 conn(0x7f56dc0d2a20 0x7f56dc0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.883+0000 7f56e3000640 1 --2- 172.21.15.71:0/1172074899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56dc161440 0x7f56dc0b3770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.883+0000 7f56e3000640 1 -- 172.21.15.71:0/1172074899 >> 172.21.15.71:0/1172074899 conn(0x7f56dc0d3e40 msgr2=0x7f56dc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.883+0000 7f56e3000640 1 -- 172.21.15.71:0/1172074899 shutdown_connections 2024-09-16T07:30:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.883+0000 7f56e3000640 1 -- 172.21.15.71:0/1172074899 wait complete. 2024-09-16T07:30:33.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.885+0000 7f56e3000640 1 Processor -- start 2024-09-16T07:30:33.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.885+0000 7f56e3000640 1 -- start start 2024-09-16T07:30:33.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56dc0b3cb0 0x7f56dc0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:33.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e3000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56dc0d2a20 0x7f56dc0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:33.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e3000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56dc161440 0x7f56dc0c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:33.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1619a0 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e3000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc161820 con 0x7f56dc161440 2024-09-16T07:30:33.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e3000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc0b5fd0 con 0x7f56dc0d2a20 2024-09-16T07:30:33.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56dc0b3cb0 0x7f56dc0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:33.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56dc0b3cb0 0x7f56dc0cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46602/0 (socket says 172.21.15.71:46602) 2024-09-16T07:30:33.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.886+0000 7f56e1c00640 1 -- 172.21.15.71:0/318563252 learned_addr learned my addr 172.21.15.71:0/318563252 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:33.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.887+0000 7f56e1200640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56dc0d2a20 0x7f56dc0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:33.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.887+0000 7f56e1c00640 1 -- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56dc0d2a20 msgr2=0x7f56dc0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:33.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.887+0000 7f56e1c00640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56dc0d2a20 0x7f56dc0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.887+0000 7f56e1c00640 1 -- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56dc161440 msgr2=0x7f56dc0c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.887+0000 7f56e1c00640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56dc161440 0x7f56dc0c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.887+0000 7f56e1c00640 1 -- 172.21.15.71:0/318563252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc0b7bb0 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.887+0000 7f56e1c00640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56dc0b3cb0 0x7f56dc0cae80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f56d0006010 tx=0x7f56d0003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.888+0000 7f56d9600640 1 -- 172.21.15.71:0/318563252 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d003e030 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.888+0000 7f56d9600640 1 -- 172.21.15.71:0/318563252 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56d0003d70 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.888+0000 7f56d9600640 1 -- 172.21.15.71:0/318563252 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0035040 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.888+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc0b6df0 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.888+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc0b73b0 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.889+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56dc0b40f0 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.891+0000 7f56d9600640 1 -- 172.21.15.71:0/318563252 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7f56d0041030 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.892+0000 7f56d9600640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56bc079810 0x7f56bc07bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:33.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.893+0000 7f56d9600640 1 -- 172.21.15.71:0/318563252 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f56d00b9ac0 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.897+0000 7f56d9600640 1 -- 172.21.15.71:0/318563252 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f56d0083b80 con 0x7f56dc0b3cb0 2024-09-16T07:30:33.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.913+0000 7f56e1200640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56bc079810 0x7f56bc07bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:33.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:33.927+0000 7f56e1200640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56bc079810 0x7f56bc07bcd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f56dc155dc0 tx=0x7f56c8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:34.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.359+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f56dc0b3780 con 0x7f56dc0b3cb0 2024-09-16T07:30:34.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.360+0000 7f56d9600640 1 -- 172.21.15.71:0/318563252 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2184 (secure 0 0 0) 0x7f56d0083480 con 0x7f56dc0b3cb0 2024-09-16T07:30:34.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:30:34.360 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 491, 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:34.361 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 79, 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:30:34.362 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 1512, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 6196068686, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 12182380544, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 760911732736, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 390208710, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 371394772, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 95, 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 102 2024-09-16T07:30:34.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 47, 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:29:54:247057+0000", 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5, 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 5, 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "c", 2024-09-16T07:30:34.364 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 6417 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:30:34.365 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 12, 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:28:56.398876+0000", 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "4 ongoing clones - average progress is 39.258% (7s)\n [==========..................] (remaining: 11s)", 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.39258000254631042, 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:34.366 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:34.367 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:30:34.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.363+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56bc079810 msgr2=0x7f56bc07bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:34.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.363+0000 7f56e3000640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56bc079810 0x7f56bc07bcd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f56dc155dc0 tx=0x7f56c8009040 comp rx=0 tx=0).stop 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.363+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56dc0b3cb0 msgr2=0x7f56dc0cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.363+0000 7f56e3000640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56dc0b3cb0 0x7f56dc0cae80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f56d0006010 tx=0x7f56d0003880 comp rx=0 tx=0).stop 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.364+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 shutdown_connections 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.364+0000 7f56e3000640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f56bc079810 0x7f56bc07bcd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.364+0000 7f56e3000640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56dc161440 0x7f56dc0c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.364+0000 7f56e3000640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56dc0d2a20 0x7f56dc0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.364+0000 7f56e3000640 1 --2- 172.21.15.71:0/318563252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56dc0b3cb0 0x7f56dc0cae80 secure :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f56d0006010 tx=0x7f56d0003880 comp rx=0 tx=0).stop 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.364+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 >> 172.21.15.71:0/318563252 conn(0x7f56dc0d3e40 msgr2=0x7f56dc0b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.364+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 shutdown_connections 2024-09-16T07:30:34.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:34.364+0000 7f56e3000640 1 -- 172.21.15.71:0/318563252 wait complete. 2024-09-16T07:30:35.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:30:35.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.681+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2929067450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda8003a00 msgr2=0x7fbda8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:35.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.681+0000 7fbdc8c00640 1 --2- 172.21.15.71:0/2929067450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda8003a00 0x7fbda8003e20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400bb20 tx=0x7fbda8005830 comp rx=0 tx=0).stop 2024-09-16T07:30:35.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.681+0000 7fbdc2800640 1 -- 172.21.15.71:0/2929067450 reap_dead start 2024-09-16T07:30:35.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.681+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2929067450 shutdown_connections 2024-09-16T07:30:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.681+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2929067450 >> 172.21.15.71:0/2929067450 conn(0x7fbdc40b18e0 msgr2=0x7fbdc40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.681+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2929067450 shutdown_connections 2024-09-16T07:30:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.681+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2929067450 wait complete. 2024-09-16T07:30:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.683+0000 7fbdc8c00640 1 Processor -- start 2024-09-16T07:30:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.683+0000 7fbdc8c00640 1 -- start start 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.684+0000 7fbdc8c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc40b0760 0x7fbdc41f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.684+0000 7fbdc8c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40b0b40 0x7fbdc40cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.684+0000 7fbdc8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc40d0030 0x7fbdc41f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.684+0000 7fbdc8c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc415fd80 con 0x7fbdc40d0030 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.684+0000 7fbdc8c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc40cd5e0 con 0x7fbdc40b0760 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.684+0000 7fbdc8c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc415ff00 con 0x7fbdc40b0b40 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.685+0000 7fbdc2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc40d0030 0x7fbdc41f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.685+0000 7fbdc2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc40d0030 0x7fbdc41f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58560/0 (socket says 172.21.15.71:58560) 2024-09-16T07:30:35.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.685+0000 7fbdc2800640 1 -- 172.21.15.71:0/2097180635 learned_addr learned my addr 172.21.15.71:0/2097180635 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:35.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.685+0000 7fbdc1400640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40b0b40 0x7fbdc40cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:35.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.685+0000 7fbdc1e00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc40b0760 0x7fbdc41f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:35.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.685+0000 7fbdc2800640 1 -- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40b0b40 msgr2=0x7fbdc40cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:35.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.685+0000 7fbdc2800640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40b0b40 0x7fbdc40cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:35.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.686+0000 7fbdc2800640 1 -- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc40b0760 msgr2=0x7fbdc41f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:35.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.686+0000 7fbdc2800640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc40b0760 0x7fbdc41f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:35.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.686+0000 7fbdc2800640 1 -- 172.21.15.71:0/2097180635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc41f5cb0 con 0x7fbdc40d0030 2024-09-16T07:30:35.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.686+0000 7fbdc1e00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc40b0760 0x7fbdc41f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:30:35.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.686+0000 7fbdc2800640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc40d0030 0x7fbdc41f8320 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4069770 tx=0x7fbdbc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:35.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.687+0000 7fbdba000640 1 -- 172.21.15.71:0/2097180635 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc00dd80 con 0x7fbdc40d0030 2024-09-16T07:30:35.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.687+0000 7fbdba000640 1 -- 172.21.15.71:0/2097180635 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdbc016950 con 0x7fbdc40d0030 2024-09-16T07:30:35.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.687+0000 7fbdba000640 1 -- 172.21.15.71:0/2097180635 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc016dc0 con 0x7fbdc40d0030 2024-09-16T07:30:35.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.688+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc40acf80 con 0x7fbdc40d0030 2024-09-16T07:30:35.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.688+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc40ad540 con 0x7fbdc40d0030 2024-09-16T07:30:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.692+0000 7fbdba000640 1 -- 172.21.15.71:0/2097180635 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7fbdbc01e400 con 0x7fbdc40d0030 2024-09-16T07:30:35.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.692+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc4069360 con 0x7fbdc40d0030 2024-09-16T07:30:35.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.693+0000 7fbdba000640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda4079910 0x7fbda407bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:35.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.694+0000 7fbdba000640 1 -- 172.21.15.71:0/2097180635 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7fbdbc0a20e0 con 0x7fbdc40d0030 2024-09-16T07:30:35.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.698+0000 7fbdc1e00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda4079910 0x7fbda407bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:35.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.704+0000 7fbdba000640 1 -- 172.21.15.71:0/2097180635 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbdbc024060 con 0x7fbdc40d0030 2024-09-16T07:30:35.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:35.711+0000 7fbdc1e00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda4079910 0x7fbda407bdd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4036000 tx=0x7fbdb40046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:36.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.121+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbdc406eb70 con 0x7fbdc40d0030 2024-09-16T07:30:36.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.123+0000 7fbdba000640 1 -- 172.21.15.71:0/2097180635 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2422 (secure 0 0 0) 0x7fbdbc06be90 con 0x7fbdc40d0030 2024-09-16T07:30:36.125 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:36.125 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:30:36.125 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:30:36.125 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:30:36.125 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:30:36.125 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:30:36.126 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 493, 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 79, 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:30:36.127 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 2296, 2024-09-16T07:30:36.129 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 9488866535, 2024-09-16T07:30:36.131 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 18617327616, 2024-09-16T07:30:36.131 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 754476785664, 2024-09-16T07:30:36.131 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:30:36.131 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_objects_per_sec": 3, 2024-09-16T07:30:36.131 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_bytes_per_sec": 13095093, 2024-09-16T07:30:36.131 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_keys_per_sec": 0, 2024-09-16T07:30:36.131 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects_recovered": 25, 2024-09-16T07:30:36.131 INFO:teuthology.orchestra.run.smithi071.stdout: "num_bytes_recovered": 104857600, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "num_keys_recovered": 0, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 804041836, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 782644785, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 194, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 212 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 47, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:29:54:247057+0000", 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:30:36.132 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 5, 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "c", 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 6417 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:30:36.133 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 12, 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:28:56.398876+0000", 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:30:36.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:36.135 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:30:36.135 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:30:36.135 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "4 ongoing clones - average progress is 39.258% (7s)\n [==========..................] (remaining: 11s)", 2024-09-16T07:30:36.135 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.39258000254631042, 2024-09-16T07:30:36.135 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:30:36.135 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:36.136 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:36.136 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.128+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda4079910 msgr2=0x7fbda407bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.128+0000 7fbdc8c00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda4079910 0x7fbda407bdd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4036000 tx=0x7fbdb40046c0 comp rx=0 tx=0).stop 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.128+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc40d0030 msgr2=0x7fbdc41f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.128+0000 7fbdc8c00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc40d0030 0x7fbdc41f8320 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4069770 tx=0x7fbdbc00d4a0 comp rx=0 tx=0).stop 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.131+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 shutdown_connections 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.131+0000 7fbdc8c00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbda4079910 0x7fbda407bdd0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.131+0000 7fbdc8c00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdc40d0030 0x7fbdc41f8320 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.131+0000 7fbdc8c00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdc40b0b40 0x7fbdc40cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.131+0000 7fbdc8c00640 1 --2- 172.21.15.71:0/2097180635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdc40b0760 0x7fbdc41f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.131+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 >> 172.21.15.71:0/2097180635 conn(0x7fbdc40b18e0 msgr2=0x7fbdc4074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.131+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 shutdown_connections 2024-09-16T07:30:36.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:36.131+0000 7fbdc8c00640 1 -- 172.21.15.71:0/2097180635 wait complete. 2024-09-16T07:30:37.165 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:30:37.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.451+0000 7f726b400640 1 -- 172.21.15.71:0/2964316792 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726402fe00 con 0x7f726c0d1260 2024-09-16T07:30:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.452+0000 7f7272200640 1 -- 172.21.15.71:0/2964316792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d1260 msgr2=0x7f726c0d4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.452+0000 7f7272200640 1 --2- 172.21.15.71:0/2964316792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d1260 0x7f726c0d4960 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7264005fd0 tx=0x7f726402aa00 comp rx=0 tx=0).stop 2024-09-16T07:30:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.452+0000 7f7272200640 1 -- 172.21.15.71:0/2964316792 shutdown_connections 2024-09-16T07:30:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.452+0000 7f7272200640 1 --2- 172.21.15.71:0/2964316792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f726c0d25d0 0x7f726c0cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.452+0000 7f7272200640 1 --2- 172.21.15.71:0/2964316792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d1260 0x7f726c0d4960 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.452+0000 7f7272200640 1 -- 172.21.15.71:0/2964316792 >> 172.21.15.71:0/2964316792 conn(0x7f726c0d3e40 msgr2=0x7f726c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.453+0000 7f7272200640 1 -- 172.21.15.71:0/2964316792 shutdown_connections 2024-09-16T07:30:37.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.453+0000 7f7272200640 1 -- 172.21.15.71:0/2964316792 wait complete. 2024-09-16T07:30:37.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.456+0000 7f7272200640 1 Processor -- start 2024-09-16T07:30:37.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.456+0000 7f7272200640 1 -- start start 2024-09-16T07:30:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f7272200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f726c0d1260 0x7f726c1f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f7272200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d25d0 0x7f726c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f7272200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f726c15e050 0x7f726c1ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:37.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f7272200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f726c0b7170 con 0x7f726c0d25d0 2024-09-16T07:30:37.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f7272200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f726c0b6ff0 con 0x7f726c15e050 2024-09-16T07:30:37.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f7272200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f726c0b72f0 con 0x7f726c0d1260 2024-09-16T07:30:37.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f726be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d25d0 0x7f726c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:37.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f726be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d25d0 0x7f726c0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58576/0 (socket says 172.21.15.71:58576) 2024-09-16T07:30:37.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.458+0000 7f726be00640 1 -- 172.21.15.71:0/3093431577 learned_addr learned my addr 172.21.15.71:0/3093431577 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:37.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.459+0000 7f726be00640 1 -- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f726c0d1260 msgr2=0x7f726c1f85a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:30:37.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.459+0000 7f726be00640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f726c0d1260 0x7f726c1f85a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:37.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.459+0000 7f726be00640 1 -- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f726c15e050 msgr2=0x7f726c1ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:37.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.459+0000 7f726be00640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f726c15e050 0x7f726c1ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:37.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.459+0000 7f726be00640 1 -- 172.21.15.71:0/3093431577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c1f8ae0 con 0x7f726c0d25d0 2024-09-16T07:30:37.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.461+0000 7f726be00640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d25d0 0x7f726c0ca660 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f725800d980 tx=0x7f725800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:37.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.462+0000 7f7268c00640 1 -- 172.21.15.71:0/3093431577 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258017070 con 0x7f726c0d25d0 2024-09-16T07:30:37.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.462+0000 7f7268c00640 1 -- 172.21.15.71:0/3093431577 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7258009d20 con 0x7f726c0d25d0 2024-09-16T07:30:37.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.462+0000 7f7268c00640 1 -- 172.21.15.71:0/3093431577 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72580106c0 con 0x7f726c0d25d0 2024-09-16T07:30:37.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.462+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726c0a05c0 con 0x7f726c0d25d0 2024-09-16T07:30:37.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.463+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726c0a0860 con 0x7f726c0d25d0 2024-09-16T07:30:37.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.465+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f726c0b7750 con 0x7f726c0d25d0 2024-09-16T07:30:37.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.466+0000 7f7268c00640 1 -- 172.21.15.71:0/3093431577 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7f7258010860 con 0x7f726c0d25d0 2024-09-16T07:30:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.467+0000 7f7268c00640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f724c079810 0x7f724c07bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.467+0000 7f7268c00640 1 -- 172.21.15.71:0/3093431577 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f7258099860 con 0x7f726c0d25d0 2024-09-16T07:30:37.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.475+0000 7f7268c00640 1 -- 172.21.15.71:0/3093431577 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f726c1f8ae0 con 0x7f726c0d25d0 2024-09-16T07:30:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.484+0000 7f7270e00640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f724c079810 0x7f724c07bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:37.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.514+0000 7f7270e00640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f724c079810 0x7f724c07bcd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f725c0095f0 tx=0x7f725c006b50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:37.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.977+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f726c0d0fe0 con 0x7f726c0d25d0 2024-09-16T07:30:37.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.979+0000 7f7268c00640 1 -- 172.21.15.71:0/3093431577 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2662 (secure 0 0 0) 0x7f7258063980 con 0x7f726c0d25d0 2024-09-16T07:30:37.980 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:30:37.981 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 494, 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:30:37.982 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 79, 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:30:37.983 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 2296, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 9488866535, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 18617327616, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 754476785664, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_objects_per_sec": 3, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_bytes_per_sec": 13092723, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_keys_per_sec": 0, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects_recovered": 25, 2024-09-16T07:30:37.984 INFO:teuthology.orchestra.run.smithi071.stdout: "num_bytes_recovered": 104857600, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "num_keys_recovered": 0, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 680823940, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 656248104, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 164, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 172 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 47, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:29:54:247057+0000", 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:30:37.985 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 5, 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "c", 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 6417 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:30:37.986 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 12, 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:28:56.398876+0000", 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:30:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "4 ongoing clones - average progress is 45.768% (11s)\n [============................] (remaining: 13s)", 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.45767998695373535, 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-total-clones": { 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Total 6 clones - average progress is 39.497% (0s)\n [===========.................] ", 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.39496999979019165, 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:37.988 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:30:37.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.984+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f724c079810 msgr2=0x7f724c07bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:37.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.984+0000 7f7272200640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f724c079810 0x7f724c07bcd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f725c0095f0 tx=0x7f725c006b50 comp rx=0 tx=0).stop 2024-09-16T07:30:37.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d25d0 msgr2=0x7f726c0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:37.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d25d0 0x7f726c0ca660 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f725800d980 tx=0x7f725800de40 comp rx=0 tx=0).stop 2024-09-16T07:30:37.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 shutdown_connections 2024-09-16T07:30:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f724c079810 0x7f724c07bcd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f726c15e050 0x7f726c1ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f726c0d25d0 0x7f726c0ca660 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 --2- 172.21.15.71:0/3093431577 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f726c0d1260 0x7f726c1f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 >> 172.21.15.71:0/3093431577 conn(0x7f726c0d3e40 msgr2=0x7f726c0d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 shutdown_connections 2024-09-16T07:30:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:37.985+0000 7f7272200640 1 -- 172.21.15.71:0/3093431577 wait complete. 2024-09-16T07:30:38.006 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone82630 2024-09-16T07:30:38.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.272+0000 7f1ae7a00640 1 -- 172.21.15.71:0/982031576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac4003a30 msgr2=0x7f1ac4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:38.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.272+0000 7f1ae7a00640 1 --2- 172.21.15.71:0/982031576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac4003a30 0x7f1ac4003e30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000c710 tx=0x7f1ac4005830 comp rx=0 tx=0).stop 2024-09-16T07:30:38.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.272+0000 7f1ae5a00640 1 -- 172.21.15.71:0/982031576 reap_dead start 2024-09-16T07:30:38.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.272+0000 7f1ae7a00640 1 -- 172.21.15.71:0/982031576 shutdown_connections 2024-09-16T07:30:38.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.272+0000 7f1ae7a00640 1 -- 172.21.15.71:0/982031576 >> 172.21.15.71:0/982031576 conn(0x7f1ae00b1840 msgr2=0x7f1ae00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:38.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.272+0000 7f1ae7a00640 1 -- 172.21.15.71:0/982031576 shutdown_connections 2024-09-16T07:30:38.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.273+0000 7f1ae7a00640 1 -- 172.21.15.71:0/982031576 wait complete. 2024-09-16T07:30:38.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.274+0000 7f1ae7a00640 1 Processor -- start 2024-09-16T07:30:38.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.275+0000 7f1ae7a00640 1 -- start start 2024-09-16T07:30:38.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.275+0000 7f1ae7a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ac4003a30 0x7f1ae01e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:38.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.275+0000 7f1ae7a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae0151930 0x7f1ae01e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:38.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.275+0000 7f1ae7a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ae01404c0 0x7f1ae0140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:38.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.275+0000 7f1ae7a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae00ce2c0 con 0x7f1ae0151930 2024-09-16T07:30:38.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.275+0000 7f1ae7a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae00736d0 con 0x7f1ac4003a30 2024-09-16T07:30:38.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.275+0000 7f1ae7a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae00ce440 con 0x7f1ae01404c0 2024-09-16T07:30:38.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.276+0000 7f1adfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae0151930 0x7f1ae01e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:38.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.276+0000 7f1adfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae0151930 0x7f1ae01e7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58596/0 (socket says 172.21.15.71:58596) 2024-09-16T07:30:38.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.276+0000 7f1adfe00640 1 -- 172.21.15.71:0/2362101030 learned_addr learned my addr 172.21.15.71:0/2362101030 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.276+0000 7f1adfe00640 1 -- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ae01404c0 msgr2=0x7f1ae0140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.276+0000 7f1adfe00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ae01404c0 0x7f1ae0140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.276+0000 7f1adfe00640 1 -- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ac4003a30 msgr2=0x7f1ae01e6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.276+0000 7f1adfe00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ac4003a30 0x7f1ae01e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.276+0000 7f1adfe00640 1 -- 172.21.15.71:0/2362101030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae01f8120 con 0x7f1ae0151930 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.277+0000 7f1adfe00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae0151930 0x7f1ae01e7370 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00e970 tx=0x7f1acc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.277+0000 7f1adcc00640 1 -- 172.21.15.71:0/2362101030 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00cc60 con 0x7f1ae0151930 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.277+0000 7f1adcc00640 1 -- 172.21.15.71:0/2362101030 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1acc00ce00 con 0x7f1ae0151930 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.277+0000 7f1adcc00640 1 -- 172.21.15.71:0/2362101030 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc005280 con 0x7f1ae0151930 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.277+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae01f83c0 con 0x7f1ae0151930 2024-09-16T07:30:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.277+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01f8980 con 0x7f1ae0151930 2024-09-16T07:30:38.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.279+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae00ce9d0 con 0x7f1ae0151930 2024-09-16T07:30:38.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.280+0000 7f1adcc00640 1 -- 172.21.15.71:0/2362101030 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7f1acc005420 con 0x7f1ae0151930 2024-09-16T07:30:38.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.281+0000 7f1adcc00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ac0079840 0x7f1ac007bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:38.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.281+0000 7f1adcc00640 1 -- 172.21.15.71:0/2362101030 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f1acc09a100 con 0x7f1ae0151930 2024-09-16T07:30:38.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.287+0000 7f1adcc00640 1 -- 172.21.15.71:0/2362101030 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1acc063f10 con 0x7f1ae0151930 2024-09-16T07:30:38.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.304+0000 7f1ae5000640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ac0079840 0x7f1ac007bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.321+0000 7f1ae5000640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ac0079840 0x7f1ac007bd00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0032000 tx=0x7f1ad00036f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:38.697+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone82630", "target": ["mon-mgr", ""]}) -- 0x7f1ae00b4970 con 0x7f1ac0079840 2024-09-16T07:30:38.741 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:38.742+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:38.741 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:38.742+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:38.742 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:38.742+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:38.742 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:38.742+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:38.742 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:38.742+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:40.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.159+0000 7f1adcc00640 1 -- 172.21.15.71:0/2362101030 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1ae00b4970 con 0x7f1ac0079840 2024-09-16T07:30:40.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ac0079840 msgr2=0x7f1ac007bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:40.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ac0079840 0x7f1ac007bd00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0032000 tx=0x7f1ad00036f0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae0151930 msgr2=0x7f1ae01e7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:40.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae0151930 0x7f1ae01e7370 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00e970 tx=0x7f1acc00ee30 comp rx=0 tx=0).stop 2024-09-16T07:30:40.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 shutdown_connections 2024-09-16T07:30:40.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ac0079840 0x7f1ac007bd00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ae01404c0 0x7f1ae0140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae0151930 0x7f1ae01e7370 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 --2- 172.21.15.71:0/2362101030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ac4003a30 0x7f1ae01e6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.164+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 >> 172.21.15.71:0/2362101030 conn(0x7f1ae00b1840 msgr2=0x7f1ae006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:40.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.165+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 shutdown_connections 2024-09-16T07:30:40.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.165+0000 7f1ae7a00640 1 -- 172.21.15.71:0/2362101030 wait complete. 2024-09-16T07:30:40.202 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone84781 2024-09-16T07:30:40.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.489+0000 7f5d9e000640 1 -- 172.21.15.71:0/398749899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d980cf830 msgr2=0x7f5d980b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.490+0000 7f5d9e000640 1 --2- 172.21.15.71:0/398749899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d980cf830 0x7f5d980b33b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009830 tx=0x7f5d8402a480 comp rx=0 tx=0).stop 2024-09-16T07:30:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.490+0000 7f5d9e000640 1 -- 172.21.15.71:0/398749899 shutdown_connections 2024-09-16T07:30:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.490+0000 7f5d9e000640 1 --2- 172.21.15.71:0/398749899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d980d2a00 0x7f5d980b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.490+0000 7f5d9e000640 1 --2- 172.21.15.71:0/398749899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d980cf830 0x7f5d980b33b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.490+0000 7f5d9e000640 1 -- 172.21.15.71:0/398749899 >> 172.21.15.71:0/398749899 conn(0x7f5d9809d0b0 msgr2=0x7f5d980d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.490+0000 7f5d9e000640 1 -- 172.21.15.71:0/398749899 shutdown_connections 2024-09-16T07:30:40.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.490+0000 7f5d9e000640 1 -- 172.21.15.71:0/398749899 wait complete. 2024-09-16T07:30:40.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.492+0000 7f5d9e000640 1 Processor -- start 2024-09-16T07:30:40.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.492+0000 7f5d9e000640 1 -- start start 2024-09-16T07:30:40.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.493+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d980cf830 0x7f5d981f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:40.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.493+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d980d2a00 0x7f5d980b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:40.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.493+0000 7f5d9e000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d98161460 0x7f5d980b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:40.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.493+0000 7f5d9e000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5d980b50c0 con 0x7f5d980cf830 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.493+0000 7f5d9e000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d980b4f40 con 0x7f5d980d2a00 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.493+0000 7f5d9e000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5d980b3e30 con 0x7f5d98161460 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d980cf830 0x7f5d981f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d980cf830 0x7f5d981f8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58610/0 (socket says 172.21.15.71:58610) 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9cc00640 1 -- 172.21.15.71:0/2429030242 learned_addr learned my addr 172.21.15.71:0/2429030242 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9d600640 1 --2- 172.21.15.71:0/2429030242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d98161460 0x7f5d980b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9cc00640 1 -- 172.21.15.71:0/2429030242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d98161460 msgr2=0x7f5d980b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9cc00640 1 --2- 172.21.15.71:0/2429030242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d98161460 0x7f5d980b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9cc00640 1 -- 172.21.15.71:0/2429030242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d980d2a00 msgr2=0x7f5d980b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9cc00640 1 --2- 172.21.15.71:0/2429030242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d980d2a00 0x7f5d980b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.494+0000 7f5d9cc00640 1 -- 172.21.15.71:0/2429030242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d980b81c0 con 0x7f5d980cf830 2024-09-16T07:30:40.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.495+0000 7f5d9cc00640 1 --2- 172.21.15.71:0/2429030242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d980cf830 0x7f5d981f8f00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c3c0 tx=0x7f5d8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:40.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.495+0000 7f5d94c00640 1 -- 172.21.15.71:0/2429030242 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c017070 con 0x7f5d980cf830 2024-09-16T07:30:40.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.495+0000 7f5d94c00640 1 -- 172.21.15.71:0/2429030242 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d8c0035f0 con 0x7f5d980cf830 2024-09-16T07:30:40.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.495+0000 7f5d94c00640 1 -- 172.21.15.71:0/2429030242 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c003a80 con 0x7f5d980cf830 2024-09-16T07:30:40.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.495+0000 7f5d9e000640 1 -- 172.21.15.71:0/2429030242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d980b84a0 con 0x7f5d980cf830 2024-09-16T07:30:40.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.495+0000 7f5d9e000640 1 -- 172.21.15.71:0/2429030242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d981f9e40 con 0x7f5d980cf830 2024-09-16T07:30:40.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.497+0000 7f5d9e000640 1 -- 172.21.15.71:0/2429030242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d980b42e0 con 0x7f5d980cf830 2024-09-16T07:30:40.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.500+0000 7f5d94c00640 1 -- 172.21.15.71:0/2429030242 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) ==== 102715+0+0 (secure 0 0 0) 0x7f5d8c01a070 con 0x7f5d980cf830 2024-09-16T07:30:40.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.501+0000 7f5d94c00640 1 --2- 172.21.15.71:0/2429030242 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d78079810 0x7f5d7807bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:40.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.501+0000 7f5d94c00640 1 -- 172.21.15.71:0/2429030242 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f5d8c098e70 con 0x7f5d980cf830 2024-09-16T07:30:40.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.502+0000 7f5d97e00640 1 --2- 172.21.15.71:0/2429030242 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d78079810 0x7f5d7807bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:40.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.503+0000 7f5d97e00640 1 --2- 172.21.15.71:0/2429030242 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d78079810 0x7f5d7807bcd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5d84034000 tx=0x7f5d840046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:40.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.504+0000 7f5d94c00640 1 -- 172.21.15.71:0/2429030242 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5d8c021020 con 0x7f5d980cf830 2024-09-16T07:30:40.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:40.959+0000 7f5d9e000640 1 -- 172.21.15.71:0/2429030242 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone84781", "target": ["mon-mgr", ""]}) -- 0x7f5d980b2f40 con 0x7f5d78079810 2024-09-16T07:30:41.142 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:41.141+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:30:41.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.147+0000 7f5d94c00640 1 -- 172.21.15.71:0/2429030242 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f5d980b2f40 con 0x7f5d78079810 2024-09-16T07:30:41.146 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:30:41.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:30:41.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone19152 2024-09-16T07:30:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.458+0000 7f47a3400640 1 -- 172.21.15.71:0/1018050755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f479c0cccd0 msgr2=0x7f479c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.458+0000 7f47a3400640 1 --2- 172.21.15.71:0/1018050755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f479c0cccd0 0x7f479c073440 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f479c0cfba0 tx=0x7f478802a4b0 comp rx=0 tx=0).stop 2024-09-16T07:30:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.458+0000 7f47a3400640 1 -- 172.21.15.71:0/1018050755 shutdown_connections 2024-09-16T07:30:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.458+0000 7f47a3400640 1 --2- 172.21.15.71:0/1018050755 >> v2:172.21.15.177:3300/0 conn(0x7f479c06efa0 0x7f479c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.458+0000 7f47a3400640 1 --2- 172.21.15.71:0/1018050755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f479c0cccd0 0x7f479c073440 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.458+0000 7f47a3400640 1 -- 172.21.15.71:0/1018050755 >> 172.21.15.71:0/1018050755 conn(0x7f479c0b1840 msgr2=0x7f479c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.458+0000 7f47a3400640 1 -- 172.21.15.71:0/1018050755 shutdown_connections 2024-09-16T07:30:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.458+0000 7f47a3400640 1 -- 172.21.15.71:0/1018050755 wait complete. 2024-09-16T07:30:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.460+0000 7f47a3400640 1 Processor -- start 2024-09-16T07:30:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.460+0000 7f47a3400640 1 -- start start 2024-09-16T07:30:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.462+0000 7f47a3400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f479c06efa0 0x7f479c13beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.462+0000 7f47a3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f479c0cccd0 0x7f479c140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.462+0000 7f47a3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c145750 0x7f479c1f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.462+0000 7f47a3400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f479c0ce480 con 0x7f479c0cccd0 2024-09-16T07:30:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.462+0000 7f47a3400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f479c0ce300 con 0x7f479c145750 2024-09-16T07:30:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.462+0000 7f47a3400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f479c0ce600 con 0x7f479c06efa0 2024-09-16T07:30:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.463+0000 7f479be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f479c0cccd0 0x7f479c140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.463+0000 7f479be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f479c0cccd0 0x7f479c140170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58620/0 (socket says 172.21.15.71:58620) 2024-09-16T07:30:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.463+0000 7f479be00640 1 -- 172.21.15.71:0/1975261151 learned_addr learned my addr 172.21.15.71:0/1975261151 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.463+0000 7f47a1400640 1 --2- 172.21.15.71:0/1975261151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c145750 0x7f479c1f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.464+0000 7f479be00640 1 -- 172.21.15.71:0/1975261151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f479c06efa0 msgr2=0x7f479c13beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.464+0000 7f479be00640 1 --2- 172.21.15.71:0/1975261151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f479c06efa0 0x7f479c13beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.464+0000 7f479be00640 1 -- 172.21.15.71:0/1975261151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c145750 msgr2=0x7f479c1f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.464+0000 7f479be00640 1 --2- 172.21.15.71:0/1975261151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c145750 0x7f479c1f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.464+0000 7f479be00640 1 -- 172.21.15.71:0/1975261151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c1406b0 con 0x7f479c0cccd0 2024-09-16T07:30:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.465+0000 7f479be00640 1 --2- 172.21.15.71:0/1975261151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f479c0cccd0 0x7f479c140170 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4788009780 tx=0x7f4788032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:41.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.466+0000 7f4798c00640 1 -- 172.21.15.71:0/1975261151 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788040070 con 0x7f479c0cccd0 2024-09-16T07:30:41.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.466+0000 7f4798c00640 1 -- 172.21.15.71:0/1975261151 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4788005310 con 0x7f479c0cccd0 2024-09-16T07:30:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.466+0000 7f4798c00640 1 -- 172.21.15.71:0/1975261151 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788003800 con 0x7f479c0cccd0 2024-09-16T07:30:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.467+0000 7f47a3400640 1 -- 172.21.15.71:0/1975261151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c151b60 con 0x7f479c0cccd0 2024-09-16T07:30:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.467+0000 7f47a3400640 1 -- 172.21.15.71:0/1975261151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c142ad0 con 0x7f479c0cccd0 2024-09-16T07:30:41.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.469+0000 7f47a3400640 1 -- 172.21.15.71:0/1975261151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c1428c0 con 0x7f479c0cccd0 2024-09-16T07:30:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.486+0000 7f4798c00640 1 -- 172.21.15.71:0/1975261151 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7f4788039040 con 0x7f479c0cccd0 2024-09-16T07:30:41.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.489+0000 7f4798c00640 1 --2- 172.21.15.71:0/1975261151 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f477c079970 0x7f477c07be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:41.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.489+0000 7f4798c00640 1 -- 172.21.15.71:0/1975261151 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f47880babb0 con 0x7f479c0cccd0 2024-09-16T07:30:41.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.494+0000 7f4798c00640 1 -- 172.21.15.71:0/1975261151 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4788084d90 con 0x7f479c0cccd0 2024-09-16T07:30:41.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.514+0000 7f47a0a00640 1 --2- 172.21.15.71:0/1975261151 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f477c079970 0x7f477c07be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:41.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.526+0000 7f47a0a00640 1 --2- 172.21.15.71:0/1975261151 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f477c079970 0x7f477c07be30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4790001500 tx=0x7f4790007a30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:41.960+0000 7f47a3400640 1 -- 172.21.15.71:0/1975261151 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone19152", "target": ["mon-mgr", ""]}) -- 0x7f479c0cdbb0 con 0x7f477c079970 2024-09-16T07:30:42.186 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:42.186+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:30:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.191+0000 7f4798c00640 1 -- 172.21.15.71:0/1975261151 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f479c0cdbb0 con 0x7f477c079970 2024-09-16T07:30:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:30:42.210 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:30:42.211 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone22153 2024-09-16T07:30:42.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.486+0000 7f36ca400640 1 -- 172.21.15.71:0/1183517443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c40af190 msgr2=0x7f36c40b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:42.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.486+0000 7f36ca400640 1 --2- 172.21.15.71:0/1183517443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c40af190 0x7f36c40b2b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009830 tx=0x7f36b002a510 comp rx=0 tx=0).stop 2024-09-16T07:30:42.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.486+0000 7f36ca400640 1 -- 172.21.15.71:0/1183517443 shutdown_connections 2024-09-16T07:30:42.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.486+0000 7f36ca400640 1 --2- 172.21.15.71:0/1183517443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c40af190 0x7f36c40b2b40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:42.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.486+0000 7f36ca400640 1 --2- 172.21.15.71:0/1183517443 >> v2:172.21.15.177:3300/0 conn(0x7f36c4161760 0x7f36c40ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:42.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.486+0000 7f36ca400640 1 -- 172.21.15.71:0/1183517443 >> 172.21.15.71:0/1183517443 conn(0x7f36c40b1840 msgr2=0x7f36c40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:42.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.486+0000 7f36ca400640 1 -- 172.21.15.71:0/1183517443 shutdown_connections 2024-09-16T07:30:42.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.486+0000 7f36ca400640 1 -- 172.21.15.71:0/1183517443 wait complete. 2024-09-16T07:30:42.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.488+0000 7f36ca400640 1 Processor -- start 2024-09-16T07:30:42.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.488+0000 7f36ca400640 1 -- start start 2024-09-16T07:30:42.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36ca400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c40af190 0x7f36c4155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:42.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36ca400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c40cec60 0x7f36c40afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:42.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36ca400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c4161760 0x7f36c40b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:42.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36ca400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4162a20 con 0x7f36c4161760 2024-09-16T07:30:42.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36ca400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41628a0 con 0x7f36c40af190 2024-09-16T07:30:42.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36ca400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36c4162ba0 con 0x7f36c40cec60 2024-09-16T07:30:42.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c4161760 0x7f36c40b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:42.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c4161760 0x7f36c40b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58622/0 (socket says 172.21.15.71:58622) 2024-09-16T07:30:42.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36c3e00640 1 -- 172.21.15.71:0/508842612 learned_addr learned my addr 172.21.15.71:0/508842612 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36c3e00640 1 -- 172.21.15.71:0/508842612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c40cec60 msgr2=0x7f36c40afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36c3e00640 1 --2- 172.21.15.71:0/508842612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c40cec60 0x7f36c40afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36c3e00640 1 -- 172.21.15.71:0/508842612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c40af190 msgr2=0x7f36c4155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36c3e00640 1 --2- 172.21.15.71:0/508842612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c40af190 0x7f36c4155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.489+0000 7f36c3e00640 1 -- 172.21.15.71:0/508842612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c41f4ae0 con 0x7f36c4161760 2024-09-16T07:30:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.490+0000 7f36c3e00640 1 --2- 172.21.15.71:0/508842612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c4161760 0x7f36c40b02a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00cbf0 tx=0x7f36bc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:42.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.490+0000 7f36bb400640 1 -- 172.21.15.71:0/508842612 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc016020 con 0x7f36c4161760 2024-09-16T07:30:42.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.490+0000 7f36bb400640 1 -- 172.21.15.71:0/508842612 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36bc00ae30 con 0x7f36c4161760 2024-09-16T07:30:42.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.490+0000 7f36bb400640 1 -- 172.21.15.71:0/508842612 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc012660 con 0x7f36c4161760 2024-09-16T07:30:42.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.490+0000 7f36ca400640 1 -- 172.21.15.71:0/508842612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c41f4da0 con 0x7f36c4161760 2024-09-16T07:30:42.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.490+0000 7f36ca400640 1 -- 172.21.15.71:0/508842612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41f5360 con 0x7f36c4161760 2024-09-16T07:30:42.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.492+0000 7f36ca400640 1 -- 172.21.15.71:0/508842612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c40cf270 con 0x7f36c4161760 2024-09-16T07:30:42.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.493+0000 7f36bb400640 1 -- 172.21.15.71:0/508842612 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7f36bc019020 con 0x7f36c4161760 2024-09-16T07:30:42.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.495+0000 7f36bb400640 1 --2- 172.21.15.71:0/508842612 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a40798a0 0x7f36a407bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:42.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.495+0000 7f36bb400640 1 -- 172.21.15.71:0/508842612 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f36bc099e30 con 0x7f36c4161760 2024-09-16T07:30:42.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.501+0000 7f36bb400640 1 -- 172.21.15.71:0/508842612 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f36bc020020 con 0x7f36c4161760 2024-09-16T07:30:42.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.544+0000 7f36c3400640 1 --2- 172.21.15.71:0/508842612 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a40798a0 0x7f36a407bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:42.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.563+0000 7f36c3400640 1 --2- 172.21.15.71:0/508842612 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a40798a0 0x7f36a407bd60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f36b40058c0 tx=0x7f36b4005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:42.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:42.882+0000 7f36ca400640 1 -- 172.21.15.71:0/508842612 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone22153", "target": ["mon-mgr", ""]}) -- 0x7f36c415b420 con 0x7f36a40798a0 2024-09-16T07:30:43.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.065+0000 7f36bb400640 1 -- 172.21.15.71:0/508842612 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f36c415b420 con 0x7f36a40798a0 2024-09-16T07:30:43.065 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:30:43.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:43.061+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:30:43.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:30:43.080 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone31984 2024-09-16T07:30:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.394+0000 7fd171600640 1 -- 172.21.15.71:0/2417499120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c0cc790 msgr2=0x7fd16c15e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.394+0000 7fd171600640 1 --2- 172.21.15.71:0/2417499120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c0cc790 0x7fd16c15e2c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd16c15f3c0 tx=0x7fd15c02a410 comp rx=0 tx=0).stop 2024-09-16T07:30:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.395+0000 7fd171600640 1 -- 172.21.15.71:0/2417499120 shutdown_connections 2024-09-16T07:30:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.395+0000 7fd171600640 1 --2- 172.21.15.71:0/2417499120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd16c15f460 0x7fd16c15f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.395+0000 7fd171600640 1 --2- 172.21.15.71:0/2417499120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c0cc790 0x7fd16c15e2c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.395+0000 7fd171600640 1 -- 172.21.15.71:0/2417499120 >> 172.21.15.71:0/2417499120 conn(0x7fd16c0d0b20 msgr2=0x7fd16c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.395+0000 7fd171600640 1 -- 172.21.15.71:0/2417499120 shutdown_connections 2024-09-16T07:30:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.395+0000 7fd171600640 1 -- 172.21.15.71:0/2417499120 wait complete. 2024-09-16T07:30:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.398+0000 7fd171600640 1 Processor -- start 2024-09-16T07:30:43.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.398+0000 7fd171600640 1 -- start start 2024-09-16T07:30:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.401+0000 7fd171600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c06cb70 0x7fd16c0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.401+0000 7fd171600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd16c0cc790 0x7fd16c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.401+0000 7fd16be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c06cb70 0x7fd16c0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.401+0000 7fd16be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c06cb70 0x7fd16c0c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58630/0 (socket says 172.21.15.71:58630) 2024-09-16T07:30:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.401+0000 7fd16be00640 1 -- 172.21.15.71:0/1166546825 learned_addr learned my addr 172.21.15.71:0/1166546825 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.401+0000 7fd171600640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd16c15f460 0x7fd16c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.402+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c162060 con 0x7fd16c06cb70 2024-09-16T07:30:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.402+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c161ee0 con 0x7fd16c0cc790 2024-09-16T07:30:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.402+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd16c1621e0 con 0x7fd16c15f460 2024-09-16T07:30:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.403+0000 7fd16b400640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd16c0cc790 0x7fd16c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.403+0000 7fd170c00640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd16c15f460 0x7fd16c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.404+0000 7fd16be00640 1 -- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd16c15f460 msgr2=0x7fd16c0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.404+0000 7fd16be00640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd16c15f460 0x7fd16c0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.404+0000 7fd16be00640 1 -- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd16c0cc790 msgr2=0x7fd16c0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.405+0000 7fd16be00640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd16c0cc790 0x7fd16c0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.405+0000 7fd16be00640 1 -- 172.21.15.71:0/1166546825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c0b62d0 con 0x7fd16c06cb70 2024-09-16T07:30:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.405+0000 7fd16be00640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c06cb70 0x7fd16c0c6230 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd15c000c00 tx=0x7fd15c032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.406+0000 7fd163e00640 1 -- 172.21.15.71:0/1166546825 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c040070 con 0x7fd16c06cb70 2024-09-16T07:30:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.406+0000 7fd163e00640 1 -- 172.21.15.71:0/1166546825 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd15c003af0 con 0x7fd16c06cb70 2024-09-16T07:30:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.406+0000 7fd163e00640 1 -- 172.21.15.71:0/1166546825 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c045460 con 0x7fd16c06cb70 2024-09-16T07:30:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.406+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c0b6820 con 0x7fd16c06cb70 2024-09-16T07:30:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.406+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c0b6540 con 0x7fd16c06cb70 2024-09-16T07:30:43.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.408+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16c0c9460 con 0x7fd16c06cb70 2024-09-16T07:30:43.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.410+0000 7fd163e00640 1 -- 172.21.15.71:0/1166546825 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7fd15c039040 con 0x7fd16c06cb70 2024-09-16T07:30:43.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.412+0000 7fd163e00640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd14c0798a0 0x7fd14c07bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:43.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.413+0000 7fd163e00640 1 -- 172.21.15.71:0/1166546825 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7fd15c0bac20 con 0x7fd16c06cb70 2024-09-16T07:30:43.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.419+0000 7fd163e00640 1 -- 172.21.15.71:0/1166546825 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd15c084ce0 con 0x7fd16c06cb70 2024-09-16T07:30:43.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.433+0000 7fd16b400640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd14c0798a0 0x7fd14c07bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.453+0000 7fd16b400640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd14c0798a0 0x7fd14c07bd60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd16c1505b0 tx=0x7fd158009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:43.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:43.843+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone31984", "target": ["mon-mgr", ""]}) -- 0x7fd16c06aaf0 con 0x7fd14c0798a0 2024-09-16T07:30:44.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.569+0000 7fd163e00640 1 -- 172.21.15.71:0/1166546825 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd16c06aaf0 con 0x7fd14c0798a0 2024-09-16T07:30:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd14c0798a0 msgr2=0x7fd14c07bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd14c0798a0 0x7fd14c07bd60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd16c1505b0 tx=0x7fd158009040 comp rx=0 tx=0).stop 2024-09-16T07:30:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c06cb70 msgr2=0x7fd16c0c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c06cb70 0x7fd16c0c6230 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd15c000c00 tx=0x7fd15c032cd0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 shutdown_connections 2024-09-16T07:30:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd14c0798a0 0x7fd14c07bd60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd16c15f460 0x7fd16c0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd16c0cc790 0x7fd16c0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 --2- 172.21.15.71:0/1166546825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd16c06cb70 0x7fd16c0c6230 secure :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd15c000c00 tx=0x7fd15c032cd0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.576+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 >> 172.21.15.71:0/1166546825 conn(0x7fd16c0d0b20 msgr2=0x7fd16c0735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.578+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 shutdown_connections 2024-09-16T07:30:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.578+0000 7fd171600640 1 -- 172.21.15.71:0/1166546825 wait complete. 2024-09-16T07:30:44.602 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone20415 2024-09-16T07:30:44.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.913+0000 7fc93fe00640 1 -- 172.21.15.71:0/2654415022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400d2560 msgr2=0x7fc940161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:44.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.913+0000 7fc93fe00640 1 --2- 172.21.15.71:0/2654415022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400d2560 0x7fc940161c60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc928009830 tx=0x7fc92802a4a0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.913+0000 7fc93fe00640 1 -- 172.21.15.71:0/2654415022 shutdown_connections 2024-09-16T07:30:44.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.913+0000 7fc93fe00640 1 --2- 172.21.15.71:0/2654415022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400d2560 0x7fc940161c60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.913+0000 7fc93fe00640 1 -- 172.21.15.71:0/2654415022 >> 172.21.15.71:0/2654415022 conn(0x7fc94009d0b0 msgr2=0x7fc9400d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:44.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.913+0000 7fc93fe00640 1 -- 172.21.15.71:0/2654415022 shutdown_connections 2024-09-16T07:30:44.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.913+0000 7fc93fe00640 1 -- 172.21.15.71:0/2654415022 wait complete. 2024-09-16T07:30:44.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.915+0000 7fc93fe00640 1 Processor -- start 2024-09-16T07:30:44.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.915+0000 7fc93fe00640 1 -- start start 2024-09-16T07:30:44.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b4f40 0x7fc9400ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400b5480 0x7fc9400cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc93fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9400d2560 0x7fc9400cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc93fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9400d2940 con 0x7fc9400b5480 2024-09-16T07:30:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc93fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc940162ac0 con 0x7fc9400b4f40 2024-09-16T07:30:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc93fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9400d2ac0 con 0x7fc9400d2560 2024-09-16T07:30:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc937e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400b5480 0x7fc9400cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc937e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400b5480 0x7fc9400cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35674/0 (socket says 172.21.15.71:35674) 2024-09-16T07:30:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.916+0000 7fc937e00640 1 -- 172.21.15.71:0/1440596985 learned_addr learned my addr 172.21.15.71:0/1440596985 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.917+0000 7fc93ea00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b4f40 0x7fc9400ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.917+0000 7fc937e00640 1 -- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9400d2560 msgr2=0x7fc9400cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.917+0000 7fc937e00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9400d2560 0x7fc9400cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.917+0000 7fc937e00640 1 -- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b4f40 msgr2=0x7fc9400ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.917+0000 7fc937e00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b4f40 0x7fc9400ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.917+0000 7fc937e00640 1 -- 172.21.15.71:0/1440596985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9400cb060 con 0x7fc9400b5480 2024-09-16T07:30:44.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.917+0000 7fc937e00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400b5480 0x7fc9400cd5c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc93000cad0 tx=0x7fc93000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:44.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.918+0000 7fc937400640 1 -- 172.21.15.71:0/1440596985 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930016020 con 0x7fc9400b5480 2024-09-16T07:30:44.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.918+0000 7fc937400640 1 -- 172.21.15.71:0/1440596985 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc93000a940 con 0x7fc9400b5480 2024-09-16T07:30:44.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.918+0000 7fc937400640 1 -- 172.21.15.71:0/1440596985 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93000ad90 con 0x7fc9400b5480 2024-09-16T07:30:44.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.918+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9400cb340 con 0x7fc9400b5480 2024-09-16T07:30:44.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.918+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9400cb900 con 0x7fc9400b5480 2024-09-16T07:30:44.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.920+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9400b3490 con 0x7fc9400b5480 2024-09-16T07:30:44.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.921+0000 7fc937400640 1 -- 172.21.15.71:0/1440596985 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7fc930019020 con 0x7fc9400b5480 2024-09-16T07:30:44.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.922+0000 7fc937400640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc91c0798d0 0x7fc91c07bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:44.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.922+0000 7fc937400640 1 -- 172.21.15.71:0/1440596985 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7fc930099670 con 0x7fc9400b5480 2024-09-16T07:30:44.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.927+0000 7fc937400640 1 -- 172.21.15.71:0/1440596985 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc930020030 con 0x7fc9400b5480 2024-09-16T07:30:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.952+0000 7fc93ea00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc91c0798d0 0x7fc91c07bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:44.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:44.966+0000 7fc93ea00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc91c0798d0 0x7fc91c07bd90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc928034000 tx=0x7fc928004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:45.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:30:45.117 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:30:45.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.372+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone20415", "target": ["mon-mgr", ""]}) -- 0x7fc940069d80 con 0x7fc91c0798d0 2024-09-16T07:30:45.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.827+0000 7fc937400640 1 -- 172.21.15.71:0/1440596985 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc928004340 con 0x7fc91c0798d0 2024-09-16T07:30:45.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.833+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc91c0798d0 msgr2=0x7fc91c07bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:45.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.833+0000 7fc93fe00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc91c0798d0 0x7fc91c07bd90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc928034000 tx=0x7fc928004730 comp rx=0 tx=0).stop 2024-09-16T07:30:45.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.833+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400b5480 msgr2=0x7fc9400cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:45.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.833+0000 7fc93fe00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400b5480 0x7fc9400cd5c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc93000cad0 tx=0x7fc93000cf90 comp rx=0 tx=0).stop 2024-09-16T07:30:45.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.834+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 shutdown_connections 2024-09-16T07:30:45.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.834+0000 7fc93fe00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc91c0798d0 0x7fc91c07bd90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:45.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.834+0000 7fc93fe00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9400d2560 0x7fc9400cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:45.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.834+0000 7fc93fe00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400b5480 0x7fc9400cd5c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:45.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.834+0000 7fc93fe00640 1 --2- 172.21.15.71:0/1440596985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b4f40 0x7fc9400ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:45.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.834+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 >> 172.21.15.71:0/1440596985 conn(0x7fc94009d0b0 msgr2=0x7fc94015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:45.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.837+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 shutdown_connections 2024-09-16T07:30:45.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:45.837+0000 7fc93fe00640 1 -- 172.21.15.71:0/1440596985 wait complete. 2024-09-16T07:30:45.873 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone52566 2024-09-16T07:30:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.118+0000 7f4e4fc00640 1 -- 172.21.15.71:0/434968202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cec60 msgr2=0x7f4e480cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.118+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/434968202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cec60 0x7f4e480cf0b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4e40005fd0 tx=0x7f4e4002a960 comp rx=0 tx=0).stop 2024-09-16T07:30:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.118+0000 7f4e4fc00640 1 -- 172.21.15.71:0/434968202 shutdown_connections 2024-09-16T07:30:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.118+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/434968202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e480af190 0x7f4e480b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.118+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/434968202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cec60 0x7f4e480cf0b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.118+0000 7f4e4fc00640 1 -- 172.21.15.71:0/434968202 >> 172.21.15.71:0/434968202 conn(0x7f4e480b1840 msgr2=0x7f4e480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:46.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.118+0000 7f4e4fc00640 1 -- 172.21.15.71:0/434968202 shutdown_connections 2024-09-16T07:30:46.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.118+0000 7f4e4fc00640 1 -- 172.21.15.71:0/434968202 wait complete. 2024-09-16T07:30:46.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.120+0000 7f4e4fc00640 1 Processor -- start 2024-09-16T07:30:46.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.120+0000 7f4e4fc00640 1 -- start start 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.121+0000 7f4e4fc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e480af190 0x7f4e48159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.121+0000 7f4e4fc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cec60 0x7f4e48154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.121+0000 7f4e4fc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e48161760 0x7f4e480c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.121+0000 7f4e4fc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48162310 con 0x7f4e48161760 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.121+0000 7f4e4fc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e48162190 con 0x7f4e480cec60 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.121+0000 7f4e4fc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4e48162490 con 0x7f4e480af190 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.121+0000 7f4e4dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e48161760 0x7f4e480c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.121+0000 7f4e4dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e48161760 0x7f4e480c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35696/0 (socket says 172.21.15.71:35696) 2024-09-16T07:30:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e4dc00640 1 -- 172.21.15.71:0/1419913986 learned_addr learned my addr 172.21.15.71:0/1419913986 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e4d200640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e480af190 0x7f4e48159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e3fe00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cec60 0x7f4e48154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e4dc00640 1 -- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e480af190 msgr2=0x7f4e48159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e4dc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e480af190 0x7f4e48159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e4dc00640 1 -- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cec60 msgr2=0x7f4e48154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e4dc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cec60 0x7f4e48154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e4dc00640 1 -- 172.21.15.71:0/1419913986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4815f3c0 con 0x7f4e48161760 2024-09-16T07:30:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e3fe00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cec60 0x7f4e48154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:30:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.122+0000 7f4e4dc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e48161760 0x7f4e480c4c70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4e40006010 tx=0x7f4e40031f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.123+0000 7f4e45600640 1 -- 172.21.15.71:0/1419913986 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40047070 con 0x7f4e48161760 2024-09-16T07:30:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.123+0000 7f4e45600640 1 -- 172.21.15.71:0/1419913986 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e40004540 con 0x7f4e48161760 2024-09-16T07:30:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.123+0000 7f4e45600640 1 -- 172.21.15.71:0/1419913986 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40034070 con 0x7f4e48161760 2024-09-16T07:30:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.123+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4815a6f0 con 0x7f4e48161760 2024-09-16T07:30:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.124+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4815a370 con 0x7f4e48161760 2024-09-16T07:30:46.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.125+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e481629a0 con 0x7f4e48161760 2024-09-16T07:30:46.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.126+0000 7f4e45600640 1 -- 172.21.15.71:0/1419913986 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7f4e4004a070 con 0x7f4e48161760 2024-09-16T07:30:46.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.128+0000 7f4e45600640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e240798f0 0x7f4e2407bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:46.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.128+0000 7f4e45600640 1 -- 172.21.15.71:0/1419913986 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f4e400ba430 con 0x7f4e48161760 2024-09-16T07:30:46.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.135+0000 7f4e45600640 1 -- 172.21.15.71:0/1419913986 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e4003f070 con 0x7f4e48161760 2024-09-16T07:30:46.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.136+0000 7f4e4d200640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e240798f0 0x7f4e2407bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:46.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.144+0000 7f4e4d200640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e240798f0 0x7f4e2407bdb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4e38006fd0 tx=0x7f4e38009060 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.541+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone52566", "target": ["mon-mgr", ""]}) -- 0x7f4e480cdc00 con 0x7f4e240798f0 2024-09-16T07:30:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.663+0000 7f4e45600640 1 -- 172.21.15.71:0/1419913986 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e480cdc00 con 0x7f4e240798f0 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e240798f0 msgr2=0x7f4e2407bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e240798f0 0x7f4e2407bdb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4e38006fd0 tx=0x7f4e38009060 comp rx=0 tx=0).stop 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e48161760 msgr2=0x7f4e480c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e48161760 0x7f4e480c4c70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4e40006010 tx=0x7f4e40031f00 comp rx=0 tx=0).stop 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 shutdown_connections 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e240798f0 0x7f4e2407bdb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e48161760 0x7f4e480c4c70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cec60 0x7f4e48154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 --2- 172.21.15.71:0/1419913986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e480af190 0x7f4e48159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 >> 172.21.15.71:0/1419913986 conn(0x7f4e480b1840 msgr2=0x7f4e48072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 shutdown_connections 2024-09-16T07:30:46.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.666+0000 7f4e4fc00640 1 -- 172.21.15.71:0/1419913986 wait complete. 2024-09-16T07:30:46.682 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:30:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 -- 172.21.15.71:0/3492827453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 msgr2=0x7fe6f80b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3492827453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 0x7fe6f80b28f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8070640 tx=0x7fe6ec02a560 comp rx=0 tx=0).stop 2024-09-16T07:30:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 -- 172.21.15.71:0/3492827453 shutdown_connections 2024-09-16T07:30:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3492827453 >> v2:172.21.15.177:3300/0 conn(0x7fe6f80d0540 0x7fe6f8074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3492827453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6f80725d0 0x7fe6f8073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3492827453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 0x7fe6f80b28f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 -- 172.21.15.71:0/3492827453 >> 172.21.15.71:0/3492827453 conn(0x7fe6f80cfa20 msgr2=0x7fe6f80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 -- 172.21.15.71:0/3492827453 shutdown_connections 2024-09-16T07:30:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.840+0000 7fe6fee00640 1 -- 172.21.15.71:0/3492827453 wait complete. 2024-09-16T07:30:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.842+0000 7fe6fee00640 1 Processor -- start 2024-09-16T07:30:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.842+0000 7fe6fee00640 1 -- start start 2024-09-16T07:30:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6f80d0540 0x7fe6f80c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 0x7fe6f815e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fee00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6f815e5d0 0x7fe6f80c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f806f320 con 0x7fe6f815bc30 2024-09-16T07:30:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fee00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f806f4a0 con 0x7fe6f80d0540 2024-09-16T07:30:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fee00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f806f620 con 0x7fe6f815e5d0 2024-09-16T07:30:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fd000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 0x7fe6f815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fd000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 0x7fe6f815e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35710/0 (socket says 172.21.15.71:35710) 2024-09-16T07:30:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fd000640 1 -- 172.21.15.71:0/3187964468 learned_addr learned my addr 172.21.15.71:0/3187964468 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fe400640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6f815e5d0 0x7fe6f80c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.843+0000 7fe6fda00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6f80d0540 0x7fe6f80c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6fd000640 1 -- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6f815e5d0 msgr2=0x7fe6f80c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6fd000640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6f815e5d0 0x7fe6f80c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6fd000640 1 -- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6f80d0540 msgr2=0x7fe6f80c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6fd000640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6f80d0540 0x7fe6f80c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6fd000640 1 -- 172.21.15.71:0/3187964468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f81f28f0 con 0x7fe6f815bc30 2024-09-16T07:30:46.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6fd000640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 0x7fe6f815e090 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400d750 tx=0x7fe6e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:46.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6f5600640 1 -- 172.21.15.71:0/3187964468 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4017070 con 0x7fe6f815bc30 2024-09-16T07:30:46.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6f5600640 1 -- 172.21.15.71:0/3187964468 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6e4009cc0 con 0x7fe6f815bc30 2024-09-16T07:30:46.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6f5600640 1 -- 172.21.15.71:0/3187964468 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400de60 con 0x7fe6f815bc30 2024-09-16T07:30:46.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f81f2bd0 con 0x7fe6f815bc30 2024-09-16T07:30:46.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.844+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f81f25a0 con 0x7fe6f815bc30 2024-09-16T07:30:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.846+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f80b28f0 con 0x7fe6f815bc30 2024-09-16T07:30:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.847+0000 7fe6f5600640 1 -- 172.21.15.71:0/3187964468 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7fe6e4005280 con 0x7fe6f815bc30 2024-09-16T07:30:46.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.848+0000 7fe6f5600640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe6d80798a0 0x7fe6d807bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:46.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.848+0000 7fe6f5600640 1 -- 172.21.15.71:0/3187964468 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7fe6e4099830 con 0x7fe6f815bc30 2024-09-16T07:30:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.848+0000 7fe6fda00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe6d80798a0 0x7fe6d807bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.849+0000 7fe6fda00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe6d80798a0 0x7fe6d807bd60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec033000 tx=0x7fe6ec004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:46.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:46.852+0000 7fe6f5600640 1 -- 172.21.15.71:0/3187964468 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe6e4062f80 con 0x7fe6f815bc30 2024-09-16T07:30:47.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.060+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe6f80cfea0 con 0x7fe6d80798a0 2024-09-16T07:30:47.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.062+0000 7fe6f5600640 1 -- 172.21.15.71:0/3187964468 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe6f80cfea0 con 0x7fe6d80798a0 2024-09-16T07:30:47.062 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:30:47.062 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.062 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:30:47.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:47.063 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:30:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.063+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe6d80798a0 msgr2=0x7fe6d807bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.063+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe6d80798a0 0x7fe6d807bd60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec033000 tx=0x7fe6ec004730 comp rx=0 tx=0).stop 2024-09-16T07:30:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.063+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 msgr2=0x7fe6f815e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.063+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 0x7fe6f815e090 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400d750 tx=0x7fe6e400dc10 comp rx=0 tx=0).stop 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.064+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 shutdown_connections 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.064+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe6d80798a0 0x7fe6d807bd60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.064+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6f815e5d0 0x7fe6f80c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.064+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f815bc30 0x7fe6f815e090 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.064+0000 7fe6fee00640 1 --2- 172.21.15.71:0/3187964468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6f80d0540 0x7fe6f80c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.064+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 >> 172.21.15.71:0/3187964468 conn(0x7fe6f80cfa20 msgr2=0x7fe6f80b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.064+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 shutdown_connections 2024-09-16T07:30:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.064+0000 7fe6fee00640 1 -- 172.21.15.71:0/3187964468 wait complete. 2024-09-16T07:30:47.081 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:30:47.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.241+0000 7f294a600640 1 -- 172.21.15.71:0/2120252291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f294406efa0 msgr2=0x7f29440cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.241+0000 7f294a600640 1 --2- 172.21.15.71:0/2120252291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f294406efa0 0x7f29440cc5c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2930005fd0 tx=0x7f293002a540 comp rx=0 tx=0).stop 2024-09-16T07:30:47.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.241+0000 7f294a600640 1 -- 172.21.15.71:0/2120252291 shutdown_connections 2024-09-16T07:30:47.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.241+0000 7f294a600640 1 --2- 172.21.15.71:0/2120252291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f294406efa0 0x7f29440cc5c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.241+0000 7f294a600640 1 --2- 172.21.15.71:0/2120252291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2944145750 0x7f2944073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.241+0000 7f294a600640 1 -- 172.21.15.71:0/2120252291 >> 172.21.15.71:0/2120252291 conn(0x7f29440b1840 msgr2=0x7f29440b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:47.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.241+0000 7f294a600640 1 -- 172.21.15.71:0/2120252291 shutdown_connections 2024-09-16T07:30:47.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.241+0000 7f294a600640 1 -- 172.21.15.71:0/2120252291 wait complete. 2024-09-16T07:30:47.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.243+0000 7f294a600640 1 Processor -- start 2024-09-16T07:30:47.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.243+0000 7f294a600640 1 -- start start 2024-09-16T07:30:47.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f294a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f294406efa0 0x7f29440b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:47.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f294a600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29440ccb00 0x7f294413be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:47.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f294a600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2944145750 0x7f294413c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:47.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f294a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29440b26b0 con 0x7f294406efa0 2024-09-16T07:30:47.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f294a600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f29440cedd0 con 0x7f2944145750 2024-09-16T07:30:47.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f294a600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2944160100 con 0x7f29440ccb00 2024-09-16T07:30:47.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f2943400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f294406efa0 0x7f29440b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:47.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f2943400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f294406efa0 0x7f29440b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35712/0 (socket says 172.21.15.71:35712) 2024-09-16T07:30:47.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f2943400640 1 -- 172.21.15.71:0/2931788338 learned_addr learned my addr 172.21.15.71:0/2931788338 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:47.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f2942a00640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29440ccb00 0x7f294413be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:47.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.244+0000 7f2943e00640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2944145750 0x7f294413c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:47.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f2942a00640 1 -- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2944145750 msgr2=0x7f294413c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f2942a00640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2944145750 0x7f294413c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f2942a00640 1 -- 172.21.15.71:0/2931788338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f294406efa0 msgr2=0x7f29440b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f2942a00640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f294406efa0 0x7f29440b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f2942a00640 1 -- 172.21.15.71:0/2931788338 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29440cbb20 con 0x7f29440ccb00 2024-09-16T07:30:47.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f2943e00640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2944145750 0x7f294413c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:30:47.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f2943400640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f294406efa0 0x7f29440b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:30:47.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f2942a00640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29440ccb00 0x7f294413be60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2930031000 tx=0x7f29300038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:47.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f293b400640 1 -- 172.21.15.71:0/2931788338 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293003f070 con 0x7f29440ccb00 2024-09-16T07:30:47.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f293b400640 1 -- 172.21.15.71:0/2931788338 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29300047a0 con 0x7f29440ccb00 2024-09-16T07:30:47.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294415ec80 con 0x7f29440ccb00 2024-09-16T07:30:47.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.245+0000 7f293b400640 1 -- 172.21.15.71:0/2931788338 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930035030 con 0x7f29440ccb00 2024-09-16T07:30:47.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.246+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29440cc070 con 0x7f29440ccb00 2024-09-16T07:30:47.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.246+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2944073390 con 0x7f29440ccb00 2024-09-16T07:30:47.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.248+0000 7f293b400640 1 -- 172.21.15.71:0/2931788338 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7f2930042070 con 0x7f29440ccb00 2024-09-16T07:30:47.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.249+0000 7f293b400640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f29240798a0 0x7f292407bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:47.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.250+0000 7f293b400640 1 -- 172.21.15.71:0/2931788338 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f29300ba6f0 con 0x7f29440ccb00 2024-09-16T07:30:47.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.250+0000 7f2943400640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f29240798a0 0x7f292407bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:47.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.251+0000 7f2943400640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f29240798a0 0x7f292407bd60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2934011440 tx=0x7f2934005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:47.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.252+0000 7f293b400640 1 -- 172.21.15.71:0/2931788338 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2944073390 con 0x7f29440ccb00 2024-09-16T07:30:47.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.465+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29440af3a0 con 0x7f29240798a0 2024-09-16T07:30:47.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.473+0000 7f293b400640 1 -- 172.21.15.71:0/2931788338 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f29440af3a0 con 0x7f29240798a0 2024-09-16T07:30:47.473 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:47.473 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:30:47.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone84781" 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sv1" 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone20415" 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone22153" 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone31984" 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone82630" 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone52566" 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone19152" 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:47.475 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:30:47.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f29240798a0 msgr2=0x7f292407bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f29240798a0 0x7f292407bd60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2934011440 tx=0x7f2934005f90 comp rx=0 tx=0).stop 2024-09-16T07:30:47.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29440ccb00 msgr2=0x7f294413be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29440ccb00 0x7f294413be60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2930031000 tx=0x7f29300038a0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 shutdown_connections 2024-09-16T07:30:47.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f29240798a0 0x7f292407bd60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2944145750 0x7f294413c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29440ccb00 0x7f294413be60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 --2- 172.21.15.71:0/2931788338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f294406efa0 0x7f29440b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 >> 172.21.15.71:0/2931788338 conn(0x7f29440b1840 msgr2=0x7f29440ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:47.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 shutdown_connections 2024-09-16T07:30:47.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.475+0000 7f294a600640 1 -- 172.21.15.71:0/2931788338 wait complete. 2024-09-16T07:30:47.492 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone84781 2024-09-16T07:30:47.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.652+0000 7fc21b000640 1 -- 172.21.15.71:0/1224562078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc214152e70 msgr2=0x7fc21415cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.652+0000 7fc21b000640 1 --2- 172.21.15.71:0/1224562078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc214152e70 0x7fc21415cb50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc204005fd0 tx=0x7fc20402aac0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.652+0000 7fc21b000640 1 -- 172.21.15.71:0/1224562078 shutdown_connections 2024-09-16T07:30:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.652+0000 7fc21b000640 1 --2- 172.21.15.71:0/1224562078 >> v2:172.21.15.177:3300/0 conn(0x7fc21406cb70 0x7fc21406d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.652+0000 7fc21b000640 1 --2- 172.21.15.71:0/1224562078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc214152e70 0x7fc21415cb50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.652+0000 7fc21b000640 1 -- 172.21.15.71:0/1224562078 >> 172.21.15.71:0/1224562078 conn(0x7fc21406dc80 msgr2=0x7fc21406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.652+0000 7fc21b000640 1 -- 172.21.15.71:0/1224562078 shutdown_connections 2024-09-16T07:30:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.652+0000 7fc21b000640 1 -- 172.21.15.71:0/1224562078 wait complete. 2024-09-16T07:30:47.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.653+0000 7fc21b000640 1 Processor -- start 2024-09-16T07:30:47.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.654+0000 7fc21b000640 1 -- start start 2024-09-16T07:30:47.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.654+0000 7fc21b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc21406cb70 0x7fc2141f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:47.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.654+0000 7fc21b000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc214152e70 0x7fc2140b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:47.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.654+0000 7fc21b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc214070ba0 0x7fc2141f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:47.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.654+0000 7fc21b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc2140af9c0 con 0x7fc21406cb70 2024-09-16T07:30:47.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.654+0000 7fc21b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc2140af840 con 0x7fc214070ba0 2024-09-16T07:30:47.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.654+0000 7fc21b000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc2140afb40 con 0x7fc214152e70 2024-09-16T07:30:47.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc21406cb70 0x7fc2141f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:47.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc21406cb70 0x7fc2141f7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35726/0 (socket says 172.21.15.71:35726) 2024-09-16T07:30:47.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 -- 172.21.15.71:0/3639043135 learned_addr learned my addr 172.21.15.71:0/3639043135 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:47.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213400640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc214152e70 0x7fc2140b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:47.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc219000640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc214070ba0 0x7fc2141f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:47.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 -- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc214152e70 msgr2=0x7fc2140b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc214152e70 0x7fc2140b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 -- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc214070ba0 msgr2=0x7fc2141f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc214070ba0 0x7fc2141f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 -- 172.21.15.71:0/3639043135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2141f83d0 con 0x7fc21406cb70 2024-09-16T07:30:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.655+0000 7fc213e00640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc21406cb70 0x7fc2141f7d60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc204006010 tx=0x7fc204003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.656+0000 7fc20be00640 1 -- 172.21.15.71:0/3639043135 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204044070 con 0x7fc21406cb70 2024-09-16T07:30:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.656+0000 7fc20be00640 1 -- 172.21.15.71:0/3639043135 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc204003e00 con 0x7fc21406cb70 2024-09-16T07:30:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.656+0000 7fc20be00640 1 -- 172.21.15.71:0/3639043135 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2040044b0 con 0x7fc21406cb70 2024-09-16T07:30:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.656+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2141f86b0 con 0x7fc21406cb70 2024-09-16T07:30:47.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.656+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2141f4980 con 0x7fc21406cb70 2024-09-16T07:30:47.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.657+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2141f4770 con 0x7fc21406cb70 2024-09-16T07:30:47.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.658+0000 7fc20be00640 1 -- 172.21.15.71:0/3639043135 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7fc204004650 con 0x7fc21406cb70 2024-09-16T07:30:47.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.659+0000 7fc20be00640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1f40798a0 0x7fc1f407bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:47.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.659+0000 7fc20be00640 1 -- 172.21.15.71:0/3639043135 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7fc2040b9c60 con 0x7fc21406cb70 2024-09-16T07:30:47.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.660+0000 7fc213400640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1f40798a0 0x7fc1f407bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:47.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.660+0000 7fc213400640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1f40798a0 0x7fc1f407bd60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc200007fd0 tx=0x7fc200009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:47.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.663+0000 7fc20be00640 1 -- 172.21.15.71:0/3639043135 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc204032030 con 0x7fc21406cb70 2024-09-16T07:30:47.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.875+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone84781", "target": ["mon-mgr", ""]}) -- 0x7fc2140afff0 con 0x7fc1f40798a0 2024-09-16T07:30:47.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.899+0000 7fc20be00640 1 -- 172.21.15.71:0/3639043135 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc2140afff0 con 0x7fc1f40798a0 2024-09-16T07:30:47.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1f40798a0 msgr2=0x7fc1f407bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1f40798a0 0x7fc1f407bd60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc200007fd0 tx=0x7fc200009040 comp rx=0 tx=0).stop 2024-09-16T07:30:47.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc21406cb70 msgr2=0x7fc2141f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:47.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc21406cb70 0x7fc2141f7d60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc204006010 tx=0x7fc204003c10 comp rx=0 tx=0).stop 2024-09-16T07:30:47.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 shutdown_connections 2024-09-16T07:30:47.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1f40798a0 0x7fc1f407bd60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc214070ba0 0x7fc2141f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc214152e70 0x7fc2140b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 --2- 172.21.15.71:0/3639043135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc21406cb70 0x7fc2141f7d60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:47.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 >> 172.21.15.71:0/3639043135 conn(0x7fc21406dc80 msgr2=0x7fc21406e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:47.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 shutdown_connections 2024-09-16T07:30:47.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:47.902+0000 7fc21b000640 1 -- 172.21.15.71:0/3639043135 wait complete. 2024-09-16T07:30:47.912 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:47.912+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:47.913 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:47.912+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:47.913 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:47.912+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:47.913 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:47.912+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:47.913 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:30:47.912+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:30:47.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T07:30:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.084+0000 7fc77be00640 1 -- 172.21.15.71:0/3372739255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc77415f9a0 msgr2=0x7fc77415fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.084+0000 7fc77be00640 1 --2- 172.21.15.71:0/3372739255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc77415f9a0 0x7fc77415fd80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc768009830 tx=0x7fc76802a480 comp rx=0 tx=0).stop 2024-09-16T07:30:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.084+0000 7fc77be00640 1 -- 172.21.15.71:0/3372739255 shutdown_connections 2024-09-16T07:30:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.084+0000 7fc77be00640 1 --2- 172.21.15.71:0/3372739255 >> v2:172.21.15.177:3300/0 conn(0x7fc774073de0 0x7fc774074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.084+0000 7fc77be00640 1 --2- 172.21.15.71:0/3372739255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc77415f9a0 0x7fc77415fd80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.084+0000 7fc77be00640 1 -- 172.21.15.71:0/3372739255 >> 172.21.15.71:0/3372739255 conn(0x7fc7740b18e0 msgr2=0x7fc7740b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:48.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.084+0000 7fc77be00640 1 -- 172.21.15.71:0/3372739255 shutdown_connections 2024-09-16T07:30:48.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.084+0000 7fc77be00640 1 -- 172.21.15.71:0/3372739255 wait complete. 2024-09-16T07:30:48.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.086+0000 7fc77be00640 1 Processor -- start 2024-09-16T07:30:48.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.086+0000 7fc77be00640 1 -- start start 2024-09-16T07:30:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc77be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc774073de0 0x7fc7740ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc77be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc77415c3f0 0x7fc77414b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc77be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc77415f9a0 0x7fc7740cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc77be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7740cd3d0 con 0x7fc77415c3f0 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc77be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7740ccbd0 con 0x7fc77415f9a0 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc77be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc7740cd550 con 0x7fc774073de0 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc778a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc77415c3f0 0x7fc77414b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc778a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc77415c3f0 0x7fc77414b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35740/0 (socket says 172.21.15.71:35740) 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc778a00640 1 -- 172.21.15.71:0/4124198117 learned_addr learned my addr 172.21.15.71:0/4124198117 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc779400640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc774073de0 0x7fc7740ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc779e00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc77415f9a0 0x7fc7740cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc778a00640 1 -- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc774073de0 msgr2=0x7fc7740ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc778a00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc774073de0 0x7fc7740ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc778a00640 1 -- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc77415f9a0 msgr2=0x7fc7740cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc778a00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc77415f9a0 0x7fc7740cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc778a00640 1 -- 172.21.15.71:0/4124198117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc774152cd0 con 0x7fc77415c3f0 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.087+0000 7fc779e00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc77415f9a0 0x7fc7740cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.088+0000 7fc778a00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc77415c3f0 0x7fc77414b5f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc76000e9f0 tx=0x7fc76000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.088+0000 7fc771600640 1 -- 172.21.15.71:0/4124198117 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76000cde0 con 0x7fc77415c3f0 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.088+0000 7fc771600640 1 -- 172.21.15.71:0/4124198117 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc760010520 con 0x7fc77415c3f0 2024-09-16T07:30:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.088+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7740cdb70 con 0x7fc77415c3f0 2024-09-16T07:30:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.088+0000 7fc771600640 1 -- 172.21.15.71:0/4124198117 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760010970 con 0x7fc77415c3f0 2024-09-16T07:30:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.088+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7740ce130 con 0x7fc77415c3f0 2024-09-16T07:30:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.089+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc774069360 con 0x7fc77415c3f0 2024-09-16T07:30:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.090+0000 7fc771600640 1 -- 172.21.15.71:0/4124198117 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7fc76001f020 con 0x7fc77415c3f0 2024-09-16T07:30:48.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.092+0000 7fc771600640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc7540798a0 0x7fc75407bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.092+0000 7fc771600640 1 -- 172.21.15.71:0/4124198117 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7fc760099f80 con 0x7fc77415c3f0 2024-09-16T07:30:48.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.092+0000 7fc779400640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc7540798a0 0x7fc75407bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.093+0000 7fc779400640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc7540798a0 0x7fc75407bd60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc768034000 tx=0x7fc768004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:48.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.096+0000 7fc771600640 1 -- 172.21.15.71:0/4124198117 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc760063d20 con 0x7fc77415c3f0 2024-09-16T07:30:48.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.316+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7740cb740 con 0x7fc7540798a0 2024-09-16T07:30:48.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.330+0000 7fc771600640 1 -- 172.21.15.71:0/4124198117 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc7740cb740 con 0x7fc7540798a0 2024-09-16T07:30:48.331 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:48.331 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:30:48.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:30:48.331 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1" 2024-09-16T07:30:48.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:30:48.331 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:30:48.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc7540798a0 msgr2=0x7fc75407bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc7540798a0 0x7fc75407bd60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc768034000 tx=0x7fc768004730 comp rx=0 tx=0).stop 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc77415c3f0 msgr2=0x7fc77414b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc77415c3f0 0x7fc77414b5f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc76000e9f0 tx=0x7fc76000eeb0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 shutdown_connections 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc7540798a0 0x7fc75407bd60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc77415f9a0 0x7fc7740cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc77415c3f0 0x7fc77414b5f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 --2- 172.21.15.71:0/4124198117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc774073de0 0x7fc7740ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 >> 172.21.15.71:0/4124198117 conn(0x7fc7740b18e0 msgr2=0x7fc7740735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 shutdown_connections 2024-09-16T07:30:48.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.333+0000 7fc77be00640 1 -- 172.21.15.71:0/4124198117 wait complete. 2024-09-16T07:30:48.349 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T07:30:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.513+0000 7f26ed000640 1 -- 172.21.15.71:0/1200878277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e80c5020 msgr2=0x7f26e80ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.513+0000 7f26ed000640 1 --2- 172.21.15.71:0/1200878277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e80c5020 0x7f26e80ad870 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f26e8069450 tx=0x7f26e002aaa0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.513+0000 7f26ed000640 1 -- 172.21.15.71:0/1200878277 shutdown_connections 2024-09-16T07:30:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.513+0000 7f26ed000640 1 --2- 172.21.15.71:0/1200878277 >> v2:172.21.15.177:3300/0 conn(0x7f26e80cd0a0 0x7f26e80c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.513+0000 7f26ed000640 1 --2- 172.21.15.71:0/1200878277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e80c5020 0x7f26e80ad870 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.513+0000 7f26ed000640 1 -- 172.21.15.71:0/1200878277 >> 172.21.15.71:0/1200878277 conn(0x7f26e80cc490 msgr2=0x7f26e80cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.513+0000 7f26ed000640 1 -- 172.21.15.71:0/1200878277 shutdown_connections 2024-09-16T07:30:48.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.513+0000 7f26ed000640 1 -- 172.21.15.71:0/1200878277 wait complete. 2024-09-16T07:30:48.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.515+0000 7f26ed000640 1 Processor -- start 2024-09-16T07:30:48.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.515+0000 7f26ed000640 1 -- start start 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26ed000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e80c5020 0x7f26e80b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26ed000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e8147740 0x7f26e8131040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26ed000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e8131580 0x7f26e80b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26ed000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80ae1d0 con 0x7f26e8147740 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26ed000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80ae050 con 0x7f26e80c5020 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26ed000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26e80ae350 con 0x7f26e8131580 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e8147740 0x7f26e8131040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e8147740 0x7f26e8131040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35746/0 (socket says 172.21.15.71:35746) 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26e6a00640 1 -- 172.21.15.71:0/385691560 learned_addr learned my addr 172.21.15.71:0/385691560 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26e7400640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e80c5020 0x7f26e80b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.516+0000 7f26e7e00640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e8131580 0x7f26e80b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26e6a00640 1 -- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e8131580 msgr2=0x7f26e80b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26e6a00640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e8131580 0x7f26e80b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26e6a00640 1 -- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e80c5020 msgr2=0x7f26e80b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26e6a00640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e80c5020 0x7f26e80b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26e6a00640 1 -- 172.21.15.71:0/385691560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e8143de0 con 0x7f26e8147740 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26e6a00640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e8147740 0x7f26e8131040 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f26d400c920 tx=0x7f26d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26df400640 1 -- 172.21.15.71:0/385691560 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4007c70 con 0x7f26e8147740 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26df400640 1 -- 172.21.15.71:0/385691560 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26d400f450 con 0x7f26e8147740 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81440c0 con 0x7f26e8147740 2024-09-16T07:30:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26df400640 1 -- 172.21.15.71:0/385691560 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d400f8a0 con 0x7f26e8147740 2024-09-16T07:30:48.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.517+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81443a0 con 0x7f26e8147740 2024-09-16T07:30:48.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.518+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e80c5af0 con 0x7f26e8147740 2024-09-16T07:30:48.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.519+0000 7f26df400640 1 -- 172.21.15.71:0/385691560 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7f26d400fa40 con 0x7f26e8147740 2024-09-16T07:30:48.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.520+0000 7f26df400640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26c80798a0 0x7f26c807bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.520+0000 7f26df400640 1 -- 172.21.15.71:0/385691560 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f26d4099bf0 con 0x7f26e8147740 2024-09-16T07:30:48.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.521+0000 7f26e7400640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26c80798a0 0x7f26c807bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.522+0000 7f26e7400640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26c80798a0 0x7f26c807bd60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8009060 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:48.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.524+0000 7f26df400640 1 -- 172.21.15.71:0/385691560 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f26e80c5af0 con 0x7f26e8147740 2024-09-16T07:30:48.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.739+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26e80c4ae0 con 0x7f26c80798a0 2024-09-16T07:30:48.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.773+0000 7f26df400640 1 -- 172.21.15.71:0/385691560 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26e80c4ae0 con 0x7f26c80798a0 2024-09-16T07:30:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26c80798a0 msgr2=0x7f26c807bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26c80798a0 0x7f26c807bd60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8009060 comp rx=0 tx=0).stop 2024-09-16T07:30:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e8147740 msgr2=0x7f26e8131040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e8147740 0x7f26e8131040 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f26d400c920 tx=0x7f26d400cde0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 shutdown_connections 2024-09-16T07:30:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f26c80798a0 0x7f26c807bd60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e8131580 0x7f26e80b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e8147740 0x7f26e8131040 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 --2- 172.21.15.71:0/385691560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e80c5020 0x7f26e80b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 >> 172.21.15.71:0/385691560 conn(0x7f26e80cc490 msgr2=0x7f26e815ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:48.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 shutdown_connections 2024-09-16T07:30:48.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.775+0000 7f26ed000640 1 -- 172.21.15.71:0/385691560 wait complete. 2024-09-16T07:30:48.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T07:30:48.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.951+0000 7f292b200640 1 -- 172.21.15.71:0/2313004597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 msgr2=0x7f292c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.951+0000 7f292b200640 1 --2- 172.21.15.71:0/2313004597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 0x7f292c06fce0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f291c009830 tx=0x7f291c02a510 comp rx=0 tx=0).stop 2024-09-16T07:30:48.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.951+0000 7f292b200640 1 -- 172.21.15.71:0/2313004597 shutdown_connections 2024-09-16T07:30:48.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.951+0000 7f292b200640 1 --2- 172.21.15.71:0/2313004597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f292c072560 0x7f292c073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.951+0000 7f292b200640 1 --2- 172.21.15.71:0/2313004597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 0x7f292c06fce0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.951+0000 7f292b200640 1 -- 172.21.15.71:0/2313004597 >> 172.21.15.71:0/2313004597 conn(0x7f292c0cf9f0 msgr2=0x7f292c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:48.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.951+0000 7f292b200640 1 -- 172.21.15.71:0/2313004597 shutdown_connections 2024-09-16T07:30:48.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.951+0000 7f292b200640 1 -- 172.21.15.71:0/2313004597 wait complete. 2024-09-16T07:30:48.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.953+0000 7f292b200640 1 Processor -- start 2024-09-16T07:30:48.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.953+0000 7f292b200640 1 -- start start 2024-09-16T07:30:48.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f292c072560 0x7f292c14a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f292c0cb4f0 0x7f292c14b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 0x7f292c14bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f292c0cbdb0 con 0x7f292c15bec0 2024-09-16T07:30:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f292c0cbc30 con 0x7f292c072560 2024-09-16T07:30:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f292c0aebd0 con 0x7f292c0cb4f0 2024-09-16T07:30:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 0x7f292c14bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 0x7f292c14bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35774/0 (socket says 172.21.15.71:35774) 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f292a800640 1 -- 172.21.15.71:0/426758181 learned_addr learned my addr 172.21.15.71:0/426758181 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.954+0000 7f2929400640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f292c0cb4f0 0x7f292c14b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f2929e00640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f292c072560 0x7f292c14a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f292a800640 1 -- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f292c0cb4f0 msgr2=0x7f292c14b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f292a800640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f292c0cb4f0 0x7f292c14b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f292a800640 1 -- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f292c072560 msgr2=0x7f292c14a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f292a800640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f292c072560 0x7f292c14a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f292a800640 1 -- 172.21.15.71:0/426758181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c14aa90 con 0x7f292c15bec0 2024-09-16T07:30:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f2929e00640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f292c072560 0x7f292c14a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:30:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f292a800640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 0x7f292c14bc00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f292400cbc0 tx=0x7f292400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f2922000640 1 -- 172.21.15.71:0/426758181 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924016020 con 0x7f292c15bec0 2024-09-16T07:30:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f2922000640 1 -- 172.21.15.71:0/426758181 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f292400ae30 con 0x7f292c15bec0 2024-09-16T07:30:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292c14c230 con 0x7f292c15bec0 2024-09-16T07:30:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f2922000640 1 -- 172.21.15.71:0/426758181 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924012660 con 0x7f292c15bec0 2024-09-16T07:30:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.955+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292c14c7f0 con 0x7f292c15bec0 2024-09-16T07:30:48.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.957+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292c14e480 con 0x7f292c15bec0 2024-09-16T07:30:48.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.958+0000 7f2922000640 1 -- 172.21.15.71:0/426758181 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7f2924019020 con 0x7f292c15bec0 2024-09-16T07:30:48.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.959+0000 7f2922000640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f290c0798a0 0x7f290c07bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:48.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.959+0000 7f2922000640 1 -- 172.21.15.71:0/426758181 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f2924099ce0 con 0x7f292c15bec0 2024-09-16T07:30:48.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.959+0000 7f2929e00640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f290c0798a0 0x7f290c07bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:48.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.960+0000 7f2929e00640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f290c0798a0 0x7f290c07bd60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f291c034000 tx=0x7f291c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:48.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:48.963+0000 7f2922000640 1 -- 172.21.15.71:0/426758181 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f292c14e480 con 0x7f292c15bec0 2024-09-16T07:30:49.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.172+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f292c0af0f0 con 0x7f290c0798a0 2024-09-16T07:30:49.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.195+0000 7f2922000640 1 -- 172.21.15.71:0/426758181 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f292c0af0f0 con 0x7f290c0798a0 2024-09-16T07:30:49.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.197+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f290c0798a0 msgr2=0x7f290c07bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:49.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f290c0798a0 0x7f290c07bd60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f291c034000 tx=0x7f291c004730 comp rx=0 tx=0).stop 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 msgr2=0x7f292c14bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 0x7f292c14bc00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f292400cbc0 tx=0x7f292400a430 comp rx=0 tx=0).stop 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 shutdown_connections 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f290c0798a0 0x7f290c07bd60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f292c15bec0 0x7f292c14bc00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f292c0cb4f0 0x7f292c14b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 --2- 172.21.15.71:0/426758181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f292c072560 0x7f292c14a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 >> 172.21.15.71:0/426758181 conn(0x7f292c0cf9f0 msgr2=0x7f292c0aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 shutdown_connections 2024-09-16T07:30:49.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.198+0000 7f292b200640 1 -- 172.21.15.71:0/426758181 wait complete. 2024-09-16T07:30:49.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20415 2024-09-16T07:30:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.381+0000 7f5baa200640 1 -- 172.21.15.71:0/2709635829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 msgr2=0x7f5ba40d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.381+0000 7f5baa200640 1 --2- 172.21.15.71:0/2709635829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 0x7f5ba40d2610 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009860 tx=0x7f5b9002a540 comp rx=0 tx=0).stop 2024-09-16T07:30:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.382+0000 7f5baa200640 1 -- 172.21.15.71:0/2709635829 shutdown_connections 2024-09-16T07:30:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.382+0000 7f5baa200640 1 --2- 172.21.15.71:0/2709635829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 0x7f5ba40d2610 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.382+0000 7f5baa200640 1 --2- 172.21.15.71:0/2709635829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ba415dfb0 0x7f5ba40d4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.382+0000 7f5baa200640 1 -- 172.21.15.71:0/2709635829 >> 172.21.15.71:0/2709635829 conn(0x7f5ba409d180 msgr2=0x7f5ba40d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.382+0000 7f5baa200640 1 -- 172.21.15.71:0/2709635829 shutdown_connections 2024-09-16T07:30:49.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.382+0000 7f5baa200640 1 -- 172.21.15.71:0/2709635829 wait complete. 2024-09-16T07:30:49.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.383+0000 7f5baa200640 1 Processor -- start 2024-09-16T07:30:49.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.383+0000 7f5baa200640 1 -- start start 2024-09-16T07:30:49.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5baa200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 0x7f5ba4160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:49.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5baa200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba40d2b50 0x7f5ba41f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:49.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5baa200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ba415dfb0 0x7f5ba41f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5baa200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40b6d10 con 0x7f5ba40cf470 2024-09-16T07:30:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5baa200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40b6e90 con 0x7f5ba40d2b50 2024-09-16T07:30:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5baa200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba40b7010 con 0x7f5ba415dfb0 2024-09-16T07:30:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5ba8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 0x7f5ba4160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5ba8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 0x7f5ba4160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35786/0 (socket says 172.21.15.71:35786) 2024-09-16T07:30:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.384+0000 7f5ba8e00640 1 -- 172.21.15.71:0/1101800424 learned_addr learned my addr 172.21.15.71:0/1101800424 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.385+0000 7f5ba3e00640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba40d2b50 0x7f5ba41f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:49.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.385+0000 7f5ba9800640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ba415dfb0 0x7f5ba41f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.385+0000 7f5ba8e00640 1 -- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ba415dfb0 msgr2=0x7f5ba41f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.385+0000 7f5ba8e00640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ba415dfb0 0x7f5ba41f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.385+0000 7f5ba8e00640 1 -- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba40d2b50 msgr2=0x7f5ba41f8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.385+0000 7f5ba8e00640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba40d2b50 0x7f5ba41f8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.385+0000 7f5ba8e00640 1 -- 172.21.15.71:0/1101800424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba41f9120 con 0x7f5ba40cf470 2024-09-16T07:30:49.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.385+0000 7f5ba8e00640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 0x7f5ba4160f60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800e790 tx=0x7f5b9800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.387+0000 7f5ba0c00640 1 -- 172.21.15.71:0/1101800424 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9800eea0 con 0x7f5ba40cf470 2024-09-16T07:30:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.389+0000 7f5ba0c00640 1 -- 172.21.15.71:0/1101800424 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b9800cdb0 con 0x7f5ba40cf470 2024-09-16T07:30:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.389+0000 7f5ba0c00640 1 -- 172.21.15.71:0/1101800424 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98010690 con 0x7f5ba40cf470 2024-09-16T07:30:49.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.389+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba4159100 con 0x7f5ba40cf470 2024-09-16T07:30:49.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.389+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba41f4df0 con 0x7f5ba40cf470 2024-09-16T07:30:49.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.389+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba41f4be0 con 0x7f5ba40cf470 2024-09-16T07:30:49.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.392+0000 7f5ba0c00640 1 -- 172.21.15.71:0/1101800424 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 29) ==== 102766+0+0 (secure 0 0 0) 0x7f5b98010830 con 0x7f5ba40cf470 2024-09-16T07:30:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.393+0000 7f5ba0c00640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b840798d0 0x7f5b8407bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:49.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.393+0000 7f5ba0c00640 1 -- 172.21.15.71:0/1101800424 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f5b9809a080 con 0x7f5ba40cf470 2024-09-16T07:30:49.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.394+0000 7f5ba3e00640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b840798d0 0x7f5b8407bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.396+0000 7f5ba0c00640 1 -- 172.21.15.71:0/1101800424 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f5b9805fa40 con 0x7f5ba40cf470 2024-09-16T07:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.398+0000 7f5ba3e00640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b840798d0 0x7f5b8407bd90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b90034000 tx=0x7f5b90004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:49.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.406+0000 7f5ba0c00640 1 -- 172.21.15.71:0/1101800424 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5ba41f9120 con 0x7f5ba40cf470 2024-09-16T07:30:49.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:49.619+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20415", "target": ["mon-mgr", ""]}) -- 0x7f5ba40b75a0 con 0x7f5b840798d0 2024-09-16T07:30:50.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.533+0000 7f5ba0c00640 1 -- 172.21.15.71:0/1101800424 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ba40b75a0 con 0x7f5b840798d0 2024-09-16T07:30:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.536+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b840798d0 msgr2=0x7f5b8407bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.536+0000 7f5baa200640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b840798d0 0x7f5b8407bd90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b90034000 tx=0x7f5b90004730 comp rx=0 tx=0).stop 2024-09-16T07:30:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.536+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 msgr2=0x7f5ba4160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.536+0000 7f5baa200640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 0x7f5ba4160f60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800e790 tx=0x7f5b9800ec50 comp rx=0 tx=0).stop 2024-09-16T07:30:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.537+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 shutdown_connections 2024-09-16T07:30:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.537+0000 7f5baa200640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b840798d0 0x7f5b8407bd90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.537+0000 7f5baa200640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ba415dfb0 0x7f5ba41f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.537+0000 7f5baa200640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba40d2b50 0x7f5ba41f8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.537+0000 7f5baa200640 1 --2- 172.21.15.71:0/1101800424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ba40cf470 0x7f5ba4160f60 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.537+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 >> 172.21.15.71:0/1101800424 conn(0x7f5ba409d180 msgr2=0x7f5ba40b7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.537+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 shutdown_connections 2024-09-16T07:30:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.538+0000 7f5baa200640 1 -- 172.21.15.71:0/1101800424 wait complete. 2024-09-16T07:30:50.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone22153 2024-09-16T07:30:50.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.751+0000 7f3d47600640 1 -- 172.21.15.71:0/2722463254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 msgr2=0x7f3d4006eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:50.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.751+0000 7f3d47600640 1 --2- 172.21.15.71:0/2722463254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 0x7f3d4006eb20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3d4006aa00 tx=0x7f3d3802aa30 comp rx=0 tx=0).stop 2024-09-16T07:30:50.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.751+0000 7f3d47600640 1 -- 172.21.15.71:0/2722463254 shutdown_connections 2024-09-16T07:30:50.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.751+0000 7f3d47600640 1 --2- 172.21.15.71:0/2722463254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 0x7f3d4006eb20 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:50.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.751+0000 7f3d47600640 1 --2- 172.21.15.71:0/2722463254 >> v2:172.21.15.177:3300/0 conn(0x7f3d40158900 0x7f3d4015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:50.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.751+0000 7f3d47600640 1 -- 172.21.15.71:0/2722463254 >> 172.21.15.71:0/2722463254 conn(0x7f3d400b18e0 msgr2=0x7f3d400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:50.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.751+0000 7f3d47600640 1 -- 172.21.15.71:0/2722463254 shutdown_connections 2024-09-16T07:30:50.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.751+0000 7f3d47600640 1 -- 172.21.15.71:0/2722463254 wait complete. 2024-09-16T07:30:50.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.753+0000 7f3d47600640 1 Processor -- start 2024-09-16T07:30:50.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.753+0000 7f3d47600640 1 -- start start 2024-09-16T07:30:50.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d47600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 0x7f3d401e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:50.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d47600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d40158900 0x7f3d400b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d47600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d4015b700 0x7f3d40148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d47600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3d400ced60 con 0x7f3d4006e6f0 2024-09-16T07:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d47600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d400cebe0 con 0x7f3d40158900 2024-09-16T07:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d47600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3d400ceee0 con 0x7f3d4015b700 2024-09-16T07:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d44c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 0x7f3d401e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d44c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 0x7f3d401e3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35806/0 (socket says 172.21.15.71:35806) 2024-09-16T07:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d44c00640 1 -- 172.21.15.71:0/4109363135 learned_addr learned my addr 172.21.15.71:0/4109363135 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:50.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d44c00640 1 -- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d4015b700 msgr2=0x7f3d40148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:50.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d44c00640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d4015b700 0x7f3d40148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:50.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d44c00640 1 -- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d40158900 msgr2=0x7f3d400b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:50.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d44c00640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d40158900 0x7f3d400b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:50.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.754+0000 7f3d44c00640 1 -- 172.21.15.71:0/4109363135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d401e33e0 con 0x7f3d4006e6f0 2024-09-16T07:30:50.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.755+0000 7f3d44c00640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 0x7f3d401e3ea0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000b4d0 tx=0x7f3d3000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:50.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.755+0000 7f3d3cc00640 1 -- 172.21.15.71:0/4109363135 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30014070 con 0x7f3d4006e6f0 2024-09-16T07:30:50.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.755+0000 7f3d3cc00640 1 -- 172.21.15.71:0/4109363135 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d30010960 con 0x7f3d4006e6f0 2024-09-16T07:30:50.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.755+0000 7f3d3cc00640 1 -- 172.21.15.71:0/4109363135 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30010db0 con 0x7f3d4006e6f0 2024-09-16T07:30:50.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.755+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d401f35b0 con 0x7f3d4006e6f0 2024-09-16T07:30:50.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.755+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d400b1050 con 0x7f3d4006e6f0 2024-09-16T07:30:50.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.756+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d40069360 con 0x7f3d4006e6f0 2024-09-16T07:30:50.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.757+0000 7f3d3cc00640 1 -- 172.21.15.71:0/4109363135 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f3d30017070 con 0x7f3d4006e6f0 2024-09-16T07:30:50.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.758+0000 7f3d3cc00640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d20079960 0x7f3d2007be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:50.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.758+0000 7f3d3cc00640 1 -- 172.21.15.71:0/4109363135 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5795+0+0 (secure 0 0 0) 0x7f3d30099650 con 0x7f3d4006e6f0 2024-09-16T07:30:50.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.758+0000 7f3d3fe00640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d20079960 0x7f3d2007be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:50.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.759+0000 7f3d3fe00640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d20079960 0x7f3d2007be20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3d401e4ad0 tx=0x7f3d2c006ba0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:50.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.762+0000 7f3d3cc00640 1 -- 172.21.15.71:0/4109363135 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3d300633d0 con 0x7f3d4006e6f0 2024-09-16T07:30:50.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:50.973+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone22153", "target": ["mon-mgr", ""]}) -- 0x7f3d400cf450 con 0x7f3d20079960 2024-09-16T07:30:51.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.157+0000 7f3d3cc00640 1 -- 172.21.15.71:0/4109363135 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d400cf450 con 0x7f3d20079960 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d20079960 msgr2=0x7f3d2007be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d20079960 0x7f3d2007be20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3d401e4ad0 tx=0x7f3d2c006ba0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 msgr2=0x7f3d401e3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 0x7f3d401e3ea0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000b4d0 tx=0x7f3d3000b990 comp rx=0 tx=0).stop 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 shutdown_connections 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3d20079960 0x7f3d2007be20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d4015b700 0x7f3d40148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d40158900 0x7f3d400b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 --2- 172.21.15.71:0/4109363135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d4006e6f0 0x7f3d401e3ea0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 >> 172.21.15.71:0/4109363135 conn(0x7f3d400b18e0 msgr2=0x7f3d400ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:51.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 shutdown_connections 2024-09-16T07:30:51.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.159+0000 7f3d47600640 1 -- 172.21.15.71:0/4109363135 wait complete. 2024-09-16T07:30:51.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone31984 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 -- 172.21.15.71:0/1835259500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 msgr2=0x7f7f440cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 --2- 172.21.15.71:0/1835259500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 0x7f7f440cefb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7f44076820 tx=0x7f7f3402b1e0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 -- 172.21.15.71:0/1835259500 shutdown_connections 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 --2- 172.21.15.71:0/1835259500 >> v2:172.21.15.71:3300/0 conn(0x7f7f4406fa00 0x7f7f4406feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 --2- 172.21.15.71:0/1835259500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f440b3e20 0x7f7f440cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 --2- 172.21.15.71:0/1835259500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 0x7f7f440cefb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 -- 172.21.15.71:0/1835259500 >> 172.21.15.71:0/1835259500 conn(0x7f7f440b18e0 msgr2=0x7f7f440b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 -- 172.21.15.71:0/1835259500 shutdown_connections 2024-09-16T07:30:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.401+0000 7f7f4a200640 1 -- 172.21.15.71:0/1835259500 wait complete. 2024-09-16T07:30:51.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.404+0000 7f7f4a200640 1 Processor -- start 2024-09-16T07:30:51.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.404+0000 7f7f4a200640 1 -- start start 2024-09-16T07:30:51.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.406+0000 7f7f4a200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f4406fa00 0x7f7f4415b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:51.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.406+0000 7f7f4a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f440b3e20 0x7f7f441f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:51.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.406+0000 7f7f4a200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 0x7f7f441f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:51.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.406+0000 7f7f4a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f440cc6d0 con 0x7f7f440b3e20 2024-09-16T07:30:51.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.406+0000 7f7f4a200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f440cc550 con 0x7f7f441512b0 2024-09-16T07:30:51.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.406+0000 7f7f4a200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7f440cc850 con 0x7f7f4406fa00 2024-09-16T07:30:51.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f43e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 0x7f7f441f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:51.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f43e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 0x7f7f441f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:49766/0 (socket says 172.21.15.71:49766) 2024-09-16T07:30:51.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f43e00640 1 -- 172.21.15.71:0/3215506812 learned_addr learned my addr 172.21.15.71:0/3215506812 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:51.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f42a00640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f440b3e20 0x7f7f441f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:51.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f43e00640 1 -- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f4406fa00 msgr2=0x7f7f4415b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:51.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f43e00640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f4406fa00 0x7f7f4415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f43e00640 1 -- 172.21.15.71:0/3215506812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f440b3e20 msgr2=0x7f7f441f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:51.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f43e00640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f440b3e20 0x7f7f441f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.407+0000 7f7f43e00640 1 -- 172.21.15.71:0/3215506812 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4415bc80 con 0x7f7f441512b0 2024-09-16T07:30:51.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.408+0000 7f7f43e00640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 0x7f7f441f8bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00da90 tx=0x7f7f3c00df50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:51.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.409+0000 7f7f3b400640 1 -- 172.21.15.71:0/3215506812 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c017070 con 0x7f7f441512b0 2024-09-16T07:30:51.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.409+0000 7f7f3b400640 1 -- 172.21.15.71:0/3215506812 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f3c003970 con 0x7f7f441512b0 2024-09-16T07:30:51.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.410+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4415d650 con 0x7f7f441512b0 2024-09-16T07:30:51.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.410+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4415dc10 con 0x7f7f441512b0 2024-09-16T07:30:51.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.411+0000 7f7f3b400640 1 -- 172.21.15.71:0/3215506812 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c003d80 con 0x7f7f441512b0 2024-09-16T07:30:51.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.411+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f440cccd0 con 0x7f7f441512b0 2024-09-16T07:30:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.415+0000 7f7f3b400640 1 -- 172.21.15.71:0/3215506812 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f7f3c004b20 con 0x7f7f441512b0 2024-09-16T07:30:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.421+0000 7f7f3b400640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f24079a00 0x7f7f2407bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.422+0000 7f7f3b400640 1 -- 172.21.15.71:0/3215506812 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f7f3c099d90 con 0x7f7f441512b0 2024-09-16T07:30:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.422+0000 7f7f43400640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f24079a00 0x7f7f2407bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.424+0000 7f7f3b400640 1 -- 172.21.15.71:0/3215506812 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7f3c063e50 con 0x7f7f441512b0 2024-09-16T07:30:51.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.426+0000 7f7f43400640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f24079a00 0x7f7f2407bec0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7f34032000 tx=0x7f7f340046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:51.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.778+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone31984", "target": ["mon-mgr", ""]}) -- 0x7f7f4406f700 con 0x7f7f24079a00 2024-09-16T07:30:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.820+0000 7f7f3b400640 1 -- 172.21.15.71:0/3215506812 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f4406f700 con 0x7f7f24079a00 2024-09-16T07:30:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f24079a00 msgr2=0x7f7f2407bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f24079a00 0x7f7f2407bec0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7f34032000 tx=0x7f7f340046c0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 msgr2=0x7f7f441f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 0x7f7f441f8bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00da90 tx=0x7f7f3c00df50 comp rx=0 tx=0).stop 2024-09-16T07:30:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 shutdown_connections 2024-09-16T07:30:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f24079a00 0x7f7f2407bec0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f441512b0 0x7f7f441f8bb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f440b3e20 0x7f7f441f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 --2- 172.21.15.71:0/3215506812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f4406fa00 0x7f7f4415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.822+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 >> 172.21.15.71:0/3215506812 conn(0x7f7f440b18e0 msgr2=0x7f7f440acda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.823+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 shutdown_connections 2024-09-16T07:30:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:51.823+0000 7f7f4a200640 1 -- 172.21.15.71:0/3215506812 wait complete. 2024-09-16T07:30:51.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone82630 2024-09-16T07:30:52.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 -- 172.21.15.71:0/2374702824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c15bda0 msgr2=0x7f849c073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 --2- 172.21.15.71:0/2374702824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c15bda0 0x7f849c073010 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f849c072590 tx=0x7f848c02a560 comp rx=0 tx=0).stop 2024-09-16T07:30:52.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 -- 172.21.15.71:0/2374702824 shutdown_connections 2024-09-16T07:30:52.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 --2- 172.21.15.71:0/2374702824 >> v2:172.21.15.177:3300/0 conn(0x7f849c070380 0x7f849c0739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 --2- 172.21.15.71:0/2374702824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f849c06e590 0x7f849c06fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 --2- 172.21.15.71:0/2374702824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c15bda0 0x7f849c073010 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 -- 172.21.15.71:0/2374702824 >> 172.21.15.71:0/2374702824 conn(0x7f849c0cfa70 msgr2=0x7f849c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:52.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 -- 172.21.15.71:0/2374702824 shutdown_connections 2024-09-16T07:30:52.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.000+0000 7f849b200640 1 -- 172.21.15.71:0/2374702824 wait complete. 2024-09-16T07:30:52.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.001+0000 7f849b200640 1 Processor -- start 2024-09-16T07:30:52.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.001+0000 7f849b200640 1 -- start start 2024-09-16T07:30:52.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.002+0000 7f849b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f849c06e590 0x7f849c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.002+0000 7f849b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c070380 0x7f849c1f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.002+0000 7f849b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f849c15bda0 0x7f849c0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.002+0000 7f849b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f849c074090 con 0x7f849c070380 2024-09-16T07:30:52.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.002+0000 7f849b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f849c073f10 con 0x7f849c06e590 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.002+0000 7f849b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f849c074210 con 0x7f849c15bda0 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.002+0000 7f8499400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c070380 0x7f849c1f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c070380 0x7f849c1f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35844/0 (socket says 172.21.15.71:35844) 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499400640 1 -- 172.21.15.71:0/1862943737 learned_addr learned my addr 172.21.15.71:0/1862943737 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f849a800640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f849c15bda0 0x7f849c0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499e00640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f849c06e590 0x7f849c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499400640 1 -- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f849c15bda0 msgr2=0x7f849c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499400640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f849c15bda0 0x7f849c0ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499400640 1 -- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f849c06e590 msgr2=0x7f849c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499400640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f849c06e590 0x7f849c0c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499400640 1 -- 172.21.15.71:0/1862943737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c1f5740 con 0x7f849c070380 2024-09-16T07:30:52.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.003+0000 7f8499400640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c070380 0x7f849c1f5200 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f848800b450 tx=0x7f848800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:52.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.004+0000 7f8492000640 1 -- 172.21.15.71:0/1862943737 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488015020 con 0x7f849c070380 2024-09-16T07:30:52.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.004+0000 7f8492000640 1 -- 172.21.15.71:0/1862943737 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8488011980 con 0x7f849c070380 2024-09-16T07:30:52.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.004+0000 7f8492000640 1 -- 172.21.15.71:0/1862943737 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488011dd0 con 0x7f849c070380 2024-09-16T07:30:52.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.004+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c0cd640 con 0x7f849c070380 2024-09-16T07:30:52.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.004+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c0cdbc0 con 0x7f849c070380 2024-09-16T07:30:52.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.005+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849c074740 con 0x7f849c070380 2024-09-16T07:30:52.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.006+0000 7f8492000640 1 -- 172.21.15.71:0/1862943737 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f8488018020 con 0x7f849c070380 2024-09-16T07:30:52.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.007+0000 7f8492000640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f847c079930 0x7f847c07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.007+0000 7f8492000640 1 -- 172.21.15.71:0/1862943737 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f8488099550 con 0x7f849c070380 2024-09-16T07:30:52.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.007+0000 7f8499e00640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f847c079930 0x7f847c07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.008+0000 7f8499e00640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f847c079930 0x7f847c07bdf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f848c033000 tx=0x7f848c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:52.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.011+0000 7f8492000640 1 -- 172.21.15.71:0/1862943737 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f848801f020 con 0x7f849c070380 2024-09-16T07:30:52.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.223+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone82630", "target": ["mon-mgr", ""]}) -- 0x7f849c0b4380 con 0x7f847c079930 2024-09-16T07:30:52.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.243+0000 7f8492000640 1 -- 172.21.15.71:0/1862943737 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f849c0b4380 con 0x7f847c079930 2024-09-16T07:30:52.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.245+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f847c079930 msgr2=0x7f847c07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.245+0000 7f849b200640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f847c079930 0x7f847c07bdf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f848c033000 tx=0x7f848c004730 comp rx=0 tx=0).stop 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c070380 msgr2=0x7f849c1f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c070380 0x7f849c1f5200 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f848800b450 tx=0x7f848800b910 comp rx=0 tx=0).stop 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 shutdown_connections 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f847c079930 0x7f847c07bdf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f849c15bda0 0x7f849c0ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f849c070380 0x7f849c1f5200 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 --2- 172.21.15.71:0/1862943737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f849c06e590 0x7f849c0c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 >> 172.21.15.71:0/1862943737 conn(0x7f849c0cfa70 msgr2=0x7f849c0b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:52.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 shutdown_connections 2024-09-16T07:30:52.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.246+0000 7f849b200640 1 -- 172.21.15.71:0/1862943737 wait complete. 2024-09-16T07:30:52.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone52566 2024-09-16T07:30:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71b2000640 1 -- 172.21.15.71:0/2576168108 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0047a0 con 0x7f71b406f220 2024-09-16T07:30:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71ba600640 1 -- 172.21.15.71:0/2576168108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 msgr2=0x7f71b406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71ba600640 1 --2- 172.21.15.71:0/2576168108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 0x7f71b406f6d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009900 tx=0x7f71ac02a610 comp rx=0 tx=0).stop 2024-09-16T07:30:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71ba600640 1 -- 172.21.15.71:0/2576168108 shutdown_connections 2024-09-16T07:30:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71ba600640 1 --2- 172.21.15.71:0/2576168108 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71b415cff0 0x7f71b406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71ba600640 1 --2- 172.21.15.71:0/2576168108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 0x7f71b406f6d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71ba600640 1 -- 172.21.15.71:0/2576168108 >> 172.21.15.71:0/2576168108 conn(0x7f71b40b18e0 msgr2=0x7f71b40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71ba600640 1 -- 172.21.15.71:0/2576168108 shutdown_connections 2024-09-16T07:30:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.472+0000 7f71ba600640 1 -- 172.21.15.71:0/2576168108 wait complete. 2024-09-16T07:30:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.474+0000 7f71ba600640 1 Processor -- start 2024-09-16T07:30:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.474+0000 7f71ba600640 1 -- start start 2024-09-16T07:30:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71ba600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 0x7f71b40cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71ba600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71b4159aa0 0x7f71b41f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71ba600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71b415cff0 0x7f71b41eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71ba600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f71b40cc5a0 con 0x7f71b406f220 2024-09-16T07:30:52.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71ba600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71b40cc420 con 0x7f71b415cff0 2024-09-16T07:30:52.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71ba600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f71b40cc720 con 0x7f71b4159aa0 2024-09-16T07:30:52.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 0x7f71b40cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 0x7f71b40cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35862/0 (socket says 172.21.15.71:35862) 2024-09-16T07:30:52.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71b4159aa0 0x7f71b41f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71b415cff0 0x7f71b41eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3400640 1 -- 172.21.15.71:0/491916737 learned_addr learned my addr 172.21.15.71:0/491916737 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:52.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3400640 1 -- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71b4159aa0 msgr2=0x7f71b41f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3400640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71b4159aa0 0x7f71b41f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3400640 1 -- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71b415cff0 msgr2=0x7f71b41eebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3400640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71b415cff0 0x7f71b41eebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.475+0000 7f71b3400640 1 -- 172.21.15.71:0/491916737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b4157a70 con 0x7f71b406f220 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.476+0000 7f71b3400640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 0x7f71b40cdad0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f71a400c870 tx=0x7f71a400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.476+0000 7f71ab400640 1 -- 172.21.15.71:0/491916737 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003b70 con 0x7f71b406f220 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.476+0000 7f71ab400640 1 -- 172.21.15.71:0/491916737 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a4003d10 con 0x7f71b406f220 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.476+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b40ce0b0 con 0x7f71b406f220 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.476+0000 7f71ab400640 1 -- 172.21.15.71:0/491916737 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400aab0 con 0x7f71b406f220 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.476+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b4157ce0 con 0x7f71b406f220 2024-09-16T07:30:52.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.477+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b4069360 con 0x7f71b406f220 2024-09-16T07:30:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.478+0000 7f71ab400640 1 -- 172.21.15.71:0/491916737 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f71a400ac50 con 0x7f71b406f220 2024-09-16T07:30:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.479+0000 7f71ab400640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7194079930 0x7f719407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.479+0000 7f71ab400640 1 -- 172.21.15.71:0/491916737 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f71a4099530 con 0x7f71b406f220 2024-09-16T07:30:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.480+0000 7f71b2a00640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7194079930 0x7f719407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.480+0000 7f71b2a00640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7194079930 0x7f719407bdf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f71b4157960 tx=0x7f71a000b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:52.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.483+0000 7f71ab400640 1 -- 172.21.15.71:0/491916737 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f71a4062c60 con 0x7f71b406f220 2024-09-16T07:30:52.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.695+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone52566", "target": ["mon-mgr", ""]}) -- 0x7f71b40ccbf0 con 0x7f7194079930 2024-09-16T07:30:52.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.716+0000 7f71ab400640 1 -- 172.21.15.71:0/491916737 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71b40ccbf0 con 0x7f7194079930 2024-09-16T07:30:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7194079930 msgr2=0x7f719407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7194079930 0x7f719407bdf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f71b4157960 tx=0x7f71a000b2e0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 msgr2=0x7f71b40cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 0x7f71b40cdad0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f71a400c870 tx=0x7f71a400cd30 comp rx=0 tx=0).stop 2024-09-16T07:30:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 shutdown_connections 2024-09-16T07:30:52.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7194079930 0x7f719407bdf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71b415cff0 0x7f71b41eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71b4159aa0 0x7f71b41f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 --2- 172.21.15.71:0/491916737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71b406f220 0x7f71b40cdad0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 >> 172.21.15.71:0/491916737 conn(0x7f71b40b18e0 msgr2=0x7f71b415e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 shutdown_connections 2024-09-16T07:30:52.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.718+0000 7f71ba600640 1 -- 172.21.15.71:0/491916737 wait complete. 2024-09-16T07:30:52.735 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone19152 2024-09-16T07:30:52.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.897+0000 7f86c4200640 1 -- 172.21.15.71:0/1636194215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc15dfc0 msgr2=0x7f86bc06f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.897+0000 7f86c4200640 1 --2- 172.21.15.71:0/1636194215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc15dfc0 0x7f86bc06f300 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f86b0005fd0 tx=0x7f86b002a5d0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.897+0000 7f86c4200640 1 -- 172.21.15.71:0/1636194215 shutdown_connections 2024-09-16T07:30:52.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.898+0000 7f86c4200640 1 --2- 172.21.15.71:0/1636194215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86bc06ee00 0x7f86bc0ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.898+0000 7f86c4200640 1 --2- 172.21.15.71:0/1636194215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc15dfc0 0x7f86bc06f300 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.898+0000 7f86c4200640 1 -- 172.21.15.71:0/1636194215 >> 172.21.15.71:0/1636194215 conn(0x7f86bc0b18e0 msgr2=0x7f86bc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:52.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.898+0000 7f86c4200640 1 -- 172.21.15.71:0/1636194215 shutdown_connections 2024-09-16T07:30:52.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.898+0000 7f86c4200640 1 -- 172.21.15.71:0/1636194215 wait complete. 2024-09-16T07:30:52.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.899+0000 7f86c4200640 1 Processor -- start 2024-09-16T07:30:52.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.899+0000 7f86c4200640 1 -- start start 2024-09-16T07:30:52.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.900+0000 7f86c4200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86bc06ee00 0x7f86bc1496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.900+0000 7f86c4200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc0b4770 0x7f86bc15bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.900+0000 7f86c4200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86bc15dfc0 0x7f86bc09cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.900+0000 7f86c4200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc0adfa0 con 0x7f86bc0b4770 2024-09-16T07:30:52.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.900+0000 7f86c4200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc0ade20 con 0x7f86bc15dfc0 2024-09-16T07:30:52.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.900+0000 7f86c4200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc0ae120 con 0x7f86bc06ee00 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc0b4770 0x7f86bc15bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc0b4770 0x7f86bc15bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35888/0 (socket says 172.21.15.71:35888) 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 -- 172.21.15.71:0/998778619 learned_addr learned my addr 172.21.15.71:0/998778619 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c2200640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86bc15dfc0 0x7f86bc09cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c1800640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86bc06ee00 0x7f86bc1496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 -- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86bc06ee00 msgr2=0x7f86bc1496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86bc06ee00 0x7f86bc1496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 -- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86bc15dfc0 msgr2=0x7f86bc09cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86bc15dfc0 0x7f86bc09cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 -- 172.21.15.71:0/998778619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc15e7b0 con 0x7f86bc0b4770 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86c0e00640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc0b4770 0x7f86bc15bf60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f86a800c410 tx=0x7f86a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.901+0000 7f86b9600640 1 -- 172.21.15.71:0/998778619 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8017070 con 0x7f86bc0b4770 2024-09-16T07:30:52.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.902+0000 7f86b9600640 1 -- 172.21.15.71:0/998778619 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86a80035f0 con 0x7f86bc0b4770 2024-09-16T07:30:52.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.902+0000 7f86b9600640 1 -- 172.21.15.71:0/998778619 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8003a80 con 0x7f86bc0b4770 2024-09-16T07:30:52.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.902+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc15c9b0 con 0x7f86bc0b4770 2024-09-16T07:30:52.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.902+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc15b260 con 0x7f86bc0b4770 2024-09-16T07:30:52.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.903+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc069360 con 0x7f86bc0b4770 2024-09-16T07:30:52.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.904+0000 7f86b9600640 1 -- 172.21.15.71:0/998778619 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f86a801a070 con 0x7f86bc0b4770 2024-09-16T07:30:52.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.905+0000 7f86b9600640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f869c079930 0x7f869c07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:52.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.905+0000 7f86b9600640 1 -- 172.21.15.71:0/998778619 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f86a8098ab0 con 0x7f86bc0b4770 2024-09-16T07:30:52.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.906+0000 7f86c1800640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f869c079930 0x7f869c07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:52.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.906+0000 7f86c1800640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f869c079930 0x7f869c07bdf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f86b0033000 tx=0x7f86b0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:52.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:52.910+0000 7f86b9600640 1 -- 172.21.15.71:0/998778619 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f86a80621e0 con 0x7f86bc0b4770 2024-09-16T07:30:53.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.118+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone19152", "target": ["mon-mgr", ""]}) -- 0x7f86bc0ad6b0 con 0x7f869c079930 2024-09-16T07:30:53.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.136+0000 7f86b9600640 1 -- 172.21.15.71:0/998778619 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86bc0ad6b0 con 0x7f869c079930 2024-09-16T07:30:53.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.138+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f869c079930 msgr2=0x7f869c07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:53.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.138+0000 7f86c4200640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f869c079930 0x7f869c07bdf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f86b0033000 tx=0x7f86b0004730 comp rx=0 tx=0).stop 2024-09-16T07:30:53.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.138+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc0b4770 msgr2=0x7f86bc15bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:53.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.138+0000 7f86c4200640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc0b4770 0x7f86bc15bf60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f86a800c410 tx=0x7f86a800c8d0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.139+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 shutdown_connections 2024-09-16T07:30:53.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.139+0000 7f86c4200640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f869c079930 0x7f869c07bdf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.139+0000 7f86c4200640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86bc15dfc0 0x7f86bc09cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.139+0000 7f86c4200640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86bc0b4770 0x7f86bc15bf60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.139+0000 7f86c4200640 1 --2- 172.21.15.71:0/998778619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86bc06ee00 0x7f86bc1496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.139+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 >> 172.21.15.71:0/998778619 conn(0x7f86bc0b18e0 msgr2=0x7f86bc0740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:53.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.139+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 shutdown_connections 2024-09-16T07:30:53.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.139+0000 7f86c4200640 1 -- 172.21.15.71:0/998778619 wait complete. 2024-09-16T07:30:53.157 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:30:53.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:30:53.242 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:30:53.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:30:53.251 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:30:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.518+0000 7fea22800640 1 -- 172.21.15.71:0/1128175325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14004b00 msgr2=0x7fea140e2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.519+0000 7fea22800640 1 --2- 172.21.15.71:0/1128175325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14004b00 0x7fea140e2240 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea1404ba30 tx=0x7fea1002b760 comp rx=0 tx=0).stop 2024-09-16T07:30:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.519+0000 7fea22800640 1 -- 172.21.15.71:0/1128175325 shutdown_connections 2024-09-16T07:30:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.519+0000 7fea22800640 1 --2- 172.21.15.71:0/1128175325 >> v2:172.21.15.71:3300/0 conn(0x7fea1403a460 0x7fea14052580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.519+0000 7fea22800640 1 --2- 172.21.15.71:0/1128175325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14004b00 0x7fea140e2240 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.519+0000 7fea22800640 1 -- 172.21.15.71:0/1128175325 >> 172.21.15.71:0/1128175325 conn(0x7fea1401dbc0 msgr2=0x7fea140549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.519+0000 7fea22800640 1 -- 172.21.15.71:0/1128175325 shutdown_connections 2024-09-16T07:30:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.519+0000 7fea22800640 1 -- 172.21.15.71:0/1128175325 wait complete. 2024-09-16T07:30:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.521+0000 7fea22800640 1 Processor -- start 2024-09-16T07:30:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.521+0000 7fea22800640 1 -- start start 2024-09-16T07:30:53.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.523+0000 7fea22800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea14004b00 0x7fea140d9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:53.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.523+0000 7fea22800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14035e70 0x7fea1417a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:53.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.523+0000 7fea22800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1403a460 0x7fea14050030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:53.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.523+0000 7fea22800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea14052ed0 con 0x7fea14004b00 2024-09-16T07:30:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.523+0000 7fea22800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea14052d50 con 0x7fea14035e70 2024-09-16T07:30:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.523+0000 7fea22800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fea14053050 con 0x7fea1403a460 2024-09-16T07:30:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.524+0000 7fea21e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1403a460 0x7fea14050030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.524+0000 7fea21e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1403a460 0x7fea14050030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:59026/0 (socket says 172.21.15.71:59026) 2024-09-16T07:30:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.524+0000 7fea21e00640 1 -- 172.21.15.71:0/394076141 learned_addr learned my addr 172.21.15.71:0/394076141 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.524+0000 7fea20a00640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14035e70 0x7fea1417a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.524+0000 7fea21400640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea14004b00 0x7fea140d9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.525+0000 7fea20a00640 1 -- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1403a460 msgr2=0x7fea14050030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.525+0000 7fea20a00640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1403a460 0x7fea14050030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.525+0000 7fea20a00640 1 -- 172.21.15.71:0/394076141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea14004b00 msgr2=0x7fea140d9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:53.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.525+0000 7fea20a00640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea14004b00 0x7fea140d9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.525+0000 7fea20a00640 1 -- 172.21.15.71:0/394076141 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea140d7930 con 0x7fea14035e70 2024-09-16T07:30:53.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.525+0000 7fea21400640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea14004b00 0x7fea140d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:30:53.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.526+0000 7fea20a00640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14035e70 0x7fea1417a000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c4b0 tx=0x7fea0800c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:53.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.526+0000 7fea19600640 1 -- 172.21.15.71:0/394076141 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08016020 con 0x7fea14035e70 2024-09-16T07:30:53.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.526+0000 7fea19600640 1 -- 172.21.15.71:0/394076141 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea08012920 con 0x7fea14035e70 2024-09-16T07:30:53.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.526+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea140d7bd0 con 0x7fea14035e70 2024-09-16T07:30:53.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.526+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea140d8190 con 0x7fea14035e70 2024-09-16T07:30:53.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.526+0000 7fea19600640 1 -- 172.21.15.71:0/394076141 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08012df0 con 0x7fea14035e70 2024-09-16T07:30:53.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.528+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f8005290 con 0x7fea14035e70 2024-09-16T07:30:53.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.529+0000 7fea19600640 1 -- 172.21.15.71:0/394076141 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fea08019020 con 0x7fea14035e70 2024-09-16T07:30:53.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.530+0000 7fea19600640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9fc079930 0x7fe9fc07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:53.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.531+0000 7fea19600640 1 -- 172.21.15.71:0/394076141 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7fea080990b0 con 0x7fea14035e70 2024-09-16T07:30:53.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.531+0000 7fea21400640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9fc079930 0x7fe9fc07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:53.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.533+0000 7fea21400640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9fc079930 0x7fe9fc07bdf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fea10035000 tx=0x7fea10004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:53.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.540+0000 7fea19600640 1 -- 172.21.15.71:0/394076141 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fea08020030 con 0x7fea14035e70 2024-09-16T07:30:53.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.847+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fe9f8005580 con 0x7fea14035e70 2024-09-16T07:30:53.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.848+0000 7fea19600640 1 -- 172.21.15.71:0/394076141 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7fea08063170 con 0x7fea14035e70 2024-09-16T07:30:53.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9fc079930 msgr2=0x7fe9fc07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:53.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9fc079930 0x7fe9fc07bdf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fea10035000 tx=0x7fea10004730 comp rx=0 tx=0).stop 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14035e70 msgr2=0x7fea1417a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14035e70 0x7fea1417a000 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c4b0 tx=0x7fea0800c970 comp rx=0 tx=0).stop 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 shutdown_connections 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9fc079930 0x7fe9fc07bdf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1403a460 0x7fea14050030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea14035e70 0x7fea1417a000 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 --2- 172.21.15.71:0/394076141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea14004b00 0x7fea140d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 >> 172.21.15.71:0/394076141 conn(0x7fea1401dbc0 msgr2=0x7fea1400b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 shutdown_connections 2024-09-16T07:30:53.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:53.851+0000 7fea22800640 1 -- 172.21.15.71:0/394076141 wait complete. 2024-09-16T07:30:53.866 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:30:53.866 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:30:53.957 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:30:54.039 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:30:54.040 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:30:54.040 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:30:54.040 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:30:54.075 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:30:54.075 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:30:54.150 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:30:54.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.411+0000 7f9da0e00640 1 -- 172.21.15.71:0/3474499732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d9c15e0d0 msgr2=0x7f9d9c0d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:54.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.411+0000 7f9da0e00640 1 --2- 172.21.15.71:0/3474499732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d9c15e0d0 0x7f9d9c0d09c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c005fd0 tx=0x7f9d8c02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.411+0000 7f9da0e00640 1 -- 172.21.15.71:0/3474499732 shutdown_connections 2024-09-16T07:30:54.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.411+0000 7f9da0e00640 1 --2- 172.21.15.71:0/3474499732 >> v2:172.21.15.177:3300/0 conn(0x7f9d9c0d2050 0x7f9d9c0d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.411+0000 7f9da0e00640 1 --2- 172.21.15.71:0/3474499732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d9c15e0d0 0x7f9d9c0d09c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.411+0000 7f9da0e00640 1 -- 172.21.15.71:0/3474499732 >> 172.21.15.71:0/3474499732 conn(0x7f9d9c0d3fe0 msgr2=0x7f9d9c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:54.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.411+0000 7f9da0e00640 1 -- 172.21.15.71:0/3474499732 shutdown_connections 2024-09-16T07:30:54.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.412+0000 7f9da0e00640 1 -- 172.21.15.71:0/3474499732 wait complete. 2024-09-16T07:30:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.414+0000 7f9da0e00640 1 Processor -- start 2024-09-16T07:30:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.414+0000 7f9da0e00640 1 -- start start 2024-09-16T07:30:54.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.416+0000 7f9da0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d9c0d1440 0x7f9d9c09ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:54.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9da0e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d9c0d2050 0x7f9d9c15d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:54.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9da0e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c0a16c0 0x7f9d9c0cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:54.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9da0e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c15ebc0 con 0x7f9d9c0d1440 2024-09-16T07:30:54.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9da0e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c0cff30 con 0x7f9d9c0a16c0 2024-09-16T07:30:54.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9da0e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c15ed40 con 0x7f9d9c0d2050 2024-09-16T07:30:54.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9d99e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d9c0d1440 0x7f9d9c09ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:54.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9d9a800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c0a16c0 0x7f9d9c0cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:54.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9d99e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d9c0d1440 0x7f9d9c09ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35914/0 (socket says 172.21.15.71:35914) 2024-09-16T07:30:54.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9d99e00640 1 -- 172.21.15.71:0/852879002 learned_addr learned my addr 172.21.15.71:0/852879002 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:54.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.417+0000 7f9d99400640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d9c0d2050 0x7f9d9c15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.418+0000 7f9d99e00640 1 -- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d9c0d2050 msgr2=0x7f9d9c15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.418+0000 7f9d99e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d9c0d2050 0x7f9d9c15d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.418+0000 7f9d99e00640 1 -- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c0a16c0 msgr2=0x7f9d9c0cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.418+0000 7f9d99e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c0a16c0 0x7f9d9c0cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.418+0000 7f9d99e00640 1 -- 172.21.15.71:0/852879002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c0a0660 con 0x7f9d9c0d1440 2024-09-16T07:30:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.418+0000 7f9d9a800640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c0a16c0 0x7f9d9c0cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:30:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.418+0000 7f9d99e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d9c0d1440 0x7f9d9c09ecb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c006010 tx=0x7f9d8c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.424+0000 7f9d92000640 1 -- 172.21.15.71:0/852879002 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c03f020 con 0x7f9d9c0d1440 2024-09-16T07:30:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.424+0000 7f9d92000640 1 -- 172.21.15.71:0/852879002 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d8c005620 con 0x7f9d9c0d1440 2024-09-16T07:30:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.425+0000 7f9d92000640 1 -- 172.21.15.71:0/852879002 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c035070 con 0x7f9d9c0d1440 2024-09-16T07:30:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.425+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c0a0900 con 0x7f9d9c0d1440 2024-09-16T07:30:54.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.426+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c0a0ec0 con 0x7f9d9c0d1440 2024-09-16T07:30:54.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.427+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9c0d1b90 con 0x7f9d9c0d1440 2024-09-16T07:30:54.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.429+0000 7f9d92000640 1 -- 172.21.15.71:0/852879002 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f9d8c042020 con 0x7f9d9c0d1440 2024-09-16T07:30:54.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.430+0000 7f9d92000640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d7c0799b0 0x7f9d7c07be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:54.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.430+0000 7f9d92000640 1 -- 172.21.15.71:0/852879002 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f9d8c0b9d50 con 0x7f9d9c0d1440 2024-09-16T07:30:54.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.431+0000 7f9d99400640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d7c0799b0 0x7f9d7c07be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:54.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.433+0000 7f9d99400640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d7c0799b0 0x7f9d7c07be70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0b5220 tx=0x7f9d88005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.439+0000 7f9d92000640 1 -- 172.21.15.71:0/852879002 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d8c049030 con 0x7f9d9c0d1440 2024-09-16T07:30:54.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.728+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9d9c15f260 con 0x7f9d9c0d1440 2024-09-16T07:30:54.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.728+0000 7f9d92000640 1 -- 172.21.15.71:0/852879002 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) ==== 80+0+0 (secure 0 0 0) 0x7f9d8c083e50 con 0x7f9d9c0d1440 2024-09-16T07:30:54.728 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:30:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.730+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d7c0799b0 msgr2=0x7f9d7c07be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.730+0000 7f9da0e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d7c0799b0 0x7f9d7c07be70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0b5220 tx=0x7f9d88005f90 comp rx=0 tx=0).stop 2024-09-16T07:30:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d9c0d1440 msgr2=0x7f9d9c09ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d9c0d1440 0x7f9d9c09ecb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c006010 tx=0x7f9d8c003820 comp rx=0 tx=0).stop 2024-09-16T07:30:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 shutdown_connections 2024-09-16T07:30:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d7c0799b0 0x7f9d7c07be70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c0a16c0 0x7f9d9c0cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d9c0d2050 0x7f9d9c15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 --2- 172.21.15.71:0/852879002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d9c0d1440 0x7f9d9c09ecb0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 >> 172.21.15.71:0/852879002 conn(0x7f9d9c0d3fe0 msgr2=0x7f9d9c15f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 shutdown_connections 2024-09-16T07:30:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:54.731+0000 7f9da0e00640 1 -- 172.21.15.71:0/852879002 wait complete. 2024-09-16T07:30:54.747 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:30:54.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:30:54.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:30:54.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:30:54.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:30:54.837 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:30:54.887 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:30:54.888 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:30:54.888 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:30:54.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:30:54.924 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:30:54.924 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:30:54.924 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:30:54.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:30:54.959 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:30:54.959 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:30:54.959 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:30:54.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:30:55.000 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:30:55.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.157+0000 7f3aeb600640 1 -- 172.21.15.71:0/2919630075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae415f5a0 msgr2=0x7f3ae40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:55.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.157+0000 7f3aeb600640 1 --2- 172.21.15.71:0/2919630075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae415f5a0 0x7f3ae40b2440 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4005fd0 tx=0x7f3ad402a540 comp rx=0 tx=0).stop 2024-09-16T07:30:55.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.157+0000 7f3aeb600640 1 -- 172.21.15.71:0/2919630075 shutdown_connections 2024-09-16T07:30:55.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.157+0000 7f3aeb600640 1 --2- 172.21.15.71:0/2919630075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae415f5a0 0x7f3ae40b2440 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.157+0000 7f3aeb600640 1 -- 172.21.15.71:0/2919630075 >> 172.21.15.71:0/2919630075 conn(0x7f3ae40b1950 msgr2=0x7f3ae40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:55.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.157+0000 7f3aeb600640 1 -- 172.21.15.71:0/2919630075 shutdown_connections 2024-09-16T07:30:55.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.157+0000 7f3aeb600640 1 -- 172.21.15.71:0/2919630075 wait complete. 2024-09-16T07:30:55.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.159+0000 7f3aeb600640 1 Processor -- start 2024-09-16T07:30:55.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.159+0000 7f3aeb600640 1 -- start start 2024-09-16T07:30:55.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3aeb600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ae4073fb0 0x7f3ae4146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:55.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3aeb600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ccf00 0x7f3ae4147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3aeb600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae415f5a0 0x7f3ae40b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3aeb600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae406f9f0 con 0x7f3ae40ccf00 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3aeb600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae406f870 con 0x7f3ae415f5a0 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3aeb600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae4074420 con 0x7f3ae4073fb0 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3ae3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ccf00 0x7f3ae4147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3ae3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ccf00 0x7f3ae4147140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34676/0 (socket says 172.21.15.71:34676) 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3ae3e00640 1 -- 172.21.15.71:0/712375875 learned_addr learned my addr 172.21.15.71:0/712375875 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.160+0000 7f3ae8c00640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ae4073fb0 0x7f3ae4146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae9600640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae415f5a0 0x7f3ae40b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae3e00640 1 -- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ae4073fb0 msgr2=0x7f3ae4146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae3e00640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ae4073fb0 0x7f3ae4146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae3e00640 1 -- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae415f5a0 msgr2=0x7f3ae40b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae3e00640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae415f5a0 0x7f3ae40b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae3e00640 1 -- 172.21.15.71:0/712375875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae4147680 con 0x7f3ae40ccf00 2024-09-16T07:30:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae3e00640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ccf00 0x7f3ae4147140 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c450 tx=0x7f3ad000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae0c00640 1 -- 172.21.15.71:0/712375875 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00052d0 con 0x7f3ae40ccf00 2024-09-16T07:30:55.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae0c00640 1 -- 172.21.15.71:0/712375875 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad0005470 con 0x7f3ae40ccf00 2024-09-16T07:30:55.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae41f8f50 con 0x7f3ae40ccf00 2024-09-16T07:30:55.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.161+0000 7f3ae0c00640 1 -- 172.21.15.71:0/712375875 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0012870 con 0x7f3ae40ccf00 2024-09-16T07:30:55.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.162+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae40b3280 con 0x7f3ae40ccf00 2024-09-16T07:30:55.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.163+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae40b2440 con 0x7f3ae40ccf00 2024-09-16T07:30:55.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.164+0000 7f3ae0c00640 1 -- 172.21.15.71:0/712375875 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f3ad00035f0 con 0x7f3ae40ccf00 2024-09-16T07:30:55.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.165+0000 7f3ae0c00640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ac4079930 0x7f3ac407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:55.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.165+0000 7f3ae0c00640 1 -- 172.21.15.71:0/712375875 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f3ad00991a0 con 0x7f3ae40ccf00 2024-09-16T07:30:55.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.165+0000 7f3ae8c00640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ac4079930 0x7f3ac407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:55.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.166+0000 7f3ae8c00640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ac4079930 0x7f3ac407bdf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4033000 tx=0x7f3ad4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:55.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.169+0000 7f3ae0c00640 1 -- 172.21.15.71:0/712375875 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3ad0062930 con 0x7f3ae40ccf00 2024-09-16T07:30:55.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.530+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ae40cf880 con 0x7f3ae40ccf00 2024-09-16T07:30:55.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.531+0000 7f3ae0c00640 1 -- 172.21.15.71:0/712375875 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f3ad0067360 con 0x7f3ae40ccf00 2024-09-16T07:30:55.532 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:55.532 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":47,"btime":"2024-09-16T07:29:54:247057+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/4181257111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":4181257111},{"type":"v1","addr":"172.21.15.177:6836","nonce":4181257111}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2868012570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2868012570},{"type":"v1","addr":"172.21.15.177:6837","nonce":2868012570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6414,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3075140097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3075140097},{"type":"v1","addr":"172.21.15.71:6834","nonce":3075140097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:29:45.618471+0000","modified":"2024-09-16T07:29:48.640333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6417},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6417":{"gid":6417,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1371001821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":1371001821},{"type":"v1","addr":"172.21.15.71:6835","nonce":1371001821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6417,"qdb_cluster":[6417]},"id":5}]} 2024-09-16T07:30:55.532 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 47 2024-09-16T07:30:55.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ac4079930 msgr2=0x7f3ac407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:55.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ac4079930 0x7f3ac407bdf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4033000 tx=0x7f3ad4004730 comp rx=0 tx=0).stop 2024-09-16T07:30:55.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ccf00 msgr2=0x7f3ae4147140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:55.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ccf00 0x7f3ae4147140 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000c450 tx=0x7f3ad000c910 comp rx=0 tx=0).stop 2024-09-16T07:30:55.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 shutdown_connections 2024-09-16T07:30:55.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ac4079930 0x7f3ac407bdf0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ae415f5a0 0x7f3ae40b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ccf00 0x7f3ae4147140 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 --2- 172.21.15.71:0/712375875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ae4073fb0 0x7f3ae4146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.536+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 >> 172.21.15.71:0/712375875 conn(0x7f3ae40b1950 msgr2=0x7f3ae40b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:55.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.537+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 shutdown_connections 2024-09-16T07:30:55.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.537+0000 7f3aeb600640 1 -- 172.21.15.71:0/712375875 wait complete. 2024-09-16T07:30:55.566 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:30:55.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.808+0000 7f4810a00640 1 -- 172.21.15.71:0/549298951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0cf1f0 msgr2=0x7f480c06eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:55.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.808+0000 7f4810a00640 1 --2- 172.21.15.71:0/549298951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0cf1f0 0x7f480c06eec0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4804005fd0 tx=0x7f480402aa90 comp rx=0 tx=0).stop 2024-09-16T07:30:55.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.808+0000 7f4810a00640 1 -- 172.21.15.71:0/549298951 shutdown_connections 2024-09-16T07:30:55.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.808+0000 7f4810a00640 1 --2- 172.21.15.71:0/549298951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0cf1f0 0x7f480c06eec0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.808+0000 7f4810a00640 1 --2- 172.21.15.71:0/549298951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f480c15a280 0x7f480c0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.808+0000 7f4810a00640 1 -- 172.21.15.71:0/549298951 >> 172.21.15.71:0/549298951 conn(0x7f480c0cfa20 msgr2=0x7f480c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:55.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.808+0000 7f4810a00640 1 -- 172.21.15.71:0/549298951 shutdown_connections 2024-09-16T07:30:55.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.808+0000 7f4810a00640 1 -- 172.21.15.71:0/549298951 wait complete. 2024-09-16T07:30:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.810+0000 7f4810a00640 1 Processor -- start 2024-09-16T07:30:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.810+0000 7f4810a00640 1 -- start start 2024-09-16T07:30:55.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f4810a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0725d0 0x7f480c157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:55.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f4810a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f480c0cf1f0 0x7f480c0c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:55.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f4810a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c15a280 0x7f480c073110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:55.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f4810a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f480c072a10 con 0x7f480c0725d0 2024-09-16T07:30:55.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f4810a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0d0970 con 0x7f480c15a280 2024-09-16T07:30:55.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f4810a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f480c072b90 con 0x7f480c0cf1f0 2024-09-16T07:30:55.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f480b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0725d0 0x7f480c157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:55.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f480b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0725d0 0x7f480c157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34692/0 (socket says 172.21.15.71:34692) 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f480b400640 1 -- 172.21.15.71:0/2782485010 learned_addr learned my addr 172.21.15.71:0/2782485010 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f480aa00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f480c0cf1f0 0x7f480c0c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.811+0000 7f480be00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c15a280 0x7f480c073110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f480b400640 1 -- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f480c0cf1f0 msgr2=0x7f480c0c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f480b400640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f480c0cf1f0 0x7f480c0c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f480b400640 1 -- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c15a280 msgr2=0x7f480c073110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f480b400640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c15a280 0x7f480c073110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f480b400640 1 -- 172.21.15.71:0/2782485010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c0c9980 con 0x7f480c0725d0 2024-09-16T07:30:55.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f480aa00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f480c0cf1f0 0x7f480c0c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:30:55.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f480b400640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0725d0 0x7f480c157870 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00ea70 tx=0x7f47fc00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:55.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f4803400640 1 -- 172.21.15.71:0/2782485010 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00cd40 con 0x7f480c0725d0 2024-09-16T07:30:55.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f4803400640 1 -- 172.21.15.71:0/2782485010 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47fc005020 con 0x7f480c0725d0 2024-09-16T07:30:55.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c155a30 con 0x7f480c0725d0 2024-09-16T07:30:55.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f4803400640 1 -- 172.21.15.71:0/2782485010 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc005470 con 0x7f480c0725d0 2024-09-16T07:30:55.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.812+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c155ff0 con 0x7f480c0725d0 2024-09-16T07:30:55.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.813+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480c06f570 con 0x7f480c0725d0 2024-09-16T07:30:55.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.814+0000 7f4803400640 1 -- 172.21.15.71:0/2782485010 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f47fc0047a0 con 0x7f480c0725d0 2024-09-16T07:30:55.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.815+0000 7f4803400640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f47ec079930 0x7f47ec07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:55.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.816+0000 7f4803400640 1 -- 172.21.15.71:0/2782485010 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f47fc0999d0 con 0x7f480c0725d0 2024-09-16T07:30:55.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.816+0000 7f480aa00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f47ec079930 0x7f47ec07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.817+0000 7f480aa00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f47ec079930 0x7f47ec07bdf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f480c0c6d30 tx=0x7f47f8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:55.819+0000 7f4803400640 1 -- 172.21.15.71:0/2782485010 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f47fc063440 con 0x7f480c0725d0 2024-09-16T07:30:56.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.043+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f480c1e8280 con 0x7f480c0725d0 2024-09-16T07:30:56.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.044+0000 7f4803400640 1 -- 172.21.15.71:0/2782485010 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f47fc067e70 con 0x7f480c0725d0 2024-09-16T07:30:56.043 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:56.044 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":47,"btime":"2024-09-16T07:29:54:247057+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/4181257111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":4181257111},{"type":"v1","addr":"172.21.15.177:6836","nonce":4181257111}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2868012570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2868012570},{"type":"v1","addr":"172.21.15.177:6837","nonce":2868012570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6414,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3075140097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3075140097},{"type":"v1","addr":"172.21.15.71:6834","nonce":3075140097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:29:45.618471+0000","modified":"2024-09-16T07:29:48.640333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6417},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6417":{"gid":6417,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1371001821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":1371001821},{"type":"v1","addr":"172.21.15.71:6835","nonce":1371001821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6417,"qdb_cluster":[6417]},"id":5}]} 2024-09-16T07:30:56.044 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 47 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f47ec079930 msgr2=0x7f47ec07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f47ec079930 0x7f47ec07bdf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f480c0c6d30 tx=0x7f47f8009040 comp rx=0 tx=0).stop 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0725d0 msgr2=0x7f480c157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0725d0 0x7f480c157870 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00ea70 tx=0x7f47fc00ef30 comp rx=0 tx=0).stop 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 shutdown_connections 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f47ec079930 0x7f47ec07bdf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c15a280 0x7f480c073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f480c0cf1f0 0x7f480c0c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 --2- 172.21.15.71:0/2782485010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f480c0725d0 0x7f480c157870 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 >> 172.21.15.71:0/2782485010 conn(0x7f480c0cfa20 msgr2=0x7f480c0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 shutdown_connections 2024-09-16T07:30:56.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.046+0000 7f4810a00640 1 -- 172.21.15.71:0/2782485010 wait complete. 2024-09-16T07:30:56.063 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:30:56.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.225+0000 7f6ad0e00640 1 -- 172.21.15.71:0/1150808361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 msgr2=0x7f6ab0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.225+0000 7f6ad0e00640 1 --2- 172.21.15.71:0/1150808361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 0x7f6ab0003ec0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009b50 tx=0x7f6ab0005830 comp rx=0 tx=0).stop 2024-09-16T07:30:56.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.226+0000 7f6ad0e00640 1 -- 172.21.15.71:0/1150808361 shutdown_connections 2024-09-16T07:30:56.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.226+0000 7f6ad0e00640 1 --2- 172.21.15.71:0/1150808361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 0x7f6ab0003ec0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.226+0000 7f6ad0e00640 1 -- 172.21.15.71:0/1150808361 >> 172.21.15.71:0/1150808361 conn(0x7f6acc0b1950 msgr2=0x7f6acc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:56.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.226+0000 7f6ad0e00640 1 -- 172.21.15.71:0/1150808361 shutdown_connections 2024-09-16T07:30:56.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.226+0000 7f6ad0e00640 1 -- 172.21.15.71:0/1150808361 wait complete. 2024-09-16T07:30:56.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.227+0000 7f6ad0e00640 1 Processor -- start 2024-09-16T07:30:56.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.227+0000 7f6ad0e00640 1 -- start start 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.228+0000 7f6ad0e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 0x7f6acc0d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.228+0000 7f6ad0e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6acc158980 0x7f6acc0ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.228+0000 7f6ad0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6acc153290 0x7f6acc153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.228+0000 7f6ad0e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc06f2e0 con 0x7f6acc153290 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.228+0000 7f6ad0e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc06f160 con 0x7f6ab0003aa0 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.228+0000 7f6ad0e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6acc06f460 con 0x7f6acc158980 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.228+0000 7f6aca800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6acc153290 0x7f6acc153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6aca800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6acc153290 0x7f6acc153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34714/0 (socket says 172.21.15.71:34714) 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6aca800640 1 -- 172.21.15.71:0/4004007274 learned_addr learned my addr 172.21.15.71:0/4004007274 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:56.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6aca800640 1 -- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6acc158980 msgr2=0x7f6acc0ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6ac9e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 0x7f6acc0d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6ac9400640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6acc158980 0x7f6acc0ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6aca800640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6acc158980 0x7f6acc0ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6aca800640 1 -- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 msgr2=0x7f6acc0d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6aca800640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 0x7f6acc0d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6aca800640 1 -- 172.21.15.71:0/4004007274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6acc1f4ab0 con 0x7f6acc153290 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6ac9e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 0x7f6acc0d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6ac9400640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6acc158980 0x7f6acc0ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6aca800640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6acc153290 0x7f6acc153710 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0af050 tx=0x7f6ac4006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6ac2000640 1 -- 172.21.15.71:0/4004007274 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4011900 con 0x7f6acc153290 2024-09-16T07:30:56.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6ac2000640 1 -- 172.21.15.71:0/4004007274 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ac4018070 con 0x7f6acc153290 2024-09-16T07:30:56.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6ac2000640 1 -- 172.21.15.71:0/4004007274 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4011d40 con 0x7f6acc153290 2024-09-16T07:30:56.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.229+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6acc1f4d90 con 0x7f6acc153290 2024-09-16T07:30:56.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.230+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6acc1f5350 con 0x7f6acc153290 2024-09-16T07:30:56.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.231+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6acc06f9c0 con 0x7f6acc153290 2024-09-16T07:30:56.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.232+0000 7f6ac2000640 1 -- 172.21.15.71:0/4004007274 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f6ac401d460 con 0x7f6acc153290 2024-09-16T07:30:56.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.233+0000 7f6ac2000640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6aac079960 0x7f6aac07be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:56.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.233+0000 7f6ac2000640 1 -- 172.21.15.71:0/4004007274 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f6ac4099d10 con 0x7f6acc153290 2024-09-16T07:30:56.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.233+0000 7f6ac9e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6aac079960 0x7f6aac07be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.234+0000 7f6ac9e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6aac079960 0x7f6aac07be20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6abc035000 tx=0x7f6abc0046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.237+0000 7f6ac2000640 1 -- 172.21.15.71:0/4004007274 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6ac4063dd0 con 0x7f6acc153290 2024-09-16T07:30:56.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.422+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6acc09ddf0 con 0x7f6acc153290 2024-09-16T07:30:56.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.423+0000 7f6ac2000640 1 -- 172.21.15.71:0/4004007274 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16931 (secure 0 0 0) 0x7f6ac40636d0 con 0x7f6acc153290 2024-09-16T07:30:56.423 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:56.423 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":80,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:30:51.396445+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:29:43.380127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T07:29:44.588310+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":80,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"11.39","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:30:56.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6aac079960 msgr2=0x7f6aac07be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6aac079960 0x7f6aac07be20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6abc035000 tx=0x7f6abc0046c0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6acc153290 msgr2=0x7f6acc153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6acc153290 0x7f6acc153710 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0af050 tx=0x7f6ac4006430 comp rx=0 tx=0).stop 2024-09-16T07:30:56.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 shutdown_connections 2024-09-16T07:30:56.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6aac079960 0x7f6aac07be20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6acc153290 0x7f6acc153710 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6acc158980 0x7f6acc0ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 --2- 172.21.15.71:0/4004007274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0003aa0 0x7f6acc0d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 >> 172.21.15.71:0/4004007274 conn(0x7f6acc0b1950 msgr2=0x7f6acc073200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:56.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 shutdown_connections 2024-09-16T07:30:56.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.425+0000 7f6ad0e00640 1 -- 172.21.15.71:0/4004007274 wait complete. 2024-09-16T07:30:56.441 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:30:56.442 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:30:56.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.600+0000 7f0ae9000640 1 -- 172.21.15.71:0/639051321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae40725d0 msgr2=0x7f0ae40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.600+0000 7f0ae9000640 1 --2- 172.21.15.71:0/639051321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae40725d0 0x7f0ae40b28f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4074a80 tx=0x7f0ad002a540 comp rx=0 tx=0).stop 2024-09-16T07:30:56.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.600+0000 7f0ae9000640 1 -- 172.21.15.71:0/639051321 shutdown_connections 2024-09-16T07:30:56.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.600+0000 7f0ae9000640 1 --2- 172.21.15.71:0/639051321 >> v2:172.21.15.177:3300/0 conn(0x7f0ae40d0540 0x7f0ae406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.600+0000 7f0ae9000640 1 --2- 172.21.15.71:0/639051321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae40725d0 0x7f0ae40b28f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.600+0000 7f0ae9000640 1 -- 172.21.15.71:0/639051321 >> 172.21.15.71:0/639051321 conn(0x7f0ae40cfa20 msgr2=0x7f0ae40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:56.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.600+0000 7f0ae9000640 1 -- 172.21.15.71:0/639051321 shutdown_connections 2024-09-16T07:30:56.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.600+0000 7f0ae9000640 1 -- 172.21.15.71:0/639051321 wait complete. 2024-09-16T07:30:56.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.601+0000 7f0ae9000640 1 Processor -- start 2024-09-16T07:30:56.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.602+0000 7f0ae9000640 1 -- start start 2024-09-16T07:30:56.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.602+0000 7f0ae9000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae40d0540 0x7f0ae40cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:56.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.602+0000 7f0ae9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae415bc30 0x7f0ae4150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:56.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae9000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae40cf3d0 0x7f0ae4150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae9000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4074660 con 0x7f0ae415bc30 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae9000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae406f860 con 0x7f0ae40d0540 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae9000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40ce0a0 con 0x7f0ae40cf3d0 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae415bc30 0x7f0ae4150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae415bc30 0x7f0ae4150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34728/0 (socket says 172.21.15.71:34728) 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 -- 172.21.15.71:0/1031860354 learned_addr learned my addr 172.21.15.71:0/1031860354 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae3400640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae40d0540 0x7f0ae40cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae3e00640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae40cf3d0 0x7f0ae4150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 -- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae40cf3d0 msgr2=0x7f0ae4150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae40cf3d0 0x7f0ae4150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 -- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae40d0540 msgr2=0x7f0ae40cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae40d0540 0x7f0ae40cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 -- 172.21.15.71:0/1031860354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae40cd770 con 0x7f0ae415bc30 2024-09-16T07:30:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.603+0000 7f0ae2a00640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae415bc30 0x7f0ae4150020 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0ad002ab30 tx=0x7f0ad0032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.604+0000 7f0adb400640 1 -- 172.21.15.71:0/1031860354 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0040070 con 0x7f0ae415bc30 2024-09-16T07:30:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.604+0000 7f0adb400640 1 -- 172.21.15.71:0/1031860354 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ad0005310 con 0x7f0ae415bc30 2024-09-16T07:30:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.604+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae40cda50 con 0x7f0ae415bc30 2024-09-16T07:30:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.604+0000 7f0adb400640 1 -- 172.21.15.71:0/1031860354 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0005c90 con 0x7f0ae415bc30 2024-09-16T07:30:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.604+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae41f4210 con 0x7f0ae415bc30 2024-09-16T07:30:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.605+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae406fdf0 con 0x7f0ae415bc30 2024-09-16T07:30:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.606+0000 7f0adb400640 1 -- 172.21.15.71:0/1031860354 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f0ad0039040 con 0x7f0ae415bc30 2024-09-16T07:30:56.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.607+0000 7f0adb400640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac4079930 0x7f0ac407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:56.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.608+0000 7f0adb400640 1 -- 172.21.15.71:0/1031860354 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f0ad00ba7a0 con 0x7f0ae415bc30 2024-09-16T07:30:56.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.608+0000 7f0ae3400640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac4079930 0x7f0ac407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:56.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.608+0000 7f0ae3400640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac4079930 0x7f0ac407bdf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4001610 tx=0x7f0ad4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:56.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.612+0000 7f0adb400640 1 -- 172.21.15.71:0/1031860354 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0ad0084860 con 0x7f0ae415bc30 2024-09-16T07:30:56.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.829+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ae40b2a90 con 0x7f0ae415bc30 2024-09-16T07:30:56.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.830+0000 7f0adb400640 1 -- 172.21.15.71:0/1031860354 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f0ad0084160 con 0x7f0ae415bc30 2024-09-16T07:30:56.831 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:56.831 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":47,"btime":"2024-09-16T07:29:54:247057+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/4181257111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":4181257111},{"type":"v1","addr":"172.21.15.177:6836","nonce":4181257111}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2868012570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2868012570},{"type":"v1","addr":"172.21.15.177:6837","nonce":2868012570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6414,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3075140097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3075140097},{"type":"v1","addr":"172.21.15.71:6834","nonce":3075140097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:29:45.618471+0000","modified":"2024-09-16T07:29:48.640333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6417},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6417":{"gid":6417,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1371001821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":1371001821},{"type":"v1","addr":"172.21.15.71:6835","nonce":1371001821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6417,"qdb_cluster":[6417]},"id":5}]} 2024-09-16T07:30:56.831 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 47 2024-09-16T07:30:56.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac4079930 msgr2=0x7f0ac407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac4079930 0x7f0ac407bdf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4001610 tx=0x7f0ad4009040 comp rx=0 tx=0).stop 2024-09-16T07:30:56.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae415bc30 msgr2=0x7f0ae4150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:56.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae415bc30 0x7f0ae4150020 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0ad002ab30 tx=0x7f0ad0032c10 comp rx=0 tx=0).stop 2024-09-16T07:30:56.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 shutdown_connections 2024-09-16T07:30:56.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac4079930 0x7f0ac407bdf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae40cf3d0 0x7f0ae4150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ae415bc30 0x7f0ae4150020 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 --2- 172.21.15.71:0/1031860354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae40d0540 0x7f0ae40cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:56.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 >> 172.21.15.71:0/1031860354 conn(0x7f0ae40cfa20 msgr2=0x7f0ae406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:56.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.832+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 shutdown_connections 2024-09-16T07:30:56.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:56.833+0000 7f0ae9000640 1 -- 172.21.15.71:0/1031860354 wait complete. 2024-09-16T07:30:56.848 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:30:57.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.004+0000 7f6b3f200640 1 -- 172.21.15.71:0/2576296056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b400725d0 msgr2=0x7f6b400ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.004+0000 7f6b3f200640 1 --2- 172.21.15.71:0/2576296056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b400725d0 0x7f6b400ceae0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6b400b2530 tx=0x7f6b2c02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.004+0000 7f6b3f200640 1 -- 172.21.15.71:0/2576296056 shutdown_connections 2024-09-16T07:30:57.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.004+0000 7f6b3f200640 1 --2- 172.21.15.71:0/2576296056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b400725d0 0x7f6b400ceae0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.004+0000 7f6b3f200640 1 --2- 172.21.15.71:0/2576296056 >> v2:172.21.15.177:3300/0 conn(0x7f6b4015a280 0x7f6b400ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.004+0000 7f6b3f200640 1 -- 172.21.15.71:0/2576296056 >> 172.21.15.71:0/2576296056 conn(0x7f6b400cfa20 msgr2=0x7f6b400cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:57.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.005+0000 7f6b3f200640 1 -- 172.21.15.71:0/2576296056 shutdown_connections 2024-09-16T07:30:57.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.005+0000 7f6b3f200640 1 -- 172.21.15.71:0/2576296056 wait complete. 2024-09-16T07:30:57.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.006+0000 7f6b3f200640 1 Processor -- start 2024-09-16T07:30:57.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.006+0000 7f6b3f200640 1 -- start start 2024-09-16T07:30:57.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b400725d0 0x7f6b4014cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3f200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b40072a40 0x7f6b400c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4015a280 0x7f6b400b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b400d08e0 con 0x7f6b4015a280 2024-09-16T07:30:57.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6b400d0760 con 0x7f6b400725d0 2024-09-16T07:30:57.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3f200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4006f5d0 con 0x7f6b40072a40 2024-09-16T07:30:57.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4015a280 0x7f6b400b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4015a280 0x7f6b400b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34754/0 (socket says 172.21.15.71:34754) 2024-09-16T07:30:57.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.007+0000 7f6b3e800640 1 -- 172.21.15.71:0/4144732430 learned_addr learned my addr 172.21.15.71:0/4144732430 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:57.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3de00640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b400725d0 0x7f6b4014cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3d400640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b40072a40 0x7f6b400c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3e800640 1 -- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b40072a40 msgr2=0x7f6b400c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3e800640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b40072a40 0x7f6b400c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3e800640 1 -- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b400725d0 msgr2=0x7f6b4014cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3e800640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b400725d0 0x7f6b4014cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3e800640 1 -- 172.21.15.71:0/4144732430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b401f3420 con 0x7f6b4015a280 2024-09-16T07:30:57.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3de00640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b400725d0 0x7f6b4014cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:30:57.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3e800640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4015a280 0x7f6b400b48d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800c7a0 tx=0x7f6b3800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:57.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b36000640 1 -- 172.21.15.71:0/4144732430 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38017070 con 0x7f6b4015a280 2024-09-16T07:30:57.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b36000640 1 -- 172.21.15.71:0/4144732430 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b38005070 con 0x7f6b4015a280 2024-09-16T07:30:57.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b400c92c0 con 0x7f6b4015a280 2024-09-16T07:30:57.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.008+0000 7f6b36000640 1 -- 172.21.15.71:0/4144732430 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380054c0 con 0x7f6b4015a280 2024-09-16T07:30:57.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.009+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b401f2960 con 0x7f6b4015a280 2024-09-16T07:30:57.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.010+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b400ca070 con 0x7f6b4015a280 2024-09-16T07:30:57.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.011+0000 7f6b36000640 1 -- 172.21.15.71:0/4144732430 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f6b3801a070 con 0x7f6b4015a280 2024-09-16T07:30:57.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.012+0000 7f6b36000640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b20079930 0x7f6b2007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.012+0000 7f6b36000640 1 -- 172.21.15.71:0/4144732430 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f6b38098e20 con 0x7f6b4015a280 2024-09-16T07:30:57.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.012+0000 7f6b3de00640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b20079930 0x7f6b2007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.013+0000 7f6b3de00640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b20079930 0x7f6b2007bdf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009b80 tx=0x7f6b30005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:57.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.016+0000 7f6b36000640 1 -- 172.21.15.71:0/4144732430 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6b38062ba0 con 0x7f6b4015a280 2024-09-16T07:30:57.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.238+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b40073910 con 0x7f6b4015a280 2024-09-16T07:30:57.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.238+0000 7f6b36000640 1 -- 172.21.15.71:0/4144732430 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f6b380624a0 con 0x7f6b4015a280 2024-09-16T07:30:57.239 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:57.239 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":47,"btime":"2024-09-16T07:29:54:247057+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/4181257111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":4181257111},{"type":"v1","addr":"172.21.15.177:6836","nonce":4181257111}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2868012570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2868012570},{"type":"v1","addr":"172.21.15.177:6837","nonce":2868012570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6414,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3075140097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3075140097},{"type":"v1","addr":"172.21.15.71:6834","nonce":3075140097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:29:45.618471+0000","modified":"2024-09-16T07:29:48.640333+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6417},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6417":{"gid":6417,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1371001821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":1371001821},{"type":"v1","addr":"172.21.15.71:6835","nonce":1371001821}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6417,"qdb_cluster":[6417]},"id":5}]} 2024-09-16T07:30:57.239 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 47 2024-09-16T07:30:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.240+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b20079930 msgr2=0x7f6b2007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.240+0000 7f6b3f200640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b20079930 0x7f6b2007bdf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009b80 tx=0x7f6b30005f90 comp rx=0 tx=0).stop 2024-09-16T07:30:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4015a280 msgr2=0x7f6b400b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4015a280 0x7f6b400b48d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800c7a0 tx=0x7f6b3800cc60 comp rx=0 tx=0).stop 2024-09-16T07:30:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 shutdown_connections 2024-09-16T07:30:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b20079930 0x7f6b2007bdf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b4015a280 0x7f6b400b48d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b40072a40 0x7f6b400c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 --2- 172.21.15.71:0/4144732430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b400725d0 0x7f6b4014cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 >> 172.21.15.71:0/4144732430 conn(0x7f6b400cfa20 msgr2=0x7f6b400ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:57.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 shutdown_connections 2024-09-16T07:30:57.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.241+0000 7f6b3f200640 1 -- 172.21.15.71:0/4144732430 wait complete. 2024-09-16T07:30:57.256 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:30:57.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.414+0000 7f711b200640 1 -- 172.21.15.71:0/3157704996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c15bc30 msgr2=0x7f711c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.414+0000 7f711b200640 1 --2- 172.21.15.71:0/3157704996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c15bc30 0x7f711c0b28f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f710c005fd0 tx=0x7f710c02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.415+0000 7f711b200640 1 -- 172.21.15.71:0/3157704996 shutdown_connections 2024-09-16T07:30:57.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.415+0000 7f711b200640 1 --2- 172.21.15.71:0/3157704996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711c0725d0 0x7f711c073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.415+0000 7f711b200640 1 --2- 172.21.15.71:0/3157704996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c15bc30 0x7f711c0b28f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.415+0000 7f711b200640 1 -- 172.21.15.71:0/3157704996 >> 172.21.15.71:0/3157704996 conn(0x7f711c0cfa20 msgr2=0x7f711c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:57.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.415+0000 7f711b200640 1 -- 172.21.15.71:0/3157704996 shutdown_connections 2024-09-16T07:30:57.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.415+0000 7f711b200640 1 -- 172.21.15.71:0/3157704996 wait complete. 2024-09-16T07:30:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.416+0000 7f711b200640 1 Processor -- start 2024-09-16T07:30:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.416+0000 7f711b200640 1 -- start start 2024-09-16T07:30:57.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.417+0000 7f711b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711c0d0540 0x7f711c158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.417+0000 7f711b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711c15bc30 0x7f711c159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.417+0000 7f711b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c14e300 0x7f711c159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.417+0000 7f711b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f711c074610 con 0x7f711c14e300 2024-09-16T07:30:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.417+0000 7f711b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f711c074490 con 0x7f711c0d0540 2024-09-16T07:30:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.417+0000 7f711b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f711c074790 con 0x7f711c15bc30 2024-09-16T07:30:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f711a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c14e300 0x7f711c159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f7119400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711c15bc30 0x7f711c159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f7119e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711c0d0540 0x7f711c158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f7119400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711c15bc30 0x7f711c159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:54026/0 (socket says 172.21.15.71:54026) 2024-09-16T07:30:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f7119400640 1 -- 172.21.15.71:0/2884914260 learned_addr learned my addr 172.21.15.71:0/2884914260 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f711a800640 1 -- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711c15bc30 msgr2=0x7f711c159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f711a800640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711c15bc30 0x7f711c159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f711a800640 1 -- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711c0d0540 msgr2=0x7f711c158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f711a800640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711c0d0540 0x7f711c158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f711a800640 1 -- 172.21.15.71:0/2884914260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c14ced0 con 0x7f711c14e300 2024-09-16T07:30:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.418+0000 7f711a800640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c14e300 0x7f711c159e50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f711400c6b0 tx=0x7f711400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.419+0000 7f7112000640 1 -- 172.21.15.71:0/2884914260 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114003b70 con 0x7f711c14e300 2024-09-16T07:30:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.419+0000 7f7112000640 1 -- 172.21.15.71:0/2884914260 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7114003d10 con 0x7f711c14e300 2024-09-16T07:30:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.419+0000 7f7112000640 1 -- 172.21.15.71:0/2884914260 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114005330 con 0x7f711c14e300 2024-09-16T07:30:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.419+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c14d1b0 con 0x7f711c14e300 2024-09-16T07:30:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.419+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c14d770 con 0x7f711c14e300 2024-09-16T07:30:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.420+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c06fc60 con 0x7f711c14e300 2024-09-16T07:30:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.421+0000 7f7112000640 1 -- 172.21.15.71:0/2884914260 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f71140054d0 con 0x7f711c14e300 2024-09-16T07:30:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.422+0000 7f7112000640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70fc079930 0x7f70fc07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.422+0000 7f7112000640 1 -- 172.21.15.71:0/2884914260 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f7114099420 con 0x7f711c14e300 2024-09-16T07:30:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.422+0000 7f7119e00640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70fc079930 0x7f70fc07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.423+0000 7f7119e00640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70fc079930 0x7f70fc07bdf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f710c033000 tx=0x7f710c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.426+0000 7f7112000640 1 -- 172.21.15.71:0/2884914260 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7114062b50 con 0x7f711c14e300 2024-09-16T07:30:57.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.612+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f711c06ad00 con 0x7f711c14e300 2024-09-16T07:30:57.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.613+0000 7f7112000640 1 -- 172.21.15.71:0/2884914260 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16931 (secure 0 0 0) 0x7f7114067580 con 0x7f711c14e300 2024-09-16T07:30:57.614 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:30:57.614 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":80,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:30:51.396445+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:29:43.380127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T07:29:44.588310+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":80,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"11.39","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:30:57.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70fc079930 msgr2=0x7f70fc07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70fc079930 0x7f70fc07bdf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f710c033000 tx=0x7f710c004730 comp rx=0 tx=0).stop 2024-09-16T07:30:57.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c14e300 msgr2=0x7f711c159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c14e300 0x7f711c159e50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f711400c6b0 tx=0x7f711400cb70 comp rx=0 tx=0).stop 2024-09-16T07:30:57.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 shutdown_connections 2024-09-16T07:30:57.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70fc079930 0x7f70fc07bdf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f711c14e300 0x7f711c159e50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711c15bc30 0x7f711c159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 --2- 172.21.15.71:0/2884914260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711c0d0540 0x7f711c158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 >> 172.21.15.71:0/2884914260 conn(0x7f711c0cfa20 msgr2=0x7f711c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:57.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 shutdown_connections 2024-09-16T07:30:57.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.616+0000 7f711b200640 1 -- 172.21.15.71:0/2884914260 wait complete. 2024-09-16T07:30:57.632 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:30:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 -- 172.21.15.71:0/691387724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 msgr2=0x7f97d0073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 --2- 172.21.15.71:0/691387724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 0x7f97d0073fb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f97d0075130 tx=0x7f97c802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 -- 172.21.15.71:0/691387724 shutdown_connections 2024-09-16T07:30:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 --2- 172.21.15.71:0/691387724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 0x7f97d0073fb0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 --2- 172.21.15.71:0/691387724 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97d00ccf00 0x7f97d00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 --2- 172.21.15.71:0/691387724 >> v2:172.21.15.177:3300/0 conn(0x7f97d015f5a0 0x7f97d00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 -- 172.21.15.71:0/691387724 >> 172.21.15.71:0/691387724 conn(0x7f97d00b1950 msgr2=0x7f97d00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:57.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 -- 172.21.15.71:0/691387724 shutdown_connections 2024-09-16T07:30:57.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.791+0000 7f97d5c00640 1 -- 172.21.15.71:0/691387724 wait complete. 2024-09-16T07:30:57.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.793+0000 7f97d5c00640 1 Processor -- start 2024-09-16T07:30:57.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.793+0000 7f97d5c00640 1 -- start start 2024-09-16T07:30:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97d5c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 0x7f97d01569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97d5c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97d00ccf00 0x7f97d00b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97d5c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97d015f5a0 0x7f97d014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97d5c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f97d006f160 con 0x7f97d00ad270 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97d5c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0074780 con 0x7f97d015f5a0 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97d5c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f97d006f2e0 con 0x7f97d00ccf00 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 0x7f97d01569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 0x7f97d01569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34766/0 (socket says 172.21.15.71:34766) 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cea00640 1 -- 172.21.15.71:0/2478760506 learned_addr learned my addr 172.21.15.71:0/2478760506 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97ce000640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97d00ccf00 0x7f97d00b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cf400640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97d015f5a0 0x7f97d014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cea00640 1 -- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97d00ccf00 msgr2=0x7f97d00b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cea00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97d00ccf00 0x7f97d00b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cea00640 1 -- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97d015f5a0 msgr2=0x7f97d014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cea00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97d015f5a0 0x7f97d014ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97cea00640 1 -- 172.21.15.71:0/2478760506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d01f8380 con 0x7f97d00ad270 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.794+0000 7f97ce000640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97d00ccf00 0x7f97d00b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.795+0000 7f97cea00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 0x7f97d01569e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f97c000ea20 tx=0x7f97c000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.795+0000 7f97c6a00640 1 -- 172.21.15.71:0/2478760506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c000cde0 con 0x7f97d00ad270 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.795+0000 7f97c6a00640 1 -- 172.21.15.71:0/2478760506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97c0011560 con 0x7f97d00ad270 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.795+0000 7f97c6a00640 1 -- 172.21.15.71:0/2478760506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0011990 con 0x7f97d00ad270 2024-09-16T07:30:57.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.795+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d01f8660 con 0x7f97d00ad270 2024-09-16T07:30:57.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.795+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d01f8c20 con 0x7f97d00ad270 2024-09-16T07:30:57.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.796+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d006f870 con 0x7f97d00ad270 2024-09-16T07:30:57.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.797+0000 7f97c6a00640 1 -- 172.21.15.71:0/2478760506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f97c0011b30 con 0x7f97d00ad270 2024-09-16T07:30:57.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.798+0000 7f97c6a00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97b0079930 0x7f97b007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:57.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.798+0000 7f97c6a00640 1 -- 172.21.15.71:0/2478760506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5851+0+0 (secure 0 0 0) 0x7f97c009a4a0 con 0x7f97d00ad270 2024-09-16T07:30:57.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.799+0000 7f97ce000640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97b0079930 0x7f97b007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:57.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.800+0000 7f97ce000640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97b0079930 0x7f97b007bdf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00bbe0 tx=0x7f97bc005ef0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:57.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:57.802+0000 7f97c6a00640 1 -- 172.21.15.71:0/2478760506 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f97c0016030 con 0x7f97d00ad270 2024-09-16T07:30:58.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.023+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f97d009d8a0 con 0x7f97d00ad270 2024-09-16T07:30:58.023 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:30:58.023+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:30:58.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.025+0000 7f97c6a00640 1 -- 172.21.15.71:0/2478760506 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f97c0011e20 con 0x7f97d00ad270 2024-09-16T07:30:58.025 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:30:58.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97b0079930 msgr2=0x7f97b007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:58.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97b0079930 0x7f97b007bdf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00bbe0 tx=0x7f97bc005ef0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 msgr2=0x7f97d01569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:58.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 0x7f97d01569e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f97c000ea20 tx=0x7f97c000eee0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 shutdown_connections 2024-09-16T07:30:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97b0079930 0x7f97b007bdf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97d015f5a0 0x7f97d014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97d00ccf00 0x7f97d00b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 --2- 172.21.15.71:0/2478760506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97d00ad270 0x7f97d01569e0 secure :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f97c000ea20 tx=0x7f97c000eee0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 >> 172.21.15.71:0/2478760506 conn(0x7f97d00b1950 msgr2=0x7f97d009cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.029+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 shutdown_connections 2024-09-16T07:30:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.030+0000 7f97d5c00640 1 -- 172.21.15.71:0/2478760506 wait complete. 2024-09-16T07:30:58.046 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:30:58.094 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:30:58.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.210+0000 7f8853a00640 1 -- 172.21.15.71:0/3404229828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 msgr2=0x7f884c0734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:58.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.210+0000 7f8853a00640 1 --2- 172.21.15.71:0/3404229828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 0x7f884c0734a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8834009900 tx=0x7f883402a610 comp rx=0 tx=0).stop 2024-09-16T07:30:58.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.210+0000 7f8853a00640 1 -- 172.21.15.71:0/3404229828 shutdown_connections 2024-09-16T07:30:58.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.210+0000 7f8853a00640 1 --2- 172.21.15.71:0/3404229828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f884c06f250 0x7f884c06f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.210+0000 7f8853a00640 1 --2- 172.21.15.71:0/3404229828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 0x7f884c0734a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.210+0000 7f8853a00640 1 -- 172.21.15.71:0/3404229828 >> 172.21.15.71:0/3404229828 conn(0x7f884c0b2130 msgr2=0x7f884c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.210+0000 7f8853a00640 1 -- 172.21.15.71:0/3404229828 shutdown_connections 2024-09-16T07:30:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.210+0000 7f8853a00640 1 -- 172.21.15.71:0/3404229828 wait complete. 2024-09-16T07:30:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.212+0000 7f8853a00640 1 Processor -- start 2024-09-16T07:30:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.212+0000 7f8853a00640 1 -- start start 2024-09-16T07:30:58.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8853a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f884c06f250 0x7f884c13dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:58.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8853a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 0x7f884c13e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:58.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8853a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f884c151cb0 0x7f884c1f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:58.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8853a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f884c06fc20 con 0x7f884c151cb0 2024-09-16T07:30:58.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8853a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f884c074280 con 0x7f884c073030 2024-09-16T07:30:58.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8853a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f884c06fda0 con 0x7f884c06f250 2024-09-16T07:30:58.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8851000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f884c06f250 0x7f884c13dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:58.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8843e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 0x7f884c13e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8851a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f884c151cb0 0x7f884c1f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8851000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f884c06f250 0x7f884c13dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:54086/0 (socket says 172.21.15.71:54086) 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.213+0000 7f8851000640 1 -- 172.21.15.71:0/4272335531 learned_addr learned my addr 172.21.15.71:0/4272335531 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8851000640 1 -- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 msgr2=0x7f884c13e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8851000640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 0x7f884c13e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8851000640 1 -- 172.21.15.71:0/4272335531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f884c151cb0 msgr2=0x7f884c1f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8851000640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f884c151cb0 0x7f884c1f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8851000640 1 -- 172.21.15.71:0/4272335531 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c13e870 con 0x7f884c06f250 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8843e00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 0x7f884c13e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8851a00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f884c151cb0 0x7f884c1f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8851000640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f884c06f250 0x7f884c13dc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c3a0 tx=0x7f883c00c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.214+0000 7f8849600640 1 -- 172.21.15.71:0/4272335531 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c017070 con 0x7f884c06f250 2024-09-16T07:30:58.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.215+0000 7f8849600640 1 -- 172.21.15.71:0/4272335531 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f883c0053c0 con 0x7f884c06f250 2024-09-16T07:30:58.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.215+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c1f8920 con 0x7f884c06f250 2024-09-16T07:30:58.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.215+0000 7f8849600640 1 -- 172.21.15.71:0/4272335531 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c003850 con 0x7f884c06f250 2024-09-16T07:30:58.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.215+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c152eb0 con 0x7f884c06f250 2024-09-16T07:30:58.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.216+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c070ab0 con 0x7f884c06f250 2024-09-16T07:30:58.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.217+0000 7f8849600640 1 -- 172.21.15.71:0/4272335531 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f883c01a070 con 0x7f884c06f250 2024-09-16T07:30:58.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.218+0000 7f8849600640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8828079930 0x7f882807bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:58.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.218+0000 7f8849600640 1 -- 172.21.15.71:0/4272335531 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(82..82 src has 1..82) ==== 5909+0+0 (secure 0 0 0) 0x7f883c099190 con 0x7f884c06f250 2024-09-16T07:30:58.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.219+0000 7f8843e00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8828079930 0x7f882807bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:58.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.219+0000 7f8843e00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8828079930 0x7f882807bdf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8834033000 tx=0x7f8834004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:58.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.222+0000 7f8849600640 1 -- 172.21.15.71:0/4272335531 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f883c0628c0 con 0x7f884c06f250 2024-09-16T07:30:58.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:58.447+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f884c06d900 con 0x7f884c06f250 2024-09-16T07:30:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.031+0000 7f8849600640 1 -- 172.21.15.71:0/4272335531 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f883c0672f0 con 0x7f884c06f250 2024-09-16T07:30:59.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.033+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8828079930 msgr2=0x7f882807bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:59.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.033+0000 7f8853a00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8828079930 0x7f882807bdf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8834033000 tx=0x7f8834004730 comp rx=0 tx=0).stop 2024-09-16T07:30:59.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.033+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f884c06f250 msgr2=0x7f884c13dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:59.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.033+0000 7f8853a00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f884c06f250 0x7f884c13dc00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c3a0 tx=0x7f883c00c7a0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.034+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 shutdown_connections 2024-09-16T07:30:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.034+0000 7f8853a00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8828079930 0x7f882807bdf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.034+0000 7f8853a00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f884c151cb0 0x7f884c1f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.034+0000 7f8853a00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c073030 0x7f884c13e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.034+0000 7f8853a00640 1 --2- 172.21.15.71:0/4272335531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f884c06f250 0x7f884c13dc00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.034+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 >> 172.21.15.71:0/4272335531 conn(0x7f884c0b2130 msgr2=0x7f884c06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.034+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 shutdown_connections 2024-09-16T07:30:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.034+0000 7f8853a00640 1 -- 172.21.15.71:0/4272335531 wait complete. 2024-09-16T07:30:59.050 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:30:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.211+0000 7fa1a7400640 1 -- 172.21.15.71:0/2715360826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a006e910 msgr2=0x7fa1a00cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.211+0000 7fa1a7400640 1 --2- 172.21.15.71:0/2715360826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a006e910 0x7fa1a00cdad0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0070cd0 tx=0x7fa18c02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.212+0000 7fa1a7400640 1 -- 172.21.15.71:0/2715360826 shutdown_connections 2024-09-16T07:30:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.212+0000 7fa1a7400640 1 --2- 172.21.15.71:0/2715360826 >> v2:172.21.15.177:3300/0 conn(0x7fa1a0074460 0x7fa1a00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.212+0000 7fa1a7400640 1 --2- 172.21.15.71:0/2715360826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a006e910 0x7fa1a00cdad0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.212+0000 7fa1a7400640 1 -- 172.21.15.71:0/2715360826 >> 172.21.15.71:0/2715360826 conn(0x7fa1a00b18e0 msgr2=0x7fa1a00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:30:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.212+0000 7fa1a7400640 1 -- 172.21.15.71:0/2715360826 shutdown_connections 2024-09-16T07:30:59.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.212+0000 7fa1a7400640 1 -- 172.21.15.71:0/2715360826 wait complete. 2024-09-16T07:30:59.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.213+0000 7fa1a7400640 1 Processor -- start 2024-09-16T07:30:59.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.213+0000 7fa1a7400640 1 -- start start 2024-09-16T07:30:59.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.214+0000 7fa1a7400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a006e910 0x7fa1a014c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.214+0000 7fa1a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a0074460 0x7fa1a0152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.214+0000 7fa1a7400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015bd60 0x7fa1a00d0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.214+0000 7fa1a7400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a00ccc70 con 0x7fa1a0074460 2024-09-16T07:30:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.214+0000 7fa1a7400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a00ccaf0 con 0x7fa1a015bd60 2024-09-16T07:30:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.214+0000 7fa1a7400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a00ccdf0 con 0x7fa1a006e910 2024-09-16T07:30:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.214+0000 7fa19fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a0074460 0x7fa1a0152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa19fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a0074460 0x7fa1a0152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34814/0 (socket says 172.21.15.71:34814) 2024-09-16T07:30:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa19fe00640 1 -- 172.21.15.71:0/1681652085 learned_addr learned my addr 172.21.15.71:0/1681652085 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:30:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa1a4a00640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a006e910 0x7fa1a014c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa1a5400640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015bd60 0x7fa1a00d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa19fe00640 1 -- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a006e910 msgr2=0x7fa1a014c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa19fe00640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a006e910 0x7fa1a014c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa19fe00640 1 -- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015bd60 msgr2=0x7fa1a00d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:30:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa19fe00640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015bd60 0x7fa1a00d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:30:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa19fe00640 1 -- 172.21.15.71:0/1681652085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a00cf0a0 con 0x7fa1a0074460 2024-09-16T07:30:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.215+0000 7fa19fe00640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a0074460 0x7fa1a0152f20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009850 tx=0x7fa18c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.216+0000 7fa19cc00640 1 -- 172.21.15.71:0/1681652085 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c03f070 con 0x7fa1a0074460 2024-09-16T07:30:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.216+0000 7fa19cc00640 1 -- 172.21.15.71:0/1681652085 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa18c0047a0 con 0x7fa1a0074460 2024-09-16T07:30:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.216+0000 7fa19cc00640 1 -- 172.21.15.71:0/1681652085 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c035030 con 0x7fa1a0074460 2024-09-16T07:30:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.216+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a00cf380 con 0x7fa1a0074460 2024-09-16T07:30:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.216+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a00cf940 con 0x7fa1a0074460 2024-09-16T07:30:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.217+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a0069360 con 0x7fa1a0074460 2024-09-16T07:30:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.218+0000 7fa19cc00640 1 -- 172.21.15.71:0/1681652085 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fa18c042070 con 0x7fa1a0074460 2024-09-16T07:30:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.219+0000 7fa19cc00640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa180079960 0x7fa18007be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:30:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.219+0000 7fa19cc00640 1 -- 172.21.15.71:0/1681652085 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5909+0+0 (secure 0 0 0) 0x7fa18c0ba530 con 0x7fa1a0074460 2024-09-16T07:30:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.219+0000 7fa1a4a00640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa180079960 0x7fa18007be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:30:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.220+0000 7fa1a4a00640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa180079960 0x7fa18007be20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa190001640 tx=0x7fa190009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:30:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.223+0000 7fa19cc00640 1 -- 172.21.15.71:0/1681652085 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa18c0842b0 con 0x7fa1a0074460 2024-09-16T07:30:59.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:30:59.409+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa1a00cbcc0 con 0x7fa1a0074460 2024-09-16T07:31:00.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.029+0000 7fa19cc00640 1 -- 172.21.15.71:0/1681652085 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) ==== 181+0+0 (secure 0 0 0) 0x7fa18c083bb0 con 0x7fa1a0074460 2024-09-16T07:31:00.029 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:31:00.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa180079960 msgr2=0x7fa18007be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:00.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa180079960 0x7fa18007be20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa190001640 tx=0x7fa190009040 comp rx=0 tx=0).stop 2024-09-16T07:31:00.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a0074460 msgr2=0x7fa1a0152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:00.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a0074460 0x7fa1a0152f20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009850 tx=0x7fa18c003820 comp rx=0 tx=0).stop 2024-09-16T07:31:00.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 shutdown_connections 2024-09-16T07:31:00.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa180079960 0x7fa18007be20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:00.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015bd60 0x7fa1a00d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:00.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a0074460 0x7fa1a0152f20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:00.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 --2- 172.21.15.71:0/1681652085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a006e910 0x7fa1a014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:00.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.033+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 >> 172.21.15.71:0/1681652085 conn(0x7fa1a00b18e0 msgr2=0x7fa1a0157430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:00.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.034+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 shutdown_connections 2024-09-16T07:31:00.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.034+0000 7fa1a7400640 1 -- 172.21.15.71:0/1681652085 wait complete. 2024-09-16T07:31:00.052 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:31:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.220+0000 7f7804600640 1 -- 172.21.15.71:0/1334408536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc159aa0 msgr2=0x7f77fc0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.220+0000 7f7804600640 1 --2- 172.21.15.71:0/1334408536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc159aa0 0x7f77fc0741a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f77fc070850 tx=0x7f77f002a560 comp rx=0 tx=0).stop 2024-09-16T07:31:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.220+0000 7f7804600640 1 -- 172.21.15.71:0/1334408536 shutdown_connections 2024-09-16T07:31:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.220+0000 7f7804600640 1 --2- 172.21.15.71:0/1334408536 >> v2:172.21.15.177:3300/0 conn(0x7f77fc15cff0 0x7f77fc06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.220+0000 7f7804600640 1 --2- 172.21.15.71:0/1334408536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc159aa0 0x7f77fc0741a0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.220+0000 7f7804600640 1 -- 172.21.15.71:0/1334408536 >> 172.21.15.71:0/1334408536 conn(0x7f77fc0b18e0 msgr2=0x7f77fc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.220+0000 7f7804600640 1 -- 172.21.15.71:0/1334408536 shutdown_connections 2024-09-16T07:31:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.220+0000 7f7804600640 1 -- 172.21.15.71:0/1334408536 wait complete. 2024-09-16T07:31:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.222+0000 7f7804600640 1 Processor -- start 2024-09-16T07:31:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.222+0000 7f7804600640 1 -- start start 2024-09-16T07:31:00.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7804600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc06f220 0x7f77fc0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:00.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7804600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77fc159aa0 0x7f77fc154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7804600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77fc15cff0 0x7f77fc153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7804600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc0ccc30 con 0x7f77fc06f220 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7804600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc0ccab0 con 0x7f77fc15cff0 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7804600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f77fc0ccdb0 con 0x7f77fc159aa0 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc06f220 0x7f77fc0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc06f220 0x7f77fc0cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34834/0 (socket says 172.21.15.71:34834) 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801c00640 1 -- 172.21.15.71:0/2589023649 learned_addr learned my addr 172.21.15.71:0/2589023649 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801200640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77fc159aa0 0x7f77fc154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7802600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77fc15cff0 0x7f77fc153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801c00640 1 -- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77fc159aa0 msgr2=0x7f77fc154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801c00640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77fc159aa0 0x7f77fc154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801c00640 1 -- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77fc15cff0 msgr2=0x7f77fc153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801c00640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77fc15cff0 0x7f77fc153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801c00640 1 -- 172.21.15.71:0/2589023649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc1553a0 con 0x7f77fc06f220 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7801200640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77fc159aa0 0x7f77fc154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.223+0000 7f7802600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77fc15cff0 0x7f77fc153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.224+0000 7f7801c00640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc06f220 0x7f77fc0cfd30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f77f00099c0 tx=0x7f77f00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.224+0000 7f77f9600640 1 -- 172.21.15.71:0/2589023649 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f003f070 con 0x7f77fc06f220 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.224+0000 7f77f9600640 1 -- 172.21.15.71:0/2589023649 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f0004890 con 0x7f77fc06f220 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.224+0000 7f77f9600640 1 -- 172.21.15.71:0/2589023649 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0035030 con 0x7f77fc06f220 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.224+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc1f82f0 con 0x7f77fc06f220 2024-09-16T07:31:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.224+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1f88b0 con 0x7f77fc06f220 2024-09-16T07:31:00.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.225+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc069360 con 0x7f77fc06f220 2024-09-16T07:31:00.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.226+0000 7f77f9600640 1 -- 172.21.15.71:0/2589023649 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f77f0042070 con 0x7f77fc06f220 2024-09-16T07:31:00.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.227+0000 7f77f9600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77dc079930 0x7f77dc07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:00.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.227+0000 7f77f9600640 1 -- 172.21.15.71:0/2589023649 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5414+0+0 (secure 0 0 0) 0x7f77f00ba670 con 0x7f77fc06f220 2024-09-16T07:31:00.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.227+0000 7f7801200640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77dc079930 0x7f77dc07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:00.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.228+0000 7f7801200640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77dc079930 0x7f77dc07bdf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f77e8005fd0 tx=0x7f77e800f360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:00.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.231+0000 7f77f9600640 1 -- 172.21.15.71:0/2589023649 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f77f00843f0 con 0x7f77fc06f220 2024-09-16T07:31:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:00.416+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77fc06f9c0 con 0x7f77fc06f220 2024-09-16T07:31:01.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.035+0000 7f77f9600640 1 -- 172.21.15.71:0/2589023649 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) ==== 169+0+0 (secure 0 0 0) 0x7f77f0083cf0 con 0x7f77fc06f220 2024-09-16T07:31:01.035 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:31:01.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.040+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77dc079930 msgr2=0x7f77dc07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.040+0000 7f7804600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77dc079930 0x7f77dc07bdf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f77e8005fd0 tx=0x7f77e800f360 comp rx=0 tx=0).stop 2024-09-16T07:31:01.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.040+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc06f220 msgr2=0x7f77fc0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.040+0000 7f7804600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc06f220 0x7f77fc0cfd30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f77f00099c0 tx=0x7f77f00039e0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.041+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 shutdown_connections 2024-09-16T07:31:01.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.041+0000 7f7804600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77dc079930 0x7f77dc07bdf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.041+0000 7f7804600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77fc15cff0 0x7f77fc153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.041+0000 7f7804600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77fc159aa0 0x7f77fc154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.041+0000 7f7804600640 1 --2- 172.21.15.71:0/2589023649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77fc06f220 0x7f77fc0cfd30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.041+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 >> 172.21.15.71:0/2589023649 conn(0x7f77fc0b18e0 msgr2=0x7f77fc0aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:01.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.041+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 shutdown_connections 2024-09-16T07:31:01.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.041+0000 7f7804600640 1 -- 172.21.15.71:0/2589023649 wait complete. 2024-09-16T07:31:01.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:31:01.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.231+0000 7f44a7e00640 1 -- 172.21.15.71:0/643367277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a015f5a0 msgr2=0x7f44a00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.231+0000 7f44a7e00640 1 --2- 172.21.15.71:0/643367277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a015f5a0 0x7f44a00b2440 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4494009830 tx=0x7f449402a480 comp rx=0 tx=0).stop 2024-09-16T07:31:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.232+0000 7f44a7e00640 1 -- 172.21.15.71:0/643367277 shutdown_connections 2024-09-16T07:31:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.232+0000 7f44a7e00640 1 --2- 172.21.15.71:0/643367277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a0073fb0 0x7f44a0074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.232+0000 7f44a7e00640 1 --2- 172.21.15.71:0/643367277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a015f5a0 0x7f44a00b2440 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.232+0000 7f44a7e00640 1 -- 172.21.15.71:0/643367277 >> 172.21.15.71:0/643367277 conn(0x7f44a00b1950 msgr2=0x7f44a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.232+0000 7f44a7e00640 1 -- 172.21.15.71:0/643367277 shutdown_connections 2024-09-16T07:31:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.232+0000 7f44a7e00640 1 -- 172.21.15.71:0/643367277 wait complete. 2024-09-16T07:31:01.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.233+0000 7f44a7e00640 1 Processor -- start 2024-09-16T07:31:01.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.234+0000 7f44a7e00640 1 -- start start 2024-09-16T07:31:01.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.234+0000 7f44a7e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a0073fb0 0x7f44a0148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:01.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.234+0000 7f44a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a00ccf00 0x7f44a015d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:01.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.234+0000 7f44a7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a015f5a0 0x7f44a0156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:01.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f44a006f2e0 con 0x7f44a00ccf00 2024-09-16T07:31:01.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a7e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f44a006f160 con 0x7f44a015f5a0 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a7e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f44a006f460 con 0x7f44a0073fb0 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a00ccf00 0x7f44a015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a00ccf00 0x7f44a015d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34844/0 (socket says 172.21.15.71:34844) 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a4a00640 1 -- 172.21.15.71:0/3875041910 learned_addr learned my addr 172.21.15.71:0/3875041910 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a5400640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a0073fb0 0x7f44a0148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a5e00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a015f5a0 0x7f44a0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a4a00640 1 -- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a0073fb0 msgr2=0x7f44a0148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a4a00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a0073fb0 0x7f44a0148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a4a00640 1 -- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a015f5a0 msgr2=0x7f44a0156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a4a00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a015f5a0 0x7f44a0156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.235+0000 7f44a4a00640 1 -- 172.21.15.71:0/3875041910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a014b1a0 con 0x7f44a00ccf00 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.236+0000 7f44a4a00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a00ccf00 0x7f44a015d900 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f448c00c500 tx=0x7f448c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:01.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.236+0000 7f449d600640 1 -- 172.21.15.71:0/3875041910 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c017070 con 0x7f44a00ccf00 2024-09-16T07:31:01.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.236+0000 7f449d600640 1 -- 172.21.15.71:0/3875041910 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f448c0035f0 con 0x7f44a00ccf00 2024-09-16T07:31:01.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.236+0000 7f449d600640 1 -- 172.21.15.71:0/3875041910 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c003a40 con 0x7f44a00ccf00 2024-09-16T07:31:01.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.236+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a00b2d50 con 0x7f44a00ccf00 2024-09-16T07:31:01.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.236+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a00b3310 con 0x7f44a00ccf00 2024-09-16T07:31:01.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.237+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a006f980 con 0x7f44a00ccf00 2024-09-16T07:31:01.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.238+0000 7f449d600640 1 -- 172.21.15.71:0/3875041910 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f448c01a070 con 0x7f44a00ccf00 2024-09-16T07:31:01.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.239+0000 7f449d600640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4480079930 0x7f448007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:01.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.240+0000 7f449d600640 1 -- 172.21.15.71:0/3875041910 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7f448c098c30 con 0x7f44a00ccf00 2024-09-16T07:31:01.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.240+0000 7f44a5400640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4480079930 0x7f448007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:01.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.241+0000 7f44a5400640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4480079930 0x7f448007bdf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4494034000 tx=0x7f4494004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:01.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.244+0000 7f449d600640 1 -- 172.21.15.71:0/3875041910 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f448c062360 con 0x7f44a00ccf00 2024-09-16T07:31:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.428+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f44a00cccc0 con 0x7f44a00ccf00 2024-09-16T07:31:01.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.432+0000 7f449d600640 1 -- 172.21.15.71:0/3875041910 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v40) ==== 71+0+0 (secure 0 0 0) 0x7f448c066d90 con 0x7f44a00ccf00 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4480079930 msgr2=0x7f448007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4480079930 0x7f448007bdf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4494034000 tx=0x7f4494004730 comp rx=0 tx=0).stop 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a00ccf00 msgr2=0x7f44a015d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a00ccf00 0x7f44a015d900 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f448c00c500 tx=0x7f448c00c9c0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 shutdown_connections 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4480079930 0x7f448007bdf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a015f5a0 0x7f44a0156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a00ccf00 0x7f44a015d900 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 --2- 172.21.15.71:0/3875041910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a0073fb0 0x7f44a0148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 >> 172.21.15.71:0/3875041910 conn(0x7f44a00b1950 msgr2=0x7f44a00b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 shutdown_connections 2024-09-16T07:31:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.435+0000 7f44a7e00640 1 -- 172.21.15.71:0/3875041910 wait complete. 2024-09-16T07:31:01.451 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-16T07:31:01.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.608+0000 7efe85600640 1 -- 172.21.15.71:0/3915811333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 msgr2=0x7efe800b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.608+0000 7efe85600640 1 --2- 172.21.15.71:0/3915811333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 0x7efe800b28f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe70005fd0 tx=0x7efe7002a840 comp rx=0 tx=0).stop 2024-09-16T07:31:01.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.608+0000 7efe85600640 1 -- 172.21.15.71:0/3915811333 shutdown_connections 2024-09-16T07:31:01.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.608+0000 7efe85600640 1 --2- 172.21.15.71:0/3915811333 >> v2:172.21.15.177:3300/0 conn(0x7efe800725d0 0x7efe80073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.608+0000 7efe85600640 1 --2- 172.21.15.71:0/3915811333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 0x7efe800b28f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.608+0000 7efe85600640 1 -- 172.21.15.71:0/3915811333 >> 172.21.15.71:0/3915811333 conn(0x7efe800cfa20 msgr2=0x7efe800cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:01.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.608+0000 7efe85600640 1 -- 172.21.15.71:0/3915811333 shutdown_connections 2024-09-16T07:31:01.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.608+0000 7efe85600640 1 -- 172.21.15.71:0/3915811333 wait complete. 2024-09-16T07:31:01.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.610+0000 7efe85600640 1 Processor -- start 2024-09-16T07:31:01.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.610+0000 7efe85600640 1 -- start start 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe85600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe800d0540 0x7efe801f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe85600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 0x7efe80151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe85600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe801f7610 0x7efe801f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe85600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efe80074610 con 0x7efe801f7610 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe85600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efe80074490 con 0x7efe800d0540 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe85600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efe80074790 con 0x7efe8015bc30 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe84c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe801f7610 0x7efe801f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe84c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe801f7610 0x7efe801f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34854/0 (socket says 172.21.15.71:34854) 2024-09-16T07:31:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.611+0000 7efe84c00640 1 -- 172.21.15.71:0/1272057036 learned_addr learned my addr 172.21.15.71:0/1272057036 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe7fe00640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe800d0540 0x7efe801f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe7f400640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 0x7efe80151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe84c00640 1 -- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 msgr2=0x7efe80151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe84c00640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 0x7efe80151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe84c00640 1 -- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe800d0540 msgr2=0x7efe801f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe84c00640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe800d0540 0x7efe801f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe84c00640 1 -- 172.21.15.71:0/1272057036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe800c9730 con 0x7efe801f7610 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe7f400640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 0x7efe80151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe84c00640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe801f7610 0x7efe801f7aa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efe7800c4b0 tx=0x7efe7800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe77e00640 1 -- 172.21.15.71:0/1272057036 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78016020 con 0x7efe801f7610 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe77e00640 1 -- 172.21.15.71:0/1272057036 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe78012920 con 0x7efe801f7610 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe77e00640 1 -- 172.21.15.71:0/1272057036 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78012d70 con 0x7efe801f7610 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe801f8460 con 0x7efe801f7610 2024-09-16T07:31:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.612+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe801f4410 con 0x7efe801f7610 2024-09-16T07:31:01.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.613+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8006fc60 con 0x7efe801f7610 2024-09-16T07:31:01.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.614+0000 7efe77e00640 1 -- 172.21.15.71:0/1272057036 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7efe78019020 con 0x7efe801f7610 2024-09-16T07:31:01.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.615+0000 7efe77e00640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe60079930 0x7efe6007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:01.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.615+0000 7efe77e00640 1 -- 172.21.15.71:0/1272057036 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7efe78099550 con 0x7efe801f7610 2024-09-16T07:31:01.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.616+0000 7efe7fe00640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe60079930 0x7efe6007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:01.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.616+0000 7efe7fe00640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe60079930 0x7efe6007bdf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe70033000 tx=0x7efe70004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:01.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.619+0000 7efe77e00640 1 -- 172.21.15.71:0/1272057036 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efe78020020 con 0x7efe801f7610 2024-09-16T07:31:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:01.876+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7efe801f8180 con 0x7efe801f7610 2024-09-16T07:31:02.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.037+0000 7efe77e00640 1 -- 172.21.15.71:0/1272057036 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v379) ==== 167+0+0 (secure 0 0 0) 0x7efe78063610 con 0x7efe801f7610 2024-09-16T07:31:02.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe60079930 msgr2=0x7efe6007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe60079930 0x7efe6007bdf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe70033000 tx=0x7efe70004ab0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe801f7610 msgr2=0x7efe801f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe801f7610 0x7efe801f7aa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efe7800c4b0 tx=0x7efe7800c970 comp rx=0 tx=0).stop 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 shutdown_connections 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe60079930 0x7efe6007bdf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe801f7610 0x7efe801f7aa0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe8015bc30 0x7efe80151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 --2- 172.21.15.71:0/1272057036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe800d0540 0x7efe801f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 >> 172.21.15.71:0/1272057036 conn(0x7efe800cfa20 msgr2=0x7efe800cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 shutdown_connections 2024-09-16T07:31:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.040+0000 7efe85600640 1 -- 172.21.15.71:0/1272057036 wait complete. 2024-09-16T07:31:02.057 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed in output of "ceph status" ... ok 2024-09-16T07:31:02.058 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-09-16T07:31:02.058 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:31:02.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:31:02.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.216+0000 7f82c3200640 1 -- 172.21.15.71:0/3336786799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc0d0540 msgr2=0x7f82bc06f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.216+0000 7f82c3200640 1 --2- 172.21.15.71:0/3336786799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc0d0540 0x7f82bc06f320 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0cd570 tx=0x7f82b802aa30 comp rx=0 tx=0).stop 2024-09-16T07:31:02.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.216+0000 7f82c3200640 1 -- 172.21.15.71:0/3336786799 shutdown_connections 2024-09-16T07:31:02.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.216+0000 7f82c3200640 1 --2- 172.21.15.71:0/3336786799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc0d0540 0x7f82bc06f320 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.216+0000 7f82c3200640 1 --2- 172.21.15.71:0/3336786799 >> v2:172.21.15.177:3300/0 conn(0x7f82bc15bc30 0x7f82bc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.216+0000 7f82c3200640 1 -- 172.21.15.71:0/3336786799 >> 172.21.15.71:0/3336786799 conn(0x7f82bc0cfa20 msgr2=0x7f82bc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.216+0000 7f82c3200640 1 -- 172.21.15.71:0/3336786799 shutdown_connections 2024-09-16T07:31:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.216+0000 7f82c3200640 1 -- 172.21.15.71:0/3336786799 wait complete. 2024-09-16T07:31:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.218+0000 7f82c3200640 1 Processor -- start 2024-09-16T07:31:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.218+0000 7f82c3200640 1 -- start start 2024-09-16T07:31:02.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c3200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82bc0d0540 0x7f82bc1489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc15bc30 0x7f82bc156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c3200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82bc1f5340 0x7f82bc147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc074660 con 0x7f82bc15bc30 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c3200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc06f860 con 0x7f82bc0d0540 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c3200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc156a50 con 0x7f82bc1f5340 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc15bc30 0x7f82bc156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc15bc30 0x7f82bc156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34882/0 (socket says 172.21.15.71:34882) 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1400640 1 -- 172.21.15.71:0/43913342 learned_addr learned my addr 172.21.15.71:0/43913342 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1e00640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82bc0d0540 0x7f82bc1489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c2800640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82bc1f5340 0x7f82bc147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1400640 1 -- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82bc1f5340 msgr2=0x7f82bc147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1400640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82bc1f5340 0x7f82bc147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1400640 1 -- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82bc0d0540 msgr2=0x7f82bc1489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1400640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82bc0d0540 0x7f82bc1489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.219+0000 7f82c1400640 1 -- 172.21.15.71:0/43913342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc0c7bc0 con 0x7f82bc15bc30 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.220+0000 7f82c1400640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc15bc30 0x7f82bc156210 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c400 tx=0x7f82ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.220+0000 7f82b6000640 1 -- 172.21.15.71:0/43913342 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac011790 con 0x7f82bc15bc30 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.220+0000 7f82b6000640 1 -- 172.21.15.71:0/43913342 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82ac017070 con 0x7f82bc15bc30 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.220+0000 7f82b6000640 1 -- 172.21.15.71:0/43913342 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac011d00 con 0x7f82bc15bc30 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.220+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc0c7ea0 con 0x7f82bc15bc30 2024-09-16T07:31:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.220+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc156750 con 0x7f82bc15bc30 2024-09-16T07:31:02.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.221+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82bc06fdf0 con 0x7f82bc15bc30 2024-09-16T07:31:02.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.222+0000 7f82b6000640 1 -- 172.21.15.71:0/43913342 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f82ac01c460 con 0x7f82bc15bc30 2024-09-16T07:31:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.224+0000 7f82b6000640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82a0079930 0x7f82a007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.224+0000 7f82b6000640 1 -- 172.21.15.71:0/43913342 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7f82ac0993b0 con 0x7f82bc15bc30 2024-09-16T07:31:02.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.224+0000 7f82c1e00640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82a0079930 0x7f82a007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.225+0000 7f82c1e00640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82a0079930 0x7f82a007bdf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f82b0005fd0 tx=0x7f82b0005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:02.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.228+0000 7f82b6000640 1 -- 172.21.15.71:0/43913342 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f82ac063470 con 0x7f82bc15bc30 2024-09-16T07:31:02.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.419+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82bc1580d0 con 0x7f82bc15bc30 2024-09-16T07:31:02.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.420+0000 7f82b6000640 1 -- 172.21.15.71:0/43913342 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+13088 (secure 0 0 0) 0x7f82ac062d70 con 0x7f82bc15bc30 2024-09-16T07:31:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":84,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:31:01.031352+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/1371001821":"2024-09-17T07:30:58.024427+0000","172.21.15.71:6833/1371001821":"2024-09-17T07:30:58.024427+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82a0079930 msgr2=0x7f82a007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82a0079930 0x7f82a007bdf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f82b0005fd0 tx=0x7f82b0005850 comp rx=0 tx=0).stop 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc15bc30 msgr2=0x7f82bc156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc15bc30 0x7f82bc156210 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c400 tx=0x7f82ac00c8c0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 shutdown_connections 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82a0079930 0x7f82a007bdf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82bc1f5340 0x7f82bc147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82bc15bc30 0x7f82bc156210 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 --2- 172.21.15.71:0/43913342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82bc0d0540 0x7f82bc1489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 >> 172.21.15.71:0/43913342 conn(0x7f82bc0cfa20 msgr2=0x7f82bc0b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 shutdown_connections 2024-09-16T07:31:02.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.423+0000 7f82c3200640 1 -- 172.21.15.71:0/43913342 wait complete. 2024-09-16T07:31:02.438 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:31:02.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:31:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.597+0000 7f052c200640 1 -- 172.21.15.71:0/3211900616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240b3370 msgr2=0x7f05240cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.597+0000 7f052c200640 1 --2- 172.21.15.71:0/3211900616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240b3370 0x7f05240cebe0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0518005fd0 tx=0x7f051802a5d0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.597+0000 7f052c200640 1 -- 172.21.15.71:0/3211900616 shutdown_connections 2024-09-16T07:31:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.597+0000 7f052c200640 1 --2- 172.21.15.71:0/3211900616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f052415f090 0x7f05240cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.597+0000 7f052c200640 1 --2- 172.21.15.71:0/3211900616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240b3370 0x7f05240cebe0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.597+0000 7f052c200640 1 -- 172.21.15.71:0/3211900616 >> 172.21.15.71:0/3211900616 conn(0x7f05240b1840 msgr2=0x7f05240b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.598+0000 7f052c200640 1 -- 172.21.15.71:0/3211900616 shutdown_connections 2024-09-16T07:31:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.598+0000 7f052c200640 1 -- 172.21.15.71:0/3211900616 wait complete. 2024-09-16T07:31:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.599+0000 7f052c200640 1 Processor -- start 2024-09-16T07:31:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.599+0000 7f052c200640 1 -- start start 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.600+0000 7f052c200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05240b3370 0x7f05241f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.600+0000 7f052c200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f052415c080 0x7f052415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.600+0000 7f052c200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052415f090 0x7f052406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.600+0000 7f052c200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0524164e50 con 0x7f052415f090 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.600+0000 7f052c200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0524164cd0 con 0x7f05240b3370 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.600+0000 7f052c200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0524164fd0 con 0x7f052415c080 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.600+0000 7f052a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052415f090 0x7f052406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.600+0000 7f052a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052415f090 0x7f052406fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34902/0 (socket says 172.21.15.71:34902) 2024-09-16T07:31:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f0528e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f052415c080 0x7f052415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f052a200640 1 -- 172.21.15.71:0/3316336940 learned_addr learned my addr 172.21.15.71:0/3316336940 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f0529800640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05240b3370 0x7f05241f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f052a200640 1 -- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f052415c080 msgr2=0x7f052415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f052a200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f052415c080 0x7f052415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f052a200640 1 -- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05240b3370 msgr2=0x7f05241f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f052a200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05240b3370 0x7f05241f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f052a200640 1 -- 172.21.15.71:0/3316336940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05241f5240 con 0x7f052415f090 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.601+0000 7f052a200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052415f090 0x7f052406fc90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f051c00da90 tx=0x7f051c00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.602+0000 7f0521600640 1 -- 172.21.15.71:0/3316336940 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c017070 con 0x7f052415f090 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.602+0000 7f0521600640 1 -- 172.21.15.71:0/3316336940 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f051c003970 con 0x7f052415f090 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.602+0000 7f0521600640 1 -- 172.21.15.71:0/3316336940 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c003dc0 con 0x7f052415f090 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.602+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05241f5520 con 0x7f052415f090 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.602+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05241f82b0 con 0x7f052415f090 2024-09-16T07:31:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.603+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052415e980 con 0x7f052415f090 2024-09-16T07:31:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.604+0000 7f0521600640 1 -- 172.21.15.71:0/3316336940 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f051c003f60 con 0x7f052415f090 2024-09-16T07:31:02.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.605+0000 7f0521600640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0504079960 0x7f050407be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.605+0000 7f0521600640 1 -- 172.21.15.71:0/3316336940 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7f051c099860 con 0x7f052415f090 2024-09-16T07:31:02.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.605+0000 7f0529800640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0504079960 0x7f050407be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.606+0000 7f0529800640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0504079960 0x7f050407be20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0518033000 tx=0x7f0518004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:02.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.609+0000 7f0521600640 1 -- 172.21.15.71:0/3316336940 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f051c063250 con 0x7f052415f090 2024-09-16T07:31:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.793+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05241f8480 con 0x7f052415f090 2024-09-16T07:31:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.793+0000 7f0521600640 1 -- 172.21.15.71:0/3316336940 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f051c067c80 con 0x7f052415f090 2024-09-16T07:31:02.793 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:31:02.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0504079960 msgr2=0x7f050407be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0504079960 0x7f050407be20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0518033000 tx=0x7f0518004730 comp rx=0 tx=0).stop 2024-09-16T07:31:02.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052415f090 msgr2=0x7f052406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052415f090 0x7f052406fc90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f051c00da90 tx=0x7f051c00df50 comp rx=0 tx=0).stop 2024-09-16T07:31:02.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 shutdown_connections 2024-09-16T07:31:02.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0504079960 0x7f050407be20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052415f090 0x7f052406fc90 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f052415c080 0x7f052415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 --2- 172.21.15.71:0/3316336940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05240b3370 0x7f05241f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.795+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 >> 172.21.15.71:0/3316336940 conn(0x7f05240b1840 msgr2=0x7f052413cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:02.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.796+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 shutdown_connections 2024-09-16T07:31:02.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.796+0000 7f052c200640 1 -- 172.21.15.71:0/3316336940 wait complete. 2024-09-16T07:31:02.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-16T07:31:02.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 -- 172.21.15.71:0/2688994151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c2415f5a0 msgr2=0x7f0c240b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 --2- 172.21.15.71:0/2688994151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c2415f5a0 0x7f0c240b2440 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0c240cfbb0 tx=0x7f0c1402a980 comp rx=0 tx=0).stop 2024-09-16T07:31:02.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 -- 172.21.15.71:0/2688994151 shutdown_connections 2024-09-16T07:31:02.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 --2- 172.21.15.71:0/2688994151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c240ad270 0x7f0c24073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 --2- 172.21.15.71:0/2688994151 >> v2:172.21.15.177:3300/0 conn(0x7f0c240ccf00 0x7f0c240acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 --2- 172.21.15.71:0/2688994151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c2415f5a0 0x7f0c240b2440 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 -- 172.21.15.71:0/2688994151 >> 172.21.15.71:0/2688994151 conn(0x7f0c240b1950 msgr2=0x7f0c240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:02.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 -- 172.21.15.71:0/2688994151 shutdown_connections 2024-09-16T07:31:02.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.967+0000 7f0c29600640 1 -- 172.21.15.71:0/2688994151 wait complete. 2024-09-16T07:31:02.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.969+0000 7f0c29600640 1 Processor -- start 2024-09-16T07:31:02.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.969+0000 7f0c29600640 1 -- start start 2024-09-16T07:31:02.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c29600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c240ad270 0x7f0c241464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c29600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c240ccf00 0x7f0c240b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c29600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c2415f5a0 0x7f0c240b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c29600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24074880 con 0x7f0c240ad270 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c29600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24074700 con 0x7f0c2415f5a0 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c29600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2406f160 con 0x7f0c240ccf00 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c22a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c240ad270 0x7f0c241464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c22a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c240ad270 0x7f0c241464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34918/0 (socket says 172.21.15.71:34918) 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c22a00640 1 -- 172.21.15.71:0/3345087132 learned_addr learned my addr 172.21.15.71:0/3345087132 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c23400640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c2415f5a0 0x7f0c240b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.970+0000 7f0c22000640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c240ccf00 0x7f0c240b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c22a00640 1 -- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c240ccf00 msgr2=0x7f0c240b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c22a00640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c240ccf00 0x7f0c240b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c22a00640 1 -- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c2415f5a0 msgr2=0x7f0c240b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c22a00640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c2415f5a0 0x7f0c240b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c22a00640 1 -- 172.21.15.71:0/3345087132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c24150070 con 0x7f0c240ad270 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c23400640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c2415f5a0 0x7f0c240b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c22a00640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c240ad270 0x7f0c241464a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0c14000f80 tx=0x7f0c14003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c1aa00640 1 -- 172.21.15.71:0/3345087132 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14034040 con 0x7f0c240ad270 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c1aa00640 1 -- 172.21.15.71:0/3345087132 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c140045c0 con 0x7f0c240ad270 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c1aa00640 1 -- 172.21.15.71:0/3345087132 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14045480 con 0x7f0c240ad270 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2414d6b0 con 0x7f0c240ad270 2024-09-16T07:31:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.971+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c24150600 con 0x7f0c240ad270 2024-09-16T07:31:02.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.972+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c24073fb0 con 0x7f0c240ad270 2024-09-16T07:31:02.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.973+0000 7f0c1aa00640 1 -- 172.21.15.71:0/3345087132 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f0c14037020 con 0x7f0c240ad270 2024-09-16T07:31:02.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.974+0000 7f0c1aa00640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0c04079960 0x7f0c0407be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:02.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.974+0000 7f0c1aa00640 1 -- 172.21.15.71:0/3345087132 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7f0c140bad70 con 0x7f0c240ad270 2024-09-16T07:31:02.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.975+0000 7f0c22000640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0c04079960 0x7f0c0407be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:02.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.975+0000 7f0c22000640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0c04079960 0x7f0c0407be20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0c10005fd0 tx=0x7f0c10005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:02.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:02.978+0000 7f0c1aa00640 1 -- 172.21.15.71:0/3345087132 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0c14084db0 con 0x7f0c240ad270 2024-09-16T07:31:03.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:03.234+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f0c240afb80 con 0x7f0c240ad270 2024-09-16T07:31:04.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.045+0000 7f0c1aa00640 1 -- 172.21.15.71:0/3345087132 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v381) ==== 177+0+0 (secure 0 0 0) 0x7f0c140846b0 con 0x7f0c240ad270 2024-09-16T07:31:04.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.048+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0c04079960 msgr2=0x7f0c0407be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.048+0000 7f0c29600640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0c04079960 0x7f0c0407be20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0c10005fd0 tx=0x7f0c10005850 comp rx=0 tx=0).stop 2024-09-16T07:31:04.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.048+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c240ad270 msgr2=0x7f0c241464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.048+0000 7f0c29600640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c240ad270 0x7f0c241464a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0c14000f80 tx=0x7f0c14003f90 comp rx=0 tx=0).stop 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.049+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 shutdown_connections 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.049+0000 7f0c29600640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0c04079960 0x7f0c0407be20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.049+0000 7f0c29600640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c2415f5a0 0x7f0c240b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.049+0000 7f0c29600640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c240ccf00 0x7f0c240b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.049+0000 7f0c29600640 1 --2- 172.21.15.71:0/3345087132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c240ad270 0x7f0c241464a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.049+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 >> 172.21.15.71:0/3345087132 conn(0x7f0c240b1950 msgr2=0x7f0c2409c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.049+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 shutdown_connections 2024-09-16T07:31:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.049+0000 7f0c29600640 1 -- 172.21.15.71:0/3345087132 wait complete. 2024-09-16T07:31:04.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:31:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.229+0000 7f2b03000640 1 -- 172.21.15.71:0/1889216712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0d0540 msgr2=0x7f2afc06f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.229+0000 7f2b03000640 1 --2- 172.21.15.71:0/1889216712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0d0540 0x7f2afc06f320 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005fd0 tx=0x7f2af402aaa0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.229+0000 7f2b03000640 1 -- 172.21.15.71:0/1889216712 shutdown_connections 2024-09-16T07:31:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.229+0000 7f2b03000640 1 --2- 172.21.15.71:0/1889216712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc0725d0 0x7f2afc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.229+0000 7f2b03000640 1 --2- 172.21.15.71:0/1889216712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0d0540 0x7f2afc06f320 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.229+0000 7f2b03000640 1 -- 172.21.15.71:0/1889216712 >> 172.21.15.71:0/1889216712 conn(0x7f2afc0cfa20 msgr2=0x7f2afc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:04.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.229+0000 7f2b03000640 1 -- 172.21.15.71:0/1889216712 shutdown_connections 2024-09-16T07:31:04.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.229+0000 7f2b03000640 1 -- 172.21.15.71:0/1889216712 wait complete. 2024-09-16T07:31:04.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.231+0000 7f2b03000640 1 Processor -- start 2024-09-16T07:31:04.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.231+0000 7f2b03000640 1 -- start start 2024-09-16T07:31:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b03000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0d0540 0x7f2afc152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b03000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc15bc30 0x7f2afc0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b03000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0cc970 0x7f2afc0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b03000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc074660 con 0x7f2afc0cc970 2024-09-16T07:31:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b03000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06f860 con 0x7f2afc0d0540 2024-09-16T07:31:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b03000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc0ccda0 con 0x7f2afc15bc30 2024-09-16T07:31:04.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b02600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0cc970 0x7f2afc0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:04.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b02600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0cc970 0x7f2afc0ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34948/0 (socket says 172.21.15.71:34948) 2024-09-16T07:31:04.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b02600640 1 -- 172.21.15.71:0/215146363 learned_addr learned my addr 172.21.15.71:0/215146363 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:04.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b01200640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc15bc30 0x7f2afc0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:04.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.232+0000 7f2b01c00640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0d0540 0x7f2afc152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:04.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2b02600640 1 -- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc15bc30 msgr2=0x7f2afc0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2b02600640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc15bc30 0x7f2afc0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2b02600640 1 -- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0d0540 msgr2=0x7f2afc152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2b02600640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0d0540 0x7f2afc152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2b02600640 1 -- 172.21.15.71:0/215146363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc0ae590 con 0x7f2afc0cc970 2024-09-16T07:31:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2b02600640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0cc970 0x7f2afc0ad980 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2af402b090 tx=0x7f2af4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2af9600640 1 -- 172.21.15.71:0/215146363 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4045070 con 0x7f2afc0cc970 2024-09-16T07:31:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2af9600640 1 -- 172.21.15.71:0/215146363 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af402e800 con 0x7f2afc0cc970 2024-09-16T07:31:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc0ae870 con 0x7f2afc0cc970 2024-09-16T07:31:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2af9600640 1 -- 172.21.15.71:0/215146363 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af40044b0 con 0x7f2afc0cc970 2024-09-16T07:31:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.233+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc0aee30 con 0x7f2afc0cc970 2024-09-16T07:31:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.235+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc0c6800 con 0x7f2afc0cc970 2024-09-16T07:31:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.236+0000 7f2af9600640 1 -- 172.21.15.71:0/215146363 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f2af4048070 con 0x7f2afc0cc970 2024-09-16T07:31:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.237+0000 7f2af9600640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc079930 0x7f2adc07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.237+0000 7f2af9600640 1 -- 172.21.15.71:0/215146363 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7f2af40b9ee0 con 0x7f2afc0cc970 2024-09-16T07:31:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.237+0000 7f2b01c00640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc079930 0x7f2adc07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.238+0000 7f2b01c00640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc079930 0x7f2adc07bdf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2af0001720 tx=0x7f2af0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:04.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.241+0000 7f2af9600640 1 -- 172.21.15.71:0/215146363 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2afc0c6800 con 0x7f2afc0cc970 2024-09-16T07:31:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.427+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2afc06fdf0 con 0x7f2afc0cc970 2024-09-16T07:31:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.428+0000 7f2af9600640 1 -- 172.21.15.71:0/215146363 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) ==== 59+0+1998 (secure 0 0 0) 0x7f2af4032030 con 0x7f2afc0cc970 2024-09-16T07:31:04.427 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:31:04.428 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:31:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:31:04.430 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:31:04.430 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:31:04.430 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:31:04.430 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:31:04.430 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:31:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.430+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc079930 msgr2=0x7f2adc07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.430+0000 7f2b03000640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc079930 0x7f2adc07bdf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2af0001720 tx=0x7f2af0009040 comp rx=0 tx=0).stop 2024-09-16T07:31:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.430+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0cc970 msgr2=0x7f2afc0ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.430+0000 7f2b03000640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0cc970 0x7f2afc0ad980 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2af402b090 tx=0x7f2af4003f10 comp rx=0 tx=0).stop 2024-09-16T07:31:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.431+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 shutdown_connections 2024-09-16T07:31:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.431+0000 7f2b03000640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2adc079930 0x7f2adc07bdf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.431+0000 7f2b03000640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2afc0cc970 0x7f2afc0ad980 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.431+0000 7f2b03000640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2afc15bc30 0x7f2afc0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.431+0000 7f2b03000640 1 --2- 172.21.15.71:0/215146363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2afc0d0540 0x7f2afc152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.431+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 >> 172.21.15.71:0/215146363 conn(0x7f2afc0cfa20 msgr2=0x7f2afc0cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.431+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 shutdown_connections 2024-09-16T07:31:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.431+0000 7f2b03000640 1 -- 172.21.15.71:0/215146363 wait complete. 2024-09-16T07:31:04.446 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:31:04.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.606+0000 7f4e47200640 1 -- 172.21.15.71:0/2057928656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cf4b0 msgr2=0x7f4e480cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.606+0000 7f4e47200640 1 --2- 172.21.15.71:0/2057928656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cf4b0 0x7f4e480cf900 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4e480d4e50 tx=0x7f4e4002a980 comp rx=0 tx=0).stop 2024-09-16T07:31:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.606+0000 7f4e47200640 1 -- 172.21.15.71:0/2057928656 shutdown_connections 2024-09-16T07:31:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.606+0000 7f4e47200640 1 --2- 172.21.15.71:0/2057928656 >> v2:172.21.15.177:3300/0 conn(0x7f4e480b3450 0x7f4e480b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.606+0000 7f4e47200640 1 --2- 172.21.15.71:0/2057928656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cf4b0 0x7f4e480cf900 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.606+0000 7f4e47200640 1 -- 172.21.15.71:0/2057928656 >> 172.21.15.71:0/2057928656 conn(0x7f4e480d3d50 msgr2=0x7f4e480d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.606+0000 7f4e47200640 1 -- 172.21.15.71:0/2057928656 shutdown_connections 2024-09-16T07:31:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.606+0000 7f4e47200640 1 -- 172.21.15.71:0/2057928656 wait complete. 2024-09-16T07:31:04.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.607+0000 7f4e47200640 1 Processor -- start 2024-09-16T07:31:04.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.607+0000 7f4e47200640 1 -- start start 2024-09-16T07:31:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.608+0000 7f4e47200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480b3450 0x7f4e48152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.608+0000 7f4e47200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cf4b0 0x7f4e4815f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.608+0000 7f4e47200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4815df60 0x7f4e48160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.608+0000 7f4e47200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4e480d2330 con 0x7f4e480b3450 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.608+0000 7f4e47200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e480cea70 con 0x7f4e480cf4b0 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.608+0000 7f4e47200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4e480d24b0 con 0x7f4e4815df60 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480b3450 0x7f4e48152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480b3450 0x7f4e48152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34952/0 (socket says 172.21.15.71:34952) 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 -- 172.21.15.71:0/2121754039 learned_addr learned my addr 172.21.15.71:0/2121754039 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e46800640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4815df60 0x7f4e48160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45400640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cf4b0 0x7f4e4815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 -- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4815df60 msgr2=0x7f4e48160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4815df60 0x7f4e48160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 -- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cf4b0 msgr2=0x7f4e4815f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cf4b0 0x7f4e4815f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 -- 172.21.15.71:0/2121754039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e481f5500 con 0x7f4e480b3450 2024-09-16T07:31:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.609+0000 7f4e45e00640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480b3450 0x7f4e48152140 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800b3f0 tx=0x7f4e3800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.610+0000 7f4e3e000640 1 -- 172.21.15.71:0/2121754039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38015020 con 0x7f4e480b3450 2024-09-16T07:31:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.610+0000 7f4e3e000640 1 -- 172.21.15.71:0/2121754039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e38011920 con 0x7f4e480b3450 2024-09-16T07:31:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.610+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e48160560 con 0x7f4e480b3450 2024-09-16T07:31:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.610+0000 7f4e3e000640 1 -- 172.21.15.71:0/2121754039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011d70 con 0x7f4e480b3450 2024-09-16T07:31:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.610+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4815fda0 con 0x7f4e480b3450 2024-09-16T07:31:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.611+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e480d2a40 con 0x7f4e480b3450 2024-09-16T07:31:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.612+0000 7f4e3e000640 1 -- 172.21.15.71:0/2121754039 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f4e38018020 con 0x7f4e480b3450 2024-09-16T07:31:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.613+0000 7f4e3e000640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e28079930 0x7f4e2807bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.613+0000 7f4e3e000640 1 -- 172.21.15.71:0/2121754039 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7f4e38098d10 con 0x7f4e480b3450 2024-09-16T07:31:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.613+0000 7f4e45400640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e28079930 0x7f4e2807bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.614+0000 7f4e45400640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e28079930 0x7f4e2807bdf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4e480b9d10 tx=0x7f4e3400b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:04.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.617+0000 7f4e3e000640 1 -- 172.21.15.71:0/2121754039 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e3801f070 con 0x7f4e480b3450 2024-09-16T07:31:04.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.801+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4e4806ed80 con 0x7f4e480b3450 2024-09-16T07:31:04.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.806+0000 7f4e3e000640 1 -- 172.21.15.71:0/2121754039 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f4e38062780 con 0x7f4e480b3450 2024-09-16T07:31:04.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.808+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e28079930 msgr2=0x7f4e2807bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.808+0000 7f4e47200640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e28079930 0x7f4e2807bdf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4e480b9d10 tx=0x7f4e3400b390 comp rx=0 tx=0).stop 2024-09-16T07:31:04.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.808+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480b3450 msgr2=0x7f4e48152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.808+0000 7f4e47200640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480b3450 0x7f4e48152140 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800b3f0 tx=0x7f4e3800b8b0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.808+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 shutdown_connections 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.808+0000 7f4e47200640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e28079930 0x7f4e2807bdf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.809+0000 7f4e47200640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4815df60 0x7f4e48160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.809+0000 7f4e47200640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e480cf4b0 0x7f4e4815f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.809+0000 7f4e47200640 1 --2- 172.21.15.71:0/2121754039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480b3450 0x7f4e48152140 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.809+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 >> 172.21.15.71:0/2121754039 conn(0x7f4e480d3d50 msgr2=0x7f4e480b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.809+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 shutdown_connections 2024-09-16T07:31:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:04.809+0000 7f4e47200640 1 -- 172.21.15.71:0/2121754039 wait complete. 2024-09-16T07:31:04.824 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:31:04.824 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:31:04.923 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:31:05.019 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:31:05.020 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:31:05.020 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:31:05.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:31:05.057 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:31:05.057 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:31:05.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:31:05.095 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:31:05.095 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:31:05.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:31:05.136 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:31:05.136 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:31:05.172 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:31:05.173 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:31:05.173 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:31:05.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.335+0000 7fbcaa200640 1 -- 172.21.15.71:0/533432728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca415f5a0 msgr2=0x7fbca40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:05.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.335+0000 7fbcaa200640 1 --2- 172.21.15.71:0/533432728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca415f5a0 0x7fbca40b2440 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbca40cfbb0 tx=0x7fbc9402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.335+0000 7fbcaa200640 1 -- 172.21.15.71:0/533432728 shutdown_connections 2024-09-16T07:31:05.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.335+0000 7fbcaa200640 1 --2- 172.21.15.71:0/533432728 >> v2:172.21.15.177:3300/0 conn(0x7fbca40ccf00 0x7fbca40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.335+0000 7fbcaa200640 1 --2- 172.21.15.71:0/533432728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca415f5a0 0x7fbca40b2440 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.335+0000 7fbcaa200640 1 -- 172.21.15.71:0/533432728 >> 172.21.15.71:0/533432728 conn(0x7fbca40b1950 msgr2=0x7fbca40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:05.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.335+0000 7fbcaa200640 1 -- 172.21.15.71:0/533432728 shutdown_connections 2024-09-16T07:31:05.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.335+0000 7fbcaa200640 1 -- 172.21.15.71:0/533432728 wait complete. 2024-09-16T07:31:05.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.337+0000 7fbcaa200640 1 Processor -- start 2024-09-16T07:31:05.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.337+0000 7fbcaa200640 1 -- start start 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbcaa200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca40ad270 0x7fbca41f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbcaa200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbca40ccf00 0x7fbca40cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbcaa200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbca415f5a0 0x7fbca40d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbcaa200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbca406f160 con 0x7fbca40ad270 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbcaa200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbca406f2e0 con 0x7fbca40ccf00 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbcaa200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbca406f460 con 0x7fbca415f5a0 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbca3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca40ad270 0x7fbca41f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbca3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca40ad270 0x7fbca41f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51768/0 (socket says 172.21.15.71:51768) 2024-09-16T07:31:05.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbca3400640 1 -- 172.21.15.71:0/1666841580 learned_addr learned my addr 172.21.15.71:0/1666841580 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbca2a00640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbca40ccf00 0x7fbca40cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.338+0000 7fbca3e00640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbca415f5a0 0x7fbca40d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbca3400640 1 -- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbca415f5a0 msgr2=0x7fbca40d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbca3400640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbca415f5a0 0x7fbca40d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbca3400640 1 -- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbca40ccf00 msgr2=0x7fbca40cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbca3400640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbca40ccf00 0x7fbca40cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbca3400640 1 -- 172.21.15.71:0/1666841580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca41f5180 con 0x7fbca40ad270 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbca3400640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca40ad270 0x7fbca41f8ec0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbc94034040 tx=0x7fbc940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbc9b400640 1 -- 172.21.15.71:0/1666841580 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9403e030 con 0x7fbca40ad270 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbc9b400640 1 -- 172.21.15.71:0/1666841580 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc940047a0 con 0x7fbca40ad270 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca41f5460 con 0x7fbca40ad270 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbc9b400640 1 -- 172.21.15.71:0/1666841580 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94035040 con 0x7fbca40ad270 2024-09-16T07:31:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.339+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca40cb300 con 0x7fbca40ad270 2024-09-16T07:31:05.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.340+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca406fa10 con 0x7fbca40ad270 2024-09-16T07:31:05.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.342+0000 7fbc9b400640 1 -- 172.21.15.71:0/1666841580 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fbc94041030 con 0x7fbca40ad270 2024-09-16T07:31:05.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.342+0000 7fbc9b400640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc84079930 0x7fbc8407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:05.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.343+0000 7fbc9b400640 1 -- 172.21.15.71:0/1666841580 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7fbc940b9e60 con 0x7fbca40ad270 2024-09-16T07:31:05.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.343+0000 7fbca2a00640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc84079930 0x7fbc8407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:05.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.343+0000 7fbca2a00640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc84079930 0x7fbc8407bdf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbca40ca5e0 tx=0x7fbc90009110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:05.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.346+0000 7fbc9b400640 1 -- 172.21.15.71:0/1666841580 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbc94083590 con 0x7fbca40ad270 2024-09-16T07:31:05.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.567+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbca41f57f0 con 0x7fbca40ad270 2024-09-16T07:31:05.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.567+0000 7fbc9b400640 1 -- 172.21.15.71:0/1666841580 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3917 (secure 0 0 0) 0x7fbc94087fc0 con 0x7fbca40ad270 2024-09-16T07:31:05.567 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:05.567 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":49,"btime":"2024-09-16T07:30:59:026770+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5690,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/4181257111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":4181257111},{"type":"v1","addr":"172.21.15.177:6836","nonce":4181257111}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2868012570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2868012570},{"type":"v1","addr":"172.21.15.177:6837","nonce":2868012570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6414,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3075140097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3075140097},{"type":"v1","addr":"172.21.15.71:6834","nonce":3075140097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4118710617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":4118710617},{"type":"v1","addr":"172.21.15.71:6835","nonce":4118710617}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-09-16T07:31:05.567 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 49 2024-09-16T07:31:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc84079930 msgr2=0x7fbc8407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc84079930 0x7fbc8407bdf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbca40ca5e0 tx=0x7fbc90009110 comp rx=0 tx=0).stop 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca40ad270 msgr2=0x7fbca41f8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca40ad270 0x7fbca41f8ec0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbc94034040 tx=0x7fbc940039e0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 shutdown_connections 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc84079930 0x7fbc8407bdf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbca415f5a0 0x7fbca40d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbca40ccf00 0x7fbca40cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 --2- 172.21.15.71:0/1666841580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbca40ad270 0x7fbca41f8ec0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 >> 172.21.15.71:0/1666841580 conn(0x7fbca40b1950 msgr2=0x7fbca4073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 shutdown_connections 2024-09-16T07:31:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.570+0000 7fbcaa200640 1 -- 172.21.15.71:0/1666841580 wait complete. 2024-09-16T07:31:05.586 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:31:05.586 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:31:05.586 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:31:05.587 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:31:05.587 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:31:05.587 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:31:05.587 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:31:05.588 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:31:05.588 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:31:05.588 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:31:05.588 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:31:05.588 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:31:05.588 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:31:05.589 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:31:05.589 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:31:05.589 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:31:05.602 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:31:05.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:31:05.604 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:31:05.604 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:31:05.606 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:31:05.606 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:31:05.607 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:31:05.608 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:31:05.646 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:31:05.647 DEBUG:teuthology.parallel:result is None 2024-09-16T07:31:05.649 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:31:05.649 DEBUG:teuthology.parallel:result is None 2024-09-16T07:31:05.651 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:31:05.651 DEBUG:teuthology.parallel:result is None 2024-09-16T07:31:05.653 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:31:05.654 DEBUG:teuthology.parallel:result is None 2024-09-16T07:31:05.654 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:31:05.778 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:31:05.778 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:31:05.784 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:31:05.784 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:31:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.823+0000 7f2b96a00640 1 -- 172.21.15.71:0/3210427876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b901438b0 msgr2=0x7f2b9006ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.823+0000 7f2b96a00640 1 --2- 172.21.15.71:0/3210427876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b901438b0 0x7f2b9006ff60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2b9015b6d0 tx=0x7f2b7802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.823+0000 7f2b96a00640 1 -- 172.21.15.71:0/3210427876 shutdown_connections 2024-09-16T07:31:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.823+0000 7f2b96a00640 1 --2- 172.21.15.71:0/3210427876 >> v2:172.21.15.177:3300/0 conn(0x7f2b900b3f00 0x7f2b900b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.823+0000 7f2b96a00640 1 --2- 172.21.15.71:0/3210427876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b901438b0 0x7f2b9006ff60 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.823+0000 7f2b96a00640 1 -- 172.21.15.71:0/3210427876 >> 172.21.15.71:0/3210427876 conn(0x7f2b900b1950 msgr2=0x7f2b900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.823+0000 7f2b96a00640 1 -- 172.21.15.71:0/3210427876 shutdown_connections 2024-09-16T07:31:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.823+0000 7f2b96a00640 1 -- 172.21.15.71:0/3210427876 wait complete. 2024-09-16T07:31:05.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.824+0000 7f2b96a00640 1 Processor -- start 2024-09-16T07:31:05.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.825+0000 7f2b96a00640 1 -- start start 2024-09-16T07:31:05.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.825+0000 7f2b96a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90070920 0x7f2b900cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:05.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.825+0000 7f2b96a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b900b3f00 0x7f2b9009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:05.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.825+0000 7f2b96a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b901438b0 0x7f2b9009af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:05.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b96a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b900b4d40 con 0x7f2b900b3f00 2024-09-16T07:31:05.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b96a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2b900b4bc0 con 0x7f2b901438b0 2024-09-16T07:31:05.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b96a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2b900b4ec0 con 0x7f2b90070920 2024-09-16T07:31:05.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b87e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b900b3f00 0x7f2b9009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:05.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b87e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b900b3f00 0x7f2b9009aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51816/0 (socket says 172.21.15.71:51816) 2024-09-16T07:31:05.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b87e00640 1 -- 172.21.15.71:0/2741196781 learned_addr learned my addr 172.21.15.71:0/2741196781 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:05.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b94a00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b901438b0 0x7f2b9009af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:05.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b8fe00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90070920 0x7f2b900cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:05.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b87e00640 1 -- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90070920 msgr2=0x7f2b900cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:05.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b87e00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90070920 0x7f2b900cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b87e00640 1 -- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b901438b0 msgr2=0x7f2b9009af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:05.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b87e00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b901438b0 0x7f2b9009af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:05.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b87e00640 1 -- 172.21.15.71:0/2741196781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b900cba10 con 0x7f2b900b3f00 2024-09-16T07:31:05.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.826+0000 7f2b94a00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b901438b0 0x7f2b9009af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:31:05.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.827+0000 7f2b87e00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b900b3f00 0x7f2b9009aa00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2b78034040 tx=0x7f2b78003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:05.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.827+0000 7f2b8cc00640 1 -- 172.21.15.71:0/2741196781 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7803e030 con 0x7f2b900b3f00 2024-09-16T07:31:05.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.827+0000 7f2b8cc00640 1 -- 172.21.15.71:0/2741196781 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b78003d10 con 0x7f2b900b3f00 2024-09-16T07:31:05.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.827+0000 7f2b8cc00640 1 -- 172.21.15.71:0/2741196781 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78035040 con 0x7f2b900b3f00 2024-09-16T07:31:05.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.827+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9009d840 con 0x7f2b900b3f00 2024-09-16T07:31:05.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.827+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9015b6f0 con 0x7f2b900b3f00 2024-09-16T07:31:05.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.828+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b900b4390 con 0x7f2b900b3f00 2024-09-16T07:31:05.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.829+0000 7f2b8cc00640 1 -- 172.21.15.71:0/2741196781 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f2b78041030 con 0x7f2b900b3f00 2024-09-16T07:31:05.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.831+0000 7f2b8cc00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b6c079930 0x7f2b6c07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:05.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.831+0000 7f2b8cc00640 1 -- 172.21.15.71:0/2741196781 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4971+0+0 (secure 0 0 0) 0x7f2b780b9da0 con 0x7f2b900b3f00 2024-09-16T07:31:05.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.831+0000 7f2b8fe00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b6c079930 0x7f2b6c07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:05.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.832+0000 7f2b8fe00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b6c079930 0x7f2b6c07bdf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2b800017c0 tx=0x7f2b80009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:05.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:05.835+0000 7f2b8cc00640 1 -- 172.21.15.71:0/2741196781 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2b780834d0 con 0x7f2b900b3f00 2024-09-16T07:31:06.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.031+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2b900cd430 con 0x7f2b900b3f00 2024-09-16T07:31:06.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.062+0000 7f2b8cc00640 1 -- 172.21.15.71:0/2741196781 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) ==== 97+0+0 (secure 0 0 0) 0x7f2b78087f00 con 0x7f2b900b3f00 2024-09-16T07:31:06.062 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:31:06.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.064+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b6c079930 msgr2=0x7f2b6c07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b6c079930 0x7f2b6c07bdf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2b800017c0 tx=0x7f2b80009040 comp rx=0 tx=0).stop 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b900b3f00 msgr2=0x7f2b9009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b900b3f00 0x7f2b9009aa00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2b78034040 tx=0x7f2b78003820 comp rx=0 tx=0).stop 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 shutdown_connections 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b6c079930 0x7f2b6c07bdf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b901438b0 0x7f2b9009af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b900b3f00 0x7f2b9009aa00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 --2- 172.21.15.71:0/2741196781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b90070920 0x7f2b900cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 >> 172.21.15.71:0/2741196781 conn(0x7f2b900b1950 msgr2=0x7f2b9006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 shutdown_connections 2024-09-16T07:31:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.065+0000 7f2b96a00640 1 -- 172.21.15.71:0/2741196781 wait complete. 2024-09-16T07:31:06.080 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:31:06.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.237+0000 7f1230a00640 1 -- 172.21.15.71:0/414348926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1616b0 msgr2=0x7f122c0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.237+0000 7f1230a00640 1 --2- 172.21.15.71:0/414348926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1616b0 0x7f122c0d2370 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f122c0d4ef0 tx=0x7f122402aa30 comp rx=0 tx=0).stop 2024-09-16T07:31:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.237+0000 7f1230a00640 1 -- 172.21.15.71:0/414348926 shutdown_connections 2024-09-16T07:31:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.237+0000 7f1230a00640 1 --2- 172.21.15.71:0/414348926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1616b0 0x7f122c0d2370 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.237+0000 7f1230a00640 1 --2- 172.21.15.71:0/414348926 >> v2:172.21.15.177:3300/0 conn(0x7f122c1612d0 0x7f122c0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.237+0000 7f1230a00640 1 -- 172.21.15.71:0/414348926 >> 172.21.15.71:0/414348926 conn(0x7f122c0d3df0 msgr2=0x7f122c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.237+0000 7f1230a00640 1 -- 172.21.15.71:0/414348926 shutdown_connections 2024-09-16T07:31:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.237+0000 7f1230a00640 1 -- 172.21.15.71:0/414348926 wait complete. 2024-09-16T07:31:06.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.238+0000 7f1230a00640 1 Processor -- start 2024-09-16T07:31:06.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.238+0000 7f1230a00640 1 -- start start 2024-09-16T07:31:06.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.239+0000 7f1230a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122c0d4980 0x7f122c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:06.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.239+0000 7f1230a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1612d0 0x7f122c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:06.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.239+0000 7f1230a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f122c1616b0 0x7f122c158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:06.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.239+0000 7f1230a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f122c0b1b60 con 0x7f122c1612d0 2024-09-16T07:31:06.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.239+0000 7f1230a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f122c0b19e0 con 0x7f122c0d4980 2024-09-16T07:31:06.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.239+0000 7f1230a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f122c0b1ce0 con 0x7f122c1616b0 2024-09-16T07:31:06.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1612d0 0x7f122c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1612d0 0x7f122c1589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51842/0 (socket says 172.21.15.71:51842) 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 -- 172.21.15.71:0/2524029365 learned_addr learned my addr 172.21.15.71:0/2524029365 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f122a800640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122c0d4980 0x7f122c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f122b200640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f122c1616b0 0x7f122c158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 -- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f122c1616b0 msgr2=0x7f122c158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f122c1616b0 0x7f122c158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 -- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122c0d4980 msgr2=0x7f122c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122c0d4980 0x7f122c157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 -- 172.21.15.71:0/2524029365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c1595b0 con 0x7f122c1612d0 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.240+0000 7f1229e00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1612d0 0x7f122c1589f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f121800c450 tx=0x7f121800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.241+0000 7f1222a00640 1 -- 172.21.15.71:0/2524029365 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12180052d0 con 0x7f122c1612d0 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.241+0000 7f1222a00640 1 -- 172.21.15.71:0/2524029365 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1218005470 con 0x7f122c1612d0 2024-09-16T07:31:06.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.241+0000 7f1222a00640 1 -- 172.21.15.71:0/2524029365 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218012870 con 0x7f122c1612d0 2024-09-16T07:31:06.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.241+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c159890 con 0x7f122c1612d0 2024-09-16T07:31:06.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.241+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c157870 con 0x7f122c1612d0 2024-09-16T07:31:06.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.242+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c0b4e00 con 0x7f122c1612d0 2024-09-16T07:31:06.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.243+0000 7f1222a00640 1 -- 172.21.15.71:0/2524029365 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f12180035f0 con 0x7f122c1612d0 2024-09-16T07:31:06.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.244+0000 7f1222a00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f120c079930 0x7f120c07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:06.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.244+0000 7f1222a00640 1 -- 172.21.15.71:0/2524029365 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4885+0+0 (secure 0 0 0) 0x7f1218099200 con 0x7f122c1612d0 2024-09-16T07:31:06.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.244+0000 7f122a800640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f120c079930 0x7f120c07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:06.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.245+0000 7f122a800640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f120c079930 0x7f120c07bdf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f121c009bd0 tx=0x7f121c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:06.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.248+0000 7f1222a00640 1 -- 172.21.15.71:0/2524029365 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f122c0d2cc0 con 0x7f122c1612d0 2024-09-16T07:31:06.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.499+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f122c0d0bb0 con 0x7f122c1612d0 2024-09-16T07:31:06.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.500+0000 7f1222a00640 1 -- 172.21.15.71:0/2524029365 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f1218062930 con 0x7f122c1612d0 2024-09-16T07:31:06.499 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:31:06.501 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:31:06.502 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:31:06.503 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.504 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.505 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:31:06.506 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.508 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:31:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:31:06.511 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:31:06.512 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:31:06.513 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:31:06.513 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:31:06.513 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:31:06.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.513 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:31:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:31:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f120c079930 msgr2=0x7f120c07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f120c079930 0x7f120c07bdf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f121c009bd0 tx=0x7f121c005f90 comp rx=0 tx=0).stop 2024-09-16T07:31:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1612d0 msgr2=0x7f122c1589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1612d0 0x7f122c1589f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f121800c450 tx=0x7f121800c910 comp rx=0 tx=0).stop 2024-09-16T07:31:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 shutdown_connections 2024-09-16T07:31:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f120c079930 0x7f120c07bdf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f122c1616b0 0x7f122c158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f122c1612d0 0x7f122c1589f0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 --2- 172.21.15.71:0/2524029365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122c0d4980 0x7f122c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 >> 172.21.15.71:0/2524029365 conn(0x7f122c0d3df0 msgr2=0x7f122c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:06.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 shutdown_connections 2024-09-16T07:31:06.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.502+0000 7f1230a00640 1 -- 172.21.15.71:0/2524029365 wait complete. 2024-09-16T07:31:06.517 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:31:06.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.673+0000 7fb66ca00640 1 -- 172.21.15.71:0/1754870348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6680b2440 msgr2=0x7fb66806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.673+0000 7fb66ca00640 1 --2- 172.21.15.71:0/1754870348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6680b2440 0x7fb66806f160 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fd0 tx=0x7fb66002a540 comp rx=0 tx=0).stop 2024-09-16T07:31:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.673+0000 7fb66ca00640 1 -- 172.21.15.71:0/1754870348 shutdown_connections 2024-09-16T07:31:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.673+0000 7fb66ca00640 1 --2- 172.21.15.71:0/1754870348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6680b2440 0x7fb66806f160 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.673+0000 7fb66ca00640 1 -- 172.21.15.71:0/1754870348 >> 172.21.15.71:0/1754870348 conn(0x7fb6680b1950 msgr2=0x7fb6680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.674+0000 7fb66ca00640 1 -- 172.21.15.71:0/1754870348 shutdown_connections 2024-09-16T07:31:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.674+0000 7fb66ca00640 1 -- 172.21.15.71:0/1754870348 wait complete. 2024-09-16T07:31:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.675+0000 7fb66ca00640 1 Processor -- start 2024-09-16T07:31:06.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.675+0000 7fb66ca00640 1 -- start start 2024-09-16T07:31:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb66ca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6680b2440 0x7fb668156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb66ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6680ccf00 0x7fb668157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb66ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb66815f5a0 0x7fb668158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb66ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb668074840 con 0x7fb66815f5a0 2024-09-16T07:31:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb66ca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680746c0 con 0x7fb6680ccf00 2024-09-16T07:31:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb66ca00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb66806f6a0 con 0x7fb6680b2440 2024-09-16T07:31:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb666800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb66815f5a0 0x7fb668158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb666800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb66815f5a0 0x7fb668158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51852/0 (socket says 172.21.15.71:51852) 2024-09-16T07:31:06.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.676+0000 7fb666800640 1 -- 172.21.15.71:0/857686809 learned_addr learned my addr 172.21.15.71:0/857686809 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:06.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.677+0000 7fb665e00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6680b2440 0x7fb668156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:06.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.677+0000 7fb665400640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6680ccf00 0x7fb668157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.677+0000 7fb666800640 1 -- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6680b2440 msgr2=0x7fb668156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.677+0000 7fb666800640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6680b2440 0x7fb668156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.677+0000 7fb666800640 1 -- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6680ccf00 msgr2=0x7fb668157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.677+0000 7fb666800640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6680ccf00 0x7fb668157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.677+0000 7fb666800640 1 -- 172.21.15.71:0/857686809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66815c3c0 con 0x7fb66815f5a0 2024-09-16T07:31:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.677+0000 7fb666800640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb66815f5a0 0x7fb668158230 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb660006010 tx=0x7fb660003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.678+0000 7fb65e000640 1 -- 172.21.15.71:0/857686809 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66003f070 con 0x7fb66815f5a0 2024-09-16T07:31:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.678+0000 7fb65e000640 1 -- 172.21.15.71:0/857686809 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb660004890 con 0x7fb66815f5a0 2024-09-16T07:31:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.678+0000 7fb65e000640 1 -- 172.21.15.71:0/857686809 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660035030 con 0x7fb66815f5a0 2024-09-16T07:31:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.678+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66815c6a0 con 0x7fb66815f5a0 2024-09-16T07:31:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.678+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66809bdd0 con 0x7fb66815f5a0 2024-09-16T07:31:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.679+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66806f160 con 0x7fb66815f5a0 2024-09-16T07:31:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.680+0000 7fb65e000640 1 -- 172.21.15.71:0/857686809 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fb660042070 con 0x7fb66815f5a0 2024-09-16T07:31:06.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.681+0000 7fb65e000640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb644079930 0x7fb64407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:06.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.682+0000 7fb65e000640 1 -- 172.21.15.71:0/857686809 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4885+0+0 (secure 0 0 0) 0x7fb6600ba670 con 0x7fb66815f5a0 2024-09-16T07:31:06.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.682+0000 7fb665e00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb644079930 0x7fb64407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:06.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.682+0000 7fb665e00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb644079930 0x7fb64407bdf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb65800b450 tx=0x7fb65800d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:06.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.686+0000 7fb65e000640 1 -- 172.21.15.71:0/857686809 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb660083da0 con 0x7fb66815f5a0 2024-09-16T07:31:06.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.872+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb6680aceb0 con 0x7fb66815f5a0 2024-09-16T07:31:06.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.873+0000 7fb65e000640 1 -- 172.21.15.71:0/857686809 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) ==== 84+0+65 (secure 0 0 0) 0x7fb6600887d0 con 0x7fb66815f5a0 2024-09-16T07:31:06.872 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:06.872 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:31:06.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:06.873 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:31:06.873 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:31:06.873 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:06.873 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb644079930 msgr2=0x7fb64407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb644079930 0x7fb64407bdf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb65800b450 tx=0x7fb65800d040 comp rx=0 tx=0).stop 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb66815f5a0 msgr2=0x7fb668158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb66815f5a0 0x7fb668158230 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb660006010 tx=0x7fb660003960 comp rx=0 tx=0).stop 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 shutdown_connections 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb644079930 0x7fb64407bdf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb66815f5a0 0x7fb668158230 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6680ccf00 0x7fb668157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 --2- 172.21.15.71:0/857686809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6680b2440 0x7fb668156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:06.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 >> 172.21.15.71:0/857686809 conn(0x7fb6680b1950 msgr2=0x7fb6680b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:06.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 shutdown_connections 2024-09-16T07:31:06.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:06.875+0000 7fb66ca00640 1 -- 172.21.15.71:0/857686809 wait complete. 2024-09-16T07:31:06.890 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:31:06.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:31:07.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.054+0000 7f6a7b000640 1 -- 172.21.15.71:0/9016350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a58004800 msgr2=0x7f6a58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:07.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.054+0000 7f6a7b000640 1 --2- 172.21.15.71:0/9016350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a58004800 0x7f6a58004c80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c310 tx=0x7f6a58005830 comp rx=0 tx=0).stop 2024-09-16T07:31:07.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.054+0000 7f6a7b000640 1 -- 172.21.15.71:0/9016350 shutdown_connections 2024-09-16T07:31:07.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.054+0000 7f6a7b000640 1 --2- 172.21.15.71:0/9016350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a58004800 0x7f6a58004c80 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:07.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.054+0000 7f6a7b000640 1 -- 172.21.15.71:0/9016350 >> 172.21.15.71:0/9016350 conn(0x7f6a740b18e0 msgr2=0x7f6a740b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:07.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.054+0000 7f6a7b000640 1 -- 172.21.15.71:0/9016350 shutdown_connections 2024-09-16T07:31:07.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.055+0000 7f6a7b000640 1 -- 172.21.15.71:0/9016350 wait complete. 2024-09-16T07:31:07.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.056+0000 7f6a7b000640 1 Processor -- start 2024-09-16T07:31:07.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.056+0000 7f6a7b000640 1 -- start start 2024-09-16T07:31:07.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a7b000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a58004800 0x7f6a741522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:07.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a7b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a74152950 0x7f6a7409d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a7b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a74158b80 0x7f6a7409d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a7b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74074320 con 0x7f6a74158b80 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a7b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740741a0 con 0x7f6a74152950 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a7b000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6a740744a0 con 0x7f6a58004800 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a79000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a74158b80 0x7f6a7409d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a79000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a74158b80 0x7f6a7409d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51872/0 (socket says 172.21.15.71:51872) 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.057+0000 7f6a79000640 1 -- 172.21.15.71:0/1150111806 learned_addr learned my addr 172.21.15.71:0/1150111806 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a73400640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a74152950 0x7f6a7409d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a73e00640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a58004800 0x7f6a741522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a79000640 1 -- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a58004800 msgr2=0x7f6a741522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a79000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a58004800 0x7f6a741522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a79000640 1 -- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a74152950 msgr2=0x7f6a7409d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a79000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a74152950 0x7f6a7409d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:07.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a79000640 1 -- 172.21.15.71:0/1150111806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a741f3150 con 0x7f6a74158b80 2024-09-16T07:31:07.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a73e00640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a58004800 0x7f6a741522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:07.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a79000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a74158b80 0x7f6a7409d8f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6a74076510 tx=0x7f6a6c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:07.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.058+0000 7f6a6be00640 1 -- 172.21.15.71:0/1150111806 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c003d80 con 0x7f6a74158b80 2024-09-16T07:31:07.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.059+0000 7f6a6be00640 1 -- 172.21.15.71:0/1150111806 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a6c00fe50 con 0x7f6a74158b80 2024-09-16T07:31:07.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.059+0000 7f6a6be00640 1 -- 172.21.15.71:0/1150111806 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c004d80 con 0x7f6a74158b80 2024-09-16T07:31:07.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.059+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a741f3430 con 0x7f6a74158b80 2024-09-16T07:31:07.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.059+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a741f39f0 con 0x7f6a74158b80 2024-09-16T07:31:07.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.060+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a74069360 con 0x7f6a74158b80 2024-09-16T07:31:07.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.064+0000 7f6a6be00640 1 -- 172.21.15.71:0/1150111806 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f6a6c004050 con 0x7f6a74158b80 2024-09-16T07:31:07.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.065+0000 7f6a6be00640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a54079960 0x7f6a5407be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:07.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.066+0000 7f6a6be00640 1 -- 172.21.15.71:0/1150111806 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4885+0+0 (secure 0 0 0) 0x7f6a6c09ac70 con 0x7f6a74158b80 2024-09-16T07:31:07.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.066+0000 7f6a73e00640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a54079960 0x7f6a5407be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:07.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.067+0000 7f6a73e00640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a54079960 0x7f6a5407be20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6a741558f0 tx=0x7f6a64006b70 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:07.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.068+0000 7f6a6be00640 1 -- 172.21.15.71:0/1150111806 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6a6c0649f0 con 0x7f6a74158b80 2024-09-16T07:31:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:07.255+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6a7406e740 con 0x7f6a74158b80 2024-09-16T07:31:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.081+0000 7f6a6be00640 1 -- 172.21.15.71:0/1150111806 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) ==== 138+0+0 (secure 0 0 0) 0x7f6a6c0642f0 con 0x7f6a74158b80 2024-09-16T07:31:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a54079960 msgr2=0x7f6a5407be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a54079960 0x7f6a5407be20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6a741558f0 tx=0x7f6a64006b70 comp rx=0 tx=0).stop 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a74158b80 msgr2=0x7f6a7409d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a74158b80 0x7f6a7409d8f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6a74076510 tx=0x7f6a6c00f790 comp rx=0 tx=0).stop 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 shutdown_connections 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a54079960 0x7f6a5407be20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a74158b80 0x7f6a7409d8f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a74152950 0x7f6a7409d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 --2- 172.21.15.71:0/1150111806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a58004800 0x7f6a741522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 >> 172.21.15.71:0/1150111806 conn(0x7f6a740b18e0 msgr2=0x7f6a7415ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:08.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 shutdown_connections 2024-09-16T07:31:08.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.084+0000 7f6a7b000640 1 -- 172.21.15.71:0/1150111806 wait complete. 2024-09-16T07:31:08.105 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:31:08.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.273+0000 7f74c6000640 1 -- 172.21.15.71:0/2723923322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c006ed00 msgr2=0x7f74c0161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:08.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.273+0000 7f74c6000640 1 --2- 172.21.15.71:0/2723923322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c006ed00 0x7f74c0161c20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f74ac005fd0 tx=0x7f74ac02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.274+0000 7f74c6000640 1 -- 172.21.15.71:0/2723923322 shutdown_connections 2024-09-16T07:31:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.274+0000 7f74c6000640 1 --2- 172.21.15.71:0/2723923322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c0162160 0x7f74c00b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.274+0000 7f74c6000640 1 --2- 172.21.15.71:0/2723923322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c006ed00 0x7f74c0161c20 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.274+0000 7f74c6000640 1 -- 172.21.15.71:0/2723923322 >> 172.21.15.71:0/2723923322 conn(0x7f74c00d1e20 msgr2=0x7f74c00d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.274+0000 7f74c6000640 1 -- 172.21.15.71:0/2723923322 shutdown_connections 2024-09-16T07:31:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.274+0000 7f74c6000640 1 -- 172.21.15.71:0/2723923322 wait complete. 2024-09-16T07:31:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.275+0000 7f74c6000640 1 Processor -- start 2024-09-16T07:31:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.275+0000 7f74c6000640 1 -- start start 2024-09-16T07:31:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.276+0000 7f74c6000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c006ed00 0x7f74c01614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.276+0000 7f74c6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c015e250 0x7f74c014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.276+0000 7f74c6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0162160 0x7f74c01f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.276+0000 7f74c6000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74c00b1e20 con 0x7f74c0162160 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.276+0000 7f74c6000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f74c00b1ca0 con 0x7f74c015e250 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.276+0000 7f74c6000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f74c00b1fa0 con 0x7f74c006ed00 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0162160 0x7f74c01f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0162160 0x7f74c01f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51894/0 (socket says 172.21.15.71:51894) 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 -- 172.21.15.71:0/2860349344 learned_addr learned my addr 172.21.15.71:0/2860349344 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bf400640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c006ed00 0x7f74c01614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bea00640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c015e250 0x7f74c014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 -- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c006ed00 msgr2=0x7f74c01614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c006ed00 0x7f74c01614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 -- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c015e250 msgr2=0x7f74c014e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c015e250 0x7f74c014e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 -- 172.21.15.71:0/2860349344 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c00c7120 con 0x7f74c0162160 2024-09-16T07:31:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.277+0000 7f74bfe00640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0162160 0x7f74c01f4d00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f74b800ea80 tx=0x7f74b800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.278+0000 7f74b7400640 1 -- 172.21.15.71:0/2860349344 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8004b20 con 0x7f74c0162160 2024-09-16T07:31:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.278+0000 7f74b7400640 1 -- 172.21.15.71:0/2860349344 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74b8004ce0 con 0x7f74c0162160 2024-09-16T07:31:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.278+0000 7f74b7400640 1 -- 172.21.15.71:0/2860349344 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8005280 con 0x7f74c0162160 2024-09-16T07:31:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.278+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c00c7400 con 0x7f74c0162160 2024-09-16T07:31:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.278+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c00c79c0 con 0x7f74c0162160 2024-09-16T07:31:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.279+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c00b2530 con 0x7f74c0162160 2024-09-16T07:31:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.280+0000 7f74b7400640 1 -- 172.21.15.71:0/2860349344 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f74b8005420 con 0x7f74c0162160 2024-09-16T07:31:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.281+0000 7f74b7400640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f74a0079930 0x7f74a007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.281+0000 7f74b7400640 1 -- 172.21.15.71:0/2860349344 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5255+0+0 (secure 0 0 0) 0x7f74b809a840 con 0x7f74c0162160 2024-09-16T07:31:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.282+0000 7f74bf400640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f74a0079930 0x7f74a007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.282+0000 7f74bf400640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f74a0079930 0x7f74a007bdf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f74b00017c0 tx=0x7f74b0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:08.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.285+0000 7f74b7400640 1 -- 172.21.15.71:0/2860349344 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f74b8063f70 con 0x7f74c0162160 2024-09-16T07:31:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:08.473+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f74c00b7ca0 con 0x7f74c0162160 2024-09-16T07:31:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.088+0000 7f74b7400640 1 -- 172.21.15.71:0/2860349344 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) ==== 170+0+0 (secure 0 0 0) 0x7f74b80689a0 con 0x7f74c0162160 2024-09-16T07:31:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:31:09.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.092+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f74a0079930 msgr2=0x7f74a007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:09.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.092+0000 7f74c6000640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f74a0079930 0x7f74a007bdf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f74b00017c0 tx=0x7f74b0009040 comp rx=0 tx=0).stop 2024-09-16T07:31:09.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.092+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0162160 msgr2=0x7f74c01f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:09.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.092+0000 7f74c6000640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0162160 0x7f74c01f4d00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f74b800ea80 tx=0x7f74b800ef40 comp rx=0 tx=0).stop 2024-09-16T07:31:09.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.093+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 shutdown_connections 2024-09-16T07:31:09.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.093+0000 7f74c6000640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f74a0079930 0x7f74a007bdf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:09.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.093+0000 7f74c6000640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0162160 0x7f74c01f4d00 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:09.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.093+0000 7f74c6000640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74c015e250 0x7f74c014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:09.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.093+0000 7f74c6000640 1 --2- 172.21.15.71:0/2860349344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74c006ed00 0x7f74c01614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:09.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.093+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 >> 172.21.15.71:0/2860349344 conn(0x7f74c00d1e20 msgr2=0x7f74c00b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:09.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.093+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 shutdown_connections 2024-09-16T07:31:09.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.094+0000 7f74c6000640 1 -- 172.21.15.71:0/2860349344 wait complete. 2024-09-16T07:31:09.123 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:31:09.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.284+0000 7fde43200640 1 -- 172.21.15.71:0/1388271795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde440b43c0 msgr2=0x7fde440d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:09.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.284+0000 7fde43200640 1 --2- 172.21.15.71:0/1388271795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde440b43c0 0x7fde440d2830 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005fd0 tx=0x7fde3c02aa10 comp rx=0 tx=0).stop 2024-09-16T07:31:09.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.285+0000 7fde43200640 1 -- 172.21.15.71:0/1388271795 shutdown_connections 2024-09-16T07:31:09.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.285+0000 7fde43200640 1 --2- 172.21.15.71:0/1388271795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde440b43c0 0x7fde440d2830 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:09.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.285+0000 7fde43200640 1 --2- 172.21.15.71:0/1388271795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde44161460 0x7fde440d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:09.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.285+0000 7fde43200640 1 -- 172.21.15.71:0/1388271795 >> 172.21.15.71:0/1388271795 conn(0x7fde4409d0b0 msgr2=0x7fde440d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:09.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.285+0000 7fde43200640 1 -- 172.21.15.71:0/1388271795 shutdown_connections 2024-09-16T07:31:09.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.285+0000 7fde43200640 1 -- 172.21.15.71:0/1388271795 wait complete. 2024-09-16T07:31:09.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.286+0000 7fde43200640 1 Processor -- start 2024-09-16T07:31:09.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.286+0000 7fde43200640 1 -- start start 2024-09-16T07:31:09.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde43200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde440b43c0 0x7fde4415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:09.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde43200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde440b5480 0x7fde441582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:09.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde43200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde44161460 0x7fde4415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:09.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde43200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde440cf0b0 con 0x7fde44161460 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde43200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde440cef30 con 0x7fde440b43c0 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde43200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fde440cf230 con 0x7fde440b5480 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde42800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde44161460 0x7fde4415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde42800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde44161460 0x7fde4415c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51916/0 (socket says 172.21.15.71:51916) 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.287+0000 7fde42800640 1 -- 172.21.15.71:0/861316193 learned_addr learned my addr 172.21.15.71:0/861316193 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde41e00640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde440b43c0 0x7fde4415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde41400640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde440b5480 0x7fde441582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde42800640 1 -- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde440b5480 msgr2=0x7fde441582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde42800640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde440b5480 0x7fde441582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde42800640 1 -- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde440b43c0 msgr2=0x7fde4415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde42800640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde440b43c0 0x7fde4415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde42800640 1 -- 172.21.15.71:0/861316193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde44158830 con 0x7fde44161460 2024-09-16T07:31:09.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde41e00640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde440b43c0 0x7fde4415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:09.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde42800640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde44161460 0x7fde4415c590 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fde3c006010 tx=0x7fde3c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:09.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde3a000640 1 -- 172.21.15.71:0/861316193 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c047070 con 0x7fde44161460 2024-09-16T07:31:09.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde3a000640 1 -- 172.21.15.71:0/861316193 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde3c004250 con 0x7fde44161460 2024-09-16T07:31:09.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4415b1e0 con 0x7fde44161460 2024-09-16T07:31:09.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.288+0000 7fde3a000640 1 -- 172.21.15.71:0/861316193 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c033050 con 0x7fde44161460 2024-09-16T07:31:09.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.289+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde441564b0 con 0x7fde44161460 2024-09-16T07:31:09.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.290+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde440cf750 con 0x7fde44161460 2024-09-16T07:31:09.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.291+0000 7fde3a000640 1 -- 172.21.15.71:0/861316193 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fde3c04a070 con 0x7fde44161460 2024-09-16T07:31:09.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.292+0000 7fde3a000640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde24079930 0x7fde2407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:09.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.292+0000 7fde3a000640 1 -- 172.21.15.71:0/861316193 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5637+0+0 (secure 0 0 0) 0x7fde3c0baa90 con 0x7fde44161460 2024-09-16T07:31:09.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.292+0000 7fde41e00640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde24079930 0x7fde2407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:09.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.293+0000 7fde41e00640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde24079930 0x7fde2407bdf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde4406ea80 tx=0x7fde34005f10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:09.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.296+0000 7fde3a000640 1 -- 172.21.15.71:0/861316193 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fde3c084810 con 0x7fde44161460 2024-09-16T07:31:09.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:09.519+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fde4415ef20 con 0x7fde44161460 2024-09-16T07:31:10.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.193+0000 7fde3a000640 1 -- 172.21.15.71:0/861316193 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) ==== 174+0+0 (secure 0 0 0) 0x7fde3c08b110 con 0x7fde44161460 2024-09-16T07:31:10.193 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 12 and data pool 13 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde24079930 msgr2=0x7fde2407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde24079930 0x7fde2407bdf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde4406ea80 tx=0x7fde34005f10 comp rx=0 tx=0).stop 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde44161460 msgr2=0x7fde4415c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde44161460 0x7fde4415c590 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fde3c006010 tx=0x7fde3c003f10 comp rx=0 tx=0).stop 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 shutdown_connections 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde24079930 0x7fde2407bdf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde44161460 0x7fde4415c590 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde440b5480 0x7fde441582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 --2- 172.21.15.71:0/861316193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde440b43c0 0x7fde4415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 >> 172.21.15.71:0/861316193 conn(0x7fde4409d0b0 msgr2=0x7fde440d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:10.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 shutdown_connections 2024-09-16T07:31:10.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.196+0000 7fde43200640 1 -- 172.21.15.71:0/861316193 wait complete. 2024-09-16T07:31:10.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:31:10.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.373+0000 7f64ca200640 1 -- 172.21.15.71:0/1894462084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c4073fb0 msgr2=0x7f64c4074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.373+0000 7f64ca200640 1 --2- 172.21.15.71:0/1894462084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c4073fb0 0x7f64c4074420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f64c40cfbb0 tx=0x7f64bc02a580 comp rx=0 tx=0).stop 2024-09-16T07:31:10.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.373+0000 7f64ca200640 1 -- 172.21.15.71:0/1894462084 shutdown_connections 2024-09-16T07:31:10.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.373+0000 7f64ca200640 1 --2- 172.21.15.71:0/1894462084 >> v2:172.21.15.177:3300/0 conn(0x7f64c40ccf00 0x7f64c40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.373+0000 7f64ca200640 1 --2- 172.21.15.71:0/1894462084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c4073fb0 0x7f64c4074420 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.373+0000 7f64ca200640 1 -- 172.21.15.71:0/1894462084 >> 172.21.15.71:0/1894462084 conn(0x7f64c40b1950 msgr2=0x7f64c40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:10.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.373+0000 7f64ca200640 1 -- 172.21.15.71:0/1894462084 shutdown_connections 2024-09-16T07:31:10.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.373+0000 7f64ca200640 1 -- 172.21.15.71:0/1894462084 wait complete. 2024-09-16T07:31:10.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.375+0000 7f64ca200640 1 Processor -- start 2024-09-16T07:31:10.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.375+0000 7f64ca200640 1 -- start start 2024-09-16T07:31:10.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64ca200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4073fb0 0x7f64c41f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:10.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64ca200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c40ccf00 0x7f64c4153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:10.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64ca200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c415f5a0 0x7f64c4154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:10.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64ca200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f64c406f2e0 con 0x7f64c40ccf00 2024-09-16T07:31:10.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64ca200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64c406f160 con 0x7f64c4073fb0 2024-09-16T07:31:10.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64ca200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64c406f460 con 0x7f64c415f5a0 2024-09-16T07:31:10.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c40ccf00 0x7f64c4153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:10.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c40ccf00 0x7f64c4153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51938/0 (socket says 172.21.15.71:51938) 2024-09-16T07:31:10.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64baa00640 1 -- 172.21.15.71:0/3147120071 learned_addr learned my addr 172.21.15.71:0/3147120071 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:10.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64c3400640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4073fb0 0x7f64c41f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:10.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.376+0000 7f64c3e00640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c415f5a0 0x7f64c4154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:10.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64baa00640 1 -- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c415f5a0 msgr2=0x7f64c4154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64baa00640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c415f5a0 0x7f64c4154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64baa00640 1 -- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4073fb0 msgr2=0x7f64c41f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64baa00640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4073fb0 0x7f64c41f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64baa00640 1 -- 172.21.15.71:0/3147120071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c40d0080 con 0x7f64c40ccf00 2024-09-16T07:31:10.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64baa00640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c40ccf00 0x7f64c4153ad0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00d770 tx=0x7f64ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:10.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64bbe00640 1 -- 172.21.15.71:0/3147120071 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac017070 con 0x7f64c40ccf00 2024-09-16T07:31:10.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64bbe00640 1 -- 172.21.15.71:0/3147120071 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64ac009d20 con 0x7f64c40ccf00 2024-09-16T07:31:10.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c40ce620 con 0x7f64c40ccf00 2024-09-16T07:31:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64bbe00640 1 -- 172.21.15.71:0/3147120071 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac005280 con 0x7f64c40ccf00 2024-09-16T07:31:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.377+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c40ce8c0 con 0x7f64c40ccf00 2024-09-16T07:31:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.378+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c406f9f0 con 0x7f64c40ccf00 2024-09-16T07:31:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.379+0000 7f64bbe00640 1 -- 172.21.15.71:0/3147120071 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f64ac005420 con 0x7f64c40ccf00 2024-09-16T07:31:10.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.380+0000 7f64bbe00640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64a0079930 0x7f64a007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:10.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.381+0000 7f64bbe00640 1 -- 172.21.15.71:0/3147120071 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5737+0+0 (secure 0 0 0) 0x7f64ac099610 con 0x7f64c40ccf00 2024-09-16T07:31:10.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.381+0000 7f64c3400640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64a0079930 0x7f64a007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:10.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.382+0000 7f64c3400640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64a0079930 0x7f64a007bdf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f64b4001e20 tx=0x7f64b4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:10.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.384+0000 7f64bbe00640 1 -- 172.21.15.71:0/3147120071 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f64ac063080 con 0x7f64c40ccf00 2024-09-16T07:31:10.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.571+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64c415d760 con 0x7f64c40ccf00 2024-09-16T07:31:10.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.572+0000 7f64bbe00640 1 -- 172.21.15.71:0/3147120071 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16800 (secure 0 0 0) 0x7f64ac067ab0 con 0x7f64c40ccf00 2024-09-16T07:31:10.572 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:10.572 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:31:09.520361+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:31:07.265563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T07:31:08.483774+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64a0079930 msgr2=0x7f64a007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64a0079930 0x7f64a007bdf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f64b4001e20 tx=0x7f64b4009040 comp rx=0 tx=0).stop 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c40ccf00 msgr2=0x7f64c4153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c40ccf00 0x7f64c4153ad0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00d770 tx=0x7f64ac00dc30 comp rx=0 tx=0).stop 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 shutdown_connections 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64a0079930 0x7f64a007bdf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c415f5a0 0x7f64c4154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64c40ccf00 0x7f64c4153ad0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 --2- 172.21.15.71:0/3147120071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4073fb0 0x7f64c41f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 >> 172.21.15.71:0/3147120071 conn(0x7f64c40b1950 msgr2=0x7f64c409d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:10.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 shutdown_connections 2024-09-16T07:31:10.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.575+0000 7f64ca200640 1 -- 172.21.15.71:0/3147120071 wait complete. 2024-09-16T07:31:10.591 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:31:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.748+0000 7fdb7be00640 1 -- 172.21.15.71:0/479272164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 msgr2=0x7fdb74071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.748+0000 7fdb7be00640 1 --2- 172.21.15.71:0/479272164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 0x7fdb74071bf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdb740cea00 tx=0x7fdb6802a580 comp rx=0 tx=0).stop 2024-09-16T07:31:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.748+0000 7fdb7be00640 1 -- 172.21.15.71:0/479272164 shutdown_connections 2024-09-16T07:31:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.748+0000 7fdb7be00640 1 --2- 172.21.15.71:0/479272164 >> v2:172.21.15.177:3300/0 conn(0x7fdb74071260 0x7fdb740cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.748+0000 7fdb7be00640 1 --2- 172.21.15.71:0/479272164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 0x7fdb74071bf0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.748+0000 7fdb7be00640 1 -- 172.21.15.71:0/479272164 >> 172.21.15.71:0/479272164 conn(0x7fdb7406d230 msgr2=0x7fdb740688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.748+0000 7fdb7be00640 1 -- 172.21.15.71:0/479272164 shutdown_connections 2024-09-16T07:31:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.748+0000 7fdb7be00640 1 -- 172.21.15.71:0/479272164 wait complete. 2024-09-16T07:31:10.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.750+0000 7fdb7be00640 1 Processor -- start 2024-09-16T07:31:10.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.750+0000 7fdb7be00640 1 -- start start 2024-09-16T07:31:10.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb7be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb74071260 0x7fdb740b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 0x7fdb741f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb7be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb740cefa0 0x7fdb741593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb7be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740b7920 con 0x7fdb740b73c0 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb7be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740b77a0 con 0x7fdb740cefa0 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb7be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdb740d2350 con 0x7fdb74071260 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb78a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 0x7fdb741f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb78a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 0x7fdb741f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51958/0 (socket says 172.21.15.71:51958) 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb78a00640 1 -- 172.21.15.71:0/831334223 learned_addr learned my addr 172.21.15.71:0/831334223 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb79400640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb74071260 0x7fdb740b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb79e00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb740cefa0 0x7fdb741593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb78a00640 1 -- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb74071260 msgr2=0x7fdb740b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb78a00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb74071260 0x7fdb740b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb78a00640 1 -- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb740cefa0 msgr2=0x7fdb741593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb78a00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb740cefa0 0x7fdb741593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.751+0000 7fdb78a00640 1 -- 172.21.15.71:0/831334223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb741f2a00 con 0x7fdb740b73c0 2024-09-16T07:31:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.752+0000 7fdb78a00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 0x7fdb741f3800 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000d950 tx=0x7fdb6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.752+0000 7fdb71600640 1 -- 172.21.15.71:0/831334223 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60017070 con 0x7fdb740b73c0 2024-09-16T07:31:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.752+0000 7fdb71600640 1 -- 172.21.15.71:0/831334223 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb60009d20 con 0x7fdb740b73c0 2024-09-16T07:31:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.752+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb741f2ce0 con 0x7fdb740b73c0 2024-09-16T07:31:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.752+0000 7fdb71600640 1 -- 172.21.15.71:0/831334223 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb600106c0 con 0x7fdb740b73c0 2024-09-16T07:31:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.752+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741f32a0 con 0x7fdb740b73c0 2024-09-16T07:31:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.753+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb740d28e0 con 0x7fdb740b73c0 2024-09-16T07:31:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.754+0000 7fdb71600640 1 -- 172.21.15.71:0/831334223 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fdb60010860 con 0x7fdb740b73c0 2024-09-16T07:31:10.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.755+0000 7fdb71600640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb54079930 0x7fdb5407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:10.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.755+0000 7fdb71600640 1 -- 172.21.15.71:0/831334223 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5737+0+0 (secure 0 0 0) 0x7fdb600998d0 con 0x7fdb740b73c0 2024-09-16T07:31:10.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.756+0000 7fdb79400640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb54079930 0x7fdb5407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:10.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.757+0000 7fdb79400640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb54079930 0x7fdb5407bdf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdb68033000 tx=0x7fdb68004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:10.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.759+0000 7fdb71600640 1 -- 172.21.15.71:0/831334223 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdb60063990 con 0x7fdb740b73c0 2024-09-16T07:31:10.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.947+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb740c54d0 con 0x7fdb740b73c0 2024-09-16T07:31:10.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.949+0000 7fdb71600640 1 -- 172.21.15.71:0/831334223 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16800 (secure 0 0 0) 0x7fdb60063290 con 0x7fdb740b73c0 2024-09-16T07:31:10.948 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:10.949 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:31:09.520361+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:31:07.265563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T07:31:08.483774+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:31:10.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb54079930 msgr2=0x7fdb5407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb54079930 0x7fdb5407bdf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdb68033000 tx=0x7fdb68004730 comp rx=0 tx=0).stop 2024-09-16T07:31:10.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 msgr2=0x7fdb741f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:10.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 0x7fdb741f3800 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000d950 tx=0x7fdb6000de10 comp rx=0 tx=0).stop 2024-09-16T07:31:10.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 shutdown_connections 2024-09-16T07:31:10.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdb54079930 0x7fdb5407bdf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb740cefa0 0x7fdb741593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb740b73c0 0x7fdb741f3800 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 --2- 172.21.15.71:0/831334223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb74071260 0x7fdb740b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:10.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.951+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 >> 172.21.15.71:0/831334223 conn(0x7fdb7406d230 msgr2=0x7fdb740688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:10.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.952+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 shutdown_connections 2024-09-16T07:31:10.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:10.952+0000 7fdb7be00640 1 -- 172.21.15.71:0/831334223 wait complete. 2024-09-16T07:31:10.967 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:31:11.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.129+0000 7fb935000640 1 -- 172.21.15.71:0/4042922473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 msgr2=0x7fb91400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:11.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.129+0000 7fb935000640 1 --2- 172.21.15.71:0/4042922473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 0x7fb91400b9c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb928001d90 tx=0x7fb914005830 comp rx=0 tx=0).stop 2024-09-16T07:31:11.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.129+0000 7fb935000640 1 -- 172.21.15.71:0/4042922473 shutdown_connections 2024-09-16T07:31:11.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.129+0000 7fb935000640 1 --2- 172.21.15.71:0/4042922473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 0x7fb91400b9c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:11.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.129+0000 7fb935000640 1 -- 172.21.15.71:0/4042922473 >> 172.21.15.71:0/4042922473 conn(0x7fb9300b1840 msgr2=0x7fb9300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:11.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.129+0000 7fb935000640 1 -- 172.21.15.71:0/4042922473 shutdown_connections 2024-09-16T07:31:11.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.129+0000 7fb935000640 1 -- 172.21.15.71:0/4042922473 wait complete. 2024-09-16T07:31:11.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.131+0000 7fb935000640 1 Processor -- start 2024-09-16T07:31:11.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.131+0000 7fb935000640 1 -- start start 2024-09-16T07:31:11.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb935000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 0x7fb93015e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:11.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb935000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9301f54b0 0x7fb9301423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:11.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb935000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9301e6b40 0x7fb9301e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb935000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb9300737d0 con 0x7fb9140052e0 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb935000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb930073650 con 0x7fb9301e6b40 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb935000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9300cc5c0 con 0x7fb9301f54b0 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 0x7fb93015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 0x7fb93015e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51972/0 (socket says 172.21.15.71:51972) 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 -- 172.21.15.71:0/325012408 learned_addr learned my addr 172.21.15.71:0/325012408 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92e800640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9301e6b40 0x7fb9301e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92d400640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9301f54b0 0x7fb9301423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 -- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9301f54b0 msgr2=0x7fb9301423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9301f54b0 0x7fb9301423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 -- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9301e6b40 msgr2=0x7fb9301e6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9301e6b40 0x7fb9301e6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:11.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 -- 172.21.15.71:0/325012408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93015ea60 con 0x7fb9140052e0 2024-09-16T07:31:11.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.132+0000 7fb92de00640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 0x7fb93015e520 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb9200066b0 tx=0x7fb9200039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:11.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.133+0000 7fb926000640 1 -- 172.21.15.71:0/325012408 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920011700 con 0x7fb9140052e0 2024-09-16T07:31:11.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.133+0000 7fb926000640 1 -- 172.21.15.71:0/325012408 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb920016020 con 0x7fb9140052e0 2024-09-16T07:31:11.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.133+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9300b3ea0 con 0x7fb9140052e0 2024-09-16T07:31:11.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.133+0000 7fb926000640 1 -- 172.21.15.71:0/325012408 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920011b40 con 0x7fb9140052e0 2024-09-16T07:31:11.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.133+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9300b4460 con 0x7fb9140052e0 2024-09-16T07:31:11.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.134+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93015f9f0 con 0x7fb9140052e0 2024-09-16T07:31:11.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.135+0000 7fb926000640 1 -- 172.21.15.71:0/325012408 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fb920011ce0 con 0x7fb9140052e0 2024-09-16T07:31:11.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.136+0000 7fb926000640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb910079960 0x7fb91007be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:11.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.136+0000 7fb926000640 1 -- 172.21.15.71:0/325012408 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fb92009a2a0 con 0x7fb9140052e0 2024-09-16T07:31:11.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.136+0000 7fb92d400640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb910079960 0x7fb91007be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:11.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.137+0000 7fb92d400640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb910079960 0x7fb91007be20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb93013d610 tx=0x7fb91c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:11.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.140+0000 7fb926000640 1 -- 172.21.15.71:0/325012408 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb920020020 con 0x7fb9140052e0 2024-09-16T07:31:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:11.365+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb9300d06f0 con 0x7fb9140052e0 2024-09-16T07:31:12.077 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-09-16T07:31:12.077+0000 7f8789800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:31:12.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.203+0000 7fb926000640 1 -- 172.21.15.71:0/325012408 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7fb920063fa0 con 0x7fb9140052e0 2024-09-16T07:31:12.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb910079960 msgr2=0x7fb91007be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:12.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb910079960 0x7fb91007be20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb93013d610 tx=0x7fb91c005f90 comp rx=0 tx=0).stop 2024-09-16T07:31:12.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 msgr2=0x7fb93015e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:12.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 0x7fb93015e520 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb9200066b0 tx=0x7fb9200039c0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 shutdown_connections 2024-09-16T07:31:12.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb910079960 0x7fb91007be20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9301e6b40 0x7fb9301e6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9301f54b0 0x7fb9301423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 --2- 172.21.15.71:0/325012408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9140052e0 0x7fb93015e520 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 >> 172.21.15.71:0/325012408 conn(0x7fb9300b1840 msgr2=0x7fb9300a9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:12.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 shutdown_connections 2024-09-16T07:31:12.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.206+0000 7fb935000640 1 -- 172.21.15.71:0/325012408 wait complete. 2024-09-16T07:31:12.224 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:31:12.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:31:12.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 -- 172.21.15.71:0/734660091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f00145750 msgr2=0x7f1f00073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:12.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 --2- 172.21.15.71:0/734660091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f00145750 0x7f1f00073440 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f1f000cc550 tx=0x7f1ef002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 -- 172.21.15.71:0/734660091 shutdown_connections 2024-09-16T07:31:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 --2- 172.21.15.71:0/734660091 >> v2:172.21.15.177:3300/0 conn(0x7f1f000ccb00 0x7f1f0015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 --2- 172.21.15.71:0/734660091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f0006efa0 0x7f1f000cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 --2- 172.21.15.71:0/734660091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f00145750 0x7f1f00073440 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 -- 172.21.15.71:0/734660091 >> 172.21.15.71:0/734660091 conn(0x7f1f000b1840 msgr2=0x7f1f000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 -- 172.21.15.71:0/734660091 shutdown_connections 2024-09-16T07:31:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.385+0000 7f1f06800640 1 -- 172.21.15.71:0/734660091 wait complete. 2024-09-16T07:31:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.387+0000 7f1f06800640 1 Processor -- start 2024-09-16T07:31:12.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.387+0000 7f1f06800640 1 -- start start 2024-09-16T07:31:12.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1f06800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f0006efa0 0x7f1f0013da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:12.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1f06800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f000ccb00 0x7f1f000ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1f06800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f00145750 0x7f1f000ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1f06800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001600b0 con 0x7f1f0006efa0 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1f06800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0015ff30 con 0x7f1f000ccb00 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1f06800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1f00160230 con 0x7f1f00145750 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1eff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f0006efa0 0x7f1f0013da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1eff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f0006efa0 0x7f1f0013da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51992/0 (socket says 172.21.15.71:51992) 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1eff400640 1 -- 172.21.15.71:0/896820186 learned_addr learned my addr 172.21.15.71:0/896820186 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1efea00640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f000ccb00 0x7f1f000ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1effe00640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f00145750 0x7f1f000ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1eff400640 1 -- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f00145750 msgr2=0x7f1f000ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1eff400640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f00145750 0x7f1f000ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1eff400640 1 -- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f000ccb00 msgr2=0x7f1f000ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1eff400640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f000ccb00 0x7f1f000ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1eff400640 1 -- 172.21.15.71:0/896820186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0013edc0 con 0x7f1f0006efa0 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.388+0000 7f1effe00640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f00145750 0x7f1f000ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.389+0000 7f1eff400640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f0006efa0 0x7f1f0013da30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0034040 tx=0x7f1ef00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.389+0000 7f1ef7400640 1 -- 172.21.15.71:0/896820186 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef003e030 con 0x7f1f0006efa0 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.389+0000 7f1ef7400640 1 -- 172.21.15.71:0/896820186 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ef00047a0 con 0x7f1f0006efa0 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.389+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0013f0a0 con 0x7f1f0006efa0 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.389+0000 7f1ef7400640 1 -- 172.21.15.71:0/896820186 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0035040 con 0x7f1f0006efa0 2024-09-16T07:31:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.389+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0013f660 con 0x7f1f0006efa0 2024-09-16T07:31:12.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.390+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f00160710 con 0x7f1f0006efa0 2024-09-16T07:31:12.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.391+0000 7f1ef7400640 1 -- 172.21.15.71:0/896820186 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f1ef0041030 con 0x7f1f0006efa0 2024-09-16T07:31:12.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.392+0000 7f1ef7400640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ee0079930 0x7f1ee007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:12.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.393+0000 7f1ef7400640 1 -- 172.21.15.71:0/896820186 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f1ef00b9e90 con 0x7f1f0006efa0 2024-09-16T07:31:12.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.393+0000 7f1efea00640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ee0079930 0x7f1ee007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:12.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.394+0000 7f1efea00640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ee0079930 0x7f1ee007bdf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1f0006ac10 tx=0x7f1eec00a2d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:12.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.397+0000 7f1ef7400640 1 -- 172.21.15.71:0/896820186 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1ef0083620 con 0x7f1f0006efa0 2024-09-16T07:31:12.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:12.615+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1f0015f710 con 0x7f1f0006efa0 2024-09-16T07:31:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.211+0000 7f1ef7400640 1 -- 172.21.15.71:0/896820186 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7f1ef0088050 con 0x7f1f0006efa0 2024-09-16T07:31:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ee0079930 msgr2=0x7f1ee007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ee0079930 0x7f1ee007bdf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1f0006ac10 tx=0x7f1eec00a2d0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f0006efa0 msgr2=0x7f1f0013da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f0006efa0 0x7f1f0013da30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0034040 tx=0x7f1ef00039e0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 shutdown_connections 2024-09-16T07:31:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ee0079930 0x7f1ee007bdf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f00145750 0x7f1f000ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f000ccb00 0x7f1f000ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 --2- 172.21.15.71:0/896820186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f0006efa0 0x7f1f0013da30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 >> 172.21.15.71:0/896820186 conn(0x7f1f000b1840 msgr2=0x7f1f000cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 shutdown_connections 2024-09-16T07:31:13.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.214+0000 7f1f06800640 1 -- 172.21.15.71:0/896820186 wait complete. 2024-09-16T07:31:13.231 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:31:13.231 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:31:13.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.390+0000 7f5a04600640 1 -- 172.21.15.71:0/3642541139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 msgr2=0x7f59e4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.390+0000 7f5a04600640 1 --2- 172.21.15.71:0/3642541139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 0x7f59e4004800 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009b50 tx=0x7f59e4005830 comp rx=0 tx=0).stop 2024-09-16T07:31:13.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.390+0000 7f5a02600640 1 -- 172.21.15.71:0/3642541139 reap_dead start 2024-09-16T07:31:13.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.390+0000 7f5a04600640 1 -- 172.21.15.71:0/3642541139 shutdown_connections 2024-09-16T07:31:13.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.390+0000 7f5a04600640 1 -- 172.21.15.71:0/3642541139 >> 172.21.15.71:0/3642541139 conn(0x7f59fc0b1950 msgr2=0x7f59fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:13.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.390+0000 7f5a04600640 1 -- 172.21.15.71:0/3642541139 shutdown_connections 2024-09-16T07:31:13.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.390+0000 7f5a04600640 1 -- 172.21.15.71:0/3642541139 wait complete. 2024-09-16T07:31:13.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.392+0000 7f5a04600640 1 Processor -- start 2024-09-16T07:31:13.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.392+0000 7f5a04600640 1 -- start start 2024-09-16T07:31:13.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a04600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 0x7f59fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:13.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a04600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59fc142c10 0x7f59fc144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:13.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a04600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59fc15e9b0 0x7f59fc15ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:13.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a04600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc0729a0 con 0x7f59e4004000 2024-09-16T07:31:13.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a04600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc0b4eb0 con 0x7f59fc142c10 2024-09-16T07:31:13.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a04600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f59fc072b20 con 0x7f59fc15e9b0 2024-09-16T07:31:13.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a01c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 0x7f59fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:13.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a01c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 0x7f59fc06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52002/0 (socket says 172.21.15.71:52002) 2024-09-16T07:31:13.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.393+0000 7f5a01c00640 1 -- 172.21.15.71:0/3074225215 learned_addr learned my addr 172.21.15.71:0/3074225215 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:13.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a01c00640 1 -- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59fc15e9b0 msgr2=0x7f59fc15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a01c00640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59fc15e9b0 0x7f59fc15ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a01c00640 1 -- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59fc142c10 msgr2=0x7f59fc144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a01200640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59fc142c10 0x7f59fc144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:13.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a01c00640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59fc142c10 0x7f59fc144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a01c00640 1 -- 172.21.15.71:0/3074225215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc06f2f0 con 0x7f59e4004000 2024-09-16T07:31:13.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a01c00640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 0x7f59fc06edb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f59f0000fb0 tx=0x7f59f00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:13.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f59f9600640 1 -- 172.21.15.71:0/3074225215 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0015070 con 0x7f59e4004000 2024-09-16T07:31:13.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f59f9600640 1 -- 172.21.15.71:0/3074225215 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59f0011940 con 0x7f59e4004000 2024-09-16T07:31:13.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f59f9600640 1 -- 172.21.15.71:0/3074225215 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0011d90 con 0x7f59e4004000 2024-09-16T07:31:13.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc06f590 con 0x7f59e4004000 2024-09-16T07:31:13.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.394+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc06fb50 con 0x7f59e4004000 2024-09-16T07:31:13.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.395+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc0cebf0 con 0x7f59e4004000 2024-09-16T07:31:13.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.396+0000 7f59f9600640 1 -- 172.21.15.71:0/3074225215 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f59f0018070 con 0x7f59e4004000 2024-09-16T07:31:13.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.398+0000 7f59f9600640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f59dc079930 0x7f59dc07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:13.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.398+0000 7f59f9600640 1 -- 172.21.15.71:0/3074225215 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f59f009a490 con 0x7f59e4004000 2024-09-16T07:31:13.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.398+0000 7f5a01200640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f59dc079930 0x7f59dc07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:13.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.399+0000 7f5a01200640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f59dc079930 0x7f59dc07bdf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009b50 tx=0x7f59e802d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:13.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.402+0000 7f59f9600640 1 -- 172.21.15.71:0/3074225215 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f59f0064210 con 0x7f59e4004000 2024-09-16T07:31:13.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.622+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59fc070a20 con 0x7f59e4004000 2024-09-16T07:31:13.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.623+0000 7f59f9600640 1 -- 172.21.15.71:0/3074225215 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7f59f0063b10 con 0x7f59e4004000 2024-09-16T07:31:13.623 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:13.623 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":56,"btime":"2024-09-16T07:31:13:204547+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1642949519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1642949519},{"type":"v1","addr":"172.21.15.177:6836","nonce":1642949519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/2594338176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2594338176},{"type":"v1","addr":"172.21.15.71:6834","nonce":2594338176}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/482130020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":482130020},{"type":"v1","addr":"172.21.15.71:6835","nonce":482130020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:31:09.520327+0000","modified":"2024-09-16T07:31:12.617058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/4183855149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":4183855149},{"type":"v1","addr":"172.21.15.177:6837","nonce":4183855149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":6}]} 2024-09-16T07:31:13.623 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 56 2024-09-16T07:31:13.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.625+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f59dc079930 msgr2=0x7f59dc07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.625+0000 7f5a04600640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f59dc079930 0x7f59dc07bdf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009b50 tx=0x7f59e802d040 comp rx=0 tx=0).stop 2024-09-16T07:31:13.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.625+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 msgr2=0x7f59fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.625+0000 7f5a04600640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 0x7f59fc06edb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f59f0000fb0 tx=0x7f59f00039c0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.626+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 shutdown_connections 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.626+0000 7f5a04600640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f59dc079930 0x7f59dc07bdf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.626+0000 7f5a04600640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59fc15e9b0 0x7f59fc15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.626+0000 7f5a04600640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59fc142c10 0x7f59fc144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.626+0000 7f5a04600640 1 --2- 172.21.15.71:0/3074225215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59e4004000 0x7f59fc06edb0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.626+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 >> 172.21.15.71:0/3074225215 conn(0x7f59fc0b1950 msgr2=0x7f59fc0ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.626+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 shutdown_connections 2024-09-16T07:31:13.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.626+0000 7f5a04600640 1 -- 172.21.15.71:0/3074225215 wait complete. 2024-09-16T07:31:13.642 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:31:13.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.800+0000 7fa10b600640 1 -- 172.21.15.71:0/2247766410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 msgr2=0x7fa1040729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.800+0000 7fa10b600640 1 --2- 172.21.15.71:0/2247766410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 0x7fa1040729a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0005fd0 tx=0x7fa0f002a620 comp rx=0 tx=0).stop 2024-09-16T07:31:13.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.800+0000 7fa10b600640 1 -- 172.21.15.71:0/2247766410 shutdown_connections 2024-09-16T07:31:13.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.800+0000 7fa10b600640 1 --2- 172.21.15.71:0/2247766410 >> v2:172.21.15.177:3300/0 conn(0x7fa1040ccfd0 0x7fa104072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.800+0000 7fa10b600640 1 --2- 172.21.15.71:0/2247766410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 0x7fa1040729a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.800+0000 7fa10b600640 1 -- 172.21.15.71:0/2247766410 >> 172.21.15.71:0/2247766410 conn(0x7fa1040b1950 msgr2=0x7fa1040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:13.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.800+0000 7fa10b600640 1 -- 172.21.15.71:0/2247766410 shutdown_connections 2024-09-16T07:31:13.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.800+0000 7fa10b600640 1 -- 172.21.15.71:0/2247766410 wait complete. 2024-09-16T07:31:13.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.802+0000 7fa10b600640 1 Processor -- start 2024-09-16T07:31:13.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.802+0000 7fa10b600640 1 -- start start 2024-09-16T07:31:13.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa10b600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1040ccfd0 0x7fa1041e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:13.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa10b600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 0x7fa1041f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:13.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa10b600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1041445d0 0x7fa104141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:13.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa10b600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa104073420 con 0x7fa1040cf300 2024-09-16T07:31:13.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa10b600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040b4eb0 con 0x7fa1041445d0 2024-09-16T07:31:13.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa10b600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1040735a0 con 0x7fa1040ccfd0 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa103e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 0x7fa1041f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa103e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 0x7fa1041f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52022/0 (socket says 172.21.15.71:52022) 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa103e00640 1 -- 172.21.15.71:0/1961922224 learned_addr learned my addr 172.21.15.71:0/1961922224 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa108c00640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1040ccfd0 0x7fa1041e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.803+0000 7fa109600640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1041445d0 0x7fa104141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa103e00640 1 -- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1040ccfd0 msgr2=0x7fa1041e47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa103e00640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1040ccfd0 0x7fa1041e47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa103e00640 1 -- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1041445d0 msgr2=0x7fa104141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa103e00640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1041445d0 0x7fa104141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa103e00640 1 -- 172.21.15.71:0/1961922224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1041f3e30 con 0x7fa1040cf300 2024-09-16T07:31:13.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa103e00640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 0x7fa1041f38f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009850 tx=0x7fa0f002ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:13.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa100c00640 1 -- 172.21.15.71:0/1961922224 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f003f070 con 0x7fa1040cf300 2024-09-16T07:31:13.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa100c00640 1 -- 172.21.15.71:0/1961922224 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0f0004d20 con 0x7fa1040cf300 2024-09-16T07:31:13.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa100c00640 1 -- 172.21.15.71:0/1961922224 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0035030 con 0x7fa1040cf300 2024-09-16T07:31:13.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa104141bc0 con 0x7fa1040cf300 2024-09-16T07:31:13.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.804+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1041e5070 con 0x7fa1040cf300 2024-09-16T07:31:13.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.805+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa104073a00 con 0x7fa1040cf300 2024-09-16T07:31:13.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.806+0000 7fa100c00640 1 -- 172.21.15.71:0/1961922224 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fa0f0042070 con 0x7fa1040cf300 2024-09-16T07:31:13.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.807+0000 7fa100c00640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0e4079930 0x7fa0e407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:13.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.807+0000 7fa100c00640 1 -- 172.21.15.71:0/1961922224 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fa0f00ba500 con 0x7fa1040cf300 2024-09-16T07:31:13.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.808+0000 7fa108c00640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0e4079930 0x7fa0e407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:13.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.808+0000 7fa108c00640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0e4079930 0x7fa0e407bdf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80015e0 tx=0x7fa0f8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:13.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.812+0000 7fa100c00640 1 -- 172.21.15.71:0/1961922224 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa1041f3e30 con 0x7fa1040cf300 2024-09-16T07:31:13.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.995+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa104143ba0 con 0x7fa1040cf300 2024-09-16T07:31:13.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.996+0000 7fa100c00640 1 -- 172.21.15.71:0/1961922224 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16787 (secure 0 0 0) 0x7fa0f0084280 con 0x7fa1040cf300 2024-09-16T07:31:13.996 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:13.996 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":92,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:31:11.091513+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:31:07.265563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T07:31:08.483774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0e4079930 msgr2=0x7fa0e407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0e4079930 0x7fa0e407bdf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80015e0 tx=0x7fa0f8009040 comp rx=0 tx=0).stop 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 msgr2=0x7fa1041f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 0x7fa1041f38f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009850 tx=0x7fa0f002ae70 comp rx=0 tx=0).stop 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 shutdown_connections 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0e4079930 0x7fa0e407bdf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1041445d0 0x7fa104141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1040cf300 0x7fa1041f38f0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 --2- 172.21.15.71:0/1961922224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1040ccfd0 0x7fa1041e47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 >> 172.21.15.71:0/1961922224 conn(0x7fa1040b1950 msgr2=0x7fa1040ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 shutdown_connections 2024-09-16T07:31:13.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:13.999+0000 7fa10b600640 1 -- 172.21.15.71:0/1961922224 wait complete. 2024-09-16T07:31:14.016 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:31:14.016 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:31:14.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.173+0000 7fd062e00640 1 -- 172.21.15.71:0/2059653469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 msgr2=0x7fd05c06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.173+0000 7fd062e00640 1 --2- 172.21.15.71:0/2059653469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 0x7fd05c06ef50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0cd5d0 tx=0x7fd04c02a580 comp rx=0 tx=0).stop 2024-09-16T07:31:14.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.173+0000 7fd062e00640 1 -- 172.21.15.71:0/2059653469 shutdown_connections 2024-09-16T07:31:14.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.173+0000 7fd062e00640 1 --2- 172.21.15.71:0/2059653469 >> v2:172.21.15.177:3300/0 conn(0x7fd05c072df0 0x7fd05c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.173+0000 7fd062e00640 1 --2- 172.21.15.71:0/2059653469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 0x7fd05c06ef50 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.173+0000 7fd062e00640 1 -- 172.21.15.71:0/2059653469 >> 172.21.15.71:0/2059653469 conn(0x7fd05c0b1950 msgr2=0x7fd05c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:14.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.173+0000 7fd062e00640 1 -- 172.21.15.71:0/2059653469 shutdown_connections 2024-09-16T07:31:14.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.173+0000 7fd062e00640 1 -- 172.21.15.71:0/2059653469 wait complete. 2024-09-16T07:31:14.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.175+0000 7fd062e00640 1 Processor -- start 2024-09-16T07:31:14.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.175+0000 7fd062e00640 1 -- start start 2024-09-16T07:31:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd062e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05c072df0 0x7fd05c142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd062e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd05c0af3d0 0x7fd05c160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd062e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 0x7fd05c160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd062e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c0737e0 con 0x7fd05c161550 2024-09-16T07:31:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd062e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c06fdb0 con 0x7fd05c072df0 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd062e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c073960 con 0x7fd05c0af3d0 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd060e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 0x7fd05c160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd060e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 0x7fd05c160560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52038/0 (socket says 172.21.15.71:52038) 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd060e00640 1 -- 172.21.15.71:0/3168372412 learned_addr learned my addr 172.21.15.71:0/3168372412 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd05be00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05c072df0 0x7fd05c142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.176+0000 7fd05b400640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd05c0af3d0 0x7fd05c160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd060e00640 1 -- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd05c0af3d0 msgr2=0x7fd05c160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd060e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd05c0af3d0 0x7fd05c160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd060e00640 1 -- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05c072df0 msgr2=0x7fd05c142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd060e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05c072df0 0x7fd05c142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd060e00640 1 -- 172.21.15.71:0/3168372412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c142c60 con 0x7fd05c161550 2024-09-16T07:31:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd05be00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05c072df0 0x7fd05c142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd060e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 0x7fd05c160560 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd05400cce0 tx=0x7fd05400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd053e00640 1 -- 172.21.15.71:0/3168372412 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05400f450 con 0x7fd05c161550 2024-09-16T07:31:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd053e00640 1 -- 172.21.15.71:0/3168372412 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd05400f700 con 0x7fd05c161550 2024-09-16T07:31:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd053e00640 1 -- 172.21.15.71:0/3168372412 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05400fb50 con 0x7fd05c161550 2024-09-16T07:31:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.177+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c1f7ee0 con 0x7fd05c161550 2024-09-16T07:31:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.178+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c1f84a0 con 0x7fd05c161550 2024-09-16T07:31:14.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.178+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c073de0 con 0x7fd05c161550 2024-09-16T07:31:14.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.180+0000 7fd053e00640 1 -- 172.21.15.71:0/3168372412 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fd05400fcf0 con 0x7fd05c161550 2024-09-16T07:31:14.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.180+0000 7fd053e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd03c079930 0x7fd03c07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:14.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.181+0000 7fd053e00640 1 -- 172.21.15.71:0/3168372412 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fd05409a320 con 0x7fd05c161550 2024-09-16T07:31:14.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.181+0000 7fd05be00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd03c079930 0x7fd03c07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:14.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.182+0000 7fd05be00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd03c079930 0x7fd03c07bdf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd04c033000 tx=0x7fd04c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:14.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.185+0000 7fd053e00640 1 -- 172.21.15.71:0/3168372412 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd05c142c60 con 0x7fd05c161550 2024-09-16T07:31:14.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.371+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd05c1f7c90 con 0x7fd03c079930 2024-09-16T07:31:14.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.376+0000 7fd053e00640 1 -- 172.21.15.71:0/3168372412 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239966 (secure 0 0 0) 0x7fd05c1f7c90 con 0x7fd03c079930 2024-09-16T07:31:14.379 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:14.380 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:31:14.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd03c079930 msgr2=0x7fd03c07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd03c079930 0x7fd03c07bdf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd04c033000 tx=0x7fd04c004730 comp rx=0 tx=0).stop 2024-09-16T07:31:14.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 msgr2=0x7fd05c160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 0x7fd05c160560 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd05400cce0 tx=0x7fd05400a620 comp rx=0 tx=0).stop 2024-09-16T07:31:14.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 shutdown_connections 2024-09-16T07:31:14.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd03c079930 0x7fd03c07bdf0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd05c161550 0x7fd05c160560 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd05c0af3d0 0x7fd05c160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 --2- 172.21.15.71:0/3168372412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd05c072df0 0x7fd05c142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 >> 172.21.15.71:0/3168372412 conn(0x7fd05c0b1950 msgr2=0x7fd05c15ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:14.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 shutdown_connections 2024-09-16T07:31:14.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.380+0000 7fd062e00640 1 -- 172.21.15.71:0/3168372412 wait complete. 2024-09-16T07:31:14.395 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-16T07:31:12.450800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":468528,"kb_used_data":9528,"kb_used_omap":12,"kb_used_meta":458803,"kb_avail":754506192,"statfs":{"total":773094113280,"available":772614340608,"internally_reserved":0,"allocated":9756672,"data_stored":4860203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":469814832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.366048"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098080+0000","last_change":"2024-09-16T07:31:10.110323+0000","last_active":"2024-09-16T07:31:11.098080+0000","last_peered":"2024-09-16T07:31:11.098080+0000","last_clean":"2024-09-16T07:31:11.098080+0000","last_became_active":"2024-09-16T07:31:10.109903+0000","last_became_peered":"2024-09-16T07:31:10.109903+0000","last_unstale":"2024-09-16T07:31:11.098080+0000","last_undegraded":"2024-09-16T07:31:11.098080+0000","last_fullsized":"2024-09-16T07:31:11.098080+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:07:37.868651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097320+0000","last_change":"2024-09-16T07:31:10.100078+0000","last_active":"2024-09-16T07:31:11.097320+0000","last_peered":"2024-09-16T07:31:11.097320+0000","last_clean":"2024-09-16T07:31:11.097320+0000","last_became_active":"2024-09-16T07:31:10.099758+0000","last_became_peered":"2024-09-16T07:31:10.099758+0000","last_unstale":"2024-09-16T07:31:11.097320+0000","last_undegraded":"2024-09-16T07:31:11.097320+0000","last_fullsized":"2024-09-16T07:31:11.097320+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:54.609963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097537+0000","last_change":"2024-09-16T07:31:10.102966+0000","last_active":"2024-09-16T07:31:11.097537+0000","last_peered":"2024-09-16T07:31:11.097537+0000","last_clean":"2024-09-16T07:31:11.097537+0000","last_became_active":"2024-09-16T07:31:10.101792+0000","last_became_peered":"2024-09-16T07:31:10.101792+0000","last_unstale":"2024-09-16T07:31:11.097537+0000","last_undegraded":"2024-09-16T07:31:11.097537+0000","last_fullsized":"2024-09-16T07:31:11.097537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:43.430870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.091714+0000","last_change":"2024-09-16T07:31:08.091714+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.091714+0000","last_undegraded":"2024-09-16T07:31:08.091714+0000","last_fullsized":"2024-09-16T07:31:08.091714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.089629+0000","last_change":"2024-09-16T07:31:08.089629+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.089629+0000","last_undegraded":"2024-09-16T07:31:08.089629+0000","last_fullsized":"2024-09-16T07:31:08.089629+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.092255+0000","last_change":"2024-09-16T07:31:08.092255+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.092255+0000","last_undegraded":"2024-09-16T07:31:08.092255+0000","last_fullsized":"2024-09-16T07:31:08.092255+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.089574+0000","last_change":"2024-09-16T07:31:08.089574+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.089574+0000","last_undegraded":"2024-09-16T07:31:08.089574+0000","last_fullsized":"2024-09-16T07:31:08.089574+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.096174+0000","last_change":"2024-09-16T07:31:08.096174+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.096174+0000","last_undegraded":"2024-09-16T07:31:08.096174+0000","last_fullsized":"2024-09-16T07:31:08.096174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.088240+0000","last_change":"2024-09-16T07:31:08.088240+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.088240+0000","last_undegraded":"2024-09-16T07:31:08.088240+0000","last_fullsized":"2024-09-16T07:31:08.088240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.096116+0000","last_change":"2024-09-16T07:31:08.096116+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.096116+0000","last_undegraded":"2024-09-16T07:31:08.096116+0000","last_fullsized":"2024-09-16T07:31:08.096116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.092311+0000","last_change":"2024-09-16T07:31:08.092311+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.092311+0000","last_undegraded":"2024-09-16T07:31:08.092311+0000","last_fullsized":"2024-09-16T07:31:08.092311+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097457+0000","last_change":"2024-09-16T07:31:09.098750+0000","last_active":"2024-09-16T07:31:11.097457+0000","last_peered":"2024-09-16T07:31:11.097457+0000","last_clean":"2024-09-16T07:31:11.097457+0000","last_became_active":"2024-09-16T07:31:09.098434+0000","last_became_peered":"2024-09-16T07:31:09.098434+0000","last_unstale":"2024-09-16T07:31:11.097457+0000","last_undegraded":"2024-09-16T07:31:11.097457+0000","last_fullsized":"2024-09-16T07:31:11.097457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:15:30.043800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.088238+0000","last_change":"2024-09-16T07:31:08.088238+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.088238+0000","last_undegraded":"2024-09-16T07:31:08.088238+0000","last_fullsized":"2024-09-16T07:31:08.088238+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.091041+0000","last_change":"2024-09-16T07:31:08.091041+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.091041+0000","last_undegraded":"2024-09-16T07:31:08.091041+0000","last_fullsized":"2024-09-16T07:31:08.091041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097728+0000","last_change":"2024-09-16T07:31:10.112501+0000","last_active":"2024-09-16T07:31:11.097728+0000","last_peered":"2024-09-16T07:31:11.097728+0000","last_clean":"2024-09-16T07:31:11.097728+0000","last_became_active":"2024-09-16T07:31:10.112301+0000","last_became_peered":"2024-09-16T07:31:10.112301+0000","last_unstale":"2024-09-16T07:31:11.097728+0000","last_undegraded":"2024-09-16T07:31:11.097728+0000","last_fullsized":"2024-09-16T07:31:11.097728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:36:26.532895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097554+0000","last_change":"2024-09-16T07:31:09.106277+0000","last_active":"2024-09-16T07:31:11.097554+0000","last_peered":"2024-09-16T07:31:11.097554+0000","last_clean":"2024-09-16T07:31:11.097554+0000","last_became_active":"2024-09-16T07:31:09.105995+0000","last_became_peered":"2024-09-16T07:31:09.105995+0000","last_unstale":"2024-09-16T07:31:11.097554+0000","last_undegraded":"2024-09-16T07:31:11.097554+0000","last_fullsized":"2024-09-16T07:31:11.097554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:17.850572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:08.079884+0000","last_change":"2024-09-16T07:31:08.079884+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.079884+0000","last_undegraded":"2024-09-16T07:31:08.079884+0000","last_fullsized":"2024-09-16T07:31:08.079884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.093329+0000","last_change":"2024-09-16T07:31:08.093329+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.093329+0000","last_undegraded":"2024-09-16T07:31:08.093329+0000","last_fullsized":"2024-09-16T07:31:08.093329+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.088192+0000","last_change":"2024-09-16T07:31:08.088192+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.088192+0000","last_undegraded":"2024-09-16T07:31:08.088192+0000","last_fullsized":"2024-09-16T07:31:08.088192+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098141+0000","last_change":"2024-09-16T07:31:09.106829+0000","last_active":"2024-09-16T07:31:11.098141+0000","last_peered":"2024-09-16T07:31:11.098141+0000","last_clean":"2024-09-16T07:31:11.098141+0000","last_became_active":"2024-09-16T07:31:09.106064+0000","last_became_peered":"2024-09-16T07:31:09.106064+0000","last_unstale":"2024-09-16T07:31:11.098141+0000","last_undegraded":"2024-09-16T07:31:11.098141+0000","last_fullsized":"2024-09-16T07:31:11.098141+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:23.786094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.088210+0000","last_change":"2024-09-16T07:31:08.088210+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.088210+0000","last_undegraded":"2024-09-16T07:31:08.088210+0000","last_fullsized":"2024-09-16T07:31:08.088210+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.095333+0000","last_change":"2024-09-16T07:31:08.095333+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.095333+0000","last_undegraded":"2024-09-16T07:31:08.095333+0000","last_fullsized":"2024-09-16T07:31:08.095333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.092835+0000","last_change":"2024-09-16T07:31:08.092835+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.092835+0000","last_undegraded":"2024-09-16T07:31:08.092835+0000","last_fullsized":"2024-09-16T07:31:08.092835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:31:09.082762+0000","last_change":"2024-09-16T07:31:09.082762+0000","last_active":"2024-09-16T07:31:09.082762+0000","last_peered":"2024-09-16T07:31:09.082762+0000","last_clean":"2024-09-16T07:31:09.082762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:09.082762+0000","last_undegraded":"2024-09-16T07:31:09.082762+0000","last_fullsized":"2024-09-16T07:31:09.082762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T07:31:08.091122+0000","last_change":"2024-09-16T07:31:08.091122+0000","last_active":"2024-09-16T07:31:08.079884+0000","last_peered":"2024-09-16T07:31:08.079884+0000","last_clean":"2024-09-16T07:31:08.079884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:31:08.091122+0000","last_undegraded":"2024-09-16T07:31:08.091122+0000","last_fullsized":"2024-09-16T07:31:08.091122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":235,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T07:31:08.096670+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:31:08.096670+0000","last_peered":"2024-09-16T07:31:08.096670+0000","last_clean":"2024-09-16T07:31:08.096670+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:31:08.096670+0000","last_undegraded":"2024-09-16T07:31:08.096670+0000","last_fullsized":"2024-09-16T07:31:08.096670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58600,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313240,"statfs":{"total":96636764160,"available":96576757760,"internally_reserved":0,"allocated":1069056,"data_stored":458661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1081344,"data_stored":464506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476843,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59192,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312648,"statfs":{"total":96636764160,"available":96576151552,"internally_reserved":0,"allocated":1675264,"data_stored":1054874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476842,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58600,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313240,"statfs":{"total":96636764160,"available":96576757760,"internally_reserved":0,"allocated":1069056,"data_stored":458661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58608,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313232,"statfs":{"total":96636764160,"available":96576749568,"internally_reserved":0,"allocated":1069056,"data_stored":458661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64060,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307780,"statfs":{"total":96636764160,"available":96571166720,"internally_reserved":0,"allocated":1019904,"data_stored":438165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509546,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46704,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325136,"statfs":{"total":96636764160,"available":96588939264,"internally_reserved":0,"allocated":1662976,"data_stored":1049029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64148,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307692,"statfs":{"total":96636764160,"available":96571076608,"internally_reserved":0,"allocated":1110016,"data_stored":477646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:31:14.400 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:31:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.593+0000 7fc3b5000640 1 -- 172.21.15.71:0/1989461902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 msgr2=0x7fc394003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.593+0000 7fc3b5000640 1 --2- 172.21.15.71:0/1989461902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 0x7fc394003de0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00017c0 tx=0x7fc394005830 comp rx=0 tx=0).stop 2024-09-16T07:31:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.593+0000 7fc3b5000640 1 -- 172.21.15.71:0/1989461902 shutdown_connections 2024-09-16T07:31:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.593+0000 7fc3b5000640 1 --2- 172.21.15.71:0/1989461902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 0x7fc394003de0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.593+0000 7fc3b5000640 1 -- 172.21.15.71:0/1989461902 >> 172.21.15.71:0/1989461902 conn(0x7fc3b00b1950 msgr2=0x7fc3b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.593+0000 7fc3b5000640 1 -- 172.21.15.71:0/1989461902 shutdown_connections 2024-09-16T07:31:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.593+0000 7fc3b5000640 1 -- 172.21.15.71:0/1989461902 wait complete. 2024-09-16T07:31:14.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.594+0000 7fc3b5000640 1 Processor -- start 2024-09-16T07:31:14.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.595+0000 7fc3b5000640 1 -- start start 2024-09-16T07:31:14.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.595+0000 7fc3b5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 0x7fc3b00b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:14.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.595+0000 7fc3b5000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3b00d02b0 0x7fc3b00d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:14.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.595+0000 7fc3b5000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3b0158b30 0x7fc3b0158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:14.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3b5000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00ad0c0 con 0x7fc394003a00 2024-09-16T07:31:14.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3b5000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00acf40 con 0x7fc3b0158b30 2024-09-16T07:31:14.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3b5000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b00ad240 con 0x7fc3b00d02b0 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 0x7fc3b00b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 0x7fc3b00b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52050/0 (socket says 172.21.15.71:52050) 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 -- 172.21.15.71:0/1931447659 learned_addr learned my addr 172.21.15.71:0/1931447659 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ae800640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3b0158b30 0x7fc3b0158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ad400640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3b00d02b0 0x7fc3b00d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 -- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3b00d02b0 msgr2=0x7fc3b00d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3b00d02b0 0x7fc3b00d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 -- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3b0158b30 msgr2=0x7fc3b0158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3b0158b30 0x7fc3b0158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 -- 172.21.15.71:0/1931447659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b01f4980 con 0x7fc394003a00 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.596+0000 7fc3ade00640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 0x7fc3b00b48e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0000c00 tx=0x7fc3a0006140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:14.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.597+0000 7fc3a6000640 1 -- 172.21.15.71:0/1931447659 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0040030 con 0x7fc394003a00 2024-09-16T07:31:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.597+0000 7fc3a6000640 1 -- 172.21.15.71:0/1931447659 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3a0004640 con 0x7fc394003a00 2024-09-16T07:31:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.597+0000 7fc3a6000640 1 -- 172.21.15.71:0/1931447659 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0037040 con 0x7fc394003a00 2024-09-16T07:31:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.597+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b01f4c40 con 0x7fc394003a00 2024-09-16T07:31:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.597+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b01f5200 con 0x7fc394003a00 2024-09-16T07:31:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.598+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b0074000 con 0x7fc394003a00 2024-09-16T07:31:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.599+0000 7fc3a6000640 1 -- 172.21.15.71:0/1931447659 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fc3a0043070 con 0x7fc394003a00 2024-09-16T07:31:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.600+0000 7fc3a6000640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc390079960 0x7fc39007be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.600+0000 7fc3a6000640 1 -- 172.21.15.71:0/1931447659 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fc3a00b9130 con 0x7fc394003a00 2024-09-16T07:31:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.600+0000 7fc3ad400640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc390079960 0x7fc39007be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:14.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.601+0000 7fc3ad400640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc390079960 0x7fc39007be20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc39c001c80 tx=0x7fc39c001c10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:14.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.604+0000 7fc3a6000640 1 -- 172.21.15.71:0/1931447659 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc3a004a020 con 0x7fc394003a00 2024-09-16T07:31:14.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.787+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3b00b21a0 con 0x7fc390079960 2024-09-16T07:31:14.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.793+0000 7fc3a6000640 1 -- 172.21.15.71:0/1931447659 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252074 (secure 0 0 0) 0x7fc3b00b21a0 con 0x7fc390079960 2024-09-16T07:31:14.795 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:14.796 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:31:14.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.796+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc390079960 msgr2=0x7fc39007be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc390079960 0x7fc39007be20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc39c001c80 tx=0x7fc39c001c10 comp rx=0 tx=0).stop 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 msgr2=0x7fc3b00b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 0x7fc3b00b48e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0000c00 tx=0x7fc3a0006140 comp rx=0 tx=0).stop 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 shutdown_connections 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc390079960 0x7fc39007be20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3b0158b30 0x7fc3b0158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3b00d02b0 0x7fc3b00d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 --2- 172.21.15.71:0/1931447659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc394003a00 0x7fc3b00b48e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 >> 172.21.15.71:0/1931447659 conn(0x7fc3b00b1950 msgr2=0x7fc3b006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 shutdown_connections 2024-09-16T07:31:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:14.797+0000 7fc3b5000640 1 -- 172.21.15.71:0/1931447659 wait complete. 2024-09-16T07:31:14.812 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-16T07:31:14.451292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":468956,"kb_used_data":9948,"kb_used_omap":12,"kb_used_meta":458803,"kb_avail":754505764,"statfs":{"total":773094113280,"available":772613902336,"internally_reserved":0,"allocated":10186752,"data_stored":5050748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":469814832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001360"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562461+0000","last_change":"2024-09-16T07:31:10.109940+0000","last_active":"2024-09-16T07:31:11.562461+0000","last_peered":"2024-09-16T07:31:11.562461+0000","last_clean":"2024-09-16T07:31:11.562461+0000","last_became_active":"2024-09-16T07:31:10.109684+0000","last_became_peered":"2024-09-16T07:31:10.109684+0000","last_unstale":"2024-09-16T07:31:11.562461+0000","last_undegraded":"2024-09-16T07:31:11.562461+0000","last_fullsized":"2024-09-16T07:31:11.562461+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:20.000856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098080+0000","last_change":"2024-09-16T07:31:10.110323+0000","last_active":"2024-09-16T07:31:11.098080+0000","last_peered":"2024-09-16T07:31:11.098080+0000","last_clean":"2024-09-16T07:31:11.098080+0000","last_became_active":"2024-09-16T07:31:10.109903+0000","last_became_peered":"2024-09-16T07:31:10.109903+0000","last_unstale":"2024-09-16T07:31:11.098080+0000","last_undegraded":"2024-09-16T07:31:11.098080+0000","last_fullsized":"2024-09-16T07:31:11.098080+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:07:37.868651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096317+0000","last_change":"2024-09-16T07:31:10.121005+0000","last_active":"2024-09-16T07:31:11.096317+0000","last_peered":"2024-09-16T07:31:11.096317+0000","last_clean":"2024-09-16T07:31:11.096317+0000","last_became_active":"2024-09-16T07:31:10.120570+0000","last_became_peered":"2024-09-16T07:31:10.120570+0000","last_unstale":"2024-09-16T07:31:11.096317+0000","last_undegraded":"2024-09-16T07:31:11.096317+0000","last_fullsized":"2024-09-16T07:31:11.096317+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:15.021815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562766+0000","last_change":"2024-09-16T07:31:10.111734+0000","last_active":"2024-09-16T07:31:11.562766+0000","last_peered":"2024-09-16T07:31:11.562766+0000","last_clean":"2024-09-16T07:31:11.562766+0000","last_became_active":"2024-09-16T07:31:10.111052+0000","last_became_peered":"2024-09-16T07:31:10.111052+0000","last_unstale":"2024-09-16T07:31:11.562766+0000","last_undegraded":"2024-09-16T07:31:11.562766+0000","last_fullsized":"2024-09-16T07:31:11.562766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:34:09.480342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561668+0000","last_change":"2024-09-16T07:31:10.113918+0000","last_active":"2024-09-16T07:31:11.561668+0000","last_peered":"2024-09-16T07:31:11.561668+0000","last_clean":"2024-09-16T07:31:11.561668+0000","last_became_active":"2024-09-16T07:31:10.113543+0000","last_became_peered":"2024-09-16T07:31:10.113543+0000","last_unstale":"2024-09-16T07:31:11.561668+0000","last_undegraded":"2024-09-16T07:31:11.561668+0000","last_fullsized":"2024-09-16T07:31:11.561668+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:07:53.347063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096065+0000","last_change":"2024-09-16T07:31:10.119763+0000","last_active":"2024-09-16T07:31:11.096065+0000","last_peered":"2024-09-16T07:31:11.096065+0000","last_clean":"2024-09-16T07:31:11.096065+0000","last_became_active":"2024-09-16T07:31:10.119503+0000","last_became_peered":"2024-09-16T07:31:10.119503+0000","last_unstale":"2024-09-16T07:31:11.096065+0000","last_undegraded":"2024-09-16T07:31:11.096065+0000","last_fullsized":"2024-09-16T07:31:11.096065+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:19.849181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097320+0000","last_change":"2024-09-16T07:31:10.100078+0000","last_active":"2024-09-16T07:31:11.097320+0000","last_peered":"2024-09-16T07:31:11.097320+0000","last_clean":"2024-09-16T07:31:11.097320+0000","last_became_active":"2024-09-16T07:31:10.099758+0000","last_became_peered":"2024-09-16T07:31:10.099758+0000","last_unstale":"2024-09-16T07:31:11.097320+0000","last_undegraded":"2024-09-16T07:31:11.097320+0000","last_fullsized":"2024-09-16T07:31:11.097320+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:54.609963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561266+0000","last_change":"2024-09-16T07:31:10.110406+0000","last_active":"2024-09-16T07:31:11.561266+0000","last_peered":"2024-09-16T07:31:11.561266+0000","last_clean":"2024-09-16T07:31:11.561266+0000","last_became_active":"2024-09-16T07:31:10.110186+0000","last_became_peered":"2024-09-16T07:31:10.110186+0000","last_unstale":"2024-09-16T07:31:11.561266+0000","last_undegraded":"2024-09-16T07:31:11.561266+0000","last_fullsized":"2024-09-16T07:31:11.561266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:33.424176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562564+0000","last_change":"2024-09-16T07:31:10.112395+0000","last_active":"2024-09-16T07:31:11.562564+0000","last_peered":"2024-09-16T07:31:11.562564+0000","last_clean":"2024-09-16T07:31:11.562564+0000","last_became_active":"2024-09-16T07:31:10.112149+0000","last_became_peered":"2024-09-16T07:31:10.112149+0000","last_unstale":"2024-09-16T07:31:11.562564+0000","last_undegraded":"2024-09-16T07:31:11.562564+0000","last_fullsized":"2024-09-16T07:31:11.562564+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:15.076603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096303+0000","last_change":"2024-09-16T07:31:10.116551+0000","last_active":"2024-09-16T07:31:11.096303+0000","last_peered":"2024-09-16T07:31:11.096303+0000","last_clean":"2024-09-16T07:31:11.096303+0000","last_became_active":"2024-09-16T07:31:10.115213+0000","last_became_peered":"2024-09-16T07:31:10.115213+0000","last_unstale":"2024-09-16T07:31:11.096303+0000","last_undegraded":"2024-09-16T07:31:11.096303+0000","last_fullsized":"2024-09-16T07:31:11.096303+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:54.653867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562518+0000","last_change":"2024-09-16T07:31:10.122229+0000","last_active":"2024-09-16T07:31:11.562518+0000","last_peered":"2024-09-16T07:31:11.562518+0000","last_clean":"2024-09-16T07:31:11.562518+0000","last_became_active":"2024-09-16T07:31:10.120746+0000","last_became_peered":"2024-09-16T07:31:10.120746+0000","last_unstale":"2024-09-16T07:31:11.562518+0000","last_undegraded":"2024-09-16T07:31:11.562518+0000","last_fullsized":"2024-09-16T07:31:11.562518+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:02.007902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561184+0000","last_change":"2024-09-16T07:31:10.110079+0000","last_active":"2024-09-16T07:31:11.561184+0000","last_peered":"2024-09-16T07:31:11.561184+0000","last_clean":"2024-09-16T07:31:11.561184+0000","last_became_active":"2024-09-16T07:31:10.109368+0000","last_became_peered":"2024-09-16T07:31:10.109368+0000","last_unstale":"2024-09-16T07:31:11.561184+0000","last_undegraded":"2024-09-16T07:31:11.561184+0000","last_fullsized":"2024-09-16T07:31:11.561184+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:20.422206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561233+0000","last_change":"2024-09-16T07:31:10.115824+0000","last_active":"2024-09-16T07:31:11.561233+0000","last_peered":"2024-09-16T07:31:11.561233+0000","last_clean":"2024-09-16T07:31:11.561233+0000","last_became_active":"2024-09-16T07:31:10.112474+0000","last_became_peered":"2024-09-16T07:31:10.112474+0000","last_unstale":"2024-09-16T07:31:11.561233+0000","last_undegraded":"2024-09-16T07:31:11.561233+0000","last_fullsized":"2024-09-16T07:31:11.561233+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:44.846895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098429+0000","last_change":"2024-09-16T07:31:10.121387+0000","last_active":"2024-09-16T07:31:11.098429+0000","last_peered":"2024-09-16T07:31:11.098429+0000","last_clean":"2024-09-16T07:31:11.098429+0000","last_became_active":"2024-09-16T07:31:10.121088+0000","last_became_peered":"2024-09-16T07:31:10.121088+0000","last_unstale":"2024-09-16T07:31:11.098429+0000","last_undegraded":"2024-09-16T07:31:11.098429+0000","last_fullsized":"2024-09-16T07:31:11.098429+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:23.888808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097537+0000","last_change":"2024-09-16T07:31:10.102966+0000","last_active":"2024-09-16T07:31:11.097537+0000","last_peered":"2024-09-16T07:31:11.097537+0000","last_clean":"2024-09-16T07:31:11.097537+0000","last_became_active":"2024-09-16T07:31:10.101792+0000","last_became_peered":"2024-09-16T07:31:10.101792+0000","last_unstale":"2024-09-16T07:31:11.097537+0000","last_undegraded":"2024-09-16T07:31:11.097537+0000","last_fullsized":"2024-09-16T07:31:11.097537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:43.430870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.095958+0000","last_change":"2024-09-16T07:31:10.119486+0000","last_active":"2024-09-16T07:31:11.095958+0000","last_peered":"2024-09-16T07:31:11.095958+0000","last_clean":"2024-09-16T07:31:11.095958+0000","last_became_active":"2024-09-16T07:31:10.118827+0000","last_became_peered":"2024-09-16T07:31:10.118827+0000","last_unstale":"2024-09-16T07:31:11.095958+0000","last_undegraded":"2024-09-16T07:31:11.095958+0000","last_fullsized":"2024-09-16T07:31:11.095958+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:33.948095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562302+0000","last_change":"2024-09-16T07:31:10.112293+0000","last_active":"2024-09-16T07:31:11.562302+0000","last_peered":"2024-09-16T07:31:11.562302+0000","last_clean":"2024-09-16T07:31:11.562302+0000","last_became_active":"2024-09-16T07:31:10.112038+0000","last_became_peered":"2024-09-16T07:31:10.112038+0000","last_unstale":"2024-09-16T07:31:11.562302+0000","last_undegraded":"2024-09-16T07:31:11.562302+0000","last_fullsized":"2024-09-16T07:31:11.562302+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:33.948095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561985+0000","last_change":"2024-09-16T07:31:10.116392+0000","last_active":"2024-09-16T07:31:11.561985+0000","last_peered":"2024-09-16T07:31:11.561985+0000","last_clean":"2024-09-16T07:31:11.561985+0000","last_became_active":"2024-09-16T07:31:10.116034+0000","last_became_peered":"2024-09-16T07:31:10.116034+0000","last_unstale":"2024-09-16T07:31:11.561985+0000","last_undegraded":"2024-09-16T07:31:11.561985+0000","last_fullsized":"2024-09-16T07:31:11.561985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:58:51.808435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098829+0000","last_change":"2024-09-16T07:31:10.118415+0000","last_active":"2024-09-16T07:31:11.098829+0000","last_peered":"2024-09-16T07:31:11.098829+0000","last_clean":"2024-09-16T07:31:11.098829+0000","last_became_active":"2024-09-16T07:31:10.118071+0000","last_became_peered":"2024-09-16T07:31:10.118071+0000","last_unstale":"2024-09-16T07:31:11.098829+0000","last_undegraded":"2024-09-16T07:31:11.098829+0000","last_fullsized":"2024-09-16T07:31:11.098829+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:58:51.808435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098381+0000","last_change":"2024-09-16T07:31:10.117005+0000","last_active":"2024-09-16T07:31:11.098381+0000","last_peered":"2024-09-16T07:31:11.098381+0000","last_clean":"2024-09-16T07:31:11.098381+0000","last_became_active":"2024-09-16T07:31:10.116729+0000","last_became_peered":"2024-09-16T07:31:10.116729+0000","last_unstale":"2024-09-16T07:31:11.098381+0000","last_undegraded":"2024-09-16T07:31:11.098381+0000","last_fullsized":"2024-09-16T07:31:11.098381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:48.847787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562455+0000","last_change":"2024-09-16T07:31:10.103496+0000","last_active":"2024-09-16T07:31:11.562455+0000","last_peered":"2024-09-16T07:31:11.562455+0000","last_clean":"2024-09-16T07:31:11.562455+0000","last_became_active":"2024-09-16T07:31:10.103239+0000","last_became_peered":"2024-09-16T07:31:10.103239+0000","last_unstale":"2024-09-16T07:31:11.562455+0000","last_undegraded":"2024-09-16T07:31:11.562455+0000","last_fullsized":"2024-09-16T07:31:11.562455+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:18.547637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098872+0000","last_change":"2024-09-16T07:31:10.115631+0000","last_active":"2024-09-16T07:31:11.098872+0000","last_peered":"2024-09-16T07:31:11.098872+0000","last_clean":"2024-09-16T07:31:11.098872+0000","last_became_active":"2024-09-16T07:31:10.114671+0000","last_became_peered":"2024-09-16T07:31:10.114671+0000","last_unstale":"2024-09-16T07:31:11.098872+0000","last_undegraded":"2024-09-16T07:31:11.098872+0000","last_fullsized":"2024-09-16T07:31:11.098872+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:33.570183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561230+0000","last_change":"2024-09-16T07:31:10.109526+0000","last_active":"2024-09-16T07:31:11.561230+0000","last_peered":"2024-09-16T07:31:11.561230+0000","last_clean":"2024-09-16T07:31:11.561230+0000","last_became_active":"2024-09-16T07:31:10.109257+0000","last_became_peered":"2024-09-16T07:31:10.109257+0000","last_unstale":"2024-09-16T07:31:11.561230+0000","last_undegraded":"2024-09-16T07:31:11.561230+0000","last_fullsized":"2024-09-16T07:31:11.561230+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:52.441261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562011+0000","last_change":"2024-09-16T07:31:10.113492+0000","last_active":"2024-09-16T07:31:11.562011+0000","last_peered":"2024-09-16T07:31:11.562011+0000","last_clean":"2024-09-16T07:31:11.562011+0000","last_became_active":"2024-09-16T07:31:10.113204+0000","last_became_peered":"2024-09-16T07:31:10.113204+0000","last_unstale":"2024-09-16T07:31:11.562011+0000","last_undegraded":"2024-09-16T07:31:11.562011+0000","last_fullsized":"2024-09-16T07:31:11.562011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:33.570183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562324+0000","last_change":"2024-09-16T07:31:10.119852+0000","last_active":"2024-09-16T07:31:11.562324+0000","last_peered":"2024-09-16T07:31:11.562324+0000","last_clean":"2024-09-16T07:31:11.562324+0000","last_became_active":"2024-09-16T07:31:10.118968+0000","last_became_peered":"2024-09-16T07:31:10.118968+0000","last_unstale":"2024-09-16T07:31:11.562324+0000","last_undegraded":"2024-09-16T07:31:11.562324+0000","last_fullsized":"2024-09-16T07:31:11.562324+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:46.362060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562228+0000","last_change":"2024-09-16T07:31:10.110372+0000","last_active":"2024-09-16T07:31:11.562228+0000","last_peered":"2024-09-16T07:31:11.562228+0000","last_clean":"2024-09-16T07:31:11.562228+0000","last_became_active":"2024-09-16T07:31:10.107926+0000","last_became_peered":"2024-09-16T07:31:10.107926+0000","last_unstale":"2024-09-16T07:31:11.562228+0000","last_undegraded":"2024-09-16T07:31:11.562228+0000","last_fullsized":"2024-09-16T07:31:11.562228+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:14.966715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561779+0000","last_change":"2024-09-16T07:31:10.108617+0000","last_active":"2024-09-16T07:31:11.561779+0000","last_peered":"2024-09-16T07:31:11.561779+0000","last_clean":"2024-09-16T07:31:11.561779+0000","last_became_active":"2024-09-16T07:31:10.102086+0000","last_became_peered":"2024-09-16T07:31:10.102086+0000","last_unstale":"2024-09-16T07:31:11.561779+0000","last_undegraded":"2024-09-16T07:31:11.561779+0000","last_fullsized":"2024-09-16T07:31:11.561779+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:02.900952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562122+0000","last_change":"2024-09-16T07:31:10.106505+0000","last_active":"2024-09-16T07:31:11.562122+0000","last_peered":"2024-09-16T07:31:11.562122+0000","last_clean":"2024-09-16T07:31:11.562122+0000","last_became_active":"2024-09-16T07:31:10.106242+0000","last_became_peered":"2024-09-16T07:31:10.106242+0000","last_unstale":"2024-09-16T07:31:11.562122+0000","last_undegraded":"2024-09-16T07:31:11.562122+0000","last_fullsized":"2024-09-16T07:31:11.562122+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:37.954316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562539+0000","last_change":"2024-09-16T07:31:10.103524+0000","last_active":"2024-09-16T07:31:11.562539+0000","last_peered":"2024-09-16T07:31:11.562539+0000","last_clean":"2024-09-16T07:31:11.562539+0000","last_became_active":"2024-09-16T07:31:10.103229+0000","last_became_peered":"2024-09-16T07:31:10.103229+0000","last_unstale":"2024-09-16T07:31:11.562539+0000","last_undegraded":"2024-09-16T07:31:11.562539+0000","last_fullsized":"2024-09-16T07:31:11.562539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:30.211385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098665+0000","last_change":"2024-09-16T07:31:10.122043+0000","last_active":"2024-09-16T07:31:11.098665+0000","last_peered":"2024-09-16T07:31:11.098665+0000","last_clean":"2024-09-16T07:31:11.098665+0000","last_became_active":"2024-09-16T07:31:10.121135+0000","last_became_peered":"2024-09-16T07:31:10.121135+0000","last_unstale":"2024-09-16T07:31:11.098665+0000","last_undegraded":"2024-09-16T07:31:11.098665+0000","last_fullsized":"2024-09-16T07:31:11.098665+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:15.076603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.095842+0000","last_change":"2024-09-16T07:31:10.117390+0000","last_active":"2024-09-16T07:31:11.095842+0000","last_peered":"2024-09-16T07:31:11.095842+0000","last_clean":"2024-09-16T07:31:11.095842+0000","last_became_active":"2024-09-16T07:31:10.117103+0000","last_became_peered":"2024-09-16T07:31:10.117103+0000","last_unstale":"2024-09-16T07:31:11.095842+0000","last_undegraded":"2024-09-16T07:31:11.095842+0000","last_fullsized":"2024-09-16T07:31:11.095842+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:12.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096190+0000","last_change":"2024-09-16T07:31:10.116426+0000","last_active":"2024-09-16T07:31:11.096190+0000","last_peered":"2024-09-16T07:31:11.096190+0000","last_clean":"2024-09-16T07:31:11.096190+0000","last_became_active":"2024-09-16T07:31:10.116146+0000","last_became_peered":"2024-09-16T07:31:10.116146+0000","last_unstale":"2024-09-16T07:31:11.096190+0000","last_undegraded":"2024-09-16T07:31:11.096190+0000","last_fullsized":"2024-09-16T07:31:11.096190+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:33.247532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562881+0000","last_change":"2024-09-16T07:31:10.114361+0000","last_active":"2024-09-16T07:31:11.562881+0000","last_peered":"2024-09-16T07:31:11.562881+0000","last_clean":"2024-09-16T07:31:11.562881+0000","last_became_active":"2024-09-16T07:31:10.113516+0000","last_became_peered":"2024-09-16T07:31:10.113516+0000","last_unstale":"2024-09-16T07:31:11.562881+0000","last_undegraded":"2024-09-16T07:31:11.562881+0000","last_fullsized":"2024-09-16T07:31:11.562881+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:14.029753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562281+0000","last_change":"2024-09-16T07:31:09.110085+0000","last_active":"2024-09-16T07:31:11.562281+0000","last_peered":"2024-09-16T07:31:11.562281+0000","last_clean":"2024-09-16T07:31:11.562281+0000","last_became_active":"2024-09-16T07:31:09.109833+0000","last_became_peered":"2024-09-16T07:31:09.109833+0000","last_unstale":"2024-09-16T07:31:11.562281+0000","last_undegraded":"2024-09-16T07:31:11.562281+0000","last_fullsized":"2024-09-16T07:31:11.562281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:15:17.171055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562666+0000","last_change":"2024-09-16T07:31:10.109034+0000","last_active":"2024-09-16T07:31:11.562666+0000","last_peered":"2024-09-16T07:31:11.562666+0000","last_clean":"2024-09-16T07:31:11.562666+0000","last_became_active":"2024-09-16T07:31:10.106720+0000","last_became_peered":"2024-09-16T07:31:10.106720+0000","last_unstale":"2024-09-16T07:31:11.562666+0000","last_undegraded":"2024-09-16T07:31:11.562666+0000","last_fullsized":"2024-09-16T07:31:11.562666+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:58:51.808435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096295+0000","last_change":"2024-09-16T07:31:09.112200+0000","last_active":"2024-09-16T07:31:11.096295+0000","last_peered":"2024-09-16T07:31:11.096295+0000","last_clean":"2024-09-16T07:31:11.096295+0000","last_became_active":"2024-09-16T07:31:09.111193+0000","last_became_peered":"2024-09-16T07:31:09.111193+0000","last_unstale":"2024-09-16T07:31:11.096295+0000","last_undegraded":"2024-09-16T07:31:11.096295+0000","last_fullsized":"2024-09-16T07:31:11.096295+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:27.986940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561466+0000","last_change":"2024-09-16T07:31:10.113361+0000","last_active":"2024-09-16T07:31:11.561466+0000","last_peered":"2024-09-16T07:31:11.561466+0000","last_clean":"2024-09-16T07:31:11.561466+0000","last_became_active":"2024-09-16T07:31:10.113092+0000","last_became_peered":"2024-09-16T07:31:10.113092+0000","last_unstale":"2024-09-16T07:31:11.561466+0000","last_undegraded":"2024-09-16T07:31:11.561466+0000","last_fullsized":"2024-09-16T07:31:11.561466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:17.216432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562011+0000","last_change":"2024-09-16T07:31:09.110861+0000","last_active":"2024-09-16T07:31:11.562011+0000","last_peered":"2024-09-16T07:31:11.562011+0000","last_clean":"2024-09-16T07:31:11.562011+0000","last_became_active":"2024-09-16T07:31:09.109219+0000","last_became_peered":"2024-09-16T07:31:09.109219+0000","last_unstale":"2024-09-16T07:31:11.562011+0000","last_undegraded":"2024-09-16T07:31:11.562011+0000","last_fullsized":"2024-09-16T07:31:11.562011+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:30.052394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562466+0000","last_change":"2024-09-16T07:31:10.117223+0000","last_active":"2024-09-16T07:31:11.562466+0000","last_peered":"2024-09-16T07:31:11.562466+0000","last_clean":"2024-09-16T07:31:11.562466+0000","last_became_active":"2024-09-16T07:31:10.116932+0000","last_became_peered":"2024-09-16T07:31:10.116932+0000","last_unstale":"2024-09-16T07:31:11.562466+0000","last_undegraded":"2024-09-16T07:31:11.562466+0000","last_fullsized":"2024-09-16T07:31:11.562466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:48.847787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562788+0000","last_change":"2024-09-16T07:31:09.115012+0000","last_active":"2024-09-16T07:31:11.562788+0000","last_peered":"2024-09-16T07:31:11.562788+0000","last_clean":"2024-09-16T07:31:11.562788+0000","last_became_active":"2024-09-16T07:31:09.114763+0000","last_became_peered":"2024-09-16T07:31:09.114763+0000","last_unstale":"2024-09-16T07:31:11.562788+0000","last_undegraded":"2024-09-16T07:31:11.562788+0000","last_fullsized":"2024-09-16T07:31:11.562788+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:17.233719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562462+0000","last_change":"2024-09-16T07:31:10.114314+0000","last_active":"2024-09-16T07:31:11.562462+0000","last_peered":"2024-09-16T07:31:11.562462+0000","last_clean":"2024-09-16T07:31:11.562462+0000","last_became_active":"2024-09-16T07:31:10.114045+0000","last_became_peered":"2024-09-16T07:31:10.114045+0000","last_unstale":"2024-09-16T07:31:11.562462+0000","last_undegraded":"2024-09-16T07:31:11.562462+0000","last_fullsized":"2024-09-16T07:31:11.562462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:34.128888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561687+0000","last_change":"2024-09-16T07:31:09.108597+0000","last_active":"2024-09-16T07:31:11.561687+0000","last_peered":"2024-09-16T07:31:11.561687+0000","last_clean":"2024-09-16T07:31:11.561687+0000","last_became_active":"2024-09-16T07:31:09.107264+0000","last_became_peered":"2024-09-16T07:31:09.107264+0000","last_unstale":"2024-09-16T07:31:11.561687+0000","last_undegraded":"2024-09-16T07:31:11.561687+0000","last_fullsized":"2024-09-16T07:31:11.561687+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:18.997978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096138+0000","last_change":"2024-09-16T07:31:10.112497+0000","last_active":"2024-09-16T07:31:11.096138+0000","last_peered":"2024-09-16T07:31:11.096138+0000","last_clean":"2024-09-16T07:31:11.096138+0000","last_became_active":"2024-09-16T07:31:10.111725+0000","last_became_peered":"2024-09-16T07:31:10.111725+0000","last_unstale":"2024-09-16T07:31:11.096138+0000","last_undegraded":"2024-09-16T07:31:11.096138+0000","last_fullsized":"2024-09-16T07:31:11.096138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:14.029753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.095940+0000","last_change":"2024-09-16T07:31:09.108397+0000","last_active":"2024-09-16T07:31:11.095940+0000","last_peered":"2024-09-16T07:31:11.095940+0000","last_clean":"2024-09-16T07:31:11.095940+0000","last_became_active":"2024-09-16T07:31:09.108100+0000","last_became_peered":"2024-09-16T07:31:09.108100+0000","last_unstale":"2024-09-16T07:31:11.095940+0000","last_undegraded":"2024-09-16T07:31:11.095940+0000","last_fullsized":"2024-09-16T07:31:11.095940+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:13.095611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561178+0000","last_change":"2024-09-16T07:31:10.113304+0000","last_active":"2024-09-16T07:31:11.561178+0000","last_peered":"2024-09-16T07:31:11.561178+0000","last_clean":"2024-09-16T07:31:11.561178+0000","last_became_active":"2024-09-16T07:31:10.113069+0000","last_became_peered":"2024-09-16T07:31:10.113069+0000","last_unstale":"2024-09-16T07:31:11.561178+0000","last_undegraded":"2024-09-16T07:31:11.561178+0000","last_fullsized":"2024-09-16T07:31:11.561178+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:40.391661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098196+0000","last_change":"2024-09-16T07:31:09.113806+0000","last_active":"2024-09-16T07:31:11.098196+0000","last_peered":"2024-09-16T07:31:11.098196+0000","last_clean":"2024-09-16T07:31:11.098196+0000","last_became_active":"2024-09-16T07:31:09.113523+0000","last_became_peered":"2024-09-16T07:31:09.113523+0000","last_unstale":"2024-09-16T07:31:11.098196+0000","last_undegraded":"2024-09-16T07:31:11.098196+0000","last_fullsized":"2024-09-16T07:31:11.098196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:33.776337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098087+0000","last_change":"2024-09-16T07:31:10.122375+0000","last_active":"2024-09-16T07:31:11.098087+0000","last_peered":"2024-09-16T07:31:11.098087+0000","last_clean":"2024-09-16T07:31:11.098087+0000","last_became_active":"2024-09-16T07:31:10.121950+0000","last_became_peered":"2024-09-16T07:31:10.121950+0000","last_unstale":"2024-09-16T07:31:11.098087+0000","last_undegraded":"2024-09-16T07:31:11.098087+0000","last_fullsized":"2024-09-16T07:31:11.098087+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:24.427972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097712+0000","last_change":"2024-09-16T07:31:09.111652+0000","last_active":"2024-09-16T07:31:11.097712+0000","last_peered":"2024-09-16T07:31:11.097712+0000","last_clean":"2024-09-16T07:31:11.097712+0000","last_became_active":"2024-09-16T07:31:09.111362+0000","last_became_peered":"2024-09-16T07:31:09.111362+0000","last_unstale":"2024-09-16T07:31:11.097712+0000","last_undegraded":"2024-09-16T07:31:11.097712+0000","last_fullsized":"2024-09-16T07:31:11.097712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:53:30.419144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098963+0000","last_change":"2024-09-16T07:31:10.117395+0000","last_active":"2024-09-16T07:31:11.098963+0000","last_peered":"2024-09-16T07:31:11.098963+0000","last_clean":"2024-09-16T07:31:11.098963+0000","last_became_active":"2024-09-16T07:31:10.116969+0000","last_became_peered":"2024-09-16T07:31:10.116969+0000","last_unstale":"2024-09-16T07:31:11.098963+0000","last_undegraded":"2024-09-16T07:31:11.098963+0000","last_fullsized":"2024-09-16T07:31:11.098963+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:45.163969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562697+0000","last_change":"2024-09-16T07:31:09.108368+0000","last_active":"2024-09-16T07:31:11.562697+0000","last_peered":"2024-09-16T07:31:11.562697+0000","last_clean":"2024-09-16T07:31:11.562697+0000","last_became_active":"2024-09-16T07:31:09.108117+0000","last_became_peered":"2024-09-16T07:31:09.108117+0000","last_unstale":"2024-09-16T07:31:11.562697+0000","last_undegraded":"2024-09-16T07:31:11.562697+0000","last_fullsized":"2024-09-16T07:31:11.562697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:23.959830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562811+0000","last_change":"2024-09-16T07:31:10.117053+0000","last_active":"2024-09-16T07:31:11.562811+0000","last_peered":"2024-09-16T07:31:11.562811+0000","last_clean":"2024-09-16T07:31:11.562811+0000","last_became_active":"2024-09-16T07:31:10.116783+0000","last_became_peered":"2024-09-16T07:31:10.116783+0000","last_unstale":"2024-09-16T07:31:11.562811+0000","last_undegraded":"2024-09-16T07:31:11.562811+0000","last_fullsized":"2024-09-16T07:31:11.562811+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:33.570183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096334+0000","last_change":"2024-09-16T07:31:09.114390+0000","last_active":"2024-09-16T07:31:11.096334+0000","last_peered":"2024-09-16T07:31:11.096334+0000","last_clean":"2024-09-16T07:31:11.096334+0000","last_became_active":"2024-09-16T07:31:09.114096+0000","last_became_peered":"2024-09-16T07:31:09.114096+0000","last_unstale":"2024-09-16T07:31:11.096334+0000","last_undegraded":"2024-09-16T07:31:11.096334+0000","last_fullsized":"2024-09-16T07:31:11.096334+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:31.298135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561549+0000","last_change":"2024-09-16T07:31:10.119123+0000","last_active":"2024-09-16T07:31:11.561549+0000","last_peered":"2024-09-16T07:31:11.561549+0000","last_clean":"2024-09-16T07:31:11.561549+0000","last_became_active":"2024-09-16T07:31:10.118852+0000","last_became_peered":"2024-09-16T07:31:10.118852+0000","last_unstale":"2024-09-16T07:31:11.561549+0000","last_undegraded":"2024-09-16T07:31:11.561549+0000","last_fullsized":"2024-09-16T07:31:11.561549+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:20.000856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562522+0000","last_change":"2024-09-16T07:31:09.112811+0000","last_active":"2024-09-16T07:31:11.562522+0000","last_peered":"2024-09-16T07:31:11.562522+0000","last_clean":"2024-09-16T07:31:11.562522+0000","last_became_active":"2024-09-16T07:31:09.112545+0000","last_became_peered":"2024-09-16T07:31:09.112545+0000","last_unstale":"2024-09-16T07:31:11.562522+0000","last_undegraded":"2024-09-16T07:31:11.562522+0000","last_fullsized":"2024-09-16T07:31:11.562522+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:50:45.563441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562187+0000","last_change":"2024-09-16T07:31:10.114836+0000","last_active":"2024-09-16T07:31:11.562187+0000","last_peered":"2024-09-16T07:31:11.562187+0000","last_clean":"2024-09-16T07:31:11.562187+0000","last_became_active":"2024-09-16T07:31:10.114549+0000","last_became_peered":"2024-09-16T07:31:10.114549+0000","last_unstale":"2024-09-16T07:31:11.562187+0000","last_undegraded":"2024-09-16T07:31:11.562187+0000","last_fullsized":"2024-09-16T07:31:11.562187+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:29.196014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561626+0000","last_change":"2024-09-16T07:31:09.104393+0000","last_active":"2024-09-16T07:31:11.561626+0000","last_peered":"2024-09-16T07:31:11.561626+0000","last_clean":"2024-09-16T07:31:11.561626+0000","last_became_active":"2024-09-16T07:31:09.104105+0000","last_became_peered":"2024-09-16T07:31:09.104105+0000","last_unstale":"2024-09-16T07:31:11.561626+0000","last_undegraded":"2024-09-16T07:31:11.561626+0000","last_fullsized":"2024-09-16T07:31:11.561626+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:13.095611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562558+0000","last_change":"2024-09-16T07:31:10.117870+0000","last_active":"2024-09-16T07:31:11.562558+0000","last_peered":"2024-09-16T07:31:11.562558+0000","last_clean":"2024-09-16T07:31:11.562558+0000","last_became_active":"2024-09-16T07:31:10.116200+0000","last_became_peered":"2024-09-16T07:31:10.116200+0000","last_unstale":"2024-09-16T07:31:11.562558+0000","last_undegraded":"2024-09-16T07:31:11.562558+0000","last_fullsized":"2024-09-16T07:31:11.562558+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:27:17.774632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562580+0000","last_change":"2024-09-16T07:31:09.103293+0000","last_active":"2024-09-16T07:31:11.562580+0000","last_peered":"2024-09-16T07:31:11.562580+0000","last_clean":"2024-09-16T07:31:11.562580+0000","last_became_active":"2024-09-16T07:31:09.102732+0000","last_became_peered":"2024-09-16T07:31:09.102732+0000","last_unstale":"2024-09-16T07:31:11.562580+0000","last_undegraded":"2024-09-16T07:31:11.562580+0000","last_fullsized":"2024-09-16T07:31:11.562580+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:42.795057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561984+0000","last_change":"2024-09-16T07:31:10.113551+0000","last_active":"2024-09-16T07:31:11.561984+0000","last_peered":"2024-09-16T07:31:11.561984+0000","last_clean":"2024-09-16T07:31:11.561984+0000","last_became_active":"2024-09-16T07:31:10.113000+0000","last_became_peered":"2024-09-16T07:31:10.113000+0000","last_unstale":"2024-09-16T07:31:11.561984+0000","last_undegraded":"2024-09-16T07:31:11.561984+0000","last_fullsized":"2024-09-16T07:31:11.561984+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:43.198429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097457+0000","last_change":"2024-09-16T07:31:09.098750+0000","last_active":"2024-09-16T07:31:11.097457+0000","last_peered":"2024-09-16T07:31:11.097457+0000","last_clean":"2024-09-16T07:31:11.097457+0000","last_became_active":"2024-09-16T07:31:09.098434+0000","last_became_peered":"2024-09-16T07:31:09.098434+0000","last_unstale":"2024-09-16T07:31:11.097457+0000","last_undegraded":"2024-09-16T07:31:11.097457+0000","last_fullsized":"2024-09-16T07:31:11.097457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:15:30.043800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097606+0000","last_change":"2024-09-16T07:31:10.116680+0000","last_active":"2024-09-16T07:31:11.097606+0000","last_peered":"2024-09-16T07:31:11.097606+0000","last_clean":"2024-09-16T07:31:11.097606+0000","last_became_active":"2024-09-16T07:31:10.115045+0000","last_became_peered":"2024-09-16T07:31:10.115045+0000","last_unstale":"2024-09-16T07:31:11.097606+0000","last_undegraded":"2024-09-16T07:31:11.097606+0000","last_fullsized":"2024-09-16T07:31:11.097606+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:55.688012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.095776+0000","last_change":"2024-09-16T07:31:09.110936+0000","last_active":"2024-09-16T07:31:11.095776+0000","last_peered":"2024-09-16T07:31:11.095776+0000","last_clean":"2024-09-16T07:31:11.095776+0000","last_became_active":"2024-09-16T07:31:09.110669+0000","last_became_peered":"2024-09-16T07:31:09.110669+0000","last_unstale":"2024-09-16T07:31:11.095776+0000","last_undegraded":"2024-09-16T07:31:11.095776+0000","last_fullsized":"2024-09-16T07:31:11.095776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:02.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561619+0000","last_change":"2024-09-16T07:31:10.123218+0000","last_active":"2024-09-16T07:31:11.561619+0000","last_peered":"2024-09-16T07:31:11.561619+0000","last_clean":"2024-09-16T07:31:11.561619+0000","last_became_active":"2024-09-16T07:31:10.119656+0000","last_became_peered":"2024-09-16T07:31:10.119656+0000","last_unstale":"2024-09-16T07:31:11.561619+0000","last_undegraded":"2024-09-16T07:31:11.561619+0000","last_fullsized":"2024-09-16T07:31:11.561619+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:26.111982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562040+0000","last_change":"2024-09-16T07:31:09.115163+0000","last_active":"2024-09-16T07:31:11.562040+0000","last_peered":"2024-09-16T07:31:11.562040+0000","last_clean":"2024-09-16T07:31:11.562040+0000","last_became_active":"2024-09-16T07:31:09.114887+0000","last_became_peered":"2024-09-16T07:31:09.114887+0000","last_unstale":"2024-09-16T07:31:11.562040+0000","last_undegraded":"2024-09-16T07:31:11.562040+0000","last_fullsized":"2024-09-16T07:31:11.562040+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:33.776337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562992+0000","last_change":"2024-09-16T07:31:10.111177+0000","last_active":"2024-09-16T07:31:11.562992+0000","last_peered":"2024-09-16T07:31:11.562992+0000","last_clean":"2024-09-16T07:31:11.562992+0000","last_became_active":"2024-09-16T07:31:10.110739+0000","last_became_peered":"2024-09-16T07:31:10.110739+0000","last_unstale":"2024-09-16T07:31:11.562992+0000","last_undegraded":"2024-09-16T07:31:11.562992+0000","last_fullsized":"2024-09-16T07:31:11.562992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:33.247532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562322+0000","last_change":"2024-09-16T07:31:09.116759+0000","last_active":"2024-09-16T07:31:11.562322+0000","last_peered":"2024-09-16T07:31:11.562322+0000","last_clean":"2024-09-16T07:31:11.562322+0000","last_became_active":"2024-09-16T07:31:09.116518+0000","last_became_peered":"2024-09-16T07:31:09.116518+0000","last_unstale":"2024-09-16T07:31:11.562322+0000","last_undegraded":"2024-09-16T07:31:11.562322+0000","last_fullsized":"2024-09-16T07:31:11.562322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:10.636529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096617+0000","last_change":"2024-09-16T07:31:10.118389+0000","last_active":"2024-09-16T07:31:11.096617+0000","last_peered":"2024-09-16T07:31:11.096617+0000","last_clean":"2024-09-16T07:31:11.096617+0000","last_became_active":"2024-09-16T07:31:10.118139+0000","last_became_peered":"2024-09-16T07:31:10.118139+0000","last_unstale":"2024-09-16T07:31:11.096617+0000","last_undegraded":"2024-09-16T07:31:11.096617+0000","last_fullsized":"2024-09-16T07:31:11.096617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:15.709142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098799+0000","last_change":"2024-09-16T07:31:09.117294+0000","last_active":"2024-09-16T07:31:11.098799+0000","last_peered":"2024-09-16T07:31:11.098799+0000","last_clean":"2024-09-16T07:31:11.098799+0000","last_became_active":"2024-09-16T07:31:09.117074+0000","last_became_peered":"2024-09-16T07:31:09.117074+0000","last_unstale":"2024-09-16T07:31:11.098799+0000","last_undegraded":"2024-09-16T07:31:11.098799+0000","last_fullsized":"2024-09-16T07:31:11.098799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:03.155631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562570+0000","last_change":"2024-09-16T07:31:10.114083+0000","last_active":"2024-09-16T07:31:11.562570+0000","last_peered":"2024-09-16T07:31:11.562570+0000","last_clean":"2024-09-16T07:31:11.562570+0000","last_became_active":"2024-09-16T07:31:10.113842+0000","last_became_peered":"2024-09-16T07:31:10.113842+0000","last_unstale":"2024-09-16T07:31:11.562570+0000","last_undegraded":"2024-09-16T07:31:11.562570+0000","last_fullsized":"2024-09-16T07:31:11.562570+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:19.849181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098303+0000","last_change":"2024-09-16T07:31:09.108194+0000","last_active":"2024-09-16T07:31:11.098303+0000","last_peered":"2024-09-16T07:31:11.098303+0000","last_clean":"2024-09-16T07:31:11.098303+0000","last_became_active":"2024-09-16T07:31:09.107901+0000","last_became_peered":"2024-09-16T07:31:09.107901+0000","last_unstale":"2024-09-16T07:31:11.098303+0000","last_undegraded":"2024-09-16T07:31:11.098303+0000","last_fullsized":"2024-09-16T07:31:11.098303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:05.119300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097728+0000","last_change":"2024-09-16T07:31:10.112501+0000","last_active":"2024-09-16T07:31:11.097728+0000","last_peered":"2024-09-16T07:31:11.097728+0000","last_clean":"2024-09-16T07:31:11.097728+0000","last_became_active":"2024-09-16T07:31:10.112301+0000","last_became_peered":"2024-09-16T07:31:10.112301+0000","last_unstale":"2024-09-16T07:31:11.097728+0000","last_undegraded":"2024-09-16T07:31:11.097728+0000","last_fullsized":"2024-09-16T07:31:11.097728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:36:26.532895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097613+0000","last_change":"2024-09-16T07:31:09.113003+0000","last_active":"2024-09-16T07:31:11.097613+0000","last_peered":"2024-09-16T07:31:11.097613+0000","last_clean":"2024-09-16T07:31:11.097613+0000","last_became_active":"2024-09-16T07:31:09.112733+0000","last_became_peered":"2024-09-16T07:31:09.112733+0000","last_unstale":"2024-09-16T07:31:11.097613+0000","last_undegraded":"2024-09-16T07:31:11.097613+0000","last_fullsized":"2024-09-16T07:31:11.097613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:43.750421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561429+0000","last_change":"2024-09-16T07:31:10.111042+0000","last_active":"2024-09-16T07:31:11.561429+0000","last_peered":"2024-09-16T07:31:11.561429+0000","last_clean":"2024-09-16T07:31:11.561429+0000","last_became_active":"2024-09-16T07:31:10.110780+0000","last_became_peered":"2024-09-16T07:31:10.110780+0000","last_unstale":"2024-09-16T07:31:11.561429+0000","last_undegraded":"2024-09-16T07:31:11.561429+0000","last_fullsized":"2024-09-16T07:31:11.561429+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:45.975248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097554+0000","last_change":"2024-09-16T07:31:09.106277+0000","last_active":"2024-09-16T07:31:11.097554+0000","last_peered":"2024-09-16T07:31:11.097554+0000","last_clean":"2024-09-16T07:31:11.097554+0000","last_became_active":"2024-09-16T07:31:09.105995+0000","last_became_peered":"2024-09-16T07:31:09.105995+0000","last_unstale":"2024-09-16T07:31:11.097554+0000","last_undegraded":"2024-09-16T07:31:11.097554+0000","last_fullsized":"2024-09-16T07:31:11.097554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:17.850572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561773+0000","last_change":"2024-09-16T07:31:10.105835+0000","last_active":"2024-09-16T07:31:11.561773+0000","last_peered":"2024-09-16T07:31:11.561773+0000","last_clean":"2024-09-16T07:31:11.561773+0000","last_became_active":"2024-09-16T07:31:10.105521+0000","last_became_peered":"2024-09-16T07:31:10.105521+0000","last_unstale":"2024-09-16T07:31:11.561773+0000","last_undegraded":"2024-09-16T07:31:11.561773+0000","last_fullsized":"2024-09-16T07:31:11.561773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:34.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561974+0000","last_change":"2024-09-16T07:31:09.110007+0000","last_active":"2024-09-16T07:31:11.561974+0000","last_peered":"2024-09-16T07:31:11.561974+0000","last_clean":"2024-09-16T07:31:11.561974+0000","last_became_active":"2024-09-16T07:31:09.109703+0000","last_became_peered":"2024-09-16T07:31:09.109703+0000","last_unstale":"2024-09-16T07:31:11.561974+0000","last_undegraded":"2024-09-16T07:31:11.561974+0000","last_fullsized":"2024-09-16T07:31:11.561974+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:27.551579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561425+0000","last_change":"2024-09-16T07:31:10.114400+0000","last_active":"2024-09-16T07:31:11.561425+0000","last_peered":"2024-09-16T07:31:11.561425+0000","last_clean":"2024-09-16T07:31:11.561425+0000","last_became_active":"2024-09-16T07:31:10.112966+0000","last_became_peered":"2024-09-16T07:31:10.112966+0000","last_unstale":"2024-09-16T07:31:11.561425+0000","last_undegraded":"2024-09-16T07:31:11.561425+0000","last_fullsized":"2024-09-16T07:31:11.561425+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:00:39.418017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.097797+0000","last_change":"2024-09-16T07:31:09.113204+0000","last_active":"2024-09-16T07:31:11.097797+0000","last_peered":"2024-09-16T07:31:11.097797+0000","last_clean":"2024-09-16T07:31:11.097797+0000","last_became_active":"2024-09-16T07:31:09.112870+0000","last_became_peered":"2024-09-16T07:31:09.112870+0000","last_unstale":"2024-09-16T07:31:11.097797+0000","last_undegraded":"2024-09-16T07:31:11.097797+0000","last_fullsized":"2024-09-16T07:31:11.097797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:18.973335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561987+0000","last_change":"2024-09-16T07:31:10.109791+0000","last_active":"2024-09-16T07:31:11.561987+0000","last_peered":"2024-09-16T07:31:11.561987+0000","last_clean":"2024-09-16T07:31:11.561987+0000","last_became_active":"2024-09-16T07:31:10.107931+0000","last_became_peered":"2024-09-16T07:31:10.107931+0000","last_unstale":"2024-09-16T07:31:11.561987+0000","last_undegraded":"2024-09-16T07:31:11.561987+0000","last_fullsized":"2024-09-16T07:31:11.561987+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:29.177930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.099057+0000","last_change":"2024-09-16T07:31:09.114389+0000","last_active":"2024-09-16T07:31:11.099057+0000","last_peered":"2024-09-16T07:31:11.099057+0000","last_clean":"2024-09-16T07:31:11.099057+0000","last_became_active":"2024-09-16T07:31:09.114099+0000","last_became_peered":"2024-09-16T07:31:09.114099+0000","last_unstale":"2024-09-16T07:31:11.099057+0000","last_undegraded":"2024-09-16T07:31:11.099057+0000","last_fullsized":"2024-09-16T07:31:11.099057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:27.551579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562223+0000","last_change":"2024-09-16T07:31:10.115496+0000","last_active":"2024-09-16T07:31:11.562223+0000","last_peered":"2024-09-16T07:31:11.562223+0000","last_clean":"2024-09-16T07:31:11.562223+0000","last_became_active":"2024-09-16T07:31:10.114907+0000","last_became_peered":"2024-09-16T07:31:10.114907+0000","last_unstale":"2024-09-16T07:31:11.562223+0000","last_undegraded":"2024-09-16T07:31:11.562223+0000","last_fullsized":"2024-09-16T07:31:11.562223+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:05.220733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:12.200956+0000","last_change":"2024-09-16T07:31:09.110100+0000","last_active":"2024-09-16T07:31:12.200956+0000","last_peered":"2024-09-16T07:31:12.200956+0000","last_clean":"2024-09-16T07:31:12.200956+0000","last_became_active":"2024-09-16T07:31:09.109723+0000","last_became_peered":"2024-09-16T07:31:09.109723+0000","last_unstale":"2024-09-16T07:31:12.200956+0000","last_undegraded":"2024-09-16T07:31:12.200956+0000","last_fullsized":"2024-09-16T07:31:12.200956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:49.631013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562028+0000","last_change":"2024-09-16T07:31:10.109497+0000","last_active":"2024-09-16T07:31:11.562028+0000","last_peered":"2024-09-16T07:31:11.562028+0000","last_clean":"2024-09-16T07:31:11.562028+0000","last_became_active":"2024-09-16T07:31:10.108929+0000","last_became_peered":"2024-09-16T07:31:10.108929+0000","last_unstale":"2024-09-16T07:31:11.562028+0000","last_undegraded":"2024-09-16T07:31:11.562028+0000","last_fullsized":"2024-09-16T07:31:11.562028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:13.831347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562622+0000","last_change":"2024-09-16T07:31:09.111472+0000","last_active":"2024-09-16T07:31:11.562622+0000","last_peered":"2024-09-16T07:31:11.562622+0000","last_clean":"2024-09-16T07:31:11.562622+0000","last_became_active":"2024-09-16T07:31:09.111166+0000","last_became_peered":"2024-09-16T07:31:09.111166+0000","last_unstale":"2024-09-16T07:31:11.562622+0000","last_undegraded":"2024-09-16T07:31:11.562622+0000","last_fullsized":"2024-09-16T07:31:11.562622+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:47:02.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096507+0000","last_change":"2024-09-16T07:31:10.108477+0000","last_active":"2024-09-16T07:31:11.096507+0000","last_peered":"2024-09-16T07:31:11.096507+0000","last_clean":"2024-09-16T07:31:11.096507+0000","last_became_active":"2024-09-16T07:31:10.107884+0000","last_became_peered":"2024-09-16T07:31:10.107884+0000","last_unstale":"2024-09-16T07:31:11.096507+0000","last_undegraded":"2024-09-16T07:31:11.096507+0000","last_fullsized":"2024-09-16T07:31:11.096507+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:28.104996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096212+0000","last_change":"2024-09-16T07:31:09.109361+0000","last_active":"2024-09-16T07:31:11.096212+0000","last_peered":"2024-09-16T07:31:11.096212+0000","last_clean":"2024-09-16T07:31:11.096212+0000","last_became_active":"2024-09-16T07:31:09.109073+0000","last_became_peered":"2024-09-16T07:31:09.109073+0000","last_unstale":"2024-09-16T07:31:11.096212+0000","last_undegraded":"2024-09-16T07:31:11.096212+0000","last_fullsized":"2024-09-16T07:31:11.096212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:01.005024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562118+0000","last_change":"2024-09-16T07:31:10.107943+0000","last_active":"2024-09-16T07:31:11.562118+0000","last_peered":"2024-09-16T07:31:11.562118+0000","last_clean":"2024-09-16T07:31:11.562118+0000","last_became_active":"2024-09-16T07:31:10.107334+0000","last_became_peered":"2024-09-16T07:31:10.107334+0000","last_unstale":"2024-09-16T07:31:11.562118+0000","last_undegraded":"2024-09-16T07:31:11.562118+0000","last_fullsized":"2024-09-16T07:31:11.562118+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:55.688012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.098141+0000","last_change":"2024-09-16T07:31:09.106829+0000","last_active":"2024-09-16T07:31:11.098141+0000","last_peered":"2024-09-16T07:31:11.098141+0000","last_clean":"2024-09-16T07:31:11.098141+0000","last_became_active":"2024-09-16T07:31:09.106064+0000","last_became_peered":"2024-09-16T07:31:09.106064+0000","last_unstale":"2024-09-16T07:31:11.098141+0000","last_undegraded":"2024-09-16T07:31:11.098141+0000","last_fullsized":"2024-09-16T07:31:11.098141+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:23.786094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562065+0000","last_change":"2024-09-16T07:31:10.118302+0000","last_active":"2024-09-16T07:31:11.562065+0000","last_peered":"2024-09-16T07:31:11.562065+0000","last_clean":"2024-09-16T07:31:11.562065+0000","last_became_active":"2024-09-16T07:31:10.117463+0000","last_became_peered":"2024-09-16T07:31:10.117463+0000","last_unstale":"2024-09-16T07:31:11.562065+0000","last_undegraded":"2024-09-16T07:31:11.562065+0000","last_fullsized":"2024-09-16T07:31:11.562065+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:53:14.098489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.096035+0000","last_change":"2024-09-16T07:31:09.107674+0000","last_active":"2024-09-16T07:31:11.096035+0000","last_peered":"2024-09-16T07:31:11.096035+0000","last_clean":"2024-09-16T07:31:11.096035+0000","last_became_active":"2024-09-16T07:31:09.107310+0000","last_became_peered":"2024-09-16T07:31:09.107310+0000","last_unstale":"2024-09-16T07:31:11.096035+0000","last_undegraded":"2024-09-16T07:31:11.096035+0000","last_fullsized":"2024-09-16T07:31:11.096035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:28:45.359182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561365+0000","last_change":"2024-09-16T07:31:10.107183+0000","last_active":"2024-09-16T07:31:11.561365+0000","last_peered":"2024-09-16T07:31:11.561365+0000","last_clean":"2024-09-16T07:31:11.561365+0000","last_became_active":"2024-09-16T07:31:10.106918+0000","last_became_peered":"2024-09-16T07:31:10.106918+0000","last_unstale":"2024-09-16T07:31:11.561365+0000","last_undegraded":"2024-09-16T07:31:11.561365+0000","last_fullsized":"2024-09-16T07:31:11.561365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:17.468806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561941+0000","last_change":"2024-09-16T07:31:09.111346+0000","last_active":"2024-09-16T07:31:11.561941+0000","last_peered":"2024-09-16T07:31:11.561941+0000","last_clean":"2024-09-16T07:31:11.561941+0000","last_became_active":"2024-09-16T07:31:09.111058+0000","last_became_peered":"2024-09-16T07:31:09.111058+0000","last_unstale":"2024-09-16T07:31:11.561941+0000","last_undegraded":"2024-09-16T07:31:11.561941+0000","last_fullsized":"2024-09-16T07:31:11.561941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:32.040048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561684+0000","last_change":"2024-09-16T07:31:10.112150+0000","last_active":"2024-09-16T07:31:11.561684+0000","last_peered":"2024-09-16T07:31:11.561684+0000","last_clean":"2024-09-16T07:31:11.561684+0000","last_became_active":"2024-09-16T07:31:10.110175+0000","last_became_peered":"2024-09-16T07:31:10.110175+0000","last_unstale":"2024-09-16T07:31:11.561684+0000","last_undegraded":"2024-09-16T07:31:11.561684+0000","last_fullsized":"2024-09-16T07:31:11.561684+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:11.273878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561323+0000","last_change":"2024-09-16T07:31:09.104477+0000","last_active":"2024-09-16T07:31:11.561323+0000","last_peered":"2024-09-16T07:31:11.561323+0000","last_clean":"2024-09-16T07:31:11.561323+0000","last_became_active":"2024-09-16T07:31:09.104186+0000","last_became_peered":"2024-09-16T07:31:09.104186+0000","last_unstale":"2024-09-16T07:31:11.561323+0000","last_undegraded":"2024-09-16T07:31:11.561323+0000","last_fullsized":"2024-09-16T07:31:11.561323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:25.109105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.562370+0000","last_change":"2024-09-16T07:31:10.118923+0000","last_active":"2024-09-16T07:31:11.562370+0000","last_peered":"2024-09-16T07:31:11.562370+0000","last_clean":"2024-09-16T07:31:11.562370+0000","last_became_active":"2024-09-16T07:31:10.118384+0000","last_became_peered":"2024-09-16T07:31:10.118384+0000","last_unstale":"2024-09-16T07:31:11.562370+0000","last_undegraded":"2024-09-16T07:31:11.562370+0000","last_fullsized":"2024-09-16T07:31:11.562370+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:09.082762+0000","last_clean_scrub_stamp":"2024-09-16T07:31:09.082762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:32.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561836+0000","last_change":"2024-09-16T07:31:09.115329+0000","last_active":"2024-09-16T07:31:11.561836+0000","last_peered":"2024-09-16T07:31:11.561836+0000","last_clean":"2024-09-16T07:31:11.561836+0000","last_became_active":"2024-09-16T07:31:09.115068+0000","last_became_peered":"2024-09-16T07:31:09.115068+0000","last_unstale":"2024-09-16T07:31:11.561836+0000","last_undegraded":"2024-09-16T07:31:11.561836+0000","last_fullsized":"2024-09-16T07:31:11.561836+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:31:08.079884+0000","last_clean_scrub_stamp":"2024-09-16T07:31:08.079884+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:14:03.692603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":241,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T07:31:11.561424+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:31:11.561424+0000","last_peered":"2024-09-16T07:31:11.561424+0000","last_clean":"2024-09-16T07:31:11.561424+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:31:11.561424+0000","last_undegraded":"2024-09-16T07:31:11.561424+0000","last_fullsized":"2024-09-16T07:31:11.561424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476844,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58664,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313176,"statfs":{"total":96636764160,"available":96576692224,"internally_reserved":0,"allocated":1130496,"data_stored":484614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476844,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58652,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313188,"statfs":{"total":96636764160,"available":96576704512,"internally_reserved":0,"allocated":1118208,"data_stored":483940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476844,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59232,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312608,"statfs":{"total":96636764160,"available":96576110592,"internally_reserved":0,"allocated":1716224,"data_stored":1074398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476843,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58668,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313172,"statfs":{"total":96636764160,"available":96576688128,"internally_reserved":0,"allocated":1134592,"data_stored":485574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58660,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313180,"statfs":{"total":96636764160,"available":96576696320,"internally_reserved":0,"allocated":1122304,"data_stored":484010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509547,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64160,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307680,"statfs":{"total":96636764160,"available":96571064320,"internally_reserved":0,"allocated":1126400,"data_stored":484612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46772,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325068,"statfs":{"total":96636764160,"available":96588869632,"internally_reserved":0,"allocated":1728512,"data_stored":1075954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64148,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307692,"statfs":{"total":96636764160,"available":96571076608,"internally_reserved":0,"allocated":1110016,"data_stored":477646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:31:14.816 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:31:14.816 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:31:15.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.018+0000 7f114aa00640 1 -- 172.21.15.71:0/2521456904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1128004080 msgr2=0x7f1128004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.018+0000 7f114aa00640 1 --2- 172.21.15.71:0/2521456904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1128004080 0x7f1128004800 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f112c00bbe0 tx=0x7f1128005830 comp rx=0 tx=0).stop 2024-09-16T07:31:15.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.018+0000 7f1148a00640 1 -- 172.21.15.71:0/2521456904 reap_dead start 2024-09-16T07:31:15.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.018+0000 7f114aa00640 1 -- 172.21.15.71:0/2521456904 shutdown_connections 2024-09-16T07:31:15.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.018+0000 7f114aa00640 1 -- 172.21.15.71:0/2521456904 >> 172.21.15.71:0/2521456904 conn(0x7f11440b1840 msgr2=0x7f11440b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:15.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.018+0000 7f114aa00640 1 -- 172.21.15.71:0/2521456904 shutdown_connections 2024-09-16T07:31:15.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.018+0000 7f114aa00640 1 -- 172.21.15.71:0/2521456904 wait complete. 2024-09-16T07:31:15.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.019+0000 7f114aa00640 1 Processor -- start 2024-09-16T07:31:15.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.020+0000 7f114aa00640 1 -- start start 2024-09-16T07:31:15.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.020+0000 7f114aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1128004080 0x7f1144133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.020+0000 7f114aa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11441e7c20 0x7f11441e8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.020+0000 7f114aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11441f7e30 0x7f11441f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.020+0000 7f114aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11440cdf40 con 0x7f11441f7e30 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.020+0000 7f114aa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11440cddc0 con 0x7f1128004080 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.020+0000 7f114aa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11440ce0c0 con 0x7f11441e7c20 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11441f7e30 0x7f11441f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11441f7e30 0x7f11441f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47696/0 (socket says 172.21.15.71:47696) 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 -- 172.21.15.71:0/1748436167 learned_addr learned my addr 172.21.15.71:0/1748436167 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f113be00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11441e7c20 0x7f11441e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1143e00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1128004080 0x7f1144133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 -- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11441e7c20 msgr2=0x7f11441e8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11441e7c20 0x7f11441e8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 -- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1128004080 msgr2=0x7f1144133ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1128004080 0x7f1144133ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 -- 172.21.15.71:0/1748436167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11441e8670 con 0x7f11441f7e30 2024-09-16T07:31:15.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.021+0000 7f1148a00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11441f7e30 0x7f11441f82b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f113c005fd0 tx=0x7f113c010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:15.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.022+0000 7f1140c00640 1 -- 172.21.15.71:0/1748436167 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c003d80 con 0x7f11441f7e30 2024-09-16T07:31:15.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.022+0000 7f1140c00640 1 -- 172.21.15.71:0/1748436167 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f113c004b20 con 0x7f11441f7e30 2024-09-16T07:31:15.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.022+0000 7f1140c00640 1 -- 172.21.15.71:0/1748436167 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c00e970 con 0x7f11441f7e30 2024-09-16T07:31:15.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.022+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11441e8950 con 0x7f11441f7e30 2024-09-16T07:31:15.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.022+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11441e8f10 con 0x7f11441f7e30 2024-09-16T07:31:15.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.023+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1144071150 con 0x7f11441f7e30 2024-09-16T07:31:15.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.024+0000 7f1140c00640 1 -- 172.21.15.71:0/1748436167 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f113c00eb10 con 0x7f11441f7e30 2024-09-16T07:31:15.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.025+0000 7f1140c00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1120079960 0x7f112007be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:15.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.025+0000 7f1140c00640 1 -- 172.21.15.71:0/1748436167 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f113c099db0 con 0x7f11441f7e30 2024-09-16T07:31:15.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.026+0000 7f1143e00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1120079960 0x7f112007be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:15.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.026+0000 7f1143e00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1120079960 0x7f112007be20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1134001860 tx=0x7f1134006040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:15.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.029+0000 7f1140c00640 1 -- 172.21.15.71:0/1748436167 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1144071150 con 0x7f11441f7e30 2024-09-16T07:31:15.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.284+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1144073440 con 0x7f11441f7e30 2024-09-16T07:31:15.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.287+0000 7f1140c00640 1 -- 172.21.15.71:0/1748436167 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f113c016030 con 0x7f11441f7e30 2024-09-16T07:31:15.286 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:31:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:31:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:31:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:31:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:31:15.287 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:31:15.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1120079960 msgr2=0x7f112007be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1120079960 0x7f112007be20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1134001860 tx=0x7f1134006040 comp rx=0 tx=0).stop 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11441f7e30 msgr2=0x7f11441f82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11441f7e30 0x7f11441f82b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f113c005fd0 tx=0x7f113c010ee0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 shutdown_connections 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1120079960 0x7f112007be20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11441f7e30 0x7f11441f82b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11441e7c20 0x7f11441e8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 --2- 172.21.15.71:0/1748436167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1128004080 0x7f1144133ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 >> 172.21.15.71:0/1748436167 conn(0x7f11440b1840 msgr2=0x7f11440b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 shutdown_connections 2024-09-16T07:31:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.290+0000 7f114aa00640 1 -- 172.21.15.71:0/1748436167 wait complete. 2024-09-16T07:31:15.301 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:31:15.306 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:31:15.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:31:15.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.509+0000 7f6dbe600640 1 -- 172.21.15.71:0/2247524558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db80af3d0 msgr2=0x7f6db806f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.509+0000 7f6dbe600640 1 --2- 172.21.15.71:0/2247524558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db80af3d0 0x7f6db806f660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009860 tx=0x7f6da402a4a0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.509+0000 7f6dbe600640 1 -- 172.21.15.71:0/2247524558 shutdown_connections 2024-09-16T07:31:15.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.509+0000 7f6dbe600640 1 --2- 172.21.15.71:0/2247524558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db80af3d0 0x7f6db806f660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.509+0000 7f6dbe600640 1 -- 172.21.15.71:0/2247524558 >> 172.21.15.71:0/2247524558 conn(0x7f6db80b1950 msgr2=0x7f6db80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:15.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.509+0000 7f6dbe600640 1 -- 172.21.15.71:0/2247524558 shutdown_connections 2024-09-16T07:31:15.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.509+0000 7f6dbe600640 1 -- 172.21.15.71:0/2247524558 wait complete. 2024-09-16T07:31:15.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.510+0000 7f6dbe600640 1 Processor -- start 2024-09-16T07:31:15.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.510+0000 7f6dbe600640 1 -- start start 2024-09-16T07:31:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.511+0000 7f6dbe600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6db8072df0 0x7f6db80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.511+0000 7f6dbe600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6db80af3d0 0x7f6db814c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.511+0000 7f6dbe600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db8161550 0x7f6db80cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.511+0000 7f6dbe600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6db80739b0 con 0x7f6db80af3d0 2024-09-16T07:31:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.511+0000 7f6dbe600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6db806fdb0 con 0x7f6db8072df0 2024-09-16T07:31:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.511+0000 7f6dbe600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6db8073b30 con 0x7f6db8161550 2024-09-16T07:31:15.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6db80af3d0 0x7f6db814c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:15.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6db80af3d0 0x7f6db814c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47700/0 (socket says 172.21.15.71:47700) 2024-09-16T07:31:15.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 -- 172.21.15.71:0/2695538043 learned_addr learned my addr 172.21.15.71:0/2695538043 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db7e00640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db8161550 0x7f6db80cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 -- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db8161550 msgr2=0x7f6db80cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db8161550 0x7f6db80cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 -- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6db8072df0 msgr2=0x7f6db80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6db8072df0 0x7f6db80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 -- 172.21.15.71:0/2695538043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db80cec00 con 0x7f6db80af3d0 2024-09-16T07:31:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db7e00640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db8161550 0x7f6db80cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:31:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6db6a00640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6db80af3d0 0x7f6db814c7e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009980 tx=0x7f6da4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.512+0000 7f6daf400640 1 -- 172.21.15.71:0/2695538043 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4005c90 con 0x7f6db80af3d0 2024-09-16T07:31:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.513+0000 7f6daf400640 1 -- 172.21.15.71:0/2695538043 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6da4005020 con 0x7f6db80af3d0 2024-09-16T07:31:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.513+0000 7f6daf400640 1 -- 172.21.15.71:0/2695538043 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4005410 con 0x7f6db80af3d0 2024-09-16T07:31:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.513+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db80ceee0 con 0x7f6db80af3d0 2024-09-16T07:31:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.513+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db80cf4a0 con 0x7f6db80af3d0 2024-09-16T07:31:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.514+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db8073f60 con 0x7f6db80af3d0 2024-09-16T07:31:15.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.515+0000 7f6daf400640 1 -- 172.21.15.71:0/2695538043 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f6da4039040 con 0x7f6db80af3d0 2024-09-16T07:31:15.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.516+0000 7f6daf400640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d98079930 0x7f6d9807bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:15.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.516+0000 7f6daf400640 1 -- 172.21.15.71:0/2695538043 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f6da40ba490 con 0x7f6db80af3d0 2024-09-16T07:31:15.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.516+0000 7f6db7400640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d98079930 0x7f6d9807bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:15.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.517+0000 7f6db7400640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d98079930 0x7f6d9807bdf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6da80014e0 tx=0x7f6da80078a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:15.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.520+0000 7f6daf400640 1 -- 172.21.15.71:0/2695538043 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6da4040030 con 0x7f6db80af3d0 2024-09-16T07:31:15.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.738+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6db8157de0 con 0x7f6db80af3d0 2024-09-16T07:31:15.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.739+0000 7f6daf400640 1 -- 172.21.15.71:0/2695538043 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7f6da4084550 con 0x7f6db80af3d0 2024-09-16T07:31:15.738 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:15.738 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":56,"btime":"2024-09-16T07:31:13:204547+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1642949519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1642949519},{"type":"v1","addr":"172.21.15.177:6836","nonce":1642949519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/2594338176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2594338176},{"type":"v1","addr":"172.21.15.71:6834","nonce":2594338176}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/482130020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":482130020},{"type":"v1","addr":"172.21.15.71:6835","nonce":482130020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:31:09.520327+0000","modified":"2024-09-16T07:31:12.617058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/4183855149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":4183855149},{"type":"v1","addr":"172.21.15.177:6837","nonce":4183855149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":6}]} 2024-09-16T07:31:15.739 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 56 2024-09-16T07:31:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d98079930 msgr2=0x7f6d9807bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d98079930 0x7f6d9807bdf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6da80014e0 tx=0x7f6da80078a0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6db80af3d0 msgr2=0x7f6db814c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6db80af3d0 0x7f6db814c7e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009980 tx=0x7f6da4003c10 comp rx=0 tx=0).stop 2024-09-16T07:31:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 shutdown_connections 2024-09-16T07:31:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6d98079930 0x7f6d9807bdf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6db8161550 0x7f6db80cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6db80af3d0 0x7f6db814c7e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 --2- 172.21.15.71:0/2695538043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6db8072df0 0x7f6db80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:15.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 >> 172.21.15.71:0/2695538043 conn(0x7f6db80b1950 msgr2=0x7f6db806ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:15.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 shutdown_connections 2024-09-16T07:31:15.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:15.741+0000 7f6dbe600640 1 -- 172.21.15.71:0/2695538043 wait complete. 2024-09-16T07:31:16.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:31:09.520327+0000', 'modified': '2024-09-16T07:31:12.617058+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7008}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7008': {'gid': 7008, 'name': 'd', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6837/4183855149', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6835', 'nonce': 4183855149}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 4183855149}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7008, 'qdb_cluster': [7008]} 2024-09-16T07:31:16.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:31:16.161 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-16T07:31:16.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.321+0000 7fccff200640 1 -- 172.21.15.71:0/1670075246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd000725d0 msgr2=0x7fcd00073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.321+0000 7fccff200640 1 --2- 172.21.15.71:0/1670075246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd000725d0 0x7fcd00073d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fccec009830 tx=0x7fccec02a520 comp rx=0 tx=0).stop 2024-09-16T07:31:16.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.321+0000 7fccff200640 1 -- 172.21.15.71:0/1670075246 shutdown_connections 2024-09-16T07:31:16.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.321+0000 7fccff200640 1 --2- 172.21.15.71:0/1670075246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd000725d0 0x7fcd00073d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.321+0000 7fccff200640 1 --2- 172.21.15.71:0/1670075246 >> v2:172.21.15.177:3300/0 conn(0x7fcd0015bc30 0x7fcd000b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.321+0000 7fccff200640 1 -- 172.21.15.71:0/1670075246 >> 172.21.15.71:0/1670075246 conn(0x7fcd000cfa20 msgr2=0x7fcd000cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:16.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.321+0000 7fccff200640 1 -- 172.21.15.71:0/1670075246 shutdown_connections 2024-09-16T07:31:16.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.321+0000 7fccff200640 1 -- 172.21.15.71:0/1670075246 wait complete. 2024-09-16T07:31:16.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.322+0000 7fccff200640 1 Processor -- start 2024-09-16T07:31:16.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.323+0000 7fccff200640 1 -- start start 2024-09-16T07:31:16.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.323+0000 7fccff200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd000d0540 0x7fcd000c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.323+0000 7fccff200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd0015bc30 0x7fcd001f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.323+0000 7fccff200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd001f2070 0x7fcd0014fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.323+0000 7fccff200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00074610 con 0x7fcd0015bc30 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.323+0000 7fccff200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00074490 con 0x7fcd000d0540 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.323+0000 7fccff200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcd00074790 con 0x7fcd001f2070 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd0015bc30 0x7fcd001f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd0015bc30 0x7fcd001f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47702/0 (socket says 172.21.15.71:47702) 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 -- 172.21.15.71:0/2168313273 learned_addr learned my addr 172.21.15.71:0/2168313273 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfde00640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd000d0540 0x7fcd000c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfe800640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd001f2070 0x7fcd0014fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 -- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd001f2070 msgr2=0x7fcd0014fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd001f2070 0x7fcd0014fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 -- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd000d0540 msgr2=0x7fcd000c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd000d0540 0x7fcd000c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 -- 172.21.15.71:0/2168313273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd000c7030 con 0x7fcd0015bc30 2024-09-16T07:31:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfde00640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd000d0540 0x7fcd000c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.324+0000 7fccfd400640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd0015bc30 0x7fcd001f6040 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fccec000c00 tx=0x7fccec032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.325+0000 7fccf6000640 1 -- 172.21.15.71:0/2168313273 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec040070 con 0x7fcd0015bc30 2024-09-16T07:31:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.325+0000 7fccf6000640 1 -- 172.21.15.71:0/2168313273 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccec0038e0 con 0x7fcd0015bc30 2024-09-16T07:31:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.325+0000 7fccff200640 1 -- 172.21.15.71:0/2168313273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd001f6580 con 0x7fcd0015bc30 2024-09-16T07:31:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.325+0000 7fccf6000640 1 -- 172.21.15.71:0/2168313273 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec005c90 con 0x7fcd0015bc30 2024-09-16T07:31:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.325+0000 7fccff200640 1 -- 172.21.15.71:0/2168313273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001f2860 con 0x7fcd0015bc30 2024-09-16T07:31:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.327+0000 7fccf6000640 1 -- 172.21.15.71:0/2168313273 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fccec039040 con 0x7fcd0015bc30 2024-09-16T07:31:16.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.329+0000 7fccf6000640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcce0079930 0x7fcce007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.329+0000 7fcd08015740 1 Processor -- start 2024-09-16T07:31:16.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.329+0000 7fccf6000640 1 -- 172.21.15.71:0/2168313273 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fccec0baa50 con 0x7fcd0015bc30 2024-09-16T07:31:16.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.329+0000 7fcd08015740 1 -- start start 2024-09-16T07:31:16.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.330+0000 7fccfde00640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcce0079930 0x7fcce007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.330+0000 7fcd08015740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a25155ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.330+0000 7fcd08015740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a251519c10 0x55a25155d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.330+0000 7fcd08015740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a25155dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.330+0000 7fcd08015740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a25149c430 con 0x55a251519c10 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.330+0000 7fcd08015740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a25148c5d0 con 0x55a25151a190 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.330+0000 7fcd08015740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55a25147ba00 con 0x55a25155eaa0 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a251519c10 0x55a25155d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfde00640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcce0079930 0x7fcce007bdf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcd000b56c0 tx=0x7fccf0005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a251519c10 0x55a25155d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47710/0 (socket says 172.21.15.71:47710) 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 -- 172.21.15.71:0/1578929842 learned_addr learned my addr 172.21.15.71:0/1578929842 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfde00640 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a25155ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfe800640 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a25155dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 -- 172.21.15.71:0/1578929842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 msgr2=0x55a25155ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a25155ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 -- 172.21.15.71:0/1578929842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 msgr2=0x55a25155dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a25155dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 -- 172.21.15.71:0/1578929842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a251444520 con 0x55a251519c10 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fccfd400640 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a251519c10 0x55a25155d580 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fccec035040 tx=0x7fccec083ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e2ca3ab50b62ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fcceaa00640 1 -- 172.21.15.71:0/1578929842 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec04a040 con 0x55a251519c10 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fcceaa00640 1 -- 172.21.15.71:0/1578929842 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccec084a90 con 0x55a251519c10 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.331+0000 7fcceaa00640 1 -- 172.21.15.71:0/1578929842 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec084390 con 0x55a251519c10 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 -- 172.21.15.71:0/1578929842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a251519c10 msgr2=0x55a25155d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a251519c10 0x55a25155d580 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fccec035040 tx=0x7fccec083ce0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 -- 172.21.15.71:0/1578929842 shutdown_connections 2024-09-16T07:31:16.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a25155dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a251519c10 0x55a25155d580 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 --2- 172.21.15.71:0/1578929842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a25155ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 -- 172.21.15.71:0/1578929842 >> 172.21.15.71:0/1578929842 conn(0x55a2515151e0 msgr2=0x55a251518c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:16.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 -- 172.21.15.71:0/1578929842 shutdown_connections 2024-09-16T07:31:16.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.332+0000 7fcd08015740 1 -- 172.21.15.71:0/1578929842 wait complete. 2024-09-16T07:31:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.333+0000 7fcd08015740 1 -- start start 2024-09-16T07:31:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.333+0000 7fcd08015740 1 Processor -- start 2024-09-16T07:31:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fcd08015740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a25156e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fcd08015740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a25155dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fcd08015740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a25155e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fcd08015740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a251422330 con 0x55a25152e620 2024-09-16T07:31:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fcd08015740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a25147f810 con 0x55a25151a190 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fcd08015740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55a2511ec620 con 0x55a25155eaa0 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fcd08015740 10 client.0 _ll_register_callbacks cb 0x55a251427420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fccfe800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a25155e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fccfe800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a25155e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47724/0 (socket says 172.21.15.71:47724) 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fccfe800640 1 -- 172.21.15.71:0/878940144 learned_addr learned my addr 172.21.15.71:0/878940144 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fccfde00640 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a25156e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.334+0000 7fccfd400640 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a25155dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccfe800640 1 -- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 msgr2=0x55a25155dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccfe800640 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a25155dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccfe800640 1 -- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 msgr2=0x55a25156e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccfe800640 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a25156e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccfe800640 1 -- 172.21.15.71:0/878940144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2511dd850 con 0x55a25152e620 2024-09-16T07:31:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccfe800640 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a25155e150 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fccf8011d40 tx=0x7fccf800f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7b5d8f908b338d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccdf400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccdf400640 1 -- 172.21.15.71:0/878940144 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8019070 con 0x55a25152e620 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccdf400640 1 -- 172.21.15.71:0/878940144 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fccf800fc30 con 0x55a25152e620 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fcd08015740 1 -- 172.21.15.71:0/878940144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccec084a90 con 0x55a25152e620 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fccdf400640 1 -- 172.21.15.71:0/878940144 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fccf8023050 con 0x55a25152e620 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fcd08015740 10 client.7125 fetch_fsmap learned FSMap version 56 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.335+0000 7fcd08015740 1 -- 172.21.15.71:0/878940144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55a2512546b0 con 0x55a25152e620 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fccdf400640 1 -- 172.21.15.71:0/878940144 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccec084a90 con 0x55a25152e620 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fccdf400640 1 -- 172.21.15.71:0/878940144 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7fccf8026020 con 0x55a25152e620 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fcd08015740 10 client.7125 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fcd08015740 10 client.7125 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:31:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fcd08015740 20 client.7125 populate_metadata read hostname 'smithi071' 2024-09-16T07:31:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fcd08015740 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] conn(0x55a2515429b0 0x55a251542d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fcd08015740 4 client.7125 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:31:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fcd08015740 1 -- 172.21.15.71:0/878940144 --> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a25148e520 con 0x55a2515429b0 2024-09-16T07:31:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.336+0000 7fccfde00640 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] conn(0x55a2515429b0 0x55a251542d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.337+0000 7fccdf400640 10 client.7125 ms_handle_connect on v2:172.21.15.177:6835/4183855149 2024-09-16T07:31:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.337+0000 7fccfde00640 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] conn(0x55a2515429b0 0x55a251542d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d48c2fbd2e237b server_cookie=48c254e879a0d5f0 in_seq=0 out_seq=0 2024-09-16T07:31:16.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.338+0000 7fccdf400640 1 -- 172.21.15.71:0/878940144 <== mds.0 v2:172.21.15.177:6835/4183855149 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a25148e520 con 0x55a2515429b0 2024-09-16T07:31:16.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.338+0000 7fccdf400640 10 client.7125 handle_command_reply: tid=0 2024-09-16T07:31:16.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.338+0000 7fcd08015740 1 client.7125 shutdown 2024-09-16T07:31:16.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 -- 172.21.15.71:0/878940144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 msgr2=0x55a25155e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a25155e150 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fccf8011d40 tx=0x7fccf800f5a0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 -- 172.21.15.71:0/878940144 shutdown_connections 2024-09-16T07:31:16.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] conn(0x55a2515429b0 0x55a251542d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a25155e150 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a25155dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fccdf400640 0 client.7125 ms_handle_reset on v2:172.21.15.177:6835/4183855149 2024-09-16T07:31:16.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 --2- 172.21.15.71:0/878940144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a25156e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 -- 172.21.15.71:0/878940144 >> 172.21.15.71:0/878940144 conn(0x55a251519c10 msgr2=0x55a251556820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:16.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 -- 172.21.15.71:0/878940144 shutdown_connections 2024-09-16T07:31:16.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 1 -- 172.21.15.71:0/878940144 wait complete. 2024-09-16T07:31:16.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.339+0000 7fcd08015740 20 client.7125 trim_cache size 0 max 16384 2024-09-16T07:31:16.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.358+0000 7fcd08015740 1 Processor -- start 2024-09-16T07:31:16.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.358+0000 7fcd08015740 1 -- start start 2024-09-16T07:31:16.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fcd08015740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a251542f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fcd08015740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a251512160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fcd08015740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a2515126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fcd08015740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd000cec00 con 0x55a25152e620 2024-09-16T07:31:16.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fcd08015740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd000c9a60 con 0x55a25151a190 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fcd08015740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0015d190 con 0x55a25155eaa0 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a251512160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a251512160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47736/0 (socket says 172.21.15.71:47736) 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfd400640 1 -- 172.21.15.71:0/3023724944 learned_addr learned my addr 172.21.15.71:0/3023724944 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfde00640 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a251542f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfe800640 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a2515126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfd400640 1 -- 172.21.15.71:0/3023724944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 msgr2=0x55a2515126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfd400640 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a2515126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfd400640 1 -- 172.21.15.71:0/3023724944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 msgr2=0x55a251542f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfd400640 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a251542f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.359+0000 7fccfd400640 1 -- 172.21.15.71:0/3023724944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a25155dc10 con 0x55a25152e620 2024-09-16T07:31:16.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.360+0000 7fccfd400640 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a251512160 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fccec035040 tx=0x7fccec047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2309b14316832d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:16.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.360+0000 7fccea000640 1 -- 172.21.15.71:0/3023724944 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec04a040 con 0x55a25152e620 2024-09-16T07:31:16.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.360+0000 7fccea000640 1 -- 172.21.15.71:0/3023724944 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccec084a90 con 0x55a25152e620 2024-09-16T07:31:16.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.360+0000 7fccea000640 1 -- 172.21.15.71:0/3023724944 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec088e70 con 0x55a25152e620 2024-09-16T07:31:16.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.360+0000 7fcd08015740 1 -- 172.21.15.71:0/3023724944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 msgr2=0x55a251512160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.360+0000 7fcd08015740 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a251512160 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fccec035040 tx=0x7fccec047000 comp rx=0 tx=0).stop 2024-09-16T07:31:16.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.360+0000 7fcd08015740 1 -- 172.21.15.71:0/3023724944 shutdown_connections 2024-09-16T07:31:16.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.361+0000 7fcd08015740 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25155eaa0 0x55a2515126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.361+0000 7fcd08015740 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25152e620 0x55a251512160 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.361+0000 7fcd08015740 1 --2- 172.21.15.71:0/3023724944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25151a190 0x55a251542f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.361+0000 7fcd08015740 1 -- 172.21.15.71:0/3023724944 >> 172.21.15.71:0/3023724944 conn(0x55a251519c10 msgr2=0x55a25155cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.361+0000 7fcd08015740 1 -- 172.21.15.71:0/3023724944 shutdown_connections 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.361+0000 7fcd08015740 1 -- 172.21.15.71:0/3023724944 wait complete. 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.361+0000 7fcd08015740 1 -- start start 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.361+0000 7fcd08015740 1 Processor -- start 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.362+0000 7fcd08015740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25151a190 0x55a251578c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.362+0000 7fcd08015740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25152e620 0x55a2515827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.362+0000 7fcd08015740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25155eaa0 0x55a251582d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.362+0000 7fcd08015740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a251571910 con 0x55a25151a190 2024-09-16T07:31:16.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.362+0000 7fcd08015740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a25156e9f0 con 0x55a25155eaa0 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.362+0000 7fcd08015740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55a251530ef0 con 0x55a25152e620 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.362+0000 7fcd08015740 10 client.0 _ll_register_callbacks cb 0x7fccf8011940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.362+0000 7fccfde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25151a190 0x55a251578c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25151a190 0x55a251578c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47752/0 (socket says 172.21.15.71:47752) 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfde00640 1 -- 172.21.15.71:0/2638441923 learned_addr learned my addr 172.21.15.71:0/2638441923 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfe800640 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25155eaa0 0x55a251582d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfd400640 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25152e620 0x55a2515827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfde00640 1 -- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25152e620 msgr2=0x55a2515827d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfde00640 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25152e620 0x55a2515827d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfde00640 1 -- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25155eaa0 msgr2=0x55a251582d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfde00640 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25155eaa0 0x55a251582d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfde00640 1 -- 172.21.15.71:0/2638441923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a251519380 con 0x55a25151a190 2024-09-16T07:31:16.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfe800640 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25155eaa0 0x55a251582d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccfde00640 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25151a190 0x55a251578c60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x55a2515179e0 tx=0x7fccf0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ef2ad7ba114fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccdf400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccdf400640 1 -- 172.21.15.71:0/2638441923 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0004220 con 0x55a25151a190 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fccdf400640 1 -- 172.21.15.71:0/2638441923 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fccf00043c0 con 0x55a25151a190 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.363+0000 7fcd08015740 1 -- 172.21.15.71:0/2638441923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccec084a90 con 0x55a25151a190 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fccdf400640 1 -- 172.21.15.71:0/2638441923 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fccf003b050 con 0x55a25151a190 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fcd08015740 10 client.7131 fetch_fsmap learned FSMap version 56 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fcd08015740 1 -- 172.21.15.71:0/2638441923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55a251530150 con 0x55a25151a190 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fccdf400640 1 -- 172.21.15.71:0/2638441923 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccec084a90 con 0x55a25151a190 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fccdf400640 1 -- 172.21.15.71:0/2638441923 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7fccf003e020 con 0x55a25151a190 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fcd08015740 10 client.7131 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fcd08015740 10 client.7131 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:31:16.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fcd08015740 20 client.7131 populate_metadata read hostname 'smithi071' 2024-09-16T07:31:16.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fcd08015740 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] conn(0x55a2515de390 0x55a2515de770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fcd08015740 4 client.7131 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:31:16.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fcd08015740 1 -- 172.21.15.71:0/2638441923 --> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] -- command(tid 0: {"prefix": "status"}) -- 0x55a251489fd0 con 0x55a2515de390 2024-09-16T07:31:16.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.364+0000 7fccfd400640 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] conn(0x55a2515de390 0x55a2515de770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fccfd400640 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] conn(0x55a2515de390 0x55a2515de770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88667ebe607e7606 server_cookie=4f08c06352fbdeee in_seq=0 out_seq=0 2024-09-16T07:31:16.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fccdf400640 10 client.7131 ms_handle_connect on v2:172.21.15.177:6835/4183855149 2024-09-16T07:31:16.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fccdf400640 1 -- 172.21.15.71:0/2638441923 <== mds.0 v2:172.21.15.177:6835/4183855149 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55a251489fd0 con 0x55a2515de390 2024-09-16T07:31:16.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fccdf400640 10 client.7131 handle_command_reply: tid=0 2024-09-16T07:31:16.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fcd08015740 1 client.7131 shutdown 2024-09-16T07:31:16.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fcd08015740 1 -- 172.21.15.71:0/2638441923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25151a190 msgr2=0x55a251578c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fcd08015740 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25151a190 0x55a251578c60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x55a2515179e0 tx=0x7fccf0014070 comp rx=0 tx=0).stop 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fcd08015740 1 -- 172.21.15.71:0/2638441923 shutdown_connections 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.365+0000 7fcd08015740 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:6835/4183855149,v1:172.21.15.177:6837/4183855149] conn(0x55a2515de390 0x55a2515de770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fcd08015740 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a25155eaa0 0x55a251582d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fcd08015740 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a25152e620 0x55a2515827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fcd08015740 1 --2- 172.21.15.71:0/2638441923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a25151a190 0x55a251578c60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccdf400640 0 client.7131 ms_handle_reset on v2:172.21.15.177:6835/4183855149 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fcd08015740 1 -- 172.21.15.71:0/2638441923 >> 172.21.15.71:0/2638441923 conn(0x55a251519c10 msgr2=0x55a25155cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fcd08015740 1 -- 172.21.15.71:0/2638441923 shutdown_connections 2024-09-16T07:31:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fcd08015740 1 -- 172.21.15.71:0/2638441923 wait complete. 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fcd08015740 20 client.7131 trim_cache size 0 max 16384 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7008, 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 6.1708440270000002, 2024-09-16T07:31:16.371 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 56, 2024-09-16T07:31:16.372 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 92, 2024-09-16T07:31:16.372 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 92, 2024-09-16T07:31:16.372 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 10.57608156 2024-09-16T07:31:16.372 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 -- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcce0079930 msgr2=0x7fcce007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcce0079930 0x7fcce007bdf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcd000b56c0 tx=0x7fccf0005f90 comp rx=0 tx=0).stop 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 -- 172.21.15.71:0/2168313273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd0015bc30 msgr2=0x7fcd001f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd0015bc30 0x7fcd001f6040 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fccec000c00 tx=0x7fccec032c10 comp rx=0 tx=0).stop 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 -- 172.21.15.71:0/2168313273 shutdown_connections 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcce0079930 0x7fcce007bdf0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd001f2070 0x7fcd0014fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd0015bc30 0x7fcd001f6040 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 --2- 172.21.15.71:0/2168313273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd000d0540 0x7fcd000c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.366+0000 7fccff200640 1 -- 172.21.15.71:0/2168313273 >> 172.21.15.71:0/2168313273 conn(0x7fcd000cfa20 msgr2=0x7fcd000cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.367+0000 7fccff200640 1 -- 172.21.15.71:0/2168313273 shutdown_connections 2024-09-16T07:31:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.367+0000 7fccff200640 1 -- 172.21.15.71:0/2168313273 wait complete. 2024-09-16T07:31:16.381 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:31:16.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.540+0000 7f16afc00640 1 -- 172.21.15.71:0/3421178274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1690004800 msgr2=0x7f1690004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.540+0000 7f16afc00640 1 --2- 172.21.15.71:0/3421178274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1690004800 0x7f1690004c80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1694009b50 tx=0x7f1690005830 comp rx=0 tx=0).stop 2024-09-16T07:31:16.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.540+0000 7f16afc00640 1 -- 172.21.15.71:0/3421178274 shutdown_connections 2024-09-16T07:31:16.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.540+0000 7f16afc00640 1 --2- 172.21.15.71:0/3421178274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1690004800 0x7f1690004c80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.540+0000 7f16afc00640 1 -- 172.21.15.71:0/3421178274 >> 172.21.15.71:0/3421178274 conn(0x7f16a80b1950 msgr2=0x7f16a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:16.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.540+0000 7f16afc00640 1 -- 172.21.15.71:0/3421178274 shutdown_connections 2024-09-16T07:31:16.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.540+0000 7f16afc00640 1 -- 172.21.15.71:0/3421178274 wait complete. 2024-09-16T07:31:16.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.542+0000 7f16afc00640 1 Processor -- start 2024-09-16T07:31:16.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.542+0000 7f16afc00640 1 -- start start 2024-09-16T07:31:16.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16afc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1690004800 0x7f16a809cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16afc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a81f2ab0 0x7f16a81f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16afc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a81f38d0 0x7f16a81f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16afc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8074420 con 0x7f16a81f2ab0 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16afc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f16a80b2700 con 0x7f16a81f38d0 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16afc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f16a80745a0 con 0x7f1690004800 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a81f2ab0 0x7f16a81f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a81f2ab0 0x7f16a81f2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47774/0 (socket says 172.21.15.71:47774) 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16a7e00640 1 -- 172.21.15.71:0/2126001045 learned_addr learned my addr 172.21.15.71:0/2126001045 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16ad200640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1690004800 0x7f16a809cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16adc00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a81f38d0 0x7f16a81f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16a7e00640 1 -- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1690004800 msgr2=0x7f16a809cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16a7e00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1690004800 0x7f16a809cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16a7e00640 1 -- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a81f38d0 msgr2=0x7f16a81f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16a7e00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a81f38d0 0x7f16a81f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16a7e00640 1 -- 172.21.15.71:0/2126001045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a80c6fa0 con 0x7f16a81f2ab0 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.543+0000 7f16adc00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a81f38d0 0x7f16a81f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.544+0000 7f16a7e00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a81f2ab0 0x7f16a81f2f30 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1694036040 tx=0x7f16940336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.544+0000 7f16a4c00640 1 -- 172.21.15.71:0/2126001045 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694038040 con 0x7f16a81f2ab0 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.544+0000 7f16a4c00640 1 -- 172.21.15.71:0/2126001045 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1694041070 con 0x7f16a81f2ab0 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.544+0000 7f16a4c00640 1 -- 172.21.15.71:0/2126001045 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169403c430 con 0x7f16a81f2ab0 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.544+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a80c7280 con 0x7f16a81f2ab0 2024-09-16T07:31:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.544+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a80c7840 con 0x7f16a81f2ab0 2024-09-16T07:31:16.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.545+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a80acd30 con 0x7f16a81f2ab0 2024-09-16T07:31:16.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.546+0000 7f16a4c00640 1 -- 172.21.15.71:0/2126001045 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f169403c5d0 con 0x7f16a81f2ab0 2024-09-16T07:31:16.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.547+0000 7f16a4c00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1688079930 0x7f168807bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:16.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.547+0000 7f16a4c00640 1 -- 172.21.15.71:0/2126001045 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f16940b9d30 con 0x7f16a81f2ab0 2024-09-16T07:31:16.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.548+0000 7f16ad200640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1688079930 0x7f168807bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.549+0000 7f16ad200640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1688079930 0x7f168807bdf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f169c000fb0 tx=0x7f169c006040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:16.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.551+0000 7f16a4c00640 1 -- 172.21.15.71:0/2126001045 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f16940837a0 con 0x7f16a81f2ab0 2024-09-16T07:31:16.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.773+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f16a80d0ad0 con 0x7f16a81f2ab0 2024-09-16T07:31:16.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.773+0000 7f16a4c00640 1 -- 172.21.15.71:0/2126001045 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7f16940881d0 con 0x7f16a81f2ab0 2024-09-16T07:31:16.773 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:31:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.775+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1688079930 msgr2=0x7f168807bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.775+0000 7f16afc00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1688079930 0x7f168807bdf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f169c000fb0 tx=0x7f169c006040 comp rx=0 tx=0).stop 2024-09-16T07:31:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.775+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a81f2ab0 msgr2=0x7f16a81f2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.775+0000 7f16afc00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a81f2ab0 0x7f16a81f2f30 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1694036040 tx=0x7f16940336f0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.775+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 shutdown_connections 2024-09-16T07:31:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.775+0000 7f16afc00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1688079930 0x7f168807bdf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.776+0000 7f16afc00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a81f38d0 0x7f16a81f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.776+0000 7f16afc00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16a81f2ab0 0x7f16a81f2f30 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.776+0000 7f16afc00640 1 --2- 172.21.15.71:0/2126001045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1690004800 0x7f16a809cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:16.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.776+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 >> 172.21.15.71:0/2126001045 conn(0x7f16a80b1950 msgr2=0x7f16a80cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:16.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.776+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 shutdown_connections 2024-09-16T07:31:16.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:16.776+0000 7f16afc00640 1 -- 172.21.15.71:0/2126001045 wait complete. 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:31:16.791 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:31:16.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:16.792 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:31:16.827 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:31:16.828 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42200sec preferred_lft 42200sec 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:31:16.829 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:31:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:31:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:31:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:31:16.830 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:31:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:31:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:31:16.830 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:31:16.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:16.830 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:31:16.921 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:31:16.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:16.924 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:31:16.960 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:31:16.960 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:31:16.960 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:31:16.960 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:31:16.960 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:31:16.961 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:31:16.961 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:31:16.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:31:16.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:31:16.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:31:16.961 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:31:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:31:17.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:31:17.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:17.156 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:31:17.192 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:31:17.192 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:31:17.192 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:31:17.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:17.193 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:31:17.289 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:31:17.289 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:31:17.324 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:31:17.324 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:31:17.325 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:31:17.399 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:31:17.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:17.399 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:31:17.527 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:31:17.527 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:31:17.555 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:31:17.555 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:31:17.555 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:31:17.555 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:31:17.555 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:31:17.557 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:31:17.557 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:31:17.557 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:31:17.557 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:31:17.557 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:31:17.560 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:31:17.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:17.560 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:31:17.616 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:31:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:31:17.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:31:17.718 INFO:teuthology.orchestra.run.smithi071.stdout:client6712 2024-09-16T07:31:17.718 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:31:17.718 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:31:17.753 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:31:17.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.954+0000 7f6445c00640 1 -- 172.21.15.71:0/2178956529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644015dfc0 msgr2=0x7f644006f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:17.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.954+0000 7f6445c00640 1 --2- 172.21.15.71:0/2178956529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644015dfc0 0x7f644006f300 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f64400b3050 tx=0x7f643002a560 comp rx=0 tx=0).stop 2024-09-16T07:31:17.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.954+0000 7f6445c00640 1 -- 172.21.15.71:0/2178956529 shutdown_connections 2024-09-16T07:31:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.954+0000 7f6445c00640 1 --2- 172.21.15.71:0/2178956529 >> v2:172.21.15.177:3300/0 conn(0x7f64400b4770 0x7f644006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.954+0000 7f6445c00640 1 --2- 172.21.15.71:0/2178956529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f644015dfc0 0x7f644006f300 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.954+0000 7f6445c00640 1 -- 172.21.15.71:0/2178956529 >> 172.21.15.71:0/2178956529 conn(0x7f64400b18e0 msgr2=0x7f64400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.954+0000 7f6445c00640 1 -- 172.21.15.71:0/2178956529 shutdown_connections 2024-09-16T07:31:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.954+0000 7f6445c00640 1 -- 172.21.15.71:0/2178956529 wait complete. 2024-09-16T07:31:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.956+0000 7f6445c00640 1 Processor -- start 2024-09-16T07:31:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.956+0000 7f6445c00640 1 -- start start 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f6445c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644006ec30 0x7f644009d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f6445c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64400b4770 0x7f6440153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f6445c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644015dfc0 0x7f6440152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f6445c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f644015e460 con 0x7f64400b4770 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f6445c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64400ae8f0 con 0x7f644015dfc0 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f6445c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64401622c0 con 0x7f644006ec30 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f643e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64400b4770 0x7f6440153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f643e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64400b4770 0x7f6440153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47794/0 (socket says 172.21.15.71:47794) 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.957+0000 7f643e000640 1 -- 172.21.15.71:0/800122595 learned_addr learned my addr 172.21.15.71:0/800122595 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f643ea00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644006ec30 0x7f644009d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f643f400640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644015dfc0 0x7f6440152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f643e000640 1 -- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644006ec30 msgr2=0x7f644009d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f643e000640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644006ec30 0x7f644009d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f643e000640 1 -- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644015dfc0 msgr2=0x7f6440152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f643e000640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644015dfc0 0x7f6440152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f643e000640 1 -- 172.21.15.71:0/800122595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644009bbf0 con 0x7f64400b4770 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f643e000640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64400b4770 0x7f6440153500 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c920 tx=0x7f642c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f6436a00640 1 -- 172.21.15.71:0/800122595 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c007c70 con 0x7f64400b4770 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f6436a00640 1 -- 172.21.15.71:0/800122595 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f642c00f450 con 0x7f64400b4770 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f6436a00640 1 -- 172.21.15.71:0/800122595 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c00f8a0 con 0x7f64400b4770 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.958+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644009bed0 con 0x7f64400b4770 2024-09-16T07:31:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.959+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644009c490 con 0x7f64400b4770 2024-09-16T07:31:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.959+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6440069360 con 0x7f64400b4770 2024-09-16T07:31:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.961+0000 7f6436a00640 1 -- 172.21.15.71:0/800122595 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f642c00fa40 con 0x7f64400b4770 2024-09-16T07:31:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.962+0000 7f6436a00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6420079930 0x7f642007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.962+0000 7f6436a00640 1 -- 172.21.15.71:0/800122595 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f642c099f70 con 0x7f64400b4770 2024-09-16T07:31:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.962+0000 7f643ea00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6420079930 0x7f642007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.963+0000 7f643ea00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6420079930 0x7f642007bdf0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6430033000 tx=0x7f6430004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:17.966+0000 7f6436a00640 1 -- 172.21.15.71:0/800122595 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f642c014030 con 0x7f64400b4770 2024-09-16T07:31:18.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.196+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f6440162720 con 0x7f64400b4770 2024-09-16T07:31:18.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.229+0000 7f6436a00640 1 -- 172.21.15.71:0/800122595 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) ==== 136+0+0 (secure 0 0 0) 0x7f642c0636a0 con 0x7f64400b4770 2024-09-16T07:31:18.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6420079930 msgr2=0x7f642007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6420079930 0x7f642007bdf0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6430033000 tx=0x7f6430004730 comp rx=0 tx=0).stop 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64400b4770 msgr2=0x7f6440153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64400b4770 0x7f6440153500 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c920 tx=0x7f642c00cde0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 shutdown_connections 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6420079930 0x7f642007bdf0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644015dfc0 0x7f6440152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64400b4770 0x7f6440153500 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 --2- 172.21.15.71:0/800122595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644006ec30 0x7f644009d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 >> 172.21.15.71:0/800122595 conn(0x7f64400b18e0 msgr2=0x7f64400b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 shutdown_connections 2024-09-16T07:31:18.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.232+0000 7f6445c00640 1 -- 172.21.15.71:0/800122595 wait complete. 2024-09-16T07:31:18.302 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:31:18.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.460+0000 7ff1d0200640 1 -- 172.21.15.71:0/246496139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1b0004d40 msgr2=0x7ff1b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.460+0000 7ff1d0200640 1 --2- 172.21.15.71:0/246496139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1b0004d40 0x7ff1b00051c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0001d60 tx=0x7ff1b0005830 comp rx=0 tx=0).stop 2024-09-16T07:31:18.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.460+0000 7ff1ce200640 1 -- 172.21.15.71:0/246496139 reap_dead start 2024-09-16T07:31:18.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.460+0000 7ff1d0200640 1 -- 172.21.15.71:0/246496139 shutdown_connections 2024-09-16T07:31:18.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.461+0000 7ff1d0200640 1 -- 172.21.15.71:0/246496139 >> 172.21.15.71:0/246496139 conn(0x7ff1c80b1950 msgr2=0x7ff1c80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:18.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.461+0000 7ff1d0200640 1 -- 172.21.15.71:0/246496139 shutdown_connections 2024-09-16T07:31:18.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.461+0000 7ff1d0200640 1 -- 172.21.15.71:0/246496139 wait complete. 2024-09-16T07:31:18.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.462+0000 7ff1d0200640 1 Processor -- start 2024-09-16T07:31:18.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.462+0000 7ff1d0200640 1 -- start start 2024-09-16T07:31:18.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.463+0000 7ff1d0200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0004d40 0x7ff1c80cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:18.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.463+0000 7ff1d0200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8156a80 0x7ff1c8156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:18.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.463+0000 7ff1d0200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c81f4bc0 0x7ff1c81f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:18.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.463+0000 7ff1d0200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c806f9f0 con 0x7ff1c8156a80 2024-09-16T07:31:18.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.463+0000 7ff1d0200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c806f870 con 0x7ff1b0004d40 2024-09-16T07:31:18.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.463+0000 7ff1d0200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c80b2440 con 0x7ff1c81f4bc0 2024-09-16T07:31:18.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8156a80 0x7ff1c8156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:18.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8156a80 0x7ff1c8156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47818/0 (socket says 172.21.15.71:47818) 2024-09-16T07:31:18.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 -- 172.21.15.71:0/1660695014 learned_addr learned my addr 172.21.15.71:0/1660695014 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:18.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cd800640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0004d40 0x7ff1c80cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:18.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1ce200640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c81f4bc0 0x7ff1c81f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:18.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 -- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c81f4bc0 msgr2=0x7ff1c81f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c81f4bc0 0x7ff1c81f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 -- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0004d40 msgr2=0x7ff1c80cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0004d40 0x7ff1c80cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 -- 172.21.15.71:0/1660695014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c80cdb80 con 0x7ff1c8156a80 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.464+0000 7ff1cce00640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8156a80 0x7ff1c8156f00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4011ae0 tx=0x7ff1b4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.465+0000 7ff1c5600640 1 -- 172.21.15.71:0/1660695014 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b401e070 con 0x7ff1c8156a80 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.465+0000 7ff1c5600640 1 -- 172.21.15.71:0/1660695014 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1b400fe70 con 0x7ff1c8156a80 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.465+0000 7ff1c5600640 1 -- 172.21.15.71:0/1660695014 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4004290 con 0x7ff1c8156a80 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.465+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c80cde60 con 0x7ff1c8156a80 2024-09-16T07:31:18.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.465+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c80ce420 con 0x7ff1c8156a80 2024-09-16T07:31:18.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.466+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c8074440 con 0x7ff1c8156a80 2024-09-16T07:31:18.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.467+0000 7ff1c5600640 1 -- 172.21.15.71:0/1660695014 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7ff1b4021070 con 0x7ff1c8156a80 2024-09-16T07:31:18.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.468+0000 7ff1c5600640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff1a8079960 0x7ff1a807be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:18.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.468+0000 7ff1c5600640 1 -- 172.21.15.71:0/1660695014 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7ff1b409a280 con 0x7ff1c8156a80 2024-09-16T07:31:18.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.468+0000 7ff1cd800640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff1a8079960 0x7ff1a807be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:18.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.469+0000 7ff1cd800640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff1a8079960 0x7ff1a807be20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc006cd0 tx=0x7ff1bc0067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:18.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.472+0000 7ff1c5600640 1 -- 172.21.15.71:0/1660695014 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff1b4063a70 con 0x7ff1c8156a80 2024-09-16T07:31:18.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.680+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff1c809dea0 con 0x7ff1a8079960 2024-09-16T07:31:18.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.682+0000 7ff1c5600640 1 -- 172.21.15.71:0/1660695014 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff1c809dea0 con 0x7ff1a8079960 2024-09-16T07:31:18.681 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:31:18.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:18.682 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:31:18.682 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:18.682 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:31:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff1a8079960 msgr2=0x7ff1a807be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff1a8079960 0x7ff1a807be20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc006cd0 tx=0x7ff1bc0067a0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8156a80 msgr2=0x7ff1c8156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8156a80 0x7ff1c8156f00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4011ae0 tx=0x7ff1b4009e90 comp rx=0 tx=0).stop 2024-09-16T07:31:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 shutdown_connections 2024-09-16T07:31:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff1a8079960 0x7ff1a807be20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c81f4bc0 0x7ff1c81f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8156a80 0x7ff1c8156f00 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 --2- 172.21.15.71:0/1660695014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0004d40 0x7ff1c80cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 >> 172.21.15.71:0/1660695014 conn(0x7ff1c80b1950 msgr2=0x7ff1c809cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 shutdown_connections 2024-09-16T07:31:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.684+0000 7ff1d0200640 1 -- 172.21.15.71:0/1660695014 wait complete. 2024-09-16T07:31:18.700 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:31:18.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.861+0000 7f6657e00640 1 -- 172.21.15.71:0/2271991523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c003a00 msgr2=0x7f663c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.861+0000 7f6657e00640 1 --2- 172.21.15.71:0/2271991523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c003a00 0x7f663c003e20 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6648005fd0 tx=0x7f663c005830 comp rx=0 tx=0).stop 2024-09-16T07:31:18.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.861+0000 7f6657400640 1 -- 172.21.15.71:0/2271991523 reap_dead start 2024-09-16T07:31:18.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.861+0000 7f6657e00640 1 -- 172.21.15.71:0/2271991523 shutdown_connections 2024-09-16T07:31:18.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.861+0000 7f6657e00640 1 -- 172.21.15.71:0/2271991523 >> 172.21.15.71:0/2271991523 conn(0x7f66580cf9f0 msgr2=0x7f66580cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:18.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.861+0000 7f6657e00640 1 -- 172.21.15.71:0/2271991523 shutdown_connections 2024-09-16T07:31:18.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.861+0000 7f6657e00640 1 -- 172.21.15.71:0/2271991523 wait complete. 2024-09-16T07:31:18.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.862+0000 7f6657e00640 1 Processor -- start 2024-09-16T07:31:18.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.863+0000 7f6657e00640 1 -- start start 2024-09-16T07:31:18.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.863+0000 7f6657e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f663c003a00 0x7f66581f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:18.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.863+0000 7f6657e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580cf280 0x7f66580adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:18.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.863+0000 7f6657e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66581f4b20 0x7f6658157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:18.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f6657e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f66580ae4c0 con 0x7f66580cf280 2024-09-16T07:31:18.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f6657e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6658074180 con 0x7f66581f4b20 2024-09-16T07:31:18.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f6657e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f66580ae640 con 0x7f663c003a00 2024-09-16T07:31:18.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f664fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580cf280 0x7f66580adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:18.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f664fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580cf280 0x7f66580adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47834/0 (socket says 172.21.15.71:47834) 2024-09-16T07:31:18.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f664fe00640 1 -- 172.21.15.71:0/391539386 learned_addr learned my addr 172.21.15.71:0/391539386 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:18.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f6656a00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f663c003a00 0x7f66581f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:18.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f6657400640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66581f4b20 0x7f6658157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:18.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f664fe00640 1 -- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f663c003a00 msgr2=0x7f66581f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f664fe00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f663c003a00 0x7f66581f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f664fe00640 1 -- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66581f4b20 msgr2=0x7f6658157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:18.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f664fe00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66581f4b20 0x7f6658157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:18.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f664fe00640 1 -- 172.21.15.71:0/391539386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66581f4340 con 0x7f66580cf280 2024-09-16T07:31:18.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.864+0000 7f6656a00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f663c003a00 0x7f66581f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:31:18.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.865+0000 7f664fe00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580cf280 0x7f66580adaf0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f664000e5c0 tx=0x7f664000ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:18.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.865+0000 7f664f400640 1 -- 172.21.15.71:0/391539386 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640015070 con 0x7f66580cf280 2024-09-16T07:31:18.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.865+0000 7f664f400640 1 -- 172.21.15.71:0/391539386 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6640010980 con 0x7f66580cf280 2024-09-16T07:31:18.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.865+0000 7f664f400640 1 -- 172.21.15.71:0/391539386 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640010db0 con 0x7f66580cf280 2024-09-16T07:31:18.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.865+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66581f80a0 con 0x7f66580cf280 2024-09-16T07:31:18.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.865+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66581f8660 con 0x7f66580cf280 2024-09-16T07:31:18.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.866+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6658074300 con 0x7f66580cf280 2024-09-16T07:31:18.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.867+0000 7f664f400640 1 -- 172.21.15.71:0/391539386 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f6640018070 con 0x7f66580cf280 2024-09-16T07:31:18.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.868+0000 7f664f400640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6634079930 0x7f663407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:18.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.868+0000 7f664f400640 1 -- 172.21.15.71:0/391539386 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f6640099bc0 con 0x7f66580cf280 2024-09-16T07:31:18.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.868+0000 7f6656a00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6634079930 0x7f663407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:18.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.869+0000 7f6656a00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6634079930 0x7f663407bdf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6648036000 tx=0x7f66480046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:18.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:18.872+0000 7f664f400640 1 -- 172.21.15.71:0/391539386 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6640063940 con 0x7f66580cf280 2024-09-16T07:31:19.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.057+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f665806fea0 con 0x7f66580cf280 2024-09-16T07:31:19.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.057+0000 7f664f400640 1 -- 172.21.15.71:0/391539386 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f6640063240 con 0x7f66580cf280 2024-09-16T07:31:19.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6634079930 msgr2=0x7f663407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6634079930 0x7f663407bdf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6648036000 tx=0x7f66480046c0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580cf280 msgr2=0x7f66580adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580cf280 0x7f66580adaf0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f664000e5c0 tx=0x7f664000ea80 comp rx=0 tx=0).stop 2024-09-16T07:31:19.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 shutdown_connections 2024-09-16T07:31:19.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6634079930 0x7f663407bdf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66581f4b20 0x7f6658157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66580cf280 0x7f66580adaf0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 --2- 172.21.15.71:0/391539386 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f663c003a00 0x7f66581f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 >> 172.21.15.71:0/391539386 conn(0x7f66580cf9f0 msgr2=0x7f66580b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:19.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 shutdown_connections 2024-09-16T07:31:19.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.060+0000 7f6657e00640 1 -- 172.21.15.71:0/391539386 wait complete. 2024-09-16T07:31:19.077 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:31:19.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.242+0000 7f1e18800640 1 -- 172.21.15.71:0/1427245277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e1006efa0 msgr2=0x7f1e10073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.242+0000 7f1e18800640 1 --2- 172.21.15.71:0/1427245277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e1006efa0 0x7f1e10073440 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009be0 tx=0x7f1e0c02a890 comp rx=0 tx=0).stop 2024-09-16T07:31:19.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.242+0000 7f1e18800640 1 -- 172.21.15.71:0/1427245277 shutdown_connections 2024-09-16T07:31:19.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.242+0000 7f1e18800640 1 --2- 172.21.15.71:0/1427245277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e1006efa0 0x7f1e10073440 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.242+0000 7f1e18800640 1 -- 172.21.15.71:0/1427245277 >> 172.21.15.71:0/1427245277 conn(0x7f1e100b1840 msgr2=0x7f1e100b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:19.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.242+0000 7f1e18800640 1 -- 172.21.15.71:0/1427245277 shutdown_connections 2024-09-16T07:31:19.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.242+0000 7f1e18800640 1 -- 172.21.15.71:0/1427245277 wait complete. 2024-09-16T07:31:19.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.243+0000 7f1e18800640 1 Processor -- start 2024-09-16T07:31:19.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.244+0000 7f1e18800640 1 -- start start 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.244+0000 7f1e18800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e1006efa0 0x7f1e10142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.244+0000 7f1e18800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e100cccd0 0x7f1e100b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.244+0000 7f1e18800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e10145750 0x7f1e101f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.244+0000 7f1e18800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e100ce700 con 0x7f1e10145750 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e18800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1e100ce580 con 0x7f1e100cccd0 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e18800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1e100ce880 con 0x7f1e1006efa0 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e16800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e10145750 0x7f1e101f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e15e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e1006efa0 0x7f1e10142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e16800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e10145750 0x7f1e101f8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47838/0 (socket says 172.21.15.71:47838) 2024-09-16T07:31:19.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e16800640 1 -- 172.21.15.71:0/3914586156 learned_addr learned my addr 172.21.15.71:0/3914586156 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:19.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e15400640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e100cccd0 0x7f1e100b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:19.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e16800640 1 -- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e1006efa0 msgr2=0x7f1e10142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e16800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e1006efa0 0x7f1e10142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e16800640 1 -- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e100cccd0 msgr2=0x7f1e100b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e16800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e100cccd0 0x7f1e100b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.245+0000 7f1e16800640 1 -- 172.21.15.71:0/3914586156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e100b3470 con 0x7f1e10145750 2024-09-16T07:31:19.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.246+0000 7f1e16800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e10145750 0x7f1e101f8180 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c006010 tx=0x7f1e0c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:19.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.246+0000 7f1e0a000640 1 -- 172.21.15.71:0/3914586156 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c031d80 con 0x7f1e10145750 2024-09-16T07:31:19.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.246+0000 7f1e0a000640 1 -- 172.21.15.71:0/3914586156 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e0c004130 con 0x7f1e10145750 2024-09-16T07:31:19.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.246+0000 7f1e0a000640 1 -- 172.21.15.71:0/3914586156 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c004560 con 0x7f1e10145750 2024-09-16T07:31:19.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.246+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e100b3750 con 0x7f1e10145750 2024-09-16T07:31:19.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.246+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e100b3d10 con 0x7f1e10145750 2024-09-16T07:31:19.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.247+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e100ced30 con 0x7f1e10145750 2024-09-16T07:31:19.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.248+0000 7f1e0a000640 1 -- 172.21.15.71:0/3914586156 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f1e0c039040 con 0x7f1e10145750 2024-09-16T07:31:19.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.249+0000 7f1e0a000640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1df4079930 0x7f1df407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:19.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.249+0000 7f1e0a000640 1 -- 172.21.15.71:0/3914586156 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f1e0c0ba990 con 0x7f1e10145750 2024-09-16T07:31:19.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.250+0000 7f1e15e00640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1df4079930 0x7f1df407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:19.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.250+0000 7f1e15e00640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1df4079930 0x7f1df407bdf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1e040046b0 tx=0x7f1e04007e90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:19.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.253+0000 7f1e0a000640 1 -- 172.21.15.71:0/3914586156 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1e0c040030 con 0x7f1e10145750 2024-09-16T07:31:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.440+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f1e100b1fa0 con 0x7f1e10145750 2024-09-16T07:31:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.441+0000 7f1e0a000640 1 -- 172.21.15.71:0/3914586156 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v41) ==== 116+0+2 (secure 0 0 0) 0x7f1e0c0840c0 con 0x7f1e10145750 2024-09-16T07:31:19.441 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1df4079930 msgr2=0x7f1df407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1df4079930 0x7f1df407bdf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1e040046b0 tx=0x7f1e04007e90 comp rx=0 tx=0).stop 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e10145750 msgr2=0x7f1e101f8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e10145750 0x7f1e101f8180 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c006010 tx=0x7f1e0c003f10 comp rx=0 tx=0).stop 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 shutdown_connections 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1df4079930 0x7f1df407bdf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e10145750 0x7f1e101f8180 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e100cccd0 0x7f1e100b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 --2- 172.21.15.71:0/3914586156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e1006efa0 0x7f1e10142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 >> 172.21.15.71:0/3914586156 conn(0x7f1e100b1840 msgr2=0x7f1e100ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 shutdown_connections 2024-09-16T07:31:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.443+0000 7f1e18800640 1 -- 172.21.15.71:0/3914586156 wait complete. 2024-09-16T07:31:19.459 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:31:19.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.620+0000 7f69e2e00640 1 -- 172.21.15.71:0/3227570003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 msgr2=0x7f69c0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.620+0000 7f69e2e00640 1 --2- 172.21.15.71:0/3227570003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 0x7f69c0003e50 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00c610 tx=0x7f69c0005830 comp rx=0 tx=0).stop 2024-09-16T07:31:19.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.620+0000 7f69e0e00640 1 -- 172.21.15.71:0/3227570003 reap_dead start 2024-09-16T07:31:19.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.620+0000 7f69e2e00640 1 -- 172.21.15.71:0/3227570003 shutdown_connections 2024-09-16T07:31:19.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.620+0000 7f69e2e00640 1 -- 172.21.15.71:0/3227570003 >> 172.21.15.71:0/3227570003 conn(0x7f69dc0b1840 msgr2=0x7f69dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:19.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.620+0000 7f69e2e00640 1 -- 172.21.15.71:0/3227570003 shutdown_connections 2024-09-16T07:31:19.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.620+0000 7f69e2e00640 1 -- 172.21.15.71:0/3227570003 wait complete. 2024-09-16T07:31:19.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.622+0000 7f69e2e00640 1 Processor -- start 2024-09-16T07:31:19.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.622+0000 7f69e2e00640 1 -- start start 2024-09-16T07:31:19.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69e2e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 0x7f69dc1f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:19.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69e2e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69dc1438f0 0x7f69dc0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:19.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69e2e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69dc143d50 0x7f69dc0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:19.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69e2e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc0ce2c0 con 0x7f69c0003a30 2024-09-16T07:31:19.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69e2e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc0736a0 con 0x7f69dc1438f0 2024-09-16T07:31:19.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69e2e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f69dc0ce440 con 0x7f69dc143d50 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 0x7f69dc1f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 0x7f69dc1f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47866/0 (socket says 172.21.15.71:47866) 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69dbe00640 1 -- 172.21.15.71:0/2102379590 learned_addr learned my addr 172.21.15.71:0/2102379590 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69e0e00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69dc143d50 0x7f69dc0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.623+0000 7f69db400640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69dc1438f0 0x7f69dc0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69dbe00640 1 -- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69dc143d50 msgr2=0x7f69dc0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69dbe00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69dc143d50 0x7f69dc0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69dbe00640 1 -- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69dc1438f0 msgr2=0x7f69dc0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69dbe00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69dc1438f0 0x7f69dc0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69dbe00640 1 -- 172.21.15.71:0/2102379590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc1f5710 con 0x7f69c0003a30 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69db400640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69dc1438f0 0x7f69dc0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:31:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69dbe00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 0x7f69dc1f8710 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f69cc02b4d0 tx=0x7f69cc004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:19.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69d3e00640 1 -- 172.21.15.71:0/2102379590 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc039070 con 0x7f69c0003a30 2024-09-16T07:31:19.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69d3e00640 1 -- 172.21.15.71:0/2102379590 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69cc02bce0 con 0x7f69c0003a30 2024-09-16T07:31:19.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc1f8c50 con 0x7f69c0003a30 2024-09-16T07:31:19.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69d3e00640 1 -- 172.21.15.71:0/2102379590 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0062d0 con 0x7f69c0003a30 2024-09-16T07:31:19.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.624+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc1f4e70 con 0x7f69c0003a30 2024-09-16T07:31:19.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.625+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc0ce9d0 con 0x7f69c0003a30 2024-09-16T07:31:19.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.627+0000 7f69d3e00640 1 -- 172.21.15.71:0/2102379590 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f69cc006470 con 0x7f69c0003a30 2024-09-16T07:31:19.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.627+0000 7f69d3e00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f69bc079930 0x7f69bc07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:19.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.628+0000 7f69d3e00640 1 -- 172.21.15.71:0/2102379590 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f69cc0b9b10 con 0x7f69c0003a30 2024-09-16T07:31:19.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.628+0000 7f69db400640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f69bc079930 0x7f69bc07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:19.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.629+0000 7f69db400640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f69bc079930 0x7f69bc07bdf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f69dc1404a0 tx=0x7f69c8006b70 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:19.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.632+0000 7f69d3e00640 1 -- 172.21.15.71:0/2102379590 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f69cc004e20 con 0x7f69c0003a30 2024-09-16T07:31:19.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.817+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f69dc0ae7b0 con 0x7f69c0003a30 2024-09-16T07:31:19.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.821+0000 7f69d3e00640 1 -- 172.21.15.71:0/2102379590 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v42)=0 v42) ==== 131+0+0 (secure 0 0 0) 0x7f69cc083c90 con 0x7f69c0003a30 2024-09-16T07:31:19.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f69bc079930 msgr2=0x7f69bc07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f69bc079930 0x7f69bc07bdf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f69dc1404a0 tx=0x7f69c8006b70 comp rx=0 tx=0).stop 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 msgr2=0x7f69dc1f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 0x7f69dc1f8710 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f69cc02b4d0 tx=0x7f69cc004c30 comp rx=0 tx=0).stop 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 shutdown_connections 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f69bc079930 0x7f69bc07bdf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69dc143d50 0x7f69dc0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69dc1438f0 0x7f69dc0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 --2- 172.21.15.71:0/2102379590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c0003a30 0x7f69dc1f8710 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 >> 172.21.15.71:0/2102379590 conn(0x7f69dc0b1840 msgr2=0x7f69dc06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 shutdown_connections 2024-09-16T07:31:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:19.824+0000 7f69e2e00640 1 -- 172.21.15.71:0/2102379590 wait complete. 2024-09-16T07:31:19.841 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-16T07:31:19.842 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:19.842+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:19.842 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:19.842+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:19.842 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:19.842+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:19.842 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:19.842+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:19.842 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:19.842+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:20.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.002+0000 7f7aa7a00640 1 -- 172.21.15.71:0/4079353856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 msgr2=0x7f7aa00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.002+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/4079353856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 0x7f7aa00cdbb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005fd0 tx=0x7f7a9802aa10 comp rx=0 tx=0).stop 2024-09-16T07:31:20.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.002+0000 7f7aa7a00640 1 -- 172.21.15.71:0/4079353856 shutdown_connections 2024-09-16T07:31:20.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.002+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/4079353856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 0x7f7aa00cdbb0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.002+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/4079353856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7aa00cccd0 0x7f7aa0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.002+0000 7f7aa7a00640 1 -- 172.21.15.71:0/4079353856 >> 172.21.15.71:0/4079353856 conn(0x7f7aa00b1840 msgr2=0x7f7aa00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:20.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.002+0000 7f7aa7a00640 1 -- 172.21.15.71:0/4079353856 shutdown_connections 2024-09-16T07:31:20.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.002+0000 7f7aa7a00640 1 -- 172.21.15.71:0/4079353856 wait complete. 2024-09-16T07:31:20.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.004+0000 7f7aa7a00640 1 Processor -- start 2024-09-16T07:31:20.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.004+0000 7f7aa7a00640 1 -- start start 2024-09-16T07:31:20.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa7a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 0x7f7aa013bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa7a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aa00cccd0 0x7f7aa00b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa7a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7aa0145750 0x7f7aa01f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa7a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa00ce480 con 0x7f7aa006efa0 2024-09-16T07:31:20.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa7a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa00ce300 con 0x7f7aa00cccd0 2024-09-16T07:31:20.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa7a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa00ce600 con 0x7f7aa0145750 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 0x7f7aa013bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 0x7f7aa013bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47886/0 (socket says 172.21.15.71:47886) 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.005+0000 7f7aa5000640 1 -- 172.21.15.71:0/1639655689 learned_addr learned my addr 172.21.15.71:0/1639655689 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7aa5000640 1 -- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7aa0145750 msgr2=0x7f7aa01f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7aa5000640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7aa0145750 0x7f7aa01f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7aa5000640 1 -- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aa00cccd0 msgr2=0x7f7aa00b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7aa5000640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aa00cccd0 0x7f7aa00b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7aa5000640 1 -- 172.21.15.71:0/1639655689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa013fc60 con 0x7f7aa006efa0 2024-09-16T07:31:20.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7aa5000640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 0x7f7aa013bd90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c3a0 tx=0x7f7a9000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:20.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7a9cc00640 1 -- 172.21.15.71:0/1639655689 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90017070 con 0x7f7aa006efa0 2024-09-16T07:31:20.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7a9cc00640 1 -- 172.21.15.71:0/1639655689 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a900053c0 con 0x7f7aa006efa0 2024-09-16T07:31:20.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa013ff40 con 0x7f7aa006efa0 2024-09-16T07:31:20.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7a9cc00640 1 -- 172.21.15.71:0/1639655689 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90003850 con 0x7f7aa006efa0 2024-09-16T07:31:20.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.006+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa00b3a40 con 0x7f7aa006efa0 2024-09-16T07:31:20.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.007+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa00b3c40 con 0x7f7aa006efa0 2024-09-16T07:31:20.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.009+0000 7f7a9cc00640 1 -- 172.21.15.71:0/1639655689 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f7a9001a070 con 0x7f7aa006efa0 2024-09-16T07:31:20.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.009+0000 7f7a9cc00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a80079960 0x7f7a8007be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.010+0000 7f7a9cc00640 1 -- 172.21.15.71:0/1639655689 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f7a90098ba0 con 0x7f7aa006efa0 2024-09-16T07:31:20.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.010+0000 7f7a9fe00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a80079960 0x7f7a8007be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:20.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.011+0000 7f7a9fe00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a80079960 0x7f7a8007be20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c005fd0 tx=0x7f7a8c005f40 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:20.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.014+0000 7f7a9cc00640 1 -- 172.21.15.71:0/1639655689 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7aa00b3c40 con 0x7f7aa006efa0 2024-09-16T07:31:20.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.197+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f7aa00cdbb0 con 0x7f7aa006efa0 2024-09-16T07:31:20.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.201+0000 7f7a9cc00640 1 -- 172.21.15.71:0/1639655689 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v43)=0 v43) ==== 136+0+0 (secure 0 0 0) 0x7f7a90062920 con 0x7f7aa006efa0 2024-09-16T07:31:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.203+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a80079960 msgr2=0x7f7a8007be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a80079960 0x7f7a8007be20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c005fd0 tx=0x7f7a8c005f40 comp rx=0 tx=0).stop 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 msgr2=0x7f7aa013bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 0x7f7aa013bd90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c3a0 tx=0x7f7a9000c860 comp rx=0 tx=0).stop 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 shutdown_connections 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7a80079960 0x7f7a8007be20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7aa0145750 0x7f7aa01f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aa00cccd0 0x7f7aa00b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 --2- 172.21.15.71:0/1639655689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7aa006efa0 0x7f7aa013bd90 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 >> 172.21.15.71:0/1639655689 conn(0x7f7aa00b1840 msgr2=0x7f7aa00af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 shutdown_connections 2024-09-16T07:31:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.204+0000 7f7aa7a00640 1 -- 172.21.15.71:0/1639655689 wait complete. 2024-09-16T07:31:20.219 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T07:31:20.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.382+0000 7f395be00640 1 -- 172.21.15.71:0/3752547219 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f395c06e590 msgr2=0x7f395c06fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.382+0000 7f395be00640 1 --2- 172.21.15.71:0/3752547219 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f395c06e590 0x7f395c06fe40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3944005fd0 tx=0x7f394402a5d0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.382+0000 7f395be00640 1 -- 172.21.15.71:0/3752547219 shutdown_connections 2024-09-16T07:31:20.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.382+0000 7f395be00640 1 --2- 172.21.15.71:0/3752547219 >> v2:172.21.15.177:3300/0 conn(0x7f395c070380 0x7f395c0cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.382+0000 7f395be00640 1 --2- 172.21.15.71:0/3752547219 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f395c06e590 0x7f395c06fe40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.382+0000 7f395be00640 1 -- 172.21.15.71:0/3752547219 >> 172.21.15.71:0/3752547219 conn(0x7f395c0cfa70 msgr2=0x7f395c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:20.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.382+0000 7f395be00640 1 -- 172.21.15.71:0/3752547219 shutdown_connections 2024-09-16T07:31:20.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.383+0000 7f395be00640 1 -- 172.21.15.71:0/3752547219 wait complete. 2024-09-16T07:31:20.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.384+0000 7f395be00640 1 Processor -- start 2024-09-16T07:31:20.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.384+0000 7f395be00640 1 -- start start 2024-09-16T07:31:20.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f395c06e590 0x7f395c15ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f395c070380 0x7f395c1f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f395c15bda0 0x7f395c1f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f395c0cb910 con 0x7f395c06e590 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f395c0708f0 con 0x7f395c070380 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f395c0cba90 con 0x7f395c15bda0 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f395c06e590 0x7f395c15ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f395c06e590 0x7f395c15ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47888/0 (socket says 172.21.15.71:47888) 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.385+0000 7f395aa00640 1 -- 172.21.15.71:0/328376820 learned_addr learned my addr 172.21.15.71:0/328376820 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f3953e00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f395c070380 0x7f395c1f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f395b400640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f395c15bda0 0x7f395c1f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f395aa00640 1 -- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f395c15bda0 msgr2=0x7f395c1f8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f395aa00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f395c15bda0 0x7f395c1f8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f395aa00640 1 -- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f395c070380 msgr2=0x7f395c1f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f395aa00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f395c070380 0x7f395c1f7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f395aa00640 1 -- 172.21.15.71:0/328376820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395c0c7c80 con 0x7f395c06e590 2024-09-16T07:31:20.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f395aa00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f395c06e590 0x7f395c15ee70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f394c00c3a0 tx=0x7f394c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:20.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f3952a00640 1 -- 172.21.15.71:0/328376820 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c017070 con 0x7f395c06e590 2024-09-16T07:31:20.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f3952a00640 1 -- 172.21.15.71:0/328376820 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f394c0053c0 con 0x7f395c06e590 2024-09-16T07:31:20.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395c0c7f60 con 0x7f395c06e590 2024-09-16T07:31:20.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.386+0000 7f3952a00640 1 -- 172.21.15.71:0/328376820 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c003850 con 0x7f395c06e590 2024-09-16T07:31:20.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.387+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395c0c8520 con 0x7f395c06e590 2024-09-16T07:31:20.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.388+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395c0c8860 con 0x7f395c06e590 2024-09-16T07:31:20.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.389+0000 7f3952a00640 1 -- 172.21.15.71:0/328376820 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7f394c01a070 con 0x7f395c06e590 2024-09-16T07:31:20.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.390+0000 7f3952a00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3938079930 0x7f393807bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.390+0000 7f3952a00640 1 -- 172.21.15.71:0/328376820 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f394c098bb0 con 0x7f395c06e590 2024-09-16T07:31:20.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.390+0000 7f3953e00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3938079930 0x7f393807bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:20.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.391+0000 7f3953e00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3938079930 0x7f393807bdf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3944033000 tx=0x7f3944004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:20.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.394+0000 7f3952a00640 1 -- 172.21.15.71:0/328376820 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f394c062910 con 0x7f395c06e590 2024-09-16T07:31:20.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.604+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f395c14dc70 con 0x7f3938079930 2024-09-16T07:31:20.614 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:20.614+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:20.614 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:20.614+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:20.614 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:20.614+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:20.614 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:20.614+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:20.614 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:20.614+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:20.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.655+0000 7f3952a00640 1 -- 172.21.15.71:0/328376820 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f395c14dc70 con 0x7f3938079930 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3938079930 msgr2=0x7f393807bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3938079930 0x7f393807bdf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3944033000 tx=0x7f3944004730 comp rx=0 tx=0).stop 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f395c06e590 msgr2=0x7f395c15ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f395c06e590 0x7f395c15ee70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f394c00c3a0 tx=0x7f394c00c860 comp rx=0 tx=0).stop 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 shutdown_connections 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3938079930 0x7f393807bdf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f395c15bda0 0x7f395c1f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f395c070380 0x7f395c1f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 --2- 172.21.15.71:0/328376820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f395c06e590 0x7f395c15ee70 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 >> 172.21.15.71:0/328376820 conn(0x7f395c0cfa70 msgr2=0x7f395c0ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:20.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 shutdown_connections 2024-09-16T07:31:20.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.657+0000 7f395be00640 1 -- 172.21.15.71:0/328376820 wait complete. 2024-09-16T07:31:20.674 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:31:20.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.835+0000 7fbfbbe00640 1 -- 172.21.15.71:0/321531283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b4e70 msgr2=0x7fbfbc0b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.835+0000 7fbfbbe00640 1 --2- 172.21.15.71:0/321531283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b4e70 0x7fbfbc0b34c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009900 tx=0x7fbfa802a6a0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.835+0000 7fbfbbe00640 1 -- 172.21.15.71:0/321531283 shutdown_connections 2024-09-16T07:31:20.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.835+0000 7fbfbbe00640 1 --2- 172.21.15.71:0/321531283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b4e70 0x7fbfbc0b34c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.835+0000 7fbfbbe00640 1 -- 172.21.15.71:0/321531283 >> 172.21.15.71:0/321531283 conn(0x7fbfbc0d3e20 msgr2=0x7fbfbc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:20.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.836+0000 7fbfbbe00640 1 -- 172.21.15.71:0/321531283 shutdown_connections 2024-09-16T07:31:20.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.836+0000 7fbfbbe00640 1 -- 172.21.15.71:0/321531283 wait complete. 2024-09-16T07:31:20.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.837+0000 7fbfbbe00640 1 Processor -- start 2024-09-16T07:31:20.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.837+0000 7fbfbbe00640 1 -- start start 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.838+0000 7fbfbbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b3a00 0x7fbfbc0b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.838+0000 7fbfbbe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfbc0b4e70 0x7fbfbc0b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.838+0000 7fbfbbe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfbc0d2000 0x7fbfbc0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.838+0000 7fbfbbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc0b43a0 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.838+0000 7fbfbbe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc0b4220 con 0x7fbfbc0b4e70 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.838+0000 7fbfbbe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbfbc0b1d90 con 0x7fbfbc0d2000 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.838+0000 7fbfbaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b3a00 0x7fbfbc0b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.838+0000 7fbfbaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b3a00 0x7fbfbc0b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47914/0 (socket says 172.21.15.71:47914) 2024-09-16T07:31:20.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfbaa00640 1 -- 172.21.15.71:0/1903850586 learned_addr learned my addr 172.21.15.71:0/1903850586 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfbaa00640 1 -- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfbc0d2000 msgr2=0x7fbfbc0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfbaa00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfbc0d2000 0x7fbfbc0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfbaa00640 1 -- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfbc0b4e70 msgr2=0x7fbfbc0b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfbaa00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfbc0b4e70 0x7fbfbc0b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfbaa00640 1 -- 172.21.15.71:0/1903850586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc1f8760 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfbaa00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b3a00 0x7fbfbc0b8c60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00ba50 tx=0x7fbfac00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfb2a00640 1 -- 172.21.15.71:0/1903850586 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac019070 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfb2a00640 1 -- 172.21.15.71:0/1903850586 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfac00cd00 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc1f8a20 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.839+0000 7fbfb2a00640 1 -- 172.21.15.71:0/1903850586 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0146a0 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.840+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc1f8fe0 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.841+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfbc0b34c0 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.842+0000 7fbfb2a00640 1 -- 172.21.15.71:0/1903850586 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) ==== 102817+0+0 (secure 0 0 0) 0x7fbfac01c070 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.843+0000 7fbfb2a00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbf9c079930 0x7fbf9c07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:20.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.843+0000 7fbfb2a00640 1 -- 172.21.15.71:0/1903850586 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fbfac0992c0 con 0x7fbfbc0b3a00 2024-09-16T07:31:20.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.843+0000 7fbfba000640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbf9c079930 0x7fbf9c07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:20.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.844+0000 7fbfba000640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbf9c079930 0x7fbf9c07bdf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8033000 tx=0x7fbfa80046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:20.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:20.847+0000 7fbfb2a00640 1 -- 172.21.15.71:0/1903850586 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbfac063380 con 0x7fbfbc0b3a00 2024-09-16T07:31:21.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.057+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fbfbc0760b0 con 0x7fbf9c079930 2024-09-16T07:31:21.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.063+0000 7fbfb2a00640 1 -- 172.21.15.71:0/1903850586 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fbfbc0760b0 con 0x7fbf9c079930 2024-09-16T07:31:21.064 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32 2024-09-16T07:31:21.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbf9c079930 msgr2=0x7fbf9c07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:21.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbf9c079930 0x7fbf9c07bdf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8033000 tx=0x7fbfa80046c0 comp rx=0 tx=0).stop 2024-09-16T07:31:21.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b3a00 msgr2=0x7fbfbc0b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b3a00 0x7fbfbc0b8c60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00ba50 tx=0x7fbfac00bf10 comp rx=0 tx=0).stop 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 shutdown_connections 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbf9c079930 0x7fbf9c07bdf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfbc0d2000 0x7fbfbc0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfbc0b4e70 0x7fbfbc0b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 --2- 172.21.15.71:0/1903850586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfbc0b3a00 0x7fbfbc0b8c60 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 >> 172.21.15.71:0/1903850586 conn(0x7fbfbc0d3e20 msgr2=0x7fbfbc161670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 shutdown_connections 2024-09-16T07:31:21.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:21.066+0000 7fbfbbe00640 1 -- 172.21.15.71:0/1903850586 wait complete. 2024-09-16T07:31:21.081 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32 2024-09-16T07:31:21.081 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:31:21.081 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:31:21.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:21.117 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:31:28.438 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:31:28.438 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:31:28.438 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.23909 s, 148 MB/s 2024-09-16T07:31:28.440 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:31:28.440 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:31:28.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:28.475 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:31:35.778 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:31:35.779 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:31:35.779 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.22578 s, 149 MB/s 2024-09-16T07:31:35.780 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:31:35.780 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:31:35.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:35.816 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:31:43.216 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:31:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:31:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.3235 s, 147 MB/s 2024-09-16T07:31:43.219 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T07:31:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.382+0000 7f770be00640 1 -- 172.21.15.71:0/2690047096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 msgr2=0x7f770c0d4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.382+0000 7f770be00640 1 --2- 172.21.15.71:0/2690047096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 0x7f770c0d4790 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f76fc005fd0 tx=0x7f76fc02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.383+0000 7f770be00640 1 -- 172.21.15.71:0/2690047096 shutdown_connections 2024-09-16T07:31:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.383+0000 7f770be00640 1 --2- 172.21.15.71:0/2690047096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 0x7f770c0d4790 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.383+0000 7f770be00640 1 -- 172.21.15.71:0/2690047096 >> 172.21.15.71:0/2690047096 conn(0x7f770c09d180 msgr2=0x7f770c0d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.383+0000 7f770be00640 1 -- 172.21.15.71:0/2690047096 shutdown_connections 2024-09-16T07:31:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.383+0000 7f770be00640 1 -- 172.21.15.71:0/2690047096 wait complete. 2024-09-16T07:31:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.384+0000 7f770be00640 1 Processor -- start 2024-09-16T07:31:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.384+0000 7f770be00640 1 -- start start 2024-09-16T07:31:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.385+0000 7f770be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f770c0cf470 0x7f770c0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.385+0000 7f770be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f770c0d0500 0x7f770c0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.385+0000 7f770be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 0x7f770c1f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.385+0000 7f770be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0d2e30 con 0x7f770c15dfb0 2024-09-16T07:31:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.385+0000 7f770be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0b7630 con 0x7f770c0cf470 2024-09-16T07:31:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.385+0000 7f770be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f770c0d0990 con 0x7f770c0d0500 2024-09-16T07:31:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.385+0000 7f770b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 0x7f770c1f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 0x7f770c1f5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53352/0 (socket says 172.21.15.71:53352) 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770b400640 1 -- 172.21.15.71:0/2393196516 learned_addr learned my addr 172.21.15.71:0/2393196516 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770aa00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f770c0cf470 0x7f770c0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770a000640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f770c0d0500 0x7f770c0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770b400640 1 -- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f770c0d0500 msgr2=0x7f770c0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770b400640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f770c0d0500 0x7f770c0c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770b400640 1 -- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f770c0cf470 msgr2=0x7f770c0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770b400640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f770c0cf470 0x7f770c0c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770b400640 1 -- 172.21.15.71:0/2393196516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c0c9c10 con 0x7f770c15dfb0 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770b400640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 0x7f770c1f5720 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f770400c7a0 tx=0x7f770400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f7702a00640 1 -- 172.21.15.71:0/2393196516 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704003b70 con 0x7f770c15dfb0 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f7702a00640 1 -- 172.21.15.71:0/2393196516 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7704003d30 con 0x7f770c15dfb0 2024-09-16T07:31:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c0c7b00 con 0x7f770c15dfb0 2024-09-16T07:31:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.386+0000 7f7702a00640 1 -- 172.21.15.71:0/2393196516 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400aab0 con 0x7f770c15dfb0 2024-09-16T07:31:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.387+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c0c80c0 con 0x7f770c15dfb0 2024-09-16T07:31:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.388+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c0d1330 con 0x7f770c15dfb0 2024-09-16T07:31:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.389+0000 7f7702a00640 1 -- 172.21.15.71:0/2393196516 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) ==== 102919+0+0 (secure 0 0 0) 0x7f770400ac50 con 0x7f770c15dfb0 2024-09-16T07:31:43.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.389+0000 7f7702a00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f76ec079a50 0x7f76ec07bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:43.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.390+0000 7f7702a00640 1 -- 172.21.15.71:0/2393196516 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f77040995a0 con 0x7f770c15dfb0 2024-09-16T07:31:43.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.390+0000 7f770aa00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f76ec079a50 0x7f76ec07bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.391+0000 7f770aa00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f76ec079a50 0x7f76ec07bf10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f76fc033000 tx=0x7f76fc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.394+0000 7f7702a00640 1 -- 172.21.15.71:0/2393196516 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f770c0d1330 con 0x7f770c15dfb0 2024-09-16T07:31:43.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.607+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f770c08df30 con 0x7f76ec079a50 2024-09-16T07:31:43.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.620+0000 7f7702a00640 1 -- 172.21.15.71:0/2393196516 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f770c08df30 con 0x7f76ec079a50 2024-09-16T07:31:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f76ec079a50 msgr2=0x7f76ec07bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f76ec079a50 0x7f76ec07bf10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f76fc033000 tx=0x7f76fc004730 comp rx=0 tx=0).stop 2024-09-16T07:31:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 msgr2=0x7f770c1f5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 0x7f770c1f5720 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f770400c7a0 tx=0x7f770400cc60 comp rx=0 tx=0).stop 2024-09-16T07:31:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 shutdown_connections 2024-09-16T07:31:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f76ec079a50 0x7f76ec07bf10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f770c15dfb0 0x7f770c1f5720 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f770c0d0500 0x7f770c0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 --2- 172.21.15.71:0/2393196516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f770c0cf470 0x7f770c0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.622+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 >> 172.21.15.71:0/2393196516 conn(0x7f770c09d180 msgr2=0x7f770c075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.623+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 shutdown_connections 2024-09-16T07:31:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.623+0000 7f770be00640 1 -- 172.21.15.71:0/2393196516 wait complete. 2024-09-16T07:31:43.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:31:43.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.799+0000 7fe3daa00640 1 -- 172.21.15.71:0/3399316833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3d4161760 msgr2=0x7fe3d40ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:43.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.799+0000 7fe3daa00640 1 --2- 172.21.15.71:0/3399316833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3d4161760 0x7fe3d40ce4a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009be0 tx=0x7fe3c402a890 comp rx=0 tx=0).stop 2024-09-16T07:31:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.799+0000 7fe3daa00640 1 -- 172.21.15.71:0/3399316833 shutdown_connections 2024-09-16T07:31:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.799+0000 7fe3daa00640 1 --2- 172.21.15.71:0/3399316833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3d4161760 0x7fe3d40ce4a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.799+0000 7fe3daa00640 1 -- 172.21.15.71:0/3399316833 >> 172.21.15.71:0/3399316833 conn(0x7fe3d40b1840 msgr2=0x7fe3d40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.799+0000 7fe3daa00640 1 -- 172.21.15.71:0/3399316833 shutdown_connections 2024-09-16T07:31:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.799+0000 7fe3daa00640 1 -- 172.21.15.71:0/3399316833 wait complete. 2024-09-16T07:31:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.800+0000 7fe3daa00640 1 Processor -- start 2024-09-16T07:31:43.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.800+0000 7fe3daa00640 1 -- start start 2024-09-16T07:31:43.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.801+0000 7fe3daa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3d40af190 0x7fe3d41528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.801+0000 7fe3daa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3d40cec60 0x7fe3d40af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.801+0000 7fe3daa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3d4161760 0x7fe3d40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.801+0000 7fe3daa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4162c30 con 0x7fe3d40cec60 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.801+0000 7fe3daa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4162ab0 con 0x7fe3d40af190 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.801+0000 7fe3daa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d415de50 con 0x7fe3d4161760 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3d40cec60 0x7fe3d40af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3d40cec60 0x7fe3d40af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53356/0 (socket says 172.21.15.71:53356) 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 -- 172.21.15.71:0/1138824358 learned_addr learned my addr 172.21.15.71:0/1138824358 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3e00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3d40af190 0x7fe3d41528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d8a00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3d4161760 0x7fe3d40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 -- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3d4161760 msgr2=0x7fe3d40afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3d4161760 0x7fe3d40afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 -- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3d40af190 msgr2=0x7fe3d41528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3d40af190 0x7fe3d41528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 -- 172.21.15.71:0/1138824358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d41f5af0 con 0x7fe3d40cec60 2024-09-16T07:31:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.802+0000 7fe3d3400640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3d40cec60 0x7fe3d40af7f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c490 tx=0x7fe3c000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.803+0000 7fe3cbe00640 1 -- 172.21.15.71:0/1138824358 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0016020 con 0x7fe3d40cec60 2024-09-16T07:31:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.803+0000 7fe3cbe00640 1 -- 172.21.15.71:0/1138824358 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3c0012940 con 0x7fe3d40cec60 2024-09-16T07:31:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.803+0000 7fe3cbe00640 1 -- 172.21.15.71:0/1138824358 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0012d90 con 0x7fe3d40cec60 2024-09-16T07:31:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.803+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d4158360 con 0x7fe3d40cec60 2024-09-16T07:31:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.803+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d4152df0 con 0x7fe3d40cec60 2024-09-16T07:31:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.804+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d4158880 con 0x7fe3d40cec60 2024-09-16T07:31:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.805+0000 7fe3cbe00640 1 -- 172.21.15.71:0/1138824358 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) ==== 102919+0+0 (secure 0 0 0) 0x7fe3c0019020 con 0x7fe3d40cec60 2024-09-16T07:31:43.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.806+0000 7fe3cbe00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3b4079a50 0x7fe3b407bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.806+0000 7fe3cbe00640 1 -- 172.21.15.71:0/1138824358 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fe3c00994a0 con 0x7fe3d40cec60 2024-09-16T07:31:43.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.806+0000 7fe3d3e00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3b4079a50 0x7fe3b407bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:43.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.807+0000 7fe3d3e00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3b4079a50 0x7fe3b407bf10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4034000 tx=0x7fe3c4004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:43.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:43.810+0000 7fe3cbe00640 1 -- 172.21.15.71:0/1138824358 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe3d4158880 con 0x7fe3d40cec60 2024-09-16T07:31:44.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.021+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe3d40cf340 con 0x7fe3b4079a50 2024-09-16T07:31:44.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.027+0000 7fe3cbe00640 1 -- 172.21.15.71:0/1138824358 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fe3d40cf340 con 0x7fe3b4079a50 2024-09-16T07:31:44.027 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32 2024-09-16T07:31:44.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3b4079a50 msgr2=0x7fe3b407bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:44.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3b4079a50 0x7fe3b407bf10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4034000 tx=0x7fe3c4004ab0 comp rx=0 tx=0).stop 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3d40cec60 msgr2=0x7fe3d40af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3d40cec60 0x7fe3d40af7f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c490 tx=0x7fe3c000c950 comp rx=0 tx=0).stop 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 shutdown_connections 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3b4079a50 0x7fe3b407bf10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3d4161760 0x7fe3d40afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3d40cec60 0x7fe3d40af7f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 --2- 172.21.15.71:0/1138824358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3d40af190 0x7fe3d41528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 >> 172.21.15.71:0/1138824358 conn(0x7fe3d40b1840 msgr2=0x7fe3d4072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 shutdown_connections 2024-09-16T07:31:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:44.029+0000 7fe3daa00640 1 -- 172.21.15.71:0/1138824358 wait complete. 2024-09-16T07:31:44.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:44.045 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32 2024-09-16T07:31:44.045 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:31:44.081 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32 2024-09-16T07:31:44.081 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="3055190016" 2024-09-16T07:31:44.081 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:44.082 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-16T07:31:45.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:31:45.426 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:31:46.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:31:46.083 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32 2024-09-16T07:31:46.083 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:31:46.125 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/62ba5c24-f30b-4fe4-b3d5-e0b1657b5a32 2024-09-16T07:31:46.125 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T07:31:46.125 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:46.126 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T07:31:46.126 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone98650 2024-09-16T07:31:46.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.332+0000 7fc544400640 1 -- 172.21.15.71:0/2853886393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c0cd490 msgr2=0x7fc53c0732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.332+0000 7fc544400640 1 --2- 172.21.15.71:0/2853886393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c0cd490 0x7fc53c0732c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc53c06db10 tx=0x7fc52802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.333+0000 7fc544400640 1 -- 172.21.15.71:0/2853886393 shutdown_connections 2024-09-16T07:31:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.333+0000 7fc544400640 1 --2- 172.21.15.71:0/2853886393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c0cd490 0x7fc53c0732c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.333+0000 7fc544400640 1 --2- 172.21.15.71:0/2853886393 >> v2:172.21.15.177:3300/0 conn(0x7fc53c1517a0 0x7fc53c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.333+0000 7fc544400640 1 -- 172.21.15.71:0/2853886393 >> 172.21.15.71:0/2853886393 conn(0x7fc53c0b1840 msgr2=0x7fc53c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.333+0000 7fc544400640 1 -- 172.21.15.71:0/2853886393 shutdown_connections 2024-09-16T07:31:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.333+0000 7fc544400640 1 -- 172.21.15.71:0/2853886393 wait complete. 2024-09-16T07:31:46.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.334+0000 7fc544400640 1 Processor -- start 2024-09-16T07:31:46.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.334+0000 7fc544400640 1 -- start start 2024-09-16T07:31:46.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.335+0000 7fc544400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c073800 0x7fc53c14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:46.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.335+0000 7fc544400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc53c0cd490 0x7fc53c06f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:46.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.335+0000 7fc544400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c1517a0 0x7fc53c1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:46.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.335+0000 7fc544400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c074040 con 0x7fc53c1517a0 2024-09-16T07:31:46.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.335+0000 7fc544400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c073ec0 con 0x7fc53c073800 2024-09-16T07:31:46.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.335+0000 7fc544400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c0cc650 con 0x7fc53c0cd490 2024-09-16T07:31:46.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc541a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c073800 0x7fc53c14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:46.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc542400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c1517a0 0x7fc53c1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:46.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc541000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc53c0cd490 0x7fc53c06f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:46.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc541a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c073800 0x7fc53c14ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:52992/0 (socket says 172.21.15.71:52992) 2024-09-16T07:31:46.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc541a00640 1 -- 172.21.15.71:0/763905732 learned_addr learned my addr 172.21.15.71:0/763905732 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:46.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc542400640 1 -- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc53c0cd490 msgr2=0x7fc53c06f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:46.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc542400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc53c0cd490 0x7fc53c06f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc542400640 1 -- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c073800 msgr2=0x7fc53c14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:46.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc542400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c073800 0x7fc53c14ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc542400640 1 -- 172.21.15.71:0/763905732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c06f100 con 0x7fc53c1517a0 2024-09-16T07:31:46.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.336+0000 7fc542400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c1517a0 0x7fc53c1551c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c3c0 tx=0x7fc53400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:46.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.337+0000 7fc539600640 1 -- 172.21.15.71:0/763905732 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534017070 con 0x7fc53c1517a0 2024-09-16T07:31:46.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.337+0000 7fc539600640 1 -- 172.21.15.71:0/763905732 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5340053c0 con 0x7fc53c1517a0 2024-09-16T07:31:46.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.337+0000 7fc539600640 1 -- 172.21.15.71:0/763905732 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534003850 con 0x7fc53c1517a0 2024-09-16T07:31:46.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.337+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c149fd0 con 0x7fc53c1517a0 2024-09-16T07:31:46.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.337+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c06f390 con 0x7fc53c1517a0 2024-09-16T07:31:46.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.338+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c0ccaa0 con 0x7fc53c1517a0 2024-09-16T07:31:46.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.339+0000 7fc539600640 1 -- 172.21.15.71:0/763905732 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) ==== 102919+0+0 (secure 0 0 0) 0x7fc53401a070 con 0x7fc53c1517a0 2024-09-16T07:31:46.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.340+0000 7fc539600640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc51c079a50 0x7fc51c07bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:46.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.340+0000 7fc539600640 1 -- 172.21.15.71:0/763905732 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fc534098b80 con 0x7fc53c1517a0 2024-09-16T07:31:46.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.340+0000 7fc541a00640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc51c079a50 0x7fc51c07bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:46.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.341+0000 7fc541a00640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc51c079a50 0x7fc51c07bf10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc530009fd0 tx=0x7fc530009760 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:46.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.344+0000 7fc539600640 1 -- 172.21.15.71:0/763905732 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc534062280 con 0x7fc53c1517a0 2024-09-16T07:31:46.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.561+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone98650", "target": ["mon-mgr", ""]}) -- 0x7fc53c0cd190 con 0x7fc51c079a50 2024-09-16T07:31:46.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.629+0000 7fc539600640 1 -- 172.21.15.71:0/763905732 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc53c0cd190 con 0x7fc51c079a50 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc51c079a50 msgr2=0x7fc51c07bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc51c079a50 0x7fc51c07bf10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc530009fd0 tx=0x7fc530009760 comp rx=0 tx=0).stop 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c1517a0 msgr2=0x7fc53c1551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c1517a0 0x7fc53c1551c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c3c0 tx=0x7fc53400c7c0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 shutdown_connections 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc51c079a50 0x7fc51c07bf10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc53c1517a0 0x7fc53c1551c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc53c0cd490 0x7fc53c06f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 --2- 172.21.15.71:0/763905732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c073800 0x7fc53c14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 >> 172.21.15.71:0/763905732 conn(0x7fc53c0b1840 msgr2=0x7fc53c14b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:46.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 shutdown_connections 2024-09-16T07:31:46.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.632+0000 7fc544400640 1 -- 172.21.15.71:0/763905732 wait complete. 2024-09-16T07:31:46.638 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.638+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.638 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.638+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.638 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.638+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.638 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.638+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.638 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.638+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.650 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone50521 2024-09-16T07:31:46.655 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.656+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.656 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.656+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.656 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.656+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.656 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.656+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.656 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:46.656+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 -- 172.21.15.71:0/391509324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40b6c50 msgr2=0x7fd3c40b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 --2- 172.21.15.71:0/391509324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40b6c50 0x7fd3c40b70e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40cff60 tx=0x7fd3bc02a980 comp rx=0 tx=0).stop 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 -- 172.21.15.71:0/391509324 shutdown_connections 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 --2- 172.21.15.71:0/391509324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40b6c50 0x7fd3c40b70e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 --2- 172.21.15.71:0/391509324 >> v2:172.21.15.177:3300/0 conn(0x7fd3c40cf110 0x7fd3c40d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 --2- 172.21.15.71:0/391509324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40ced30 0x7fd3c40b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 -- 172.21.15.71:0/391509324 >> 172.21.15.71:0/391509324 conn(0x7fd3c40d3e20 msgr2=0x7fd3c40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 -- 172.21.15.71:0/391509324 shutdown_connections 2024-09-16T07:31:46.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.813+0000 7fd3c3200640 1 -- 172.21.15.71:0/391509324 wait complete. 2024-09-16T07:31:46.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.815+0000 7fd3c3200640 1 Processor -- start 2024-09-16T07:31:46.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.815+0000 7fd3c3200640 1 -- start start 2024-09-16T07:31:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c3200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b6c50 0x7fd3c414e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c3200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40ced30 0x7fd3c41f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40cf110 0x7fd3c40c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c4161f20 con 0x7fd3c40cf110 2024-09-16T07:31:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c3200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c4161da0 con 0x7fd3c40b6c50 2024-09-16T07:31:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c3200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c41620a0 con 0x7fd3c40ced30 2024-09-16T07:31:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40cf110 0x7fd3c40c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:46.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40cf110 0x7fd3c40c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56802/0 (socket says 172.21.15.71:56802) 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c2800640 1 -- 172.21.15.71:0/269193613 learned_addr learned my addr 172.21.15.71:0/269193613 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c1e00640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b6c50 0x7fd3c414e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.816+0000 7fd3c1400640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40ced30 0x7fd3c41f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c2800640 1 -- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40ced30 msgr2=0x7fd3c41f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c2800640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40ced30 0x7fd3c41f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c2800640 1 -- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b6c50 msgr2=0x7fd3c414e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c2800640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b6c50 0x7fd3c414e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c2800640 1 -- 172.21.15.71:0/269193613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c41e7590 con 0x7fd3c40cf110 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c1e00640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b6c50 0x7fd3c414e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c1400640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40ced30 0x7fd3c41f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:31:46.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c2800640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40cf110 0x7fd3c40c9710 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc005fd0 tx=0x7fd3bc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:46.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3ba000640 1 -- 172.21.15.71:0/269193613 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc034050 con 0x7fd3c40cf110 2024-09-16T07:31:46.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3ba000640 1 -- 172.21.15.71:0/269193613 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3bc004450 con 0x7fd3c40cf110 2024-09-16T07:31:46.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3ba000640 1 -- 172.21.15.71:0/269193613 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc043440 con 0x7fd3c40cf110 2024-09-16T07:31:46.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c41f51e0 con 0x7fd3c40cf110 2024-09-16T07:31:46.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.817+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c4150fd0 con 0x7fd3c40cf110 2024-09-16T07:31:46.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.818+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c40b4920 con 0x7fd3c40cf110 2024-09-16T07:31:46.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.819+0000 7fd3ba000640 1 -- 172.21.15.71:0/269193613 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) ==== 102919+0+0 (secure 0 0 0) 0x7fd3bc037020 con 0x7fd3c40cf110 2024-09-16T07:31:46.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.820+0000 7fd3ba000640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3a4079a80 0x7fd3a407bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:46.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.821+0000 7fd3ba000640 1 -- 172.21.15.71:0/269193613 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fd3bc0ba7d0 con 0x7fd3c40cf110 2024-09-16T07:31:46.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.821+0000 7fd3c1e00640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3a4079a80 0x7fd3a407bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:46.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.821+0000 7fd3c1e00640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3a4079a80 0x7fd3a407bf40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40d06a0 tx=0x7fd3b4005f30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:46.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:46.824+0000 7fd3ba000640 1 -- 172.21.15.71:0/269193613 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd3bc084890 con 0x7fd3c40cf110 2024-09-16T07:31:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.044+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone50521", "target": ["mon-mgr", ""]}) -- 0x7fd3c4076170 con 0x7fd3a4079a80 2024-09-16T07:31:47.636 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.636+0000 7fe71fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.636 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.636+0000 7fe71fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.636 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.636+0000 7fe71fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.636 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.636+0000 7fe71fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.636 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.636+0000 7fe71fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.738+0000 7fd3ba000640 1 -- 172.21.15.71:0/269193613 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3c4076170 con 0x7fd3a4079a80 2024-09-16T07:31:47.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3a4079a80 msgr2=0x7fd3a407bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:47.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3a4079a80 0x7fd3a407bf40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40d06a0 tx=0x7fd3b4005f30 comp rx=0 tx=0).stop 2024-09-16T07:31:47.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40cf110 msgr2=0x7fd3c40c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:47.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40cf110 0x7fd3c40c9710 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc005fd0 tx=0x7fd3bc003f90 comp rx=0 tx=0).stop 2024-09-16T07:31:47.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 shutdown_connections 2024-09-16T07:31:47.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3a4079a80 0x7fd3a407bf40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:47.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3c40cf110 0x7fd3c40c9710 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:47.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40ced30 0x7fd3c41f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:47.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 --2- 172.21.15.71:0/269193613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b6c50 0x7fd3c414e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:47.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.741+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 >> 172.21.15.71:0/269193613 conn(0x7fd3c40d3e20 msgr2=0x7fd3c40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:47.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.742+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 shutdown_connections 2024-09-16T07:31:47.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:47.742+0000 7fd3c3200640 1 -- 172.21.15.71:0/269193613 wait complete. 2024-09-16T07:31:47.766 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone31792 2024-09-16T07:31:47.773 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.773+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.773 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.773+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.773 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.773+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.773 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.773+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:47.773 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:47.773+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:48.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.020+0000 7fd930800640 1 -- 172.21.15.71:0/1554205179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 msgr2=0x7fd9280d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:48.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.020+0000 7fd930800640 1 --2- 172.21.15.71:0/1554205179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 0x7fd9280d2530 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd924005fd0 tx=0x7fd92402aa10 comp rx=0 tx=0).stop 2024-09-16T07:31:48.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.021+0000 7fd930800640 1 -- 172.21.15.71:0/1554205179 shutdown_connections 2024-09-16T07:31:48.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.021+0000 7fd930800640 1 --2- 172.21.15.71:0/1554205179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 0x7fd9280d2530 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:48.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.021+0000 7fd930800640 1 --2- 172.21.15.71:0/1554205179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9280d0cc0 0x7fd9280d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:48.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.021+0000 7fd930800640 1 -- 172.21.15.71:0/1554205179 >> 172.21.15.71:0/1554205179 conn(0x7fd9280d4040 msgr2=0x7fd9280d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:48.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.022+0000 7fd930800640 1 -- 172.21.15.71:0/1554205179 shutdown_connections 2024-09-16T07:31:48.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.022+0000 7fd930800640 1 -- 172.21.15.71:0/1554205179 wait complete. 2024-09-16T07:31:48.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.025+0000 7fd930800640 1 Processor -- start 2024-09-16T07:31:48.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.026+0000 7fd930800640 1 -- start start 2024-09-16T07:31:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd930800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 0x7fd9280cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd930800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9280d0cc0 0x7fd9280cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd930800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9280d4bd0 0x7fd9281e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd930800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9280d2bf0 con 0x7fd9280b4e90 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd930800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd9280d2a70 con 0x7fd9280d4bd0 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd930800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd9280cea30 con 0x7fd9280d0cc0 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd92de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 0x7fd9280cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd92de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 0x7fd9280cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56822/0 (socket says 172.21.15.71:56822) 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd92de00640 1 -- 172.21.15.71:0/1506254607 learned_addr learned my addr 172.21.15.71:0/1506254607 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.028+0000 7fd92e800640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9280d4bd0 0x7fd9281e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.029+0000 7fd92d400640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9280d0cc0 0x7fd9280cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.029+0000 7fd92de00640 1 -- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9280d0cc0 msgr2=0x7fd9280cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.029+0000 7fd92de00640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9280d0cc0 0x7fd9280cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.029+0000 7fd92de00640 1 -- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9280d4bd0 msgr2=0x7fd9281e94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.029+0000 7fd92de00640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9280d4bd0 0x7fd9281e94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.029+0000 7fd92de00640 1 -- 172.21.15.71:0/1506254607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9280cb480 con 0x7fd9280b4e90 2024-09-16T07:31:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.030+0000 7fd92de00640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 0x7fd9280cc520 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c890 tx=0x7fd91c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.030+0000 7fd922000640 1 -- 172.21.15.71:0/1506254607 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c004220 con 0x7fd9280b4e90 2024-09-16T07:31:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.030+0000 7fd922000640 1 -- 172.21.15.71:0/1506254607 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd91c0079b0 con 0x7fd9280b4e90 2024-09-16T07:31:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.030+0000 7fd922000640 1 -- 172.21.15.71:0/1506254607 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c007de0 con 0x7fd9280b4e90 2024-09-16T07:31:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.030+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9280cb760 con 0x7fd9280b4e90 2024-09-16T07:31:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.031+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9280cbd20 con 0x7fd9280b4e90 2024-09-16T07:31:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.036+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9280d10a0 con 0x7fd9280b4e90 2024-09-16T07:31:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.036+0000 7fd922000640 1 -- 172.21.15.71:0/1506254607 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) ==== 102919+0+0 (secure 0 0 0) 0x7fd91c005020 con 0x7fd9280b4e90 2024-09-16T07:31:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.038+0000 7fd922000640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd90c079a50 0x7fd90c07bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:48.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.038+0000 7fd922000640 1 -- 172.21.15.71:0/1506254607 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fd91c0999e0 con 0x7fd9280b4e90 2024-09-16T07:31:48.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.042+0000 7fd92d400640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd90c079a50 0x7fd90c07bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:48.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.043+0000 7fd922000640 1 -- 172.21.15.71:0/1506254607 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd91c014030 con 0x7fd9280b4e90 2024-09-16T07:31:48.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.044+0000 7fd92d400640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd90c079a50 0x7fd90c07bf10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd918006fd0 tx=0x7fd918008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:48.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:48.424+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone31792", "target": ["mon-mgr", ""]}) -- 0x7fd9280cda70 con 0x7fd90c079a50 2024-09-16T07:31:49.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.102+0000 7fd922000640 1 -- 172.21.15.71:0/1506254607 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd9280cda70 con 0x7fd90c079a50 2024-09-16T07:31:49.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd90c079a50 msgr2=0x7fd90c07bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:49.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd90c079a50 0x7fd90c07bf10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd918006fd0 tx=0x7fd918008040 comp rx=0 tx=0).stop 2024-09-16T07:31:49.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 msgr2=0x7fd9280cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:49.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 0x7fd9280cc520 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c890 tx=0x7fd91c00cd50 comp rx=0 tx=0).stop 2024-09-16T07:31:49.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 shutdown_connections 2024-09-16T07:31:49.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd90c079a50 0x7fd90c07bf10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:49.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9280d4bd0 0x7fd9281e94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:49.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9280d0cc0 0x7fd9280cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:49.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 --2- 172.21.15.71:0/1506254607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9280b4e90 0x7fd9280cc520 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:49.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.108+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 >> 172.21.15.71:0/1506254607 conn(0x7fd9280d4040 msgr2=0x7fd92806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:49.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.109+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 shutdown_connections 2024-09-16T07:31:49.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.109+0000 7fd930800640 1 -- 172.21.15.71:0/1506254607 wait complete. 2024-09-16T07:31:49.134 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:49.134+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:49.135 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:49.134+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:49.135 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:49.134+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:49.135 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:49.134+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:49.135 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:31:49.134+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:31:49.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone51223 2024-09-16T07:31:49.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.444+0000 7effaaa00640 1 -- 172.21.15.71:0/3912946200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa41618c0 msgr2=0x7effa40d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:49.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.444+0000 7effaaa00640 1 --2- 172.21.15.71:0/3912946200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa41618c0 0x7effa40d4bc0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7effa40d4000 tx=0x7eff9002a540 comp rx=0 tx=0).stop 2024-09-16T07:31:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.444+0000 7effaaa00640 1 -- 172.21.15.71:0/3912946200 shutdown_connections 2024-09-16T07:31:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.444+0000 7effaaa00640 1 --2- 172.21.15.71:0/3912946200 >> v2:172.21.15.177:3300/0 conn(0x7effa40cf630 0x7effa40cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.444+0000 7effaaa00640 1 --2- 172.21.15.71:0/3912946200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa41618c0 0x7effa40d4bc0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.444+0000 7effaaa00640 1 -- 172.21.15.71:0/3912946200 >> 172.21.15.71:0/3912946200 conn(0x7effa40d40a0 msgr2=0x7effa40d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.444+0000 7effaaa00640 1 -- 172.21.15.71:0/3912946200 shutdown_connections 2024-09-16T07:31:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.445+0000 7effaaa00640 1 -- 172.21.15.71:0/3912946200 wait complete. 2024-09-16T07:31:49.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.446+0000 7effaaa00640 1 Processor -- start 2024-09-16T07:31:49.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.447+0000 7effaaa00640 1 -- start start 2024-09-16T07:31:49.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.447+0000 7effaaa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa40cf630 0x7effa40b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:49.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.447+0000 7effaaa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa41614e0 0x7effa414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:49.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.448+0000 7effa3400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa41614e0 0x7effa414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:49.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.448+0000 7effa3400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa41614e0 0x7effa414ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:41936/0 (socket says 172.21.15.71:41936) 2024-09-16T07:31:49.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.449+0000 7effa3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa40cf630 0x7effa40b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:49.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.449+0000 7effa3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa40cf630 0x7effa40b91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:53014/0 (socket says 172.21.15.71:53014) 2024-09-16T07:31:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.449+0000 7effaaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa41618c0 0x7effa414f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.449+0000 7effa3400640 1 -- 172.21.15.71:0/1918977795 learned_addr learned my addr 172.21.15.71:0/1918977795 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.449+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7effa40d24e0 con 0x7effa41618c0 2024-09-16T07:31:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.449+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effa40d2360 con 0x7effa40cf630 2024-09-16T07:31:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.449+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effa40d2660 con 0x7effa41614e0 2024-09-16T07:31:49.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.450+0000 7effa8a00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa41618c0 0x7effa414f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.451+0000 7effa3e00640 1 -- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa41614e0 msgr2=0x7effa414ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.451+0000 7effa3e00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa41614e0 0x7effa414ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.451+0000 7effa3e00640 1 -- 172.21.15.71:0/1918977795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa41618c0 msgr2=0x7effa414f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.451+0000 7effa3e00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa41618c0 0x7effa414f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.451+0000 7effa3e00640 1 -- 172.21.15.71:0/1918977795 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa41f4dd0 con 0x7effa40cf630 2024-09-16T07:31:49.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.454+0000 7effa3e00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa40cf630 0x7effa40b91f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff9400c3c0 tx=0x7eff9400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:49.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.457+0000 7eff9be00640 1 -- 172.21.15.71:0/1918977795 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94017070 con 0x7effa40cf630 2024-09-16T07:31:49.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.457+0000 7eff9be00640 1 -- 172.21.15.71:0/1918977795 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff940035f0 con 0x7effa40cf630 2024-09-16T07:31:49.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.457+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa41f50b0 con 0x7effa40cf630 2024-09-16T07:31:49.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.458+0000 7eff9be00640 1 -- 172.21.15.71:0/1918977795 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94003a80 con 0x7effa40cf630 2024-09-16T07:31:49.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.458+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa41f5670 con 0x7effa40cf630 2024-09-16T07:31:49.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.459+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa40cfae0 con 0x7effa40cf630 2024-09-16T07:31:49.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.465+0000 7eff9be00640 1 -- 172.21.15.71:0/1918977795 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 33) ==== 103123+0+0 (secure 0 0 0) 0x7eff9401a070 con 0x7effa40cf630 2024-09-16T07:31:49.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.467+0000 7eff9be00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff84079d60 0x7eff8407c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:49.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.467+0000 7eff9be00640 1 -- 172.21.15.71:0/1918977795 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7eff94098fd0 con 0x7effa40cf630 2024-09-16T07:31:49.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.475+0000 7eff9be00640 1 -- 172.21.15.71:0/1918977795 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7eff94021020 con 0x7effa40cf630 2024-09-16T07:31:49.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.490+0000 7effa3400640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff84079d60 0x7eff8407c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:49.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.491+0000 7effa3400640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff84079d60 0x7eff8407c220 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7effa40c88f0 tx=0x7eff90004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:49.834+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone51223", "target": ["mon-mgr", ""]}) -- 0x7effa40ba020 con 0x7eff84079d60 2024-09-16T07:31:50.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.706+0000 7eff9be00640 1 -- 172.21.15.71:0/1918977795 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7effa40ba020 con 0x7eff84079d60 2024-09-16T07:31:50.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff84079d60 msgr2=0x7eff8407c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:50.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff84079d60 0x7eff8407c220 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7effa40c88f0 tx=0x7eff90004730 comp rx=0 tx=0).stop 2024-09-16T07:31:50.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa40cf630 msgr2=0x7effa40b91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:50.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa40cf630 0x7effa40b91f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff9400c3c0 tx=0x7eff9400c880 comp rx=0 tx=0).stop 2024-09-16T07:31:50.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 shutdown_connections 2024-09-16T07:31:50.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff84079d60 0x7eff8407c220 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:50.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa41618c0 0x7effa414f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:50.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa41614e0 0x7effa414ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:50.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 --2- 172.21.15.71:0/1918977795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa40cf630 0x7effa40b91f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:50.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 >> 172.21.15.71:0/1918977795 conn(0x7effa40d40a0 msgr2=0x7effa406e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:50.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 shutdown_connections 2024-09-16T07:31:50.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:50.712+0000 7effaaa00640 1 -- 172.21.15.71:0/1918977795 wait complete. 2024-09-16T07:31:50.752 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone73294 2024-09-16T07:31:51.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.023+0000 7f19e8a00640 1 -- 172.21.15.71:0/2804625450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19c8003a50 msgr2=0x7f19c8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:51.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.023+0000 7f19e8a00640 1 --2- 172.21.15.71:0/2804625450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19c8003a50 0x7f19c8003e70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f19d400bb20 tx=0x7f19c8005830 comp rx=0 tx=0).stop 2024-09-16T07:31:51.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.023+0000 7f19e3e00640 1 -- 172.21.15.71:0/2804625450 reap_dead start 2024-09-16T07:31:51.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.023+0000 7f19e8a00640 1 -- 172.21.15.71:0/2804625450 shutdown_connections 2024-09-16T07:31:51.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.023+0000 7f19e8a00640 1 -- 172.21.15.71:0/2804625450 >> 172.21.15.71:0/2804625450 conn(0x7f19e40cc490 msgr2=0x7f19e40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:51.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.024+0000 7f19e8a00640 1 -- 172.21.15.71:0/2804625450 shutdown_connections 2024-09-16T07:31:51.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.024+0000 7f19e8a00640 1 -- 172.21.15.71:0/2804625450 wait complete. 2024-09-16T07:31:51.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.027+0000 7f19e8a00640 1 Processor -- start 2024-09-16T07:31:51.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.027+0000 7f19e8a00640 1 -- start start 2024-09-16T07:31:51.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.029+0000 7f19e8a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e42018f0 0x7f19e40af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:51.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.029+0000 7f19e8a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e4201cd0 0x7f19e40c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:51.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.029+0000 7f19e8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19e414c380 0x7f19e414c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:51.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.029+0000 7f19e8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40ca4f0 con 0x7f19e414c380 2024-09-16T07:31:51.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.029+0000 7f19e8a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40ca370 con 0x7f19e42018f0 2024-09-16T07:31:51.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.029+0000 7f19e8a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f19e40c6430 con 0x7f19e4201cd0 2024-09-16T07:31:51.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.030+0000 7f19e3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e42018f0 0x7f19e40af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:51.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.030+0000 7f19e3400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e42018f0 0x7f19e40af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:53032/0 (socket says 172.21.15.71:53032) 2024-09-16T07:31:51.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.030+0000 7f19e3400640 1 -- 172.21.15.71:0/536888358 learned_addr learned my addr 172.21.15.71:0/536888358 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.032+0000 7f19dbe00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e4201cd0 0x7f19e40c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:51.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.032+0000 7f19e3400640 1 -- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e4201cd0 msgr2=0x7f19e40c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.033+0000 7f19e3400640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e4201cd0 0x7f19e40c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.033+0000 7f19e3400640 1 -- 172.21.15.71:0/536888358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19e414c380 msgr2=0x7f19e414c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.033+0000 7f19e3400640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19e414c380 0x7f19e414c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.033+0000 7f19e3400640 1 -- 172.21.15.71:0/536888358 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e40c97c0 con 0x7f19e42018f0 2024-09-16T07:31:51.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.034+0000 7f19dbe00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e4201cd0 0x7f19e40c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:31:51.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.034+0000 7f19e3400640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e42018f0 0x7f19e40af590 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f19d400bc40 tx=0x7f19d4005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:51.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.036+0000 7f19db400640 1 -- 172.21.15.71:0/536888358 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4040070 con 0x7f19e42018f0 2024-09-16T07:31:51.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.036+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e42021b0 con 0x7f19e42018f0 2024-09-16T07:31:51.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.036+0000 7f19db400640 1 -- 172.21.15.71:0/536888358 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d4004030 con 0x7f19e42018f0 2024-09-16T07:31:51.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.036+0000 7f19db400640 1 -- 172.21.15.71:0/536888358 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4035040 con 0x7f19e42018f0 2024-09-16T07:31:51.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.037+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e4202770 con 0x7f19e42018f0 2024-09-16T07:31:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.039+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e415fa50 con 0x7f19e42018f0 2024-09-16T07:31:51.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.043+0000 7f19db400640 1 -- 172.21.15.71:0/536888358 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 33) ==== 103123+0+0 (secure 0 0 0) 0x7f19d4038020 con 0x7f19e42018f0 2024-09-16T07:31:51.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.045+0000 7f19db400640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19c0079cc0 0x7f19c007c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:51.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.045+0000 7f19db400640 1 -- 172.21.15.71:0/536888358 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f19d40ba630 con 0x7f19e42018f0 2024-09-16T07:31:51.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.051+0000 7f19db400640 1 -- 172.21.15.71:0/536888358 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f19d4084670 con 0x7f19e42018f0 2024-09-16T07:31:51.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.053+0000 7f19dbe00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19c0079cc0 0x7f19c007c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:51.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.072+0000 7f19dbe00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19c0079cc0 0x7f19c007c180 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f19e4156540 tx=0x7f19cc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:51.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.473+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone73294", "target": ["mon-mgr", ""]}) -- 0x7f19e414e270 con 0x7f19c0079cc0 2024-09-16T07:31:51.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:51.592+0000 7f19db400640 1 -- 172.21.15.71:0/536888358 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f19d40801e0 con 0x7f19e42018f0 2024-09-16T07:31:52.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.636+0000 7f19db400640 1 -- 172.21.15.71:0/536888358 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19e414e270 con 0x7f19c0079cc0 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19c0079cc0 msgr2=0x7f19c007c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19c0079cc0 0x7f19c007c180 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f19e4156540 tx=0x7f19cc009040 comp rx=0 tx=0).stop 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e42018f0 msgr2=0x7f19e40af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e42018f0 0x7f19e40af590 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f19d400bc40 tx=0x7f19d4005870 comp rx=0 tx=0).stop 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 shutdown_connections 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f19c0079cc0 0x7f19c007c180 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19e414c380 0x7f19e414c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e4201cd0 0x7f19e40c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 --2- 172.21.15.71:0/536888358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e42018f0 0x7f19e40af590 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 >> 172.21.15.71:0/536888358 conn(0x7f19e40cc490 msgr2=0x7f19e40cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 shutdown_connections 2024-09-16T07:31:52.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.639+0000 7f19e8a00640 1 -- 172.21.15.71:0/536888358 wait complete. 2024-09-16T07:31:52.675 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:31:52.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.930+0000 7f75da600640 1 -- 172.21.15.71:0/73672625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d415f3e0 msgr2=0x7f75d40d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:52.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.930+0000 7f75da600640 1 --2- 172.21.15.71:0/73672625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d415f3e0 0x7f75d40d09a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009830 tx=0x7f75c402a510 comp rx=0 tx=0).stop 2024-09-16T07:31:52.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.930+0000 7f75da600640 1 -- 172.21.15.71:0/73672625 shutdown_connections 2024-09-16T07:31:52.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.930+0000 7f75da600640 1 --2- 172.21.15.71:0/73672625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d415f3e0 0x7f75d40d09a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:52.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.930+0000 7f75da600640 1 -- 172.21.15.71:0/73672625 >> 172.21.15.71:0/73672625 conn(0x7f75d40d1e00 msgr2=0x7f75d40d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:52.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.930+0000 7f75da600640 1 -- 172.21.15.71:0/73672625 shutdown_connections 2024-09-16T07:31:52.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.930+0000 7f75da600640 1 -- 172.21.15.71:0/73672625 wait complete. 2024-09-16T07:31:52.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.932+0000 7f75da600640 1 Processor -- start 2024-09-16T07:31:52.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.932+0000 7f75da600640 1 -- start start 2024-09-16T07:31:52.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.933+0000 7f75da600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75d40b3480 0x7f75d40c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:52.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.934+0000 7f75da600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40cef20 0x7f75d415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:52.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.934+0000 7f75da600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75d415f3e0 0x7f75d415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:52.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.934+0000 7f75da600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75d40cd020 con 0x7f75d40cef20 2024-09-16T07:31:52.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.934+0000 7f75da600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f75d40b2bf0 con 0x7f75d415f3e0 2024-09-16T07:31:52.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.934+0000 7f75da600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f75d40cd1a0 con 0x7f75d40b3480 2024-09-16T07:31:52.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.934+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40cef20 0x7f75d415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:52.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.934+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40cef20 0x7f75d415a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56902/0 (socket says 172.21.15.71:56902) 2024-09-16T07:31:52.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.934+0000 7f75cbe00640 1 -- 172.21.15.71:0/4282137461 learned_addr learned my addr 172.21.15.71:0/4282137461 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:52.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.935+0000 7f75d3400640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75d40b3480 0x7f75d40c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:52.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.935+0000 7f75cbe00640 1 -- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75d40b3480 msgr2=0x7f75d40c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:52.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.935+0000 7f75cbe00640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75d40b3480 0x7f75d40c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:52.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.935+0000 7f75cbe00640 1 -- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75d415f3e0 msgr2=0x7f75d415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:52.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.935+0000 7f75cbe00640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75d415f3e0 0x7f75d415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:52.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.935+0000 7f75cbe00640 1 -- 172.21.15.71:0/4282137461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d40c7650 con 0x7f75d40cef20 2024-09-16T07:31:52.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.936+0000 7f75cbe00640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40cef20 0x7f75d415a0f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00c410 tx=0x7f75bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:52.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.937+0000 7f75cb400640 1 -- 172.21.15.71:0/4282137461 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc017070 con 0x7f75d40cef20 2024-09-16T07:31:52.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.937+0000 7f75cb400640 1 -- 172.21.15.71:0/4282137461 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75bc0035f0 con 0x7f75d40cef20 2024-09-16T07:31:52.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.937+0000 7f75cb400640 1 -- 172.21.15.71:0/4282137461 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc003a80 con 0x7f75d40cef20 2024-09-16T07:31:52.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.937+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d40c5eb0 con 0x7f75d40cef20 2024-09-16T07:31:52.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.937+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d40c7bd0 con 0x7f75d40cef20 2024-09-16T07:31:52.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.940+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d40d1220 con 0x7f75d40cef20 2024-09-16T07:31:52.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.941+0000 7f75cb400640 1 -- 172.21.15.71:0/4282137461 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f75bc01a070 con 0x7f75d40cef20 2024-09-16T07:31:52.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.943+0000 7f75cb400640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f75b0079df0 0x7f75b007c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:52.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.944+0000 7f75cb400640 1 -- 172.21.15.71:0/4282137461 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f75bc0989d0 con 0x7f75d40cef20 2024-09-16T07:31:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.949+0000 7f75d3400640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f75b0079df0 0x7f75b007c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:52.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.951+0000 7f75cb400640 1 -- 172.21.15.71:0/4282137461 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f75d40d1220 con 0x7f75d40cef20 2024-09-16T07:31:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:52.975+0000 7f75d3400640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f75b0079df0 0x7f75b007c2b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f75c4034000 tx=0x7f75c4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:53.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.441+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f75d40cd730 con 0x7f75d40cef20 2024-09-16T07:31:53.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.441+0000 7f75cb400640 1 -- 172.21.15.71:0/4282137461 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2180 (secure 0 0 0) 0x7f75bc0010c0 con 0x7f75d40cef20 2024-09-16T07:31:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:53.441 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:31:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:31:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:31:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 570, 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 92, 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:31:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:31:53.444 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 915, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 3712911482, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 7738572800, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 765355540480, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 46467068, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 130334817, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 11, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 35 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 57, 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:31:18:228480+0000", 2024-09-16T07:31:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6, 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 6, 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:31:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 7008 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 14, 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:30:52.446300+0000", 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "3 ongoing clones - average progress is 20.052% (3s)\n [=====.......................] (remaining: 12s)", 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.20051999390125275, 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:53.449 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f75b0079df0 msgr2=0x7f75b007c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f75b0079df0 0x7f75b007c2b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f75c4034000 tx=0x7f75c4004730 comp rx=0 tx=0).stop 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40cef20 msgr2=0x7f75d415a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40cef20 0x7f75d415a0f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00c410 tx=0x7f75bc00c8d0 comp rx=0 tx=0).stop 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 shutdown_connections 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f75b0079df0 0x7f75b007c2b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75d415f3e0 0x7f75d415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40cef20 0x7f75d415a0f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 --2- 172.21.15.71:0/4282137461 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75d40b3480 0x7f75d40c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 >> 172.21.15.71:0/4282137461 conn(0x7f75d40d1e00 msgr2=0x7f75d40cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.444+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 shutdown_connections 2024-09-16T07:31:53.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:53.445+0000 7f75da600640 1 -- 172.21.15.71:0/4282137461 wait complete. 2024-09-16T07:31:54.466 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:31:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.761+0000 7f8b62400640 1 -- 172.21.15.71:0/534474695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b40004e40 msgr2=0x7f8b400052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.761+0000 7f8b62400640 1 --2- 172.21.15.71:0/534474695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b40004e40 0x7f8b400052c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009f90 tx=0x7f8b40005830 comp rx=0 tx=0).stop 2024-09-16T07:31:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.761+0000 7f8b5be00640 1 -- 172.21.15.71:0/534474695 reap_dead start 2024-09-16T07:31:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.762+0000 7f8b62400640 1 -- 172.21.15.71:0/534474695 shutdown_connections 2024-09-16T07:31:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.762+0000 7f8b62400640 1 -- 172.21.15.71:0/534474695 >> 172.21.15.71:0/534474695 conn(0x7f8b5c0b18e0 msgr2=0x7f8b5c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.762+0000 7f8b62400640 1 -- 172.21.15.71:0/534474695 shutdown_connections 2024-09-16T07:31:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.762+0000 7f8b62400640 1 -- 172.21.15.71:0/534474695 wait complete. 2024-09-16T07:31:54.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.765+0000 7f8b62400640 1 Processor -- start 2024-09-16T07:31:54.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.765+0000 7f8b62400640 1 -- start start 2024-09-16T07:31:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.766+0000 7f8b62400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b5c14bb60 0x7f8b5c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.766+0000 7f8b62400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b5c15f0f0 0x7f8b5c09cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.766+0000 7f8b62400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c09d2d0 0x7f8b5c1f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.766+0000 7f8b62400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c0b2ce0 con 0x7f8b5c14bb60 2024-09-16T07:31:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.766+0000 7f8b62400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c0cd5e0 con 0x7f8b5c15f0f0 2024-09-16T07:31:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.766+0000 7f8b62400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5c0b2e60 con 0x7f8b5c09d2d0 2024-09-16T07:31:54.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.767+0000 7f8b5b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b5c14bb60 0x7f8b5c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:54.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.767+0000 7f8b5b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b5c14bb60 0x7f8b5c0ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56204/0 (socket says 172.21.15.71:56204) 2024-09-16T07:31:54.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.767+0000 7f8b5b400640 1 -- 172.21.15.71:0/1212284367 learned_addr learned my addr 172.21.15.71:0/1212284367 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:54.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.768+0000 7f8b5b400640 1 -- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c09d2d0 msgr2=0x7f8b5c1f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:54.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.768+0000 7f8b5b400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c09d2d0 0x7f8b5c1f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:54.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.768+0000 7f8b5b400640 1 -- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b5c15f0f0 msgr2=0x7f8b5c09cd90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:31:54.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.768+0000 7f8b5b400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b5c15f0f0 0x7f8b5c09cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:54.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.768+0000 7f8b5b400640 1 -- 172.21.15.71:0/1212284367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c0aef80 con 0x7f8b5c14bb60 2024-09-16T07:31:54.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.769+0000 7f8b5b400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b5c14bb60 0x7f8b5c0ce2b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00c410 tx=0x7f8b4c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:54.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.769+0000 7f8b53400640 1 -- 172.21.15.71:0/1212284367 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c017070 con 0x7f8b5c14bb60 2024-09-16T07:31:54.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.769+0000 7f8b53400640 1 -- 172.21.15.71:0/1212284367 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b4c0035f0 con 0x7f8b5c14bb60 2024-09-16T07:31:54.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.769+0000 7f8b53400640 1 -- 172.21.15.71:0/1212284367 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c003a80 con 0x7f8b5c14bb60 2024-09-16T07:31:54.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.770+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5c0cdb70 con 0x7f8b5c14bb60 2024-09-16T07:31:54.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.770+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5c09aba0 con 0x7f8b5c14bb60 2024-09-16T07:31:54.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.771+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b5c069360 con 0x7f8b5c14bb60 2024-09-16T07:31:54.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.773+0000 7f8b53400640 1 -- 172.21.15.71:0/1212284367 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f8b4c003c20 con 0x7f8b5c14bb60 2024-09-16T07:31:54.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.774+0000 7f8b53400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b3c079d20 0x7f8b3c07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:54.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.774+0000 7f8b53400640 1 -- 172.21.15.71:0/1212284367 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f8b4c098cd0 con 0x7f8b5c14bb60 2024-09-16T07:31:54.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.781+0000 7f8b53400640 1 -- 172.21.15.71:0/1212284367 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b4c0629b0 con 0x7f8b5c14bb60 2024-09-16T07:31:54.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.786+0000 7f8b5aa00640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b3c079d20 0x7f8b3c07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:54.861+0000 7f8b5aa00640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b3c079d20 0x7f8b3c07c1e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b48001260 tx=0x7f8b480035f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.273+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8b5c074240 con 0x7f8b5c14bb60 2024-09-16T07:31:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.274+0000 7f8b53400640 1 -- 172.21.15.71:0/1212284367 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2184 (secure 0 0 0) 0x7f8b4c0622b0 con 0x7f8b5c14bb60 2024-09-16T07:31:55.274 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:55.274 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:31:55.274 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:31:55.274 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:31:55.274 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:31:55.274 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:31:55.274 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:31:55.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 572, 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:31:55.275 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 92, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:55.276 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 1677, 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 6888299677, 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 13543419904, 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 759550693376, 2024-09-16T07:31:55.277 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 327694099, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 391952870, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 79, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 107 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 57, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:31:18:228480+0000", 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:31:55.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 6, 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 7008 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:31:55.279 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 14, 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:30:52.446300+0000", 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:31:55.280 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:31:55.281 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "4 ongoing clones - average progress is 27.148% (5s)\n [=======.....................] (remaining: 14s)", 2024-09-16T07:31:55.281 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.27147999405860901, 2024-09-16T07:31:55.281 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:31:55.281 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:55.281 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:55.281 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:31:55.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.279+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b3c079d20 msgr2=0x7f8b3c07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:55.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.279+0000 7f8b62400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b3c079d20 0x7f8b3c07c1e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b48001260 tx=0x7f8b480035f0 comp rx=0 tx=0).stop 2024-09-16T07:31:55.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.279+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b5c14bb60 msgr2=0x7f8b5c0ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:55.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.279+0000 7f8b62400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b5c14bb60 0x7f8b5c0ce2b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00c410 tx=0x7f8b4c00c8d0 comp rx=0 tx=0).stop 2024-09-16T07:31:55.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.283+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 shutdown_connections 2024-09-16T07:31:55.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.283+0000 7f8b62400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b3c079d20 0x7f8b3c07c1e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:55.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.283+0000 7f8b62400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c09d2d0 0x7f8b5c1f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:55.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.283+0000 7f8b62400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b5c15f0f0 0x7f8b5c09cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:55.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.283+0000 7f8b62400640 1 --2- 172.21.15.71:0/1212284367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b5c14bb60 0x7f8b5c0ce2b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:55.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.283+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 >> 172.21.15.71:0/1212284367 conn(0x7f8b5c0b18e0 msgr2=0x7f8b5c0b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:55.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.285+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 shutdown_connections 2024-09-16T07:31:55.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:55.285+0000 7f8b62400640 1 -- 172.21.15.71:0/1212284367 wait complete. 2024-09-16T07:31:56.313 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:31:56.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.580+0000 7fbb7d600640 1 -- 172.21.15.71:0/682698632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780d1260 msgr2=0x7fbb780d4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:56.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.580+0000 7fbb7d600640 1 --2- 172.21.15.71:0/682698632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780d1260 0x7fbb780d4960 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fbb68005fd0 tx=0x7fbb6802a530 comp rx=0 tx=0).stop 2024-09-16T07:31:56.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.581+0000 7fbb7d600640 1 -- 172.21.15.71:0/682698632 shutdown_connections 2024-09-16T07:31:56.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.581+0000 7fbb7d600640 1 --2- 172.21.15.71:0/682698632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780d1260 0x7fbb780d4960 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:56.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.581+0000 7fbb7d600640 1 --2- 172.21.15.71:0/682698632 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb7815e050 0x7fbb780cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:56.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.581+0000 7fbb7d600640 1 -- 172.21.15.71:0/682698632 >> 172.21.15.71:0/682698632 conn(0x7fbb780d3e40 msgr2=0x7fbb780d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:56.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.581+0000 7fbb7d600640 1 -- 172.21.15.71:0/682698632 shutdown_connections 2024-09-16T07:31:56.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.581+0000 7fbb7d600640 1 -- 172.21.15.71:0/682698632 wait complete. 2024-09-16T07:31:56.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.586+0000 7fbb7d600640 1 Processor -- start 2024-09-16T07:31:56.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.589+0000 7fbb7d600640 1 -- start start 2024-09-16T07:31:56.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.592+0000 7fbb7d600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb780d1260 0x7fbb780a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:56.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.592+0000 7fbb7d600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb780d1ec0 0x7fbb780a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:56.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.592+0000 7fbb7d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7815e050 0x7fbb781fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:56.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.592+0000 7fbb7d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbb780d2820 con 0x7fbb7815e050 2024-09-16T07:31:56.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.592+0000 7fbb7d600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb780d26a0 con 0x7fbb780d1ec0 2024-09-16T07:31:56.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.592+0000 7fbb7d600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb780b7380 con 0x7fbb780d1260 2024-09-16T07:31:56.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.595+0000 7fbb7cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7815e050 0x7fbb781fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:56.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.595+0000 7fbb7cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7815e050 0x7fbb781fbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56214/0 (socket says 172.21.15.71:56214) 2024-09-16T07:31:56.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.595+0000 7fbb7cc00640 1 -- 172.21.15.71:0/2640831641 learned_addr learned my addr 172.21.15.71:0/2640831641 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:56.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.596+0000 7fbb77e00640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb780d1260 0x7fbb780a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:56.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.597+0000 7fbb7cc00640 1 -- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb780d1260 msgr2=0x7fbb780a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:56.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.597+0000 7fbb7cc00640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb780d1260 0x7fbb780a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:56.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.597+0000 7fbb7cc00640 1 -- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb780d1ec0 msgr2=0x7fbb780a1120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:31:56.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.597+0000 7fbb7cc00640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb780d1ec0 0x7fbb780a1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:56.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.597+0000 7fbb7cc00640 1 -- 172.21.15.71:0/2640831641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7815f910 con 0x7fbb7815e050 2024-09-16T07:31:56.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.598+0000 7fbb7cc00640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7815e050 0x7fbb781fbee0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000b9c0 tx=0x7fbb7000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:56.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.601+0000 7fbb6fe00640 1 -- 172.21.15.71:0/2640831641 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70019070 con 0x7fbb7815e050 2024-09-16T07:31:56.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.602+0000 7fbb6fe00640 1 -- 172.21.15.71:0/2640831641 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb7000cc90 con 0x7fbb7815e050 2024-09-16T07:31:56.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.602+0000 7fbb6fe00640 1 -- 172.21.15.71:0/2640831641 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700146a0 con 0x7fbb7815e050 2024-09-16T07:31:56.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.602+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7815fbf0 con 0x7fbb7815e050 2024-09-16T07:31:56.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.602+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb781601b0 con 0x7fbb7815e050 2024-09-16T07:31:56.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.605+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb50005290 con 0x7fbb7815e050 2024-09-16T07:31:56.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.608+0000 7fbb6fe00640 1 -- 172.21.15.71:0/2640831641 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7fbb7001c070 con 0x7fbb7815e050 2024-09-16T07:31:56.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.610+0000 7fbb6fe00640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb58079df0 0x7fbb5807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:56.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.610+0000 7fbb6fe00640 1 -- 172.21.15.71:0/2640831641 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fbb70099800 con 0x7fbb7815e050 2024-09-16T07:31:56.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.612+0000 7fbb77e00640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb58079df0 0x7fbb5807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:56.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.615+0000 7fbb6fe00640 1 -- 172.21.15.71:0/2640831641 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb70023020 con 0x7fbb7815e050 2024-09-16T07:31:56.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:56.623+0000 7fbb77e00640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb58079df0 0x7fbb5807c2b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbb78160df0 tx=0x7fbb64008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:57.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.035+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbb50005850 con 0x7fbb7815e050 2024-09-16T07:31:57.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.036+0000 7fbb6fe00640 1 -- 172.21.15.71:0/2640831641 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2422 (secure 0 0 0) 0x7fbb700638c0 con 0x7fbb7815e050 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:31:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 573, 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:31:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 92, 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 1677, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 6888299677, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 13543419904, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 759550693376, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 327560355, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 312255698, 2024-09-16T07:31:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 79, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 87 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 57, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:31:18:228480+0000", 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 6, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:31:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 7008 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:31:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 14, 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:30:52.446300+0000", 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "4 ongoing clones - average progress is 40.821% (8s)\n [===========.................] (remaining: 11s)", 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.40821000933647156, 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:31:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:31:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-total-clones": { 2024-09-16T07:31:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Total 5 clones - average progress is 32.656% (0s)\n [=========...................] ", 2024-09-16T07:31:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.32655999064445496, 2024-09-16T07:31:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:31:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:31:57.043 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:31:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb58079df0 msgr2=0x7fbb5807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb58079df0 0x7fbb5807c2b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbb78160df0 tx=0x7fbb64008040 comp rx=0 tx=0).stop 2024-09-16T07:31:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7815e050 msgr2=0x7fbb781fbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7815e050 0x7fbb781fbee0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000b9c0 tx=0x7fbb7000be80 comp rx=0 tx=0).stop 2024-09-16T07:31:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 shutdown_connections 2024-09-16T07:31:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb58079df0 0x7fbb5807c2b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7815e050 0x7fbb781fbee0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb780d1ec0 0x7fbb780a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 --2- 172.21.15.71:0/2640831641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb780d1260 0x7fbb780a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.040+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 >> 172.21.15.71:0/2640831641 conn(0x7fbb780d3e40 msgr2=0x7fbb780cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.041+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 shutdown_connections 2024-09-16T07:31:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.041+0000 7fbb7d600640 1 -- 172.21.15.71:0/2640831641 wait complete. 2024-09-16T07:31:57.074 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 40.821% (8s)\n [===========.................] (remaining: 11s)', 'progress': 0.40821000933647156, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 5 clones - average progress is 32.656% (0s)\n [=========...................] ', 'progress': 0.32655999064445496, 'add_to_ceph_s': True}} 2024-09-16T07:31:57.074 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone69795 2024-09-16T07:31:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.402+0000 7f273ae00640 1 -- 172.21.15.71:0/2298635889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27340b4bd0 msgr2=0x7f27340b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.402+0000 7f273ae00640 1 --2- 172.21.15.71:0/2298635889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27340b4bd0 0x7f27340b3ec0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f272c005fd0 tx=0x7f272c02aaa0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.402+0000 7f273ae00640 1 -- 172.21.15.71:0/2298635889 shutdown_connections 2024-09-16T07:31:57.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.403+0000 7f273ae00640 1 --2- 172.21.15.71:0/2298635889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27340b4400 0x7f273406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.403+0000 7f273ae00640 1 --2- 172.21.15.71:0/2298635889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27340b4bd0 0x7f27340b3ec0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.403+0000 7f273ae00640 1 -- 172.21.15.71:0/2298635889 >> 172.21.15.71:0/2298635889 conn(0x7f27340b1840 msgr2=0x7f27340b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:57.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.403+0000 7f273ae00640 1 -- 172.21.15.71:0/2298635889 shutdown_connections 2024-09-16T07:31:57.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.403+0000 7f273ae00640 1 -- 172.21.15.71:0/2298635889 wait complete. 2024-09-16T07:31:57.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.405+0000 7f273ae00640 1 Processor -- start 2024-09-16T07:31:57.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.405+0000 7f273ae00640 1 -- start start 2024-09-16T07:31:57.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.407+0000 7f273ae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27340b4400 0x7f27340a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:57.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.407+0000 7f273ae00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27340b4bd0 0x7f273413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:57.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.407+0000 7f273ae00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27341437c0 0x7f27341f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:57.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.407+0000 7f273ae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f273406ef20 con 0x7f27340b4400 2024-09-16T07:31:57.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.407+0000 7f273ae00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f273406eda0 con 0x7f27341437c0 2024-09-16T07:31:57.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.407+0000 7f273ae00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f273406f0a0 con 0x7f27340b4bd0 2024-09-16T07:31:57.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.407+0000 7f2733e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27340b4400 0x7f27340a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:57.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.408+0000 7f2733e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27340b4400 0x7f27340a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56222/0 (socket says 172.21.15.71:56222) 2024-09-16T07:31:57.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.408+0000 7f2733e00640 1 -- 172.21.15.71:0/3240939140 learned_addr learned my addr 172.21.15.71:0/3240939140 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:57.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.408+0000 7f2733e00640 1 -- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27340b4bd0 msgr2=0x7f273413d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:57.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.408+0000 7f2733e00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27340b4bd0 0x7f273413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.408+0000 7f2733e00640 1 -- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27341437c0 msgr2=0x7f27341f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:57.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.408+0000 7f2733e00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27341437c0 0x7f27341f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:57.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.408+0000 7f2733e00640 1 -- 172.21.15.71:0/3240939140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27340aa0c0 con 0x7f27340b4400 2024-09-16T07:31:57.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.409+0000 7f2733e00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27340b4400 0x7f27340a9950 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f272400c430 tx=0x7f272400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:57.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.409+0000 7f272be00640 1 -- 172.21.15.71:0/3240939140 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724017070 con 0x7f27340b4400 2024-09-16T07:31:57.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.409+0000 7f272be00640 1 -- 172.21.15.71:0/3240939140 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27240053e0 con 0x7f27340b4400 2024-09-16T07:31:57.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.409+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27340aa3a0 con 0x7f27340b4400 2024-09-16T07:31:57.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.409+0000 7f272be00640 1 -- 172.21.15.71:0/3240939140 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724003850 con 0x7f27340b4400 2024-09-16T07:31:57.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.410+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27340aa960 con 0x7f27340b4400 2024-09-16T07:31:57.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.412+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273406e860 con 0x7f27340b4400 2024-09-16T07:31:57.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.413+0000 7f272be00640 1 -- 172.21.15.71:0/3240939140 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f272401a070 con 0x7f27340b4400 2024-09-16T07:31:57.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.414+0000 7f272be00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2714079d20 0x7f271407c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:57.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.414+0000 7f272be00640 1 -- 172.21.15.71:0/3240939140 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f2724098990 con 0x7f27340b4400 2024-09-16T07:31:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.423+0000 7f272be00640 1 -- 172.21.15.71:0/3240939140 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2724062ab0 con 0x7f27340b4400 2024-09-16T07:31:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.444+0000 7f2733400640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2714079d20 0x7f271407c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:57.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.464+0000 7f2733400640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2714079d20 0x7f271407c1e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f272c035000 tx=0x7f272c004a40 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:57.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:57.818+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone69795", "target": ["mon-mgr", ""]}) -- 0x7f2734167090 con 0x7f2714079d20 2024-09-16T07:31:59.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.410+0000 7f272be00640 1 -- 172.21.15.71:0/3240939140 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f272c003970 con 0x7f2714079d20 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2714079d20 msgr2=0x7f271407c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2714079d20 0x7f271407c1e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f272c035000 tx=0x7f272c004a40 comp rx=0 tx=0).stop 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27340b4400 msgr2=0x7f27340a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27340b4400 0x7f27340a9950 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f272400c430 tx=0x7f272400c8f0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 shutdown_connections 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2714079d20 0x7f271407c1e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27341437c0 0x7f27341f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27340b4bd0 0x7f273413d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 --2- 172.21.15.71:0/3240939140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27340b4400 0x7f27340a9950 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 >> 172.21.15.71:0/3240939140 conn(0x7f27340b1840 msgr2=0x7f273413a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 shutdown_connections 2024-09-16T07:31:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.413+0000 7f273ae00640 1 -- 172.21.15.71:0/3240939140 wait complete. 2024-09-16T07:31:59.437 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone71046 2024-09-16T07:31:59.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.742+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2339506311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 msgr2=0x7f1cbc073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:59.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.742+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2339506311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 0x7f1cbc073440 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0af750 tx=0x7f1ca802a4b0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.742+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2339506311 shutdown_connections 2024-09-16T07:31:59.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.742+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2339506311 >> v2:172.21.15.177:3300/0 conn(0x7f1cbc06efa0 0x7f1cbc0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.742+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2339506311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 0x7f1cbc073440 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.742+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2339506311 >> 172.21.15.71:0/2339506311 conn(0x7f1cbc0b1840 msgr2=0x7f1cbc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:31:59.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.742+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2339506311 shutdown_connections 2024-09-16T07:31:59.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.742+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2339506311 wait complete. 2024-09-16T07:31:59.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.745+0000 7f1cc3a00640 1 Processor -- start 2024-09-16T07:31:59.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.745+0000 7f1cc3a00640 1 -- start start 2024-09-16T07:31:59.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.747+0000 7f1cc3a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc06efa0 0x7f1cbc135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:59.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.747+0000 7f1cc3a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0ccb00 0x7f1cbc1f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:59.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.747+0000 7f1cc3a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 0x7f1cbc1fc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:59.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.747+0000 7f1cc3a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0ced40 con 0x7f1cbc145750 2024-09-16T07:31:59.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.747+0000 7f1cc3a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0cebc0 con 0x7f1cbc06efa0 2024-09-16T07:31:59.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.747+0000 7f1cc3a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc0ceec0 con 0x7f1cbc0ccb00 2024-09-16T07:31:59.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.748+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0ccb00 0x7f1cbc1f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:59.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.748+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0ccb00 0x7f1cbc1f97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:38138/0 (socket says 172.21.15.71:38138) 2024-09-16T07:31:59.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.748+0000 7f1cc1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 0x7f1cbc1fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:59.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.748+0000 7f1cbbe00640 1 -- 172.21.15.71:0/2983408766 learned_addr learned my addr 172.21.15.71:0/2983408766 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:31:59.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.749+0000 7f1cc1000640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc06efa0 0x7f1cbc135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:59.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.749+0000 7f1cc1a00640 1 -- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0ccb00 msgr2=0x7f1cbc1f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:59.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.749+0000 7f1cc1a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0ccb00 0x7f1cbc1f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.749+0000 7f1cc1a00640 1 -- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc06efa0 msgr2=0x7f1cbc135100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:31:59.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.749+0000 7f1cc1a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc06efa0 0x7f1cbc135100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:31:59.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.749+0000 7f1cc1a00640 1 -- 172.21.15.71:0/2983408766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc1ea340 con 0x7f1cbc145750 2024-09-16T07:31:59.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.750+0000 7f1cc1000640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc06efa0 0x7f1cbc135100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:31:59.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.750+0000 7f1cc1a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 0x7f1cbc1fc490 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400cdd0 tx=0x7f1cb400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:31:59.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.751+0000 7f1cb8c00640 1 -- 172.21.15.71:0/2983408766 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4016020 con 0x7f1cbc145750 2024-09-16T07:31:59.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.752+0000 7f1cb8c00640 1 -- 172.21.15.71:0/2983408766 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb400ae30 con 0x7f1cbc145750 2024-09-16T07:31:59.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.752+0000 7f1cb8c00640 1 -- 172.21.15.71:0/2983408766 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4016020 con 0x7f1cbc145750 2024-09-16T07:31:59.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.752+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc1ea8b0 con 0x7f1cbc145750 2024-09-16T07:31:59.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.752+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc1eae70 con 0x7f1cbc145750 2024-09-16T07:31:59.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.756+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc0cc5c0 con 0x7f1cbc145750 2024-09-16T07:31:59.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.758+0000 7f1cb8c00640 1 -- 172.21.15.71:0/2983408766 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f1cb4019020 con 0x7f1cbc145750 2024-09-16T07:31:59.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.760+0000 7f1cb8c00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1c9c079f20 0x7f1c9c07c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:31:59.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.761+0000 7f1cb8c00640 1 -- 172.21.15.71:0/2983408766 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f1cb40994b0 con 0x7f1cbc145750 2024-09-16T07:31:59.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.770+0000 7f1cb8c00640 1 -- 172.21.15.71:0/2983408766 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1cb4020030 con 0x7f1cbc145750 2024-09-16T07:31:59.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.801+0000 7f1cc1000640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1c9c079f20 0x7f1c9c07c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:31:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:31:59.823+0000 7f1cc1000640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1c9c079f20 0x7f1c9c07c3e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8034000 tx=0x7f1ca8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:00.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:00.166+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone71046", "target": ["mon-mgr", ""]}) -- 0x7f1cbc1356a0 con 0x7f1c9c079f20 2024-09-16T07:32:01.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.564+0000 7f1cb8c00640 1 -- 172.21.15.71:0/2983408766 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1cbc1356a0 con 0x7f1c9c079f20 2024-09-16T07:32:01.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1c9c079f20 msgr2=0x7f1c9c07c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1c9c079f20 0x7f1c9c07c3e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8034000 tx=0x7f1ca8004730 comp rx=0 tx=0).stop 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 msgr2=0x7f1cbc1fc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 0x7f1cbc1fc490 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400cdd0 tx=0x7f1cb400a430 comp rx=0 tx=0).stop 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 shutdown_connections 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1c9c079f20 0x7f1c9c07c3e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cbc145750 0x7f1cbc1fc490 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0ccb00 0x7f1cbc1f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 --2- 172.21.15.71:0/2983408766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc06efa0 0x7f1cbc135100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:01.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.567+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 >> 172.21.15.71:0/2983408766 conn(0x7f1cbc0b1840 msgr2=0x7f1cbc15ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:01.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.568+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 shutdown_connections 2024-09-16T07:32:01.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.568+0000 7f1cc3a00640 1 -- 172.21.15.71:0/2983408766 wait complete. 2024-09-16T07:32:01.587 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone61947 2024-09-16T07:32:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.880+0000 7fe7bc000640 1 -- 172.21.15.71:0/3353603214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b40cec60 msgr2=0x7fe7b40cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.880+0000 7fe7bc000640 1 --2- 172.21.15.71:0/3353603214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b40cec60 0x7fe7b40cf070 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40cfba0 tx=0x7fe7ac02aa30 comp rx=0 tx=0).stop 2024-09-16T07:32:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.881+0000 7fe7bc000640 1 -- 172.21.15.71:0/3353603214 shutdown_connections 2024-09-16T07:32:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.881+0000 7fe7bc000640 1 --2- 172.21.15.71:0/3353603214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b40cec60 0x7fe7b40cf070 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.881+0000 7fe7bc000640 1 --2- 172.21.15.71:0/3353603214 >> v2:172.21.15.177:3300/0 conn(0x7fe7b4161760 0x7fe7b40ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.881+0000 7fe7bc000640 1 -- 172.21.15.71:0/3353603214 >> 172.21.15.71:0/3353603214 conn(0x7fe7b40b1840 msgr2=0x7fe7b40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.881+0000 7fe7bc000640 1 -- 172.21.15.71:0/3353603214 shutdown_connections 2024-09-16T07:32:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.881+0000 7fe7bc000640 1 -- 172.21.15.71:0/3353603214 wait complete. 2024-09-16T07:32:01.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.883+0000 7fe7bc000640 1 Processor -- start 2024-09-16T07:32:01.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.883+0000 7fe7bc000640 1 -- start start 2024-09-16T07:32:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7bc000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7b40af190 0x7fe7b41f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7bc000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7b40cec60 0x7fe7b4160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:01.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7bc000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b4161760 0x7fe7b414a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:01.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7bc000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4162310 con 0x7fe7b4161760 2024-09-16T07:32:01.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7bc000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4162190 con 0x7fe7b40cec60 2024-09-16T07:32:01.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7bc000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b4162490 con 0x7fe7b40af190 2024-09-16T07:32:01.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7ba000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b4161760 0x7fe7b414a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:01.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7ba000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b4161760 0x7fe7b414a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56270/0 (socket says 172.21.15.71:56270) 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.885+0000 7fe7ba000640 1 -- 172.21.15.71:0/1636281611 learned_addr learned my addr 172.21.15.71:0/1636281611 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.886+0000 7fe7b9600640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7b40af190 0x7fe7b41f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.886+0000 7fe7ba000640 1 -- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7b40af190 msgr2=0x7fe7b41f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.886+0000 7fe7ba000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7b40af190 0x7fe7b41f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.886+0000 7fe7ba000640 1 -- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7b40cec60 msgr2=0x7fe7b4160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.886+0000 7fe7ba000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7b40cec60 0x7fe7b4160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.886+0000 7fe7ba000640 1 -- 172.21.15.71:0/1636281611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b41f5690 con 0x7fe7b4161760 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.886+0000 7fe7ba000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b4161760 0x7fe7b414a300 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009d60 tx=0x7fe7ac003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.887+0000 7fe7b1600640 1 -- 172.21.15.71:0/1636281611 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac047070 con 0x7fe7b4161760 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.887+0000 7fe7b1600640 1 -- 172.21.15.71:0/1636281611 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7ac004250 con 0x7fe7b4161760 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.887+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b40c4f00 con 0x7fe7b4161760 2024-09-16T07:32:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.887+0000 7fe7b1600640 1 -- 172.21.15.71:0/1636281611 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac033050 con 0x7fe7b4161760 2024-09-16T07:32:01.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.887+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b41f5dc0 con 0x7fe7b4161760 2024-09-16T07:32:01.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.888+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b4162a70 con 0x7fe7b4161760 2024-09-16T07:32:01.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.889+0000 7fe7b1600640 1 -- 172.21.15.71:0/1636281611 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7fe7ac04a070 con 0x7fe7b4161760 2024-09-16T07:32:01.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.890+0000 7fe7b1600640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe794079d20 0x7fe79407c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:01.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.890+0000 7fe7b1600640 1 -- 172.21.15.71:0/1636281611 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fe7ac0baa70 con 0x7fe7b4161760 2024-09-16T07:32:01.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.893+0000 7fe7b9600640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe794079d20 0x7fe79407c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.895+0000 7fe7b9600640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe794079d20 0x7fe79407c1e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b810 tx=0x7fe7a0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:01.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:01.896+0000 7fe7b1600640 1 -- 172.21.15.71:0/1636281611 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe7b41f5690 con 0x7fe7b4161760 2024-09-16T07:32:02.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:02.285+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone61947", "target": ["mon-mgr", ""]}) -- 0x7fe7b4069a10 con 0x7fe794079d20 2024-09-16T07:32:03.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.193+0000 7fe7b1600640 1 -- 172.21.15.71:0/1636281611 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7b4069a10 con 0x7fe794079d20 2024-09-16T07:32:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.198+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe794079d20 msgr2=0x7fe79407c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.198+0000 7fe7bc000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe794079d20 0x7fe79407c1e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b810 tx=0x7fe7a0008040 comp rx=0 tx=0).stop 2024-09-16T07:32:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.198+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b4161760 msgr2=0x7fe7b414a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.198+0000 7fe7bc000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b4161760 0x7fe7b414a300 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009d60 tx=0x7fe7ac003f10 comp rx=0 tx=0).stop 2024-09-16T07:32:03.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.204+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 shutdown_connections 2024-09-16T07:32:03.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.204+0000 7fe7bc000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe794079d20 0x7fe79407c1e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:03.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.204+0000 7fe7bc000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7b4161760 0x7fe7b414a300 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:03.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.204+0000 7fe7bc000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7b40cec60 0x7fe7b4160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:03.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.204+0000 7fe7bc000640 1 --2- 172.21.15.71:0/1636281611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7b40af190 0x7fe7b41f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:03.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.204+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 >> 172.21.15.71:0/1636281611 conn(0x7fe7b40b1840 msgr2=0x7fe7b4072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:03.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.204+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 shutdown_connections 2024-09-16T07:32:03.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.204+0000 7fe7bc000640 1 -- 172.21.15.71:0/1636281611 wait complete. 2024-09-16T07:32:03.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone30628 2024-09-16T07:32:03.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.506+0000 7faa44800640 1 -- 172.21.15.71:0/1443836997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0cfaf0 msgr2=0x7faa3c0d2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:03.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.506+0000 7faa44800640 1 --2- 172.21.15.71:0/1443836997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0cfaf0 0x7faa3c0d2030 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0c7750 tx=0x7faa3802aa30 comp rx=0 tx=0).stop 2024-09-16T07:32:03.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.507+0000 7faa44800640 1 -- 172.21.15.71:0/1443836997 shutdown_connections 2024-09-16T07:32:03.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.507+0000 7faa44800640 1 --2- 172.21.15.71:0/1443836997 >> v2:172.21.15.177:3300/0 conn(0x7faa3c0b5630 0x7faa3c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:03.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.507+0000 7faa44800640 1 --2- 172.21.15.71:0/1443836997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0cfaf0 0x7faa3c0d2030 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:03.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.507+0000 7faa44800640 1 -- 172.21.15.71:0/1443836997 >> 172.21.15.71:0/1443836997 conn(0x7faa3c0d3fb0 msgr2=0x7faa3c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:03.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.507+0000 7faa44800640 1 -- 172.21.15.71:0/1443836997 shutdown_connections 2024-09-16T07:32:03.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.507+0000 7faa44800640 1 -- 172.21.15.71:0/1443836997 wait complete. 2024-09-16T07:32:03.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.509+0000 7faa44800640 1 Processor -- start 2024-09-16T07:32:03.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.510+0000 7faa44800640 1 -- start start 2024-09-16T07:32:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.511+0000 7faa44800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0b5630 0x7faa3c0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.511+0000 7faa44800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa3c0cfaf0 0x7faa3c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.511+0000 7faa44800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa3c161590 0x7faa3c15e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.511+0000 7faa44800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c0d26f0 con 0x7faa3c0b5630 2024-09-16T07:32:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.511+0000 7faa44800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c0d2570 con 0x7faa3c0cfaf0 2024-09-16T07:32:03.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.511+0000 7faa44800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7faa3c0d2870 con 0x7faa3c161590 2024-09-16T07:32:03.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.512+0000 7faa41e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0b5630 0x7faa3c0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:03.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.512+0000 7faa41e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0b5630 0x7faa3c0b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56278/0 (socket says 172.21.15.71:56278) 2024-09-16T07:32:03.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.512+0000 7faa41e00640 1 -- 172.21.15.71:0/3874239998 learned_addr learned my addr 172.21.15.71:0/3874239998 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:03.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.512+0000 7faa41400640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa3c0cfaf0 0x7faa3c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:03.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.514+0000 7faa41e00640 1 -- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa3c161590 msgr2=0x7faa3c15e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:03.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.514+0000 7faa41e00640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa3c161590 0x7faa3c15e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:03.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.514+0000 7faa41e00640 1 -- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa3c0cfaf0 msgr2=0x7faa3c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:03.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.514+0000 7faa41e00640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa3c0cfaf0 0x7faa3c0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:03.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.514+0000 7faa41e00640 1 -- 172.21.15.71:0/3874239998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3c14e230 con 0x7faa3c0b5630 2024-09-16T07:32:03.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.514+0000 7faa41e00640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0b5630 0x7faa3c0b4a50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7faa3000b3e0 tx=0x7faa3000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:03.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.515+0000 7faa36000640 1 -- 172.21.15.71:0/3874239998 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30015020 con 0x7faa3c0b5630 2024-09-16T07:32:03.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.515+0000 7faa36000640 1 -- 172.21.15.71:0/3874239998 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa30011940 con 0x7faa3c0b5630 2024-09-16T07:32:03.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.515+0000 7faa36000640 1 -- 172.21.15.71:0/3874239998 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa30011d90 con 0x7faa3c0b5630 2024-09-16T07:32:03.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.515+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3c1f4f70 con 0x7faa3c0b5630 2024-09-16T07:32:03.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.515+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3c1f5510 con 0x7faa3c0b5630 2024-09-16T07:32:03.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.517+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa3c0b9f30 con 0x7faa3c0b5630 2024-09-16T07:32:03.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.519+0000 7faa36000640 1 -- 172.21.15.71:0/3874239998 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7faa30018020 con 0x7faa3c0b5630 2024-09-16T07:32:03.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.520+0000 7faa36000640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faa20079d20 0x7faa2007c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:03.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.520+0000 7faa36000640 1 -- 172.21.15.71:0/3874239998 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7faa30099510 con 0x7faa3c0b5630 2024-09-16T07:32:03.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.525+0000 7faa36000640 1 -- 172.21.15.71:0/3874239998 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faa3001f020 con 0x7faa3c0b5630 2024-09-16T07:32:03.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.531+0000 7faa41400640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faa20079d20 0x7faa2007c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:03.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.538+0000 7faa41400640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faa20079d20 0x7faa2007c1e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7faa3c160240 tx=0x7faa2c009110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:03.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:03.905+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone30628", "target": ["mon-mgr", ""]}) -- 0x7faa3c074730 con 0x7faa20079d20 2024-09-16T07:32:04.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.972+0000 7faa36000640 1 -- 172.21.15.71:0/3874239998 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa3c074730 con 0x7faa20079d20 2024-09-16T07:32:04.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faa20079d20 msgr2=0x7faa2007c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faa20079d20 0x7faa2007c1e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7faa3c160240 tx=0x7faa2c009110 comp rx=0 tx=0).stop 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0b5630 msgr2=0x7faa3c0b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0b5630 0x7faa3c0b4a50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7faa3000b3e0 tx=0x7faa3000b7e0 comp rx=0 tx=0).stop 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 shutdown_connections 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faa20079d20 0x7faa2007c1e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa3c161590 0x7faa3c15e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa3c0cfaf0 0x7faa3c0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 --2- 172.21.15.71:0/3874239998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa3c0b5630 0x7faa3c0b4a50 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.978+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 >> 172.21.15.71:0/3874239998 conn(0x7faa3c0d3fb0 msgr2=0x7faa3c0b8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:04.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.979+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 shutdown_connections 2024-09-16T07:32:04.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:04.979+0000 7faa44800640 1 -- 172.21.15.71:0/3874239998 wait complete. 2024-09-16T07:32:05.032 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone86499 2024-09-16T07:32:05.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.291+0000 7f8557e00640 1 -- 172.21.15.71:0/3179921747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855015d8d0 msgr2=0x7f8550070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:05.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.291+0000 7f8557e00640 1 --2- 172.21.15.71:0/3179921747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855015d8d0 0x7f8550070310 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f853c00a940 tx=0x7f853c02b860 comp rx=0 tx=0).stop 2024-09-16T07:32:05.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.292+0000 7f8557e00640 1 -- 172.21.15.71:0/3179921747 shutdown_connections 2024-09-16T07:32:05.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.292+0000 7f8557e00640 1 --2- 172.21.15.71:0/3179921747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855015d8d0 0x7f8550070310 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:05.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.292+0000 7f8557e00640 1 -- 172.21.15.71:0/3179921747 >> 172.21.15.71:0/3179921747 conn(0x7f85500b3560 msgr2=0x7f85500b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:05.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.292+0000 7f8557e00640 1 -- 172.21.15.71:0/3179921747 shutdown_connections 2024-09-16T07:32:05.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.293+0000 7f8557e00640 1 -- 172.21.15.71:0/3179921747 wait complete. 2024-09-16T07:32:05.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.296+0000 7f8557e00640 1 Processor -- start 2024-09-16T07:32:05.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.296+0000 7f8557e00640 1 -- start start 2024-09-16T07:32:05.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8557e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cf790 0x7f8550150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:05.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8557e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85500cfb70 0x7f85500b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:05.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8557e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85501f4190 0x7f85500b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:05.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8557e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8550070850 con 0x7f85500cfb70 2024-09-16T07:32:05.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8557e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85500c7f10 con 0x7f85500cf790 2024-09-16T07:32:05.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8557e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f85500709d0 con 0x7f85501f4190 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8554a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85500cfb70 0x7f85500b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8554a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85500cfb70 0x7f85500b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47280/0 (socket says 172.21.15.71:47280) 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.297+0000 7f8554a00640 1 -- 172.21.15.71:0/3731826649 learned_addr learned my addr 172.21.15.71:0/3731826649 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.298+0000 7f8554a00640 1 -- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85501f4190 msgr2=0x7f85500b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.298+0000 7f8554a00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85501f4190 0x7f85500b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.298+0000 7f8554a00640 1 -- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cf790 msgr2=0x7f8550150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.298+0000 7f8554a00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cf790 0x7f8550150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.298+0000 7f8554a00640 1 -- 172.21.15.71:0/3731826649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85501f4620 con 0x7f85500cfb70 2024-09-16T07:32:05.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.298+0000 7f8554a00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85500cfb70 0x7f85500b1530 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f853c00a890 tx=0x7f853c031c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:05.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.299+0000 7f854d600640 1 -- 172.21.15.71:0/3731826649 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c0035f0 con 0x7f85500cfb70 2024-09-16T07:32:05.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.299+0000 7f854d600640 1 -- 172.21.15.71:0/3731826649 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f853c0037b0 con 0x7f85500cfb70 2024-09-16T07:32:05.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.299+0000 7f854d600640 1 -- 172.21.15.71:0/3731826649 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c003c00 con 0x7f85500cfb70 2024-09-16T07:32:05.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.299+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85501f02a0 con 0x7f85500cfb70 2024-09-16T07:32:05.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.299+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85501f0860 con 0x7f85500cfb70 2024-09-16T07:32:05.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.301+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8550070310 con 0x7f85500cfb70 2024-09-16T07:32:05.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.303+0000 7f854d600640 1 -- 172.21.15.71:0/3731826649 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f853c04e430 con 0x7f85500cfb70 2024-09-16T07:32:05.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.305+0000 7f854d600640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8530079d20 0x7f853007c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:05.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.306+0000 7f854d600640 1 -- 172.21.15.71:0/3731826649 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f853c0baee0 con 0x7f85500cfb70 2024-09-16T07:32:05.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.307+0000 7f8555400640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8530079d20 0x7f853007c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:05.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.310+0000 7f854d600640 1 -- 172.21.15.71:0/3731826649 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f853c084bc0 con 0x7f85500cfb70 2024-09-16T07:32:05.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.313+0000 7f8555400640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8530079d20 0x7f853007c1e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8544001220 tx=0x7f85440011b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:05.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:05.646+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone86499", "target": ["mon-mgr", ""]}) -- 0x7f85500d2340 con 0x7f8530079d20 2024-09-16T07:32:06.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.651+0000 7f854d600640 1 -- 172.21.15.71:0/3731826649 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85500d2340 con 0x7f8530079d20 2024-09-16T07:32:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.656+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8530079d20 msgr2=0x7f853007c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.656+0000 7f8557e00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8530079d20 0x7f853007c1e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8544001220 tx=0x7f85440011b0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.656+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85500cfb70 msgr2=0x7f85500b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:06.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.656+0000 7f8557e00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85500cfb70 0x7f85500b1530 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f853c00a890 tx=0x7f853c031c20 comp rx=0 tx=0).stop 2024-09-16T07:32:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.657+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 shutdown_connections 2024-09-16T07:32:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.657+0000 7f8557e00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8530079d20 0x7f853007c1e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.657+0000 7f8557e00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85501f4190 0x7f85500b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.657+0000 7f8557e00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85500cfb70 0x7f85500b1530 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.657+0000 7f8557e00640 1 --2- 172.21.15.71:0/3731826649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cf790 0x7f8550150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.657+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 >> 172.21.15.71:0/3731826649 conn(0x7f85500b3560 msgr2=0x7f85500b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.657+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 shutdown_connections 2024-09-16T07:32:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.657+0000 7f8557e00640 1 -- 172.21.15.71:0/3731826649 wait complete. 2024-09-16T07:32:06.679 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone614610 2024-09-16T07:32:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.974+0000 7f0128a00640 1 -- 172.21.15.71:0/2275368288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01240b4400 msgr2=0x7f012406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.974+0000 7f0128a00640 1 --2- 172.21.15.71:0/2275368288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01240b4400 0x7f012406e860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f011c005fd0 tx=0x7f011c02b880 comp rx=0 tx=0).stop 2024-09-16T07:32:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.975+0000 7f0128a00640 1 -- 172.21.15.71:0/2275368288 shutdown_connections 2024-09-16T07:32:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.975+0000 7f0128a00640 1 --2- 172.21.15.71:0/2275368288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01240b4400 0x7f012406e860 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.975+0000 7f0128a00640 1 --2- 172.21.15.71:0/2275368288 >> v2:172.21.15.71:3300/0 conn(0x7f01240b4bd0 0x7f01240b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.975+0000 7f0128a00640 1 -- 172.21.15.71:0/2275368288 >> 172.21.15.71:0/2275368288 conn(0x7f01240b1840 msgr2=0x7f01240b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.975+0000 7f0128a00640 1 -- 172.21.15.71:0/2275368288 shutdown_connections 2024-09-16T07:32:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.976+0000 7f0128a00640 1 -- 172.21.15.71:0/2275368288 wait complete. 2024-09-16T07:32:06.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.978+0000 7f0128a00640 1 Processor -- start 2024-09-16T07:32:06.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.978+0000 7f0128a00640 1 -- start start 2024-09-16T07:32:06.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.979+0000 7f0128a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01240b4400 0x7f01240aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:06.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.979+0000 7f0128a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01240b4bd0 0x7f01241f8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:06.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.979+0000 7f0128a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01241437c0 0x7f0124141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:06.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.979+0000 7f0128a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f012406ef20 con 0x7f01240b4400 2024-09-16T07:32:06.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.979+0000 7f0128a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f012406eda0 con 0x7f01240b4bd0 2024-09-16T07:32:06.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.979+0000 7f0128a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f012406f0a0 con 0x7f01241437c0 2024-09-16T07:32:06.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.980+0000 7f0121e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01240b4400 0x7f01240aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:06.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.980+0000 7f0121e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01240b4400 0x7f01240aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47300/0 (socket says 172.21.15.71:47300) 2024-09-16T07:32:06.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.980+0000 7f0121e00640 1 -- 172.21.15.71:0/2361770725 learned_addr learned my addr 172.21.15.71:0/2361770725 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:06.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.980+0000 7f0121400640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01240b4bd0 0x7f01241f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:06.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.980+0000 7f0122800640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01241437c0 0x7f0124141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:06.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.981+0000 7f0121e00640 1 -- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01241437c0 msgr2=0x7f0124141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:06.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.981+0000 7f0121e00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01241437c0 0x7f0124141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.981+0000 7f0121e00640 1 -- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01240b4bd0 msgr2=0x7f01241f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:06.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.981+0000 7f0121e00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01240b4bd0 0x7f01241f8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:06.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.981+0000 7f0121e00640 1 -- 172.21.15.71:0/2361770725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01240af3e0 con 0x7f01240b4400 2024-09-16T07:32:06.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.982+0000 7f0121e00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01240b4400 0x7f01240aeea0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f011400c400 tx=0x7f011400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:06.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.983+0000 7f011a000640 1 -- 172.21.15.71:0/2361770725 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114017070 con 0x7f01240b4400 2024-09-16T07:32:06.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.983+0000 7f011a000640 1 -- 172.21.15.71:0/2361770725 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01140053c0 con 0x7f01240b4400 2024-09-16T07:32:06.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.983+0000 7f011a000640 1 -- 172.21.15.71:0/2361770725 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114003850 con 0x7f01240b4400 2024-09-16T07:32:06.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.983+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01241fcb50 con 0x7f01240b4400 2024-09-16T07:32:06.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.983+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01241fd110 con 0x7f01240b4400 2024-09-16T07:32:06.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.984+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012406e860 con 0x7f01240b4400 2024-09-16T07:32:06.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.991+0000 7f011a000640 1 -- 172.21.15.71:0/2361770725 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f011401a070 con 0x7f01240b4400 2024-09-16T07:32:06.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.993+0000 7f011a000640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0104079d20 0x7f010407c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:06.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.993+0000 7f011a000640 1 -- 172.21.15.71:0/2361770725 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f0114098d20 con 0x7f01240b4400 2024-09-16T07:32:06.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.996+0000 7f011a000640 1 -- 172.21.15.71:0/2361770725 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0114021020 con 0x7f01240b4400 2024-09-16T07:32:06.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.996+0000 7f0121400640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0104079d20 0x7f010407c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:06.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:06.999+0000 7f0121400640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0104079d20 0x7f010407c1e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f01241fc5c0 tx=0x7f0110005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:07.323+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone614610", "target": ["mon-mgr", ""]}) -- 0x7f01240cd570 con 0x7f0104079d20 2024-09-16T07:32:08.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.584+0000 7f011a000640 1 -- 172.21.15.71:0/2361770725 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f01240cd570 con 0x7f0104079d20 2024-09-16T07:32:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0104079d20 msgr2=0x7f010407c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0104079d20 0x7f010407c1e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f01241fc5c0 tx=0x7f0110005f90 comp rx=0 tx=0).stop 2024-09-16T07:32:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01240b4400 msgr2=0x7f01240aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01240b4400 0x7f01240aeea0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f011400c400 tx=0x7f011400c8c0 comp rx=0 tx=0).stop 2024-09-16T07:32:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 shutdown_connections 2024-09-16T07:32:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0104079d20 0x7f010407c1e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01241437c0 0x7f0124141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01240b4bd0 0x7f01241f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 --2- 172.21.15.71:0/2361770725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01240b4400 0x7f01240aeea0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 >> 172.21.15.71:0/2361770725 conn(0x7f01240b1840 msgr2=0x7f01240d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.587+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 shutdown_connections 2024-09-16T07:32:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.588+0000 7f0128a00640 1 -- 172.21.15.71:0/2361770725 wait complete. 2024-09-16T07:32:08.611 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone673111 2024-09-16T07:32:08.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.908+0000 7f6fe2200640 1 -- 172.21.15.71:0/360620027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc15e110 msgr2=0x7f6fdc0cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:08.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.908+0000 7f6fe2200640 1 --2- 172.21.15.71:0/360620027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc15e110 0x7f6fdc0cf490 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0cfed0 tx=0x7f6fd002a800 comp rx=0 tx=0).stop 2024-09-16T07:32:08.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.908+0000 7f6fe2200640 1 -- 172.21.15.71:0/360620027 shutdown_connections 2024-09-16T07:32:08.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.908+0000 7f6fe2200640 1 --2- 172.21.15.71:0/360620027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc15e110 0x7f6fdc0cf490 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:08.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.908+0000 7f6fe2200640 1 -- 172.21.15.71:0/360620027 >> 172.21.15.71:0/360620027 conn(0x7f6fdc09d0e0 msgr2=0x7f6fdc0d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:08.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.908+0000 7f6fe2200640 1 -- 172.21.15.71:0/360620027 shutdown_connections 2024-09-16T07:32:08.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.908+0000 7f6fe2200640 1 -- 172.21.15.71:0/360620027 wait complete. 2024-09-16T07:32:08.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.911+0000 7f6fe2200640 1 Processor -- start 2024-09-16T07:32:08.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.912+0000 7f6fe2200640 1 -- start start 2024-09-16T07:32:08.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fe2200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fdc0d0f90 0x7f6fdc0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:08.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fe2200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc0d47b0 0x7f6fdc1f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:08.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fe2200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc15e110 0x7f6fdc155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:08.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fe2200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b8640 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fe2200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b84c0 con 0x7f6fdc15e110 2024-09-16T07:32:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fe2200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b87c0 con 0x7f6fdc0d0f90 2024-09-16T07:32:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc0d47b0 0x7f6fdc1f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc0d47b0 0x7f6fdc1f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47306/0 (socket says 172.21.15.71:47306) 2024-09-16T07:32:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.913+0000 7f6fdbe00640 1 -- 172.21.15.71:0/456596906 learned_addr learned my addr 172.21.15.71:0/456596906 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.914+0000 7f6fdbe00640 1 -- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fdc0d0f90 msgr2=0x7f6fdc0c76c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.914+0000 7f6fdbe00640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fdc0d0f90 0x7f6fdc0c76c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.914+0000 7f6fdbe00640 1 -- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc15e110 msgr2=0x7f6fdc155290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.914+0000 7f6fdbe00640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc15e110 0x7f6fdc155290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.914+0000 7f6fdbe00640 1 -- 172.21.15.71:0/456596906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc1f5430 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.915+0000 7f6fdbe00640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc0d47b0 0x7f6fdc1f4ef0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800c500 tx=0x7f6fc800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.915+0000 7f6fd8c00640 1 -- 172.21.15.71:0/456596906 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8012830 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.915+0000 7f6fd8c00640 1 -- 172.21.15.71:0/456596906 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fc8017070 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.915+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc1f56a0 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.915+0000 7f6fd8c00640 1 -- 172.21.15.71:0/456596906 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8012d60 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.915+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc1f5c60 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.917+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc0cf490 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.920+0000 7f6fd8c00640 1 -- 172.21.15.71:0/456596906 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f6fc801c460 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.921+0000 7f6fd8c00640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fbc079d20 0x7f6fbc07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.921+0000 7f6fd8c00640 1 -- 172.21.15.71:0/456596906 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f6fc8099880 con 0x7f6fdc0d47b0 2024-09-16T07:32:08.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.923+0000 7f6fe0e00640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fbc079d20 0x7f6fbc07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:08.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.930+0000 7f6fe0e00640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fbc079d20 0x7f6fbc07c1e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0034000 tx=0x7f6fd0004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:08.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:08.941+0000 7f6fd8c00640 1 -- 172.21.15.71:0/456596906 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6fc80634e0 con 0x7f6fdc0d47b0 2024-09-16T07:32:09.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.369+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone673111", "target": ["mon-mgr", ""]}) -- 0x7f6fdc0cacd0 con 0x7f6fbc079d20 2024-09-16T07:32:09.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.674+0000 7f6fd8c00640 1 -- 172.21.15.71:0/456596906 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6fdc0cacd0 con 0x7f6fbc079d20 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fbc079d20 msgr2=0x7f6fbc07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fbc079d20 0x7f6fbc07c1e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0034000 tx=0x7f6fd0004ab0 comp rx=0 tx=0).stop 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc0d47b0 msgr2=0x7f6fdc1f4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc0d47b0 0x7f6fdc1f4ef0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800c500 tx=0x7f6fc800c9c0 comp rx=0 tx=0).stop 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 shutdown_connections 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fbc079d20 0x7f6fbc07c1e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc15e110 0x7f6fdc155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fdc0d47b0 0x7f6fdc1f4ef0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 --2- 172.21.15.71:0/456596906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fdc0d0f90 0x7f6fdc0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 >> 172.21.15.71:0/456596906 conn(0x7f6fdc09d0e0 msgr2=0x7f6fdc06e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 shutdown_connections 2024-09-16T07:32:09.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:09.679+0000 7f6fe2200640 1 -- 172.21.15.71:0/456596906 wait complete. 2024-09-16T07:32:09.712 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone219912 2024-09-16T07:32:10.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.003+0000 7f724ba00640 1 -- 172.21.15.71:0/544368631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f722c004800 msgr2=0x7f722c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:10.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.003+0000 7f724ba00640 1 --2- 172.21.15.71:0/544368631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f722c004800 0x7f722c004c80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f7230001670 tx=0x7f722c005830 comp rx=0 tx=0).stop 2024-09-16T07:32:10.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.003+0000 7f724ba00640 1 -- 172.21.15.71:0/544368631 shutdown_connections 2024-09-16T07:32:10.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.003+0000 7f724ba00640 1 --2- 172.21.15.71:0/544368631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f722c004800 0x7f722c004c80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:10.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.003+0000 7f724ba00640 1 -- 172.21.15.71:0/544368631 >> 172.21.15.71:0/544368631 conn(0x7f72440b1840 msgr2=0x7f72440b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:10.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.004+0000 7f724ba00640 1 -- 172.21.15.71:0/544368631 shutdown_connections 2024-09-16T07:32:10.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.004+0000 7f724ba00640 1 -- 172.21.15.71:0/544368631 wait complete. 2024-09-16T07:32:10.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.007+0000 7f724ba00640 1 Processor -- start 2024-09-16T07:32:10.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.007+0000 7f724ba00640 1 -- start start 2024-09-16T07:32:10.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.009+0000 7f724ba00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c004800 0x7f72441fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:10.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.009+0000 7f724ba00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724413d8c0 0x7f724413dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:10.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.009+0000 7f724ba00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72440aa570 0x7f72440aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:10.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.009+0000 7f724ba00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f72440cc5c0 con 0x7f72440aa570 2024-09-16T07:32:10.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.009+0000 7f724ba00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f72440736a0 con 0x7f722c004800 2024-09-16T07:32:10.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.009+0000 7f724ba00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f72440cc740 con 0x7f724413d8c0 2024-09-16T07:32:10.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.009+0000 7f7249a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72440aa570 0x7f72440aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:10.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.010+0000 7f7249a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72440aa570 0x7f72440aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47324/0 (socket says 172.21.15.71:47324) 2024-09-16T07:32:10.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.010+0000 7f7249a00640 1 -- 172.21.15.71:0/2761647277 learned_addr learned my addr 172.21.15.71:0/2761647277 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:10.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.010+0000 7f7249000640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c004800 0x7f72441fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:10.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.010+0000 7f7243e00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724413d8c0 0x7f724413dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:10.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.011+0000 7f7249a00640 1 -- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724413d8c0 msgr2=0x7f724413dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:10.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.011+0000 7f7249a00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724413d8c0 0x7f724413dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:10.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.011+0000 7f7249a00640 1 -- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c004800 msgr2=0x7f72441fb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:10.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.011+0000 7f7249a00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c004800 0x7f72441fb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:10.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.011+0000 7f7249a00640 1 -- 172.21.15.71:0/2761647277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72441f8dc0 con 0x7f72440aa570 2024-09-16T07:32:10.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.011+0000 7f7249a00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72440aa570 0x7f72440aa9d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f723c002230 tx=0x7f723c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:10.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.012+0000 7f7240c00640 1 -- 172.21.15.71:0/2761647277 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c00cc20 con 0x7f72440aa570 2024-09-16T07:32:10.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.012+0000 7f7240c00640 1 -- 172.21.15.71:0/2761647277 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f723c017070 con 0x7f72440aa570 2024-09-16T07:32:10.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.012+0000 7f7240c00640 1 -- 172.21.15.71:0/2761647277 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c0116c0 con 0x7f72440aa570 2024-09-16T07:32:10.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.012+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72440aaf10 con 0x7f72440aa570 2024-09-16T07:32:10.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.012+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724413e780 con 0x7f72440aa570 2024-09-16T07:32:10.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.015+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724415f9f0 con 0x7f72440aa570 2024-09-16T07:32:10.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.017+0000 7f7240c00640 1 -- 172.21.15.71:0/2761647277 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f723c011860 con 0x7f72440aa570 2024-09-16T07:32:10.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.018+0000 7f7240c00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7224079da0 0x7f722407c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:10.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.018+0000 7f7240c00640 1 -- 172.21.15.71:0/2761647277 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f723c099ad0 con 0x7f72440aa570 2024-09-16T07:32:10.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.025+0000 7f7240c00640 1 -- 172.21.15.71:0/2761647277 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f72440aaf10 con 0x7f72440aa570 2024-09-16T07:32:10.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.069+0000 7f7249000640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7224079da0 0x7f722407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:10.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.163+0000 7f7249000640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7224079da0 0x7f722407c260 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7238003700 tx=0x7f723800b260 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:10.412+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone219912", "target": ["mon-mgr", ""]}) -- 0x7f72440afb30 con 0x7f7224079da0 2024-09-16T07:32:11.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.495+0000 7f7240c00640 1 -- 172.21.15.71:0/2761647277 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f72440afb30 con 0x7f7224079da0 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7224079da0 msgr2=0x7f722407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7224079da0 0x7f722407c260 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7238003700 tx=0x7f723800b260 comp rx=0 tx=0).stop 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72440aa570 msgr2=0x7f72440aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72440aa570 0x7f72440aa9d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f723c002230 tx=0x7f723c00ed30 comp rx=0 tx=0).stop 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 shutdown_connections 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7224079da0 0x7f722407c260 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72440aa570 0x7f72440aa9d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f724413d8c0 0x7f724413dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:11.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 --2- 172.21.15.71:0/2761647277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c004800 0x7f72441fb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:11.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 >> 172.21.15.71:0/2761647277 conn(0x7f72440b1840 msgr2=0x7f72440cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:11.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.499+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 shutdown_connections 2024-09-16T07:32:11.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.500+0000 7f724ba00640 1 -- 172.21.15.71:0/2761647277 wait complete. 2024-09-16T07:32:11.530 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone594713 2024-09-16T07:32:11.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.837+0000 7fe874000640 1 -- 172.21.15.71:0/466912370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe854003fe0 msgr2=0x7fe854004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:11.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.837+0000 7fe874000640 1 --2- 172.21.15.71:0/466912370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe854003fe0 0x7fe854004800 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe858009b50 tx=0x7fe854005830 comp rx=0 tx=0).stop 2024-09-16T07:32:11.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.838+0000 7fe872000640 1 -- 172.21.15.71:0/466912370 reap_dead start 2024-09-16T07:32:11.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.838+0000 7fe874000640 1 -- 172.21.15.71:0/466912370 shutdown_connections 2024-09-16T07:32:11.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.838+0000 7fe874000640 1 -- 172.21.15.71:0/466912370 >> 172.21.15.71:0/466912370 conn(0x7fe86c0b1840 msgr2=0x7fe86c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:11.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.838+0000 7fe874000640 1 -- 172.21.15.71:0/466912370 shutdown_connections 2024-09-16T07:32:11.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.838+0000 7fe874000640 1 -- 172.21.15.71:0/466912370 wait complete. 2024-09-16T07:32:11.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.841+0000 7fe874000640 1 Processor -- start 2024-09-16T07:32:11.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.842+0000 7fe874000640 1 -- start start 2024-09-16T07:32:11.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.843+0000 7fe874000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe854003fe0 0x7fe86c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:11.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.844+0000 7fe874000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe86c13e990 0x7fe86c150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:11.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.844+0000 7fe874000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe86c13ee10 0x7fe86c151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:11.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.844+0000 7fe874000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c0cdc50 con 0x7fe86c13e990 2024-09-16T07:32:11.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.844+0000 7fe874000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c0cdad0 con 0x7fe86c13ee10 2024-09-16T07:32:11.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.844+0000 7fe874000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe86c0cddd0 con 0x7fe854003fe0 2024-09-16T07:32:11.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.844+0000 7fe870c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe86c13e990 0x7fe86c150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:11.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.844+0000 7fe870c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe86c13e990 0x7fe86c150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47346/0 (socket says 172.21.15.71:47346) 2024-09-16T07:32:11.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.844+0000 7fe870c00640 1 -- 172.21.15.71:0/3133260308 learned_addr learned my addr 172.21.15.71:0/3133260308 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:11.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.845+0000 7fe870c00640 1 -- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe854003fe0 msgr2=0x7fe86c0b06d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:32:11.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.845+0000 7fe870c00640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe854003fe0 0x7fe86c0b06d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:11.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.845+0000 7fe870c00640 1 -- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe86c13ee10 msgr2=0x7fe86c151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:11.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.845+0000 7fe870c00640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe86c13ee10 0x7fe86c151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:11.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.845+0000 7fe870c00640 1 -- 172.21.15.71:0/3133260308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c153490 con 0x7fe86c13e990 2024-09-16T07:32:11.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.846+0000 7fe870c00640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe86c13e990 0x7fe86c150f50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe858036040 tx=0x7fe858033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:11.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.847+0000 7fe869600640 1 -- 172.21.15.71:0/3133260308 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858043070 con 0x7fe86c13e990 2024-09-16T07:32:11.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.847+0000 7fe869600640 1 -- 172.21.15.71:0/3133260308 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe858004320 con 0x7fe86c13e990 2024-09-16T07:32:11.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.847+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c13f370 con 0x7fe86c13e990 2024-09-16T07:32:11.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.848+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c153730 con 0x7fe86c13e990 2024-09-16T07:32:11.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.848+0000 7fe869600640 1 -- 172.21.15.71:0/3133260308 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858038040 con 0x7fe86c13e990 2024-09-16T07:32:11.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.850+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe848005290 con 0x7fe86c13e990 2024-09-16T07:32:11.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.851+0000 7fe869600640 1 -- 172.21.15.71:0/3133260308 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7fe858046070 con 0x7fe86c13e990 2024-09-16T07:32:11.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.853+0000 7fe869600640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe84c079d50 0x7fe84c07c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:11.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.854+0000 7fe869600640 1 -- 172.21.15.71:0/3133260308 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fe8580b9880 con 0x7fe86c13e990 2024-09-16T07:32:11.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.861+0000 7fe869600640 1 -- 172.21.15.71:0/3133260308 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe858083350 con 0x7fe86c13e990 2024-09-16T07:32:11.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.866+0000 7fe871600640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe84c079d50 0x7fe84c07c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:11.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:11.917+0000 7fe871600640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe84c079d50 0x7fe84c07c210 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe860003c40 tx=0x7fe8600079b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:12.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.235+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone594713", "target": ["mon-mgr", ""]}) -- 0x7fe848000c80 con 0x7fe84c079d50 2024-09-16T07:32:12.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.948+0000 7fe869600640 1 -- 172.21.15.71:0/3133260308 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe848000c80 con 0x7fe84c079d50 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe84c079d50 msgr2=0x7fe84c07c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe84c079d50 0x7fe84c07c210 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe860003c40 tx=0x7fe8600079b0 comp rx=0 tx=0).stop 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe86c13e990 msgr2=0x7fe86c150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe86c13e990 0x7fe86c150f50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe858036040 tx=0x7fe858033670 comp rx=0 tx=0).stop 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 shutdown_connections 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe84c079d50 0x7fe84c07c210 secure :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe860003c40 tx=0x7fe8600079b0 comp rx=0 tx=0).stop 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe86c13ee10 0x7fe86c151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe86c13e990 0x7fe86c150f50 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:12.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 --2- 172.21.15.71:0/3133260308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe854003fe0 0x7fe86c0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:12.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.951+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 >> 172.21.15.71:0/3133260308 conn(0x7fe86c0b1840 msgr2=0x7fe86c0ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:12.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.952+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 shutdown_connections 2024-09-16T07:32:12.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:12.952+0000 7fe874000640 1 -- 172.21.15.71:0/3133260308 wait complete. 2024-09-16T07:32:12.983 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone43314 2024-09-16T07:32:13.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.310+0000 7f7c1ee00640 1 -- 172.21.15.71:0/3207391699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 msgr2=0x7f7c181620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:13.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.311+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/3207391699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 0x7f7c181620f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f7c00005fd0 tx=0x7f7c0002a4b0 comp rx=0 tx=0).stop 2024-09-16T07:32:13.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.311+0000 7f7c1ee00640 1 -- 172.21.15.71:0/3207391699 shutdown_connections 2024-09-16T07:32:13.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.311+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/3207391699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 0x7f7c181620f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:13.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.311+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/3207391699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1806fba0 0x7f7c18161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:13.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.311+0000 7f7c1ee00640 1 -- 172.21.15.71:0/3207391699 >> 172.21.15.71:0/3207391699 conn(0x7f7c180b1840 msgr2=0x7f7c180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:13.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.311+0000 7f7c1ee00640 1 -- 172.21.15.71:0/3207391699 shutdown_connections 2024-09-16T07:32:13.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.311+0000 7f7c1ee00640 1 -- 172.21.15.71:0/3207391699 wait complete. 2024-09-16T07:32:13.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.314+0000 7f7c1ee00640 1 Processor -- start 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.315+0000 7f7c1ee00640 1 -- start start 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.316+0000 7f7c1ee00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1806fba0 0x7f7c18201e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.316+0000 7f7c1ee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 0x7f7c1815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.316+0000 7f7c1ee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c180b2b40 0x7f7c1815ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.317+0000 7f7c1ee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18070100 con 0x7f7c180af170 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.317+0000 7f7c1ee00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1806ff80 con 0x7f7c180b2b40 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.317+0000 7f7c1ee00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7c18070280 con 0x7f7c1806fba0 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.317+0000 7f7c0f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 0x7f7c1815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.317+0000 7f7c0f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 0x7f7c1815c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47356/0 (socket says 172.21.15.71:47356) 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.317+0000 7f7c0f400640 1 -- 172.21.15.71:0/1325769204 learned_addr learned my addr 172.21.15.71:0/1325769204 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.318+0000 7f7c0f400640 1 -- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1806fba0 msgr2=0x7f7c18201e60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:32:13.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.318+0000 7f7c0f400640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1806fba0 0x7f7c18201e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:13.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.318+0000 7f7c0f400640 1 -- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c180b2b40 msgr2=0x7f7c1815ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:13.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.318+0000 7f7c0f400640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c180b2b40 0x7f7c1815ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:13.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.318+0000 7f7c0f400640 1 -- 172.21.15.71:0/1325769204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c18160e20 con 0x7f7c180af170 2024-09-16T07:32:13.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.319+0000 7f7c0f400640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 0x7f7c1815c450 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7c00000c00 tx=0x7f7c000326c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:13.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.319+0000 7f7c14c00640 1 -- 172.21.15.71:0/1325769204 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0003f070 con 0x7f7c180af170 2024-09-16T07:32:13.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.319+0000 7f7c14c00640 1 -- 172.21.15.71:0/1325769204 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c000328b0 con 0x7f7c180af170 2024-09-16T07:32:13.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.319+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c181610a0 con 0x7f7c180af170 2024-09-16T07:32:13.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.319+0000 7f7c14c00640 1 -- 172.21.15.71:0/1325769204 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00035030 con 0x7f7c180af170 2024-09-16T07:32:13.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.320+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c18161660 con 0x7f7c180af170 2024-09-16T07:32:13.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.324+0000 7f7c14c00640 1 -- 172.21.15.71:0/1325769204 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f7c00042070 con 0x7f7c180af170 2024-09-16T07:32:13.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.326+0000 7f7c14c00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7bf4079d20 0x7f7bf407c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:13.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.326+0000 7f7c14c00640 1 -- 172.21.15.71:0/1325769204 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f7c000ba9a0 con 0x7f7c180af170 2024-09-16T07:32:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.328+0000 7f7c17e00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7bf4079d20 0x7f7bf407c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:13.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.336+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c18158060 con 0x7f7c180af170 2024-09-16T07:32:13.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.346+0000 7f7c14c00640 1 -- 172.21.15.71:0/1325769204 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7c18160e20 con 0x7f7c180af170 2024-09-16T07:32:13.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.353+0000 7f7c17e00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7bf4079d20 0x7f7bf407c1e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800b990 tx=0x7f7c0800b370 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:13.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:13.725+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone43314", "target": ["mon-mgr", ""]}) -- 0x7f7c180b1f00 con 0x7f7bf4079d20 2024-09-16T07:32:14.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.889+0000 7f7c14c00640 1 -- 172.21.15.71:0/1325769204 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c180b1f00 con 0x7f7bf4079d20 2024-09-16T07:32:14.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7bf4079d20 msgr2=0x7f7bf407c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:14.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7bf4079d20 0x7f7bf407c1e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800b990 tx=0x7f7c0800b370 comp rx=0 tx=0).stop 2024-09-16T07:32:14.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 msgr2=0x7f7c1815c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:14.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 0x7f7c1815c450 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7c00000c00 tx=0x7f7c000326c0 comp rx=0 tx=0).stop 2024-09-16T07:32:14.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 shutdown_connections 2024-09-16T07:32:14.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7bf4079d20 0x7f7bf407c1e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:14.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c180b2b40 0x7f7c1815ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:14.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c180af170 0x7f7c1815c450 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:14.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 --2- 172.21.15.71:0/1325769204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1806fba0 0x7f7c18201e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:14.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 >> 172.21.15.71:0/1325769204 conn(0x7f7c180b1840 msgr2=0x7f7c18072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:14.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 shutdown_connections 2024-09-16T07:32:14.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:14.893+0000 7f7c1ee00640 1 -- 172.21.15.71:0/1325769204 wait complete. 2024-09-16T07:32:14.920 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone644815 2024-09-16T07:32:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.263+0000 7efd77200640 1 -- 172.21.15.71:0/2182411759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd780cdc60 msgr2=0x7efd7806f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.263+0000 7efd77200640 1 --2- 172.21.15.71:0/2182411759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd780cdc60 0x7efd7806f270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efd70005fd0 tx=0x7efd7002a970 comp rx=0 tx=0).stop 2024-09-16T07:32:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.263+0000 7efd77200640 1 -- 172.21.15.71:0/2182411759 shutdown_connections 2024-09-16T07:32:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.263+0000 7efd77200640 1 --2- 172.21.15.71:0/2182411759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd780cdc60 0x7efd7806f270 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.263+0000 7efd77200640 1 --2- 172.21.15.71:0/2182411759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd78070730 0x7efd780ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.263+0000 7efd77200640 1 -- 172.21.15.71:0/2182411759 >> 172.21.15.71:0/2182411759 conn(0x7efd780cc490 msgr2=0x7efd780cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.263+0000 7efd77200640 1 -- 172.21.15.71:0/2182411759 shutdown_connections 2024-09-16T07:32:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.263+0000 7efd77200640 1 -- 172.21.15.71:0/2182411759 wait complete. 2024-09-16T07:32:15.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.266+0000 7efd77200640 1 Processor -- start 2024-09-16T07:32:15.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.266+0000 7efd77200640 1 -- start start 2024-09-16T07:32:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.267+0000 7efd77200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78070730 0x7efd781f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.267+0000 7efd77200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780cdc60 0x7efd781f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.267+0000 7efd77200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd781430b0 0x7efd7813ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.267+0000 7efd77200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd780c5970 con 0x7efd780cdc60 2024-09-16T07:32:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.267+0000 7efd77200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd780c57f0 con 0x7efd78070730 2024-09-16T07:32:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.267+0000 7efd77200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efd780c5af0 con 0x7efd781430b0 2024-09-16T07:32:15.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.267+0000 7efd75400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780cdc60 0x7efd781f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:15.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.267+0000 7efd75400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780cdc60 0x7efd781f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41604/0 (socket says 172.21.15.71:41604) 2024-09-16T07:32:15.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.268+0000 7efd75400640 1 -- 172.21.15.71:0/3839254683 learned_addr learned my addr 172.21.15.71:0/3839254683 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:15.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.268+0000 7efd75400640 1 -- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd781430b0 msgr2=0x7efd7813ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:15.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.268+0000 7efd75400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd781430b0 0x7efd7813ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:15.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.268+0000 7efd75400640 1 -- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78070730 msgr2=0x7efd781f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:15.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.268+0000 7efd75400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78070730 0x7efd781f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:15.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.268+0000 7efd75400640 1 -- 172.21.15.71:0/3839254683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780733c0 con 0x7efd780cdc60 2024-09-16T07:32:15.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.269+0000 7efd75400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780cdc60 0x7efd781f53d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b410 tx=0x7efd6400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:15.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.269+0000 7efd6e000640 1 -- 172.21.15.71:0/3839254683 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64015020 con 0x7efd780cdc60 2024-09-16T07:32:15.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.269+0000 7efd6e000640 1 -- 172.21.15.71:0/3839254683 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd64011920 con 0x7efd780cdc60 2024-09-16T07:32:15.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.269+0000 7efd77200640 1 -- 172.21.15.71:0/3839254683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd780736a0 con 0x7efd780cdc60 2024-09-16T07:32:15.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.269+0000 7efd6e000640 1 -- 172.21.15.71:0/3839254683 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64011d70 con 0x7efd780cdc60 2024-09-16T07:32:15.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.269+0000 7efd77200640 1 -- 172.21.15.71:0/3839254683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd78073c60 con 0x7efd780cdc60 2024-09-16T07:32:15.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.271+0000 7efd63400640 1 -- 172.21.15.71:0/3839254683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd50005290 con 0x7efd780cdc60 2024-09-16T07:32:15.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.273+0000 7efd6e000640 1 -- 172.21.15.71:0/3839254683 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7efd64018020 con 0x7efd780cdc60 2024-09-16T07:32:15.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.274+0000 7efd6e000640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd58079d20 0x7efd5807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:15.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.274+0000 7efd6e000640 1 -- 172.21.15.71:0/3839254683 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7efd640995a0 con 0x7efd780cdc60 2024-09-16T07:32:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.280+0000 7efd6e000640 1 -- 172.21.15.71:0/3839254683 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efd6401f020 con 0x7efd780cdc60 2024-09-16T07:32:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.294+0000 7efd75e00640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd58079d20 0x7efd5807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:15.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.304+0000 7efd75e00640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd58079d20 0x7efd5807c1e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd68005fd0 tx=0x7efd68005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:15.534 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:32:15.542 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:32:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:15.710+0000 7efd63400640 1 -- 172.21.15.71:0/3839254683 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone644815", "target": ["mon-mgr", ""]}) -- 0x7efd50000c80 con 0x7efd58079d20 2024-09-16T07:32:17.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.400+0000 7efd6e000640 1 -- 172.21.15.71:0/3839254683 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd50000c80 con 0x7efd58079d20 2024-09-16T07:32:17.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.409+0000 7efd63400640 1 -- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd58079d20 msgr2=0x7efd5807c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:17.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.409+0000 7efd63400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd58079d20 0x7efd5807c1e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd68005fd0 tx=0x7efd68005850 comp rx=0 tx=0).stop 2024-09-16T07:32:17.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.409+0000 7efd63400640 1 -- 172.21.15.71:0/3839254683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780cdc60 msgr2=0x7efd781f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:17.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.409+0000 7efd63400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780cdc60 0x7efd781f53d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b410 tx=0x7efd6400b8d0 comp rx=0 tx=0).stop 2024-09-16T07:32:17.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.411+0000 7efd63400640 1 -- 172.21.15.71:0/3839254683 shutdown_connections 2024-09-16T07:32:17.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.411+0000 7efd63400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efd58079d20 0x7efd5807c1e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:17.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.411+0000 7efd63400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd781430b0 0x7efd7813ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:17.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.411+0000 7efd63400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780cdc60 0x7efd781f53d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:17.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.411+0000 7efd63400640 1 --2- 172.21.15.71:0/3839254683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78070730 0x7efd781f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:17.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.411+0000 7efd63400640 1 -- 172.21.15.71:0/3839254683 >> 172.21.15.71:0/3839254683 conn(0x7efd780cc490 msgr2=0x7efd780b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:17.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.414+0000 7efd63400640 1 -- 172.21.15.71:0/3839254683 shutdown_connections 2024-09-16T07:32:17.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.415+0000 7efd63400640 1 -- 172.21.15.71:0/3839254683 wait complete. 2024-09-16T07:32:17.453 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone418816 2024-09-16T07:32:17.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.765+0000 7feb1f000640 1 -- 172.21.15.71:0/3006782316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feafc0043b0 msgr2=0x7feafc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.765+0000 7feb1f000640 1 --2- 172.21.15.71:0/3006782316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feafc0043b0 0x7feafc004800 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7feb1815c490 tx=0x7feafc005830 comp rx=0 tx=0).stop 2024-09-16T07:32:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.766+0000 7feb1f000640 1 -- 172.21.15.71:0/3006782316 shutdown_connections 2024-09-16T07:32:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.766+0000 7feb1f000640 1 --2- 172.21.15.71:0/3006782316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feafc0043b0 0x7feafc004800 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.766+0000 7feb1f000640 1 -- 172.21.15.71:0/3006782316 >> 172.21.15.71:0/3006782316 conn(0x7feb180b1840 msgr2=0x7feb180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.766+0000 7feb1f000640 1 -- 172.21.15.71:0/3006782316 shutdown_connections 2024-09-16T07:32:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.766+0000 7feb1f000640 1 -- 172.21.15.71:0/3006782316 wait complete. 2024-09-16T07:32:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.768+0000 7feb1f000640 1 Processor -- start 2024-09-16T07:32:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.768+0000 7feb1f000640 1 -- start start 2024-09-16T07:32:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.769+0000 7feb1f000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feafc0043b0 0x7feb1813d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.769+0000 7feb1f000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb18155060 0x7feb1815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.769+0000 7feb1f000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb181554e0 0x7feb1815ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.769+0000 7feb1f000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb1815ed60 con 0x7feb181554e0 2024-09-16T07:32:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.769+0000 7feb1f000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feb1815ebe0 con 0x7feb18155060 2024-09-16T07:32:17.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.769+0000 7feb1f000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feb1815eee0 con 0x7feafc0043b0 2024-09-16T07:32:17.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.770+0000 7feb1d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb181554e0 0x7feb1815ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:17.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.770+0000 7feb1d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb181554e0 0x7feb1815ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41630/0 (socket says 172.21.15.71:41630) 2024-09-16T07:32:17.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.770+0000 7feb1d000640 1 -- 172.21.15.71:0/2400621040 learned_addr learned my addr 172.21.15.71:0/2400621040 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:17.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.770+0000 7feb1d000640 1 -- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feafc0043b0 msgr2=0x7feb1813d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:17.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.770+0000 7feb1d000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feafc0043b0 0x7feb1813d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:17.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.770+0000 7feb1d000640 1 -- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb18155060 msgr2=0x7feb1815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:17.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.770+0000 7feb1d000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb18155060 0x7feb1815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:17.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.770+0000 7feb1d000640 1 -- 172.21.15.71:0/2400621040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb181f3a00 con 0x7feb181554e0 2024-09-16T07:32:17.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.771+0000 7feb1d000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb181554e0 0x7feb1815ca00 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e890 tx=0x7feb1000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:17.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.771+0000 7feb0fe00640 1 -- 172.21.15.71:0/2400621040 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10004b20 con 0x7feb181554e0 2024-09-16T07:32:17.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.771+0000 7feb0fe00640 1 -- 172.21.15.71:0/2400621040 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb10004dd0 con 0x7feb181554e0 2024-09-16T07:32:17.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.771+0000 7feb0fe00640 1 -- 172.21.15.71:0/2400621040 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000ccc0 con 0x7feb181554e0 2024-09-16T07:32:17.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.771+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb181f3ce0 con 0x7feb181554e0 2024-09-16T07:32:17.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.771+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb181f42a0 con 0x7feb181554e0 2024-09-16T07:32:17.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.774+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb180cde40 con 0x7feb181554e0 2024-09-16T07:32:17.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.775+0000 7feb0fe00640 1 -- 172.21.15.71:0/2400621040 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7feb1001b020 con 0x7feb181554e0 2024-09-16T07:32:17.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.777+0000 7feb0fe00640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaf8079d20 0x7feaf807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:17.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.777+0000 7feb0fe00640 1 -- 172.21.15.71:0/2400621040 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7feb10099260 con 0x7feb181554e0 2024-09-16T07:32:17.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.782+0000 7feb0fe00640 1 -- 172.21.15.71:0/2400621040 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feb1000ce60 con 0x7feb181554e0 2024-09-16T07:32:17.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.811+0000 7feb17e00640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaf8079d20 0x7feaf807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:17.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:17.842+0000 7feb17e00640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaf8079d20 0x7feaf807c1e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feb080058c0 tx=0x7feb08005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:18.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:18.261+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone418816", "target": ["mon-mgr", ""]}) -- 0x7feb1806f3b0 con 0x7feaf8079d20 2024-09-16T07:32:20.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.231+0000 7feb0fe00640 1 -- 172.21.15.71:0/2400621040 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feb1806f3b0 con 0x7feaf8079d20 2024-09-16T07:32:20.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaf8079d20 msgr2=0x7feaf807c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:20.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaf8079d20 0x7feaf807c1e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feb080058c0 tx=0x7feb08005850 comp rx=0 tx=0).stop 2024-09-16T07:32:20.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb181554e0 msgr2=0x7feb1815ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:20.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb181554e0 0x7feb1815ca00 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e890 tx=0x7feb1000ed50 comp rx=0 tx=0).stop 2024-09-16T07:32:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 shutdown_connections 2024-09-16T07:32:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaf8079d20 0x7feaf807c1e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb181554e0 0x7feb1815ca00 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb18155060 0x7feb1815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 --2- 172.21.15.71:0/2400621040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feafc0043b0 0x7feb1813d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.234+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 >> 172.21.15.71:0/2400621040 conn(0x7feb180b1840 msgr2=0x7feb1806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.235+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 shutdown_connections 2024-09-16T07:32:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.235+0000 7feb1f000640 1 -- 172.21.15.71:0/2400621040 wait complete. 2024-09-16T07:32:20.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99717 2024-09-16T07:32:20.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 -- 172.21.15.71:0/1123587374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c145750 msgr2=0x7fb22c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 --2- 172.21.15.71:0/1123587374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c145750 0x7fb22c073440 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0af750 tx=0x7fb21c02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 -- 172.21.15.71:0/1123587374 shutdown_connections 2024-09-16T07:32:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 --2- 172.21.15.71:0/1123587374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c0ccb00 0x7fb22c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 --2- 172.21.15.71:0/1123587374 >> v2:172.21.15.177:3300/0 conn(0x7fb22c06efa0 0x7fb22c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 --2- 172.21.15.71:0/1123587374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c145750 0x7fb22c073440 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 -- 172.21.15.71:0/1123587374 >> 172.21.15.71:0/1123587374 conn(0x7fb22c0b1840 msgr2=0x7fb22c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 -- 172.21.15.71:0/1123587374 shutdown_connections 2024-09-16T07:32:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.517+0000 7fb233200640 1 -- 172.21.15.71:0/1123587374 wait complete. 2024-09-16T07:32:20.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.520+0000 7fb233200640 1 Processor -- start 2024-09-16T07:32:20.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.520+0000 7fb233200640 1 -- start start 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb233200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c06efa0 0x7fb22c13e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb233200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c0ccb00 0x7fb22c1f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb233200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c145750 0x7fb22c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb233200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c1602c0 con 0x7fb22c06efa0 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb233200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c160140 con 0x7fb22c145750 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb233200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb22c160440 con 0x7fb22c0ccb00 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb22be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c06efa0 0x7fb22c13e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb22be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c06efa0 0x7fb22c13e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41638/0 (socket says 172.21.15.71:41638) 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb22be00640 1 -- 172.21.15.71:0/3031090425 learned_addr learned my addr 172.21.15.71:0/3031090425 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb231200640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c145750 0x7fb22c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.521+0000 7fb22b400640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c0ccb00 0x7fb22c1f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:20.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.522+0000 7fb22be00640 1 -- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c0ccb00 msgr2=0x7fb22c1f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.522+0000 7fb22be00640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c0ccb00 0x7fb22c1f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.522+0000 7fb22be00640 1 -- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c145750 msgr2=0x7fb22c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.522+0000 7fb22be00640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c145750 0x7fb22c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.522+0000 7fb22be00640 1 -- 172.21.15.71:0/3031090425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c1f4e40 con 0x7fb22c06efa0 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.522+0000 7fb231200640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c145750 0x7fb22c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.522+0000 7fb22be00640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c06efa0 0x7fb22c13e530 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb21c034040 tx=0x7fb21c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.523+0000 7fb223e00640 1 -- 172.21.15.71:0/3031090425 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c03e030 con 0x7fb22c06efa0 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.523+0000 7fb223e00640 1 -- 172.21.15.71:0/3031090425 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb21c0047a0 con 0x7fb22c06efa0 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.523+0000 7fb223e00640 1 -- 172.21.15.71:0/3031090425 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c035040 con 0x7fb22c06efa0 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.523+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22c1f5120 con 0x7fb22c06efa0 2024-09-16T07:32:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.523+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22c13ed60 con 0x7fb22c06efa0 2024-09-16T07:32:20.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.524+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22c1f5640 con 0x7fb22c06efa0 2024-09-16T07:32:20.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.525+0000 7fb223e00640 1 -- 172.21.15.71:0/3031090425 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7fb21c041030 con 0x7fb22c06efa0 2024-09-16T07:32:20.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.526+0000 7fb223e00640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb20c079d20 0x7fb20c07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:20.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.526+0000 7fb223e00640 1 -- 172.21.15.71:0/3031090425 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fb21c0ba1a0 con 0x7fb22c06efa0 2024-09-16T07:32:20.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.533+0000 7fb223e00640 1 -- 172.21.15.71:0/3031090425 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb21c083e80 con 0x7fb22c06efa0 2024-09-16T07:32:20.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.559+0000 7fb22b400640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb20c079d20 0x7fb20c07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:20.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.578+0000 7fb22b400640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb20c079d20 0x7fb20c07c1e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb21800a810 tx=0x7fb218008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:20.929+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99717", "target": ["mon-mgr", ""]}) -- 0x7fb22c1f4be0 con 0x7fb20c079d20 2024-09-16T07:32:22.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.147+0000 7fb223e00640 1 -- 172.21.15.71:0/3031090425 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb21800adf0 con 0x7fb20c079d20 2024-09-16T07:32:22.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb20c079d20 msgr2=0x7fb20c07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb20c079d20 0x7fb20c07c1e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb21800a810 tx=0x7fb218008040 comp rx=0 tx=0).stop 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c06efa0 msgr2=0x7fb22c13e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c06efa0 0x7fb22c13e530 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb21c034040 tx=0x7fb21c0039e0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 shutdown_connections 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb20c079d20 0x7fb20c07c1e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb22c145750 0x7fb22c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb22c0ccb00 0x7fb22c1f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 --2- 172.21.15.71:0/3031090425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb22c06efa0 0x7fb22c13e530 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 >> 172.21.15.71:0/3031090425 conn(0x7fb22c0b1840 msgr2=0x7fb22c0cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 shutdown_connections 2024-09-16T07:32:22.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.151+0000 7fb233200640 1 -- 172.21.15.71:0/3031090425 wait complete. 2024-09-16T07:32:22.180 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone648918 2024-09-16T07:32:22.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.463+0000 7fe60e800640 1 -- 172.21.15.71:0/2620088011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080b8a60 msgr2=0x7fe6080d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:22.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.463+0000 7fe60e800640 1 --2- 172.21.15.71:0/2620088011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080b8a60 0x7fe6080d1190 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe600009be0 tx=0x7fe60002a8c0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.463+0000 7fe60e800640 1 -- 172.21.15.71:0/2620088011 shutdown_connections 2024-09-16T07:32:22.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.463+0000 7fe60e800640 1 --2- 172.21.15.71:0/2620088011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080b8a60 0x7fe6080d1190 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.463+0000 7fe60e800640 1 --2- 172.21.15.71:0/2620088011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6080d0870 0x7fe6080d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.463+0000 7fe60e800640 1 -- 172.21.15.71:0/2620088011 >> 172.21.15.71:0/2620088011 conn(0x7fe6080d3e60 msgr2=0x7fe6080d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:22.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.463+0000 7fe60e800640 1 -- 172.21.15.71:0/2620088011 shutdown_connections 2024-09-16T07:32:22.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.463+0000 7fe60e800640 1 -- 172.21.15.71:0/2620088011 wait complete. 2024-09-16T07:32:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.466+0000 7fe60e800640 1 Processor -- start 2024-09-16T07:32:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.466+0000 7fe60e800640 1 -- start start 2024-09-16T07:32:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60e800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6080b8a60 0x7fe60815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6080d0870 0x7fe6081f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080d1f00 0x7fe60814aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60e800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6080cf190 con 0x7fe6080d1f00 2024-09-16T07:32:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60e800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe6080cf010 con 0x7fe6080d0870 2024-09-16T07:32:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60e800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe6080cf310 con 0x7fe6080b8a60 2024-09-16T07:32:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080d1f00 0x7fe60814aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080d1f00 0x7fe60814aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41658/0 (socket says 172.21.15.71:41658) 2024-09-16T07:32:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.468+0000 7fe60de00640 1 -- 172.21.15.71:0/2475026990 learned_addr learned my addr 172.21.15.71:0/2475026990 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:22.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.469+0000 7fe60ca00640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6080d0870 0x7fe6081f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:22.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.469+0000 7fe60de00640 1 -- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6080b8a60 msgr2=0x7fe60815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:22.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.469+0000 7fe60de00640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6080b8a60 0x7fe60815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.469+0000 7fe60de00640 1 -- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6080d0870 msgr2=0x7fe6081f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:22.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.469+0000 7fe60de00640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6080d0870 0x7fe6081f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:22.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.469+0000 7fe60de00640 1 -- 172.21.15.71:0/2475026990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60815ceb0 con 0x7fe6080d1f00 2024-09-16T07:32:22.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.469+0000 7fe60ca00640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6080d0870 0x7fe6081f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:32:22.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.470+0000 7fe60de00640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080d1f00 0x7fe60814aaf0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe600009be0 tx=0x7fe600032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:22.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.471+0000 7fe605600640 1 -- 172.21.15.71:0/2475026990 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600040070 con 0x7fe6080d1f00 2024-09-16T07:32:22.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.471+0000 7fe605600640 1 -- 172.21.15.71:0/2475026990 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe600004540 con 0x7fe6080d1f00 2024-09-16T07:32:22.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.471+0000 7fe605600640 1 -- 172.21.15.71:0/2475026990 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600032e40 con 0x7fe6080d1f00 2024-09-16T07:32:22.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.471+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6081f5c40 con 0x7fe6080d1f00 2024-09-16T07:32:22.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.471+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60814b800 con 0x7fe6080d1f00 2024-09-16T07:32:22.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.473+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6080ce900 con 0x7fe6080d1f00 2024-09-16T07:32:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.485+0000 7fe605600640 1 -- 172.21.15.71:0/2475026990 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7fe600039040 con 0x7fe6080d1f00 2024-09-16T07:32:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.487+0000 7fe605600640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5e8079d20 0x7fe5e807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.488+0000 7fe605600640 1 -- 172.21.15.71:0/2475026990 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fe6000ba810 con 0x7fe6080d1f00 2024-09-16T07:32:22.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.493+0000 7fe605600640 1 -- 172.21.15.71:0/2475026990 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe60002e050 con 0x7fe6080d1f00 2024-09-16T07:32:22.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.498+0000 7fe60d400640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5e8079d20 0x7fe5e807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.568+0000 7fe60d400640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5e8079d20 0x7fe5e807c1e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009710 tx=0x7fe5fc006c90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:22.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:22.871+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone648918", "target": ["mon-mgr", ""]}) -- 0x7fe6080d0650 con 0x7fe5e8079d20 2024-09-16T07:32:24.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.293+0000 7fe605600640 1 -- 172.21.15.71:0/2475026990 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe6080d0650 con 0x7fe5e8079d20 2024-09-16T07:32:24.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5e8079d20 msgr2=0x7fe5e807c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:24.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5e8079d20 0x7fe5e807c1e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009710 tx=0x7fe5fc006c90 comp rx=0 tx=0).stop 2024-09-16T07:32:24.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080d1f00 msgr2=0x7fe60814aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:24.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080d1f00 0x7fe60814aaf0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe600009be0 tx=0x7fe600032c10 comp rx=0 tx=0).stop 2024-09-16T07:32:24.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 shutdown_connections 2024-09-16T07:32:24.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe5e8079d20 0x7fe5e807c1e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6080d1f00 0x7fe60814aaf0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6080d0870 0x7fe6081f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 --2- 172.21.15.71:0/2475026990 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6080b8a60 0x7fe60815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.298+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 >> 172.21.15.71:0/2475026990 conn(0x7fe6080d3e60 msgr2=0x7fe6080d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:24.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.299+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 shutdown_connections 2024-09-16T07:32:24.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.300+0000 7fe60e800640 1 -- 172.21.15.71:0/2475026990 wait complete. 2024-09-16T07:32:24.328 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone849419 2024-09-16T07:32:24.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.600+0000 7f58a1c00640 1 -- 172.21.15.71:0/3031872832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 msgr2=0x7f589c0c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:24.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.600+0000 7f58a1c00640 1 --2- 172.21.15.71:0/3031872832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 0x7f589c0c6430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f589c075410 tx=0x7f588c02a580 comp rx=0 tx=0).stop 2024-09-16T07:32:24.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.600+0000 7f58a1c00640 1 -- 172.21.15.71:0/3031872832 shutdown_connections 2024-09-16T07:32:24.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.600+0000 7f58a1c00640 1 --2- 172.21.15.71:0/3031872832 >> v2:172.21.15.177:3300/0 conn(0x7f589c15c0f0 0x7f589c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.601+0000 7f58a1c00640 1 --2- 172.21.15.71:0/3031872832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f589c0c5730 0x7f589c15fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.601+0000 7f58a1c00640 1 --2- 172.21.15.71:0/3031872832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 0x7f589c0c6430 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.601+0000 7f58a1c00640 1 -- 172.21.15.71:0/3031872832 >> 172.21.15.71:0/3031872832 conn(0x7f589c0cc490 msgr2=0x7f589c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:24.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.601+0000 7f58a1c00640 1 -- 172.21.15.71:0/3031872832 shutdown_connections 2024-09-16T07:32:24.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.601+0000 7f58a1c00640 1 -- 172.21.15.71:0/3031872832 wait complete. 2024-09-16T07:32:24.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.603+0000 7f58a1c00640 1 Processor -- start 2024-09-16T07:32:24.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.603+0000 7f58a1c00640 1 -- start start 2024-09-16T07:32:24.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f589c0c5730 0x7f589c1e6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:24.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f589c15c0f0 0x7f589c0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:24.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 0x7f589c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:24.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f589c0cd2c0 con 0x7f589c15f670 2024-09-16T07:32:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f589c0cd140 con 0x7f589c0c5730 2024-09-16T07:32:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f589c0ca370 con 0x7f589c15c0f0 2024-09-16T07:32:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 0x7f589c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 0x7f589c14fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41672/0 (socket says 172.21.15.71:41672) 2024-09-16T07:32:24.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.604+0000 7f58a1200640 1 -- 172.21.15.71:0/22471232 learned_addr learned my addr 172.21.15.71:0/22471232 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.605+0000 7f58a1200640 1 -- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f589c15c0f0 msgr2=0x7f589c0b19d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.605+0000 7f58a1200640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f589c15c0f0 0x7f589c0b19d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.605+0000 7f58a1200640 1 -- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f589c0c5730 msgr2=0x7f589c1e6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.605+0000 7f58a1200640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f589c0c5730 0x7f589c1e6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.605+0000 7f58a1200640 1 -- 172.21.15.71:0/22471232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c1e4e70 con 0x7f589c15f670 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.605+0000 7f58a1200640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 0x7f589c14fef0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f589400b990 tx=0x7f589400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.605+0000 7f5898c00640 1 -- 172.21.15.71:0/22471232 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894019070 con 0x7f589c15f670 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.606+0000 7f5898c00640 1 -- 172.21.15.71:0/22471232 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f589400cc90 con 0x7f589c15f670 2024-09-16T07:32:24.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.606+0000 7f5898c00640 1 -- 172.21.15.71:0/22471232 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58940146a0 con 0x7f589c15f670 2024-09-16T07:32:24.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.606+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c1e5410 con 0x7f589c15f670 2024-09-16T07:32:24.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.607+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1e59d0 con 0x7f589c15f670 2024-09-16T07:32:24.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.610+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c1e74b0 con 0x7f589c15f670 2024-09-16T07:32:24.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.611+0000 7f5898c00640 1 -- 172.21.15.71:0/22471232 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f589401c070 con 0x7f589c15f670 2024-09-16T07:32:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.613+0000 7f5898c00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f587807e130 0x7f58780805f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.614+0000 7f5898c00640 1 -- 172.21.15.71:0/22471232 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f5894099380 con 0x7f589c15f670 2024-09-16T07:32:24.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.619+0000 7f589be00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f587807e130 0x7f58780805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:24.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.621+0000 7f5898c00640 1 -- 172.21.15.71:0/22471232 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f58940634a0 con 0x7f589c15f670 2024-09-16T07:32:24.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.635+0000 7f589be00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f587807e130 0x7f58780805f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f588c033000 tx=0x7f588c0046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:24.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:24.965+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone849419", "target": ["mon-mgr", ""]}) -- 0x7f589c1e5130 con 0x7f587807e130 2024-09-16T07:32:25.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.447+0000 7f5898c00640 1 -- 172.21.15.71:0/22471232 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f589c1e5130 con 0x7f587807e130 2024-09-16T07:32:25.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.450+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f587807e130 msgr2=0x7f58780805f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:25.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.450+0000 7f58a1c00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f587807e130 0x7f58780805f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f588c033000 tx=0x7f588c0046c0 comp rx=0 tx=0).stop 2024-09-16T07:32:25.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.450+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 msgr2=0x7f589c14fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:25.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.450+0000 7f58a1c00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 0x7f589c14fef0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f589400b990 tx=0x7f589400be50 comp rx=0 tx=0).stop 2024-09-16T07:32:25.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.451+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 shutdown_connections 2024-09-16T07:32:25.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.451+0000 7f58a1c00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f587807e130 0x7f58780805f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:25.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.451+0000 7f58a1c00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f589c15f670 0x7f589c14fef0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:25.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.451+0000 7f58a1c00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f589c15c0f0 0x7f589c0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:25.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.451+0000 7f58a1c00640 1 --2- 172.21.15.71:0/22471232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f589c0c5730 0x7f589c1e6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:25.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.451+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 >> 172.21.15.71:0/22471232 conn(0x7f589c0cc490 msgr2=0x7f589c1523e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:25.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.451+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 shutdown_connections 2024-09-16T07:32:25.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:25.451+0000 7f58a1c00640 1 -- 172.21.15.71:0/22471232 wait complete. 2024-09-16T07:32:27.481 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:32:27.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.739+0000 7f41a1e00640 1 -- 172.21.15.71:0/3316717090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 msgr2=0x7f419c167110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:27.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.739+0000 7f41a1e00640 1 --2- 172.21.15.71:0/3316717090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 0x7f419c167110 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4194005fd0 tx=0x7f419402a8b0 comp rx=0 tx=0).stop 2024-09-16T07:32:27.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.740+0000 7f41a1e00640 1 -- 172.21.15.71:0/3316717090 shutdown_connections 2024-09-16T07:32:27.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.740+0000 7f41a1e00640 1 --2- 172.21.15.71:0/3316717090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 0x7f419c167110 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:27.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.740+0000 7f41a1e00640 1 -- 172.21.15.71:0/3316717090 >> 172.21.15.71:0/3316717090 conn(0x7f419c0b18e0 msgr2=0x7f419c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:27.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.740+0000 7f41a1e00640 1 -- 172.21.15.71:0/3316717090 shutdown_connections 2024-09-16T07:32:27.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.740+0000 7f41a1e00640 1 -- 172.21.15.71:0/3316717090 wait complete. 2024-09-16T07:32:27.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.742+0000 7f41a1e00640 1 Processor -- start 2024-09-16T07:32:27.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.742+0000 7f41a1e00640 1 -- start start 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f41a1e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f419c073de0 0x7f419c0d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f41a1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419c15f9a0 0x7f419c1f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f41a1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 0x7f419c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f41a1e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f419c1677d0 con 0x7f419c164d20 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f41a1e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f419c167650 con 0x7f419c15f9a0 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f41a1e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f419c167950 con 0x7f419c073de0 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f419b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 0x7f419c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f419b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 0x7f419c0b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55332/0 (socket says 172.21.15.71:55332) 2024-09-16T07:32:27.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.743+0000 7f419b400640 1 -- 172.21.15.71:0/182843598 learned_addr learned my addr 172.21.15.71:0/182843598 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:27.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f419b400640 1 -- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f419c073de0 msgr2=0x7f419c0d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:27.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f419b400640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f419c073de0 0x7f419c0d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:27.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f419b400640 1 -- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419c15f9a0 msgr2=0x7f419c1f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f419b400640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419c15f9a0 0x7f419c1f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f419b400640 1 -- 172.21.15.71:0/182843598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c1f67e0 con 0x7f419c164d20 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f419b400640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 0x7f419c0b41c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4194006010 tx=0x7f4194003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f4193400640 1 -- 172.21.15.71:0/182843598 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194031b90 con 0x7f419c164d20 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f4193400640 1 -- 172.21.15.71:0/182843598 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4194031e20 con 0x7f419c164d20 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419c1f6ac0 con 0x7f419c164d20 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.744+0000 7f4193400640 1 -- 172.21.15.71:0/182843598 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41940332f0 con 0x7f419c164d20 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.745+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419c0ce790 con 0x7f419c164d20 2024-09-16T07:32:27.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.746+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f419c0b4980 con 0x7f419c164d20 2024-09-16T07:32:27.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.747+0000 7f4193400640 1 -- 172.21.15.71:0/182843598 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 34) ==== 103174+0+0 (secure 0 0 0) 0x7f4194031660 con 0x7f419c164d20 2024-09-16T07:32:27.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.748+0000 7f4193400640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f417c079d20 0x7f417c07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:27.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.748+0000 7f4193400640 1 -- 172.21.15.71:0/182843598 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f41940bb000 con 0x7f419c164d20 2024-09-16T07:32:27.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.752+0000 7f4193400640 1 -- 172.21.15.71:0/182843598 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f419403f030 con 0x7f419c164d20 2024-09-16T07:32:27.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.760+0000 7f419aa00640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f417c079d20 0x7f417c07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:27.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:27.762+0000 7f419aa00640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f417c079d20 0x7f417c07c1e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f419c09dc00 tx=0x7f418c007970 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:28.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.204+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f419c069360 con 0x7f419c164d20 2024-09-16T07:32:28.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.206+0000 7f4193400640 1 -- 172.21.15.71:0/182843598 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2444 (secure 0 0 0) 0x7f4194084c60 con 0x7f419c164d20 2024-09-16T07:32:28.206 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:32:28.207 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 605, 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:28.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 92, 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:32:28.209 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 6613, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 27530119258, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 54789500928, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 718304612352, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 554871133, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 559802906, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 133, 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 146 2024-09-16T07:32:28.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 57, 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:31:18:228480+0000", 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6, 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 6, 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:32:28.211 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 7008 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:32:28.212 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 17, 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:32:18.525085+0000", 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "4 ongoing clones - average progress is 37.956% (34s)\n [==========..................] (remaining: 56s)", 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.37955999374389648, 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:28.213 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-total-clones": { 2024-09-16T07:32:28.214 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Total 10 clones - average progress is 33.073% (26s)\n [=========...................] (remaining: 53s)", 2024-09-16T07:32:28.214 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.33072999119758606, 2024-09-16T07:32:28.214 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:32:28.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:32:28.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:32:28.214 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:32:28.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.212+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f417c079d20 msgr2=0x7f417c07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:28.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.212+0000 7f41a1e00640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f417c079d20 0x7f417c07c1e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f419c09dc00 tx=0x7f418c007970 comp rx=0 tx=0).stop 2024-09-16T07:32:28.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.212+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 msgr2=0x7f419c0b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:28.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.212+0000 7f41a1e00640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 0x7f419c0b41c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4194006010 tx=0x7f4194003ba0 comp rx=0 tx=0).stop 2024-09-16T07:32:28.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.213+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 shutdown_connections 2024-09-16T07:32:28.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.213+0000 7f41a1e00640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f417c079d20 0x7f417c07c1e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:28.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.213+0000 7f41a1e00640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f419c164d20 0x7f419c0b41c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:28.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.213+0000 7f41a1e00640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419c15f9a0 0x7f419c1f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:28.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.213+0000 7f41a1e00640 1 --2- 172.21.15.71:0/182843598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f419c073de0 0x7f419c0d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:28.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.213+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 >> 172.21.15.71:0/182843598 conn(0x7f419c0b18e0 msgr2=0x7f419c1656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:28.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.214+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 shutdown_connections 2024-09-16T07:32:28.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:28.214+0000 7f41a1e00640 1 -- 172.21.15.71:0/182843598 wait complete. 2024-09-16T07:32:28.235 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 37.956% (34s)\n [==========..................] (remaining: 56s)', 'progress': 0.3795599937438965, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 10 clones - average progress is 33.073% (26s)\n [=========...................] (remaining: 53s)', 'progress': 0.33072999119758606, 'add_to_ceph_s': True}} 2024-09-16T07:32:28.236 INFO:tasks.cephfs.test_volumes:on_p = 0.40821000933647156 new_on_p = 0.3795599937438965 2024-09-16T07:32:28.236 INFO:tasks.cephfs.test_volumes:onpen_p = 0.32655999064445496 new_onpen_p = 0.33072999119758606 2024-09-16T07:32:28.236 INFO:tasks.cephfs.test_volumes:onpen_count = None new_onpen_count = None 2024-09-16T07:32:28.737 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:32:29.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1366037063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 msgr2=0x7f7f080cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:29.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1366037063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 0x7f7f080cf9b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8005fd0 tx=0x7f7ef802a580 comp rx=0 tx=0).stop 2024-09-16T07:32:29.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1366037063 shutdown_connections 2024-09-16T07:32:29.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1366037063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f080d1ec0 0x7f7f080d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1366037063 >> v2:172.21.15.177:3300/0 conn(0x7f7f080d1260 0x7f7f080d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1366037063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 0x7f7f080cf9b0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1366037063 >> 172.21.15.71:0/1366037063 conn(0x7f7f080d3e40 msgr2=0x7f7f080d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:29.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1366037063 shutdown_connections 2024-09-16T07:32:29.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.012+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1366037063 wait complete. 2024-09-16T07:32:29.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.014+0000 7f7f0ca00640 1 Processor -- start 2024-09-16T07:32:29.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.015+0000 7f7f0ca00640 1 -- start start 2024-09-16T07:32:29.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.016+0000 7f7f0ca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f080d1260 0x7f7f0809dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:29.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.016+0000 7f7f0ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f080d1ec0 0x7f7f080cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:29.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.016+0000 7f7f0ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 0x7f7f080ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:29.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.016+0000 7f7f0ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f080b7000 con 0x7f7f0815e050 2024-09-16T07:32:29.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.016+0000 7f7f0ca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f080b6e80 con 0x7f7f080d1ec0 2024-09-16T07:32:29.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.016+0000 7f7f0ca00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7f080b7180 con 0x7f7f080d1260 2024-09-16T07:32:29.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.016+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 0x7f7f080ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:29.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.017+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 0x7f7f080ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55346/0 (socket says 172.21.15.71:55346) 2024-09-16T07:32:29.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.017+0000 7f7f07e00640 1 -- 172.21.15.71:0/1828917199 learned_addr learned my addr 172.21.15.71:0/1828917199 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:29.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.017+0000 7f7f06a00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f080d1ec0 0x7f7f080cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:29.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.018+0000 7f7f07e00640 1 -- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f080d1260 msgr2=0x7f7f0809dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:29.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.018+0000 7f7f07e00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f080d1260 0x7f7f0809dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.018+0000 7f7f07e00640 1 -- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f080d1ec0 msgr2=0x7f7f080cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:29.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.018+0000 7f7f07e00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f080d1ec0 0x7f7f080cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.018+0000 7f7f07e00640 1 -- 172.21.15.71:0/1828917199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f080cceb0 con 0x7f7f0815e050 2024-09-16T07:32:29.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.018+0000 7f7f07e00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 0x7f7f080ca3e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000eaa0 tx=0x7f7f0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:29.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.019+0000 7f7eff400640 1 -- 172.21.15.71:0/1828917199 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000cdb0 con 0x7f7f0815e050 2024-09-16T07:32:29.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.019+0000 7f7eff400640 1 -- 172.21.15.71:0/1828917199 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f00004c10 con 0x7f7f0815e050 2024-09-16T07:32:29.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.019+0000 7f7eff400640 1 -- 172.21.15.71:0/1828917199 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00005020 con 0x7f7f0815e050 2024-09-16T07:32:29.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.019+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f080cd150 con 0x7f7f0815e050 2024-09-16T07:32:29.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.020+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f080cd710 con 0x7f7f0815e050 2024-09-16T07:32:29.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.021+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f080d2350 con 0x7f7f0815e050 2024-09-16T07:32:29.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.024+0000 7f7eff400640 1 -- 172.21.15.71:0/1828917199 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 35) ==== 103123+0+0 (secure 0 0 0) 0x7f7f000051c0 con 0x7f7f0815e050 2024-09-16T07:32:29.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.025+0000 7f7eff400640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7ee8079ce0 0x7f7ee807c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:29.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.025+0000 7f7eff400640 1 -- 172.21.15.71:0/1828917199 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f7f0009a300 con 0x7f7f0815e050 2024-09-16T07:32:29.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.030+0000 7f7eff400640 1 -- 172.21.15.71:0/1828917199 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7f00016030 con 0x7f7f0815e050 2024-09-16T07:32:29.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.062+0000 7f7f07400640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7ee8079ce0 0x7f7ee807c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:29.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.113+0000 7f7f07400640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7ee8079ce0 0x7f7ee807c1a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8033000 tx=0x7f7ef80046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.533+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7f080d0120 con 0x7f7f0815e050 2024-09-16T07:32:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.534+0000 7f7eff400640 1 -- 172.21.15.71:0/1828917199 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2425 (secure 0 0 0) 0x7f7f00064030 con 0x7f7f0815e050 2024-09-16T07:32:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:32:29.534 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:32:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:32:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:32:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:32:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:32:29.534 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 606, 2024-09-16T07:32:29.535 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 92, 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:32:29.536 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:29.537 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:32:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:32:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 6928, 2024-09-16T07:32:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 28834548362, 2024-09-16T07:32:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 57986064384, 2024-09-16T07:32:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 715108048896, 2024-09-16T07:32:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:32:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 664711090, 2024-09-16T07:32:29.538 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 668596745, 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 160, 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 174 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 57, 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:31:18:228480+0000", 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6, 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:32:29.539 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 6, 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 7008 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:29.540 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:32:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:32:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:32:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:32:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:32:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:32:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:32:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:32:29.541 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 17, 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:32:18.525085+0000", 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "4 ongoing clones - average progress is 25.0% (40s)\n [=======.....................] (remaining: 2m)", 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.25, 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:32:29.542 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-total-clones": { 2024-09-16T07:32:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Total 11 clones - average progress is 13.329% (32s)\n [===.........................] (remaining: 3m)", 2024-09-16T07:32:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0.13328999280929565, 2024-09-16T07:32:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:32:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:32:29.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:32:29.543 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7ee8079ce0 msgr2=0x7f7ee807c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7ee8079ce0 0x7f7ee807c1a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8033000 tx=0x7f7ef80046c0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 msgr2=0x7f7f080ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 0x7f7f080ca3e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000eaa0 tx=0x7f7f0000ef60 comp rx=0 tx=0).stop 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 shutdown_connections 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7ee8079ce0 0x7f7ee807c1a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0815e050 0x7f7f080ca3e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f080d1ec0 0x7f7f080cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 --2- 172.21.15.71:0/1828917199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f080d1260 0x7f7f0809dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 >> 172.21.15.71:0/1828917199 conn(0x7f7f080d3e40 msgr2=0x7f7f080b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.539+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 shutdown_connections 2024-09-16T07:32:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.540+0000 7f7f0ca00640 1 -- 172.21.15.71:0/1828917199 wait complete. 2024-09-16T07:32:29.574 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 25.0% (40s)\n [=======.....................] (remaining: 2m)', 'progress': 0.25, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 11 clones - average progress is 13.329% (32s)\n [===.........................] (remaining: 3m)', 'progress': 0.13328999280929565, 'add_to_ceph_s': True}} 2024-09-16T07:32:29.575 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-09-16T07:32:29.575 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.13328999280929565; onpen_p = 0.32655999064445496 2024-09-16T07:32:29.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone98650 2024-09-16T07:32:29.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.910+0000 7fcca7e00640 1 -- 172.21.15.71:0/200323525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca80cf500 msgr2=0x7fcca80cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:29.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.910+0000 7fcca7e00640 1 --2- 172.21.15.71:0/200323525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca80cf500 0x7fcca80cb610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcca0005fd0 tx=0x7fcca002a9f0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.910+0000 7fcca7e00640 1 -- 172.21.15.71:0/200323525 shutdown_connections 2024-09-16T07:32:29.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.910+0000 7fcca7e00640 1 --2- 172.21.15.71:0/200323525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca80cf500 0x7fcca80cb610 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.910+0000 7fcca7e00640 1 --2- 172.21.15.71:0/200323525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca806f6b0 0x7fcca80cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.910+0000 7fcca7e00640 1 -- 172.21.15.71:0/200323525 >> 172.21.15.71:0/200323525 conn(0x7fcca80b2670 msgr2=0x7fcca80b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:29.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.911+0000 7fcca7e00640 1 -- 172.21.15.71:0/200323525 shutdown_connections 2024-09-16T07:32:29.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.911+0000 7fcca7e00640 1 -- 172.21.15.71:0/200323525 wait complete. 2024-09-16T07:32:29.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.913+0000 7fcca7e00640 1 Processor -- start 2024-09-16T07:32:29.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.914+0000 7fcca7e00640 1 -- start start 2024-09-16T07:32:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.915+0000 7fcca7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca806f6b0 0x7fcca80c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.915+0000 7fcca7e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca80cf500 0x7fcca814a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.915+0000 7fcca7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcca815aad0 0x7fcca80b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.915+0000 7fcca7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcca80cc080 con 0x7fcca815aad0 2024-09-16T07:32:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.915+0000 7fcca7e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcca80cbf00 con 0x7fcca806f6b0 2024-09-16T07:32:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.915+0000 7fcca7e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcca80cc200 con 0x7fcca80cf500 2024-09-16T07:32:29.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.916+0000 7fcca7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcca815aad0 0x7fcca80b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:29.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.916+0000 7fcca7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcca815aad0 0x7fcca80b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55348/0 (socket says 172.21.15.71:55348) 2024-09-16T07:32:29.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.916+0000 7fcca7400640 1 -- 172.21.15.71:0/1922565819 learned_addr learned my addr 172.21.15.71:0/1922565819 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.916+0000 7fcca7400640 1 -- 172.21.15.71:0/1922565819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca80cf500 msgr2=0x7fcca814a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.916+0000 7fcca7400640 1 --2- 172.21.15.71:0/1922565819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca80cf500 0x7fcca814a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.916+0000 7fcca7400640 1 -- 172.21.15.71:0/1922565819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca806f6b0 msgr2=0x7fcca80c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.916+0000 7fcca7400640 1 --2- 172.21.15.71:0/1922565819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca806f6b0 0x7fcca80c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.916+0000 7fcca7400640 1 -- 172.21.15.71:0/1922565819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca81f8010 con 0x7fcca815aad0 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.917+0000 7fcca7400640 1 --2- 172.21.15.71:0/1922565819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcca815aad0 0x7fcca80b3b80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcca0006010 tx=0x7fcca0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.917+0000 7fcc9ea00640 1 -- 172.21.15.71:0/1922565819 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0047070 con 0x7fcca815aad0 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.917+0000 7fcc9ea00640 1 -- 172.21.15.71:0/1922565819 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcca0004250 con 0x7fcca815aad0 2024-09-16T07:32:29.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.917+0000 7fcca7e00640 1 -- 172.21.15.71:0/1922565819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca81f82f0 con 0x7fcca815aad0 2024-09-16T07:32:29.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.917+0000 7fcc9ea00640 1 -- 172.21.15.71:0/1922565819 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0033050 con 0x7fcca815aad0 2024-09-16T07:32:29.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.918+0000 7fcca7e00640 1 -- 172.21.15.71:0/1922565819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81f88b0 con 0x7fcca815aad0 2024-09-16T07:32:29.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.919+0000 7fcca7e00640 1 -- 172.21.15.71:0/1922565819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca80cc720 con 0x7fcca815aad0 2024-09-16T07:32:29.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.922+0000 7fcc9ea00640 1 -- 172.21.15.71:0/1922565819 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 35) ==== 103123+0+0 (secure 0 0 0) 0x7fcca004a070 con 0x7fcca815aad0 2024-09-16T07:32:29.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.924+0000 7fcc9ea00640 1 --2- 172.21.15.71:0/1922565819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcc88079c90 0x7fcc8807c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:29.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.924+0000 7fcc9ea00640 1 -- 172.21.15.71:0/1922565819 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fcca00ba850 con 0x7fcca815aad0 2024-09-16T07:32:29.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.940+0000 7fcc9ea00640 1 -- 172.21.15.71:0/1922565819 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcca0084910 con 0x7fcca815aad0 2024-09-16T07:32:29.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.940+0000 7fcca6a00640 1 --2- 172.21.15.71:0/1922565819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcc88079c90 0x7fcc8807c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:29.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:29.990+0000 7fcca6a00640 1 --2- 172.21.15.71:0/1922565819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcc88079c90 0x7fcc8807c150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcca80b2640 tx=0x7fcc98006ba0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:30.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:30.378+0000 7fcca7e00640 1 -- 172.21.15.71:0/1922565819 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone98650", "target": ["mon-mgr", ""]}) -- 0x7fcca80c3f80 con 0x7fcc88079c90 2024-09-16T07:32:31.023 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:30.794+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:32:31.023 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:30.794+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:32:31.023 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:30.794+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:32:31.023 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:30.794+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:32:31.023 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:30.794+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:32:32.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.142+0000 7fcc9ea00640 1 -- 172.21.15.71:0/1922565819 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fcca80c3f80 con 0x7fcc88079c90 2024-09-16T07:32:32.142 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:32.143 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:32.140+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:32.154 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:32.155 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone50521 2024-09-16T07:32:32.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.438+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3536947722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8acc004d40 msgr2=0x7f8acc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:32.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.438+0000 7f8ae7e00640 1 --2- 172.21.15.71:0/3536947722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8acc004d40 0x7f8acc0051c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8ae808d200 tx=0x7f8acc005830 comp rx=0 tx=0).stop 2024-09-16T07:32:32.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.439+0000 7f8ae7400640 1 -- 172.21.15.71:0/3536947722 reap_dead start 2024-09-16T07:32:32.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.439+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3536947722 shutdown_connections 2024-09-16T07:32:32.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.439+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3536947722 >> 172.21.15.71:0/3536947722 conn(0x7f8ae809d0b0 msgr2=0x7f8ae80d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.439+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3536947722 shutdown_connections 2024-09-16T07:32:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.440+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3536947722 wait complete. 2024-09-16T07:32:32.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.443+0000 7f8ae7e00640 1 Processor -- start 2024-09-16T07:32:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.444+0000 7f8ae7e00640 1 -- start start 2024-09-16T07:32:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.445+0000 7f8ae7e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8acc004d40 0x7f8ae80ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.445+0000 7f8ae7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ae81f5350 0x7f8ae815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.446+0000 7f8ae7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ae81f8490 0x7f8ae806e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.446+0000 7f8ae7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae80b50c0 con 0x7f8ae81f8490 2024-09-16T07:32:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.446+0000 7f8ae7e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae80b4f40 con 0x7f8ae81f5350 2024-09-16T07:32:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.446+0000 7f8ae7e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae80b33b0 con 0x7f8acc004d40 2024-09-16T07:32:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.446+0000 7f8ae6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ae81f5350 0x7f8ae815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.446+0000 7f8ae6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ae81f5350 0x7f8ae815e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:33992/0 (socket says 172.21.15.71:33992) 2024-09-16T07:32:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.446+0000 7f8ae6000640 1 -- 172.21.15.71:0/3106381748 learned_addr learned my addr 172.21.15.71:0/3106381748 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.447+0000 7f8ae7400640 1 --2- 172.21.15.71:0/3106381748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ae81f8490 0x7f8ae806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.447+0000 7f8ae6a00640 1 --2- 172.21.15.71:0/3106381748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8acc004d40 0x7f8ae80ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.448+0000 7f8ae7400640 1 -- 172.21.15.71:0/3106381748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8acc004d40 msgr2=0x7f8ae80ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.448+0000 7f8ae7400640 1 --2- 172.21.15.71:0/3106381748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8acc004d40 0x7f8ae80ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.448+0000 7f8ae7400640 1 -- 172.21.15.71:0/3106381748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ae81f5350 msgr2=0x7f8ae815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.448+0000 7f8ae7400640 1 --2- 172.21.15.71:0/3106381748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ae81f5350 0x7f8ae815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.448+0000 7f8ae7400640 1 -- 172.21.15.71:0/3106381748 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae80cdee0 con 0x7f8ae81f8490 2024-09-16T07:32:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.448+0000 7f8ae7400640 1 --2- 172.21.15.71:0/3106381748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ae81f8490 0x7f8ae806e450 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8032000 tx=0x7f8ad8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.449+0000 7f8adea00640 1 -- 172.21.15.71:0/3106381748 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8042070 con 0x7f8ae81f8490 2024-09-16T07:32:32.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.449+0000 7f8adea00640 1 -- 172.21.15.71:0/3106381748 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ad80044d0 con 0x7f8ae81f8490 2024-09-16T07:32:32.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.449+0000 7f8adea00640 1 -- 172.21.15.71:0/3106381748 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8038040 con 0x7f8ae81f8490 2024-09-16T07:32:32.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.449+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3106381748 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae80cd060 con 0x7f8ae81f8490 2024-09-16T07:32:32.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.450+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3106381748 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae80cd920 con 0x7f8ae81f8490 2024-09-16T07:32:32.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.451+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3106381748 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae8160a40 con 0x7f8ae81f8490 2024-09-16T07:32:32.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.456+0000 7f8adea00640 1 -- 172.21.15.71:0/3106381748 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 35) ==== 103123+0+0 (secure 0 0 0) 0x7f8ad8003830 con 0x7f8ae81f8490 2024-09-16T07:32:32.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.457+0000 7f8adea00640 1 --2- 172.21.15.71:0/3106381748 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ac8079cc0 0x7f8ac807c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:32.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.458+0000 7f8adea00640 1 -- 172.21.15.71:0/3106381748 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f8ad80b95b0 con 0x7f8ae81f8490 2024-09-16T07:32:32.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.463+0000 7f8adea00640 1 -- 172.21.15.71:0/3106381748 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ad8083060 con 0x7f8ae81f8490 2024-09-16T07:32:32.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.515+0000 7f8ae6a00640 1 --2- 172.21.15.71:0/3106381748 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ac8079cc0 0x7f8ac807c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:32.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.578+0000 7f8ae6a00640 1 --2- 172.21.15.71:0/3106381748 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ac8079cc0 0x7f8ac807c180 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0004dc0 tx=0x7f8ae0006040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:32.906+0000 7f8ae7e00640 1 -- 172.21.15.71:0/3106381748 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone50521", "target": ["mon-mgr", ""]}) -- 0x7f8ae8069d80 con 0x7f8ac8079cc0 2024-09-16T07:32:33.450 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:33.449+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:33.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.451+0000 7f8adea00640 1 -- 172.21.15.71:0/3106381748 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f8ae0016070 con 0x7f8ac8079cc0 2024-09-16T07:32:33.451 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:33.463 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:33.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone31792 2024-09-16T07:32:33.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.734+0000 7f4077000640 1 -- 172.21.15.71:0/3207010788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700cbc00 msgr2=0x7f407006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:33.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.734+0000 7f4077000640 1 --2- 172.21.15.71:0/3207010788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700cbc00 0x7f407006fce0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f4064009830 tx=0x7f406402a540 comp rx=0 tx=0).stop 2024-09-16T07:32:33.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.736+0000 7f4077000640 1 -- 172.21.15.71:0/3207010788 shutdown_connections 2024-09-16T07:32:33.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.736+0000 7f4077000640 1 --2- 172.21.15.71:0/3207010788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700cbc00 0x7f407006fce0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.736+0000 7f4077000640 1 -- 172.21.15.71:0/3207010788 >> 172.21.15.71:0/3207010788 conn(0x7f40700cf9f0 msgr2=0x7f40700cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.736+0000 7f4077000640 1 -- 172.21.15.71:0/3207010788 shutdown_connections 2024-09-16T07:32:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.736+0000 7f4077000640 1 -- 172.21.15.71:0/3207010788 wait complete. 2024-09-16T07:32:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.739+0000 7f4077000640 1 Processor -- start 2024-09-16T07:32:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.739+0000 7f4077000640 1 -- start start 2024-09-16T07:32:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.741+0000 7f4077000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4070072560 0x7f4070151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.741+0000 7f4077000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40700cbc00 0x7f40700b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.741+0000 7f4077000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f407015bec0 0x7f40700b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.741+0000 7f4077000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f40700aed50 con 0x7f407015bec0 2024-09-16T07:32:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.741+0000 7f4077000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40700aebd0 con 0x7f4070072560 2024-09-16T07:32:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.741+0000 7f4077000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f40700aeed0 con 0x7f40700cbc00 2024-09-16T07:32:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.741+0000 7f4076600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f407015bec0 0x7f40700b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.742+0000 7f4076600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f407015bec0 0x7f40700b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55378/0 (socket says 172.21.15.71:55378) 2024-09-16T07:32:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.742+0000 7f4076600640 1 -- 172.21.15.71:0/1417344040 learned_addr learned my addr 172.21.15.71:0/1417344040 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.742+0000 7f4075c00640 1 --2- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4070072560 0x7f4070151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.742+0000 7f4075200640 1 --2- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40700cbc00 0x7f40700b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.742+0000 7f4076600640 1 -- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40700cbc00 msgr2=0x7f40700b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.742+0000 7f4076600640 1 --2- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40700cbc00 0x7f40700b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.742+0000 7f4076600640 1 -- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4070072560 msgr2=0x7f4070151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.742+0000 7f4076600640 1 --2- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4070072560 0x7f4070151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.743+0000 7f4076600640 1 -- 172.21.15.71:0/1417344040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4070157c80 con 0x7f407015bec0 2024-09-16T07:32:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.743+0000 7f4076600640 1 --2- 172.21.15.71:0/1417344040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f407015bec0 0x7f40700b4840 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f406800cc30 tx=0x7f406800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.743+0000 7f406d600640 1 -- 172.21.15.71:0/1417344040 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406800ce40 con 0x7f407015bec0 2024-09-16T07:32:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.744+0000 7f406d600640 1 -- 172.21.15.71:0/1417344040 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4068017070 con 0x7f407015bec0 2024-09-16T07:32:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.744+0000 7f406d600640 1 -- 172.21.15.71:0/1417344040 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068012790 con 0x7f407015bec0 2024-09-16T07:32:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.744+0000 7f4077000640 1 -- 172.21.15.71:0/1417344040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4070157f60 con 0x7f407015bec0 2024-09-16T07:32:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.744+0000 7f4077000640 1 -- 172.21.15.71:0/1417344040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4070158520 con 0x7f407015bec0 2024-09-16T07:32:33.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.746+0000 7f4077000640 1 -- 172.21.15.71:0/1417344040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407006fce0 con 0x7f407015bec0 2024-09-16T07:32:33.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.748+0000 7f406d600640 1 -- 172.21.15.71:0/1417344040 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f4068012930 con 0x7f407015bec0 2024-09-16T07:32:33.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.750+0000 7f406d600640 1 --2- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4054079d20 0x7f405407c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:33.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.750+0000 7f406d600640 1 -- 172.21.15.71:0/1417344040 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f4068099be0 con 0x7f407015bec0 2024-09-16T07:32:33.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.756+0000 7f4075c00640 1 --2- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4054079d20 0x7f405407c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:33.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.760+0000 7f4075c00640 1 --2- 172.21.15.71:0/1417344040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4054079d20 0x7f405407c1e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f40600015e0 tx=0x7f4060009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:33.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:33.763+0000 7f406d600640 1 -- 172.21.15.71:0/1417344040 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4068063650 con 0x7f407015bec0 2024-09-16T07:32:34.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.213+0000 7f4077000640 1 -- 172.21.15.71:0/1417344040 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone31792", "target": ["mon-mgr", ""]}) -- 0x7f40701541c0 con 0x7f4054079d20 2024-09-16T07:32:34.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.539+0000 7f406d600640 1 -- 172.21.15.71:0/1417344040 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f40701541c0 con 0x7f4054079d20 2024-09-16T07:32:34.539 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:34.540 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:34.537+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:34.552 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:34.552 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone51223 2024-09-16T07:32:34.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.837+0000 7fcd4f600640 1 -- 172.21.15.71:0/1247979927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd481437c0 msgr2=0x7fcd480cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.837+0000 7fcd4f600640 1 --2- 172.21.15.71:0/1247979927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd481437c0 0x7fcd480cdad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd38005fd0 tx=0x7fcd3802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:32:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.837+0000 7fcd4f600640 1 -- 172.21.15.71:0/1247979927 shutdown_connections 2024-09-16T07:32:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.837+0000 7fcd4f600640 1 --2- 172.21.15.71:0/1247979927 >> v2:172.21.15.71:3300/0 conn(0x7fcd480b4bd0 0x7fcd480b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.837+0000 7fcd4f600640 1 --2- 172.21.15.71:0/1247979927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd481437c0 0x7fcd480cdad0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:34.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.837+0000 7fcd4f600640 1 -- 172.21.15.71:0/1247979927 >> 172.21.15.71:0/1247979927 conn(0x7fcd480b1840 msgr2=0x7fcd480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:34.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.838+0000 7fcd4f600640 1 -- 172.21.15.71:0/1247979927 shutdown_connections 2024-09-16T07:32:34.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.838+0000 7fcd4f600640 1 -- 172.21.15.71:0/1247979927 wait complete. 2024-09-16T07:32:34.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.841+0000 7fcd4f600640 1 Processor -- start 2024-09-16T07:32:34.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.856+0000 7fcd4f600640 1 -- start start 2024-09-16T07:32:34.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.858+0000 7fcd4f600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd480b4400 0x7fcd4813c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:34.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.858+0000 7fcd4f600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd480b4bd0 0x7fcd481537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:34.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.859+0000 7fcd4f600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd481437c0 0x7fcd48073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:34.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.859+0000 7fcd4f600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4806f870 con 0x7fcd480b4bd0 2024-09-16T07:32:34.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.859+0000 7fcd4f600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48070a80 con 0x7fcd481437c0 2024-09-16T07:32:34.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.859+0000 7fcd4f600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4806f9f0 con 0x7fcd480b4400 2024-09-16T07:32:34.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.862+0000 7fcd4d600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd481437c0 0x7fcd48073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:34.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.863+0000 7fcd4d600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd481437c0 0x7fcd48073d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:35656/0 (socket says 172.21.15.71:35656) 2024-09-16T07:32:34.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.863+0000 7fcd4d600640 1 -- 172.21.15.71:0/2743873843 learned_addr learned my addr 172.21.15.71:0/2743873843 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:34.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.864+0000 7fcd4cc00640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd480b4400 0x7fcd4813c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:34.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.865+0000 7fcd3fe00640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd480b4bd0 0x7fcd481537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:34.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.867+0000 7fcd4d600640 1 -- 172.21.15.71:0/2743873843 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd480b4400 msgr2=0x7fcd4813c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:34.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.867+0000 7fcd4d600640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd480b4400 0x7fcd4813c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:34.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.867+0000 7fcd4d600640 1 -- 172.21.15.71:0/2743873843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd480b4bd0 msgr2=0x7fcd481537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:34.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.869+0000 7fcd4d600640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd480b4bd0 0x7fcd481537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:34.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.869+0000 7fcd4d600640 1 -- 172.21.15.71:0/2743873843 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd48074520 con 0x7fcd481437c0 2024-09-16T07:32:34.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.869+0000 7fcd3fe00640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd480b4bd0 0x7fcd481537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:32:34.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.872+0000 7fcd4d600640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd481437c0 0x7fcd48073d40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c790 tx=0x7fcd4000cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:34.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.874+0000 7fcd45600640 1 -- 172.21.15.71:0/2743873843 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40003b70 con 0x7fcd481437c0 2024-09-16T07:32:34.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.875+0000 7fcd45600640 1 -- 172.21.15.71:0/2743873843 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd40003d30 con 0x7fcd481437c0 2024-09-16T07:32:34.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.875+0000 7fcd45600640 1 -- 172.21.15.71:0/2743873843 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000aab0 con 0x7fcd481437c0 2024-09-16T07:32:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.875+0000 7fcd4f600640 1 -- 172.21.15.71:0/2743873843 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd48074280 con 0x7fcd481437c0 2024-09-16T07:32:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.875+0000 7fcd4f600640 1 -- 172.21.15.71:0/2743873843 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd48072b90 con 0x7fcd481437c0 2024-09-16T07:32:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.881+0000 7fcd4f600640 1 -- 172.21.15.71:0/2743873843 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd48070580 con 0x7fcd481437c0 2024-09-16T07:32:34.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.886+0000 7fcd45600640 1 -- 172.21.15.71:0/2743873843 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7fcd4000ac50 con 0x7fcd481437c0 2024-09-16T07:32:34.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.887+0000 7fcd45600640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd24079df0 0x7fcd2407c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:34.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.888+0000 7fcd45600640 1 -- 172.21.15.71:0/2743873843 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fcd40099810 con 0x7fcd481437c0 2024-09-16T07:32:34.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.897+0000 7fcd45600640 1 -- 172.21.15.71:0/2743873843 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcd48070580 con 0x7fcd481437c0 2024-09-16T07:32:34.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.909+0000 7fcd4cc00640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd24079df0 0x7fcd2407c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:34.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:34.920+0000 7fcd4cc00640 1 --2- 172.21.15.71:0/2743873843 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fcd24079df0 0x7fcd2407c2b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcd38033000 tx=0x7fcd38004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:35.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.306+0000 7fcd4f600640 1 -- 172.21.15.71:0/2743873843 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone51223", "target": ["mon-mgr", ""]}) -- 0x7fcd480cd570 con 0x7fcd24079df0 2024-09-16T07:32:35.588 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:35.587+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:35.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.592+0000 7fcd45600640 1 -- 172.21.15.71:0/2743873843 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fcd480cd570 con 0x7fcd24079df0 2024-09-16T07:32:35.593 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:35.622 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:35.622 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone73294 2024-09-16T07:32:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.954+0000 7f7a19c00640 1 -- 172.21.15.71:0/2865128823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79f8004080 msgr2=0x7f79f8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:35.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.954+0000 7f7a19c00640 1 --2- 172.21.15.71:0/2865128823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79f8004080 0x7f79f8004800 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000c260 tx=0x7f79f8005830 comp rx=0 tx=0).stop 2024-09-16T07:32:35.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.954+0000 7f7a19200640 1 -- 172.21.15.71:0/2865128823 reap_dead start 2024-09-16T07:32:35.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.954+0000 7f7a19c00640 1 -- 172.21.15.71:0/2865128823 shutdown_connections 2024-09-16T07:32:35.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.954+0000 7f7a19c00640 1 -- 172.21.15.71:0/2865128823 >> 172.21.15.71:0/2865128823 conn(0x7f7a140cf9f0 msgr2=0x7f7a140cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:35.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.955+0000 7f7a19c00640 1 -- 172.21.15.71:0/2865128823 shutdown_connections 2024-09-16T07:32:35.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.955+0000 7f7a19c00640 1 -- 172.21.15.71:0/2865128823 wait complete. 2024-09-16T07:32:35.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.958+0000 7f7a19c00640 1 Processor -- start 2024-09-16T07:32:35.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.959+0000 7f7a19c00640 1 -- start start 2024-09-16T07:32:35.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.960+0000 7f7a19c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a141f4120 0x7f7a140c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:35.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.960+0000 7f7a19c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a141f4500 0x7f7a1415f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:35.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.960+0000 7f7a19c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a140c4f40 0x7f7a1415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:35.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.960+0000 7f7a19c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7a14070210 con 0x7f7a141f4500 2024-09-16T07:32:35.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.960+0000 7f7a19c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a14070090 con 0x7f7a141f4120 2024-09-16T07:32:35.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.960+0000 7f7a19c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7a14070390 con 0x7f7a140c4f40 2024-09-16T07:32:35.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a141f4500 0x7f7a1415f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:35.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a141f4500 0x7f7a1415f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48870/0 (socket says 172.21.15.71:48870) 2024-09-16T07:32:35.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13400640 1 -- 172.21.15.71:0/2456622664 learned_addr learned my addr 172.21.15.71:0/2456622664 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:35.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13e00640 1 --2- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a141f4120 0x7f7a140c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:35.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a19200640 1 --2- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a140c4f40 0x7f7a1415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:35.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13400640 1 -- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a140c4f40 msgr2=0x7f7a1415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:35.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13400640 1 --2- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a140c4f40 0x7f7a1415f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:35.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13400640 1 -- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a141f4120 msgr2=0x7f7a140c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:35.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13400640 1 --2- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a141f4120 0x7f7a140c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:35.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.961+0000 7f7a13400640 1 -- 172.21.15.71:0/2456622664 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a140c4270 con 0x7f7a141f4500 2024-09-16T07:32:35.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.962+0000 7f7a13400640 1 --2- 172.21.15.71:0/2456622664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a141f4500 0x7f7a1415f070 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000c260 tx=0x7f7a00004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:35.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.962+0000 7f7a0be00640 1 -- 172.21.15.71:0/2456622664 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00031030 con 0x7f7a141f4500 2024-09-16T07:32:35.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.962+0000 7f7a0be00640 1 -- 172.21.15.71:0/2456622664 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a00006440 con 0x7f7a141f4500 2024-09-16T07:32:35.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.962+0000 7f7a0be00640 1 -- 172.21.15.71:0/2456622664 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a000065f0 con 0x7f7a141f4500 2024-09-16T07:32:35.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.963+0000 7f7a19c00640 1 -- 172.21.15.71:0/2456622664 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a140c93d0 con 0x7f7a141f4500 2024-09-16T07:32:35.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.963+0000 7f7a19c00640 1 -- 172.21.15.71:0/2456622664 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a140c44e0 con 0x7f7a141f4500 2024-09-16T07:32:35.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.965+0000 7f7a19c00640 1 -- 172.21.15.71:0/2456622664 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a140ae1e0 con 0x7f7a141f4500 2024-09-16T07:32:35.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.968+0000 7f7a0be00640 1 -- 172.21.15.71:0/2456622664 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f7a00034070 con 0x7f7a141f4500 2024-09-16T07:32:35.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.971+0000 7f7a0be00640 1 --2- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79f4079fc0 0x7f79f407c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:35.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.971+0000 7f7a0be00640 1 -- 172.21.15.71:0/2456622664 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f7a000b9c20 con 0x7f7a141f4500 2024-09-16T07:32:35.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.974+0000 7f7a0be00640 1 -- 172.21.15.71:0/2456622664 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7a00045060 con 0x7f7a141f4500 2024-09-16T07:32:35.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:35.990+0000 7f7a13e00640 1 --2- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79f4079fc0 0x7f79f407c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:36.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:36.037+0000 7f7a13e00640 1 --2- 172.21.15.71:0/2456622664 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f79f4079fc0 0x7f79f407c480 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400fb40 tx=0x7f7a04005f30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:36.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:36.382+0000 7f7a19c00640 1 -- 172.21.15.71:0/2456622664 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone73294", "target": ["mon-mgr", ""]}) -- 0x7f7a1414d710 con 0x7f79f4079fc0 2024-09-16T07:32:37.361 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:37.361+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:37.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.364+0000 7f7a0be00640 1 -- 172.21.15.71:0/2456622664 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f7a04011b80 con 0x7f79f4079fc0 2024-09-16T07:32:37.365 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:37.377 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:37.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone69795 2024-09-16T07:32:37.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.663+0000 7fd1b6400640 1 -- 172.21.15.71:0/608425476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0162330 msgr2=0x7fd1b00b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:37.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.663+0000 7fd1b6400640 1 --2- 172.21.15.71:0/608425476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0162330 0x7fd1b00b1760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005fd0 tx=0x7fd1a802aa00 comp rx=0 tx=0).stop 2024-09-16T07:32:37.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.666+0000 7fd1b6400640 1 -- 172.21.15.71:0/608425476 shutdown_connections 2024-09-16T07:32:37.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.666+0000 7fd1b6400640 1 --2- 172.21.15.71:0/608425476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0162330 0x7fd1b00b1760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:37.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.666+0000 7fd1b6400640 1 --2- 172.21.15.71:0/608425476 >> v2:172.21.15.177:3300/0 conn(0x7fd1b015e250 0x7fd1b00d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:37.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.666+0000 7fd1b6400640 1 -- 172.21.15.71:0/608425476 >> 172.21.15.71:0/608425476 conn(0x7fd1b00d1e20 msgr2=0x7fd1b00d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:37.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.667+0000 7fd1b6400640 1 -- 172.21.15.71:0/608425476 shutdown_connections 2024-09-16T07:32:37.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.667+0000 7fd1b6400640 1 -- 172.21.15.71:0/608425476 wait complete. 2024-09-16T07:32:37.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.670+0000 7fd1b6400640 1 Processor -- start 2024-09-16T07:32:37.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.670+0000 7fd1b6400640 1 -- start start 2024-09-16T07:32:37.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.674+0000 7fd1b6400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b006ed00 0x7fd1b01fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.674+0000 7fd1b6400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b015e250 0x7fd1b00b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.674+0000 7fd1b6400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b0162330 0x7fd1b00c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1b6400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b00b1ca0 con 0x7fd1b0162330 2024-09-16T07:32:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1b6400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b00d1250 con 0x7fd1b006ed00 2024-09-16T07:32:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1b6400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b00b1e20 con 0x7fd1b015e250 2024-09-16T07:32:37.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1afe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b0162330 0x7fd1b00c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:37.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1afe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b0162330 0x7fd1b00c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48878/0 (socket says 172.21.15.71:48878) 2024-09-16T07:32:37.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1afe00640 1 -- 172.21.15.71:0/1570386812 learned_addr learned my addr 172.21.15.71:0/1570386812 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:37.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1af400640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b006ed00 0x7fd1b01fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:37.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1afe00640 1 -- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b015e250 msgr2=0x7fd1b00b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:37.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.675+0000 7fd1aea00640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b015e250 0x7fd1b00b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:37.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.676+0000 7fd1afe00640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b015e250 0x7fd1b00b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:37.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.676+0000 7fd1afe00640 1 -- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b006ed00 msgr2=0x7fd1b01fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:37.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.676+0000 7fd1afe00640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b006ed00 0x7fd1b01fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:37.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.676+0000 7fd1afe00640 1 -- 172.21.15.71:0/1570386812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b01f9610 con 0x7fd1b0162330 2024-09-16T07:32:37.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.676+0000 7fd1aea00640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b015e250 0x7fd1b00b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:32:37.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.677+0000 7fd1afe00640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b0162330 0x7fd1b00c5910 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8006010 tx=0x7fd1a8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:37.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.678+0000 7fd1a7400640 1 -- 172.21.15.71:0/1570386812 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8046070 con 0x7fd1b0162330 2024-09-16T07:32:37.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.678+0000 7fd1a7400640 1 -- 172.21.15.71:0/1570386812 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1a8004540 con 0x7fd1b0162330 2024-09-16T07:32:37.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.678+0000 7fd1a7400640 1 -- 172.21.15.71:0/1570386812 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8033070 con 0x7fd1b0162330 2024-09-16T07:32:37.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.678+0000 7fd1b6400640 1 -- 172.21.15.71:0/1570386812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b01f98a0 con 0x7fd1b0162330 2024-09-16T07:32:37.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.678+0000 7fd1b6400640 1 -- 172.21.15.71:0/1570386812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b01f9e60 con 0x7fd1b0162330 2024-09-16T07:32:37.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.682+0000 7fd1b6400640 1 -- 172.21.15.71:0/1570386812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b00b2380 con 0x7fd1b0162330 2024-09-16T07:32:37.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.683+0000 7fd1a7400640 1 -- 172.21.15.71:0/1570386812 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7fd1a8049070 con 0x7fd1b0162330 2024-09-16T07:32:37.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.685+0000 7fd1a7400640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd190079d20 0x7fd19007c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:37.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.685+0000 7fd1a7400640 1 -- 172.21.15.71:0/1570386812 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fd1a80ba6f0 con 0x7fd1b0162330 2024-09-16T07:32:37.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.686+0000 7fd1af400640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd190079d20 0x7fd19007c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:37.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.690+0000 7fd1af400640 1 --2- 172.21.15.71:0/1570386812 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd190079d20 0x7fd19007c1e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0005e10 tx=0x7fd1a0005da0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:37.691+0000 7fd1a7400640 1 -- 172.21.15.71:0/1570386812 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd1a803e070 con 0x7fd1b0162330 2024-09-16T07:32:38.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:38.164+0000 7fd1b6400640 1 -- 172.21.15.71:0/1570386812 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone69795", "target": ["mon-mgr", ""]}) -- 0x7fd1b01fa490 con 0x7fd190079d20 2024-09-16T07:32:38.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:38.741+0000 7fd1a7400640 1 -- 172.21.15.71:0/1570386812 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fd1b01fa490 con 0x7fd190079d20 2024-09-16T07:32:38.741 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:38.742 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:38.739+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:38.757 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:38.758 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone71046 2024-09-16T07:32:39.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.017+0000 7f9b94c00640 1 -- 172.21.15.71:0/1623915937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b900cbc00 msgr2=0x7f9b9006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:39.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.017+0000 7f9b94c00640 1 --2- 172.21.15.71:0/1623915937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b900cbc00 0x7f9b9006fce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c005fd0 tx=0x7f9b7c02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:32:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.018+0000 7f9b94c00640 1 -- 172.21.15.71:0/1623915937 shutdown_connections 2024-09-16T07:32:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.018+0000 7f9b94c00640 1 --2- 172.21.15.71:0/1623915937 >> v2:172.21.15.177:3300/0 conn(0x7f9b90072560 0x7f9b90073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.018+0000 7f9b94c00640 1 --2- 172.21.15.71:0/1623915937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b900cbc00 0x7f9b9006fce0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.018+0000 7f9b94c00640 1 -- 172.21.15.71:0/1623915937 >> 172.21.15.71:0/1623915937 conn(0x7f9b900cf9f0 msgr2=0x7f9b900cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.019+0000 7f9b94c00640 1 -- 172.21.15.71:0/1623915937 shutdown_connections 2024-09-16T07:32:39.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.019+0000 7f9b94c00640 1 -- 172.21.15.71:0/1623915937 wait complete. 2024-09-16T07:32:39.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.022+0000 7f9b94c00640 1 Processor -- start 2024-09-16T07:32:39.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.023+0000 7f9b94c00640 1 -- start start 2024-09-16T07:32:39.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.024+0000 7f9b94c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b90072560 0x7f9b901522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:39.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.024+0000 7f9b94c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b900cbc00 0x7f9b900cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:39.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.024+0000 7f9b94c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9015bec0 0x7f9b900ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:39.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.024+0000 7f9b94c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900ae9e0 con 0x7f9b9015bec0 2024-09-16T07:32:39.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.024+0000 7f9b94c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900ae860 con 0x7f9b900cbc00 2024-09-16T07:32:39.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.024+0000 7f9b94c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9b900aeb60 con 0x7f9b90072560 2024-09-16T07:32:39.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.025+0000 7f9b8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9015bec0 0x7f9b900ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:39.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.025+0000 7f9b8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9015bec0 0x7f9b900ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48880/0 (socket says 172.21.15.71:48880) 2024-09-16T07:32:39.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.025+0000 7f9b8fe00640 1 -- 172.21.15.71:0/512134181 learned_addr learned my addr 172.21.15.71:0/512134181 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:39.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.025+0000 7f9b8ea00640 1 --2- 172.21.15.71:0/512134181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b900cbc00 0x7f9b900cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:39.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.025+0000 7f9b8fe00640 1 -- 172.21.15.71:0/512134181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b90072560 msgr2=0x7f9b901522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:39.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.025+0000 7f9b8fe00640 1 --2- 172.21.15.71:0/512134181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b90072560 0x7f9b901522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:39.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.025+0000 7f9b8fe00640 1 -- 172.21.15.71:0/512134181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b900cbc00 msgr2=0x7f9b900cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:39.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.025+0000 7f9b8fe00640 1 --2- 172.21.15.71:0/512134181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b900cbc00 0x7f9b900cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:39.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.026+0000 7f9b8fe00640 1 -- 172.21.15.71:0/512134181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b900c6ac0 con 0x7f9b9015bec0 2024-09-16T07:32:39.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.026+0000 7f9b8fe00640 1 --2- 172.21.15.71:0/512134181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9015bec0 0x7f9b900ad8c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800eb40 tx=0x7f9b8800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:39.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.026+0000 7f9b87400640 1 -- 172.21.15.71:0/512134181 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88004b20 con 0x7f9b9015bec0 2024-09-16T07:32:39.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.027+0000 7f9b87400640 1 -- 172.21.15.71:0/512134181 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b88004db0 con 0x7f9b9015bec0 2024-09-16T07:32:39.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.027+0000 7f9b87400640 1 -- 172.21.15.71:0/512134181 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88005280 con 0x7f9b9015bec0 2024-09-16T07:32:39.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.027+0000 7f9b94c00640 1 -- 172.21.15.71:0/512134181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b900c6d80 con 0x7f9b9015bec0 2024-09-16T07:32:39.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.028+0000 7f9b94c00640 1 -- 172.21.15.71:0/512134181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b900c7340 con 0x7f9b9015bec0 2024-09-16T07:32:39.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.029+0000 7f9b94c00640 1 -- 172.21.15.71:0/512134181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b900af080 con 0x7f9b9015bec0 2024-09-16T07:32:39.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.031+0000 7f9b87400640 1 -- 172.21.15.71:0/512134181 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f9b88005420 con 0x7f9b9015bec0 2024-09-16T07:32:39.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.033+0000 7f9b87400640 1 --2- 172.21.15.71:0/512134181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b70079d50 0x7f9b7007c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:39.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.033+0000 7f9b87400640 1 -- 172.21.15.71:0/512134181 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f9b8809adf0 con 0x7f9b9015bec0 2024-09-16T07:32:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.041+0000 7f9b87400640 1 -- 172.21.15.71:0/512134181 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9b88027080 con 0x7f9b9015bec0 2024-09-16T07:32:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.077+0000 7f9b8f400640 1 --2- 172.21.15.71:0/512134181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b70079d50 0x7f9b7007c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:39.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.091+0000 7f9b8f400640 1 --2- 172.21.15.71:0/512134181 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b70079d50 0x7f9b7007c210 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9b80001530 tx=0x7f9b800078d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:39.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:39.378+0000 7f9b94c00640 1 -- 172.21.15.71:0/512134181 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone71046", "target": ["mon-mgr", ""]}) -- 0x7f9b901f5820 con 0x7f9b70079d50 2024-09-16T07:32:40.217 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:40.217+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:40.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.219+0000 7f9b87400640 1 -- 172.21.15.71:0/512134181 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f9b901f5820 con 0x7f9b70079d50 2024-09-16T07:32:40.218 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:40.234 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:40.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone61947 2024-09-16T07:32:40.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.554+0000 7f5100a00640 1 -- 172.21.15.71:0/2658244117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50e0004d40 msgr2=0x7f50e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:40.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.554+0000 7f5100a00640 1 --2- 172.21.15.71:0/2658244117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50e0004d40 0x7f50e00051c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f50fc08d200 tx=0x7f50e0005830 comp rx=0 tx=0).stop 2024-09-16T07:32:40.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.555+0000 7f50fbe00640 1 -- 172.21.15.71:0/2658244117 reap_dead start 2024-09-16T07:32:40.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.555+0000 7f5100a00640 1 -- 172.21.15.71:0/2658244117 shutdown_connections 2024-09-16T07:32:40.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.555+0000 7f5100a00640 1 -- 172.21.15.71:0/2658244117 >> 172.21.15.71:0/2658244117 conn(0x7f50fc09d0b0 msgr2=0x7f50fc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:40.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.555+0000 7f5100a00640 1 -- 172.21.15.71:0/2658244117 shutdown_connections 2024-09-16T07:32:40.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.555+0000 7f5100a00640 1 -- 172.21.15.71:0/2658244117 wait complete. 2024-09-16T07:32:40.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.557+0000 7f5100a00640 1 Processor -- start 2024-09-16T07:32:40.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.557+0000 7f5100a00640 1 -- start start 2024-09-16T07:32:40.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f5100a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50fc1f2000 0x7f50fc0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:40.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f5100a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50fc1f23e0 0x7f50fc1f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:40.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f5100a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50fc1f5160 0x7f50fc0b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:40.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f5100a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc0b56d0 con 0x7f50fc1f2000 2024-09-16T07:32:40.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f5100a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc0b5150 con 0x7f50fc1f5160 2024-09-16T07:32:40.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f5100a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f50fc0b33b0 con 0x7f50fc1f23e0 2024-09-16T07:32:40.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f50fb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50fc1f2000 0x7f50fc0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:40.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f50fb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50fc1f2000 0x7f50fc0c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48892/0 (socket says 172.21.15.71:48892) 2024-09-16T07:32:40.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.559+0000 7f50fb400640 1 -- 172.21.15.71:0/1547909487 learned_addr learned my addr 172.21.15.71:0/1547909487 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:40.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.560+0000 7f50faa00640 1 --2- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50fc1f23e0 0x7f50fc1f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:40.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.560+0000 7f50fbe00640 1 --2- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50fc1f5160 0x7f50fc0b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.561+0000 7f50fb400640 1 -- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50fc1f23e0 msgr2=0x7f50fc1f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.561+0000 7f50fb400640 1 --2- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50fc1f23e0 0x7f50fc1f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.561+0000 7f50fb400640 1 -- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50fc1f5160 msgr2=0x7f50fc0b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.561+0000 7f50fb400640 1 --2- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50fc1f5160 0x7f50fc0b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.561+0000 7f50fb400640 1 -- 172.21.15.71:0/1547909487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc1f5730 con 0x7f50fc1f2000 2024-09-16T07:32:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.562+0000 7f50fb400640 1 --2- 172.21.15.71:0/1547909487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50fc1f2000 0x7f50fc0c9190 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00d990 tx=0x7f50ec00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.562+0000 7f50f3400640 1 -- 172.21.15.71:0/1547909487 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0064e0 con 0x7f50fc1f2000 2024-09-16T07:32:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.562+0000 7f50f3400640 1 -- 172.21.15.71:0/1547909487 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50ec00b900 con 0x7f50fc1f2000 2024-09-16T07:32:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.562+0000 7f50f3400640 1 -- 172.21.15.71:0/1547909487 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00bd10 con 0x7f50fc1f2000 2024-09-16T07:32:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.562+0000 7f5100a00640 1 -- 172.21.15.71:0/1547909487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc1f5a10 con 0x7f50fc1f2000 2024-09-16T07:32:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.563+0000 7f5100a00640 1 -- 172.21.15.71:0/1547909487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc0cb7e0 con 0x7f50fc1f2000 2024-09-16T07:32:40.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.564+0000 7f5100a00640 1 -- 172.21.15.71:0/1547909487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc0cb2f0 con 0x7f50fc1f2000 2024-09-16T07:32:40.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.567+0000 7f50f3400640 1 -- 172.21.15.71:0/1547909487 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f50ec0036d0 con 0x7f50fc1f2000 2024-09-16T07:32:40.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.568+0000 7f50f3400640 1 --2- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50dc079d50 0x7f50dc07c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:40.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.568+0000 7f50f3400640 1 -- 172.21.15.71:0/1547909487 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f50ec099d00 con 0x7f50fc1f2000 2024-09-16T07:32:40.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.574+0000 7f50f3400640 1 -- 172.21.15.71:0/1547909487 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f50ec020020 con 0x7f50fc1f2000 2024-09-16T07:32:40.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.577+0000 7f50faa00640 1 --2- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50dc079d50 0x7f50dc07c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:40.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.595+0000 7f50faa00640 1 --2- 172.21.15.71:0/1547909487 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50dc079d50 0x7f50dc07c210 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0ce390 tx=0x7f50e800d2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:40.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:40.957+0000 7f5100a00640 1 -- 172.21.15.71:0/1547909487 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone61947", "target": ["mon-mgr", ""]}) -- 0x7f50fc0d22f0 con 0x7f50dc079d50 2024-09-16T07:32:41.246 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:41.246+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:41.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.249+0000 7f50f3400640 1 -- 172.21.15.71:0/1547909487 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f50fc0d22f0 con 0x7f50dc079d50 2024-09-16T07:32:41.248 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:41.269 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:41.270 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone30628 2024-09-16T07:32:41.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.535+0000 7f1032200640 1 -- 172.21.15.71:0/2546785802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1010003a50 msgr2=0x7f1010003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:41.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.535+0000 7f1032200640 1 --2- 172.21.15.71:0/2546785802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1010003a50 0x7f1010003e70 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f101c00bb20 tx=0x7f1010005830 comp rx=0 tx=0).stop 2024-09-16T07:32:41.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.535+0000 7f102be00640 1 -- 172.21.15.71:0/2546785802 reap_dead start 2024-09-16T07:32:41.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.535+0000 7f1032200640 1 -- 172.21.15.71:0/2546785802 shutdown_connections 2024-09-16T07:32:41.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.535+0000 7f1032200640 1 -- 172.21.15.71:0/2546785802 >> 172.21.15.71:0/2546785802 conn(0x7f102c0b1840 msgr2=0x7f102c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:41.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.536+0000 7f1032200640 1 -- 172.21.15.71:0/2546785802 shutdown_connections 2024-09-16T07:32:41.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.536+0000 7f1032200640 1 -- 172.21.15.71:0/2546785802 wait complete. 2024-09-16T07:32:41.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.538+0000 7f1032200640 1 Processor -- start 2024-09-16T07:32:41.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.539+0000 7f1032200640 1 -- start start 2024-09-16T07:32:41.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f1032200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010003a50 0x7f102c15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:41.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f1032200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f102c0aee80 0x7f102c13fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:41.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f1032200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f102c13e430 0x7f102c13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:41.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f1032200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0b2680 con 0x7f102c13e430 2024-09-16T07:32:41.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f1032200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0cedd0 con 0x7f102c0aee80 2024-09-16T07:32:41.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f1032200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f102c073440 con 0x7f1010003a50 2024-09-16T07:32:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f102be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f102c13e430 0x7f102c13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f102be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f102c13e430 0x7f102c13e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48910/0 (socket says 172.21.15.71:48910) 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.541+0000 7f102be00640 1 -- 172.21.15.71:0/2561322738 learned_addr learned my addr 172.21.15.71:0/2561322738 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.542+0000 7f102be00640 1 -- 172.21.15.71:0/2561322738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010003a50 msgr2=0x7f102c15e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.542+0000 7f102be00640 1 --2- 172.21.15.71:0/2561322738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010003a50 0x7f102c15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.542+0000 7f102be00640 1 -- 172.21.15.71:0/2561322738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f102c0aee80 msgr2=0x7f102c13fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.542+0000 7f102be00640 1 --2- 172.21.15.71:0/2561322738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f102c0aee80 0x7f102c13fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.542+0000 7f102be00640 1 -- 172.21.15.71:0/2561322738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c140380 con 0x7f102c13e430 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.542+0000 7f102be00640 1 --2- 172.21.15.71:0/2561322738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f102c13e430 0x7f102c13e8b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f102c0d0cd0 tx=0x7f102400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.542+0000 7f1023400640 1 -- 172.21.15.71:0/2561322738 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102400cc20 con 0x7f102c13e430 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.542+0000 7f1032200640 1 -- 172.21.15.71:0/2561322738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c142ba0 con 0x7f102c13e430 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.543+0000 7f1023400640 1 -- 172.21.15.71:0/2561322738 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1024018070 con 0x7f102c13e430 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.543+0000 7f1023400640 1 -- 172.21.15.71:0/2561322738 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10240116c0 con 0x7f102c13e430 2024-09-16T07:32:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.543+0000 7f1032200640 1 -- 172.21.15.71:0/2561322738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c143160 con 0x7f102c13e430 2024-09-16T07:32:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.544+0000 7f1032200640 1 -- 172.21.15.71:0/2561322738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c0cc5c0 con 0x7f102c13e430 2024-09-16T07:32:41.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.548+0000 7f1023400640 1 -- 172.21.15.71:0/2561322738 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f1024011860 con 0x7f102c13e430 2024-09-16T07:32:41.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.550+0000 7f1023400640 1 --2- 172.21.15.71:0/2561322738 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f100c079d50 0x7f100c07c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:41.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.550+0000 7f1023400640 1 -- 172.21.15.71:0/2561322738 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f1024099b90 con 0x7f102c13e430 2024-09-16T07:32:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.553+0000 7f1023400640 1 -- 172.21.15.71:0/2561322738 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1024063c50 con 0x7f102c13e430 2024-09-16T07:32:41.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.571+0000 7f102b400640 1 --2- 172.21.15.71:0/2561322738 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f100c079d50 0x7f100c07c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:41.572+0000 7f102b400640 1 --2- 172.21.15.71:0/2561322738 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f100c079d50 0x7f100c07c210 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f101c036000 tx=0x7f101c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:42.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.000+0000 7f1032200640 1 -- 172.21.15.71:0/2561322738 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone30628", "target": ["mon-mgr", ""]}) -- 0x7f102c144f00 con 0x7f100c079d50 2024-09-16T07:32:42.383 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:42.383+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:42.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.396+0000 7f1023400640 1 -- 172.21.15.71:0/2561322738 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f102c144f00 con 0x7f100c079d50 2024-09-16T07:32:42.397 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:42.416 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:42.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone86499 2024-09-16T07:32:42.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.719+0000 7f6bc1c00640 1 -- 172.21.15.71:0/1569344486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bbc0b3370 msgr2=0x7f6bbc0cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:42.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.719+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/1569344486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bbc0b3370 0x7f6bbc0cebe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8005fd0 tx=0x7f6ba802a530 comp rx=0 tx=0).stop 2024-09-16T07:32:42.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.719+0000 7f6bc1c00640 1 -- 172.21.15.71:0/1569344486 shutdown_connections 2024-09-16T07:32:42.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.719+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/1569344486 >> v2:172.21.15.177:3300/0 conn(0x7f6bbc15c080 0x7f6bbc0cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:42.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.719+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/1569344486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bbc0b3370 0x7f6bbc0cebe0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:42.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.719+0000 7f6bc1c00640 1 -- 172.21.15.71:0/1569344486 >> 172.21.15.71:0/1569344486 conn(0x7f6bbc0b1840 msgr2=0x7f6bbc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:42.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.719+0000 7f6bc1c00640 1 -- 172.21.15.71:0/1569344486 shutdown_connections 2024-09-16T07:32:42.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.719+0000 7f6bc1c00640 1 -- 172.21.15.71:0/1569344486 wait complete. 2024-09-16T07:32:42.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.721+0000 7f6bc1c00640 1 Processor -- start 2024-09-16T07:32:42.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.722+0000 7f6bc1c00640 1 -- start start 2024-09-16T07:32:42.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.723+0000 7f6bc1c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc0b3370 0x7f6bbc13f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:42.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.723+0000 7f6bc1c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bbc15c080 0x7f6bbc1fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:42.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.723+0000 7f6bc1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15e980 0x7f6bbc13c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:42.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.723+0000 7f6bc1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc164ed0 con 0x7f6bbc15e980 2024-09-16T07:32:42.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.723+0000 7f6bc1c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc164d50 con 0x7f6bbc0b3370 2024-09-16T07:32:42.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.723+0000 7f6bc1c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc165050 con 0x7f6bbc15c080 2024-09-16T07:32:42.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.724+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15e980 0x7f6bbc13c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:42.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.724+0000 7f6bbb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15e980 0x7f6bbc13c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48914/0 (socket says 172.21.15.71:48914) 2024-09-16T07:32:42.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.724+0000 7f6bbb400640 1 -- 172.21.15.71:0/335350240 learned_addr learned my addr 172.21.15.71:0/335350240 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:42.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.724+0000 7f6bba000640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bbc15c080 0x7f6bbc1fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:42.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.724+0000 7f6bbb400640 1 -- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bbc15c080 msgr2=0x7f6bbc1fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:42.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.724+0000 7f6bbb400640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bbc15c080 0x7f6bbc1fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:42.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.724+0000 7f6bbb400640 1 -- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc0b3370 msgr2=0x7f6bbc13f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:42.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.724+0000 7f6bbb400640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc0b3370 0x7f6bbc13f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:42.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.725+0000 7f6bbb400640 1 -- 172.21.15.71:0/335350240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc1f93a0 con 0x7f6bbc15e980 2024-09-16T07:32:42.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.725+0000 7f6bbb400640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15e980 0x7f6bbc13c170 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400b990 tx=0x7f6bb400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:42.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.726+0000 7f6bb2a00640 1 -- 172.21.15.71:0/335350240 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4019070 con 0x7f6bbc15e980 2024-09-16T07:32:42.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.726+0000 7f6bb2a00640 1 -- 172.21.15.71:0/335350240 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bb400cc90 con 0x7f6bbc15e980 2024-09-16T07:32:42.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.726+0000 7f6bb2a00640 1 -- 172.21.15.71:0/335350240 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40146a0 con 0x7f6bbc15e980 2024-09-16T07:32:42.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.726+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc13c6b0 con 0x7f6bbc15e980 2024-09-16T07:32:42.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.726+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc1f9940 con 0x7f6bbc15e980 2024-09-16T07:32:42.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.729+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc165500 con 0x7f6bbc15e980 2024-09-16T07:32:42.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.734+0000 7f6bb2a00640 1 -- 172.21.15.71:0/335350240 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f6bb401c070 con 0x7f6bbc15e980 2024-09-16T07:32:42.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.736+0000 7f6bb2a00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b9c079d20 0x7f6b9c07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:42.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.736+0000 7f6bb2a00640 1 -- 172.21.15.71:0/335350240 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f6bb4099890 con 0x7f6bbc15e980 2024-09-16T07:32:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.740+0000 7f6bb2a00640 1 -- 172.21.15.71:0/335350240 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6bb4023020 con 0x7f6bbc15e980 2024-09-16T07:32:42.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.773+0000 7f6bbaa00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b9c079d20 0x7f6b9c07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:42.805+0000 7f6bbaa00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b9c079d20 0x7f6b9c07c1e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8033000 tx=0x7f6ba80046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:43.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:43.157+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone86499", "target": ["mon-mgr", ""]}) -- 0x7f6bbc0ce1a0 con 0x7f6b9c079d20 2024-09-16T07:32:44.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.966+0000 7f6bb2a00640 1 -- 172.21.15.71:0/335350240 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6bbc0ce1a0 con 0x7f6b9c079d20 2024-09-16T07:32:44.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.970+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b9c079d20 msgr2=0x7f6b9c07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:44.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.970+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b9c079d20 0x7f6b9c07c1e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8033000 tx=0x7f6ba80046c0 comp rx=0 tx=0).stop 2024-09-16T07:32:44.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.970+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15e980 msgr2=0x7f6bbc13c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.970+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15e980 0x7f6bbc13c170 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400b990 tx=0x7f6bb400be50 comp rx=0 tx=0).stop 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.971+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 shutdown_connections 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.971+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b9c079d20 0x7f6b9c07c1e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.971+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15e980 0x7f6bbc13c170 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.971+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bbc15c080 0x7f6bbc1fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.971+0000 7f6bc1c00640 1 --2- 172.21.15.71:0/335350240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bbc0b3370 0x7f6bbc13f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.971+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 >> 172.21.15.71:0/335350240 conn(0x7f6bbc0b1840 msgr2=0x7f6bbc163e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.971+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 shutdown_connections 2024-09-16T07:32:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:44.971+0000 7f6bc1c00640 1 -- 172.21.15.71:0/335350240 wait complete. 2024-09-16T07:32:44.994 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone614610 2024-09-16T07:32:45.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.283+0000 7f5d2dc00640 1 -- 172.21.15.71:0/4039733911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0c004800 msgr2=0x7f5d0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:45.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.283+0000 7f5d2dc00640 1 --2- 172.21.15.71:0/4039733911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0c004800 0x7f5d0c004c80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5d280b5880 tx=0x7f5d0c005830 comp rx=0 tx=0).stop 2024-09-16T07:32:45.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.283+0000 7f5d2dc00640 1 -- 172.21.15.71:0/4039733911 shutdown_connections 2024-09-16T07:32:45.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.283+0000 7f5d2dc00640 1 --2- 172.21.15.71:0/4039733911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d0c004800 0x7f5d0c004c80 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:45.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.283+0000 7f5d2dc00640 1 -- 172.21.15.71:0/4039733911 >> 172.21.15.71:0/4039733911 conn(0x7f5d2809d0b0 msgr2=0x7f5d280d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:45.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.283+0000 7f5d2dc00640 1 -- 172.21.15.71:0/4039733911 shutdown_connections 2024-09-16T07:32:45.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.283+0000 7f5d2dc00640 1 -- 172.21.15.71:0/4039733911 wait complete. 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.286+0000 7f5d2dc00640 1 Processor -- start 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.286+0000 7f5d2dc00640 1 -- start start 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d2dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d0c004800 0x7f5d281f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d2dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d280b0a80 0x7f5d2815e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d2dc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d280b2020 0x7f5d280b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d2dc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5d280d22f0 con 0x7f5d280b0a80 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d2dc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d280b5210 con 0x7f5d0c004800 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d2dc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5d280d2470 con 0x7f5d280b2020 2024-09-16T07:32:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d27400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d280b0a80 0x7f5d2815e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d27400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d280b0a80 0x7f5d2815e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35328/0 (socket says 172.21.15.71:35328) 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.287+0000 7f5d27400640 1 -- 172.21.15.71:0/97551972 learned_addr learned my addr 172.21.15.71:0/97551972 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.288+0000 7f5d27e00640 1 --2- 172.21.15.71:0/97551972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d0c004800 0x7f5d281f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.288+0000 7f5d27400640 1 -- 172.21.15.71:0/97551972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d280b2020 msgr2=0x7f5d280b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.288+0000 7f5d27400640 1 --2- 172.21.15.71:0/97551972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d280b2020 0x7f5d280b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.288+0000 7f5d27400640 1 -- 172.21.15.71:0/97551972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d0c004800 msgr2=0x7f5d281f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.288+0000 7f5d27400640 1 --2- 172.21.15.71:0/97551972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d0c004800 0x7f5d281f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.288+0000 7f5d27400640 1 -- 172.21.15.71:0/97551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d280b0f80 con 0x7f5d280b0a80 2024-09-16T07:32:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.288+0000 7f5d27400640 1 --2- 172.21.15.71:0/97551972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d280b0a80 0x7f5d2815e4c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5d1402a7d0 tx=0x7f5d14006150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.289+0000 7f5d1fe00640 1 -- 172.21.15.71:0/97551972 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14003810 con 0x7f5d280b0a80 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.289+0000 7f5d1fe00640 1 -- 172.21.15.71:0/97551972 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d14003aa0 con 0x7f5d280b0a80 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.289+0000 7f5d2dc00640 1 -- 172.21.15.71:0/97551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d280b1260 con 0x7f5d280b0a80 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.289+0000 7f5d1fe00640 1 -- 172.21.15.71:0/97551972 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d140064d0 con 0x7f5d280b0a80 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.289+0000 7f5d2dc00640 1 -- 172.21.15.71:0/97551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d280b1820 con 0x7f5d280b0a80 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.291+0000 7f5d2dc00640 1 -- 172.21.15.71:0/97551972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d280d29d0 con 0x7f5d280b0a80 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.292+0000 7f5d1fe00640 1 -- 172.21.15.71:0/97551972 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f5d1403e440 con 0x7f5d280b0a80 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.294+0000 7f5d1fe00640 1 --2- 172.21.15.71:0/97551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d08079d20 0x7f5d0807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.294+0000 7f5d1fe00640 1 -- 172.21.15.71:0/97551972 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f5d140b9680 con 0x7f5d280b0a80 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.297+0000 7f5d27e00640 1 --2- 172.21.15.71:0/97551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d08079d20 0x7f5d0807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.299+0000 7f5d27e00640 1 --2- 172.21.15.71:0/97551972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5d08079d20 0x7f5d0807c1e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5d280b3060 tx=0x7f5d1800b260 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.301+0000 7f5d1fe00640 1 -- 172.21.15.71:0/97551972 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5d14083320 con 0x7f5d280b0a80 2024-09-16T07:32:45.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:45.669+0000 7f5d2dc00640 1 -- 172.21.15.71:0/97551972 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone614610", "target": ["mon-mgr", ""]}) -- 0x7f5d280cb0f0 con 0x7f5d08079d20 2024-09-16T07:32:45.781 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:32:45.787 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:32:46.163 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:46.163+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.163+0000 7f5d1fe00640 1 -- 172.21.15.71:0/97551972 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f5d280cb0f0 con 0x7f5d08079d20 2024-09-16T07:32:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:46.175 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:46.176 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone673111 2024-09-16T07:32:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.420+0000 7f8402e00640 1 -- 172.21.15.71:0/779592681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83fc161460 msgr2=0x7f83fc0d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.421+0000 7f8402e00640 1 --2- 172.21.15.71:0/779592681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83fc161460 0x7f83fc0d22f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f83f0005fd0 tx=0x7f83f002aa10 comp rx=0 tx=0).stop 2024-09-16T07:32:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.421+0000 7f8402e00640 1 -- 172.21.15.71:0/779592681 shutdown_connections 2024-09-16T07:32:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.421+0000 7f8402e00640 1 --2- 172.21.15.71:0/779592681 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83fc0b5480 0x7f83fc0d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.421+0000 7f8402e00640 1 --2- 172.21.15.71:0/779592681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83fc161460 0x7f83fc0d22f0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.421+0000 7f8402e00640 1 -- 172.21.15.71:0/779592681 >> 172.21.15.71:0/779592681 conn(0x7f83fc09d0b0 msgr2=0x7f83fc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.421+0000 7f8402e00640 1 -- 172.21.15.71:0/779592681 shutdown_connections 2024-09-16T07:32:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.421+0000 7f8402e00640 1 -- 172.21.15.71:0/779592681 wait complete. 2024-09-16T07:32:46.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.423+0000 7f8402e00640 1 Processor -- start 2024-09-16T07:32:46.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.423+0000 7f8402e00640 1 -- start start 2024-09-16T07:32:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.425+0000 7f8402e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83fc0b4f40 0x7f83fc158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.425+0000 7f8402e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83fc0b5480 0x7f83fc15aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.425+0000 7f8402e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83fc161460 0x7f83fc157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.425+0000 7f8402e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc0cf0b0 con 0x7f83fc0b5480 2024-09-16T07:32:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.425+0000 7f8402e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc0cef30 con 0x7f83fc0b4f40 2024-09-16T07:32:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.425+0000 7f8402e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f83fc0cf230 con 0x7f83fc161460 2024-09-16T07:32:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.426+0000 7f8401000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83fc0b5480 0x7f83fc15aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.426+0000 7f8401000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83fc0b5480 0x7f83fc15aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35338/0 (socket says 172.21.15.71:35338) 2024-09-16T07:32:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.426+0000 7f8401000640 1 -- 172.21.15.71:0/1328556880 learned_addr learned my addr 172.21.15.71:0/1328556880 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:46.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.426+0000 7f8401000640 1 -- 172.21.15.71:0/1328556880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83fc161460 msgr2=0x7f83fc157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:46.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.426+0000 7f8401000640 1 --2- 172.21.15.71:0/1328556880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83fc161460 0x7f83fc157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:46.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.426+0000 7f8401000640 1 -- 172.21.15.71:0/1328556880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83fc0b4f40 msgr2=0x7f83fc158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:46.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.426+0000 7f8401000640 1 --2- 172.21.15.71:0/1328556880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83fc0b4f40 0x7f83fc158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:46.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.426+0000 7f8401000640 1 -- 172.21.15.71:0/1328556880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc15b580 con 0x7f83fc0b5480 2024-09-16T07:32:46.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.427+0000 7f8401000640 1 --2- 172.21.15.71:0/1328556880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83fc0b5480 0x7f83fc15aec0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f83e800d750 tx=0x7f83e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.427+0000 7f83f9600640 1 -- 172.21.15.71:0/1328556880 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8017020 con 0x7f83fc0b5480 2024-09-16T07:32:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.427+0000 7f83f9600640 1 -- 172.21.15.71:0/1328556880 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83e80097c0 con 0x7f83fc0b5480 2024-09-16T07:32:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.427+0000 7f83f9600640 1 -- 172.21.15.71:0/1328556880 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e800de60 con 0x7f83fc0b5480 2024-09-16T07:32:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.427+0000 7f8402e00640 1 -- 172.21.15.71:0/1328556880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc1f8010 con 0x7f83fc0b5480 2024-09-16T07:32:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.428+0000 7f8402e00640 1 -- 172.21.15.71:0/1328556880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc1f85d0 con 0x7f83fc0b5480 2024-09-16T07:32:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.430+0000 7f8402e00640 1 -- 172.21.15.71:0/1328556880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc1f7e30 con 0x7f83fc0b5480 2024-09-16T07:32:46.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.435+0000 7f83f9600640 1 -- 172.21.15.71:0/1328556880 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f83e801a020 con 0x7f83fc0b5480 2024-09-16T07:32:46.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.438+0000 7f83f9600640 1 --2- 172.21.15.71:0/1328556880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f83dc079df0 0x7f83dc07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:46.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.438+0000 7f83f9600640 1 -- 172.21.15.71:0/1328556880 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f83e8099a80 con 0x7f83fc0b5480 2024-09-16T07:32:46.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.439+0000 7f8401a00640 1 --2- 172.21.15.71:0/1328556880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f83dc079df0 0x7f83dc07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:46.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.442+0000 7f8401a00640 1 --2- 172.21.15.71:0/1328556880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f83dc079df0 0x7f83dc07c2b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83f0035000 tx=0x7f83f0004a40 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:46.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.443+0000 7f83f9600640 1 -- 172.21.15.71:0/1328556880 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f83fc1f7e30 con 0x7f83fc0b5480 2024-09-16T07:32:46.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:46.851+0000 7f8402e00640 1 -- 172.21.15.71:0/1328556880 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone673111", "target": ["mon-mgr", ""]}) -- 0x7f83fc069d80 con 0x7f83dc079df0 2024-09-16T07:32:47.160 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:32:47.159+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:47.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.159+0000 7f83f9600640 1 -- 172.21.15.71:0/1328556880 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f83f0003a70 con 0x7f83dc079df0 2024-09-16T07:32:47.161 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:32:47.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:32:47.173 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone219912 2024-09-16T07:32:47.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.467+0000 7fbc71a00640 1 -- 172.21.15.71:0/294223430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 msgr2=0x7fbc50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:47.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.467+0000 7fbc71a00640 1 --2- 172.21.15.71:0/294223430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 0x7fbc50004c80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c001690 tx=0x7fbc50005830 comp rx=0 tx=0).stop 2024-09-16T07:32:47.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.467+0000 7fbc71a00640 1 -- 172.21.15.71:0/294223430 shutdown_connections 2024-09-16T07:32:47.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.467+0000 7fbc71a00640 1 --2- 172.21.15.71:0/294223430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 0x7fbc50004c80 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:47.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.467+0000 7fbc71a00640 1 -- 172.21.15.71:0/294223430 >> 172.21.15.71:0/294223430 conn(0x7fbc6c0d40a0 msgr2=0x7fbc6c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:47.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.468+0000 7fbc71a00640 1 -- 172.21.15.71:0/294223430 shutdown_connections 2024-09-16T07:32:47.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.468+0000 7fbc71a00640 1 -- 172.21.15.71:0/294223430 wait complete. 2024-09-16T07:32:47.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.470+0000 7fbc71a00640 1 Processor -- start 2024-09-16T07:32:47.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.470+0000 7fbc71a00640 1 -- start start 2024-09-16T07:32:47.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.471+0000 7fbc71a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 0x7fbc6c0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:47.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.472+0000 7fbc71a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc6c06e5a0 0x7fbc6c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:47.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.472+0000 7fbc71a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc6c1f4f20 0x7fbc6c1f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:47.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.472+0000 7fbc71a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c0cfc60 con 0x7fbc50004800 2024-09-16T07:32:47.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.472+0000 7fbc71a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c0cfae0 con 0x7fbc6c1f4f20 2024-09-16T07:32:47.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.472+0000 7fbc71a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c0cfde0 con 0x7fbc6c06e5a0 2024-09-16T07:32:47.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.472+0000 7fbc6aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 0x7fbc6c0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:47.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 0x7fbc6c0cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35360/0 (socket says 172.21.15.71:35360) 2024-09-16T07:32:47.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6aa00640 1 -- 172.21.15.71:0/1813445133 learned_addr learned my addr 172.21.15.71:0/1813445133 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:47.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6b400640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc6c1f4f20 0x7fbc6c1f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:47.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6a000640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc6c06e5a0 0x7fbc6c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:47.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6aa00640 1 -- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc6c06e5a0 msgr2=0x7fbc6c06ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:47.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6aa00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc6c06e5a0 0x7fbc6c06ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:47.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6aa00640 1 -- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc6c1f4f20 msgr2=0x7fbc6c1f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:47.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6aa00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc6c1f4f20 0x7fbc6c1f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:47.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.473+0000 7fbc6aa00640 1 -- 172.21.15.71:0/1813445133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c150790 con 0x7fbc50004800 2024-09-16T07:32:47.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.474+0000 7fbc6aa00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 0x7fbc6c0cc830 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b550 tx=0x7fbc5800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:47.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.475+0000 7fbc62a00640 1 -- 172.21.15.71:0/1813445133 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58014070 con 0x7fbc50004800 2024-09-16T07:32:47.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.475+0000 7fbc62a00640 1 -- 172.21.15.71:0/1813445133 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc58005cf0 con 0x7fbc50004800 2024-09-16T07:32:47.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.475+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c150a70 con 0x7fbc50004800 2024-09-16T07:32:47.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.475+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c151030 con 0x7fbc50004800 2024-09-16T07:32:47.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.475+0000 7fbc62a00640 1 -- 172.21.15.71:0/1813445133 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc580108f0 con 0x7fbc50004800 2024-09-16T07:32:47.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.477+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc44005290 con 0x7fbc50004800 2024-09-16T07:32:47.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.478+0000 7fbc62a00640 1 -- 172.21.15.71:0/1813445133 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7fbc5800e070 con 0x7fbc50004800 2024-09-16T07:32:47.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.479+0000 7fbc62a00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc4c079d50 0x7fbc4c07c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:47.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.480+0000 7fbc62a00640 1 -- 172.21.15.71:0/1813445133 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fbc580992e0 con 0x7fbc50004800 2024-09-16T07:32:47.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.484+0000 7fbc62a00640 1 -- 172.21.15.71:0/1813445133 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbc58063020 con 0x7fbc50004800 2024-09-16T07:32:47.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.497+0000 7fbc6a000640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc4c079d50 0x7fbc4c07c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:47.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.536+0000 7fbc6a000640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc4c079d50 0x7fbc4c07c210 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c032000 tx=0x7fbc5c035040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:47.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:47.936+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone219912", "target": ["mon-mgr", ""]}) -- 0x7fbc44000bc0 con 0x7fbc4c079d50 2024-09-16T07:32:49.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.540+0000 7fbc62a00640 1 -- 172.21.15.71:0/1813445133 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc44000bc0 con 0x7fbc4c079d50 2024-09-16T07:32:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.545+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc4c079d50 msgr2=0x7fbc4c07c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.545+0000 7fbc71a00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc4c079d50 0x7fbc4c07c210 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c032000 tx=0x7fbc5c035040 comp rx=0 tx=0).stop 2024-09-16T07:32:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.545+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 msgr2=0x7fbc6c0cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.545+0000 7fbc71a00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 0x7fbc6c0cc830 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b550 tx=0x7fbc5800ba10 comp rx=0 tx=0).stop 2024-09-16T07:32:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.545+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 shutdown_connections 2024-09-16T07:32:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.546+0000 7fbc71a00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc4c079d50 0x7fbc4c07c210 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.546+0000 7fbc71a00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc6c1f4f20 0x7fbc6c1f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.546+0000 7fbc71a00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc6c06e5a0 0x7fbc6c06ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.546+0000 7fbc71a00640 1 --2- 172.21.15.71:0/1813445133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50004800 0x7fbc6c0cc830 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.546+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 >> 172.21.15.71:0/1813445133 conn(0x7fbc6c0d40a0 msgr2=0x7fbc6c1607d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.546+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 shutdown_connections 2024-09-16T07:32:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.546+0000 7fbc71a00640 1 -- 172.21.15.71:0/1813445133 wait complete. 2024-09-16T07:32:49.578 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone594713 2024-09-16T07:32:49.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.884+0000 7f7cd3000640 1 -- 172.21.15.71:0/4044511488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc0cb4a0 msgr2=0x7f7ccc15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:49.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.884+0000 7f7cd3000640 1 --2- 172.21.15.71:0/4044511488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc0cb4a0 0x7f7ccc15e660 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009830 tx=0x7f7cc002a4a0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.884+0000 7f7cd3000640 1 -- 172.21.15.71:0/4044511488 shutdown_connections 2024-09-16T07:32:49.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.884+0000 7f7cd3000640 1 --2- 172.21.15.71:0/4044511488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0cbf90 0x7f7ccc1649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.884+0000 7f7cd3000640 1 --2- 172.21.15.71:0/4044511488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc0cb4a0 0x7f7ccc15e660 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.884+0000 7f7cd3000640 1 -- 172.21.15.71:0/4044511488 >> 172.21.15.71:0/4044511488 conn(0x7f7ccc0d0b20 msgr2=0x7f7ccc0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:49.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.884+0000 7f7cd3000640 1 -- 172.21.15.71:0/4044511488 shutdown_connections 2024-09-16T07:32:49.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.884+0000 7f7cd3000640 1 -- 172.21.15.71:0/4044511488 wait complete. 2024-09-16T07:32:49.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.886+0000 7f7cd3000640 1 Processor -- start 2024-09-16T07:32:49.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.887+0000 7f7cd3000640 1 -- start start 2024-09-16T07:32:49.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.888+0000 7f7cd3000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0cb4a0 0x7f7ccc0c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:49.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.888+0000 7f7cd3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ccc0cbf90 0x7f7ccc0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:49.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.888+0000 7f7cd3000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc15eba0 0x7f7ccc0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:49.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.888+0000 7f7cd3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc06ccf0 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.888+0000 7f7cd3000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc06cb70 con 0x7f7ccc15eba0 2024-09-16T07:32:49.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.888+0000 7f7cd3000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc06ce70 con 0x7f7ccc0cb4a0 2024-09-16T07:32:49.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.889+0000 7f7cd1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ccc0cbf90 0x7f7ccc0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:49.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.889+0000 7f7cd1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ccc0cbf90 0x7f7ccc0b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35372/0 (socket says 172.21.15.71:35372) 2024-09-16T07:32:49.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.889+0000 7f7cd1200640 1 -- 172.21.15.71:0/3477347281 learned_addr learned my addr 172.21.15.71:0/3477347281 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:49.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.889+0000 7f7cd2600640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc15eba0 0x7f7ccc0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:49.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.889+0000 7f7cd1200640 1 -- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0cb4a0 msgr2=0x7f7ccc0c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.889+0000 7f7cd1200640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0cb4a0 0x7f7ccc0c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.889+0000 7f7cd1200640 1 -- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc15eba0 msgr2=0x7f7ccc0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.889+0000 7f7cd1200640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc15eba0 0x7f7ccc0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.890+0000 7f7cd1200640 1 -- 172.21.15.71:0/3477347281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc1f5460 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.890+0000 7f7cd1200640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ccc0cbf90 0x7f7ccc0b5470 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c520 tx=0x7f7cb800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.891+0000 7f7cc9600640 1 -- 172.21.15.71:0/3477347281 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8017070 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.891+0000 7f7cc9600640 1 -- 172.21.15.71:0/3477347281 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cb80035f0 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.891+0000 7f7cc9600640 1 -- 172.21.15.71:0/3477347281 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8003ac0 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.891+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ccc1f5700 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.891+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ccc1f5fa0 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.892+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ccc15e660 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.896+0000 7f7cc9600640 1 -- 172.21.15.71:0/3477347281 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f7cb801a070 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.897+0000 7f7cc9600640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7cac079d70 0x7f7cac07c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:49.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.897+0000 7f7cc9600640 1 -- 172.21.15.71:0/3477347281 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f7cb8098f50 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.901+0000 7f7cc9600640 1 -- 172.21.15.71:0/3477347281 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7cb8021020 con 0x7f7ccc0cbf90 2024-09-16T07:32:49.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.905+0000 7f7cd1c00640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7cac079d70 0x7f7cac07c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:49.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:49.910+0000 7f7cd1c00640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7cac079d70 0x7f7cac07c230 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0034000 tx=0x7f7cc0033040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:50.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:50.267+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone594713", "target": ["mon-mgr", ""]}) -- 0x7f7ccc0d18c0 con 0x7f7cac079d70 2024-09-16T07:32:52.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.137+0000 7f7cc9600640 1 -- 172.21.15.71:0/3477347281 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7ccc0d18c0 con 0x7f7cac079d70 2024-09-16T07:32:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7cac079d70 msgr2=0x7f7cac07c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7cac079d70 0x7f7cac07c230 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0034000 tx=0x7f7cc0033040 comp rx=0 tx=0).stop 2024-09-16T07:32:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ccc0cbf90 msgr2=0x7f7ccc0b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ccc0cbf90 0x7f7ccc0b5470 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800c520 tx=0x7f7cb800c9e0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 shutdown_connections 2024-09-16T07:32:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7cac079d70 0x7f7cac07c230 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc15eba0 0x7f7ccc0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ccc0cbf90 0x7f7ccc0b5470 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 --2- 172.21.15.71:0/3477347281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0cb4a0 0x7f7ccc0c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 >> 172.21.15.71:0/3477347281 conn(0x7f7ccc0d0b20 msgr2=0x7f7ccc0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:52.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 shutdown_connections 2024-09-16T07:32:52.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.140+0000 7f7cd3000640 1 -- 172.21.15.71:0/3477347281 wait complete. 2024-09-16T07:32:52.163 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone43314 2024-09-16T07:32:52.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.458+0000 7f852a600640 1 -- 172.21.15.71:0/1438548405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 msgr2=0x7f85240b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:52.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.458+0000 7f852a600640 1 --2- 172.21.15.71:0/1438548405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 0x7f85240b33b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8510005fd0 tx=0x7f851002a560 comp rx=0 tx=0).stop 2024-09-16T07:32:52.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.458+0000 7f852a600640 1 -- 172.21.15.71:0/1438548405 shutdown_connections 2024-09-16T07:32:52.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.458+0000 7f852a600640 1 --2- 172.21.15.71:0/1438548405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85240d2a00 0x7f85240b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.458+0000 7f852a600640 1 --2- 172.21.15.71:0/1438548405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 0x7f85240b33b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.458+0000 7f852a600640 1 -- 172.21.15.71:0/1438548405 >> 172.21.15.71:0/1438548405 conn(0x7f852409d0b0 msgr2=0x7f85240d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:52.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.458+0000 7f852a600640 1 -- 172.21.15.71:0/1438548405 shutdown_connections 2024-09-16T07:32:52.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.458+0000 7f852a600640 1 -- 172.21.15.71:0/1438548405 wait complete. 2024-09-16T07:32:52.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.461+0000 7f852a600640 1 Processor -- start 2024-09-16T07:32:52.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.461+0000 7f852a600640 1 -- start start 2024-09-16T07:32:52.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.463+0000 7f852a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 0x7f85241e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:52.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.463+0000 7f852a600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85240d2a00 0x7f852415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:52.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.463+0000 7f852a600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8524161460 0x7f85241f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:52.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.464+0000 7f852a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f85240b50c0 con 0x7f85240cf830 2024-09-16T07:32:52.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.464+0000 7f852a600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85240b4f40 con 0x7f85240d2a00 2024-09-16T07:32:52.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.464+0000 7f852a600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f85240b3e30 con 0x7f8524161460 2024-09-16T07:32:52.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.464+0000 7f8529200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 0x7f85241e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:52.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.464+0000 7f8529200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 0x7f85241e9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35388/0 (socket says 172.21.15.71:35388) 2024-09-16T07:32:52.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.464+0000 7f8529200640 1 -- 172.21.15.71:0/3311345715 learned_addr learned my addr 172.21.15.71:0/3311345715 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:52.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.464+0000 7f8523e00640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85240d2a00 0x7f852415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:52.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.464+0000 7f8529c00640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8524161460 0x7f85241f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:52.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.465+0000 7f8529200640 1 -- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8524161460 msgr2=0x7f85241f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:52.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.465+0000 7f8529200640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8524161460 0x7f85241f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.465+0000 7f8529200640 1 -- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85240d2a00 msgr2=0x7f852415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:52.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.465+0000 7f8529200640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85240d2a00 0x7f852415e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:52.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.465+0000 7f8529200640 1 -- 172.21.15.71:0/3311345715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85240b75a0 con 0x7f85240cf830 2024-09-16T07:32:52.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.466+0000 7f8529200640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 0x7f85241e9880 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f851800c3c0 tx=0x7f851800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:52.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.466+0000 7f8520c00640 1 -- 172.21.15.71:0/3311345715 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518017070 con 0x7f85240cf830 2024-09-16T07:32:52.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.466+0000 7f8520c00640 1 -- 172.21.15.71:0/3311345715 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85180035f0 con 0x7f85240cf830 2024-09-16T07:32:52.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.466+0000 7f8520c00640 1 -- 172.21.15.71:0/3311345715 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518003a80 con 0x7f85240cf830 2024-09-16T07:32:52.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.467+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85240b7880 con 0x7f85240cf830 2024-09-16T07:32:52.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.467+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85240b25b0 con 0x7f85240cf830 2024-09-16T07:32:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.469+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85240b42e0 con 0x7f85240cf830 2024-09-16T07:32:52.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.471+0000 7f8520c00640 1 -- 172.21.15.71:0/3311345715 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f851801a070 con 0x7f85240cf830 2024-09-16T07:32:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.473+0000 7f8520c00640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8504079d20 0x7f850407c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.473+0000 7f8520c00640 1 -- 172.21.15.71:0/3311345715 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f85180987c0 con 0x7f85240cf830 2024-09-16T07:32:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.474+0000 7f8523e00640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8504079d20 0x7f850407c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.478+0000 7f8523e00640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8504079d20 0x7f850407c1e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8510033000 tx=0x7f8510004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.480+0000 7f8520c00640 1 -- 172.21.15.71:0/3311345715 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8518062290 con 0x7f85240cf830 2024-09-16T07:32:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:52.894+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone43314", "target": ["mon-mgr", ""]}) -- 0x7f8524069d80 con 0x7f8504079d20 2024-09-16T07:32:54.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.774+0000 7f8520c00640 1 -- 172.21.15.71:0/3311345715 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8510004380 con 0x7f8504079d20 2024-09-16T07:32:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8504079d20 msgr2=0x7f850407c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8504079d20 0x7f850407c1e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8510033000 tx=0x7f8510004730 comp rx=0 tx=0).stop 2024-09-16T07:32:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 msgr2=0x7f85241e9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 0x7f85241e9880 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f851800c3c0 tx=0x7f851800c880 comp rx=0 tx=0).stop 2024-09-16T07:32:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 shutdown_connections 2024-09-16T07:32:54.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8504079d20 0x7f850407c1e0 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8510033000 tx=0x7f8510004730 comp rx=0 tx=0).stop 2024-09-16T07:32:54.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8524161460 0x7f85241f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:54.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85240d2a00 0x7f852415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:54.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 --2- 172.21.15.71:0/3311345715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85240cf830 0x7f85241e9880 secure :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f851800c3c0 tx=0x7f851800c880 comp rx=0 tx=0).stop 2024-09-16T07:32:54.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.781+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 >> 172.21.15.71:0/3311345715 conn(0x7f852409d0b0 msgr2=0x7f85240d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:54.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.783+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 shutdown_connections 2024-09-16T07:32:54.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:54.784+0000 7f852a600640 1 -- 172.21.15.71:0/3311345715 wait complete. 2024-09-16T07:32:54.819 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone644815 2024-09-16T07:32:55.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.110+0000 7fbc34400640 1 -- 172.21.15.71:0/2858644060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0cf2f0 msgr2=0x7fbc2c15e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:55.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.110+0000 7fbc34400640 1 --2- 172.21.15.71:0/2858644060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0cf2f0 0x7fbc2c15e980 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009830 tx=0x7fbc1802a510 comp rx=0 tx=0).stop 2024-09-16T07:32:55.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.110+0000 7fbc34400640 1 -- 172.21.15.71:0/2858644060 shutdown_connections 2024-09-16T07:32:55.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.110+0000 7fbc34400640 1 --2- 172.21.15.71:0/2858644060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0cf2f0 0x7fbc2c15e980 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:55.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.110+0000 7fbc34400640 1 -- 172.21.15.71:0/2858644060 >> 172.21.15.71:0/2858644060 conn(0x7fbc2c0b1840 msgr2=0x7fbc2c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:55.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.110+0000 7fbc34400640 1 -- 172.21.15.71:0/2858644060 shutdown_connections 2024-09-16T07:32:55.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.110+0000 7fbc34400640 1 -- 172.21.15.71:0/2858644060 wait complete. 2024-09-16T07:32:55.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.112+0000 7fbc34400640 1 Processor -- start 2024-09-16T07:32:55.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.112+0000 7fbc34400640 1 -- start start 2024-09-16T07:32:55.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc34400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0b3370 0x7fbc2c06fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:55.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc34400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc2c0cf2f0 0x7fbc2c1e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:55.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc34400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2c15c080 0x7fbc2c13b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:55.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc34400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c0ce0b0 con 0x7fbc2c0b3370 2024-09-16T07:32:55.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc34400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c0cdf30 con 0x7fbc2c15c080 2024-09-16T07:32:55.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc34400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c0ce230 con 0x7fbc2c0cf2f0 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc31a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0b3370 0x7fbc2c06fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc31a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0b3370 0x7fbc2c06fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39268/0 (socket says 172.21.15.71:39268) 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.114+0000 7fbc31a00640 1 -- 172.21.15.71:0/3716009850 learned_addr learned my addr 172.21.15.71:0/3716009850 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc31a00640 1 -- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc2c0cf2f0 msgr2=0x7fbc2c1e5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc31a00640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc2c0cf2f0 0x7fbc2c1e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc31a00640 1 -- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2c15c080 msgr2=0x7fbc2c13b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc31a00640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2c15c080 0x7fbc2c13b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc31a00640 1 -- 172.21.15.71:0/3716009850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c13fbc0 con 0x7fbc2c0b3370 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc31a00640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0b3370 0x7fbc2c06fa80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000c530 tx=0x7fbc2000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc29600640 1 -- 172.21.15.71:0/3716009850 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20017070 con 0x7fbc2c0b3370 2024-09-16T07:32:55.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc29600640 1 -- 172.21.15.71:0/3716009850 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc200035f0 con 0x7fbc2c0b3370 2024-09-16T07:32:55.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c13fea0 con 0x7fbc2c0b3370 2024-09-16T07:32:55.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.115+0000 7fbc29600640 1 -- 172.21.15.71:0/3716009850 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20003a40 con 0x7fbc2c0b3370 2024-09-16T07:32:55.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.116+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c06ffc0 con 0x7fbc2c0b3370 2024-09-16T07:32:55.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.118+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c0ce790 con 0x7fbc2c0b3370 2024-09-16T07:32:55.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.121+0000 7fbc29600640 1 -- 172.21.15.71:0/3716009850 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7fbc2001a070 con 0x7fbc2c0b3370 2024-09-16T07:32:55.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.123+0000 7fbc29600640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc0c079d50 0x7fbc0c07c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:55.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.123+0000 7fbc29600640 1 -- 172.21.15.71:0/3716009850 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fbc20098ab0 con 0x7fbc2c0b3370 2024-09-16T07:32:55.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.131+0000 7fbc29600640 1 -- 172.21.15.71:0/3716009850 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbc20062b70 con 0x7fbc2c0b3370 2024-09-16T07:32:55.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.139+0000 7fbc31000640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc0c079d50 0x7fbc0c07c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:55.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.182+0000 7fbc31000640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc0c079d50 0x7fbc0c07c210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbc18034000 tx=0x7fbc180046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:55.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:55.550+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone644815", "target": ["mon-mgr", ""]}) -- 0x7fbc2c073940 con 0x7fbc0c079d50 2024-09-16T07:32:56.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.379+0000 7fbc29600640 1 -- 172.21.15.71:0/3716009850 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc2c073940 con 0x7fbc0c079d50 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc0c079d50 msgr2=0x7fbc0c07c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc0c079d50 0x7fbc0c07c210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbc18034000 tx=0x7fbc180046c0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0b3370 msgr2=0x7fbc2c06fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0b3370 0x7fbc2c06fa80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000c530 tx=0x7fbc2000c9f0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 shutdown_connections 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc0c079d50 0x7fbc0c07c210 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2c15c080 0x7fbc2c13b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc2c0cf2f0 0x7fbc2c1e5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 --2- 172.21.15.71:0/3716009850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc2c0b3370 0x7fbc2c06fa80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 >> 172.21.15.71:0/3716009850 conn(0x7fbc2c0b1840 msgr2=0x7fbc2c074140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:56.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.383+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 shutdown_connections 2024-09-16T07:32:56.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.384+0000 7fbc34400640 1 -- 172.21.15.71:0/3716009850 wait complete. 2024-09-16T07:32:56.413 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone418816 2024-09-16T07:32:56.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.650+0000 7fc50be00640 1 -- 172.21.15.71:0/1997918641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50415bf10 msgr2=0x7fc5041604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:56.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.650+0000 7fc50be00640 1 --2- 172.21.15.71:0/1997918641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50415bf10 0x7fc5041604b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc5040ce340 tx=0x7fc4f802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.651+0000 7fc50be00640 1 -- 172.21.15.71:0/1997918641 shutdown_connections 2024-09-16T07:32:56.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.651+0000 7fc50be00640 1 --2- 172.21.15.71:0/1997918641 >> v2:172.21.15.177:3300/0 conn(0x7fc5040d08e0 0x7fc5040b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.651+0000 7fc50be00640 1 --2- 172.21.15.71:0/1997918641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc50415bf10 0x7fc5041604b0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.651+0000 7fc50be00640 1 -- 172.21.15.71:0/1997918641 >> 172.21.15.71:0/1997918641 conn(0x7fc5040d1e20 msgr2=0x7fc5040d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:56.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.652+0000 7fc50be00640 1 -- 172.21.15.71:0/1997918641 shutdown_connections 2024-09-16T07:32:56.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.652+0000 7fc50be00640 1 -- 172.21.15.71:0/1997918641 wait complete. 2024-09-16T07:32:56.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.654+0000 7fc50be00640 1 Processor -- start 2024-09-16T07:32:56.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.654+0000 7fc50be00640 1 -- start start 2024-09-16T07:32:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.655+0000 7fc50be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040cd6f0 0x7fc5041f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.655+0000 7fc50be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5040d08e0 0x7fc50406dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.655+0000 7fc50be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc50415bf10 0x7fc50406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.655+0000 7fc50be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc5040cdce0 con 0x7fc5040cd6f0 2024-09-16T07:32:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.655+0000 7fc50be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5040cdb60 con 0x7fc5040d08e0 2024-09-16T07:32:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.655+0000 7fc50be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc5040cf9f0 con 0x7fc50415bf10 2024-09-16T07:32:56.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.656+0000 7fc509400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040cd6f0 0x7fc5041f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:56.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.656+0000 7fc509400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040cd6f0 0x7fc5041f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39276/0 (socket says 172.21.15.71:39276) 2024-09-16T07:32:56.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.656+0000 7fc509400640 1 -- 172.21.15.71:0/301956347 learned_addr learned my addr 172.21.15.71:0/301956347 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:56.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.656+0000 7fc509e00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc50415bf10 0x7fc50406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:56.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.657+0000 7fc509400640 1 -- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc50415bf10 msgr2=0x7fc50406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:56.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.657+0000 7fc509400640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc50415bf10 0x7fc50406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.657+0000 7fc509400640 1 -- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5040d08e0 msgr2=0x7fc50406dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:56.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.657+0000 7fc509400640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5040d08e0 0x7fc50406dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:56.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.657+0000 7fc509400640 1 -- 172.21.15.71:0/301956347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50406e7f0 con 0x7fc5040cd6f0 2024-09-16T07:32:56.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.657+0000 7fc509400640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040cd6f0 0x7fc5041f5350 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8034040 tx=0x7fc4f8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:56.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.658+0000 7fc501600640 1 -- 172.21.15.71:0/301956347 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f803e030 con 0x7fc5040cd6f0 2024-09-16T07:32:56.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.658+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50406ead0 con 0x7fc5040cd6f0 2024-09-16T07:32:56.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.658+0000 7fc501600640 1 -- 172.21.15.71:0/301956347 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4f8003e00 con 0x7fc5040cd6f0 2024-09-16T07:32:56.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.658+0000 7fc501600640 1 -- 172.21.15.71:0/301956347 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8035040 con 0x7fc5040cd6f0 2024-09-16T07:32:56.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.658+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50406f090 con 0x7fc5040cd6f0 2024-09-16T07:32:56.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.660+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc50415c660 con 0x7fc5040cd6f0 2024-09-16T07:32:56.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.661+0000 7fc501600640 1 -- 172.21.15.71:0/301956347 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7fc4f8041030 con 0x7fc5040cd6f0 2024-09-16T07:32:56.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.662+0000 7fc501600640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4e4079d20 0x7fc4e407c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:56.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.662+0000 7fc501600640 1 -- 172.21.15.71:0/301956347 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fc4f80b9fe0 con 0x7fc5040cd6f0 2024-09-16T07:32:56.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.668+0000 7fc501600640 1 -- 172.21.15.71:0/301956347 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc4f80836d0 con 0x7fc5040cd6f0 2024-09-16T07:32:56.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.691+0000 7fc508a00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4e4079d20 0x7fc4e407c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:56.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:56.702+0000 7fc508a00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4e4079d20 0x7fc4e407c1e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0000f60 tx=0x7fc4f00079b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:57.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:57.048+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone418816", "target": ["mon-mgr", ""]}) -- 0x7fc504069630 con 0x7fc4e4079d20 2024-09-16T07:32:58.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.513+0000 7fc501600640 1 -- 172.21.15.71:0/301956347 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc504069630 con 0x7fc4e4079d20 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.520+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4e4079d20 msgr2=0x7fc4e407c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.520+0000 7fc50be00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4e4079d20 0x7fc4e407c1e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0000f60 tx=0x7fc4f00079b0 comp rx=0 tx=0).stop 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040cd6f0 msgr2=0x7fc5041f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040cd6f0 0x7fc5041f5350 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8034040 tx=0x7fc4f8003910 comp rx=0 tx=0).stop 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 shutdown_connections 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4e4079d20 0x7fc4e407c1e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc50415bf10 0x7fc50406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5040d08e0 0x7fc50406dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 --2- 172.21.15.71:0/301956347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040cd6f0 0x7fc5041f5350 secure :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8034040 tx=0x7fc4f8003910 comp rx=0 tx=0).stop 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 >> 172.21.15.71:0/301956347 conn(0x7fc5040d1e20 msgr2=0x7fc5040b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:58.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 shutdown_connections 2024-09-16T07:32:58.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.521+0000 7fc50be00640 1 -- 172.21.15.71:0/301956347 wait complete. 2024-09-16T07:32:58.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99717 2024-09-16T07:32:58.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.825+0000 7f0358600640 1 -- 172.21.15.71:0/3489512336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0338004800 msgr2=0x7f0338004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:58.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.825+0000 7f0358600640 1 --2- 172.21.15.71:0/3489512336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0338004800 0x7f0338004c80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c6f0 tx=0x7f0338005830 comp rx=0 tx=0).stop 2024-09-16T07:32:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.826+0000 7f0358600640 1 -- 172.21.15.71:0/3489512336 shutdown_connections 2024-09-16T07:32:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.826+0000 7f0358600640 1 --2- 172.21.15.71:0/3489512336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0338004800 0x7f0338004c80 secure :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c6f0 tx=0x7f0338005830 comp rx=0 tx=0).stop 2024-09-16T07:32:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.826+0000 7f0358600640 1 -- 172.21.15.71:0/3489512336 >> 172.21.15.71:0/3489512336 conn(0x7f03500b1840 msgr2=0x7f03500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:32:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.826+0000 7f0358600640 1 -- 172.21.15.71:0/3489512336 shutdown_connections 2024-09-16T07:32:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.826+0000 7f0358600640 1 -- 172.21.15.71:0/3489512336 wait complete. 2024-09-16T07:32:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.829+0000 7f0358600640 1 Processor -- start 2024-09-16T07:32:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.829+0000 7f0358600640 1 -- start start 2024-09-16T07:32:58.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.831+0000 7f0358600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03500abd80 0x7f03500ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:58.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.831+0000 7f0358600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03500a9f40 0x7f03500ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:58.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.831+0000 7f0358600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03500ced80 0x7f03500cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:58.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.831+0000 7f0358600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f03500cc5c0 con 0x7f03500a9f40 2024-09-16T07:32:58.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.831+0000 7f0358600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03500736a0 con 0x7f03500ced80 2024-09-16T07:32:58.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.831+0000 7f0358600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f03500cc740 con 0x7f03500abd80 2024-09-16T07:32:58.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.832+0000 7f0355200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03500a9f40 0x7f03500ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:58.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.832+0000 7f0355200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03500a9f40 0x7f03500ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39292/0 (socket says 172.21.15.71:39292) 2024-09-16T07:32:58.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.832+0000 7f0355200640 1 -- 172.21.15.71:0/791022467 learned_addr learned my addr 172.21.15.71:0/791022467 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:32:58.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.832+0000 7f0355c00640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03500abd80 0x7f03500ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:58.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.833+0000 7f0355200640 1 -- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03500abd80 msgr2=0x7f03500ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:58.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.833+0000 7f0355200640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03500abd80 0x7f03500ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:58.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.833+0000 7f0355200640 1 -- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03500ced80 msgr2=0x7f03500cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:32:58.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.833+0000 7f0355200640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03500ced80 0x7f03500cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:32:58.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.833+0000 7f0355200640 1 -- 172.21.15.71:0/791022467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0350205e30 con 0x7f03500a9f40 2024-09-16T07:32:58.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.833+0000 7f0355200640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03500a9f40 0x7f03500ae740 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c610 tx=0x7f033c0060d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:58.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.834+0000 7f034d600640 1 -- 172.21.15.71:0/791022467 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c0064d0 con 0x7f03500a9f40 2024-09-16T07:32:58.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.834+0000 7f034d600640 1 -- 172.21.15.71:0/791022467 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f033c003610 con 0x7f03500a9f40 2024-09-16T07:32:58.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.834+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03500aa420 con 0x7f03500a9f40 2024-09-16T07:32:58.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.834+0000 7f034d600640 1 -- 172.21.15.71:0/791022467 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c003a40 con 0x7f03500a9f40 2024-09-16T07:32:58.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.834+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0350205350 con 0x7f03500a9f40 2024-09-16T07:32:58.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.836+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035015f9f0 con 0x7f03500a9f40 2024-09-16T07:32:58.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.837+0000 7f034d600640 1 -- 172.21.15.71:0/791022467 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f033c003be0 con 0x7f03500a9f40 2024-09-16T07:32:58.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.838+0000 7f034d600640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0330079ec0 0x7f033007c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:32:58.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.839+0000 7f034d600640 1 -- 172.21.15.71:0/791022467 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f033c0b94a0 con 0x7f03500a9f40 2024-09-16T07:32:58.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.844+0000 7f034d600640 1 -- 172.21.15.71:0/791022467 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f033c0834e0 con 0x7f03500a9f40 2024-09-16T07:32:58.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.881+0000 7f0355c00640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0330079ec0 0x7f033007c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:32:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:58.908+0000 7f0355c00640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0330079ec0 0x7f033007c380 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0344001860 tx=0x7f0344009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:32:59.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:32:59.296+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99717", "target": ["mon-mgr", ""]}) -- 0x7f03500ce580 con 0x7f0330079ec0 2024-09-16T07:33:00.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.157+0000 7f034d600640 1 -- 172.21.15.71:0/791022467 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f03500ce580 con 0x7f0330079ec0 2024-09-16T07:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0330079ec0 msgr2=0x7f033007c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0330079ec0 0x7f033007c380 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0344001860 tx=0x7f0344009040 comp rx=0 tx=0).stop 2024-09-16T07:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03500a9f40 msgr2=0x7f03500ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03500a9f40 0x7f03500ae740 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c610 tx=0x7f033c0060d0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 shutdown_connections 2024-09-16T07:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0330079ec0 0x7f033007c380 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03500ced80 0x7f03500cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03500a9f40 0x7f03500ae740 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 --2- 172.21.15.71:0/791022467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03500abd80 0x7f03500ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.163+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 >> 172.21.15.71:0/791022467 conn(0x7f03500b1840 msgr2=0x7f03500b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.164+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 shutdown_connections 2024-09-16T07:33:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.164+0000 7f0358600640 1 -- 172.21.15.71:0/791022467 wait complete. 2024-09-16T07:33:00.192 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone648918 2024-09-16T07:33:00.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.541+0000 7fe510c00640 1 -- 172.21.15.71:0/1254804265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c070870 msgr2=0x7fe50c1437c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:00.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.541+0000 7fe510c00640 1 --2- 172.21.15.71:0/1254804265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c070870 0x7fe50c1437c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0b28c0 tx=0x7fe4fc02e400 comp rx=0 tx=0).stop 2024-09-16T07:33:00.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.541+0000 7fe510c00640 1 -- 172.21.15.71:0/1254804265 shutdown_connections 2024-09-16T07:33:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.541+0000 7fe510c00640 1 --2- 172.21.15.71:0/1254804265 >> v2:172.21.15.177:3300/0 conn(0x7fe50c0b45d0 0x7fe50c06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.541+0000 7fe510c00640 1 --2- 172.21.15.71:0/1254804265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c070870 0x7fe50c1437c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.541+0000 7fe510c00640 1 -- 172.21.15.71:0/1254804265 >> 172.21.15.71:0/1254804265 conn(0x7fe50c0b1840 msgr2=0x7fe50c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.541+0000 7fe510c00640 1 -- 172.21.15.71:0/1254804265 shutdown_connections 2024-09-16T07:33:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.542+0000 7fe510c00640 1 -- 172.21.15.71:0/1254804265 wait complete. 2024-09-16T07:33:00.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.544+0000 7fe510c00640 1 Processor -- start 2024-09-16T07:33:00.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.544+0000 7fe510c00640 1 -- start start 2024-09-16T07:33:00.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.545+0000 7fe510c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe50c070870 0x7fe50c1f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:00.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.545+0000 7fe510c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c0b45d0 0x7fe50c13f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:00.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.545+0000 7fe510c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe50c0b4bd0 0x7fe50c13f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:00.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.545+0000 7fe510c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c06ef20 con 0x7fe50c0b45d0 2024-09-16T07:33:00.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.545+0000 7fe510c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c06eda0 con 0x7fe50c070870 2024-09-16T07:33:00.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.545+0000 7fe510c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe50c06f0a0 con 0x7fe50c0b4bd0 2024-09-16T07:33:00.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.546+0000 7fe503e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c0b45d0 0x7fe50c13f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:00.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.546+0000 7fe503e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c0b45d0 0x7fe50c13f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39308/0 (socket says 172.21.15.71:39308) 2024-09-16T07:33:00.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.546+0000 7fe503e00640 1 -- 172.21.15.71:0/791460134 learned_addr learned my addr 172.21.15.71:0/791460134 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:00.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.547+0000 7fe503e00640 1 -- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe50c0b4bd0 msgr2=0x7fe50c13f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.546+0000 7fe50a800640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe50c0b4bd0 0x7fe50c13f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.547+0000 7fe503e00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe50c0b4bd0 0x7fe50c13f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.547+0000 7fe503e00640 1 -- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe50c070870 msgr2=0x7fe50c1f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.547+0000 7fe503e00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe50c070870 0x7fe50c1f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.547+0000 7fe503e00640 1 -- 172.21.15.71:0/791460134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c13cbc0 con 0x7fe50c0b45d0 2024-09-16T07:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.547+0000 7fe50a800640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe50c0b4bd0 0x7fe50c13f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.548+0000 7fe503e00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c0b45d0 0x7fe50c13f3b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc010800 tx=0x7fe4fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.548+0000 7fe502000640 1 -- 172.21.15.71:0/791460134 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc003b40 con 0x7fe50c0b45d0 2024-09-16T07:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.548+0000 7fe502000640 1 -- 172.21.15.71:0/791460134 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4fc003dd0 con 0x7fe50c0b45d0 2024-09-16T07:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.548+0000 7fe502000640 1 -- 172.21.15.71:0/791460134 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc005280 con 0x7fe50c0b45d0 2024-09-16T07:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.548+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c13cea0 con 0x7fe50c0b45d0 2024-09-16T07:33:00.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.549+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c13d460 con 0x7fe50c0b45d0 2024-09-16T07:33:00.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.551+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c1437c0 con 0x7fe50c0b45d0 2024-09-16T07:33:00.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.553+0000 7fe502000640 1 -- 172.21.15.71:0/791460134 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7fe4fc03d040 con 0x7fe50c0b45d0 2024-09-16T07:33:00.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.554+0000 7fe502000640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe4e8079d20 0x7fe4e807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:00.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.554+0000 7fe502000640 1 -- 172.21.15.71:0/791460134 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fe4fc0bef80 con 0x7fe50c0b45d0 2024-09-16T07:33:00.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.561+0000 7fe502000640 1 -- 172.21.15.71:0/791460134 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe4fc044030 con 0x7fe50c0b45d0 2024-09-16T07:33:00.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.569+0000 7fe509e00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe4e8079d20 0x7fe4e807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:00.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.587+0000 7fe509e00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe4e8079d20 0x7fe4e807c1e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4006fd0 tx=0x7fe4f4006bc0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:00.991+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone648918", "target": ["mon-mgr", ""]}) -- 0x7fe50c13d8f0 con 0x7fe4e8079d20 2024-09-16T07:33:01.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.607+0000 7fe502000640 1 -- 172.21.15.71:0/791460134 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe50c13d8f0 con 0x7fe4e8079d20 2024-09-16T07:33:01.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.610+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe4e8079d20 msgr2=0x7fe4e807c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:01.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.610+0000 7fe510c00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe4e8079d20 0x7fe4e807c1e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4006fd0 tx=0x7fe4f4006bc0 comp rx=0 tx=0).stop 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.610+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c0b45d0 msgr2=0x7fe50c13f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.610+0000 7fe510c00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c0b45d0 0x7fe50c13f3b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc010800 tx=0x7fe4fc0045e0 comp rx=0 tx=0).stop 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.611+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 shutdown_connections 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.611+0000 7fe510c00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe4e8079d20 0x7fe4e807c1e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.611+0000 7fe510c00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe50c0b4bd0 0x7fe50c13f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.611+0000 7fe510c00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe50c0b45d0 0x7fe50c13f3b0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.611+0000 7fe510c00640 1 --2- 172.21.15.71:0/791460134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe50c070870 0x7fe50c1f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.611+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 >> 172.21.15.71:0/791460134 conn(0x7fe50c0b1840 msgr2=0x7fe50c15cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.611+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 shutdown_connections 2024-09-16T07:33:01.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.611+0000 7fe510c00640 1 -- 172.21.15.71:0/791460134 wait complete. 2024-09-16T07:33:01.641 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone849419 2024-09-16T07:33:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.877+0000 7fbdd7a00640 1 -- 172.21.15.71:0/1195856713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb80039e0 msgr2=0x7fbdb8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.877+0000 7fbdd7a00640 1 --2- 172.21.15.71:0/1195856713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb80039e0 0x7fbdb8003dc0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009b20 tx=0x7fbdb8005830 comp rx=0 tx=0).stop 2024-09-16T07:33:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.877+0000 7fbdd5a00640 1 -- 172.21.15.71:0/1195856713 reap_dead start 2024-09-16T07:33:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.877+0000 7fbdd7a00640 1 -- 172.21.15.71:0/1195856713 shutdown_connections 2024-09-16T07:33:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.877+0000 7fbdd7a00640 1 -- 172.21.15.71:0/1195856713 >> 172.21.15.71:0/1195856713 conn(0x7fbdd00b1840 msgr2=0x7fbdd00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.877+0000 7fbdd7a00640 1 -- 172.21.15.71:0/1195856713 shutdown_connections 2024-09-16T07:33:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.877+0000 7fbdd7a00640 1 -- 172.21.15.71:0/1195856713 wait complete. 2024-09-16T07:33:01.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.879+0000 7fbdd7a00640 1 Processor -- start 2024-09-16T07:33:01.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.879+0000 7fbdd7a00640 1 -- start start 2024-09-16T07:33:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.880+0000 7fbdd7a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdb80039e0 0x7fbdd015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.880+0000 7fbdd7a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd00cd1b0 0x7fbdd0157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.880+0000 7fbdd7a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01e9010 0x7fbdd00cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.880+0000 7fbdd7a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0070130 con 0x7fbdd01e9010 2024-09-16T07:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.880+0000 7fbdd7a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd006ffb0 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.880+0000 7fbdd7a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd00702b0 con 0x7fbdb80039e0 2024-09-16T07:33:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd00cd1b0 0x7fbdd0157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd00cd1b0 0x7fbdd0157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:52728/0 (socket says 172.21.15.71:52728) 2024-09-16T07:33:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdcfe00640 1 -- 172.21.15.71:0/841440901 learned_addr learned my addr 172.21.15.71:0/841440901 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdd5000640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdb80039e0 0x7fbdd015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdd5a00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01e9010 0x7fbdd00cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdcfe00640 1 -- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdb80039e0 msgr2=0x7fbdd015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdcfe00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdb80039e0 0x7fbdd015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdcfe00640 1 -- 172.21.15.71:0/841440901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01e9010 msgr2=0x7fbdd00cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:01.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdcfe00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01e9010 0x7fbdd00cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:01.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.881+0000 7fbdcfe00640 1 -- 172.21.15.71:0/841440901 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd015c430 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.884+0000 7fbdcfe00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd00cd1b0 0x7fbdd0157f20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c4d0 tx=0x7fbdc400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.888+0000 7fbdccc00640 1 -- 172.21.15.71:0/841440901 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4018070 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.888+0000 7fbdccc00640 1 -- 172.21.15.71:0/841440901 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdc4007030 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.888+0000 7fbdccc00640 1 -- 172.21.15.71:0/841440901 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4007480 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.888+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd015c710 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.888+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd015ccd0 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.889+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd015e750 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.892+0000 7fbdccc00640 1 -- 172.21.15.71:0/841440901 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7fbdc40067b0 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.894+0000 7fbdccc00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdb0079d20 0x7fbdb007c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.894+0000 7fbdccc00640 1 -- 172.21.15.71:0/841440901 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fbdc40990b0 con 0x7fbdd00cd1b0 2024-09-16T07:33:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.895+0000 7fbdd5000640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdb0079d20 0x7fbdb007c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:01.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.896+0000 7fbdd5000640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdb0079d20 0x7fbdb007c1e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009b20 tx=0x7fbdbc0046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:01.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:01.899+0000 7fbdccc00640 1 -- 172.21.15.71:0/841440901 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbdc4021020 con 0x7fbdd00cd1b0 2024-09-16T07:33:02.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.232+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone849419", "target": ["mon-mgr", ""]}) -- 0x7fbdd01f4da0 con 0x7fbdb0079d20 2024-09-16T07:33:02.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.419+0000 7fbdccc00640 1 -- 172.21.15.71:0/841440901 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbdd01f4da0 con 0x7fbdb0079d20 2024-09-16T07:33:02.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.421+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdb0079d20 msgr2=0x7fbdb007c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.421+0000 7fbdd7a00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdb0079d20 0x7fbdb007c1e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009b20 tx=0x7fbdbc0046c0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.421+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd00cd1b0 msgr2=0x7fbdd0157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.421+0000 7fbdd7a00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd00cd1b0 0x7fbdd0157f20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c4d0 tx=0x7fbdc400c990 comp rx=0 tx=0).stop 2024-09-16T07:33:02.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.422+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 shutdown_connections 2024-09-16T07:33:02.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.422+0000 7fbdd7a00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbdb0079d20 0x7fbdb007c1e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.422+0000 7fbdd7a00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01e9010 0x7fbdd00cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.422+0000 7fbdd7a00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd00cd1b0 0x7fbdd0157f20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.422+0000 7fbdd7a00640 1 --2- 172.21.15.71:0/841440901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdb80039e0 0x7fbdd015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.422+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 >> 172.21.15.71:0/841440901 conn(0x7fbdd00b1840 msgr2=0x7fbdd0073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:02.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.422+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 shutdown_connections 2024-09-16T07:33:02.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.422+0000 7fbdd7a00640 1 -- 172.21.15.71:0/841440901 wait complete. 2024-09-16T07:33:02.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:33:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.595+0000 7fbf03000640 1 -- 172.21.15.71:0/2060001102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc161550 msgr2=0x7fbefc06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.595+0000 7fbf03000640 1 --2- 172.21.15.71:0/2060001102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc161550 0x7fbefc06ef50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbeec005fd0 tx=0x7fbeec02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.595+0000 7fbf03000640 1 -- 172.21.15.71:0/2060001102 shutdown_connections 2024-09-16T07:33:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.595+0000 7fbf03000640 1 --2- 172.21.15.71:0/2060001102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc161550 0x7fbefc06ef50 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.595+0000 7fbf03000640 1 -- 172.21.15.71:0/2060001102 >> 172.21.15.71:0/2060001102 conn(0x7fbefc0b1950 msgr2=0x7fbefc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.595+0000 7fbf03000640 1 -- 172.21.15.71:0/2060001102 shutdown_connections 2024-09-16T07:33:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.595+0000 7fbf03000640 1 -- 172.21.15.71:0/2060001102 wait complete. 2024-09-16T07:33:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.597+0000 7fbf03000640 1 Processor -- start 2024-09-16T07:33:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.597+0000 7fbf03000640 1 -- start start 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbf03000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbefc072df0 0x7fbefc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbf03000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc0af3d0 0x7fbefc0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbf03000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbefc161550 0x7fbefc1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbf03000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc0739b0 con 0x7fbefc0af3d0 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbf03000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc06fdb0 con 0x7fbefc161550 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbf03000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc073b30 con 0x7fbefc072df0 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbefb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc0af3d0 0x7fbefc0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbefb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc0af3d0 0x7fbefc0cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39340/0 (socket says 172.21.15.71:39340) 2024-09-16T07:33:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbefb400640 1 -- 172.21.15.71:0/2382839816 learned_addr learned my addr 172.21.15.71:0/2382839816 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbefbe00640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbefc072df0 0x7fbefc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.598+0000 7fbf01000640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbefc161550 0x7fbefc1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbefb400640 1 -- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbefc072df0 msgr2=0x7fbefc0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbefb400640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbefc072df0 0x7fbefc0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbefb400640 1 -- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbefc161550 msgr2=0x7fbefc1566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbefb400640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbefc161550 0x7fbefc1566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbefb400640 1 -- 172.21.15.71:0/2382839816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc0b08f0 con 0x7fbefc0af3d0 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbefb400640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc0af3d0 0x7fbefc0cd000 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c420 tx=0x7fbee800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbef3e00640 1 -- 172.21.15.71:0/2382839816 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8017070 con 0x7fbefc0af3d0 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbef3e00640 1 -- 172.21.15.71:0/2382839816 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbee80053c0 con 0x7fbefc0af3d0 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbef3e00640 1 -- 172.21.15.71:0/2382839816 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8003850 con 0x7fbefc0af3d0 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc0afd70 con 0x7fbefc0af3d0 2024-09-16T07:33:02.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.599+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc0b0330 con 0x7fbefc0af3d0 2024-09-16T07:33:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.600+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc15f750 con 0x7fbefc0af3d0 2024-09-16T07:33:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.602+0000 7fbef3e00640 1 -- 172.21.15.71:0/2382839816 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7fbee801a070 con 0x7fbefc0af3d0 2024-09-16T07:33:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.602+0000 7fbef3e00640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbedc079d20 0x7fbedc07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.603+0000 7fbef3e00640 1 -- 172.21.15.71:0/2382839816 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fbee8098910 con 0x7fbefc0af3d0 2024-09-16T07:33:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.603+0000 7fbefbe00640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbedc079d20 0x7fbedc07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.604+0000 7fbefbe00640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbedc079d20 0x7fbedc07c1e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbeec033000 tx=0x7fbeec004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.606+0000 7fbef3e00640 1 -- 172.21.15.71:0/2382839816 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbefc15f750 con 0x7fbefc0af3d0 2024-09-16T07:33:02.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.812+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbefc074020 con 0x7fbedc079d20 2024-09-16T07:33:02.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.814+0000 7fbef3e00640 1 -- 172.21.15.71:0/2382839816 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbefc074020 con 0x7fbedc079d20 2024-09-16T07:33:02.813 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:33:02.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:02.813 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:33:02.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:02.814 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:33:02.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbedc079d20 msgr2=0x7fbedc07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbedc079d20 0x7fbedc07c1e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbeec033000 tx=0x7fbeec004730 comp rx=0 tx=0).stop 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc0af3d0 msgr2=0x7fbefc0cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc0af3d0 0x7fbefc0cd000 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c420 tx=0x7fbee800c8e0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 shutdown_connections 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbedc079d20 0x7fbedc07c1e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbefc161550 0x7fbefc1566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbefc0af3d0 0x7fbefc0cd000 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 --2- 172.21.15.71:0/2382839816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbefc072df0 0x7fbefc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 >> 172.21.15.71:0/2382839816 conn(0x7fbefc0b1950 msgr2=0x7fbefc15f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 shutdown_connections 2024-09-16T07:33:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.816+0000 7fbf03000640 1 -- 172.21.15.71:0/2382839816 wait complete. 2024-09-16T07:33:02.832 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:33:02.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.990+0000 7f823e400640 1 -- 172.21.15.71:0/3259494944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82380cccd0 msgr2=0x7f82380cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.990+0000 7f823e400640 1 --2- 172.21.15.71:0/3259494944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82380cccd0 0x7f82380cdbb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8230005fd0 tx=0x7f823002a4a0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.990+0000 7f823e400640 1 -- 172.21.15.71:0/3259494944 shutdown_connections 2024-09-16T07:33:02.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.990+0000 7f823e400640 1 --2- 172.21.15.71:0/3259494944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82380cccd0 0x7f82380cdbb0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.990+0000 7f823e400640 1 --2- 172.21.15.71:0/3259494944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8238145750 0x7f8238073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.990+0000 7f823e400640 1 -- 172.21.15.71:0/3259494944 >> 172.21.15.71:0/3259494944 conn(0x7f82380b1840 msgr2=0x7f82380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:02.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.990+0000 7f823e400640 1 -- 172.21.15.71:0/3259494944 shutdown_connections 2024-09-16T07:33:02.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.990+0000 7f823e400640 1 -- 172.21.15.71:0/3259494944 wait complete. 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.992+0000 7f823e400640 1 Processor -- start 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.992+0000 7f823e400640 1 -- start start 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f823e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f823806efa0 0x7f823813c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f823e400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82380cccd0 0x7f823815e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f823e400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8238145750 0x7f8238140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f823e400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f82380ce480 con 0x7f823806efa0 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f823e400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f82380ce300 con 0x7f8238145750 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f823e400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f82380ce600 con 0x7f82380cccd0 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f823806efa0 0x7f823813c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f823806efa0 0x7f823813c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39356/0 (socket says 172.21.15.71:39356) 2024-09-16T07:33:02.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237400640 1 -- 172.21.15.71:0/1746706530 learned_addr learned my addr 172.21.15.71:0/1746706530 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8236a00640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82380cccd0 0x7f823815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237e00640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8238145750 0x7f8238140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237400640 1 -- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82380cccd0 msgr2=0x7f823815e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82380cccd0 0x7f823815e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237400640 1 -- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8238145750 msgr2=0x7f8238140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8238145750 0x7f8238140790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237400640 1 -- 172.21.15.71:0/1746706530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8238140f30 con 0x7f823806efa0 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8236a00640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82380cccd0 0x7f823815e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.993+0000 7f8237e00640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8238145750 0x7f8238140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.994+0000 7f8237400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f823806efa0 0x7f823813c4d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f822800cce0 tx=0x7f822800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.994+0000 7f822f400640 1 -- 172.21.15.71:0/1746706530 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822800f450 con 0x7f823806efa0 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.994+0000 7f822f400640 1 -- 172.21.15.71:0/1746706530 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f822800f6e0 con 0x7f823806efa0 2024-09-16T07:33:02.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.994+0000 7f822f400640 1 -- 172.21.15.71:0/1746706530 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822800fb30 con 0x7f823806efa0 2024-09-16T07:33:02.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.994+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8238141210 con 0x7f823806efa0 2024-09-16T07:33:02.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.994+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381f8850 con 0x7f823806efa0 2024-09-16T07:33:02.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.995+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82380cdbb0 con 0x7f823806efa0 2024-09-16T07:33:02.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.996+0000 7f822f400640 1 -- 172.21.15.71:0/1746706530 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f822800fcd0 con 0x7f823806efa0 2024-09-16T07:33:02.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.997+0000 7f822f400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8218079d20 0x7f821807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:02.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.997+0000 7f822f400640 1 -- 172.21.15.71:0/1746706530 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f822809a1f0 con 0x7f823806efa0 2024-09-16T07:33:02.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.998+0000 7f8236a00640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8218079d20 0x7f821807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:02.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:02.998+0000 7f8236a00640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8218079d20 0x7f821807c1e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f82381f2a40 tx=0x7f8224009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:03.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.001+0000 7f822f400640 1 -- 172.21.15.71:0/1746706530 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8228014030 con 0x7f823806efa0 2024-09-16T07:33:03.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.211+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8238140cd0 con 0x7f8218079d20 2024-09-16T07:33:03.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.229+0000 7f822f400640 1 -- 172.21.15.71:0/1746706530 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1026 (secure 0 0 0) 0x7f8224003aa0 con 0x7f8218079d20 2024-09-16T07:33:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:03.229 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:33:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone614610" 2024-09-16T07:33:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone673111" 2024-09-16T07:33:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone69795" 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone219912" 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone594713" 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sv1" 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone30628" 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone73294" 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone99717" 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone43314" 2024-09-16T07:33:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone98650" 2024-09-16T07:33:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone644815" 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone849419" 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone71046" 2024-09-16T07:33:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone86499" 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone648918" 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone31792" 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone50521" 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone51223" 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone61947" 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone418816" 2024-09-16T07:33:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:03.236 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:33:03.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8218079d20 msgr2=0x7f821807c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8218079d20 0x7f821807c1e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f82381f2a40 tx=0x7f8224009040 comp rx=0 tx=0).stop 2024-09-16T07:33:03.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f823806efa0 msgr2=0x7f823813c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f823806efa0 0x7f823813c4d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f822800cce0 tx=0x7f822800a620 comp rx=0 tx=0).stop 2024-09-16T07:33:03.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 shutdown_connections 2024-09-16T07:33:03.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8218079d20 0x7f821807c1e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8238145750 0x7f8238140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82380cccd0 0x7f823815e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 --2- 172.21.15.71:0/1746706530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f823806efa0 0x7f823813c4d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 >> 172.21.15.71:0/1746706530 conn(0x7f82380b1840 msgr2=0x7f82380aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:03.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 shutdown_connections 2024-09-16T07:33:03.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.231+0000 7f823e400640 1 -- 172.21.15.71:0/1746706530 wait complete. 2024-09-16T07:33:03.248 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone614610 2024-09-16T07:33:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.414+0000 7f95e1800640 1 -- 172.21.15.71:0/2530265459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 msgr2=0x7f95c0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.414+0000 7f95e1800640 1 --2- 172.21.15.71:0/2530265459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 0x7f95c0004010 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f95cc006fb0 tx=0x7f95c0005830 comp rx=0 tx=0).stop 2024-09-16T07:33:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.414+0000 7f95e0e00640 1 -- 172.21.15.71:0/2530265459 reap_dead start 2024-09-16T07:33:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.414+0000 7f95e1800640 1 -- 172.21.15.71:0/2530265459 shutdown_connections 2024-09-16T07:33:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.414+0000 7f95e1800640 1 -- 172.21.15.71:0/2530265459 >> 172.21.15.71:0/2530265459 conn(0x7f95dc0d3f00 msgr2=0x7f95dc0d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.414+0000 7f95e1800640 1 -- 172.21.15.71:0/2530265459 shutdown_connections 2024-09-16T07:33:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.414+0000 7f95e1800640 1 -- 172.21.15.71:0/2530265459 wait complete. 2024-09-16T07:33:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.415+0000 7f95e1800640 1 Processor -- start 2024-09-16T07:33:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.415+0000 7f95e1800640 1 -- start start 2024-09-16T07:33:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.416+0000 7f95e1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 0x7f95dc0ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.416+0000 7f95e1800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95dc0c77b0 0x7f95dc1f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.416+0000 7f95e1800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95dc1f8310 0x7f95dc1f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.416+0000 7f95e1800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc166ea0 con 0x7f95c0003c10 2024-09-16T07:33:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.416+0000 7f95e1800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc0d4d70 con 0x7f95dc1f8310 2024-09-16T07:33:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.416+0000 7f95e1800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f95dc167020 con 0x7f95dc0c77b0 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 0x7f95dc0ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 0x7f95dc0ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39382/0 (socket says 172.21.15.71:39382) 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 -- 172.21.15.71:0/1735562204 learned_addr learned my addr 172.21.15.71:0/1735562204 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95d3e00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95dc0c77b0 0x7f95dc1f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95e0e00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95dc1f8310 0x7f95dc1f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 -- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95dc0c77b0 msgr2=0x7f95dc1f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95dc0c77b0 0x7f95dc1f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 -- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95dc1f8310 msgr2=0x7f95dc1f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95dc1f8310 0x7f95dc1f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 -- 172.21.15.71:0/1735562204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc0b1520 con 0x7f95c0003c10 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95d3e00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95dc0c77b0 0x7f95dc1f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95e0e00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95dc1f8310 0x7f95dc1f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95dbe00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 0x7f95dc0ca340 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f95cc02b0a0 tx=0x7f95cc006cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:03.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.417+0000 7f95d8c00640 1 -- 172.21.15.71:0/1735562204 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc032030 con 0x7f95c0003c10 2024-09-16T07:33:03.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.418+0000 7f95d8c00640 1 -- 172.21.15.71:0/1735562204 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95cc003af0 con 0x7f95c0003c10 2024-09-16T07:33:03.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.418+0000 7f95d8c00640 1 -- 172.21.15.71:0/1735562204 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc02e480 con 0x7f95c0003c10 2024-09-16T07:33:03.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.418+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95dc0b17c0 con 0x7f95c0003c10 2024-09-16T07:33:03.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.418+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95dc0b1d80 con 0x7f95c0003c10 2024-09-16T07:33:03.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.419+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95dc15eaf0 con 0x7f95c0003c10 2024-09-16T07:33:03.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.420+0000 7f95d8c00640 1 -- 172.21.15.71:0/1735562204 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f95cc035020 con 0x7f95c0003c10 2024-09-16T07:33:03.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.421+0000 7f95d8c00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95b8079d50 0x7f95b807c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:03.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.421+0000 7f95d8c00640 1 -- 172.21.15.71:0/1735562204 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f95cc0b9490 con 0x7f95c0003c10 2024-09-16T07:33:03.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.421+0000 7f95d3e00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95b8079d50 0x7f95b807c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:03.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.422+0000 7f95d3e00640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95b8079d50 0x7f95b807c210 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000f60 tx=0x7f95c4006090 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:03.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.425+0000 7f95d8c00640 1 -- 172.21.15.71:0/1735562204 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f95cc083550 con 0x7f95c0003c10 2024-09-16T07:33:03.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.636+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone614610", "target": ["mon-mgr", ""]}) -- 0x7f95dc0b9e80 con 0x7f95b8079d50 2024-09-16T07:33:03.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.658+0000 7f95d8c00640 1 -- 172.21.15.71:0/1735562204 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95dc0b9e80 con 0x7f95b8079d50 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95b8079d50 msgr2=0x7f95b807c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95b8079d50 0x7f95b807c210 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000f60 tx=0x7f95c4006090 comp rx=0 tx=0).stop 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 msgr2=0x7f95dc0ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 0x7f95dc0ca340 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f95cc02b0a0 tx=0x7f95cc006cc0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 shutdown_connections 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f95b8079d50 0x7f95b807c210 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95dc1f8310 0x7f95dc1f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95dc0c77b0 0x7f95dc1f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 --2- 172.21.15.71:0/1735562204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c0003c10 0x7f95dc0ca340 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 >> 172.21.15.71:0/1735562204 conn(0x7f95dc0d3f00 msgr2=0x7f95dc0d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:03.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 shutdown_connections 2024-09-16T07:33:03.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.660+0000 7f95e1800640 1 -- 172.21.15.71:0/1735562204 wait complete. 2024-09-16T07:33:03.662 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.662+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.662 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.663+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.662 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.663+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.662 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.663+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.662 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.663+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.677 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone673111 2024-09-16T07:33:03.684 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.683+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.685 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.683+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.685 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.683+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.685 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.683+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.685 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:03.683+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.840+0000 7f1d32800640 1 -- 172.21.15.71:0/718207199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c159aa0 msgr2=0x7f1d2c0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.840+0000 7f1d32800640 1 --2- 172.21.15.71:0/718207199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c0741a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c005fd0 tx=0x7f1d1c02a8d0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.840+0000 7f1d32800640 1 -- 172.21.15.71:0/718207199 shutdown_connections 2024-09-16T07:33:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.840+0000 7f1d32800640 1 --2- 172.21.15.71:0/718207199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c0741a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.840+0000 7f1d32800640 1 -- 172.21.15.71:0/718207199 >> 172.21.15.71:0/718207199 conn(0x7f1d2c0b18e0 msgr2=0x7f1d2c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.840+0000 7f1d32800640 1 -- 172.21.15.71:0/718207199 shutdown_connections 2024-09-16T07:33:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.840+0000 7f1d32800640 1 -- 172.21.15.71:0/718207199 wait complete. 2024-09-16T07:33:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.842+0000 7f1d32800640 1 Processor -- start 2024-09-16T07:33:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.842+0000 7f1d32800640 1 -- start start 2024-09-16T07:33:03.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d32800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f220 0x7f1d2c14b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:03.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d32800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:03.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d32800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c15cff0 0x7f1d2c154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:03.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d32800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c06f6b0 con 0x7f1d2c06f220 2024-09-16T07:33:03.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d32800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c0ccc50 con 0x7f1d2c159aa0 2024-09-16T07:33:03.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d32800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2c06f830 con 0x7f1d2c15cff0 2024-09-16T07:33:03.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f220 0x7f1d2c14b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:03.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f220 0x7f1d2c14b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39394/0 (socket says 172.21.15.71:39394) 2024-09-16T07:33:03.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 -- 172.21.15.71:0/435379659 learned_addr learned my addr 172.21.15.71:0/435379659 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 -- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c15cff0 msgr2=0x7f1d2c154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c15cff0 0x7f1d2c154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 -- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c159aa0 msgr2=0x7f1d2c154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 -- 172.21.15.71:0/435379659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c1f8250 con 0x7f1d2c06f220 2024-09-16T07:33:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.843+0000 7f1d2b400640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f220 0x7f1d2c14b550 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c000f80 tx=0x7f1d1c032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.844+0000 7f1d23400640 1 -- 172.21.15.71:0/435379659 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c037040 con 0x7f1d2c06f220 2024-09-16T07:33:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.844+0000 7f1d23400640 1 -- 172.21.15.71:0/435379659 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d1c003c60 con 0x7f1d2c06f220 2024-09-16T07:33:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.844+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c1f8530 con 0x7f1d2c06f220 2024-09-16T07:33:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.844+0000 7f1d23400640 1 -- 172.21.15.71:0/435379659 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c03c990 con 0x7f1d2c06f220 2024-09-16T07:33:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.844+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c1f8af0 con 0x7f1d2c06f220 2024-09-16T07:33:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.845+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c069360 con 0x7f1d2c06f220 2024-09-16T07:33:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.846+0000 7f1d23400640 1 -- 172.21.15.71:0/435379659 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f1d1c03a070 con 0x7f1d2c06f220 2024-09-16T07:33:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.847+0000 7f1d23400640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0c079d20 0x7f1d0c07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.847+0000 7f1d23400640 1 -- 172.21.15.71:0/435379659 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f1d1c0ba5f0 con 0x7f1d2c06f220 2024-09-16T07:33:03.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.847+0000 7f1d2aa00640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0c079d20 0x7f1d0c07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:03.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.848+0000 7f1d2aa00640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0c079d20 0x7f1d0c07c1e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1d18001530 tx=0x7f1d180079d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:03.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:03.851+0000 7f1d23400640 1 -- 172.21.15.71:0/435379659 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1d1c041020 con 0x7f1d2c06f220 2024-09-16T07:33:04.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.061+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone673111", "target": ["mon-mgr", ""]}) -- 0x7f1d2c070980 con 0x7f1d0c079d20 2024-09-16T07:33:04.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.082+0000 7f1d23400640 1 -- 172.21.15.71:0/435379659 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1d2c070980 con 0x7f1d0c079d20 2024-09-16T07:33:04.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0c079d20 msgr2=0x7f1d0c07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0c079d20 0x7f1d0c07c1e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1d18001530 tx=0x7f1d180079d0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f220 msgr2=0x7f1d2c14b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f220 0x7f1d2c14b550 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c000f80 tx=0x7f1d1c032f30 comp rx=0 tx=0).stop 2024-09-16T07:33:04.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 shutdown_connections 2024-09-16T07:33:04.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0c079d20 0x7f1d0c07c1e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c15cff0 0x7f1d2c154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c159aa0 0x7f1d2c154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 --2- 172.21.15.71:0/435379659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f220 0x7f1d2c14b550 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.084+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 >> 172.21.15.71:0/435379659 conn(0x7f1d2c0b18e0 msgr2=0x7f1d2c15b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:04.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.085+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 shutdown_connections 2024-09-16T07:33:04.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.085+0000 7f1d32800640 1 -- 172.21.15.71:0/435379659 wait complete. 2024-09-16T07:33:04.101 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone69795 2024-09-16T07:33:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.263+0000 7ffb2be00640 1 -- 172.21.15.71:0/4006214566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c070550 msgr2=0x7ffb2c0cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.263+0000 7ffb2be00640 1 --2- 172.21.15.71:0/4006214566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c070550 0x7ffb2c0cb1c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009830 tx=0x7ffb2402a510 comp rx=0 tx=0).stop 2024-09-16T07:33:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.263+0000 7ffb2be00640 1 -- 172.21.15.71:0/4006214566 shutdown_connections 2024-09-16T07:33:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.263+0000 7ffb2be00640 1 --2- 172.21.15.71:0/4006214566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c070550 0x7ffb2c0cb1c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.263+0000 7ffb2be00640 1 --2- 172.21.15.71:0/4006214566 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c15bda0 0x7ffb2c073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.263+0000 7ffb2be00640 1 -- 172.21.15.71:0/4006214566 >> 172.21.15.71:0/4006214566 conn(0x7ffb2c0cfa70 msgr2=0x7ffb2c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:04.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.263+0000 7ffb2be00640 1 -- 172.21.15.71:0/4006214566 shutdown_connections 2024-09-16T07:33:04.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.263+0000 7ffb2be00640 1 -- 172.21.15.71:0/4006214566 wait complete. 2024-09-16T07:33:04.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.264+0000 7ffb2be00640 1 Processor -- start 2024-09-16T07:33:04.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.265+0000 7ffb2be00640 1 -- start start 2024-09-16T07:33:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.265+0000 7ffb2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c06e590 0x7ffb2c073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c070550 0x7ffb2c14f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c15bda0 0x7ffb2c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c0cba90 con 0x7ffb2c06e590 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c0cb910 con 0x7ffb2c15bda0 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2c0cbc10 con 0x7ffb2c070550 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c06e590 0x7ffb2c073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c06e590 0x7ffb2c073c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39408/0 (socket says 172.21.15.71:39408) 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 -- 172.21.15.71:0/3512484287 learned_addr learned my addr 172.21.15.71:0/3512484287 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb22000640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c070550 0x7ffb2c14f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2b400640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c15bda0 0x7ffb2c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 -- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c070550 msgr2=0x7ffb2c14f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c070550 0x7ffb2c14f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 -- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c15bda0 msgr2=0x7ffb2c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c15bda0 0x7ffb2c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 -- 172.21.15.71:0/3512484287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c15c6a0 con 0x7ffb2c06e590 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2b400640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c15bda0 0x7ffb2c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.266+0000 7ffb2aa00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c06e590 0x7ffb2c073c70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00dac0 tx=0x7ffb1c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.267+0000 7ffb23400640 1 -- 172.21.15.71:0/3512484287 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c017070 con 0x7ffb2c06e590 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.267+0000 7ffb23400640 1 -- 172.21.15.71:0/3512484287 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb1c003970 con 0x7ffb2c06e590 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.267+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2c15c980 con 0x7ffb2c06e590 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.267+0000 7ffb23400640 1 -- 172.21.15.71:0/3512484287 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c003dc0 con 0x7ffb2c06e590 2024-09-16T07:33:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.267+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2c15cf40 con 0x7ffb2c06e590 2024-09-16T07:33:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.268+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2c0cc1a0 con 0x7ffb2c06e590 2024-09-16T07:33:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.269+0000 7ffb23400640 1 -- 172.21.15.71:0/3512484287 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7ffb1c004020 con 0x7ffb2c06e590 2024-09-16T07:33:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.270+0000 7ffb23400640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffb08079d50 0x7ffb0807c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.270+0000 7ffb23400640 1 -- 172.21.15.71:0/3512484287 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7ffb1c09a010 con 0x7ffb2c06e590 2024-09-16T07:33:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.271+0000 7ffb22000640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffb08079d50 0x7ffb0807c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.271+0000 7ffb22000640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffb08079d50 0x7ffb0807c210 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c075200 tx=0x7ffb14009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:04.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.274+0000 7ffb23400640 1 -- 172.21.15.71:0/3512484287 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffb1c020020 con 0x7ffb2c06e590 2024-09-16T07:33:04.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.490+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone69795", "target": ["mon-mgr", ""]}) -- 0x7ffb2c14f560 con 0x7ffb08079d50 2024-09-16T07:33:04.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.513+0000 7ffb23400640 1 -- 172.21.15.71:0/3512484287 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffb14001c60 con 0x7ffb08079d50 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffb08079d50 msgr2=0x7ffb0807c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffb08079d50 0x7ffb0807c210 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c075200 tx=0x7ffb14009040 comp rx=0 tx=0).stop 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c06e590 msgr2=0x7ffb2c073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c06e590 0x7ffb2c073c70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00dac0 tx=0x7ffb1c00df80 comp rx=0 tx=0).stop 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 shutdown_connections 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffb08079d50 0x7ffb0807c210 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c15bda0 0x7ffb2c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c070550 0x7ffb2c14f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 --2- 172.21.15.71:0/3512484287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb2c06e590 0x7ffb2c073c70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 >> 172.21.15.71:0/3512484287 conn(0x7ffb2c0cfa70 msgr2=0x7ffb2c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 shutdown_connections 2024-09-16T07:33:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.516+0000 7ffb2be00640 1 -- 172.21.15.71:0/3512484287 wait complete. 2024-09-16T07:33:04.531 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone219912 2024-09-16T07:33:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.693+0000 7f646cc00640 1 -- 172.21.15.71:0/3661286945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6468159aa0 msgr2=0x7f64680741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.693+0000 7f646cc00640 1 --2- 172.21.15.71:0/3661286945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6468159aa0 0x7f64680741a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6458009ca0 tx=0x7f645802aa30 comp rx=0 tx=0).stop 2024-09-16T07:33:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.693+0000 7f646cc00640 1 -- 172.21.15.71:0/3661286945 shutdown_connections 2024-09-16T07:33:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.693+0000 7f646cc00640 1 --2- 172.21.15.71:0/3661286945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f646815cff0 0x7f646806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.693+0000 7f646cc00640 1 --2- 172.21.15.71:0/3661286945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6468159aa0 0x7f64680741a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.693+0000 7f646cc00640 1 -- 172.21.15.71:0/3661286945 >> 172.21.15.71:0/3661286945 conn(0x7f64680b18e0 msgr2=0x7f64680b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.693+0000 7f646cc00640 1 -- 172.21.15.71:0/3661286945 shutdown_connections 2024-09-16T07:33:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.693+0000 7f646cc00640 1 -- 172.21.15.71:0/3661286945 wait complete. 2024-09-16T07:33:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.695+0000 7f646cc00640 1 Processor -- start 2024-09-16T07:33:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.695+0000 7f646cc00640 1 -- start start 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f646cc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f646806f220 0x7f6468157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f646cc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6468159aa0 0x7f64681546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f646cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646815cff0 0x7f64680d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f646cc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f646806fa60 con 0x7f646815cff0 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f646cc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f646806f8e0 con 0x7f646806f220 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f646cc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64680cc8a0 con 0x7f6468159aa0 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f6466800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646815cff0 0x7f64680d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f6466800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646815cff0 0x7f64680d01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39418/0 (socket says 172.21.15.71:39418) 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f6466800640 1 -- 172.21.15.71:0/936209765 learned_addr learned my addr 172.21.15.71:0/936209765 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f6465e00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f646806f220 0x7f6468157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.696+0000 7f6465400640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6468159aa0 0x7f64681546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f6466800640 1 -- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6468159aa0 msgr2=0x7f64681546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f6466800640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6468159aa0 0x7f64681546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f6466800640 1 -- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f646806f220 msgr2=0x7f6468157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f6466800640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f646806f220 0x7f6468157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f6466800640 1 -- 172.21.15.71:0/936209765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6468157640 con 0x7f646815cff0 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f6466800640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646815cff0 0x7f64680d01d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f646000c390 tx=0x7f646000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f645e000640 1 -- 172.21.15.71:0/936209765 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6460017070 con 0x7f646815cff0 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f645e000640 1 -- 172.21.15.71:0/936209765 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64600035f0 con 0x7f646815cff0 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f645e000640 1 -- 172.21.15.71:0/936209765 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6460003a40 con 0x7f646815cff0 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64681f8910 con 0x7f646815cff0 2024-09-16T07:33:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.697+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64681f8ed0 con 0x7f646815cff0 2024-09-16T07:33:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.698+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6468069360 con 0x7f646815cff0 2024-09-16T07:33:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.700+0000 7f645e000640 1 -- 172.21.15.71:0/936209765 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f646001a070 con 0x7f646815cff0 2024-09-16T07:33:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.701+0000 7f645e000640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6448079d20 0x7f644807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.701+0000 7f645e000640 1 -- 172.21.15.71:0/936209765 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f64600988e0 con 0x7f646815cff0 2024-09-16T07:33:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.702+0000 7f6465e00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6448079d20 0x7f644807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.703+0000 7f6465e00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6448079d20 0x7f644807c1e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6458035000 tx=0x7f645803a040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.706+0000 7f645e000640 1 -- 172.21.15.71:0/936209765 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6460062350 con 0x7f646815cff0 2024-09-16T07:33:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.916+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone219912", "target": ["mon-mgr", ""]}) -- 0x7f6468070980 con 0x7f6448079d20 2024-09-16T07:33:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.938+0000 7f645e000640 1 -- 172.21.15.71:0/936209765 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6468070980 con 0x7f6448079d20 2024-09-16T07:33:04.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6448079d20 msgr2=0x7f644807c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6448079d20 0x7f644807c1e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6458035000 tx=0x7f645803a040 comp rx=0 tx=0).stop 2024-09-16T07:33:04.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646815cff0 msgr2=0x7f64680d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:04.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646815cff0 0x7f64680d01d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f646000c390 tx=0x7f646000c850 comp rx=0 tx=0).stop 2024-09-16T07:33:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 shutdown_connections 2024-09-16T07:33:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6448079d20 0x7f644807c1e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646815cff0 0x7f64680d01d0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6468159aa0 0x7f64681546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.941+0000 7f646cc00640 1 --2- 172.21.15.71:0/936209765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f646806f220 0x7f6468157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.942+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 >> 172.21.15.71:0/936209765 conn(0x7f64680b18e0 msgr2=0x7f646815b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.942+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 shutdown_connections 2024-09-16T07:33:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:04.942+0000 7f646cc00640 1 -- 172.21.15.71:0/936209765 wait complete. 2024-09-16T07:33:04.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone594713 2024-09-16T07:33:05.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.119+0000 7f250f800640 1 -- 172.21.15.71:0/1481682072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f250815f9a0 msgr2=0x7f250815fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.119+0000 7f250f800640 1 --2- 172.21.15.71:0/1481682072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f250815f9a0 0x7f250815fd80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005fd0 tx=0x7f24f802a540 comp rx=0 tx=0).stop 2024-09-16T07:33:05.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.119+0000 7f250f800640 1 -- 172.21.15.71:0/1481682072 shutdown_connections 2024-09-16T07:33:05.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.119+0000 7f250f800640 1 --2- 172.21.15.71:0/1481682072 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2508073de0 0x7f2508074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.119+0000 7f250f800640 1 --2- 172.21.15.71:0/1481682072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f250815f9a0 0x7f250815fd80 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.120+0000 7f250f800640 1 -- 172.21.15.71:0/1481682072 >> 172.21.15.71:0/1481682072 conn(0x7f25080b18e0 msgr2=0x7f25080b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:05.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.120+0000 7f250f800640 1 -- 172.21.15.71:0/1481682072 shutdown_connections 2024-09-16T07:33:05.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.120+0000 7f250f800640 1 -- 172.21.15.71:0/1481682072 wait complete. 2024-09-16T07:33:05.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.121+0000 7f250f800640 1 Processor -- start 2024-09-16T07:33:05.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.121+0000 7f250f800640 1 -- start start 2024-09-16T07:33:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250f800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2508073de0 0x7f25081f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250f800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250815c3f0 0x7f25080b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250f800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250815f9a0 0x7f25080b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250f800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f25080cd3d0 con 0x7f2508073de0 2024-09-16T07:33:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250f800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f25080ccbd0 con 0x7f250815c3f0 2024-09-16T07:33:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250f800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f25080cd550 con 0x7f250815f9a0 2024-09-16T07:33:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2508073de0 0x7f25081f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2508073de0 0x7f25081f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34462/0 (socket says 172.21.15.71:34462) 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.122+0000 7f250ce00640 1 -- 172.21.15.71:0/3443955017 learned_addr learned my addr 172.21.15.71:0/3443955017 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f250d800640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250815f9a0 0x7f25080b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f2507e00640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250815c3f0 0x7f25080b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f250ce00640 1 -- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250815f9a0 msgr2=0x7f25080b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f250ce00640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250815f9a0 0x7f25080b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f250ce00640 1 -- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250815c3f0 msgr2=0x7f25080b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f250ce00640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250815c3f0 0x7f25080b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f250ce00640 1 -- 172.21.15.71:0/3443955017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25081f8230 con 0x7f2508073de0 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f250ce00640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2508073de0 0x7f25081f89f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f24f8034040 tx=0x7f24f80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f2504c00640 1 -- 172.21.15.71:0/3443955017 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f803e030 con 0x7f2508073de0 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f2504c00640 1 -- 172.21.15.71:0/3443955017 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24f8003d90 con 0x7f2508073de0 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f2504c00640 1 -- 172.21.15.71:0/3443955017 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8035040 con 0x7f2508073de0 2024-09-16T07:33:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.123+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25080ae590 con 0x7f2508073de0 2024-09-16T07:33:05.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.124+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25081f84d0 con 0x7f2508073de0 2024-09-16T07:33:05.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.125+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2508069360 con 0x7f2508073de0 2024-09-16T07:33:05.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.126+0000 7f2504c00640 1 -- 172.21.15.71:0/3443955017 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 36) ==== 103174+0+0 (secure 0 0 0) 0x7f24f8041030 con 0x7f2508073de0 2024-09-16T07:33:05.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.127+0000 7f2504c00640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24e8079d20 0x7f24e807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.127+0000 7f2504c00640 1 -- 172.21.15.71:0/3443955017 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f24f80ba460 con 0x7f2508073de0 2024-09-16T07:33:05.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.128+0000 7f2507e00640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24e8079d20 0x7f24e807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.129+0000 7f2507e00640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24e8079d20 0x7f24e807c1e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f24f4005fd0 tx=0x7f24f4005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:05.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.132+0000 7f2504c00640 1 -- 172.21.15.71:0/3443955017 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f24f8084140 con 0x7f2508073de0 2024-09-16T07:33:05.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.340+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone594713", "target": ["mon-mgr", ""]}) -- 0x7f250815d1e0 con 0x7f24e8079d20 2024-09-16T07:33:05.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.360+0000 7f2504c00640 1 -- 172.21.15.71:0/3443955017 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f250815d1e0 con 0x7f24e8079d20 2024-09-16T07:33:05.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24e8079d20 msgr2=0x7f24e807c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24e8079d20 0x7f24e807c1e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f24f4005fd0 tx=0x7f24f4005850 comp rx=0 tx=0).stop 2024-09-16T07:33:05.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2508073de0 msgr2=0x7f25081f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2508073de0 0x7f25081f89f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f24f8034040 tx=0x7f24f80038a0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 shutdown_connections 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24e8079d20 0x7f24e807c1e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250815f9a0 0x7f25080b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250815c3f0 0x7f25080b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 --2- 172.21.15.71:0/3443955017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2508073de0 0x7f25081f89f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 >> 172.21.15.71:0/3443955017 conn(0x7f25080b18e0 msgr2=0x7f250815e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 shutdown_connections 2024-09-16T07:33:05.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.362+0000 7f250f800640 1 -- 172.21.15.71:0/3443955017 wait complete. 2024-09-16T07:33:05.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T07:33:05.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.541+0000 7fd199600640 1 -- 172.21.15.71:0/3281500178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 msgr2=0x7fd19406ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.541+0000 7fd199600640 1 --2- 172.21.15.71:0/3281500178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 0x7fd19406ff30 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd1940c6af0 tx=0x7fd18c02a8b0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.542+0000 7fd199600640 1 -- 172.21.15.71:0/3281500178 shutdown_connections 2024-09-16T07:33:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.542+0000 7fd199600640 1 --2- 172.21.15.71:0/3281500178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 0x7fd19406ff30 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.542+0000 7fd199600640 1 --2- 172.21.15.71:0/3281500178 >> v2:172.21.15.177:3300/0 conn(0x7fd19415abd0 0x7fd194072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.542+0000 7fd199600640 1 -- 172.21.15.71:0/3281500178 >> 172.21.15.71:0/3281500178 conn(0x7fd1940cfa70 msgr2=0x7fd1940cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.542+0000 7fd199600640 1 -- 172.21.15.71:0/3281500178 shutdown_connections 2024-09-16T07:33:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.542+0000 7fd199600640 1 -- 172.21.15.71:0/3281500178 wait complete. 2024-09-16T07:33:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.543+0000 7fd199600640 1 Processor -- start 2024-09-16T07:33:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.544+0000 7fd199600640 1 -- start start 2024-09-16T07:33:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.544+0000 7fd199600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 0x7fd1941532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.544+0000 7fd199600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd19415abd0 0x7fd1941f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.544+0000 7fd199600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1941f7c20 0x7fd19415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.544+0000 7fd199600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1940cd7c0 con 0x7fd194073cf0 2024-09-16T07:33:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.544+0000 7fd199600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1940cd640 con 0x7fd1941f7c20 2024-09-16T07:33:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.544+0000 7fd199600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1940cd940 con 0x7fd19415abd0 2024-09-16T07:33:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 0x7fd1941532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 0x7fd1941532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34466/0 (socket says 172.21.15.71:34466) 2024-09-16T07:33:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 -- 172.21.15.71:0/2557789887 learned_addr learned my addr 172.21.15.71:0/2557789887 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 -- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd19415abd0 msgr2=0x7fd1941f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd19415abd0 0x7fd1941f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd198c00640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1941f7c20 0x7fd19415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 -- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1941f7c20 msgr2=0x7fd19415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1941f7c20 0x7fd19415cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 -- 172.21.15.71:0/2557789887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19415d8a0 con 0x7fd194073cf0 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd198c00640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1941f7c20 0x7fd19415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd193e00640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 0x7fd1941532e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d770 tx=0x7fd18400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.545+0000 7fd18be00640 1 -- 172.21.15.71:0/2557789887 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184017070 con 0x7fd194073cf0 2024-09-16T07:33:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.546+0000 7fd18be00640 1 -- 172.21.15.71:0/2557789887 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd184009d20 con 0x7fd194073cf0 2024-09-16T07:33:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.546+0000 7fd18be00640 1 -- 172.21.15.71:0/2557789887 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1840106c0 con 0x7fd194073cf0 2024-09-16T07:33:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.546+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1940c4870 con 0x7fd194073cf0 2024-09-16T07:33:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.546+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19415d0f0 con 0x7fd194073cf0 2024-09-16T07:33:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.547+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1941f8030 con 0x7fd194073cf0 2024-09-16T07:33:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.548+0000 7fd18be00640 1 -- 172.21.15.71:0/2557789887 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fd18401a070 con 0x7fd194073cf0 2024-09-16T07:33:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.549+0000 7fd18be00640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd174079e70 0x7fd17407c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.549+0000 7fd18be00640 1 -- 172.21.15.71:0/2557789887 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fd184099b70 con 0x7fd194073cf0 2024-09-16T07:33:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.549+0000 7fd193400640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd174079e70 0x7fd17407c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.550+0000 7fd193400640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd174079e70 0x7fd17407c330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd180000f60 tx=0x7fd1800078d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.553+0000 7fd18be00640 1 -- 172.21.15.71:0/2557789887 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd1941f8030 con 0x7fd194073cf0 2024-09-16T07:33:05.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.768+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd19415db80 con 0x7fd174079e70 2024-09-16T07:33:05.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.785+0000 7fd18be00640 1 -- 172.21.15.71:0/2557789887 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd19415db80 con 0x7fd174079e70 2024-09-16T07:33:05.784 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:05.785 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:33:05.785 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:05.785 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1" 2024-09-16T07:33:05.785 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:05.785 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:33:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd174079e70 msgr2=0x7fd17407c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd174079e70 0x7fd17407c330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd180000f60 tx=0x7fd1800078d0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 msgr2=0x7fd1941532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 0x7fd1941532e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d770 tx=0x7fd18400dc30 comp rx=0 tx=0).stop 2024-09-16T07:33:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 shutdown_connections 2024-09-16T07:33:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd174079e70 0x7fd17407c330 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1941f7c20 0x7fd19415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd19415abd0 0x7fd1941f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 --2- 172.21.15.71:0/2557789887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd194073cf0 0x7fd1941532e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 >> 172.21.15.71:0/2557789887 conn(0x7fd1940cfa70 msgr2=0x7fd1940af490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 shutdown_connections 2024-09-16T07:33:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.787+0000 7fd199600640 1 -- 172.21.15.71:0/2557789887 wait complete. 2024-09-16T07:33:05.804 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T07:33:05.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.968+0000 7fa7dd400640 1 -- 172.21.15.71:0/3339304932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7d806efa0 msgr2=0x7fa7d80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.968+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3339304932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7d806efa0 0x7fa7d80cdbb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0005fd0 tx=0x7fa7d002a820 comp rx=0 tx=0).stop 2024-09-16T07:33:05.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.969+0000 7fa7dd400640 1 -- 172.21.15.71:0/3339304932 shutdown_connections 2024-09-16T07:33:05.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.969+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3339304932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7d806efa0 0x7fa7d80cdbb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.969+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3339304932 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d80cccd0 0x7fa7d8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.969+0000 7fa7dd400640 1 -- 172.21.15.71:0/3339304932 >> 172.21.15.71:0/3339304932 conn(0x7fa7d80b1840 msgr2=0x7fa7d80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:05.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.969+0000 7fa7dd400640 1 -- 172.21.15.71:0/3339304932 shutdown_connections 2024-09-16T07:33:05.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.969+0000 7fa7dd400640 1 -- 172.21.15.71:0/3339304932 wait complete. 2024-09-16T07:33:05.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.970+0000 7fa7dd400640 1 Processor -- start 2024-09-16T07:33:05.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.970+0000 7fa7dd400640 1 -- start start 2024-09-16T07:33:05.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.971+0000 7fa7dd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d806efa0 0x7fa7d81f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.971+0000 7fa7dd400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d80cccd0 0x7fa7d8141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.971+0000 7fa7dd400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7d8145750 0x7fa7d80c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.971+0000 7fa7dd400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d80ce480 con 0x7fa7d806efa0 2024-09-16T07:33:05.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.971+0000 7fa7dd400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d80ce300 con 0x7fa7d8145750 2024-09-16T07:33:05.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.971+0000 7fa7dd400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d80ce600 con 0x7fa7d80cccd0 2024-09-16T07:33:05.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d806efa0 0x7fa7d81f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d806efa0 0x7fa7d81f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34468/0 (socket says 172.21.15.71:34468) 2024-09-16T07:33:05.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d6800640 1 -- 172.21.15.71:0/3843830261 learned_addr learned my addr 172.21.15.71:0/3843830261 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:05.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7cfe00640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d80cccd0 0x7fa7d8141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d7200640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7d8145750 0x7fa7d80c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d6800640 1 -- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d80cccd0 msgr2=0x7fa7d8141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d6800640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d80cccd0 0x7fa7d8141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d6800640 1 -- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7d8145750 msgr2=0x7fa7d80c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:05.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d6800640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7d8145750 0x7fa7d80c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:05.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.972+0000 7fa7d6800640 1 -- 172.21.15.71:0/3843830261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d81427a0 con 0x7fa7d806efa0 2024-09-16T07:33:05.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.973+0000 7fa7d6800640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d806efa0 0x7fa7d81f7880 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c430 tx=0x7fa7c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:05.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.973+0000 7fa7cea00640 1 -- 172.21.15.71:0/3843830261 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8011790 con 0x7fa7d806efa0 2024-09-16T07:33:05.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.973+0000 7fa7cea00640 1 -- 172.21.15.71:0/3843830261 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7c8017070 con 0x7fa7d806efa0 2024-09-16T07:33:05.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.973+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d8155430 con 0x7fa7d806efa0 2024-09-16T07:33:05.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.973+0000 7fa7cea00640 1 -- 172.21.15.71:0/3843830261 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8011ce0 con 0x7fa7d806efa0 2024-09-16T07:33:05.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.973+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81559f0 con 0x7fa7d806efa0 2024-09-16T07:33:05.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.974+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d80cdbb0 con 0x7fa7d806efa0 2024-09-16T07:33:05.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.975+0000 7fa7cea00640 1 -- 172.21.15.71:0/3843830261 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fa7c801c460 con 0x7fa7d806efa0 2024-09-16T07:33:05.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.977+0000 7fa7cea00640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b4079e40 0x7fa7b407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:05.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.977+0000 7fa7cea00640 1 -- 172.21.15.71:0/3843830261 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fa7c8099720 con 0x7fa7d806efa0 2024-09-16T07:33:05.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.977+0000 7fa7cfe00640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b4079e40 0x7fa7b407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:05.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.978+0000 7fa7cfe00640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b4079e40 0x7fa7b407c300 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00095f0 tx=0x7fa7c0005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:05.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:05.981+0000 7fa7cea00640 1 -- 172.21.15.71:0/3843830261 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa7c8099bc0 con 0x7fa7d806efa0 2024-09-16T07:33:06.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.199+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7d815ef80 con 0x7fa7b4079e40 2024-09-16T07:33:06.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.228+0000 7fa7cea00640 1 -- 172.21.15.71:0/3843830261 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa7d815ef80 con 0x7fa7b4079e40 2024-09-16T07:33:06.229 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b4079e40 msgr2=0x7fa7b407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b4079e40 0x7fa7b407c300 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00095f0 tx=0x7fa7c0005f90 comp rx=0 tx=0).stop 2024-09-16T07:33:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d806efa0 msgr2=0x7fa7d81f7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d806efa0 0x7fa7d81f7880 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c430 tx=0x7fa7c800c8f0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 shutdown_connections 2024-09-16T07:33:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7b4079e40 0x7fa7b407c300 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7d8145750 0x7fa7d80c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7d80cccd0 0x7fa7d8141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 --2- 172.21.15.71:0/3843830261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7d806efa0 0x7fa7d81f7880 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.231+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 >> 172.21.15.71:0/3843830261 conn(0x7fa7d80b1840 msgr2=0x7fa7d80af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.232+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 shutdown_connections 2024-09-16T07:33:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.232+0000 7fa7dd400640 1 -- 172.21.15.71:0/3843830261 wait complete. 2024-09-16T07:33:06.248 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T07:33:06.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.425+0000 7f49c1800640 1 -- 172.21.15.71:0/1438586507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49bc0cf2f0 msgr2=0x7f49bc15e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:06.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.425+0000 7f49c1800640 1 --2- 172.21.15.71:0/1438586507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49bc0cf2f0 0x7f49bc15e980 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009860 tx=0x7f49a802a5a0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.426+0000 7f49c1800640 1 -- 172.21.15.71:0/1438586507 shutdown_connections 2024-09-16T07:33:06.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.426+0000 7f49c1800640 1 --2- 172.21.15.71:0/1438586507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49bc0cf2f0 0x7f49bc15e980 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.426+0000 7f49c1800640 1 -- 172.21.15.71:0/1438586507 >> 172.21.15.71:0/1438586507 conn(0x7f49bc0b1840 msgr2=0x7f49bc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:06.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.426+0000 7f49c1800640 1 -- 172.21.15.71:0/1438586507 shutdown_connections 2024-09-16T07:33:06.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.426+0000 7f49c1800640 1 -- 172.21.15.71:0/1438586507 wait complete. 2024-09-16T07:33:06.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.428+0000 7f49c1800640 1 Processor -- start 2024-09-16T07:33:06.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.429+0000 7f49c1800640 1 -- start start 2024-09-16T07:33:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.430+0000 7f49c1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49bc0b3370 0x7f49bc152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.430+0000 7f49c1800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49bc0cf2f0 0x7f49bc152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.431+0000 7f49c1800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc15c080 0x7f49bc152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.431+0000 7f49c1800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc0ce0b0 con 0x7f49bc0b3370 2024-09-16T07:33:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.431+0000 7f49c1800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc0cdf30 con 0x7f49bc15c080 2024-09-16T07:33:06.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.431+0000 7f49c1800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49bc0ce230 con 0x7f49bc0cf2f0 2024-09-16T07:33:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.431+0000 7f49baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49bc0b3370 0x7f49bc152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.431+0000 7f49baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49bc0b3370 0x7f49bc152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34484/0 (socket says 172.21.15.71:34484) 2024-09-16T07:33:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.431+0000 7f49baa00640 1 -- 172.21.15.71:0/1322746374 learned_addr learned my addr 172.21.15.71:0/1322746374 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:06.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.432+0000 7f49ba000640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49bc0cf2f0 0x7f49bc152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.432+0000 7f49bb400640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc15c080 0x7f49bc152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.433+0000 7f49baa00640 1 -- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49bc0cf2f0 msgr2=0x7f49bc152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.433+0000 7f49baa00640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49bc0cf2f0 0x7f49bc152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.433+0000 7f49baa00640 1 -- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc15c080 msgr2=0x7f49bc152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.433+0000 7f49baa00640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc15c080 0x7f49bc152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.433+0000 7f49baa00640 1 -- 172.21.15.71:0/1322746374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc154720 con 0x7f49bc0b3370 2024-09-16T07:33:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.433+0000 7f49baa00640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49bc0b3370 0x7f49bc152440 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00c680 tx=0x7f49ac00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.434+0000 7f49b2a00640 1 -- 172.21.15.71:0/1322746374 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac016020 con 0x7f49bc0b3370 2024-09-16T07:33:06.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.434+0000 7f49b2a00640 1 -- 172.21.15.71:0/1322746374 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49ac00a950 con 0x7f49bc0b3370 2024-09-16T07:33:06.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.434+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc154a00 con 0x7f49bc0b3370 2024-09-16T07:33:06.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.434+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc1552a0 con 0x7f49bc0b3370 2024-09-16T07:33:06.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.435+0000 7f49b2a00640 1 -- 172.21.15.71:0/1322746374 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00ada0 con 0x7f49bc0b3370 2024-09-16T07:33:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.437+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4994005290 con 0x7f49bc0b3370 2024-09-16T07:33:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.438+0000 7f49b2a00640 1 -- 172.21.15.71:0/1322746374 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f49ac019020 con 0x7f49bc0b3370 2024-09-16T07:33:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.439+0000 7f49b2a00640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f499c079e40 0x7f499c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.439+0000 7f49b2a00640 1 -- 172.21.15.71:0/1322746374 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f49ac099820 con 0x7f49bc0b3370 2024-09-16T07:33:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.440+0000 7f49ba000640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f499c079e40 0x7f499c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.441+0000 7f49ba000640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f499c079e40 0x7f499c07c300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f49a8034000 tx=0x7f49a8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:06.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.445+0000 7f49b2a00640 1 -- 172.21.15.71:0/1322746374 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f49ac020030 con 0x7f49bc0b3370 2024-09-16T07:33:06.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.830+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4994000bc0 con 0x7f499c079e40 2024-09-16T07:33:06.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.860+0000 7f49b2a00640 1 -- 172.21.15.71:0/1322746374 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4994000bc0 con 0x7f499c079e40 2024-09-16T07:33:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f499c079e40 msgr2=0x7f499c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f499c079e40 0x7f499c07c300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f49a8034000 tx=0x7f49a8004730 comp rx=0 tx=0).stop 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49bc0b3370 msgr2=0x7f49bc152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49bc0b3370 0x7f49bc152440 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00c680 tx=0x7f49ac00cb40 comp rx=0 tx=0).stop 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 shutdown_connections 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f499c079e40 0x7f499c07c300 secure :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f49a8034000 tx=0x7f49a8004730 comp rx=0 tx=0).stop 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc15c080 0x7f49bc152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49bc0cf2f0 0x7f49bc152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 --2- 172.21.15.71:0/1322746374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49bc0b3370 0x7f49bc152440 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 >> 172.21.15.71:0/1322746374 conn(0x7f49bc0b1840 msgr2=0x7f49bc13c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 shutdown_connections 2024-09-16T07:33:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:06.865+0000 7f49c1800640 1 -- 172.21.15.71:0/1322746374 wait complete. 2024-09-16T07:33:06.891 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone30628 2024-09-16T07:33:07.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.150+0000 7f7f76400640 1 -- 172.21.15.71:0/3750737933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7015aed0 msgr2=0x7f7f700ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.150+0000 7f7f76400640 1 --2- 172.21.15.71:0/3750737933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7015aed0 0x7f7f700ca0c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7f70069560 tx=0x7f7f6402a5f0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.151+0000 7f7f76400640 1 -- 172.21.15.71:0/3750737933 shutdown_connections 2024-09-16T07:33:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.151+0000 7f7f76400640 1 --2- 172.21.15.71:0/3750737933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f7015aed0 0x7f7f700ca0c0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.151+0000 7f7f76400640 1 --2- 172.21.15.71:0/3750737933 >> v2:172.21.15.177:3300/0 conn(0x7f7f700aec10 0x7f7f700ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.151+0000 7f7f76400640 1 -- 172.21.15.71:0/3750737933 >> 172.21.15.71:0/3750737933 conn(0x7f7f700cfa70 msgr2=0x7f7f700cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.151+0000 7f7f76400640 1 -- 172.21.15.71:0/3750737933 shutdown_connections 2024-09-16T07:33:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.151+0000 7f7f76400640 1 -- 172.21.15.71:0/3750737933 wait complete. 2024-09-16T07:33:07.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.153+0000 7f7f76400640 1 Processor -- start 2024-09-16T07:33:07.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.154+0000 7f7f76400640 1 -- start start 2024-09-16T07:33:07.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.155+0000 7f7f76400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f700aec10 0x7f7f701550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:07.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.155+0000 7f7f76400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f700cf170 0x7f7f7014a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:07.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.155+0000 7f7f76400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f7015aed0 0x7f7f700b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:07.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.155+0000 7f7f76400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700cc290 con 0x7f7f700cf170 2024-09-16T07:33:07.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.155+0000 7f7f76400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70070900 con 0x7f7f7015aed0 2024-09-16T07:33:07.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.155+0000 7f7f76400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7f700ca600 con 0x7f7f700aec10 2024-09-16T07:33:07.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f700aec10 0x7f7f701550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:07.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f700cf170 0x7f7f7014a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:07.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f700aec10 0x7f7f701550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:35638/0 (socket says 172.21.15.71:35638) 2024-09-16T07:33:07.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75000640 1 -- 172.21.15.71:0/2194598163 learned_addr learned my addr 172.21.15.71:0/2194598163 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75a00640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f7015aed0 0x7f7f700b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75000640 1 -- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f7015aed0 msgr2=0x7f7f700b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75000640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f7015aed0 0x7f7f700b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75000640 1 -- 172.21.15.71:0/2194598163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f700cf170 msgr2=0x7f7f7014a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75000640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f700cf170 0x7f7f7014a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f75000640 1 -- 172.21.15.71:0/2194598163 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7014a610 con 0x7f7f700aec10 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.156+0000 7f7f6fe00640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f700cf170 0x7f7f7014a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.157+0000 7f7f75a00640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f7015aed0 0x7f7f700b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:33:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.157+0000 7f7f75000640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f700aec10 0x7f7f701550f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00ea20 tx=0x7f7f5c00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:07.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.157+0000 7f7f6cc00640 1 -- 172.21.15.71:0/2194598163 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00cd80 con 0x7f7f700aec10 2024-09-16T07:33:07.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.157+0000 7f7f6cc00640 1 -- 172.21.15.71:0/2194598163 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f5c010430 con 0x7f7f700aec10 2024-09-16T07:33:07.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.157+0000 7f7f6cc00640 1 -- 172.21.15.71:0/2194598163 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c010840 con 0x7f7f700aec10 2024-09-16T07:33:07.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.158+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f700c6380 con 0x7f7f700aec10 2024-09-16T07:33:07.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.158+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f70151ea0 con 0x7f7f700aec10 2024-09-16T07:33:07.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.160+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f700ce530 con 0x7f7f700aec10 2024-09-16T07:33:07.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.164+0000 7f7f6cc00640 1 -- 172.21.15.71:0/2194598163 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f7f5c0109e0 con 0x7f7f700aec10 2024-09-16T07:33:07.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.166+0000 7f7f6cc00640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f50079e40 0x7f7f5007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:07.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.166+0000 7f7f6cc00640 1 -- 172.21.15.71:0/2194598163 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f7f5c09a510 con 0x7f7f700aec10 2024-09-16T07:33:07.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.167+0000 7f7f6fe00640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f50079e40 0x7f7f5007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:07.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.170+0000 7f7f6fe00640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f50079e40 0x7f7f5007c300 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7f64033000 tx=0x7f7f64004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:07.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.174+0000 7f7f6cc00640 1 -- 172.21.15.71:0/2194598163 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7f5c063b70 con 0x7f7f700aec10 2024-09-16T07:33:07.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.546+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone30628", "target": ["mon-mgr", ""]}) -- 0x7f7f700c6130 con 0x7f7f50079e40 2024-09-16T07:33:07.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.579+0000 7f7f6cc00640 1 -- 172.21.15.71:0/2194598163 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f700c6130 con 0x7f7f50079e40 2024-09-16T07:33:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f50079e40 msgr2=0x7f7f5007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f50079e40 0x7f7f5007c300 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7f64033000 tx=0x7f7f64004730 comp rx=0 tx=0).stop 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f700aec10 msgr2=0x7f7f701550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f700aec10 0x7f7f701550f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00ea20 tx=0x7f7f5c00eee0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 shutdown_connections 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f50079e40 0x7f7f5007c300 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f7015aed0 0x7f7f700b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f700cf170 0x7f7f7014a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 --2- 172.21.15.71:0/2194598163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f700aec10 0x7f7f701550f0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00ea20 tx=0x7f7f5c00eee0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.584+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 >> 172.21.15.71:0/2194598163 conn(0x7f7f700cfa70 msgr2=0x7f7f7006efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.585+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 shutdown_connections 2024-09-16T07:33:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.585+0000 7f7f76400640 1 -- 172.21.15.71:0/2194598163 wait complete. 2024-09-16T07:33:07.607 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone73294 2024-09-16T07:33:07.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.856+0000 7fa7b7000640 1 -- 172.21.15.71:0/1603828477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 msgr2=0x7fa7b0074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:07.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.856+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1603828477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 0x7fa7b0074240 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009830 tx=0x7fa79c02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.857+0000 7fa7b7000640 1 -- 172.21.15.71:0/1603828477 shutdown_connections 2024-09-16T07:33:07.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.857+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1603828477 >> v2:172.21.15.71:3300/0 conn(0x7fa7b006e6f0 0x7fa7b006eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.857+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1603828477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 0x7fa7b0074240 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.857+0000 7fa7b7000640 1 -- 172.21.15.71:0/1603828477 >> 172.21.15.71:0/1603828477 conn(0x7fa7b00b18e0 msgr2=0x7fa7b00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:07.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.857+0000 7fa7b7000640 1 -- 172.21.15.71:0/1603828477 shutdown_connections 2024-09-16T07:33:07.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.858+0000 7fa7b7000640 1 -- 172.21.15.71:0/1603828477 wait complete. 2024-09-16T07:33:07.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.861+0000 7fa7b7000640 1 Processor -- start 2024-09-16T07:33:07.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.862+0000 7fa7b7000640 1 -- start start 2024-09-16T07:33:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.863+0000 7fa7b7000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7b006e6f0 0x7fa7b014f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.863+0000 7fa7b7000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 0x7fa7b00cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.863+0000 7fa7b7000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7b015f9a0 0x7fa7b00ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.863+0000 7fa7b7000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b00ccc20 con 0x7fa7b015f9a0 2024-09-16T07:33:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.863+0000 7fa7b7000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b00cd5e0 con 0x7fa7b0073de0 2024-09-16T07:33:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.863+0000 7fa7b7000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b006f0e0 con 0x7fa7b006e6f0 2024-09-16T07:33:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7af400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 0x7fa7b00cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7af400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 0x7fa7b00cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:53852/0 (socket says 172.21.15.71:53852) 2024-09-16T07:33:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7af400640 1 -- 172.21.15.71:0/1760758747 learned_addr learned my addr 172.21.15.71:0/1760758747 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7afe00640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7b006e6f0 0x7fa7b014f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7b5000640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7b015f9a0 0x7fa7b00ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:07.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7af400640 1 -- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7b006e6f0 msgr2=0x7fa7b014f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:07.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7af400640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7b006e6f0 0x7fa7b014f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7af400640 1 -- 172.21.15.71:0/1760758747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7b015f9a0 msgr2=0x7fa7b00ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:07.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7af400640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7b015f9a0 0x7fa7b00ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:07.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.864+0000 7fa7af400640 1 -- 172.21.15.71:0/1760758747 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b00cec00 con 0x7fa7b0073de0 2024-09-16T07:33:07.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.865+0000 7fa7af400640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 0x7fa7b00cdad0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009950 tx=0x7fa79c032c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:07.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.865+0000 7fa7a7e00640 1 -- 172.21.15.71:0/1760758747 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c040070 con 0x7fa7b0073de0 2024-09-16T07:33:07.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.865+0000 7fa7a7e00640 1 -- 172.21.15.71:0/1760758747 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa79c032e70 con 0x7fa7b0073de0 2024-09-16T07:33:07.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.865+0000 7fa7a7e00640 1 -- 172.21.15.71:0/1760758747 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c003d90 con 0x7fa7b0073de0 2024-09-16T07:33:07.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.865+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b00ce640 con 0x7fa7b0073de0 2024-09-16T07:33:07.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.866+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b00cef30 con 0x7fa7b0073de0 2024-09-16T07:33:07.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.867+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b0069360 con 0x7fa7b0073de0 2024-09-16T07:33:07.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.869+0000 7fa7a7e00640 1 -- 172.21.15.71:0/1760758747 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fa79c039040 con 0x7fa7b0073de0 2024-09-16T07:33:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.870+0000 7fa7a7e00640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa790079e40 0x7fa79007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.870+0000 7fa7a7e00640 1 -- 172.21.15.71:0/1760758747 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fa79c0baf00 con 0x7fa7b0073de0 2024-09-16T07:33:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.870+0000 7fa7afe00640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa790079e40 0x7fa79007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:07.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.871+0000 7fa7afe00640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa790079e40 0x7fa79007c300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0000f60 tx=0x7fa7a0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:07.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:07.876+0000 7fa7a7e00640 1 -- 172.21.15.71:0/1760758747 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa79c084560 con 0x7fa7b0073de0 2024-09-16T07:33:08.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.143+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone73294", "target": ["mon-mgr", ""]}) -- 0x7fa7b00cd7b0 con 0x7fa790079e40 2024-09-16T07:33:08.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.160+0000 7fa7a7e00640 1 -- 172.21.15.71:0/1760758747 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa7b00cd7b0 con 0x7fa790079e40 2024-09-16T07:33:08.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa790079e40 msgr2=0x7fa79007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa790079e40 0x7fa79007c300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0000f60 tx=0x7fa7a0009040 comp rx=0 tx=0).stop 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 msgr2=0x7fa7b00cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 0x7fa7b00cdad0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009950 tx=0x7fa79c032c80 comp rx=0 tx=0).stop 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 shutdown_connections 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa790079e40 0x7fa79007c300 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7b015f9a0 0x7fa7b00ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7b0073de0 0x7fa7b00cdad0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 --2- 172.21.15.71:0/1760758747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7b006e6f0 0x7fa7b014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 >> 172.21.15.71:0/1760758747 conn(0x7fa7b00b18e0 msgr2=0x7fa7b00afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 shutdown_connections 2024-09-16T07:33:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.162+0000 7fa7b7000640 1 -- 172.21.15.71:0/1760758747 wait complete. 2024-09-16T07:33:08.177 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99717 2024-09-16T07:33:08.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.340+0000 7f2bd7200640 1 -- 172.21.15.71:0/2822977775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd815b9b0 msgr2=0x7f2bd815ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.340+0000 7f2bd7200640 1 --2- 172.21.15.71:0/2822977775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd815b9b0 0x7f2bd815ee00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009860 tx=0x7f2bc802a620 comp rx=0 tx=0).stop 2024-09-16T07:33:08.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.341+0000 7f2bd7200640 1 -- 172.21.15.71:0/2822977775 shutdown_connections 2024-09-16T07:33:08.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.341+0000 7f2bd7200640 1 --2- 172.21.15.71:0/2822977775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd815b9b0 0x7f2bd815ee00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.341+0000 7f2bd7200640 1 -- 172.21.15.71:0/2822977775 >> 172.21.15.71:0/2822977775 conn(0x7f2bd80cfa70 msgr2=0x7f2bd80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:08.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.341+0000 7f2bd7200640 1 -- 172.21.15.71:0/2822977775 shutdown_connections 2024-09-16T07:33:08.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.341+0000 7f2bd7200640 1 -- 172.21.15.71:0/2822977775 wait complete. 2024-09-16T07:33:08.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.342+0000 7f2bd7200640 1 Processor -- start 2024-09-16T07:33:08.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.342+0000 7f2bd7200640 1 -- start start 2024-09-16T07:33:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.343+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd80caa90 0x7f2bd814bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.343+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd80ce7f0 0x7f2bd80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.343+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd815b9b0 0x7f2bd814c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.343+0000 7f2bd7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80adf90 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.343+0000 7f2bd7200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80cf390 con 0x7f2bd815b9b0 2024-09-16T07:33:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.343+0000 7f2bd7200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd80ae110 con 0x7f2bd80caa90 2024-09-16T07:33:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.343+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd80ce7f0 0x7f2bd80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd80ce7f0 0x7f2bd80b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34536/0 (socket says 172.21.15.71:34536) 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5400640 1 -- 172.21.15.71:0/4212239899 learned_addr learned my addr 172.21.15.71:0/4212239899 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5e00640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd80caa90 0x7f2bd814bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd6800640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd815b9b0 0x7f2bd814c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5400640 1 -- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd80caa90 msgr2=0x7f2bd814bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5400640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd80caa90 0x7f2bd814bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5400640 1 -- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd815b9b0 msgr2=0x7f2bd814c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5400640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd815b9b0 0x7f2bd814c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5400640 1 -- 172.21.15.71:0/4212239899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80b3540 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd5400640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd80ce7f0 0x7f2bd80b4f90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400c460 tx=0x7f2bc400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bce000640 1 -- 172.21.15.71:0/4212239899 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4016020 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd80b3820 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bce000640 1 -- 172.21.15.71:0/4212239899 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bc4012a00 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.344+0000 7f2bce000640 1 -- 172.21.15.71:0/4212239899 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4012e50 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.345+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd80b4790 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.346+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd80ae5c0 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.347+0000 7f2bce000640 1 -- 172.21.15.71:0/4212239899 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f2bc4019020 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.348+0000 7f2bce000640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb8079fe0 0x7f2bb807c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:08.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.348+0000 7f2bce000640 1 -- 172.21.15.71:0/4212239899 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f2bc4099980 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.348+0000 7f2bd5e00640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb8079fe0 0x7f2bb807c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:08.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.349+0000 7f2bd5e00640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb8079fe0 0x7f2bb807c4a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8034000 tx=0x7f2bc8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:08.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.352+0000 7f2bce000640 1 -- 172.21.15.71:0/4212239899 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2bc4020020 con 0x7f2bd80ce7f0 2024-09-16T07:33:08.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.565+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99717", "target": ["mon-mgr", ""]}) -- 0x7f2bd80cfff0 con 0x7f2bb8079fe0 2024-09-16T07:33:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.583+0000 7f2bce000640 1 -- 172.21.15.71:0/4212239899 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2bd80cfff0 con 0x7f2bb8079fe0 2024-09-16T07:33:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb8079fe0 msgr2=0x7f2bb807c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb8079fe0 0x7f2bb807c4a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8034000 tx=0x7f2bc8004730 comp rx=0 tx=0).stop 2024-09-16T07:33:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd80ce7f0 msgr2=0x7f2bd80b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd80ce7f0 0x7f2bd80b4f90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400c460 tx=0x7f2bc400c920 comp rx=0 tx=0).stop 2024-09-16T07:33:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 shutdown_connections 2024-09-16T07:33:08.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb8079fe0 0x7f2bb807c4a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd815b9b0 0x7f2bd814c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd80ce7f0 0x7f2bd80b4f90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 --2- 172.21.15.71:0/4212239899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd80caa90 0x7f2bd814bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 >> 172.21.15.71:0/4212239899 conn(0x7f2bd80cfa70 msgr2=0x7f2bd80b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 shutdown_connections 2024-09-16T07:33:08.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.586+0000 7f2bd7200640 1 -- 172.21.15.71:0/4212239899 wait complete. 2024-09-16T07:33:08.601 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone43314 2024-09-16T07:33:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.765+0000 7f094fe00640 1 -- 172.21.15.71:0/3291452929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09481512b0 msgr2=0x7f09480cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.765+0000 7f094fe00640 1 --2- 172.21.15.71:0/3291452929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09481512b0 0x7f09480cefb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f093c009fd0 tx=0x7f093c02a890 comp rx=0 tx=0).stop 2024-09-16T07:33:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.765+0000 7f094fe00640 1 -- 172.21.15.71:0/3291452929 shutdown_connections 2024-09-16T07:33:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.765+0000 7f094fe00640 1 --2- 172.21.15.71:0/3291452929 >> v2:172.21.15.177:3300/0 conn(0x7f09480cf4f0 0x7f0948070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.765+0000 7f094fe00640 1 --2- 172.21.15.71:0/3291452929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09481512b0 0x7f09480cefb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.765+0000 7f094fe00640 1 -- 172.21.15.71:0/3291452929 >> 172.21.15.71:0/3291452929 conn(0x7f09480b18e0 msgr2=0x7f09480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.765+0000 7f094fe00640 1 -- 172.21.15.71:0/3291452929 shutdown_connections 2024-09-16T07:33:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.765+0000 7f094fe00640 1 -- 172.21.15.71:0/3291452929 wait complete. 2024-09-16T07:33:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.767+0000 7f094fe00640 1 Processor -- start 2024-09-16T07:33:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.767+0000 7f094fe00640 1 -- start start 2024-09-16T07:33:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09480b3e20 0x7f094814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09480cf4f0 0x7f0948073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09481512b0 0x7f09480740b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f09480cc6d0 con 0x7f09480cf4f0 2024-09-16T07:33:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09480cc550 con 0x7f09480b3e20 2024-09-16T07:33:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f09480cc850 con 0x7f09481512b0 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09480cf4f0 0x7f0948073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09480cf4f0 0x7f0948073b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34542/0 (socket says 172.21.15.71:34542) 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094ca00640 1 -- 172.21.15.71:0/3521905095 learned_addr learned my addr 172.21.15.71:0/3521905095 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.768+0000 7f094d400640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09480b3e20 0x7f094814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094de00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09481512b0 0x7f09480740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094ca00640 1 -- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09481512b0 msgr2=0x7f09480740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094ca00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09481512b0 0x7f09480740b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094ca00640 1 -- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09480b3e20 msgr2=0x7f094814fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094ca00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09480b3e20 0x7f094814fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094ca00640 1 -- 172.21.15.71:0/3521905095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0948150040 con 0x7f09480cf4f0 2024-09-16T07:33:08.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094ca00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09480cf4f0 0x7f0948073b70 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f093400c420 tx=0x7f093400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f0945600640 1 -- 172.21.15.71:0/3521905095 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934011790 con 0x7f09480cf4f0 2024-09-16T07:33:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f0945600640 1 -- 172.21.15.71:0/3521905095 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0934017070 con 0x7f09480cf4f0 2024-09-16T07:33:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09481e9390 con 0x7f09480cf4f0 2024-09-16T07:33:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f0945600640 1 -- 172.21.15.71:0/3521905095 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934011d20 con 0x7f09480cf4f0 2024-09-16T07:33:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.769+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09481e9950 con 0x7f09480cf4f0 2024-09-16T07:33:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.770+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0948069360 con 0x7f09480cf4f0 2024-09-16T07:33:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.772+0000 7f0945600640 1 -- 172.21.15.71:0/3521905095 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f093401c460 con 0x7f09480cf4f0 2024-09-16T07:33:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.772+0000 7f0945600640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0928079e40 0x7f092807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.773+0000 7f0945600640 1 -- 172.21.15.71:0/3521905095 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f09340997b0 con 0x7f09480cf4f0 2024-09-16T07:33:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.773+0000 7f094d400640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0928079e40 0x7f092807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:08.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.774+0000 7f094d400640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0928079e40 0x7f092807c300 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f093c033000 tx=0x7f093c004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:08.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.776+0000 7f0945600640 1 -- 172.21.15.71:0/3521905095 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0934063460 con 0x7f09480cf4f0 2024-09-16T07:33:08.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:08.985+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone43314", "target": ["mon-mgr", ""]}) -- 0x7f09480cd490 con 0x7f0928079e40 2024-09-16T07:33:09.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.003+0000 7f0945600640 1 -- 172.21.15.71:0/3521905095 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f09480cd490 con 0x7f0928079e40 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0928079e40 msgr2=0x7f092807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0928079e40 0x7f092807c300 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f093c033000 tx=0x7f093c004ab0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09480cf4f0 msgr2=0x7f0948073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09480cf4f0 0x7f0948073b70 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f093400c420 tx=0x7f093400c8e0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 shutdown_connections 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0928079e40 0x7f092807c300 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09481512b0 0x7f09480740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09480cf4f0 0x7f0948073b70 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 --2- 172.21.15.71:0/3521905095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09480b3e20 0x7f094814fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 >> 172.21.15.71:0/3521905095 conn(0x7f09480b18e0 msgr2=0x7f09480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:09.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 shutdown_connections 2024-09-16T07:33:09.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.005+0000 7f094fe00640 1 -- 172.21.15.71:0/3521905095 wait complete. 2024-09-16T07:33:09.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone98650 2024-09-16T07:33:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.185+0000 7f7774600640 1 -- 172.21.15.71:0/4098959894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c15dfc0 msgr2=0x7f776c06f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.185+0000 7f7774600640 1 --2- 172.21.15.71:0/4098959894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c15dfc0 0x7f776c06f300 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f776c0b3050 tx=0x7f776002a560 comp rx=0 tx=0).stop 2024-09-16T07:33:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.185+0000 7f7774600640 1 -- 172.21.15.71:0/4098959894 shutdown_connections 2024-09-16T07:33:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.185+0000 7f7774600640 1 --2- 172.21.15.71:0/4098959894 >> v2:172.21.15.177:3300/0 conn(0x7f776c0b4770 0x7f776c06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.185+0000 7f7774600640 1 --2- 172.21.15.71:0/4098959894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c15dfc0 0x7f776c06f300 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.185+0000 7f7774600640 1 -- 172.21.15.71:0/4098959894 >> 172.21.15.71:0/4098959894 conn(0x7f776c0b18e0 msgr2=0x7f776c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.185+0000 7f7774600640 1 -- 172.21.15.71:0/4098959894 shutdown_connections 2024-09-16T07:33:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.185+0000 7f7774600640 1 -- 172.21.15.71:0/4098959894 wait complete. 2024-09-16T07:33:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.187+0000 7f7774600640 1 Processor -- start 2024-09-16T07:33:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.187+0000 7f7774600640 1 -- start start 2024-09-16T07:33:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7774600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c06ec30 0x7f776c09d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7774600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776c0b4770 0x7f776c160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7774600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f776c15dfc0 0x7f776c0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7774600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0ae8f0 con 0x7f776c06ec30 2024-09-16T07:33:09.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7774600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f776c15e4b0 con 0x7f776c15dfc0 2024-09-16T07:33:09.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7774600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f776c1622c0 con 0x7f776c0b4770 2024-09-16T07:33:09.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7771c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c06ec30 0x7f776c09d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:09.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7771c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c06ec30 0x7f776c09d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34556/0 (socket says 172.21.15.71:34556) 2024-09-16T07:33:09.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.188+0000 7f7771c00640 1 -- 172.21.15.71:0/2846589797 learned_addr learned my addr 172.21.15.71:0/2846589797 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7772600640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f776c15dfc0 0x7f776c0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7771200640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776c0b4770 0x7f776c160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7771c00640 1 -- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776c0b4770 msgr2=0x7f776c160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7771c00640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776c0b4770 0x7f776c160d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7771c00640 1 -- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f776c15dfc0 msgr2=0x7f776c0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7771c00640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f776c15dfc0 0x7f776c0afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7771c00640 1 -- 172.21.15.71:0/2846589797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c09bff0 con 0x7f776c06ec30 2024-09-16T07:33:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7771c00640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c06ec30 0x7f776c09d360 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7760034040 tx=0x7f77600039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:09.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7769600640 1 -- 172.21.15.71:0/2846589797 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776003e030 con 0x7f776c06ec30 2024-09-16T07:33:09.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7769600640 1 -- 172.21.15.71:0/2846589797 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77600047a0 con 0x7f776c06ec30 2024-09-16T07:33:09.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7769600640 1 -- 172.21.15.71:0/2846589797 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760035040 con 0x7f776c06ec30 2024-09-16T07:33:09.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.189+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c09c2d0 con 0x7f776c06ec30 2024-09-16T07:33:09.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.190+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c09c890 con 0x7f776c06ec30 2024-09-16T07:33:09.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.191+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c069360 con 0x7f776c06ec30 2024-09-16T07:33:09.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.192+0000 7f7769600640 1 -- 172.21.15.71:0/2846589797 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f7760041030 con 0x7f776c06ec30 2024-09-16T07:33:09.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.192+0000 7f7769600640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f774c079e40 0x7f774c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:09.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.193+0000 7f7769600640 1 -- 172.21.15.71:0/2846589797 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7f77600ba240 con 0x7f776c06ec30 2024-09-16T07:33:09.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.193+0000 7f7771200640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f774c079e40 0x7f774c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:09.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.194+0000 7f7771200640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f774c079e40 0x7f774c07c300 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7758005e10 tx=0x7f7758005da0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:09.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.196+0000 7f7769600640 1 -- 172.21.15.71:0/2846589797 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7760083ed0 con 0x7f776c06ec30 2024-09-16T07:33:09.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.434+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone98650", "target": ["mon-mgr", ""]}) -- 0x7f776c162770 con 0x7f774c079e40 2024-09-16T07:33:09.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.456+0000 7f7769600640 1 -- 172.21.15.71:0/2846589797 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f776c162770 con 0x7f774c079e40 2024-09-16T07:33:09.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f774c079e40 msgr2=0x7f774c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f774c079e40 0x7f774c07c300 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7758005e10 tx=0x7f7758005da0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c06ec30 msgr2=0x7f776c09d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c06ec30 0x7f776c09d360 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7760034040 tx=0x7f77600039e0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 shutdown_connections 2024-09-16T07:33:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f774c079e40 0x7f774c07c300 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f776c15dfc0 0x7f776c0afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776c0b4770 0x7f776c160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 --2- 172.21.15.71:0/2846589797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f776c06ec30 0x7f776c09d360 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 >> 172.21.15.71:0/2846589797 conn(0x7f776c0b18e0 msgr2=0x7f776c0b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 shutdown_connections 2024-09-16T07:33:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.461+0000 7f7774600640 1 -- 172.21.15.71:0/2846589797 wait complete. 2024-09-16T07:33:09.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone644815 2024-09-16T07:33:09.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.738+0000 7fa0eb400640 1 -- 172.21.15.71:0/4294516401 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8031bf0 con 0x7fa0ec073de0 2024-09-16T07:33:09.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.739+0000 7fa0f3800640 1 -- 172.21.15.71:0/4294516401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec073de0 msgr2=0x7fa0ec074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.739+0000 7fa0f3800640 1 --2- 172.21.15.71:0/4294516401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec073de0 0x7fa0ec074240 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009830 tx=0x7fa0d802a490 comp rx=0 tx=0).stop 2024-09-16T07:33:09.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.740+0000 7fa0f3800640 1 -- 172.21.15.71:0/4294516401 shutdown_connections 2024-09-16T07:33:09.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.740+0000 7fa0f3800640 1 --2- 172.21.15.71:0/4294516401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec073de0 0x7fa0ec074240 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.740+0000 7fa0f3800640 1 --2- 172.21.15.71:0/4294516401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec15f9a0 0x7fa0ec15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.740+0000 7fa0f3800640 1 -- 172.21.15.71:0/4294516401 >> 172.21.15.71:0/4294516401 conn(0x7fa0ec0b18e0 msgr2=0x7fa0ec0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:09.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.740+0000 7fa0f3800640 1 -- 172.21.15.71:0/4294516401 shutdown_connections 2024-09-16T07:33:09.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.740+0000 7fa0f3800640 1 -- 172.21.15.71:0/4294516401 wait complete. 2024-09-16T07:33:09.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.744+0000 7fa0f3800640 1 Processor -- start 2024-09-16T07:33:09.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.744+0000 7fa0f3800640 1 -- start start 2024-09-16T07:33:09.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f3800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec073de0 0x7fa0ec14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:09.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f3800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec15c3f0 0x7fa0ec1fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:09.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f3800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0ec15f9a0 0x7fa0ec14a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:09.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f3800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0cd3d0 con 0x7fa0ec15f9a0 2024-09-16T07:33:09.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f3800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0cc940 con 0x7fa0ec15c3f0 2024-09-16T07:33:09.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f3800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec0cd550 con 0x7fa0ec073de0 2024-09-16T07:33:09.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f0e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec073de0 0x7fa0ec14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:09.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f0e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec073de0 0x7fa0ec14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:35690/0 (socket says 172.21.15.71:35690) 2024-09-16T07:33:09.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.747+0000 7fa0f0e00640 1 -- 172.21.15.71:0/3827236022 learned_addr learned my addr 172.21.15.71:0/3827236022 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:09.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.748+0000 7fa0f1800640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0ec15f9a0 0x7fa0ec14a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:09.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.748+0000 7fa0f0e00640 1 -- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec15c3f0 msgr2=0x7fa0ec1fcd60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:33:09.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.748+0000 7fa0f0e00640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec15c3f0 0x7fa0ec1fcd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.748+0000 7fa0f0e00640 1 -- 172.21.15.71:0/3827236022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0ec15f9a0 msgr2=0x7fa0ec14a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:09.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.748+0000 7fa0f0e00640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0ec15f9a0 0x7fa0ec14a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:09.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.748+0000 7fa0f0e00640 1 -- 172.21.15.71:0/3827236022 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec155b30 con 0x7fa0ec073de0 2024-09-16T07:33:09.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.749+0000 7fa0f1800640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0ec15f9a0 0x7fa0ec14a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:09.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.749+0000 7fa0f0e00640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec073de0 0x7fa0ec14e210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000d740 tx=0x7fa0e000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:09.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.750+0000 7fa0e8c00640 1 -- 172.21.15.71:0/3827236022 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0017070 con 0x7fa0ec073de0 2024-09-16T07:33:09.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.750+0000 7fa0e8c00640 1 -- 172.21.15.71:0/3827236022 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0e0009d20 con 0x7fa0ec073de0 2024-09-16T07:33:09.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.750+0000 7fa0e8c00640 1 -- 172.21.15.71:0/3827236022 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e00106c0 con 0x7fa0ec073de0 2024-09-16T07:33:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.751+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec154a90 con 0x7fa0ec073de0 2024-09-16T07:33:09.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.752+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec155050 con 0x7fa0ec073de0 2024-09-16T07:33:09.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.753+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec069360 con 0x7fa0ec073de0 2024-09-16T07:33:09.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.756+0000 7fa0e8c00640 1 -- 172.21.15.71:0/3827236022 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fa0e001a070 con 0x7fa0ec073de0 2024-09-16T07:33:09.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.758+0000 7fa0e8c00640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0cc079f10 0x7fa0cc07c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:09.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.758+0000 7fa0e8c00640 1 -- 172.21.15.71:0/3827236022 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7fa0e0099c10 con 0x7fa0ec073de0 2024-09-16T07:33:09.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.758+0000 7fa0ebe00640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0cc079f10 0x7fa0cc07c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:09.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.760+0000 7fa0ebe00640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0cc079f10 0x7fa0cc07c3d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8034000 tx=0x7fa0d8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:09.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:09.762+0000 7fa0e8c00640 1 -- 172.21.15.71:0/3827236022 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa0e0063cd0 con 0x7fa0ec073de0 2024-09-16T07:33:10.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.143+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone644815", "target": ["mon-mgr", ""]}) -- 0x7fa0ec0cd7b0 con 0x7fa0cc079f10 2024-09-16T07:33:10.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.178+0000 7fa0e8c00640 1 -- 172.21.15.71:0/3827236022 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa0ec0cd7b0 con 0x7fa0cc079f10 2024-09-16T07:33:10.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.182+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0cc079f10 msgr2=0x7fa0cc07c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:10.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.182+0000 7fa0f3800640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0cc079f10 0x7fa0cc07c3d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8034000 tx=0x7fa0d8004730 comp rx=0 tx=0).stop 2024-09-16T07:33:10.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.182+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec073de0 msgr2=0x7fa0ec14e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:10.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.182+0000 7fa0f3800640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec073de0 0x7fa0ec14e210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000d740 tx=0x7fa0e000dc00 comp rx=0 tx=0).stop 2024-09-16T07:33:10.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.183+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 shutdown_connections 2024-09-16T07:33:10.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.183+0000 7fa0f3800640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa0cc079f10 0x7fa0cc07c3d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.183+0000 7fa0f3800640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0ec15f9a0 0x7fa0ec14a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.183+0000 7fa0f3800640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec15c3f0 0x7fa0ec1fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.183+0000 7fa0f3800640 1 --2- 172.21.15.71:0/3827236022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec073de0 0x7fa0ec14e210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.183+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 >> 172.21.15.71:0/3827236022 conn(0x7fa0ec0b18e0 msgr2=0x7fa0ec15e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:10.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.183+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 shutdown_connections 2024-09-16T07:33:10.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.184+0000 7fa0f3800640 1 -- 172.21.15.71:0/3827236022 wait complete. 2024-09-16T07:33:10.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone849419 2024-09-16T07:33:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.379+0000 7effb7e00640 1 -- 172.21.15.71:0/1444391789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb006f220 msgr2=0x7effb006f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.379+0000 7effb7e00640 1 --2- 172.21.15.71:0/1444391789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb006f220 0x7effb006f6b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effa8009be0 tx=0x7effa802a920 comp rx=0 tx=0).stop 2024-09-16T07:33:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.379+0000 7effb7e00640 1 -- 172.21.15.71:0/1444391789 shutdown_connections 2024-09-16T07:33:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.379+0000 7effb7e00640 1 --2- 172.21.15.71:0/1444391789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb006f220 0x7effb006f6b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.379+0000 7effb7e00640 1 -- 172.21.15.71:0/1444391789 >> 172.21.15.71:0/1444391789 conn(0x7effb00b18e0 msgr2=0x7effb00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.379+0000 7effb7e00640 1 -- 172.21.15.71:0/1444391789 shutdown_connections 2024-09-16T07:33:10.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.379+0000 7effb7e00640 1 -- 172.21.15.71:0/1444391789 wait complete. 2024-09-16T07:33:10.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.381+0000 7effb7e00640 1 Processor -- start 2024-09-16T07:33:10.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.381+0000 7effb7e00640 1 -- start start 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb006f220 0x7effb0153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb0159aa0 0x7effb01f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb7e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb015cff0 0x7effb015e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7effb00cc710 con 0x7effb006f220 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb7e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effb00cc590 con 0x7effb0159aa0 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb7e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effb00cc890 con 0x7effb015cff0 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb006f220 0x7effb0153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb006f220 0x7effb0153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34582/0 (socket says 172.21.15.71:34582) 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb5400640 1 -- 172.21.15.71:0/3895467042 learned_addr learned my addr 172.21.15.71:0/3895467042 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb4a00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb0159aa0 0x7effb01f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:10.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.382+0000 7effb5e00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb015cff0 0x7effb015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:10.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.383+0000 7effb4a00640 1 -- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb015cff0 msgr2=0x7effb015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:10.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.383+0000 7effb4a00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb015cff0 0x7effb015e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.383+0000 7effb4a00640 1 -- 172.21.15.71:0/3895467042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb006f220 msgr2=0x7effb0153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:10.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.383+0000 7effb4a00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb006f220 0x7effb0153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.383+0000 7effb4a00640 1 -- 172.21.15.71:0/3895467042 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb0154090 con 0x7effb0159aa0 2024-09-16T07:33:10.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.384+0000 7effb4a00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb0159aa0 0x7effb01f88a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00c390 tx=0x7eff9c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:10.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.385+0000 7effad600640 1 -- 172.21.15.71:0/3895467042 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c011790 con 0x7effb0159aa0 2024-09-16T07:33:10.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.386+0000 7effad600640 1 -- 172.21.15.71:0/3895467042 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff9c017070 con 0x7effb0159aa0 2024-09-16T07:33:10.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.386+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb01f8de0 con 0x7effb0159aa0 2024-09-16T07:33:10.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.386+0000 7effad600640 1 -- 172.21.15.71:0/3895467042 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c011ce0 con 0x7effb0159aa0 2024-09-16T07:33:10.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.386+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb0154310 con 0x7effb0159aa0 2024-09-16T07:33:10.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.387+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb006f6b0 con 0x7effb0159aa0 2024-09-16T07:33:10.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.390+0000 7effad600640 1 -- 172.21.15.71:0/3895467042 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7eff9c01c460 con 0x7effb0159aa0 2024-09-16T07:33:10.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.392+0000 7effad600640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff90079e40 0x7eff9007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:10.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.392+0000 7effad600640 1 -- 172.21.15.71:0/3895467042 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7eff9c0996d0 con 0x7effb0159aa0 2024-09-16T07:33:10.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.393+0000 7effb5400640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff90079e40 0x7eff9007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:10.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.395+0000 7effb5400640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff90079e40 0x7eff9007c300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7effa4000c00 tx=0x7effa4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:10.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.400+0000 7effad600640 1 -- 172.21.15.71:0/3895467042 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7effb01f8de0 con 0x7effb0159aa0 2024-09-16T07:33:10.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.786+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone849419", "target": ["mon-mgr", ""]}) -- 0x7effb015bd30 con 0x7eff90079e40 2024-09-16T07:33:10.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.831+0000 7effad600640 1 -- 172.21.15.71:0/3895467042 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7effb015bd30 con 0x7eff90079e40 2024-09-16T07:33:10.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff90079e40 msgr2=0x7eff9007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:10.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff90079e40 0x7eff9007c300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7effa4000c00 tx=0x7effa4009040 comp rx=0 tx=0).stop 2024-09-16T07:33:10.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb0159aa0 msgr2=0x7effb01f88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:10.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb0159aa0 0x7effb01f88a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00c390 tx=0x7eff9c00c850 comp rx=0 tx=0).stop 2024-09-16T07:33:10.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 shutdown_connections 2024-09-16T07:33:10.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff90079e40 0x7eff9007c300 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb015cff0 0x7effb015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb0159aa0 0x7effb01f88a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 --2- 172.21.15.71:0/3895467042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb006f220 0x7effb0153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:10.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.836+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 >> 172.21.15.71:0/3895467042 conn(0x7effb00b18e0 msgr2=0x7effb015d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:10.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.837+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 shutdown_connections 2024-09-16T07:33:10.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:10.837+0000 7effb7e00640 1 -- 172.21.15.71:0/3895467042 wait complete. 2024-09-16T07:33:10.863 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone71046 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8dbe00640 1 -- 172.21.15.71:0/3751635553 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0032ab0 con 0x7ff8dc15f9a0 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8e4200640 1 -- 172.21.15.71:0/3751635553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc15f9a0 msgr2=0x7ff8dc15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8e4200640 1 --2- 172.21.15.71:0/3751635553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc15f9a0 0x7ff8dc15fd80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0005fd0 tx=0x7ff8d002a870 comp rx=0 tx=0).stop 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8e4200640 1 -- 172.21.15.71:0/3751635553 shutdown_connections 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8e4200640 1 --2- 172.21.15.71:0/3751635553 >> v2:172.21.15.177:3300/0 conn(0x7ff8dc06e6f0 0x7ff8dc06eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8e4200640 1 --2- 172.21.15.71:0/3751635553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc15f9a0 0x7ff8dc15fd80 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8e4200640 1 -- 172.21.15.71:0/3751635553 >> 172.21.15.71:0/3751635553 conn(0x7ff8dc0b18e0 msgr2=0x7ff8dc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8e4200640 1 -- 172.21.15.71:0/3751635553 shutdown_connections 2024-09-16T07:33:11.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.063+0000 7ff8e4200640 1 -- 172.21.15.71:0/3751635553 wait complete. 2024-09-16T07:33:11.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.065+0000 7ff8e4200640 1 Processor -- start 2024-09-16T07:33:11.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.065+0000 7ff8e4200640 1 -- start start 2024-09-16T07:33:11.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e4200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc06e6f0 0x7ff8dc0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:11.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e4200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8dc073de0 0x7ff8dc14b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:11.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e4200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8dc15f9a0 0x7ff8dc0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:11.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e4200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc0ccc20 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e4200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc0cd5e0 con 0x7ff8dc15f9a0 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e4200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc06f0e0 con 0x7ff8dc073de0 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc06e6f0 0x7ff8dc0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc06e6f0 0x7ff8dc0cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34592/0 (socket says 172.21.15.71:34592) 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e1800640 1 -- 172.21.15.71:0/2179250448 learned_addr learned my addr 172.21.15.71:0/2179250448 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e2200640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8dc15f9a0 0x7ff8dc0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.066+0000 7ff8e0e00640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8dc073de0 0x7ff8dc14b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e1800640 1 -- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8dc073de0 msgr2=0x7ff8dc14b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e1800640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8dc073de0 0x7ff8dc14b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e1800640 1 -- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8dc15f9a0 msgr2=0x7ff8dc0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e1800640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8dc15f9a0 0x7ff8dc0cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e1800640 1 -- 172.21.15.71:0/2179250448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc152c90 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e0e00640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8dc073de0 0x7ff8dc14b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:11.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e1800640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc06e6f0 0x7ff8dc0cdad0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0006010 tx=0x7ff8d0033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:11.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8d9600640 1 -- 172.21.15.71:0/2179250448 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0042020 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8d9600640 1 -- 172.21.15.71:0/2179250448 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8d0033db0 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8d9600640 1 -- 172.21.15.71:0/2179250448 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0037040 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc0ce160 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.067+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc0ce720 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.068+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8dc069360 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.069+0000 7ff8d9600640 1 -- 172.21.15.71:0/2179250448 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7ff8d0045020 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.070+0000 7ff8d9600640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8bc079e70 0x7ff8bc07c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:11.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.070+0000 7ff8d9600640 1 -- 172.21.15.71:0/2179250448 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5737+0+0 (secure 0 0 0) 0x7ff8d00ba6e0 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.071+0000 7ff8e0e00640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8bc079e70 0x7ff8bc07c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:11.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.071+0000 7ff8e0e00640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8bc079e70 0x7ff8bc07c330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0d0150 tx=0x7ff8c8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:11.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.075+0000 7ff8d9600640 1 -- 172.21.15.71:0/2179250448 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff8d004c030 con 0x7ff8dc06e6f0 2024-09-16T07:33:11.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.289+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone71046", "target": ["mon-mgr", ""]}) -- 0x7ff8dc0cd7b0 con 0x7ff8bc079e70 2024-09-16T07:33:11.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.304+0000 7ff8d9600640 1 -- 172.21.15.71:0/2179250448 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8dc0cd7b0 con 0x7ff8bc079e70 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8bc079e70 msgr2=0x7ff8bc07c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8bc079e70 0x7ff8bc07c330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0d0150 tx=0x7ff8c8009040 comp rx=0 tx=0).stop 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc06e6f0 msgr2=0x7ff8dc0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc06e6f0 0x7ff8dc0cdad0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0006010 tx=0x7ff8d0033ad0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 shutdown_connections 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff8bc079e70 0x7ff8bc07c330 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8dc15f9a0 0x7ff8dc0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8dc073de0 0x7ff8dc14b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 --2- 172.21.15.71:0/2179250448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8dc06e6f0 0x7ff8dc0cdad0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 >> 172.21.15.71:0/2179250448 conn(0x7ff8dc0b18e0 msgr2=0x7ff8dc0afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:11.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 shutdown_connections 2024-09-16T07:33:11.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.307+0000 7ff8e4200640 1 -- 172.21.15.71:0/2179250448 wait complete. 2024-09-16T07:33:11.322 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone86499 2024-09-16T07:33:11.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.556+0000 7fa503c00640 1 -- 172.21.15.71:0/3603892207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0708b0 msgr2=0x7fa4fc070110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.556+0000 7fa503c00640 1 --2- 172.21.15.71:0/3603892207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0708b0 0x7fa4fc070110 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009830 tx=0x7fa4ec02a510 comp rx=0 tx=0).stop 2024-09-16T07:33:11.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.557+0000 7fa503c00640 1 -- 172.21.15.71:0/3603892207 shutdown_connections 2024-09-16T07:33:11.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.557+0000 7fa503c00640 1 --2- 172.21.15.71:0/3603892207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0708b0 0x7fa4fc070110 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.557+0000 7fa503c00640 1 -- 172.21.15.71:0/3603892207 >> 172.21.15.71:0/3603892207 conn(0x7fa4fc0b18e0 msgr2=0x7fa4fc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:11.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.557+0000 7fa503c00640 1 -- 172.21.15.71:0/3603892207 shutdown_connections 2024-09-16T07:33:11.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.557+0000 7fa503c00640 1 -- 172.21.15.71:0/3603892207 wait complete. 2024-09-16T07:33:11.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.559+0000 7fa503c00640 1 Processor -- start 2024-09-16T07:33:11.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.560+0000 7fa503c00640 1 -- start start 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa503c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4fc0708b0 0x7fa4fc147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa503c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0cf6c0 0x7fa4fc14fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa503c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4fc1512b0 0x7fa4fc1500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa503c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc0cc810 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa4fbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0cf6c0 0x7fa4fc14fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa501200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4fc0708b0 0x7fa4fc147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa501c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4fc1512b0 0x7fa4fc1500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa4fbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0cf6c0 0x7fa4fc14fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34604/0 (socket says 172.21.15.71:34604) 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.562+0000 7fa4fbe00640 1 -- 172.21.15.71:0/417032498 learned_addr learned my addr 172.21.15.71:0/417032498 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:11.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.563+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc0cc690 con 0x7fa4fc0708b0 2024-09-16T07:33:11.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.563+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa4fc0cc990 con 0x7fa4fc1512b0 2024-09-16T07:33:11.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.563+0000 7fa4fbe00640 1 -- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4fc1512b0 msgr2=0x7fa4fc1500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.563+0000 7fa4fbe00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4fc1512b0 0x7fa4fc1500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.563+0000 7fa4fbe00640 1 -- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4fc0708b0 msgr2=0x7fa4fc147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.563+0000 7fa4fbe00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4fc0708b0 0x7fa4fc147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.563+0000 7fa4fbe00640 1 -- 172.21.15.71:0/417032498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc149730 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.563+0000 7fa4fbe00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0cf6c0 0x7fa4fc14fb80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c450 tx=0x7fa4e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.564+0000 7fa4f8c00640 1 -- 172.21.15.71:0/417032498 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8016020 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.564+0000 7fa4f8c00640 1 -- 172.21.15.71:0/417032498 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4e8012920 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.564+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc09af60 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.564+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc09b520 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.564+0000 7fa4f8c00640 1 -- 172.21.15.71:0/417032498 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8012d70 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.566+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d4005290 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.567+0000 7fa4f8c00640 1 -- 172.21.15.71:0/417032498 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fa4e8019020 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.568+0000 7fa4f8c00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4dc079e40 0x7fa4dc07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.568+0000 7fa4f8c00640 1 -- 172.21.15.71:0/417032498 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7fa4e8099540 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.569+0000 7fa501200640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4dc079e40 0x7fa4dc07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.571+0000 7fa501200640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4dc079e40 0x7fa4dc07c300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc1f6420 tx=0x7fa4f400d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.577+0000 7fa4f8c00640 1 -- 172.21.15.71:0/417032498 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa4e8020030 con 0x7fa4fc0cf6c0 2024-09-16T07:33:11.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.957+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone86499", "target": ["mon-mgr", ""]}) -- 0x7fa4d4000bc0 con 0x7fa4dc079e40 2024-09-16T07:33:11.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.972+0000 7fa4f8c00640 1 -- 172.21.15.71:0/417032498 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa4d4000bc0 con 0x7fa4dc079e40 2024-09-16T07:33:11.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.974+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4dc079e40 msgr2=0x7fa4dc07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.974+0000 7fa503c00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4dc079e40 0x7fa4dc07c300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc1f6420 tx=0x7fa4f400d040 comp rx=0 tx=0).stop 2024-09-16T07:33:11.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.974+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0cf6c0 msgr2=0x7fa4fc14fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:11.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.974+0000 7fa503c00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0cf6c0 0x7fa4fc14fb80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c450 tx=0x7fa4e800c910 comp rx=0 tx=0).stop 2024-09-16T07:33:11.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.975+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 shutdown_connections 2024-09-16T07:33:11.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.975+0000 7fa503c00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4dc079e40 0x7fa4dc07c300 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.975+0000 7fa503c00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4fc1512b0 0x7fa4fc1500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.975+0000 7fa503c00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4fc0cf6c0 0x7fa4fc14fb80 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.975+0000 7fa503c00640 1 --2- 172.21.15.71:0/417032498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4fc0708b0 0x7fa4fc147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.975+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 >> 172.21.15.71:0/417032498 conn(0x7fa4fc0b18e0 msgr2=0x7fa4fc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.975+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 shutdown_connections 2024-09-16T07:33:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:11.975+0000 7fa503c00640 1 -- 172.21.15.71:0/417032498 wait complete. 2024-09-16T07:33:11.990 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone648918 2024-09-16T07:33:12.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.153+0000 7ff2ac400640 1 -- 172.21.15.71:0/2954038115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a4073de0 msgr2=0x7ff2a4074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.153+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2954038115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a4073de0 0x7ff2a4074240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff290009830 tx=0x7ff29002a480 comp rx=0 tx=0).stop 2024-09-16T07:33:12.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.153+0000 7ff2ac400640 1 -- 172.21.15.71:0/2954038115 shutdown_connections 2024-09-16T07:33:12.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.153+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2954038115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a406e6f0 0x7ff2a406eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.153+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2954038115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a4073de0 0x7ff2a4074240 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.153+0000 7ff2ac400640 1 -- 172.21.15.71:0/2954038115 >> 172.21.15.71:0/2954038115 conn(0x7ff2a40b18e0 msgr2=0x7ff2a40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:12.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.153+0000 7ff2ac400640 1 -- 172.21.15.71:0/2954038115 shutdown_connections 2024-09-16T07:33:12.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.153+0000 7ff2ac400640 1 -- 172.21.15.71:0/2954038115 wait complete. 2024-09-16T07:33:12.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.154+0000 7ff2ac400640 1 Processor -- start 2024-09-16T07:33:12.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.155+0000 7ff2ac400640 1 -- start start 2024-09-16T07:33:12.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.155+0000 7ff2ac400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a406e6f0 0x7ff2a415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.155+0000 7ff2ac400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a4073de0 0x7ff2a415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.155+0000 7ff2ac400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a415f9a0 0x7ff2a415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2ac400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a406f0e0 con 0x7ff2a4073de0 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2ac400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a40cd5e0 con 0x7ff2a406e6f0 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2ac400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a406f260 con 0x7ff2a415f9a0 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a4073de0 0x7ff2a415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a4073de0 0x7ff2a415ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34620/0 (socket says 172.21.15.71:34620) 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9000640 1 -- 172.21.15.71:0/2420969307 learned_addr learned my addr 172.21.15.71:0/2420969307 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9a00640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a406e6f0 0x7ff2a415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2aa400640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a415f9a0 0x7ff2a415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9000640 1 -- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a415f9a0 msgr2=0x7ff2a415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9000640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a415f9a0 0x7ff2a415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9000640 1 -- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a406e6f0 msgr2=0x7ff2a415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9000640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a406e6f0 0x7ff2a415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.156+0000 7ff2a9000640 1 -- 172.21.15.71:0/2420969307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a41f5a90 con 0x7ff2a4073de0 2024-09-16T07:33:12.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.157+0000 7ff2a9000640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a4073de0 0x7ff2a415ce20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff290009950 tx=0x7ff290032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:12.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.157+0000 7ff2a1600640 1 -- 172.21.15.71:0/2420969307 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290040070 con 0x7ff2a4073de0 2024-09-16T07:33:12.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.157+0000 7ff2a1600640 1 -- 172.21.15.71:0/2420969307 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2900038e0 con 0x7ff2a4073de0 2024-09-16T07:33:12.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.157+0000 7ff2a1600640 1 -- 172.21.15.71:0/2420969307 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290003d30 con 0x7ff2a4073de0 2024-09-16T07:33:12.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.157+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a41f5d70 con 0x7ff2a4073de0 2024-09-16T07:33:12.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.157+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a415c620 con 0x7ff2a4073de0 2024-09-16T07:33:12.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.158+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a40d0970 con 0x7ff2a4073de0 2024-09-16T07:33:12.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.159+0000 7ff2a1600640 1 -- 172.21.15.71:0/2420969307 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7ff290039040 con 0x7ff2a4073de0 2024-09-16T07:33:12.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.160+0000 7ff2a1600640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff284079e40 0x7ff28407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:12.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.160+0000 7ff2a1600640 1 -- 172.21.15.71:0/2420969307 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7ff2900bb0c0 con 0x7ff2a4073de0 2024-09-16T07:33:12.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.160+0000 7ff2a9a00640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff284079e40 0x7ff28407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:12.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.161+0000 7ff2a9a00640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff284079e40 0x7ff28407c300 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406f030 tx=0x7ff298009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:12.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.164+0000 7ff2a1600640 1 -- 172.21.15.71:0/2420969307 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff2900bb560 con 0x7ff2a4073de0 2024-09-16T07:33:12.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.375+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone648918", "target": ["mon-mgr", ""]}) -- 0x7ff2a40cd7b0 con 0x7ff284079e40 2024-09-16T07:33:12.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.399+0000 7ff2a1600640 1 -- 172.21.15.71:0/2420969307 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2a40cd7b0 con 0x7ff284079e40 2024-09-16T07:33:12.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.403+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff284079e40 msgr2=0x7ff28407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.403+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff284079e40 0x7ff28407c300 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406f030 tx=0x7ff298009040 comp rx=0 tx=0).stop 2024-09-16T07:33:12.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.403+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a4073de0 msgr2=0x7ff2a415ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.403+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a4073de0 0x7ff2a415ce20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff290009950 tx=0x7ff290032c10 comp rx=0 tx=0).stop 2024-09-16T07:33:12.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.404+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 shutdown_connections 2024-09-16T07:33:12.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.404+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff284079e40 0x7ff28407c300 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.404+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2a415f9a0 0x7ff2a415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.404+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a4073de0 0x7ff2a415ce20 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.404+0000 7ff2ac400640 1 --2- 172.21.15.71:0/2420969307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2a406e6f0 0x7ff2a415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.404+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 >> 172.21.15.71:0/2420969307 conn(0x7ff2a40b18e0 msgr2=0x7ff2a406f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:12.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.404+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 shutdown_connections 2024-09-16T07:33:12.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.404+0000 7ff2ac400640 1 -- 172.21.15.71:0/2420969307 wait complete. 2024-09-16T07:33:12.430 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone31792 2024-09-16T07:33:12.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.661+0000 7f0b0f600640 1 -- 172.21.15.71:0/2602956630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0aec003c60 msgr2=0x7f0aec004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.661+0000 7f0b0f600640 1 --2- 172.21.15.71:0/2602956630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0aec003c60 0x7f0aec004060 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0af8001830 tx=0x7f0aec005830 comp rx=0 tx=0).stop 2024-09-16T07:33:12.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.661+0000 7f0b0d600640 1 -- 172.21.15.71:0/2602956630 reap_dead start 2024-09-16T07:33:12.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.661+0000 7f0b0f600640 1 -- 172.21.15.71:0/2602956630 shutdown_connections 2024-09-16T07:33:12.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.661+0000 7f0b0f600640 1 -- 172.21.15.71:0/2602956630 >> 172.21.15.71:0/2602956630 conn(0x7f0b080b18e0 msgr2=0x7f0b080b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:12.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.661+0000 7f0b0f600640 1 -- 172.21.15.71:0/2602956630 shutdown_connections 2024-09-16T07:33:12.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.661+0000 7f0b0f600640 1 -- 172.21.15.71:0/2602956630 wait complete. 2024-09-16T07:33:12.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.663+0000 7f0b0f600640 1 Processor -- start 2024-09-16T07:33:12.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.663+0000 7f0b0f600640 1 -- start start 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0f600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aec003c60 0x7f0b080d0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0f600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b0809ba50 0x7f0b08147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0f600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b08147b50 0x7f0b081f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0f600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0b080cca20 con 0x7f0b08147b50 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0f600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b080cc8a0 con 0x7f0aec003c60 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0f600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0b080ccba0 con 0x7f0b0809ba50 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b08147b50 0x7f0b081f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0cc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aec003c60 0x7f0b080d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:12.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b08147b50 0x7f0b081f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34636/0 (socket says 172.21.15.71:34636) 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b0d600640 1 -- 172.21.15.71:0/4254514594 learned_addr learned my addr 172.21.15.71:0/4254514594 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.664+0000 7f0b07e00640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b0809ba50 0x7f0b08147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0d600640 1 -- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b0809ba50 msgr2=0x7f0b08147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0d600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b0809ba50 0x7f0b08147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0d600640 1 -- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aec003c60 msgr2=0x7f0b080d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0d600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aec003c60 0x7f0b080d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0d600640 1 -- 172.21.15.71:0/4254514594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b08148090 con 0x7f0b08147b50 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0cc00640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aec003c60 0x7f0b080d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0d600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b08147b50 0x7f0b081f6020 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0b080d0cd0 tx=0x7f0b00006420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b04c00640 1 -- 172.21.15.71:0/4254514594 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00015020 con 0x7f0b08147b50 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b04c00640 1 -- 172.21.15.71:0/4254514594 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b000119a0 con 0x7f0b08147b50 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b04c00640 1 -- 172.21.15.71:0/4254514594 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00011df0 con 0x7f0b08147b50 2024-09-16T07:33:12.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b08148370 con 0x7f0b08147b50 2024-09-16T07:33:12.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.665+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b08158370 con 0x7f0b08147b50 2024-09-16T07:33:12.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.666+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b08069360 con 0x7f0b08147b50 2024-09-16T07:33:12.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.668+0000 7f0b04c00640 1 -- 172.21.15.71:0/4254514594 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f0b00018020 con 0x7f0b08147b50 2024-09-16T07:33:12.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.668+0000 7f0b04c00640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ae8079e40 0x7f0ae807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:12.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.669+0000 7f0b04c00640 1 -- 172.21.15.71:0/4254514594 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f0b0009a100 con 0x7f0b08147b50 2024-09-16T07:33:12.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.669+0000 7f0b0cc00640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ae8079e40 0x7f0ae807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:12.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.670+0000 7f0b0cc00640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ae8079e40 0x7f0ae807c300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0af8032000 tx=0x7f0af8001360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:12.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.673+0000 7f0b04c00640 1 -- 172.21.15.71:0/4254514594 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0b0001f030 con 0x7f0b08147b50 2024-09-16T07:33:12.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.882+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone31792", "target": ["mon-mgr", ""]}) -- 0x7f0b08070980 con 0x7f0ae8079e40 2024-09-16T07:33:12.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.905+0000 7f0b04c00640 1 -- 172.21.15.71:0/4254514594 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b08070980 con 0x7f0ae8079e40 2024-09-16T07:33:12.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ae8079e40 msgr2=0x7f0ae807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ae8079e40 0x7f0ae807c300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0af8032000 tx=0x7f0af8001360 comp rx=0 tx=0).stop 2024-09-16T07:33:12.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b08147b50 msgr2=0x7f0b081f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:12.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b08147b50 0x7f0b081f6020 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0b080d0cd0 tx=0x7f0b00006420 comp rx=0 tx=0).stop 2024-09-16T07:33:12.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 shutdown_connections 2024-09-16T07:33:12.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ae8079e40 0x7f0ae807c300 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b08147b50 0x7f0b081f6020 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b0809ba50 0x7f0b08147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.907+0000 7f0b0f600640 1 --2- 172.21.15.71:0/4254514594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aec003c60 0x7f0b080d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:12.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.908+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 >> 172.21.15.71:0/4254514594 conn(0x7f0b080b18e0 msgr2=0x7f0b0815c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:12.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.908+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 shutdown_connections 2024-09-16T07:33:12.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:12.908+0000 7f0b0f600640 1 -- 172.21.15.71:0/4254514594 wait complete. 2024-09-16T07:33:12.923 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone50521 2024-09-16T07:33:13.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.085+0000 7ff22b000640 1 -- 172.21.15.71:0/2716105995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff224159aa0 msgr2=0x7ff2240741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.085+0000 7ff22b000640 1 --2- 172.21.15.71:0/2716105995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff224159aa0 0x7ff2240741a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff214009be0 tx=0x7ff21402a8e0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.085+0000 7ff22b000640 1 -- 172.21.15.71:0/2716105995 shutdown_connections 2024-09-16T07:33:13.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.085+0000 7ff22b000640 1 --2- 172.21.15.71:0/2716105995 >> v2:172.21.15.177:3300/0 conn(0x7ff22415cff0 0x7ff22406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.085+0000 7ff22b000640 1 --2- 172.21.15.71:0/2716105995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff224159aa0 0x7ff2240741a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.085+0000 7ff22b000640 1 -- 172.21.15.71:0/2716105995 >> 172.21.15.71:0/2716105995 conn(0x7ff2240b18e0 msgr2=0x7ff2240b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:13.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.085+0000 7ff22b000640 1 -- 172.21.15.71:0/2716105995 shutdown_connections 2024-09-16T07:33:13.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.085+0000 7ff22b000640 1 -- 172.21.15.71:0/2716105995 wait complete. 2024-09-16T07:33:13.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.086+0000 7ff22b000640 1 Processor -- start 2024-09-16T07:33:13.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.086+0000 7ff22b000640 1 -- start start 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.087+0000 7ff22b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff22406f220 0x7ff224156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.087+0000 7ff22b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff224159aa0 0x7ff2241449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.087+0000 7ff22b000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff22415cff0 0x7ff224144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.087+0000 7ff22b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff22406fa60 con 0x7ff224159aa0 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.087+0000 7ff22b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff22406f8e0 con 0x7ff22406f220 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.087+0000 7ff22b000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff2240cc8a0 con 0x7ff22415cff0 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff224159aa0 0x7ff2241449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff224159aa0 0x7ff2241449a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34644/0 (socket says 172.21.15.71:34644) 2024-09-16T07:33:13.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 -- 172.21.15.71:0/3165423866 learned_addr learned my addr 172.21.15.71:0/3165423866 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:13.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223e00640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff22406f220 0x7ff224156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:13.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff229000640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff22415cff0 0x7ff224144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:13.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 -- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff22415cff0 msgr2=0x7ff224144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff22415cff0 0x7ff224144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 -- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff22406f220 msgr2=0x7ff224156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff22406f220 0x7ff224156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 -- 172.21.15.71:0/3165423866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22409ab30 con 0x7ff224159aa0 2024-09-16T07:33:13.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223e00640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff22406f220 0x7ff224156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:13.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.088+0000 7ff223400640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff224159aa0 0x7ff2241449a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff21000e940 tx=0x7ff21000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:13.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.089+0000 7ff21be00640 1 -- 172.21.15.71:0/3165423866 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000cd80 con 0x7ff224159aa0 2024-09-16T07:33:13.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.089+0000 7ff21be00640 1 -- 172.21.15.71:0/3165423866 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff210010430 con 0x7ff224159aa0 2024-09-16T07:33:13.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.089+0000 7ff21be00640 1 -- 172.21.15.71:0/3165423866 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210010880 con 0x7ff224159aa0 2024-09-16T07:33:13.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.089+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22409ae10 con 0x7ff224159aa0 2024-09-16T07:33:13.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.089+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22409d570 con 0x7ff224159aa0 2024-09-16T07:33:13.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.090+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff224069360 con 0x7ff224159aa0 2024-09-16T07:33:13.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.091+0000 7ff21be00640 1 -- 172.21.15.71:0/3165423866 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7ff210010a20 con 0x7ff224159aa0 2024-09-16T07:33:13.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.092+0000 7ff21be00640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff204079e40 0x7ff20407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:13.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.092+0000 7ff21be00640 1 -- 172.21.15.71:0/3165423866 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7ff21009a5a0 con 0x7ff224159aa0 2024-09-16T07:33:13.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.092+0000 7ff223e00640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff204079e40 0x7ff20407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:13.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.093+0000 7ff223e00640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff204079e40 0x7ff20407c300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff214034000 tx=0x7ff214004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:13.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.096+0000 7ff21be00640 1 -- 172.21.15.71:0/3165423866 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff210064250 con 0x7ff224159aa0 2024-09-16T07:33:13.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.306+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone50521", "target": ["mon-mgr", ""]}) -- 0x7ff2240b27c0 con 0x7ff204079e40 2024-09-16T07:33:13.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.324+0000 7ff21be00640 1 -- 172.21.15.71:0/3165423866 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2240b27c0 con 0x7ff204079e40 2024-09-16T07:33:13.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff204079e40 msgr2=0x7ff20407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff204079e40 0x7ff20407c300 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff214034000 tx=0x7ff214004ab0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff224159aa0 msgr2=0x7ff2241449a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff224159aa0 0x7ff2241449a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff21000e940 tx=0x7ff21000ee00 comp rx=0 tx=0).stop 2024-09-16T07:33:13.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 shutdown_connections 2024-09-16T07:33:13.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff204079e40 0x7ff20407c300 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff22415cff0 0x7ff224144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff224159aa0 0x7ff2241449a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 --2- 172.21.15.71:0/3165423866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff22406f220 0x7ff224156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 >> 172.21.15.71:0/3165423866 conn(0x7ff2240b18e0 msgr2=0x7ff22415b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 shutdown_connections 2024-09-16T07:33:13.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.326+0000 7ff22b000640 1 -- 172.21.15.71:0/3165423866 wait complete. 2024-09-16T07:33:13.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone51223 2024-09-16T07:33:13.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.581+0000 7f5eb2a00640 1 -- 172.21.15.71:0/999953419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0aec10 msgr2=0x7f5eac0ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.581+0000 7f5eb2a00640 1 --2- 172.21.15.71:0/999953419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0aec10 0x7f5eac0ce530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005fd0 tx=0x7f5ea002b1a0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.581+0000 7f5eb2a00640 1 -- 172.21.15.71:0/999953419 shutdown_connections 2024-09-16T07:33:13.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.581+0000 7f5eb2a00640 1 --2- 172.21.15.71:0/999953419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0aec10 0x7f5eac0ce530 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.581+0000 7f5eb2a00640 1 -- 172.21.15.71:0/999953419 >> 172.21.15.71:0/999953419 conn(0x7f5eac0cfa70 msgr2=0x7f5eac0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:13.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.581+0000 7f5eb2a00640 1 -- 172.21.15.71:0/999953419 shutdown_connections 2024-09-16T07:33:13.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.581+0000 7f5eb2a00640 1 -- 172.21.15.71:0/999953419 wait complete. 2024-09-16T07:33:13.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.584+0000 7f5eb2a00640 1 Processor -- start 2024-09-16T07:33:13.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.584+0000 7f5eb2a00640 1 -- start start 2024-09-16T07:33:13.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.585+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0aec10 0x7f5eac156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:13.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.585+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0ca7d0 0x7f5eac14af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:13.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.585+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5eac15aed0 0x7f5eac0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:13.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.585+0000 7f5eb2a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0cf170 con 0x7f5eac0aec10 2024-09-16T07:33:13.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.585+0000 7f5eb2a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac070900 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.585+0000 7f5eb2a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac0cf2f0 con 0x7f5eac15aed0 2024-09-16T07:33:13.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.585+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5eac15aed0 0x7f5eac0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:13.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.585+0000 7f5eb0c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0ca7d0 0x7f5eac14af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:13.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.586+0000 7f5eb0c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0ca7d0 0x7f5eac14af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:54014/0 (socket says 172.21.15.71:54014) 2024-09-16T07:33:13.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.586+0000 7f5eb0c00640 1 -- 172.21.15.71:0/467374542 learned_addr learned my addr 172.21.15.71:0/467374542 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:13.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.586+0000 7f5eb1600640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0aec10 0x7f5eac156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:13.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.586+0000 7f5eb0c00640 1 -- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5eac15aed0 msgr2=0x7f5eac0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.587+0000 7f5eb0c00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5eac15aed0 0x7f5eac0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.587+0000 7f5eb0c00640 1 -- 172.21.15.71:0/467374542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0aec10 msgr2=0x7f5eac156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.587+0000 7f5eb0c00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0aec10 0x7f5eac156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.587+0000 7f5eb0c00640 1 -- 172.21.15.71:0/467374542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac157390 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.587+0000 7f5eb1600640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0aec10 0x7f5eac156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:13.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.587+0000 7f5eb2000640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5eac15aed0 0x7f5eac0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:13.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.587+0000 7f5eb0c00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0ca7d0 0x7f5eac14af30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c400 tx=0x7f5e9800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:13.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.588+0000 7f5ea9600640 1 -- 172.21.15.71:0/467374542 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98011790 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.588+0000 7f5ea9600640 1 -- 172.21.15.71:0/467374542 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e98017070 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.588+0000 7f5ea9600640 1 -- 172.21.15.71:0/467374542 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98011ce0 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.588+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac157670 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.588+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac155810 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.589+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e88005290 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.591+0000 7f5ea9600640 1 -- 172.21.15.71:0/467374542 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f5e9803c660 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.593+0000 7f5ea9600640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e8c079e90 0x7f5e8c07c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:13.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.593+0000 7f5ea9600640 1 -- 172.21.15.71:0/467374542 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f5e98099ab0 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.594+0000 7f5eb1600640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e8c079e90 0x7f5e8c07c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.595+0000 7f5eb1600640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e8c079e90 0x7f5e8c07c350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0033000 tx=0x7f5ea0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:13.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.599+0000 7f5ea9600640 1 -- 172.21.15.71:0/467374542 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5e98063760 con 0x7f5eac0ca7d0 2024-09-16T07:33:13.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.975+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone51223", "target": ["mon-mgr", ""]}) -- 0x7f5e88000bc0 con 0x7f5e8c079e90 2024-09-16T07:33:13.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.991+0000 7f5ea9600640 1 -- 172.21.15.71:0/467374542 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e88000bc0 con 0x7f5e8c079e90 2024-09-16T07:33:13.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e8c079e90 msgr2=0x7f5e8c07c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e8c079e90 0x7f5e8c07c350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0033000 tx=0x7f5ea0004730 comp rx=0 tx=0).stop 2024-09-16T07:33:13.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0ca7d0 msgr2=0x7f5eac14af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:13.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0ca7d0 0x7f5eac14af30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c400 tx=0x7f5e9800c8c0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 shutdown_connections 2024-09-16T07:33:13.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e8c079e90 0x7f5e8c07c350 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5eac15aed0 0x7f5eac0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5eac0ca7d0 0x7f5eac14af30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 --2- 172.21.15.71:0/467374542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0aec10 0x7f5eac156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:13.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.993+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 >> 172.21.15.71:0/467374542 conn(0x7f5eac0cfa70 msgr2=0x7f5eac072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:13.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.994+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 shutdown_connections 2024-09-16T07:33:13.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:13.994+0000 7f5eb2a00640 1 -- 172.21.15.71:0/467374542 wait complete. 2024-09-16T07:33:14.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone61947 2024-09-16T07:33:14.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.171+0000 7f17fe000640 1 -- 172.21.15.71:0/1661844596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8164d20 msgr2=0x7f17f8167110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:14.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.171+0000 7f17fe000640 1 --2- 172.21.15.71:0/1661844596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8164d20 0x7f17f8167110 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f17f0005fd0 tx=0x7f17f002a8b0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.171+0000 7f17fe000640 1 -- 172.21.15.71:0/1661844596 shutdown_connections 2024-09-16T07:33:14.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.171+0000 7f17fe000640 1 --2- 172.21.15.71:0/1661844596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8164d20 0x7f17f8167110 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.171+0000 7f17fe000640 1 -- 172.21.15.71:0/1661844596 >> 172.21.15.71:0/1661844596 conn(0x7f17f80b18e0 msgr2=0x7f17f80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:14.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.171+0000 7f17fe000640 1 -- 172.21.15.71:0/1661844596 shutdown_connections 2024-09-16T07:33:14.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.171+0000 7f17fe000640 1 -- 172.21.15.71:0/1661844596 wait complete. 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.173+0000 7f17fe000640 1 Processor -- start 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.173+0000 7f17fe000640 1 -- start start 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17fe000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8073de0 0x7f17f80cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17fe000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f815f9a0 0x7f17f814f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17fe000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17f8164d20 0x7f17f80c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17fe000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f17f8167650 con 0x7f17f8073de0 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17fe000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17f80cd5e0 con 0x7f17f815f9a0 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17fe000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f17f81677d0 con 0x7f17f8164d20 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8073de0 0x7f17f80cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:14.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8073de0 0x7f17f80cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34656/0 (socket says 172.21.15.71:34656) 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7400640 1 -- 172.21.15.71:0/1857622521 learned_addr learned my addr 172.21.15.71:0/1857622521 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f6a00640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f815f9a0 0x7f17f814f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7e00640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17f8164d20 0x7f17f80c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7400640 1 -- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17f8164d20 msgr2=0x7f17f80c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7400640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17f8164d20 0x7f17f80c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7400640 1 -- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f815f9a0 msgr2=0x7f17f814f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7400640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f815f9a0 0x7f17f814f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f7400640 1 -- 172.21.15.71:0/1857622521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f814fcc0 con 0x7f17f8073de0 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.174+0000 7f17f6a00640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f815f9a0 0x7f17f814f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.175+0000 7f17f7400640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8073de0 0x7f17f80cdad0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c530 tx=0x7f17e800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.175+0000 7f17ef400640 1 -- 172.21.15.71:0/1857622521 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80052d0 con 0x7f17f8073de0 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.175+0000 7f17ef400640 1 -- 172.21.15.71:0/1857622521 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17e8005470 con 0x7f17f8073de0 2024-09-16T07:33:14.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.175+0000 7f17ef400640 1 -- 172.21.15.71:0/1857622521 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8012870 con 0x7f17f8073de0 2024-09-16T07:33:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.175+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f814e350 con 0x7f17f8073de0 2024-09-16T07:33:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.175+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f814e910 con 0x7f17f8073de0 2024-09-16T07:33:14.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.176+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f8069360 con 0x7f17f8073de0 2024-09-16T07:33:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.177+0000 7f17ef400640 1 -- 172.21.15.71:0/1857622521 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f17e80035f0 con 0x7f17f8073de0 2024-09-16T07:33:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.178+0000 7f17ef400640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17d8079e40 0x7f17d807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.178+0000 7f17ef400640 1 -- 172.21.15.71:0/1857622521 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f17e8099630 con 0x7f17f8073de0 2024-09-16T07:33:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.178+0000 7f17f6a00640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17d8079e40 0x7f17d807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:14.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.179+0000 7f17f6a00640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17d8079e40 0x7f17d807c300 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f17f814f6f0 tx=0x7f17e4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:14.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.182+0000 7f17ef400640 1 -- 172.21.15.71:0/1857622521 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f17e80632e0 con 0x7f17f8073de0 2024-09-16T07:33:14.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.399+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone61947", "target": ["mon-mgr", ""]}) -- 0x7f17f8074240 con 0x7f17d8079e40 2024-09-16T07:33:14.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.430+0000 7f17ef400640 1 -- 172.21.15.71:0/1857622521 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f17f8074240 con 0x7f17d8079e40 2024-09-16T07:33:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17d8079e40 msgr2=0x7f17d807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17d8079e40 0x7f17d807c300 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f17f814f6f0 tx=0x7f17e4009040 comp rx=0 tx=0).stop 2024-09-16T07:33:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8073de0 msgr2=0x7f17f80cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8073de0 0x7f17f80cdad0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c530 tx=0x7f17e800c9f0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 shutdown_connections 2024-09-16T07:33:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17d8079e40 0x7f17d807c300 secure :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f17f814f6f0 tx=0x7f17e4009040 comp rx=0 tx=0).stop 2024-09-16T07:33:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17f8164d20 0x7f17f80c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f815f9a0 0x7f17f814f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 --2- 172.21.15.71:0/1857622521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f8073de0 0x7f17f80cdad0 secure :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c530 tx=0x7f17e800c9f0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.434+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 >> 172.21.15.71:0/1857622521 conn(0x7f17f80b18e0 msgr2=0x7f17f80b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.435+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 shutdown_connections 2024-09-16T07:33:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.435+0000 7f17fe000640 1 -- 172.21.15.71:0/1857622521 wait complete. 2024-09-16T07:33:14.459 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone418816 2024-09-16T07:33:14.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.727+0000 7fea3b000640 1 -- 172.21.15.71:0/757892933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea3415b9b0 msgr2=0x7fea3415ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:14.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.727+0000 7fea3b000640 1 --2- 172.21.15.71:0/757892933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea3415b9b0 0x7fea3415ee00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea28009830 tx=0x7fea2802a5c0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.727+0000 7fea3b000640 1 -- 172.21.15.71:0/757892933 shutdown_connections 2024-09-16T07:33:14.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.727+0000 7fea3b000640 1 --2- 172.21.15.71:0/757892933 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea340caa90 0x7fea340ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.727+0000 7fea3b000640 1 --2- 172.21.15.71:0/757892933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea3415b9b0 0x7fea3415ee00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.727+0000 7fea3b000640 1 -- 172.21.15.71:0/757892933 >> 172.21.15.71:0/757892933 conn(0x7fea340cfa70 msgr2=0x7fea340cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:14.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.728+0000 7fea3b000640 1 -- 172.21.15.71:0/757892933 shutdown_connections 2024-09-16T07:33:14.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.728+0000 7fea3b000640 1 -- 172.21.15.71:0/757892933 wait complete. 2024-09-16T07:33:14.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.730+0000 7fea3b000640 1 Processor -- start 2024-09-16T07:33:14.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.730+0000 7fea3b000640 1 -- start start 2024-09-16T07:33:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.732+0000 7fea3b000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea340caa90 0x7fea3406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.732+0000 7fea3b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea340ce7f0 0x7fea340b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea3b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea3415b9b0 0x7fea340c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea3b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fea340addc0 con 0x7fea3415b9b0 2024-09-16T07:33:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea3b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea340cf390 con 0x7fea340ce7f0 2024-09-16T07:33:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea3b000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fea340adf40 con 0x7fea340caa90 2024-09-16T07:33:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea39c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea340caa90 0x7fea3406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea39c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea340caa90 0x7fea3406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:53708/0 (socket says 172.21.15.71:53708) 2024-09-16T07:33:14.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea39c00640 1 -- 172.21.15.71:0/12744341 learned_addr learned my addr 172.21.15.71:0/12744341 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:14.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea39200640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea340ce7f0 0x7fea340b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:14.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.733+0000 7fea3a600640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea3415b9b0 0x7fea340c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.734+0000 7fea39c00640 1 -- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea340ce7f0 msgr2=0x7fea340b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.734+0000 7fea39c00640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea340ce7f0 0x7fea340b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.734+0000 7fea39c00640 1 -- 172.21.15.71:0/12744341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea3415b9b0 msgr2=0x7fea340c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.734+0000 7fea39c00640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea3415b9b0 0x7fea340c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.734+0000 7fea39c00640 1 -- 172.21.15.71:0/12744341 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3406fee0 con 0x7fea340caa90 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.734+0000 7fea3a600640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea3415b9b0 0x7fea340c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.735+0000 7fea39c00640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea340caa90 0x7fea3406f5e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea28033c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.735+0000 7fea31600640 1 -- 172.21.15.71:0/12744341 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28031d60 con 0x7fea340caa90 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.735+0000 7fea31600640 1 -- 172.21.15.71:0/12744341 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea28033e10 con 0x7fea340caa90 2024-09-16T07:33:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.735+0000 7fea31600640 1 -- 172.21.15.71:0/12744341 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28033310 con 0x7fea340caa90 2024-09-16T07:33:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.735+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea340b2a20 con 0x7fea340caa90 2024-09-16T07:33:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.736+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea340b2fe0 con 0x7fea340caa90 2024-09-16T07:33:14.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.737+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea340ae3f0 con 0x7fea340caa90 2024-09-16T07:33:14.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.739+0000 7fea31600640 1 -- 172.21.15.71:0/12744341 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fea280334b0 con 0x7fea340caa90 2024-09-16T07:33:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.741+0000 7fea31600640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fea14079e90 0x7fea1407c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.741+0000 7fea31600640 1 -- 172.21.15.71:0/12744341 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7fea280bae40 con 0x7fea340caa90 2024-09-16T07:33:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.742+0000 7fea39200640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fea14079e90 0x7fea1407c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.743+0000 7fea39200640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fea14079e90 0x7fea1407c350 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fea34161360 tx=0x7fea20009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:14.748+0000 7fea31600640 1 -- 172.21.15.71:0/12744341 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fea2803f030 con 0x7fea340caa90 2024-09-16T07:33:15.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.016+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone418816", "target": ["mon-mgr", ""]}) -- 0x7fea340b59f0 con 0x7fea14079e90 2024-09-16T07:33:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.039+0000 7fea31600640 1 -- 172.21.15.71:0/12744341 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fea340b59f0 con 0x7fea14079e90 2024-09-16T07:33:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fea14079e90 msgr2=0x7fea1407c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fea14079e90 0x7fea1407c350 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fea34161360 tx=0x7fea20009040 comp rx=0 tx=0).stop 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea340caa90 msgr2=0x7fea3406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea340caa90 0x7fea3406f5e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea28033c20 comp rx=0 tx=0).stop 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 shutdown_connections 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fea14079e90 0x7fea1407c350 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea3415b9b0 0x7fea340c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea340ce7f0 0x7fea340b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 --2- 172.21.15.71:0/12744341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea340caa90 0x7fea3406f5e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 >> 172.21.15.71:0/12744341 conn(0x7fea340cfa70 msgr2=0x7fea340af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 shutdown_connections 2024-09-16T07:33:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.042+0000 7fea3b000640 1 -- 172.21.15.71:0/12744341 wait complete. 2024-09-16T07:33:15.058 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:33:15.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:33:15.143 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:33:15.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:33:15.153 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:33:15.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.313+0000 7f7784e00640 1 -- 172.21.15.71:0/1436594833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7780070870 msgr2=0x7f77800b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:15.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.313+0000 7f7784e00640 1 --2- 172.21.15.71:0/1436594833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7780070870 0x7f77800b3ec0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f776c009830 tx=0x7f776c02a510 comp rx=0 tx=0).stop 2024-09-16T07:33:15.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.313+0000 7f7784e00640 1 -- 172.21.15.71:0/1436594833 shutdown_connections 2024-09-16T07:33:15.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.313+0000 7f7784e00640 1 --2- 172.21.15.71:0/1436594833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7780070870 0x7f77800b3ec0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.313+0000 7f7784e00640 1 --2- 172.21.15.71:0/1436594833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77800b4bd0 0x7f77801437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.313+0000 7f7784e00640 1 -- 172.21.15.71:0/1436594833 >> 172.21.15.71:0/1436594833 conn(0x7f77800b1840 msgr2=0x7f77800b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:15.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.313+0000 7f7784e00640 1 -- 172.21.15.71:0/1436594833 shutdown_connections 2024-09-16T07:33:15.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.313+0000 7f7784e00640 1 -- 172.21.15.71:0/1436594833 wait complete. 2024-09-16T07:33:15.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.315+0000 7f7784e00640 1 Processor -- start 2024-09-16T07:33:15.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.315+0000 7f7784e00640 1 -- start start 2024-09-16T07:33:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f7784e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7780070870 0x7f77800ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f7784e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77800b4400 0x7f77800a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f7784e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77800b4bd0 0x7f7780159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f7784e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77800cde60 con 0x7f77800b4400 2024-09-16T07:33:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f7784e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f77800cdce0 con 0x7f7780070870 2024-09-16T07:33:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f7784e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f77800cdfe0 con 0x7f77800b4bd0 2024-09-16T07:33:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77800b4400 0x7f77800a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77800b4400 0x7f77800a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40512/0 (socket says 172.21.15.71:40512) 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777d400640 1 -- 172.21.15.71:0/3977648787 learned_addr learned my addr 172.21.15.71:0/3977648787 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777de00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7780070870 0x7f77800ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777e800640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77800b4bd0 0x7f7780159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777d400640 1 -- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77800b4bd0 msgr2=0x7f7780159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777d400640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77800b4bd0 0x7f7780159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777d400640 1 -- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7780070870 msgr2=0x7f77800ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777d400640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7780070870 0x7f77800ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.316+0000 7f777d400640 1 -- 172.21.15.71:0/3977648787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77801f4cc0 con 0x7f77800b4400 2024-09-16T07:33:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.317+0000 7f777d400640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77800b4400 0x7f77800a9460 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f776c000c00 tx=0x7f776c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:15.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.317+0000 7f7776000640 1 -- 172.21.15.71:0/3977648787 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c040070 con 0x7f77800b4400 2024-09-16T07:33:15.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.317+0000 7f7776000640 1 -- 172.21.15.71:0/3977648787 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f776c0038e0 con 0x7f77800b4400 2024-09-16T07:33:15.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.317+0000 7f7776000640 1 -- 172.21.15.71:0/3977648787 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c003d30 con 0x7f77800b4400 2024-09-16T07:33:15.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.317+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77800a9d60 con 0x7f77800b4400 2024-09-16T07:33:15.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.317+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77800aa320 con 0x7f77800b4400 2024-09-16T07:33:15.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.318+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77800b3ec0 con 0x7f77800b4400 2024-09-16T07:33:15.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.320+0000 7f7776000640 1 -- 172.21.15.71:0/3977648787 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f776c039040 con 0x7f77800b4400 2024-09-16T07:33:15.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.320+0000 7f7776000640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7760079e70 0x7f776007c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:15.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.321+0000 7f7776000640 1 -- 172.21.15.71:0/3977648787 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f776c0bb170 con 0x7f77800b4400 2024-09-16T07:33:15.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.321+0000 7f777de00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7760079e70 0x7f776007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:15.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.322+0000 7f777de00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7760079e70 0x7f776007c330 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f777000ab30 tx=0x7f7770005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:15.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.324+0000 7f7776000640 1 -- 172.21.15.71:0/3977648787 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f776c084e20 con 0x7f77800b4400 2024-09-16T07:33:15.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.620+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f77800b1cc0 con 0x7f77800b4400 2024-09-16T07:33:15.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.621+0000 7f7776000640 1 -- 172.21.15.71:0/3977648787 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v43)=0 v43) ==== 131+0+0 (secure 0 0 0) 0x7f776c0bb610 con 0x7f77800b4400 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7760079e70 msgr2=0x7f776007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7760079e70 0x7f776007c330 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f777000ab30 tx=0x7f7770005f90 comp rx=0 tx=0).stop 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77800b4400 msgr2=0x7f77800a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77800b4400 0x7f77800a9460 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f776c000c00 tx=0x7f776c032c10 comp rx=0 tx=0).stop 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 shutdown_connections 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7760079e70 0x7f776007c330 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77800b4bd0 0x7f7780159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77800b4400 0x7f77800a9460 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 --2- 172.21.15.71:0/3977648787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7780070870 0x7f77800ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 >> 172.21.15.71:0/3977648787 conn(0x7f77800b1840 msgr2=0x7f778015bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 shutdown_connections 2024-09-16T07:33:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:15.625+0000 7f7784e00640 1 -- 172.21.15.71:0/3977648787 wait complete. 2024-09-16T07:33:15.647 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:33:15.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:33:15.763 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:33:15.881 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:33:15.881 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:33:15.881 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:33:15.881 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:15.943 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:33:15.944 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:15.951 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:33:16.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:33:16.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:33:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.278+0000 7f7dc4800640 1 -- 172.21.15.71:0/245513492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7da4004800 msgr2=0x7f7da4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.278+0000 7f7dc4800640 1 --2- 172.21.15.71:0/245513492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7da4004800 0x7f7da4004c80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0016c0 tx=0x7f7da4005830 comp rx=0 tx=0).stop 2024-09-16T07:33:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.278+0000 7f7dc2800640 1 -- 172.21.15.71:0/245513492 reap_dead start 2024-09-16T07:33:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.278+0000 7f7dc4800640 1 -- 172.21.15.71:0/245513492 shutdown_connections 2024-09-16T07:33:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.278+0000 7f7dc4800640 1 -- 172.21.15.71:0/245513492 >> 172.21.15.71:0/245513492 conn(0x7f7dbc0b1950 msgr2=0x7f7dbc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:16.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.278+0000 7f7dc4800640 1 -- 172.21.15.71:0/245513492 shutdown_connections 2024-09-16T07:33:16.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.278+0000 7f7dc4800640 1 -- 172.21.15.71:0/245513492 wait complete. 2024-09-16T07:33:16.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.280+0000 7f7dc4800640 1 Processor -- start 2024-09-16T07:33:16.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.280+0000 7f7dc4800640 1 -- start start 2024-09-16T07:33:16.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc4800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7da4004800 0x7f7dbc14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:16.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc4800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7dbc1e5880 0x7f7dbc1e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:16.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc4800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc0cff10 0x7f7dbc0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:16.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc4800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc074840 con 0x7f7dbc0cff10 2024-09-16T07:33:16.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc4800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc0746c0 con 0x7f7da4004800 2024-09-16T07:33:16.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc4800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc06f160 con 0x7f7dbc1e5880 2024-09-16T07:33:16.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc0cff10 0x7f7dbc0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc0cff10 0x7f7dbc0d0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40524/0 (socket says 172.21.15.71:40524) 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc2800640 1 -- 172.21.15.71:0/3109258051 learned_addr learned my addr 172.21.15.71:0/3109258051 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc1400640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7dbc1e5880 0x7f7dbc1e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc1e00640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7da4004800 0x7f7dbc14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc2800640 1 -- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7dbc1e5880 msgr2=0x7f7dbc1e5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc2800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7dbc1e5880 0x7f7dbc1e5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc2800640 1 -- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7da4004800 msgr2=0x7f7dbc14a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc2800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7da4004800 0x7f7dbc14a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.281+0000 7f7dc2800640 1 -- 172.21.15.71:0/3109258051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc0cbc60 con 0x7f7dbc0cff10 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.282+0000 7f7dc2800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc0cff10 0x7f7dbc0d0390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc06e130 tx=0x7f7db800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.282+0000 7f7db6000640 1 -- 172.21.15.71:0/3109258051 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8004b20 con 0x7f7dbc0cff10 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.282+0000 7f7db6000640 1 -- 172.21.15.71:0/3109258051 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7db8004cc0 con 0x7f7dbc0cff10 2024-09-16T07:33:16.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.282+0000 7f7db6000640 1 -- 172.21.15.71:0/3109258051 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8003bd0 con 0x7f7dbc0cff10 2024-09-16T07:33:16.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.282+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc1e63f0 con 0x7f7dbc0cff10 2024-09-16T07:33:16.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.282+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc1f83c0 con 0x7f7dbc0cff10 2024-09-16T07:33:16.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.283+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc06f6c0 con 0x7f7dbc0cff10 2024-09-16T07:33:16.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.284+0000 7f7db6000640 1 -- 172.21.15.71:0/3109258051 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f7db8004030 con 0x7f7dbc0cff10 2024-09-16T07:33:16.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.285+0000 7f7db6000640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7da0079e70 0x7f7da007c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:16.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.285+0000 7f7db6000640 1 -- 172.21.15.71:0/3109258051 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f7db8099dc0 con 0x7f7dbc0cff10 2024-09-16T07:33:16.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.286+0000 7f7dc1e00640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7da0079e70 0x7f7da007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:16.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.286+0000 7f7dc1e00640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7da0079e70 0x7f7da007c330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7db0001240 tx=0x7f7db0006040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:16.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.289+0000 7f7db6000640 1 -- 172.21.15.71:0/3109258051 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7db8063480 con 0x7f7dbc0cff10 2024-09-16T07:33:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.594+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7dbc0cc670 con 0x7f7dbc0cff10 2024-09-16T07:33:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.595+0000 7f7db6000640 1 -- 172.21.15.71:0/3109258051 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7f7db8067eb0 con 0x7f7dbc0cff10 2024-09-16T07:33:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7da0079e70 msgr2=0x7f7da007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7da0079e70 0x7f7da007c330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7db0001240 tx=0x7f7db0006040 comp rx=0 tx=0).stop 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc0cff10 msgr2=0x7f7dbc0d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc0cff10 0x7f7dbc0d0390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc06e130 tx=0x7f7db800f3c0 comp rx=0 tx=0).stop 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 shutdown_connections 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7da0079e70 0x7f7da007c330 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc0cff10 0x7f7dbc0d0390 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7dbc1e5880 0x7f7dbc1e5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:16.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 --2- 172.21.15.71:0/3109258051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7da4004800 0x7f7dbc14a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:16.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.598+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 >> 172.21.15.71:0/3109258051 conn(0x7f7dbc0b1950 msgr2=0x7f7dbc0ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:16.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.599+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 shutdown_connections 2024-09-16T07:33:16.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:16.599+0000 7f7dc4800640 1 -- 172.21.15.71:0/3109258051 wait complete. 2024-09-16T07:33:16.626 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:33:16.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:16.626 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:33:16.747 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:33:16.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:16.747 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:33:16.818 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:33:16.818 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:33:16.819 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:16.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:33:16.880 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:33:16.880 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:33:16.880 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:16.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:33:16.942 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:33:16.942 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:33:16.942 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:16.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:33:17.000 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:33:17.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.233+0000 7fa7cfa00640 1 -- 172.21.15.71:0/2942870029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80ccf00 msgr2=0x7fa7c80acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.233+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/2942870029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80ccf00 0x7fa7c80acd30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4005fd0 tx=0x7fa7b402a540 comp rx=0 tx=0).stop 2024-09-16T07:33:17.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.234+0000 7fa7cfa00640 1 -- 172.21.15.71:0/2942870029 shutdown_connections 2024-09-16T07:33:17.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.234+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/2942870029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80ccf00 0x7fa7c80acd30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.234+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/2942870029 >> v2:172.21.15.177:3300/0 conn(0x7fa7c815f5a0 0x7fa7c80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.234+0000 7fa7cfa00640 1 -- 172.21.15.71:0/2942870029 >> 172.21.15.71:0/2942870029 conn(0x7fa7c80b1950 msgr2=0x7fa7c80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:17.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.234+0000 7fa7cfa00640 1 -- 172.21.15.71:0/2942870029 shutdown_connections 2024-09-16T07:33:17.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.234+0000 7fa7cfa00640 1 -- 172.21.15.71:0/2942870029 wait complete. 2024-09-16T07:33:17.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.235+0000 7fa7cfa00640 1 Processor -- start 2024-09-16T07:33:17.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.235+0000 7fa7cfa00640 1 -- start start 2024-09-16T07:33:17.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.236+0000 7fa7cfa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80ad270 0x7fa7c81f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.236+0000 7fa7cfa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80ccf00 0x7fa7c8157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.236+0000 7fa7cfa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c815f5a0 0x7fa7c80d0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.236+0000 7fa7cfa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c806f2e0 con 0x7fa7c815f5a0 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.236+0000 7fa7cfa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c806f160 con 0x7fa7c80ad270 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.236+0000 7fa7cfa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c806f460 con 0x7fa7c80ccf00 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c815f5a0 0x7fa7c80d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80ad270 0x7fa7c81f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c815f5a0 0x7fa7c80d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40528/0 (socket says 172.21.15.71:40528) 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cda00640 1 -- 172.21.15.71:0/1468942509 learned_addr learned my addr 172.21.15.71:0/1468942509 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7c7e00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80ccf00 0x7fa7c8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cd000640 1 -- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80ccf00 msgr2=0x7fa7c8157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cd000640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80ccf00 0x7fa7c8157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cd000640 1 -- 172.21.15.71:0/1468942509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c815f5a0 msgr2=0x7fa7c80d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cd000640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c815f5a0 0x7fa7c80d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cd000640 1 -- 172.21.15.71:0/1468942509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c80d0ae0 con 0x7fa7c80ad270 2024-09-16T07:33:17.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cda00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c815f5a0 0x7fa7c80d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:33:17.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.237+0000 7fa7cd000640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80ad270 0x7fa7c81f5500 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800e6d0 tx=0x7fa7b800eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:17.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.238+0000 7fa7c4c00640 1 -- 172.21.15.71:0/1468942509 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800cab0 con 0x7fa7c80ad270 2024-09-16T07:33:17.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.238+0000 7fa7c4c00640 1 -- 172.21.15.71:0/1468942509 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7b800cd40 con 0x7fa7c80ad270 2024-09-16T07:33:17.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.238+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c8158730 con 0x7fa7c80ad270 2024-09-16T07:33:17.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.238+0000 7fa7c4c00640 1 -- 172.21.15.71:0/1468942509 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8010690 con 0x7fa7c80ad270 2024-09-16T07:33:17.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.238+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c8158cf0 con 0x7fa7c80ad270 2024-09-16T07:33:17.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.239+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c806f9f0 con 0x7fa7c80ad270 2024-09-16T07:33:17.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.240+0000 7fa7c4c00640 1 -- 172.21.15.71:0/1468942509 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fa7b8010830 con 0x7fa7c80ad270 2024-09-16T07:33:17.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.241+0000 7fa7c4c00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7a8079e40 0x7fa7a807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:17.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.241+0000 7fa7c4c00640 1 -- 172.21.15.71:0/1468942509 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7fa7b809a360 con 0x7fa7c80ad270 2024-09-16T07:33:17.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.241+0000 7fa7c7e00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7a8079e40 0x7fa7a807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:17.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.242+0000 7fa7c7e00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7a8079e40 0x7fa7a807c300 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4033000 tx=0x7fa7b4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:17.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.246+0000 7fa7c4c00640 1 -- 172.21.15.71:0/1468942509 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa7b80639c0 con 0x7fa7c80ad270 2024-09-16T07:33:17.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.542+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7c80b4a50 con 0x7fa7c80ad270 2024-09-16T07:33:17.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.543+0000 7fa7c4c00640 1 -- 172.21.15.71:0/1468942509 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7fa7b80683f0 con 0x7fa7c80ad270 2024-09-16T07:33:17.543 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:17.543 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"btime":"2024-09-16T07:31:18:228480+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1642949519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1642949519},{"type":"v1","addr":"172.21.15.177:6836","nonce":1642949519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/2594338176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2594338176},{"type":"v1","addr":"172.21.15.71:6834","nonce":2594338176}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/482130020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":482130020},{"type":"v1","addr":"172.21.15.71:6835","nonce":482130020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:31:09.520327+0000","modified":"2024-09-16T07:31:12.617058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/4183855149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":4183855149},{"type":"v1","addr":"172.21.15.177:6837","nonce":4183855149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":6}]} 2024-09-16T07:33:17.544 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 57 2024-09-16T07:33:17.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7a8079e40 msgr2=0x7fa7a807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7a8079e40 0x7fa7a807c300 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4033000 tx=0x7fa7b4004730 comp rx=0 tx=0).stop 2024-09-16T07:33:17.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80ad270 msgr2=0x7fa7c81f5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80ad270 0x7fa7c81f5500 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800e6d0 tx=0x7fa7b800eb90 comp rx=0 tx=0).stop 2024-09-16T07:33:17.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 shutdown_connections 2024-09-16T07:33:17.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7a8079e40 0x7fa7a807c300 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c815f5a0 0x7fa7c80d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80ccf00 0x7fa7c8157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 --2- 172.21.15.71:0/1468942509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80ad270 0x7fa7c81f5500 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.547+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 >> 172.21.15.71:0/1468942509 conn(0x7fa7c80b1950 msgr2=0x7fa7c8073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:17.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.548+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 shutdown_connections 2024-09-16T07:33:17.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.548+0000 7fa7cfa00640 1 -- 172.21.15.71:0/1468942509 wait complete. 2024-09-16T07:33:17.565 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:33:17.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 -- 172.21.15.71:0/1315079301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 msgr2=0x7f8d1406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 --2- 172.21.15.71:0/1315079301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 0x7f8d1406d000 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8d140763d0 tx=0x7f8d0002a560 comp rx=0 tx=0).stop 2024-09-16T07:33:17.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 -- 172.21.15.71:0/1315079301 shutdown_connections 2024-09-16T07:33:17.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 --2- 172.21.15.71:0/1315079301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d140730c0 0x7f8d140734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 --2- 172.21.15.71:0/1315079301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 0x7f8d1406d000 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 --2- 172.21.15.71:0/1315079301 >> v2:172.21.15.177:3300/0 conn(0x7f8d1413efa0 0x7f8d140cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 -- 172.21.15.71:0/1315079301 >> 172.21.15.71:0/1315079301 conn(0x7f8d140b2230 msgr2=0x7f8d140b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:17.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 -- 172.21.15.71:0/1315079301 shutdown_connections 2024-09-16T07:33:17.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.722+0000 7f8d1a600640 1 -- 172.21.15.71:0/1315079301 wait complete. 2024-09-16T07:33:17.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.724+0000 7f8d1a600640 1 Processor -- start 2024-09-16T07:33:17.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.724+0000 7f8d1a600640 1 -- start start 2024-09-16T07:33:17.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d1a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 0x7f8d140a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:17.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d1a600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d140730c0 0x7f8d141f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:17.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d1a600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d1413efa0 0x7f8d140aa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:17.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d1a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8d140cd890 con 0x7f8d1406cb70 2024-09-16T07:33:17.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d1a600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d140cd710 con 0x7f8d1413efa0 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d1a600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8d14073a30 con 0x7f8d140730c0 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 0x7f8d140a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 0x7f8d140a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40546/0 (socket says 172.21.15.71:40546) 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13400640 1 -- 172.21.15.71:0/3601991760 learned_addr learned my addr 172.21.15.71:0/3601991760 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13e00640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d1413efa0 0x7f8d140aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d12a00640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d140730c0 0x7f8d141f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13400640 1 -- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d140730c0 msgr2=0x7f8d141f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13400640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d140730c0 0x7f8d141f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13400640 1 -- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d1413efa0 msgr2=0x7f8d140aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13400640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d1413efa0 0x7f8d140aa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d13400640 1 -- 172.21.15.71:0/3601991760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d140a9a40 con 0x7f8d1406cb70 2024-09-16T07:33:17.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.725+0000 7f8d12a00640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d140730c0 0x7f8d141f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:17.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.726+0000 7f8d13400640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 0x7f8d140a9460 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ea50 tx=0x7f8d0400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:17.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.726+0000 7f8d0b400640 1 -- 172.21.15.71:0/3601991760 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400cdc0 con 0x7f8d1406cb70 2024-09-16T07:33:17.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.726+0000 7f8d0b400640 1 -- 172.21.15.71:0/3601991760 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d04011560 con 0x7f8d1406cb70 2024-09-16T07:33:17.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.726+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d140a9d20 con 0x7f8d1406cb70 2024-09-16T07:33:17.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.726+0000 7f8d0b400640 1 -- 172.21.15.71:0/3601991760 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04011970 con 0x7f8d1406cb70 2024-09-16T07:33:17.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.726+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d140aa2e0 con 0x7f8d1406cb70 2024-09-16T07:33:17.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.727+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d140734f0 con 0x7f8d1406cb70 2024-09-16T07:33:17.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.728+0000 7f8d0b400640 1 -- 172.21.15.71:0/3601991760 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f8d04011b10 con 0x7f8d1406cb70 2024-09-16T07:33:17.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.729+0000 7f8d0b400640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cf4079e40 0x7f8cf407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:17.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.730+0000 7f8d0b400640 1 -- 172.21.15.71:0/3601991760 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f8d0409a690 con 0x7f8d1406cb70 2024-09-16T07:33:17.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.730+0000 7f8d12a00640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cf4079e40 0x7f8cf407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:17.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.731+0000 7f8d12a00640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cf4079e40 0x7f8cf407c300 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8d00033000 tx=0x7f8d00004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:17.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.734+0000 7f8d0b400640 1 -- 172.21.15.71:0/3601991760 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8d04016030 con 0x7f8d1406cb70 2024-09-16T07:33:17.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.954+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d1415a4f0 con 0x7f8d1406cb70 2024-09-16T07:33:17.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.954+0000 7f8d0b400640 1 -- 172.21.15.71:0/3601991760 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7f8d04064340 con 0x7f8d1406cb70 2024-09-16T07:33:17.954 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:17.954 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"btime":"2024-09-16T07:31:18:228480+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1642949519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1642949519},{"type":"v1","addr":"172.21.15.177:6836","nonce":1642949519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/2594338176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2594338176},{"type":"v1","addr":"172.21.15.71:6834","nonce":2594338176}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/482130020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":482130020},{"type":"v1","addr":"172.21.15.71:6835","nonce":482130020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:31:09.520327+0000","modified":"2024-09-16T07:31:12.617058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/4183855149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":4183855149},{"type":"v1","addr":"172.21.15.177:6837","nonce":4183855149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":6}]} 2024-09-16T07:33:17.954 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 57 2024-09-16T07:33:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cf4079e40 msgr2=0x7f8cf407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cf4079e40 0x7f8cf407c300 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8d00033000 tx=0x7f8d00004730 comp rx=0 tx=0).stop 2024-09-16T07:33:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 msgr2=0x7f8d140a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 0x7f8d140a9460 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ea50 tx=0x7f8d0400ef10 comp rx=0 tx=0).stop 2024-09-16T07:33:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 shutdown_connections 2024-09-16T07:33:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cf4079e40 0x7f8cf407c300 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d1413efa0 0x7f8d140aa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d140730c0 0x7f8d141f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 --2- 172.21.15.71:0/3601991760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d1406cb70 0x7f8d140a9460 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.956+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 >> 172.21.15.71:0/3601991760 conn(0x7f8d140b2230 msgr2=0x7f8d1415ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.957+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 shutdown_connections 2024-09-16T07:33:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:17.957+0000 7f8d1a600640 1 -- 172.21.15.71:0/3601991760 wait complete. 2024-09-16T07:33:17.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:33:18.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.129+0000 7fa446000640 1 -- 172.21.15.71:0/3031547752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4401612d0 msgr2=0x7fa4400b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.129+0000 7fa446000640 1 --2- 172.21.15.71:0/3031547752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4401612d0 0x7fa4400b4e00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa4400d1870 tx=0x7fa43402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.129+0000 7fa446000640 1 -- 172.21.15.71:0/3031547752 shutdown_connections 2024-09-16T07:33:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.129+0000 7fa446000640 1 --2- 172.21.15.71:0/3031547752 >> v2:172.21.15.177:3300/0 conn(0x7fa4401616b0 0x7fa4400d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.129+0000 7fa446000640 1 --2- 172.21.15.71:0/3031547752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4401612d0 0x7fa4400b4e00 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.129+0000 7fa446000640 1 -- 172.21.15.71:0/3031547752 >> 172.21.15.71:0/3031547752 conn(0x7fa4400d3df0 msgr2=0x7fa4400d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.130+0000 7fa446000640 1 -- 172.21.15.71:0/3031547752 shutdown_connections 2024-09-16T07:33:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.130+0000 7fa446000640 1 -- 172.21.15.71:0/3031547752 wait complete. 2024-09-16T07:33:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.131+0000 7fa446000640 1 Processor -- start 2024-09-16T07:33:18.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.131+0000 7fa446000640 1 -- start start 2024-09-16T07:33:18.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.132+0000 7fa446000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4400d4980 0x7fa4400b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.132+0000 7fa446000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4401612d0 0x7fa4400cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.132+0000 7fa446000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4401616b0 0x7fa4400cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.132+0000 7fa446000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa4400b1b60 con 0x7fa4400d4980 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.132+0000 7fa446000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa4400b19e0 con 0x7fa4401612d0 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.132+0000 7fa446000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa4400b1ce0 con 0x7fa4401616b0 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4400d4980 0x7fa4400b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4400d4980 0x7fa4400b8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40576/0 (socket says 172.21.15.71:40576) 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 -- 172.21.15.71:0/1310719721 learned_addr learned my addr 172.21.15.71:0/1310719721 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa43fe00640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4401612d0 0x7fa4400cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa445600640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4401616b0 0x7fa4400cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 -- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4401616b0 msgr2=0x7fa4400cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4401616b0 0x7fa4400cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 -- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4401612d0 msgr2=0x7fa4400cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4401612d0 0x7fa4400cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 -- 172.21.15.71:0/1310719721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4400cdaf0 con 0x7fa4400d4980 2024-09-16T07:33:18.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.133+0000 7fa444c00640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4400d4980 0x7fa4400b8ec0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa434009850 tx=0x7fa434003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.134+0000 7fa43cc00640 1 -- 172.21.15.71:0/1310719721 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43403f070 con 0x7fa4400d4980 2024-09-16T07:33:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.134+0000 7fa43cc00640 1 -- 172.21.15.71:0/1310719721 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4340047a0 con 0x7fa4400d4980 2024-09-16T07:33:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.134+0000 7fa43cc00640 1 -- 172.21.15.71:0/1310719721 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434035030 con 0x7fa4400d4980 2024-09-16T07:33:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.134+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4400c8b90 con 0x7fa4400d4980 2024-09-16T07:33:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.134+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4400c9150 con 0x7fa4400d4980 2024-09-16T07:33:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.135+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4400b4e00 con 0x7fa4400d4980 2024-09-16T07:33:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.136+0000 7fa43cc00640 1 -- 172.21.15.71:0/1310719721 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fa434042070 con 0x7fa4400d4980 2024-09-16T07:33:18.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.137+0000 7fa43cc00640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa420079e40 0x7fa42007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.137+0000 7fa43cc00640 1 -- 172.21.15.71:0/1310719721 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7fa4340ba8b0 con 0x7fa4400d4980 2024-09-16T07:33:18.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.137+0000 7fa43fe00640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa420079e40 0x7fa42007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.138+0000 7fa43fe00640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa420079e40 0x7fa42007c300 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa4400b2f00 tx=0x7fa42c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:18.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.141+0000 7fa43cc00640 1 -- 172.21.15.71:0/1310719721 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa4400d2cc0 con 0x7fa4400d4980 2024-09-16T07:33:18.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.333+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4400d0a50 con 0x7fa4400d4980 2024-09-16T07:33:18.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.334+0000 7fa43cc00640 1 -- 172.21.15.71:0/1310719721 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16853 (secure 0 0 0) 0x7fa4340845c0 con 0x7fa4400d4980 2024-09-16T07:33:18.334 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:18.334 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":94,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:33:18.221307+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:31:07.265563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T07:31:08.483774+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:33:18.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa420079e40 msgr2=0x7fa42007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa420079e40 0x7fa42007c300 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa4400b2f00 tx=0x7fa42c009040 comp rx=0 tx=0).stop 2024-09-16T07:33:18.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4400d4980 msgr2=0x7fa4400b8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4400d4980 0x7fa4400b8ec0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa434009850 tx=0x7fa434003910 comp rx=0 tx=0).stop 2024-09-16T07:33:18.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 shutdown_connections 2024-09-16T07:33:18.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa420079e40 0x7fa42007c300 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4401616b0 0x7fa4400cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4401612d0 0x7fa4400cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 --2- 172.21.15.71:0/1310719721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4400d4980 0x7fa4400b8ec0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.336+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 >> 172.21.15.71:0/1310719721 conn(0x7fa4400d3df0 msgr2=0x7fa4401605d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:18.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.337+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 shutdown_connections 2024-09-16T07:33:18.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.337+0000 7fa446000640 1 -- 172.21.15.71:0/1310719721 wait complete. 2024-09-16T07:33:18.352 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:33:18.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:33:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.510+0000 7fe67c800640 1 -- 172.21.15.71:0/1132784965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe67415f5a0 msgr2=0x7fe6740b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.510+0000 7fe67c800640 1 --2- 172.21.15.71:0/1132784965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe67415f5a0 0x7fe6740b2440 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe668005fd0 tx=0x7fe66802a540 comp rx=0 tx=0).stop 2024-09-16T07:33:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.510+0000 7fe67c800640 1 -- 172.21.15.71:0/1132784965 shutdown_connections 2024-09-16T07:33:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.510+0000 7fe67c800640 1 --2- 172.21.15.71:0/1132784965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6740ccf00 0x7fe6740acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.510+0000 7fe67c800640 1 --2- 172.21.15.71:0/1132784965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe67415f5a0 0x7fe6740b2440 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.510+0000 7fe67c800640 1 -- 172.21.15.71:0/1132784965 >> 172.21.15.71:0/1132784965 conn(0x7fe6740b1950 msgr2=0x7fe6740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:18.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.510+0000 7fe67c800640 1 -- 172.21.15.71:0/1132784965 shutdown_connections 2024-09-16T07:33:18.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.510+0000 7fe67c800640 1 -- 172.21.15.71:0/1132784965 wait complete. 2024-09-16T07:33:18.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.512+0000 7fe67c800640 1 Processor -- start 2024-09-16T07:33:18.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.512+0000 7fe67c800640 1 -- start start 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe67c800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6740ad270 0x7fe674152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe67c800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740ccf00 0x7fe6741f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe67c800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe67415f5a0 0x7fe6741461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe67c800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe67406f2e0 con 0x7fe6740ccf00 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe67c800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe67406f160 con 0x7fe6740ad270 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe67c800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe67406f460 con 0x7fe67415f5a0 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe679400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740ccf00 0x7fe6741f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe679400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740ccf00 0x7fe6741f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40602/0 (socket says 172.21.15.71:40602) 2024-09-16T07:33:18.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.513+0000 7fe679400640 1 -- 172.21.15.71:0/4129358740 learned_addr learned my addr 172.21.15.71:0/4129358740 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe679e00640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6740ad270 0x7fe674152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe67a800640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe67415f5a0 0x7fe6741461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe679400640 1 -- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe67415f5a0 msgr2=0x7fe6741461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe679400640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe67415f5a0 0x7fe6741461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe679400640 1 -- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6740ad270 msgr2=0x7fe674152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe679400640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6740ad270 0x7fe674152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe679400640 1 -- 172.21.15.71:0/4129358740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6740b2df0 con 0x7fe6740ccf00 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe67a800640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe67415f5a0 0x7fe6741461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe679400640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740ccf00 0x7fe6741f80f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9f0 tx=0x7fe66400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe66e000640 1 -- 172.21.15.71:0/4129358740 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400cd80 con 0x7fe6740ccf00 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe66e000640 1 -- 172.21.15.71:0/4129358740 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe664010430 con 0x7fe6740ccf00 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe66e000640 1 -- 172.21.15.71:0/4129358740 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664010840 con 0x7fe6740ccf00 2024-09-16T07:33:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe674153880 con 0x7fe6740ccf00 2024-09-16T07:33:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.514+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe674153de0 con 0x7fe6740ccf00 2024-09-16T07:33:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.516+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67406f9f0 con 0x7fe6740ccf00 2024-09-16T07:33:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.516+0000 7fe66e000640 1 -- 172.21.15.71:0/4129358740 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fe6640109e0 con 0x7fe6740ccf00 2024-09-16T07:33:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.517+0000 7fe66e000640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe658079e40 0x7fe65807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.517+0000 7fe66e000640 1 -- 172.21.15.71:0/4129358740 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5765+0+0 (secure 0 0 0) 0x7fe66409a680 con 0x7fe6740ccf00 2024-09-16T07:33:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.518+0000 7fe679e00640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe658079e40 0x7fe65807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.518+0000 7fe679e00640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe658079e40 0x7fe65807c300 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe668033000 tx=0x7fe668004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:18.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.522+0000 7fe66e000640 1 -- 172.21.15.71:0/4129358740 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe674153880 con 0x7fe6740ccf00 2024-09-16T07:33:18.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.747+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe67409b830 con 0x7fe6740ccf00 2024-09-16T07:33:18.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.748+0000 7fe66e000640 1 -- 172.21.15.71:0/4129358740 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7fe6640643f0 con 0x7fe6740ccf00 2024-09-16T07:33:18.748 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:18.748 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"btime":"2024-09-16T07:31:18:228480+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1642949519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1642949519},{"type":"v1","addr":"172.21.15.177:6836","nonce":1642949519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/2594338176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2594338176},{"type":"v1","addr":"172.21.15.71:6834","nonce":2594338176}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/482130020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":482130020},{"type":"v1","addr":"172.21.15.71:6835","nonce":482130020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:31:09.520327+0000","modified":"2024-09-16T07:31:12.617058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/4183855149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":4183855149},{"type":"v1","addr":"172.21.15.177:6837","nonce":4183855149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":6}]} 2024-09-16T07:33:18.748 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 57 2024-09-16T07:33:18.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe658079e40 msgr2=0x7fe65807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe658079e40 0x7fe65807c300 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe668033000 tx=0x7fe668004730 comp rx=0 tx=0).stop 2024-09-16T07:33:18.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740ccf00 msgr2=0x7fe6741f80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740ccf00 0x7fe6741f80f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9f0 tx=0x7fe66400eeb0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 shutdown_connections 2024-09-16T07:33:18.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe658079e40 0x7fe65807c300 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe67415f5a0 0x7fe6741461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740ccf00 0x7fe6741f80f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 --2- 172.21.15.71:0/4129358740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6740ad270 0x7fe674152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.750+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 >> 172.21.15.71:0/4129358740 conn(0x7fe6740b1950 msgr2=0x7fe6740af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:18.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.751+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 shutdown_connections 2024-09-16T07:33:18.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.751+0000 7fe67c800640 1 -- 172.21.15.71:0/4129358740 wait complete. 2024-09-16T07:33:18.766 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:33:18.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.925+0000 7f774bc00640 1 -- 172.21.15.71:0/2177960939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f774415f5a0 msgr2=0x7f77440b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.925+0000 7f774bc00640 1 --2- 172.21.15.71:0/2177960939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f774415f5a0 0x7f77440b2440 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7744076720 tx=0x7f773402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.925+0000 7f774bc00640 1 -- 172.21.15.71:0/2177960939 shutdown_connections 2024-09-16T07:33:18.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.925+0000 7f774bc00640 1 --2- 172.21.15.71:0/2177960939 >> v2:172.21.15.177:3300/0 conn(0x7f77440ad270 0x7f7744073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.925+0000 7f774bc00640 1 --2- 172.21.15.71:0/2177960939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77440ccf00 0x7f77440acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.925+0000 7f774bc00640 1 --2- 172.21.15.71:0/2177960939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f774415f5a0 0x7f77440b2440 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.925+0000 7f774bc00640 1 -- 172.21.15.71:0/2177960939 >> 172.21.15.71:0/2177960939 conn(0x7f77440b1950 msgr2=0x7f77440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:18.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.926+0000 7f774bc00640 1 -- 172.21.15.71:0/2177960939 shutdown_connections 2024-09-16T07:33:18.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.926+0000 7f774bc00640 1 -- 172.21.15.71:0/2177960939 wait complete. 2024-09-16T07:33:18.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.927+0000 7f774bc00640 1 Processor -- start 2024-09-16T07:33:18.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.927+0000 7f774bc00640 1 -- start start 2024-09-16T07:33:18.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.928+0000 7f774bc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77440ad270 0x7f7744150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.928+0000 7f774bc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77440ccf00 0x7f774414d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.928+0000 7f774bc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f774415f5a0 0x7f7744144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.928+0000 7f774bc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f774406f160 con 0x7f77440ad270 2024-09-16T07:33:18.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.928+0000 7f774bc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7744074780 con 0x7f774415f5a0 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.928+0000 7f774bc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f774406f2e0 con 0x7f77440ccf00 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77440ad270 0x7f7744150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77440ad270 0x7f7744150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40618/0 (socket says 172.21.15.71:40618) 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 -- 172.21.15.71:0/3762509152 learned_addr learned my addr 172.21.15.71:0/3762509152 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 -- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77440ccf00 msgr2=0x7f774414d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77440ccf00 0x7f774414d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 -- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f774415f5a0 msgr2=0x7f7744144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f774415f5a0 0x7f7744144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:18.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 -- 172.21.15.71:0/3762509152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77441f6580 con 0x7f77440ad270 2024-09-16T07:33:18.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7749200640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77440ad270 0x7f7744150220 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f7734034040 tx=0x7f77340039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:18.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.929+0000 7f7741600640 1 -- 172.21.15.71:0/3762509152 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773403e030 con 0x7f77440ad270 2024-09-16T07:33:18.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.930+0000 7f7741600640 1 -- 172.21.15.71:0/3762509152 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77340047a0 con 0x7f77440ad270 2024-09-16T07:33:18.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.930+0000 7f7741600640 1 -- 172.21.15.71:0/3762509152 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734035040 con 0x7f77440ad270 2024-09-16T07:33:18.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.930+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77441f6860 con 0x7f77440ad270 2024-09-16T07:33:18.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.930+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77441f6e20 con 0x7f77440ad270 2024-09-16T07:33:18.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.931+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774406f870 con 0x7f77440ad270 2024-09-16T07:33:18.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.932+0000 7f7741600640 1 -- 172.21.15.71:0/3762509152 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f7734041030 con 0x7f77440ad270 2024-09-16T07:33:18.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.933+0000 7f7741600640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7720079e40 0x7f772007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:18.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.933+0000 7f7741600640 1 -- 172.21.15.71:0/3762509152 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5765+0+0 (secure 0 0 0) 0x7f77340ba110 con 0x7f77440ad270 2024-09-16T07:33:18.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.933+0000 7f773be00640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7720079e40 0x7f772007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:18.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.934+0000 7f773be00640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7720079e40 0x7f772007c300 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f772c005fd0 tx=0x7f772c005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:18.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:18.937+0000 7f7741600640 1 -- 172.21.15.71:0/3762509152 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7734083dc0 con 0x7f77440ad270 2024-09-16T07:33:19.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.155+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77440afd10 con 0x7f77440ad270 2024-09-16T07:33:19.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.155+0000 7f7741600640 1 -- 172.21.15.71:0/3762509152 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5228 (secure 0 0 0) 0x7f77340836c0 con 0x7f77440ad270 2024-09-16T07:33:19.155 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:19.156 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"btime":"2024-09-16T07:31:18:228480+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1642949519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1642949519},{"type":"v1","addr":"172.21.15.177:6836","nonce":1642949519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/2594338176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2594338176},{"type":"v1","addr":"172.21.15.71:6834","nonce":2594338176}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/482130020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":482130020},{"type":"v1","addr":"172.21.15.71:6835","nonce":482130020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:31:09.520327+0000","modified":"2024-09-16T07:31:12.617058+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/4183855149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":4183855149},{"type":"v1","addr":"172.21.15.177:6837","nonce":4183855149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":6}]} 2024-09-16T07:33:19.156 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 57 2024-09-16T07:33:19.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.157+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7720079e40 msgr2=0x7f772007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.157+0000 7f774bc00640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7720079e40 0x7f772007c300 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f772c005fd0 tx=0x7f772c005850 comp rx=0 tx=0).stop 2024-09-16T07:33:19.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.157+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77440ad270 msgr2=0x7f7744150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.157+0000 7f774bc00640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77440ad270 0x7f7744150220 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f7734034040 tx=0x7f77340039e0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.157+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 shutdown_connections 2024-09-16T07:33:19.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.158+0000 7f774bc00640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7720079e40 0x7f772007c300 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.158+0000 7f774bc00640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f774415f5a0 0x7f7744144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.158+0000 7f774bc00640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77440ccf00 0x7f774414d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.158+0000 7f774bc00640 1 --2- 172.21.15.71:0/3762509152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77440ad270 0x7f7744150220 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.158+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 >> 172.21.15.71:0/3762509152 conn(0x7f77440b1950 msgr2=0x7f774415e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:19.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.158+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 shutdown_connections 2024-09-16T07:33:19.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.158+0000 7f774bc00640 1 -- 172.21.15.71:0/3762509152 wait complete. 2024-09-16T07:33:19.173 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:33:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.330+0000 7fb5f2000640 1 -- 172.21.15.71:0/2454430158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec161550 msgr2=0x7fb5ec06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.330+0000 7fb5f2000640 1 --2- 172.21.15.71:0/2454430158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec161550 0x7fb5ec06ef50 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc005fd0 tx=0x7fb5dc02a820 comp rx=0 tx=0).stop 2024-09-16T07:33:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.330+0000 7fb5f2000640 1 -- 172.21.15.71:0/2454430158 shutdown_connections 2024-09-16T07:33:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.330+0000 7fb5f2000640 1 --2- 172.21.15.71:0/2454430158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec161550 0x7fb5ec06ef50 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.330+0000 7fb5f2000640 1 -- 172.21.15.71:0/2454430158 >> 172.21.15.71:0/2454430158 conn(0x7fb5ec0b1950 msgr2=0x7fb5ec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.330+0000 7fb5f2000640 1 -- 172.21.15.71:0/2454430158 shutdown_connections 2024-09-16T07:33:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.330+0000 7fb5f2000640 1 -- 172.21.15.71:0/2454430158 wait complete. 2024-09-16T07:33:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.332+0000 7fb5f2000640 1 Processor -- start 2024-09-16T07:33:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.332+0000 7fb5f2000640 1 -- start start 2024-09-16T07:33:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5f2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec072df0 0x7fb5ec0ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5f2000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5ec0af3d0 0x7fb5ec0af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5f2000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5ec161550 0x7fb5ec0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5f2000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec0739b0 con 0x7fb5ec072df0 2024-09-16T07:33:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5f2000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec06fdb0 con 0x7fb5ec0af3d0 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5f2000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ec073b30 con 0x7fb5ec161550 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec072df0 0x7fb5ec0ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec072df0 0x7fb5ec0ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40634/0 (socket says 172.21.15.71:40634) 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eb400640 1 -- 172.21.15.71:0/1637073601 learned_addr learned my addr 172.21.15.71:0/1637073601 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5ebe00640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5ec161550 0x7fb5ec0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eaa00640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5ec0af3d0 0x7fb5ec0af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eb400640 1 -- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5ec161550 msgr2=0x7fb5ec0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eb400640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5ec161550 0x7fb5ec0afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eb400640 1 -- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5ec0af3d0 msgr2=0x7fb5ec0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eb400640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5ec0af3d0 0x7fb5ec0af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.333+0000 7fb5eb400640 1 -- 172.21.15.71:0/1637073601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec1f48f0 con 0x7fb5ec072df0 2024-09-16T07:33:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.334+0000 7fb5eb400640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec072df0 0x7fb5ec0ca1f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc035040 tx=0x7fb5dc032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:19.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.334+0000 7fb5e3400640 1 -- 172.21.15.71:0/1637073601 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc037040 con 0x7fb5ec072df0 2024-09-16T07:33:19.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.334+0000 7fb5e3400640 1 -- 172.21.15.71:0/1637073601 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5dc042070 con 0x7fb5ec072df0 2024-09-16T07:33:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.334+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec1f8c50 con 0x7fb5ec072df0 2024-09-16T07:33:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.334+0000 7fb5e3400640 1 -- 172.21.15.71:0/1637073601 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0042a0 con 0x7fb5ec072df0 2024-09-16T07:33:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.334+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1f5170 con 0x7fb5ec072df0 2024-09-16T07:33:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.335+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ec074040 con 0x7fb5ec072df0 2024-09-16T07:33:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.336+0000 7fb5e3400640 1 -- 172.21.15.71:0/1637073601 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fb5dc004440 con 0x7fb5ec072df0 2024-09-16T07:33:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.337+0000 7fb5e3400640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5cc079e40 0x7fb5cc07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:19.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.337+0000 7fb5e3400640 1 -- 172.21.15.71:0/1637073601 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5765+0+0 (secure 0 0 0) 0x7fb5dc0ba8a0 con 0x7fb5ec072df0 2024-09-16T07:33:19.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.338+0000 7fb5eaa00640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5cc079e40 0x7fb5cc07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:19.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.339+0000 7fb5eaa00640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5cc079e40 0x7fb5cc07c300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec157730 tx=0x7fb5d8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:19.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.341+0000 7fb5e3400640 1 -- 172.21.15.71:0/1637073601 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5dc0844d0 con 0x7fb5ec072df0 2024-09-16T07:33:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.526+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5ec15d4f0 con 0x7fb5ec072df0 2024-09-16T07:33:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.528+0000 7fb5e3400640 1 -- 172.21.15.71:0/1637073601 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16853 (secure 0 0 0) 0x7fb5dc083dd0 con 0x7fb5ec072df0 2024-09-16T07:33:19.527 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:19.528 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":94,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:33:18.221307+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:31:07.265563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T07:31:08.483774+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:33:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5cc079e40 msgr2=0x7fb5cc07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5cc079e40 0x7fb5cc07c300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec157730 tx=0x7fb5d8009040 comp rx=0 tx=0).stop 2024-09-16T07:33:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec072df0 msgr2=0x7fb5ec0ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec072df0 0x7fb5ec0ca1f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc035040 tx=0x7fb5dc032c90 comp rx=0 tx=0).stop 2024-09-16T07:33:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 shutdown_connections 2024-09-16T07:33:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb5cc079e40 0x7fb5cc07c300 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5ec161550 0x7fb5ec0afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5ec0af3d0 0x7fb5ec0af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 --2- 172.21.15.71:0/1637073601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5ec072df0 0x7fb5ec0ca1f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 >> 172.21.15.71:0/1637073601 conn(0x7fb5ec0b1950 msgr2=0x7fb5ec0ad740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:19.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 shutdown_connections 2024-09-16T07:33:19.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.530+0000 7fb5f2000640 1 -- 172.21.15.71:0/1637073601 wait complete. 2024-09-16T07:33:19.546 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:33:19.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.706+0000 7f9d4fe00640 1 -- 172.21.15.71:0/3400994085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d4815f5a0 msgr2=0x7f9d480b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.706+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/3400994085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d4815f5a0 0x7f9d480b2440 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009830 tx=0x7f9d3c02a480 comp rx=0 tx=0).stop 2024-09-16T07:33:19.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.706+0000 7f9d4fe00640 1 -- 172.21.15.71:0/3400994085 shutdown_connections 2024-09-16T07:33:19.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.706+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/3400994085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d480ccf00 0x7f9d480acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.706+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/3400994085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d4815f5a0 0x7f9d480b2440 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.706+0000 7f9d4fe00640 1 -- 172.21.15.71:0/3400994085 >> 172.21.15.71:0/3400994085 conn(0x7f9d480b1950 msgr2=0x7f9d480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:19.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.706+0000 7f9d4fe00640 1 -- 172.21.15.71:0/3400994085 shutdown_connections 2024-09-16T07:33:19.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.706+0000 7f9d4fe00640 1 -- 172.21.15.71:0/3400994085 wait complete. 2024-09-16T07:33:19.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.708+0000 7f9d4fe00640 1 Processor -- start 2024-09-16T07:33:19.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.708+0000 7f9d4fe00640 1 -- start start 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d480ad270 0x7f9d481f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d480ccf00 0x7f9d480b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d4815f5a0 0x7f9d480b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4806f2e0 con 0x7f9d480ad270 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4806f160 con 0x7f9d480ccf00 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9d4806f460 con 0x7f9d4815f5a0 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d480ad270 0x7f9d481f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d480ad270 0x7f9d481f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40652/0 (socket says 172.21.15.71:40652) 2024-09-16T07:33:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.709+0000 7f9d4d400640 1 -- 172.21.15.71:0/623075346 learned_addr learned my addr 172.21.15.71:0/623075346 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4ca00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d480ccf00 0x7f9d480b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4de00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d4815f5a0 0x7f9d480b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4d400640 1 -- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d4815f5a0 msgr2=0x7f9d480b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4d400640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d4815f5a0 0x7f9d480b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4d400640 1 -- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d480ccf00 msgr2=0x7f9d480b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4d400640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d480ccf00 0x7f9d480b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4d400640 1 -- 172.21.15.71:0/623075346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d480ccc60 con 0x7f9d480ad270 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4d400640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d480ad270 0x7f9d481f3d90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c000c00 tx=0x7f9d3c033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d45600640 1 -- 172.21.15.71:0/623075346 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c005c90 con 0x7f9d480ad270 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d45600640 1 -- 172.21.15.71:0/623075346 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d3c031e50 con 0x7f9d480ad270 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d45600640 1 -- 172.21.15.71:0/623075346 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0332b0 con 0x7f9d480ad270 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.710+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d480b3e00 con 0x7f9d480ad270 2024-09-16T07:33:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.711+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d480b43c0 con 0x7f9d480ad270 2024-09-16T07:33:19.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.712+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4806f9f0 con 0x7f9d480ad270 2024-09-16T07:33:19.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.713+0000 7f9d45600640 1 -- 172.21.15.71:0/623075346 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f9d3c033450 con 0x7f9d480ad270 2024-09-16T07:33:19.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.714+0000 7f9d45600640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d28079e40 0x7f9d2807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:19.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.714+0000 7f9d45600640 1 -- 172.21.15.71:0/623075346 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5765+0+0 (secure 0 0 0) 0x7f9d3c0badd0 con 0x7f9d480ad270 2024-09-16T07:33:19.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.714+0000 7f9d4ca00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d28079e40 0x7f9d2807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:19.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.715+0000 7f9d4ca00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d28079e40 0x7f9d2807c300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9d34005fd0 tx=0x7f9d34005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.717+0000 7f9d45600640 1 -- 172.21.15.71:0/623075346 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d3c03f030 con 0x7f9d480ad270 2024-09-16T07:33:19.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.938+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9d48069ac0 con 0x7f9d480ad270 2024-09-16T07:33:19.938 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:33:19.939+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:33:19.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.941+0000 7f9d45600640 1 -- 172.21.15.71:0/623075346 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7f9d3c084a80 con 0x7f9d480ad270 2024-09-16T07:33:19.940 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:33:19.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d28079e40 msgr2=0x7f9d2807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d28079e40 0x7f9d2807c300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9d34005fd0 tx=0x7f9d34005850 comp rx=0 tx=0).stop 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d480ad270 msgr2=0x7f9d481f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d480ad270 0x7f9d481f3d90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c000c00 tx=0x7f9d3c033c90 comp rx=0 tx=0).stop 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 shutdown_connections 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9d28079e40 0x7f9d2807c300 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d4815f5a0 0x7f9d480b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d480ccf00 0x7f9d480b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.944+0000 7f9d4fe00640 1 --2- 172.21.15.71:0/623075346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d480ad270 0x7f9d481f3d90 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.945+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 >> 172.21.15.71:0/623075346 conn(0x7f9d480b1950 msgr2=0x7f9d4809af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.945+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 shutdown_connections 2024-09-16T07:33:19.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:19.945+0000 7f9d4fe00640 1 -- 172.21.15.71:0/623075346 wait complete. 2024-09-16T07:33:19.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:33:20.014 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:33:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.120+0000 7fa53ea00640 1 -- 172.21.15.71:0/1959829725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa538145750 msgr2=0x7fa538073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.120+0000 7fa53ea00640 1 --2- 172.21.15.71:0/1959829725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa538145750 0x7fa538073440 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa528005fd0 tx=0x7fa52802a820 comp rx=0 tx=0).stop 2024-09-16T07:33:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.120+0000 7fa53ea00640 1 -- 172.21.15.71:0/1959829725 shutdown_connections 2024-09-16T07:33:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.120+0000 7fa53ea00640 1 --2- 172.21.15.71:0/1959829725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa53806efa0 0x7fa5380cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.120+0000 7fa53ea00640 1 --2- 172.21.15.71:0/1959829725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa538145750 0x7fa538073440 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.120+0000 7fa53ea00640 1 -- 172.21.15.71:0/1959829725 >> 172.21.15.71:0/1959829725 conn(0x7fa5380b1840 msgr2=0x7fa5380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.120+0000 7fa53ea00640 1 -- 172.21.15.71:0/1959829725 shutdown_connections 2024-09-16T07:33:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.120+0000 7fa53ea00640 1 -- 172.21.15.71:0/1959829725 wait complete. 2024-09-16T07:33:20.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.122+0000 7fa53ea00640 1 Processor -- start 2024-09-16T07:33:20.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.122+0000 7fa53ea00640 1 -- start start 2024-09-16T07:33:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa53ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53806efa0 0x7fa538141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa53ea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5380ccb00 0x7fa5380ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa53ea00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa538145750 0x7fa53815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa53ea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380ced40 con 0x7fa53806efa0 2024-09-16T07:33:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa53ea00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380cebc0 con 0x7fa5380ccb00 2024-09-16T07:33:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa53ea00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa5380ceec0 con 0x7fa538145750 2024-09-16T07:33:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53806efa0 0x7fa538141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53806efa0 0x7fa538141560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40664/0 (socket says 172.21.15.71:40664) 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537e00640 1 -- 172.21.15.71:0/3257284117 learned_addr learned my addr 172.21.15.71:0/3257284117 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537400640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5380ccb00 0x7fa5380ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa53ca00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa538145750 0x7fa53815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537e00640 1 -- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa538145750 msgr2=0x7fa53815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537e00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa538145750 0x7fa53815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537e00640 1 -- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5380ccb00 msgr2=0x7fa5380ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537e00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5380ccb00 0x7fa5380ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa537e00640 1 -- 172.21.15.71:0/3257284117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5381f5c50 con 0x7fa53806efa0 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.123+0000 7fa53ca00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa538145750 0x7fa53815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.124+0000 7fa537e00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53806efa0 0x7fa538141560 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa528009cd0 tx=0x7fa528032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:20.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.124+0000 7fa52fe00640 1 -- 172.21.15.71:0/3257284117 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528037040 con 0x7fa53806efa0 2024-09-16T07:33:20.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.124+0000 7fa52fe00640 1 -- 172.21.15.71:0/3257284117 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa528042070 con 0x7fa53806efa0 2024-09-16T07:33:20.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.124+0000 7fa52fe00640 1 -- 172.21.15.71:0/3257284117 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528004280 con 0x7fa53806efa0 2024-09-16T07:33:20.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.124+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5381f5ef0 con 0x7fa53806efa0 2024-09-16T07:33:20.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.124+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381f64b0 con 0x7fa53806efa0 2024-09-16T07:33:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.125+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5380cc5c0 con 0x7fa53806efa0 2024-09-16T07:33:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.126+0000 7fa52fe00640 1 -- 172.21.15.71:0/3257284117 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fa528004420 con 0x7fa53806efa0 2024-09-16T07:33:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.127+0000 7fa52fe00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa518079e70 0x7fa51807c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:20.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.127+0000 7fa52fe00640 1 -- 172.21.15.71:0/3257284117 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5823+0+0 (secure 0 0 0) 0x7fa5280ba6b0 con 0x7fa53806efa0 2024-09-16T07:33:20.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.128+0000 7fa537400640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa518079e70 0x7fa51807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:20.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.128+0000 7fa537400640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa518079e70 0x7fa51807c330 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa524009bd0 tx=0x7fa524005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:20.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.131+0000 7fa52fe00640 1 -- 172.21.15.71:0/3257284117 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa528084360 con 0x7fa53806efa0 2024-09-16T07:33:20.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.355+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa5380b0780 con 0x7fa53806efa0 2024-09-16T07:33:20.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.943+0000 7fa52fe00640 1 -- 172.21.15.71:0/3257284117 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7fa5280bab50 con 0x7fa53806efa0 2024-09-16T07:33:20.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa518079e70 msgr2=0x7fa51807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa518079e70 0x7fa51807c330 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa524009bd0 tx=0x7fa524005f90 comp rx=0 tx=0).stop 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53806efa0 msgr2=0x7fa538141560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53806efa0 0x7fa538141560 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa528009cd0 tx=0x7fa528032d10 comp rx=0 tx=0).stop 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 shutdown_connections 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa518079e70 0x7fa51807c330 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa538145750 0x7fa53815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5380ccb00 0x7fa5380ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 --2- 172.21.15.71:0/3257284117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa53806efa0 0x7fa538141560 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 >> 172.21.15.71:0/3257284117 conn(0x7fa5380b1840 msgr2=0x7fa5380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 shutdown_connections 2024-09-16T07:33:20.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:20.946+0000 7fa53ea00640 1 -- 172.21.15.71:0/3257284117 wait complete. 2024-09-16T07:33:20.964 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:33:21.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.127+0000 7f5b20e00640 1 -- 172.21.15.71:0/3010725296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 msgr2=0x7f5b1c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:21.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.127+0000 7f5b20e00640 1 --2- 172.21.15.71:0/3010725296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 0x7f5b1c06e6f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009830 tx=0x7f5b0402a5c0 comp rx=0 tx=0).stop 2024-09-16T07:33:21.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.128+0000 7f5b20e00640 1 -- 172.21.15.71:0/3010725296 shutdown_connections 2024-09-16T07:33:21.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.128+0000 7f5b20e00640 1 --2- 172.21.15.71:0/3010725296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 0x7f5b1c06e6f0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:21.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.128+0000 7f5b20e00640 1 -- 172.21.15.71:0/3010725296 >> 172.21.15.71:0/3010725296 conn(0x7f5b1c0b18e0 msgr2=0x7f5b1c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:21.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.128+0000 7f5b20e00640 1 -- 172.21.15.71:0/3010725296 shutdown_connections 2024-09-16T07:33:21.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.128+0000 7f5b20e00640 1 -- 172.21.15.71:0/3010725296 wait complete. 2024-09-16T07:33:21.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.129+0000 7f5b20e00640 1 Processor -- start 2024-09-16T07:33:21.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.129+0000 7f5b20e00640 1 -- start start 2024-09-16T07:33:21.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.130+0000 7f5b20e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b1c06f220 0x7f5b1c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:21.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.130+0000 7f5b20e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b1c159aa0 0x7f5b1c1f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.130+0000 7f5b20e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 0x7f5b1c09d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.130+0000 7f5b20e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c0ccc20 con 0x7f5b1c15cff0 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.130+0000 7f5b20e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c06f6b0 con 0x7f5b1c06f220 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.130+0000 7f5b20e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1c157d40 con 0x7f5b1c159aa0 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 0x7f5b1c09d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 0x7f5b1c09d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40670/0 (socket says 172.21.15.71:40670) 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 -- 172.21.15.71:0/2272039462 learned_addr learned my addr 172.21.15.71:0/2272039462 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b19e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b1c06f220 0x7f5b1c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b13e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b1c159aa0 0x7f5b1c1f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 -- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b1c159aa0 msgr2=0x7f5b1c1f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b1c159aa0 0x7f5b1c1f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:21.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 -- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b1c06f220 msgr2=0x7f5b1c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:21.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b1c06f220 0x7f5b1c0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:21.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 -- 172.21.15.71:0/2272039462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c153470 con 0x7f5b1c15cff0 2024-09-16T07:33:21.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b19e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b1c06f220 0x7f5b1c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:21.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b13e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b1c159aa0 0x7f5b1c1f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:21.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.131+0000 7f5b1a800640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 0x7f5b1c09d170 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c7a0 tx=0x7f5b1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:21.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.132+0000 7f5b12000640 1 -- 172.21.15.71:0/2272039462 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14003b70 con 0x7f5b1c15cff0 2024-09-16T07:33:21.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.132+0000 7f5b12000640 1 -- 172.21.15.71:0/2272039462 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b14003d10 con 0x7f5b1c15cff0 2024-09-16T07:33:21.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.132+0000 7f5b12000640 1 -- 172.21.15.71:0/2272039462 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400aab0 con 0x7f5b1c15cff0 2024-09-16T07:33:21.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.132+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c0cf530 con 0x7f5b1c15cff0 2024-09-16T07:33:21.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.132+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c1f3750 con 0x7f5b1c15cff0 2024-09-16T07:33:21.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.133+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1c069360 con 0x7f5b1c15cff0 2024-09-16T07:33:21.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.134+0000 7f5b12000640 1 -- 172.21.15.71:0/2272039462 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f5b1400ac50 con 0x7f5b1c15cff0 2024-09-16T07:33:21.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.135+0000 7f5b12000640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af8079e40 0x7f5af807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:21.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.135+0000 7f5b12000640 1 -- 172.21.15.71:0/2272039462 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5823+0+0 (secure 0 0 0) 0x7f5b14099900 con 0x7f5b1c15cff0 2024-09-16T07:33:21.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.135+0000 7f5b19e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af8079e40 0x7f5af807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:21.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.136+0000 7f5b19e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af8079e40 0x7f5af807c300 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c000f60 tx=0x7f5b0c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:21.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.139+0000 7f5b12000640 1 -- 172.21.15.71:0/2272039462 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5b14063610 con 0x7f5b1c15cff0 2024-09-16T07:33:21.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.323+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5b1c06e6f0 con 0x7f5b1c15cff0 2024-09-16T07:33:21.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.945+0000 7f5b12000640 1 -- 172.21.15.71:0/2272039462 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f5b14062f10 con 0x7f5b1c15cff0 2024-09-16T07:33:21.945 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:33:21.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af8079e40 msgr2=0x7f5af807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:21.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af8079e40 0x7f5af807c300 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c000f60 tx=0x7f5b0c009040 comp rx=0 tx=0).stop 2024-09-16T07:33:21.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 msgr2=0x7f5b1c09d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:21.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 0x7f5b1c09d170 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c7a0 tx=0x7f5b1400cc60 comp rx=0 tx=0).stop 2024-09-16T07:33:21.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 shutdown_connections 2024-09-16T07:33:21.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af8079e40 0x7f5af807c300 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:21.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1c15cff0 0x7f5b1c09d170 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:21.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b1c159aa0 0x7f5b1c1f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:21.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 --2- 172.21.15.71:0/2272039462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b1c06f220 0x7f5b1c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:21.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.949+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 >> 172.21.15.71:0/2272039462 conn(0x7f5b1c0b18e0 msgr2=0x7f5b1c15c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:21.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.950+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 shutdown_connections 2024-09-16T07:33:21.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:21.950+0000 7f5b20e00640 1 -- 172.21.15.71:0/2272039462 wait complete. 2024-09-16T07:33:21.967 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:33:22.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.134+0000 7f1d86400640 1 -- 172.21.15.71:0/3436542297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 msgr2=0x7f1d800d47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:22.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.134+0000 7f1d86400640 1 --2- 172.21.15.71:0/3436542297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 0x7f1d800d47e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1d8009d850 tx=0x7f1d7402a5f0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.134+0000 7f1d86400640 1 -- 172.21.15.71:0/3436542297 shutdown_connections 2024-09-16T07:33:22.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.134+0000 7f1d86400640 1 --2- 172.21.15.71:0/3436542297 >> v2:172.21.15.177:3300/0 conn(0x7f1d800d2920 0x7f1d800b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.134+0000 7f1d86400640 1 --2- 172.21.15.71:0/3436542297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 0x7f1d800d47e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.134+0000 7f1d86400640 1 -- 172.21.15.71:0/3436542297 >> 172.21.15.71:0/3436542297 conn(0x7f1d8009d180 msgr2=0x7f1d800d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:22.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.134+0000 7f1d86400640 1 -- 172.21.15.71:0/3436542297 shutdown_connections 2024-09-16T07:33:22.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.135+0000 7f1d86400640 1 -- 172.21.15.71:0/3436542297 wait complete. 2024-09-16T07:33:22.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.136+0000 7f1d86400640 1 Processor -- start 2024-09-16T07:33:22.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.136+0000 7f1d86400640 1 -- start start 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d86400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d800b5200 0x7f1d80153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d86400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d800d2920 0x7f1d800b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d86400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 0x7f1d800b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d86400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d800b2230 con 0x7f1d801623b0 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d86400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d800b20b0 con 0x7f1d800b5200 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d86400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1d800b23b0 con 0x7f1d800d2920 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d85a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 0x7f1d800b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d85a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 0x7f1d800b9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40688/0 (socket says 172.21.15.71:40688) 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.137+0000 7f1d85a00640 1 -- 172.21.15.71:0/1485395975 learned_addr learned my addr 172.21.15.71:0/1485395975 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d85a00640 1 -- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d800d2920 msgr2=0x7f1d800b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d85a00640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d800d2920 0x7f1d800b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d85a00640 1 -- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d800b5200 msgr2=0x7f1d80153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d85a00640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d800b5200 0x7f1d80153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d85a00640 1 -- 172.21.15.71:0/1485395975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8015f730 con 0x7f1d801623b0 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d85a00640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 0x7f1d800b9940 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800eaa0 tx=0x7f1d7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d7cc00640 1 -- 172.21.15.71:0/1485395975 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7800cdb0 con 0x7f1d801623b0 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d7cc00640 1 -- 172.21.15.71:0/1485395975 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d78004c10 con 0x7f1d801623b0 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8015fa10 con 0x7f1d801623b0 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d7cc00640 1 -- 172.21.15.71:0/1485395975 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78005020 con 0x7f1d801623b0 2024-09-16T07:33:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.138+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d800b7240 con 0x7f1d801623b0 2024-09-16T07:33:22.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.140+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8008d600 con 0x7f1d801623b0 2024-09-16T07:33:22.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.141+0000 7f1d7cc00640 1 -- 172.21.15.71:0/1485395975 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f1d780051c0 con 0x7f1d801623b0 2024-09-16T07:33:22.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.142+0000 7f1d7cc00640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d60079e40 0x7f1d6007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:22.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.142+0000 7f1d7cc00640 1 -- 172.21.15.71:0/1485395975 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5385+0+0 (secure 0 0 0) 0x7f1d7809a4c0 con 0x7f1d801623b0 2024-09-16T07:33:22.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.143+0000 7f1d85000640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d60079e40 0x7f1d6007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:22.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.143+0000 7f1d85000640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d60079e40 0x7f1d6007c300 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1d74033000 tx=0x7f1d740046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:22.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.146+0000 7f1d7cc00640 1 -- 172.21.15.71:0/1485395975 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1d78016030 con 0x7f1d801623b0 2024-09-16T07:33:22.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.335+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1d800b1940 con 0x7f1d801623b0 2024-09-16T07:33:22.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.953+0000 7f1d7cc00640 1 -- 172.21.15.71:0/1485395975 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f1d78064170 con 0x7f1d801623b0 2024-09-16T07:33:22.953 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d60079e40 msgr2=0x7f1d6007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d60079e40 0x7f1d6007c300 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1d74033000 tx=0x7f1d740046c0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 msgr2=0x7f1d800b9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 0x7f1d800b9940 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800eaa0 tx=0x7f1d7800ef60 comp rx=0 tx=0).stop 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 shutdown_connections 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d60079e40 0x7f1d6007c300 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801623b0 0x7f1d800b9940 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d800d2920 0x7f1d800b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 --2- 172.21.15.71:0/1485395975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d800b5200 0x7f1d80153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 >> 172.21.15.71:0/1485395975 conn(0x7f1d8009d180 msgr2=0x7f1d800b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:22.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 shutdown_connections 2024-09-16T07:33:22.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:22.957+0000 7f1d86400640 1 -- 172.21.15.71:0/1485395975 wait complete. 2024-09-16T07:33:22.983 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:33:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.145+0000 7fc6ee800640 1 -- 172.21.15.71:0/240109434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6cc003a00 msgr2=0x7fc6cc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.145+0000 7fc6ee800640 1 --2- 172.21.15.71:0/240109434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6cc003a00 0x7fc6cc003de0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80017c0 tx=0x7fc6cc005830 comp rx=0 tx=0).stop 2024-09-16T07:33:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.145+0000 7fc6ee800640 1 -- 172.21.15.71:0/240109434 shutdown_connections 2024-09-16T07:33:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.145+0000 7fc6ee800640 1 --2- 172.21.15.71:0/240109434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6cc003a00 0x7fc6cc003de0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.145+0000 7fc6ee800640 1 -- 172.21.15.71:0/240109434 >> 172.21.15.71:0/240109434 conn(0x7fc6e80b1950 msgr2=0x7fc6e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.145+0000 7fc6ee800640 1 -- 172.21.15.71:0/240109434 shutdown_connections 2024-09-16T07:33:23.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.145+0000 7fc6ee800640 1 -- 172.21.15.71:0/240109434 wait complete. 2024-09-16T07:33:23.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.147+0000 7fc6ee800640 1 Processor -- start 2024-09-16T07:33:23.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.147+0000 7fc6ee800640 1 -- start start 2024-09-16T07:33:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6ee800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc003a00 0x7fc6e814b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6ee800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e81efc80 0x7fc6e80c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6ee800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6e80c8180 0x7fc6e814d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6ee800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e80ad0c0 con 0x7fc6e81efc80 2024-09-16T07:33:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6ee800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e80acf40 con 0x7fc6e80c8180 2024-09-16T07:33:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6ee800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e80ad240 con 0x7fc6cc003a00 2024-09-16T07:33:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e81efc80 0x7fc6e80c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e81efc80 0x7fc6e80c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40712/0 (socket says 172.21.15.71:40712) 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6e6a00640 1 -- 172.21.15.71:0/1076119536 learned_addr learned my addr 172.21.15.71:0/1076119536 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6e7400640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc003a00 0x7fc6e814b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.148+0000 7fc6e7e00640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6e80c8180 0x7fc6e814d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6e6a00640 1 -- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc003a00 msgr2=0x7fc6e814b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6e6a00640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc003a00 0x7fc6e814b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6e6a00640 1 -- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6e80c8180 msgr2=0x7fc6e814d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6e6a00640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6e80c8180 0x7fc6e814d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6e6a00640 1 -- 172.21.15.71:0/1076119536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e815b2f0 con 0x7fc6e81efc80 2024-09-16T07:33:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6e6a00640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e81efc80 0x7fc6e80c7c40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4001860 tx=0x7fc6d400da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6df400640 1 -- 172.21.15.71:0/1076119536 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4018070 con 0x7fc6e81efc80 2024-09-16T07:33:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6df400640 1 -- 172.21.15.71:0/1076119536 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6d40052e0 con 0x7fc6e81efc80 2024-09-16T07:33:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6df400640 1 -- 172.21.15.71:0/1076119536 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4003850 con 0x7fc6e81efc80 2024-09-16T07:33:23.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e80c8610 con 0x7fc6e81efc80 2024-09-16T07:33:23.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.149+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e815b840 con 0x7fc6e81efc80 2024-09-16T07:33:23.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.150+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e8074000 con 0x7fc6e81efc80 2024-09-16T07:33:23.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.152+0000 7fc6df400640 1 -- 172.21.15.71:0/1076119536 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fc6d401b070 con 0x7fc6e81efc80 2024-09-16T07:33:23.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.153+0000 7fc6df400640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc6c8079e40 0x7fc6c807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:23.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.153+0000 7fc6df400640 1 -- 172.21.15.71:0/1076119536 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7fc6d4098c10 con 0x7fc6e81efc80 2024-09-16T07:33:23.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.153+0000 7fc6e7400640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc6c8079e40 0x7fc6c807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:23.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.154+0000 7fc6e7400640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc6c8079e40 0x7fc6c807c300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8032000 tx=0x7fc6d802c000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.157+0000 7fc6df400640 1 -- 172.21.15.71:0/1076119536 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc6d40990b0 con 0x7fc6e81efc80 2024-09-16T07:33:23.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.372+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fc6e80d0af0 con 0x7fc6e81efc80 2024-09-16T07:33:23.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.376+0000 7fc6df400640 1 -- 172.21.15.71:0/1076119536 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7fc6d4062680 con 0x7fc6e81efc80 2024-09-16T07:33:23.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.378+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc6c8079e40 msgr2=0x7fc6c807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.378+0000 7fc6ee800640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc6c8079e40 0x7fc6c807c300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8032000 tx=0x7fc6d802c000 comp rx=0 tx=0).stop 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e81efc80 msgr2=0x7fc6e80c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e81efc80 0x7fc6e80c7c40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4001860 tx=0x7fc6d400da00 comp rx=0 tx=0).stop 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 shutdown_connections 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc6c8079e40 0x7fc6c807c300 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6e80c8180 0x7fc6e814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e81efc80 0x7fc6e80c7c40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 --2- 172.21.15.71:0/1076119536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc003a00 0x7fc6e814b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 >> 172.21.15.71:0/1076119536 conn(0x7fc6e80b1950 msgr2=0x7fc6e80b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 shutdown_connections 2024-09-16T07:33:23.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.379+0000 7fc6ee800640 1 -- 172.21.15.71:0/1076119536 wait complete. 2024-09-16T07:33:23.394 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-16T07:33:23.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.553+0000 7f4893000640 1 -- 172.21.15.71:0/660649180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c15f5a0 msgr2=0x7f488c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.553+0000 7f4893000640 1 --2- 172.21.15.71:0/660649180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c15f5a0 0x7f488c0b2440 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f487c005fd0 tx=0x7f487c02a540 comp rx=0 tx=0).stop 2024-09-16T07:33:23.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.553+0000 7f4893000640 1 -- 172.21.15.71:0/660649180 shutdown_connections 2024-09-16T07:33:23.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.553+0000 7f4893000640 1 --2- 172.21.15.71:0/660649180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f488c073fb0 0x7f488c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.553+0000 7f4893000640 1 --2- 172.21.15.71:0/660649180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c15f5a0 0x7f488c0b2440 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.553+0000 7f4893000640 1 -- 172.21.15.71:0/660649180 >> 172.21.15.71:0/660649180 conn(0x7f488c0b1950 msgr2=0x7f488c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:23.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.553+0000 7f4893000640 1 -- 172.21.15.71:0/660649180 shutdown_connections 2024-09-16T07:33:23.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.553+0000 7f4893000640 1 -- 172.21.15.71:0/660649180 wait complete. 2024-09-16T07:33:23.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.554+0000 7f4893000640 1 Processor -- start 2024-09-16T07:33:23.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.554+0000 7f4893000640 1 -- start start 2024-09-16T07:33:23.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.555+0000 7f4893000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f488c073fb0 0x7f488c14d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:23.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.555+0000 7f4893000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c0ccf00 0x7f488c153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:23.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.555+0000 7f4893000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f488c15f5a0 0x7f488c144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.555+0000 7f4893000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f488c06f2e0 con 0x7f488c0ccf00 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.555+0000 7f4893000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f488c06f160 con 0x7f488c073fb0 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.555+0000 7f4893000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f488c06f460 con 0x7f488c15f5a0 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.555+0000 7f488b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c0ccf00 0x7f488c153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c0ccf00 0x7f488c153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40732/0 (socket says 172.21.15.71:40732) 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488b400640 1 -- 172.21.15.71:0/1575733073 learned_addr learned my addr 172.21.15.71:0/1575733073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488be00640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f488c073fb0 0x7f488c14d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f4891000640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f488c15f5a0 0x7f488c144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488b400640 1 -- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f488c15f5a0 msgr2=0x7f488c144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488b400640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f488c15f5a0 0x7f488c144210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488b400640 1 -- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f488c073fb0 msgr2=0x7f488c14d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488b400640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f488c073fb0 0x7f488c14d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488b400640 1 -- 172.21.15.71:0/1575733073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c14f310 con 0x7f488c0ccf00 2024-09-16T07:33:23.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f4891000640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f488c15f5a0 0x7f488c144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:23.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f488b400640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c0ccf00 0x7f488c153170 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f487800c450 tx=0x7f487800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:23.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f4883e00640 1 -- 172.21.15.71:0/1575733073 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48780052d0 con 0x7f488c0ccf00 2024-09-16T07:33:23.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f4883e00640 1 -- 172.21.15.71:0/1575733073 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4878005470 con 0x7f488c0ccf00 2024-09-16T07:33:23.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f4883e00640 1 -- 172.21.15.71:0/1575733073 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878012870 con 0x7f488c0ccf00 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.556+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c14e270 con 0x7f488c0ccf00 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.557+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c14eb10 con 0x7f488c0ccf00 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.558+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c06f980 con 0x7f488c0ccf00 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.559+0000 7f4883e00640 1 -- 172.21.15.71:0/1575733073 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f48780035f0 con 0x7f488c0ccf00 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.560+0000 7f4883e00640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f486c079e40 0x7f486c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.560+0000 7f4883e00640 1 -- 172.21.15.71:0/1575733073 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f4878099540 con 0x7f488c0ccf00 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.560+0000 7f488be00640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f486c079e40 0x7f486c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.561+0000 7f488be00640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f486c079e40 0x7f486c07c300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f487c033000 tx=0x7f487c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:23.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.564+0000 7f4883e00640 1 -- 172.21.15.71:0/1575733073 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f48780631f0 con 0x7f488c0ccf00 2024-09-16T07:33:23.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.814+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f488c1548a0 con 0x7f488c0ccf00 2024-09-16T07:33:23.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.955+0000 7f4883e00640 1 -- 172.21.15.71:0/1575733073 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v493) ==== 174+0+0 (secure 0 0 0) 0x7f48780999e0 con 0x7f488c0ccf00 2024-09-16T07:33:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.957+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f486c079e40 msgr2=0x7f486c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.957+0000 7f4893000640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f486c079e40 0x7f486c07c300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f487c033000 tx=0x7f487c004730 comp rx=0 tx=0).stop 2024-09-16T07:33:23.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c0ccf00 msgr2=0x7f488c153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c0ccf00 0x7f488c153170 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f487800c450 tx=0x7f487800c910 comp rx=0 tx=0).stop 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 shutdown_connections 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f486c079e40 0x7f486c07c300 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f488c15f5a0 0x7f488c144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f488c0ccf00 0x7f488c153170 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 --2- 172.21.15.71:0/1575733073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f488c073fb0 0x7f488c14d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 >> 172.21.15.71:0/1575733073 conn(0x7f488c0b1950 msgr2=0x7f488c0ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 shutdown_connections 2024-09-16T07:33:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:23.958+0000 7f4893000640 1 -- 172.21.15.71:0/1575733073 wait complete. 2024-09-16T07:33:23.975 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-09-16T07:33:23.975 INFO:tasks.cephfs_test_runner:Starting test: test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-09-16T07:33:23.975 INFO:tasks.cephfs_test_runner:test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:33:23.976 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:33:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.133+0000 7f96f4c00640 1 -- 172.21.15.71:0/1369480401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f015f5a0 msgr2=0x7f96f00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.133+0000 7f96f4c00640 1 --2- 172.21.15.71:0/1369480401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f015f5a0 0x7f96f00b2440 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f96f006dac0 tx=0x7f96e002a560 comp rx=0 tx=0).stop 2024-09-16T07:33:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.133+0000 7f96f4c00640 1 -- 172.21.15.71:0/1369480401 shutdown_connections 2024-09-16T07:33:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.133+0000 7f96f4c00640 1 --2- 172.21.15.71:0/1369480401 >> v2:172.21.15.177:3300/0 conn(0x7f96f0073fb0 0x7f96f0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.133+0000 7f96f4c00640 1 --2- 172.21.15.71:0/1369480401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f015f5a0 0x7f96f00b2440 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.133+0000 7f96f4c00640 1 -- 172.21.15.71:0/1369480401 >> 172.21.15.71:0/1369480401 conn(0x7f96f00b1950 msgr2=0x7f96f00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.133+0000 7f96f4c00640 1 -- 172.21.15.71:0/1369480401 shutdown_connections 2024-09-16T07:33:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.133+0000 7f96f4c00640 1 -- 172.21.15.71:0/1369480401 wait complete. 2024-09-16T07:33:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.135+0000 7f96f4c00640 1 Processor -- start 2024-09-16T07:33:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.135+0000 7f96f4c00640 1 -- start start 2024-09-16T07:33:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96f4c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96f0073fb0 0x7f96f00ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96f4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f00ccf00 0x7f96f014e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96f4c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96f015f5a0 0x7f96f014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96f4c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f96f006f2e0 con 0x7f96f00ccf00 2024-09-16T07:33:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96f4c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f96f006f160 con 0x7f96f0073fb0 2024-09-16T07:33:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96f4c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96f006f460 con 0x7f96f015f5a0 2024-09-16T07:33:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ed400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f00ccf00 0x7f96f014e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ed400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f00ccf00 0x7f96f014e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40744/0 (socket says 172.21.15.71:40744) 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ed400640 1 -- 172.21.15.71:0/174665761 learned_addr learned my addr 172.21.15.71:0/174665761 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ede00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96f0073fb0 0x7f96f00ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ee800640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96f015f5a0 0x7f96f014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ed400640 1 -- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96f015f5a0 msgr2=0x7f96f014e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ed400640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96f015f5a0 0x7f96f014e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ed400640 1 -- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96f0073fb0 msgr2=0x7f96f00ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ed400640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96f0073fb0 0x7f96f00ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.136+0000 7f96ed400640 1 -- 172.21.15.71:0/174665761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f01f5680 con 0x7f96f00ccf00 2024-09-16T07:33:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.137+0000 7f96ee800640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96f015f5a0 0x7f96f014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:24.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.137+0000 7f96ed400640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f00ccf00 0x7f96f014e2b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00c480 tx=0x7f96dc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:24.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.137+0000 7f96e6000640 1 -- 172.21.15.71:0/174665761 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0052d0 con 0x7f96f00ccf00 2024-09-16T07:33:24.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.137+0000 7f96e6000640 1 -- 172.21.15.71:0/174665761 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96dc005470 con 0x7f96f00ccf00 2024-09-16T07:33:24.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.137+0000 7f96e6000640 1 -- 172.21.15.71:0/174665761 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc012870 con 0x7f96f00ccf00 2024-09-16T07:33:24.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.137+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f01f5c60 con 0x7f96f00ccf00 2024-09-16T07:33:24.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.137+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f01f6220 con 0x7f96f00ccf00 2024-09-16T07:33:24.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.138+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f006f9f0 con 0x7f96f00ccf00 2024-09-16T07:33:24.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.140+0000 7f96e6000640 1 -- 172.21.15.71:0/174665761 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f96dc0035f0 con 0x7f96f00ccf00 2024-09-16T07:33:24.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.140+0000 7f96e6000640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f96d0079e40 0x7f96d007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.141+0000 7f96e6000640 1 -- 172.21.15.71:0/174665761 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f96dc099570 con 0x7f96f00ccf00 2024-09-16T07:33:24.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.141+0000 7f96ede00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f96d0079e40 0x7f96d007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.142+0000 7f96ede00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f96d0079e40 0x7f96d007c300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f96e0033000 tx=0x7f96e0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:24.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.145+0000 7f96e6000640 1 -- 172.21.15.71:0/174665761 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f96dc063280 con 0x7f96f00ccf00 2024-09-16T07:33:24.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.330+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96f00cf210 con 0x7f96f00ccf00 2024-09-16T07:33:24.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.331+0000 7f96e6000640 1 -- 172.21.15.71:0/174665761 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+13090 (secure 0 0 0) 0x7f96dc062b80 con 0x7f96f00ccf00 2024-09-16T07:33:24.331 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:24.331 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":97,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:33:22.946743+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6837/4183855149":"2024-09-17T07:33:19.939917+0000","172.21.15.177:6835/4183855149":"2024-09-17T07:33:19.939917+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:33:24.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f96d0079e40 msgr2=0x7f96d007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f96d0079e40 0x7f96d007c300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f96e0033000 tx=0x7f96e0004730 comp rx=0 tx=0).stop 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f00ccf00 msgr2=0x7f96f014e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f00ccf00 0x7f96f014e2b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00c480 tx=0x7f96dc00c940 comp rx=0 tx=0).stop 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 shutdown_connections 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f96d0079e40 0x7f96d007c300 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96f015f5a0 0x7f96f014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96f00ccf00 0x7f96f014e2b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 --2- 172.21.15.71:0/174665761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96f0073fb0 0x7f96f00ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 >> 172.21.15.71:0/174665761 conn(0x7f96f00b1950 msgr2=0x7f96f00ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 shutdown_connections 2024-09-16T07:33:24.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.334+0000 7f96f4c00640 1 -- 172.21.15.71:0/174665761 wait complete. 2024-09-16T07:33:24.349 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:33:24.350 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:33:24.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.511+0000 7f8caba00640 1 -- 172.21.15.71:0/4158096110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c88003a00 msgr2=0x7f8c88003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.511+0000 7f8caba00640 1 --2- 172.21.15.71:0/4158096110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c88003a00 0x7f8c88003e20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009b50 tx=0x7f8c88005830 comp rx=0 tx=0).stop 2024-09-16T07:33:24.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.511+0000 7f8ca9a00640 1 -- 172.21.15.71:0/4158096110 reap_dead start 2024-09-16T07:33:24.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.511+0000 7f8caba00640 1 -- 172.21.15.71:0/4158096110 shutdown_connections 2024-09-16T07:33:24.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.511+0000 7f8caba00640 1 -- 172.21.15.71:0/4158096110 >> 172.21.15.71:0/4158096110 conn(0x7f8ca40b1840 msgr2=0x7f8ca40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:24.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.511+0000 7f8caba00640 1 -- 172.21.15.71:0/4158096110 shutdown_connections 2024-09-16T07:33:24.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.511+0000 7f8caba00640 1 -- 172.21.15.71:0/4158096110 wait complete. 2024-09-16T07:33:24.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.513+0000 7f8caba00640 1 Processor -- start 2024-09-16T07:33:24.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.513+0000 7f8caba00640 1 -- start start 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8caba00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c88003a00 0x7f8ca4140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8caba00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca413d340 0x7f8ca4151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8caba00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca4154470 0x7f8ca41548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8caba00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40cdad0 con 0x7f8ca413d340 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8caba00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40cf440 con 0x7f8c88003a00 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8caba00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca40cdc50 con 0x7f8ca4154470 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8c9be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca413d340 0x7f8ca4151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8c9be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca413d340 0x7f8ca4151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40748/0 (socket says 172.21.15.71:40748) 2024-09-16T07:33:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8c9be00640 1 -- 172.21.15.71:0/3490148554 learned_addr learned my addr 172.21.15.71:0/3490148554 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.514+0000 7f8ca9000640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c88003a00 0x7f8ca4140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8ca9a00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca4154470 0x7f8ca41548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8c9be00640 1 -- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca4154470 msgr2=0x7f8ca41548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8c9be00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca4154470 0x7f8ca41548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8c9be00640 1 -- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c88003a00 msgr2=0x7f8ca4140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8c9be00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c88003a00 0x7f8ca4140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8c9be00640 1 -- 172.21.15.71:0/3490148554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca4151dc0 con 0x7f8ca413d340 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8c9be00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca413d340 0x7f8ca4151880 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00f980 tx=0x7f8c8c00fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8ca1600640 1 -- 172.21.15.71:0/3490148554 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00bcb0 con 0x7f8ca413d340 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8ca1600640 1 -- 172.21.15.71:0/3490148554 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c8c011440 con 0x7f8ca413d340 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8ca1600640 1 -- 172.21.15.71:0/3490148554 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c011870 con 0x7f8ca413d340 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca41411c0 con 0x7f8ca413d340 2024-09-16T07:33:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.515+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca4152780 con 0x7f8ca413d340 2024-09-16T07:33:24.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.516+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca413dab0 con 0x7f8ca413d340 2024-09-16T07:33:24.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.518+0000 7f8ca1600640 1 -- 172.21.15.71:0/3490148554 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f8c8c019020 con 0x7f8ca413d340 2024-09-16T07:33:24.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.518+0000 7f8ca1600640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c80079e40 0x7f8c8007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.519+0000 7f8ca1600640 1 -- 172.21.15.71:0/3490148554 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f8c8c09a380 con 0x7f8ca413d340 2024-09-16T07:33:24.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.519+0000 7f8ca9000640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c80079e40 0x7f8c8007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.520+0000 7f8ca9000640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c80079e40 0x7f8c8007c300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8c94035000 tx=0x7f8c9402d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:24.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.523+0000 7f8ca1600640 1 -- 172.21.15.71:0/3490148554 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ca413dab0 con 0x7f8ca413d340 2024-09-16T07:33:24.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.708+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ca415ee10 con 0x7f8ca413d340 2024-09-16T07:33:24.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.708+0000 7f8ca1600640 1 -- 172.21.15.71:0/3490148554 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f8c8c0639e0 con 0x7f8ca413d340 2024-09-16T07:33:24.708 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c80079e40 msgr2=0x7f8c8007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c80079e40 0x7f8c8007c300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8c94035000 tx=0x7f8c9402d040 comp rx=0 tx=0).stop 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca413d340 msgr2=0x7f8ca4151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca413d340 0x7f8ca4151880 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00f980 tx=0x7f8c8c00fe40 comp rx=0 tx=0).stop 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 shutdown_connections 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c80079e40 0x7f8c8007c300 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ca4154470 0x7f8ca41548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ca413d340 0x7f8ca4151880 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 --2- 172.21.15.71:0/3490148554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c88003a00 0x7f8ca4140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 >> 172.21.15.71:0/3490148554 conn(0x7f8ca40b1840 msgr2=0x7f8ca415c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:24.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 shutdown_connections 2024-09-16T07:33:24.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.711+0000 7f8caba00640 1 -- 172.21.15.71:0/3490148554 wait complete. 2024-09-16T07:33:24.727 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-16T07:33:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.883+0000 7f910b800640 1 -- 172.21.15.71:0/801001955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 msgr2=0x7f910406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.883+0000 7f910b800640 1 --2- 172.21.15.71:0/801001955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 0x7f910406f160 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005fd0 tx=0x7f90fc02aa10 comp rx=0 tx=0).stop 2024-09-16T07:33:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.883+0000 7f910b800640 1 -- 172.21.15.71:0/801001955 shutdown_connections 2024-09-16T07:33:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.883+0000 7f910b800640 1 --2- 172.21.15.71:0/801001955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 0x7f910406f160 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.883+0000 7f910b800640 1 -- 172.21.15.71:0/801001955 >> 172.21.15.71:0/801001955 conn(0x7f91040b1950 msgr2=0x7f91040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:24.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.883+0000 7f910b800640 1 -- 172.21.15.71:0/801001955 shutdown_connections 2024-09-16T07:33:24.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.884+0000 7f910b800640 1 -- 172.21.15.71:0/801001955 wait complete. 2024-09-16T07:33:24.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.885+0000 7f910b800640 1 Processor -- start 2024-09-16T07:33:24.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.885+0000 7f910b800640 1 -- start start 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f910b800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 0x7f91040c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f910b800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91040ccf00 0x7f91041f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f910b800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910415f5a0 0x7f91041f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f910b800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9104074840 con 0x7f91040b2440 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f910b800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f91040746c0 con 0x7f91040ccf00 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f910b800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f910406f6a0 con 0x7f910415f5a0 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f9108e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 0x7f91040c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f9108e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 0x7f91040c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47540/0 (socket says 172.21.15.71:47540) 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.886+0000 7f9108e00640 1 -- 172.21.15.71:0/1665192885 learned_addr learned my addr 172.21.15.71:0/1665192885 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9108e00640 1 -- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910415f5a0 msgr2=0x7f91041f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9108e00640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910415f5a0 0x7f91041f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9108e00640 1 -- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91040ccf00 msgr2=0x7f91041f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9108e00640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91040ccf00 0x7f91041f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9108e00640 1 -- 172.21.15.71:0/1665192885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91040c7700 con 0x7f91040b2440 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9108e00640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 0x7f91040c6d90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f90f400c3d0 tx=0x7f90f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9100c00640 1 -- 172.21.15.71:0/1665192885 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4017070 con 0x7f91040b2440 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9100c00640 1 -- 172.21.15.71:0/1665192885 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90f40053c0 con 0x7f91040b2440 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91040c79e0 con 0x7f91040b2440 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f9100c00640 1 -- 172.21.15.71:0/1665192885 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4003850 con 0x7f91040b2440 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.887+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91040c7fa0 con 0x7f91040b2440 2024-09-16T07:33:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.889+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910406f160 con 0x7f91040b2440 2024-09-16T07:33:24.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.890+0000 7f9100c00640 1 -- 172.21.15.71:0/1665192885 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f90f401a070 con 0x7f91040b2440 2024-09-16T07:33:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.891+0000 7f9100c00640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e4079e40 0x7f90e407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.891+0000 7f9100c00640 1 -- 172.21.15.71:0/1665192885 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f90f4098d80 con 0x7f91040b2440 2024-09-16T07:33:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.891+0000 7f9103e00640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e4079e40 0x7f90e407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:24.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.892+0000 7f9103e00640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e4079e40 0x7f90e407c300 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f90f0003cc0 tx=0x7f90f00079b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:24.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:24.895+0000 7f9100c00640 1 -- 172.21.15.71:0/1665192885 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f90f4062a90 con 0x7f91040b2440 2024-09-16T07:33:25.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.150+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f91040acdd0 con 0x7f91040b2440 2024-09-16T07:33:25.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.963+0000 7f9100c00640 1 -- 172.21.15.71:0/1665192885 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v495) ==== 181+0+0 (secure 0 0 0) 0x7f90f4062390 con 0x7f91040b2440 2024-09-16T07:33:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e4079e40 msgr2=0x7f90e407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e4079e40 0x7f90e407c300 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f90f0003cc0 tx=0x7f90f00079b0 comp rx=0 tx=0).stop 2024-09-16T07:33:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 msgr2=0x7f91040c6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 0x7f91040c6d90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f90f400c3d0 tx=0x7f90f400c890 comp rx=0 tx=0).stop 2024-09-16T07:33:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 shutdown_connections 2024-09-16T07:33:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e4079e40 0x7f90e407c300 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910415f5a0 0x7f91041f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91040ccf00 0x7f91041f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 --2- 172.21.15.71:0/1665192885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91040b2440 0x7f91040c6d90 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.966+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 >> 172.21.15.71:0/1665192885 conn(0x7f91040b1950 msgr2=0x7f91040ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.967+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 shutdown_connections 2024-09-16T07:33:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:25.967+0000 7f910b800640 1 -- 172.21.15.71:0/1665192885 wait complete. 2024-09-16T07:33:25.986 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:33:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.148+0000 7f49e7200640 1 -- 172.21.15.71:0/3295491952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e8161440 msgr2=0x7f49e80b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.148+0000 7f49e7200640 1 --2- 172.21.15.71:0/3295491952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e8161440 0x7f49e80b5370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f49e80d4ea0 tx=0x7f49d802a400 comp rx=0 tx=0).stop 2024-09-16T07:33:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.148+0000 7f49e7200640 1 -- 172.21.15.71:0/3295491952 shutdown_connections 2024-09-16T07:33:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.148+0000 7f49e7200640 1 --2- 172.21.15.71:0/3295491952 >> v2:172.21.15.177:3300/0 conn(0x7f49e80b34b0 0x7f49e80b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.148+0000 7f49e7200640 1 --2- 172.21.15.71:0/3295491952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e8161440 0x7f49e80b5370 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.148+0000 7f49e7200640 1 -- 172.21.15.71:0/3295491952 >> 172.21.15.71:0/3295491952 conn(0x7f49e80d3df0 msgr2=0x7f49e80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:26.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.148+0000 7f49e7200640 1 -- 172.21.15.71:0/3295491952 shutdown_connections 2024-09-16T07:33:26.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.148+0000 7f49e7200640 1 -- 172.21.15.71:0/3295491952 wait complete. 2024-09-16T07:33:26.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.150+0000 7f49e7200640 1 Processor -- start 2024-09-16T07:33:26.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.150+0000 7f49e7200640 1 -- start start 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e7200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e80b34b0 0x7f49e80b7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e7200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e80b3e50 0x7f49e80b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49e8161440 0x7f49e80b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f49e81619c0 con 0x7f49e8161440 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e7200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80b2990 con 0x7f49e80b34b0 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e7200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49e80d2470 con 0x7f49e80b3e50 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49e8161440 0x7f49e80b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49e8161440 0x7f49e80b8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47552/0 (socket says 172.21.15.71:47552) 2024-09-16T07:33:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e6800640 1 -- 172.21.15.71:0/1697490103 learned_addr learned my addr 172.21.15.71:0/1697490103 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e5e00640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e80b34b0 0x7f49e80b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e5400640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e80b3e50 0x7f49e80b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e6800640 1 -- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e80b3e50 msgr2=0x7f49e80b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e6800640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e80b3e50 0x7f49e80b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e6800640 1 -- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e80b34b0 msgr2=0x7f49e80b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e6800640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e80b34b0 0x7f49e80b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e6800640 1 -- 172.21.15.71:0/1697490103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e80b7f10 con 0x7f49e8161440 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.151+0000 7f49e5400640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e80b3e50 0x7f49e80b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.152+0000 7f49e6800640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49e8161440 0x7f49e80b8dc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f49e000cc70 tx=0x7f49e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.152+0000 7f49de000640 1 -- 172.21.15.71:0/1697490103 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0016020 con 0x7f49e8161440 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.152+0000 7f49de000640 1 -- 172.21.15.71:0/1697490103 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49e0012420 con 0x7f49e8161440 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.152+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e80b81f0 con 0x7f49e8161440 2024-09-16T07:33:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.152+0000 7f49de000640 1 -- 172.21.15.71:0/1697490103 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0012850 con 0x7f49e8161440 2024-09-16T07:33:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.152+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e80cb110 con 0x7f49e8161440 2024-09-16T07:33:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.153+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e80b3910 con 0x7f49e8161440 2024-09-16T07:33:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.154+0000 7f49de000640 1 -- 172.21.15.71:0/1697490103 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f49e0019020 con 0x7f49e8161440 2024-09-16T07:33:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.155+0000 7f49de000640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49c8079e70 0x7f49c807c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:26.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.155+0000 7f49de000640 1 -- 172.21.15.71:0/1697490103 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f49e009a270 con 0x7f49e8161440 2024-09-16T07:33:26.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.156+0000 7f49e5e00640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49c8079e70 0x7f49c807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:26.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.156+0000 7f49e5e00640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49c8079e70 0x7f49c807c330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f49d8034000 tx=0x7f49d8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:26.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.160+0000 7f49de000640 1 -- 172.21.15.71:0/1697490103 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f49e0020020 con 0x7f49e8161440 2024-09-16T07:33:26.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.347+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f49e80cb2e0 con 0x7f49e8161440 2024-09-16T07:33:26.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.348+0000 7f49de000640 1 -- 172.21.15.71:0/1697490103 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+1998 (secure 0 0 0) 0x7f49e0063f20 con 0x7f49e8161440 2024-09-16T07:33:26.347 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:33:26.348 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:33:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:33:26.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49c8079e70 msgr2=0x7f49c807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49c8079e70 0x7f49c807c330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f49d8034000 tx=0x7f49d8004730 comp rx=0 tx=0).stop 2024-09-16T07:33:26.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49e8161440 msgr2=0x7f49e80b8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49e8161440 0x7f49e80b8dc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f49e000cc70 tx=0x7f49e000a430 comp rx=0 tx=0).stop 2024-09-16T07:33:26.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 shutdown_connections 2024-09-16T07:33:26.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49c8079e70 0x7f49c807c330 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49e8161440 0x7f49e80b8dc0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49e80b3e50 0x7f49e80b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 --2- 172.21.15.71:0/1697490103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49e80b34b0 0x7f49e80b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 >> 172.21.15.71:0/1697490103 conn(0x7f49e80d3df0 msgr2=0x7f49e80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 shutdown_connections 2024-09-16T07:33:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.350+0000 7f49e7200640 1 -- 172.21.15.71:0/1697490103 wait complete. 2024-09-16T07:33:26.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:33:26.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.525+0000 7f23b3e00640 1 -- 172.21.15.71:0/3992955334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 msgr2=0x7f23b406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.525+0000 7f23b3e00640 1 --2- 172.21.15.71:0/3992955334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 0x7f23b406fce0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f23b40b5600 tx=0x7f23a402a560 comp rx=0 tx=0).stop 2024-09-16T07:33:26.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.525+0000 7f23b3e00640 1 -- 172.21.15.71:0/3992955334 shutdown_connections 2024-09-16T07:33:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.525+0000 7f23b3e00640 1 --2- 172.21.15.71:0/3992955334 >> v2:172.21.15.177:3300/0 conn(0x7f23b40cbc00 0x7f23b4073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.525+0000 7f23b3e00640 1 --2- 172.21.15.71:0/3992955334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 0x7f23b406fce0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.526+0000 7f23b3e00640 1 -- 172.21.15.71:0/3992955334 >> 172.21.15.71:0/3992955334 conn(0x7f23b40cf9f0 msgr2=0x7f23b40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.526+0000 7f23b3e00640 1 -- 172.21.15.71:0/3992955334 shutdown_connections 2024-09-16T07:33:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.526+0000 7f23b3e00640 1 -- 172.21.15.71:0/3992955334 wait complete. 2024-09-16T07:33:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.527+0000 7f23b3e00640 1 Processor -- start 2024-09-16T07:33:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.527+0000 7f23b3e00640 1 -- start start 2024-09-16T07:33:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b4072560 0x7f23b40b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b40cbc00 0x7f23b414b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 0x7f23b414b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f23b40ae990 con 0x7f23b415bec0 2024-09-16T07:33:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23b40ae810 con 0x7f23b4072560 2024-09-16T07:33:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f23b40aeb10 con 0x7f23b40cbc00 2024-09-16T07:33:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 0x7f23b414b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 0x7f23b414b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47560/0 (socket says 172.21.15.71:47560) 2024-09-16T07:33:26.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.528+0000 7f23b3400640 1 -- 172.21.15.71:0/1315556936 learned_addr learned my addr 172.21.15.71:0/1315556936 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b3400640 1 -- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b40cbc00 msgr2=0x7f23b414b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b3400640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b40cbc00 0x7f23b414b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b3400640 1 -- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b4072560 msgr2=0x7f23b40b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b2a00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b4072560 0x7f23b40b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b3400640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b4072560 0x7f23b40b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b3400640 1 -- 172.21.15.71:0/1315556936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b40c7750 con 0x7f23b415bec0 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b2a00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b4072560 0x7f23b40b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b3400640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 0x7f23b414b650 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00b9e0 tx=0x7f23ac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23aaa00640 1 -- 172.21.15.71:0/1315556936 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac019070 con 0x7f23b415bec0 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23aaa00640 1 -- 172.21.15.71:0/1315556936 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23ac00cc30 con 0x7f23b415bec0 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b414be80 con 0x7f23b415bec0 2024-09-16T07:33:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.529+0000 7f23aaa00640 1 -- 172.21.15.71:0/1315556936 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0146a0 con 0x7f23b415bec0 2024-09-16T07:33:26.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.530+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b41495b0 con 0x7f23b415bec0 2024-09-16T07:33:26.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.531+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b40af040 con 0x7f23b415bec0 2024-09-16T07:33:26.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.532+0000 7f23aaa00640 1 -- 172.21.15.71:0/1315556936 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f23ac01c070 con 0x7f23b415bec0 2024-09-16T07:33:26.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.533+0000 7f23aaa00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2394079e40 0x7f239407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:26.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.533+0000 7f23aaa00640 1 -- 172.21.15.71:0/1315556936 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f23ac0996d0 con 0x7f23b415bec0 2024-09-16T07:33:26.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.533+0000 7f23b2a00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2394079e40 0x7f239407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:26.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.534+0000 7f23b2a00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2394079e40 0x7f239407c300 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f23a4033000 tx=0x7f23a4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:26.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.537+0000 7f23aaa00640 1 -- 172.21.15.71:0/1315556936 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f23b40c7750 con 0x7f23b415bec0 2024-09-16T07:33:26.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.722+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f23b40d00b0 con 0x7f23b415bec0 2024-09-16T07:33:26.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.726+0000 7f23aaa00640 1 -- 172.21.15.71:0/1315556936 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f23ac00ce20 con 0x7f23b415bec0 2024-09-16T07:33:26.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2394079e40 msgr2=0x7f239407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2394079e40 0x7f239407c300 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f23a4033000 tx=0x7f23a4004730 comp rx=0 tx=0).stop 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 msgr2=0x7f23b414b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 0x7f23b414b650 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00b9e0 tx=0x7f23ac00bea0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 shutdown_connections 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2394079e40 0x7f239407c300 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23b415bec0 0x7f23b414b650 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b40cbc00 0x7f23b414b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 --2- 172.21.15.71:0/1315556936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b4072560 0x7f23b40b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 >> 172.21.15.71:0/1315556936 conn(0x7f23b40cf9f0 msgr2=0x7f23b40b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 shutdown_connections 2024-09-16T07:33:26.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:26.729+0000 7f23b3e00640 1 -- 172.21.15.71:0/1315556936 wait complete. 2024-09-16T07:33:26.744 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:33:26.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:33:26.842 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:33:26.929 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:33:26.929 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:33:26.929 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:26.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:33:26.965 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:33:26.965 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:27.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:33:27.007 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:33:27.007 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:27.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:33:27.043 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:33:27.044 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:27.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:33:27.083 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:33:27.083 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:33:27.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.241+0000 7f780d600640 1 -- 172.21.15.71:0/2603865333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78080ccf00 msgr2=0x7f78080b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.241+0000 7f780d600640 1 --2- 172.21.15.71:0/2603865333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78080ccf00 0x7f78080b2440 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f77f4005fd0 tx=0x7f77f402a5f0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.241+0000 7f780d600640 1 -- 172.21.15.71:0/2603865333 shutdown_connections 2024-09-16T07:33:27.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.241+0000 7f780d600640 1 --2- 172.21.15.71:0/2603865333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78080ccf00 0x7f78080b2440 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.241+0000 7f780d600640 1 -- 172.21.15.71:0/2603865333 >> 172.21.15.71:0/2603865333 conn(0x7f78080b1950 msgr2=0x7f78080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:27.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.241+0000 7f780d600640 1 -- 172.21.15.71:0/2603865333 shutdown_connections 2024-09-16T07:33:27.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.241+0000 7f780d600640 1 -- 172.21.15.71:0/2603865333 wait complete. 2024-09-16T07:33:27.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.242+0000 7f780d600640 1 Processor -- start 2024-09-16T07:33:27.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.243+0000 7f780d600640 1 -- start start 2024-09-16T07:33:27.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.243+0000 7f780d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7808073fb0 0x7f7808143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:27.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f780d600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78080ccf00 0x7f78081465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f780d600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780815f5a0 0x7f7808146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f780d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f780806f9f0 con 0x7f7808073fb0 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f780d600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f780806f870 con 0x7f780815f5a0 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f780d600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7808074420 con 0x7f78080ccf00 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7808073fb0 0x7f7808143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7808073fb0 0x7f7808143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47580/0 (socket says 172.21.15.71:47580) 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 -- 172.21.15.71:0/3494227605 learned_addr learned my addr 172.21.15.71:0/3494227605 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7807400640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780815f5a0 0x7f7808146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806000640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78080ccf00 0x7f78081465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 -- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78080ccf00 msgr2=0x7f78081465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78080ccf00 0x7f78081465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 -- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780815f5a0 msgr2=0x7f7808146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780815f5a0 0x7f7808146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 -- 172.21.15.71:0/3494227605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7808150e70 con 0x7f7808073fb0 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.244+0000 7f7806a00640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7808073fb0 0x7f7808143d50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f77f800c400 tx=0x7f77f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.245+0000 7f77fea00640 1 -- 172.21.15.71:0/3494227605 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8017070 con 0x7f7808073fb0 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.245+0000 7f77fea00640 1 -- 172.21.15.71:0/3494227605 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f80053c0 con 0x7f7808073fb0 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.245+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78080d0740 con 0x7f7808073fb0 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.245+0000 7f77fea00640 1 -- 172.21.15.71:0/3494227605 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8003850 con 0x7f7808073fb0 2024-09-16T07:33:27.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.245+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780815a4a0 con 0x7f7808073fb0 2024-09-16T07:33:27.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.246+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78080b2440 con 0x7f7808073fb0 2024-09-16T07:33:27.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.247+0000 7f77fea00640 1 -- 172.21.15.71:0/3494227605 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f77f801a070 con 0x7f7808073fb0 2024-09-16T07:33:27.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.248+0000 7f77fea00640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77e8079e40 0x7f77e807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:27.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.248+0000 7f77fea00640 1 -- 172.21.15.71:0/3494227605 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f77f8098e00 con 0x7f7808073fb0 2024-09-16T07:33:27.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.248+0000 7f7806000640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77e8079e40 0x7f77e807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:27.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.249+0000 7f7806000640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77e8079e40 0x7f77e807c300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f77f4033000 tx=0x7f77f4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:27.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.252+0000 7f77fea00640 1 -- 172.21.15.71:0/3494227605 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f77f8062ab0 con 0x7f7808073fb0 2024-09-16T07:33:27.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.473+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78080b3160 con 0x7f7808073fb0 2024-09-16T07:33:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.474+0000 7f77fea00640 1 -- 172.21.15.71:0/3494227605 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+3914 (secure 0 0 0) 0x7f77f80992a0 con 0x7f7808073fb0 2024-09-16T07:33:27.474 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:27.474 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":59,"btime":"2024-09-16T07:33:20:942250+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1642949519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1642949519},{"type":"v1","addr":"172.21.15.177:6836","nonce":1642949519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/2594338176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2594338176},{"type":"v1","addr":"172.21.15.71:6834","nonce":2594338176}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/482130020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":482130020},{"type":"v1","addr":"172.21.15.71:6835","nonce":482130020}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/4088950892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":4088950892},{"type":"v1","addr":"172.21.15.177:6837","nonce":4088950892}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-09-16T07:33:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 59 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77e8079e40 msgr2=0x7f77e807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77e8079e40 0x7f77e807c300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f77f4033000 tx=0x7f77f4004730 comp rx=0 tx=0).stop 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7808073fb0 msgr2=0x7f7808143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7808073fb0 0x7f7808143d50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f77f800c400 tx=0x7f77f800c8c0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 shutdown_connections 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f77e8079e40 0x7f77e807c300 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780815f5a0 0x7f7808146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78080ccf00 0x7f78081465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 --2- 172.21.15.71:0/3494227605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7808073fb0 0x7f7808143d50 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 >> 172.21.15.71:0/3494227605 conn(0x7f78080b1950 msgr2=0x7f78080b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 shutdown_connections 2024-09-16T07:33:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.477+0000 7f780d600640 1 -- 172.21.15.71:0/3494227605 wait complete. 2024-09-16T07:33:27.493 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:33:27.494 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:33:27.494 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:33:27.494 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:33:27.494 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:33:27.494 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:33:27.495 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:33:27.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:33:27.495 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:33:27.495 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:33:27.495 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:33:27.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:33:27.495 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:33:27.495 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:33:27.496 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:33:27.496 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:33:27.511 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:33:27.511 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:33:27.513 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:33:27.513 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:33:27.514 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:33:27.514 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:33:27.514 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:33:27.514 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:33:27.557 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:33:27.557 DEBUG:teuthology.parallel:result is None 2024-09-16T07:33:27.558 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:33:27.558 DEBUG:teuthology.parallel:result is None 2024-09-16T07:33:27.559 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:33:27.559 DEBUG:teuthology.parallel:result is None 2024-09-16T07:33:27.561 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:33:27.561 DEBUG:teuthology.parallel:result is None 2024-09-16T07:33:27.561 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:33:27.686 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:33:27.686 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:33:27.687 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:33:27.690 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:33:27.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.729+0000 7f6d13800640 1 -- 172.21.15.71:0/272137061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c161550 msgr2=0x7f6d0c06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.729+0000 7f6d13800640 1 --2- 172.21.15.71:0/272137061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c161550 0x7f6d0c06ef50 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009830 tx=0x7f6cfc02a530 comp rx=0 tx=0).stop 2024-09-16T07:33:27.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.729+0000 7f6d13800640 1 -- 172.21.15.71:0/272137061 shutdown_connections 2024-09-16T07:33:27.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.729+0000 7f6d13800640 1 --2- 172.21.15.71:0/272137061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c161550 0x7f6d0c06ef50 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.729+0000 7f6d13800640 1 -- 172.21.15.71:0/272137061 >> 172.21.15.71:0/272137061 conn(0x7f6d0c0b1950 msgr2=0x7f6d0c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:27.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.729+0000 7f6d13800640 1 -- 172.21.15.71:0/272137061 shutdown_connections 2024-09-16T07:33:27.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.729+0000 7f6d13800640 1 -- 172.21.15.71:0/272137061 wait complete. 2024-09-16T07:33:27.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.731+0000 7f6d13800640 1 Processor -- start 2024-09-16T07:33:27.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.731+0000 7f6d13800640 1 -- start start 2024-09-16T07:33:27.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d13800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0c072df0 0x7f6d0c0d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:27.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d13800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c0af3d0 0x7f6d0c1f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:27.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d13800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0c161550 0x7f6d0c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:27.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d13800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c0739b0 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d13800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c06fdb0 con 0x7f6d0c161550 2024-09-16T07:33:27.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d13800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0c073b30 con 0x7f6d0c072df0 2024-09-16T07:33:27.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d0be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c0af3d0 0x7f6d0c1f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:27.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d0be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c0af3d0 0x7f6d0c1f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47624/0 (socket says 172.21.15.71:47624) 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d0be00640 1 -- 172.21.15.71:0/1938123524 learned_addr learned my addr 172.21.15.71:0/1938123524 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d11800640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0c161550 0x7f6d0c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.732+0000 7f6d10e00640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0c072df0 0x7f6d0c0d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d0be00640 1 -- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0c072df0 msgr2=0x7f6d0c0d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d0be00640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0c072df0 0x7f6d0c0d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d0be00640 1 -- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0c161550 msgr2=0x7f6d0c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d0be00640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0c161550 0x7f6d0c09aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d0be00640 1 -- 172.21.15.71:0/1938123524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c09c300 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d10e00640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0c072df0 0x7f6d0c0d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:27.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d11800640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0c161550 0x7f6d0c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:27.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d0be00640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c0af3d0 0x7f6d0c1f5530 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c4d0 tx=0x7f6cf800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:27.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d08c00640 1 -- 172.21.15.71:0/1938123524 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80052d0 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d08c00640 1 -- 172.21.15.71:0/1938123524 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cf8005470 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d08c00640 1 -- 172.21.15.71:0/1938123524 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8012870 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c1f5a70 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.733+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c09c880 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.734+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0c09d170 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.736+0000 7f6d08c00640 1 -- 172.21.15.71:0/1938123524 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f6cf80035f0 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.737+0000 7f6d08c00640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6cec079e40 0x7f6cec07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:27.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.737+0000 7f6d08c00640 1 -- 172.21.15.71:0/1938123524 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4971+0+0 (secure 0 0 0) 0x7f6cf8099690 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.737+0000 7f6d10e00640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6cec079e40 0x7f6cec07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:27.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.737+0000 7f6d10e00640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6cec079e40 0x7f6cec07c300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc034000 tx=0x7f6cfc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:27.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.741+0000 7f6d08c00640 1 -- 172.21.15.71:0/1938123524 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6cf8063340 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.937+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6d0c074020 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.988+0000 7f6d08c00640 1 -- 172.21.15.71:0/1938123524 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7f6cf8062c40 con 0x7f6d0c0af3d0 2024-09-16T07:33:27.988 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:33:27.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6cec079e40 msgr2=0x7f6cec07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6cec079e40 0x7f6cec07c300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc034000 tx=0x7f6cfc004730 comp rx=0 tx=0).stop 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c0af3d0 msgr2=0x7f6d0c1f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c0af3d0 0x7f6d0c1f5530 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800c4d0 tx=0x7f6cf800c990 comp rx=0 tx=0).stop 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 shutdown_connections 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6cec079e40 0x7f6cec07c300 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0c161550 0x7f6d0c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d0c0af3d0 0x7f6d0c1f5530 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 --2- 172.21.15.71:0/1938123524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0c072df0 0x7f6d0c0d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 >> 172.21.15.71:0/1938123524 conn(0x7f6d0c0b1950 msgr2=0x7f6d0c06ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 shutdown_connections 2024-09-16T07:33:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:27.991+0000 7f6d13800640 1 -- 172.21.15.71:0/1938123524 wait complete. 2024-09-16T07:33:28.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:33:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.167+0000 7f5b1be00640 1 -- 172.21.15.71:0/622896048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b140ccf00 msgr2=0x7f5b140b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.167+0000 7f5b1be00640 1 --2- 172.21.15.71:0/622896048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b140ccf00 0x7f5b140b2440 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5b00005fd0 tx=0x7f5b0002a540 comp rx=0 tx=0).stop 2024-09-16T07:33:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.167+0000 7f5b1be00640 1 -- 172.21.15.71:0/622896048 shutdown_connections 2024-09-16T07:33:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.167+0000 7f5b1be00640 1 --2- 172.21.15.71:0/622896048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b14073fb0 0x7f5b14074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.167+0000 7f5b1be00640 1 --2- 172.21.15.71:0/622896048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b140ccf00 0x7f5b140b2440 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.167+0000 7f5b1be00640 1 -- 172.21.15.71:0/622896048 >> 172.21.15.71:0/622896048 conn(0x7f5b140b1950 msgr2=0x7f5b140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.167+0000 7f5b1be00640 1 -- 172.21.15.71:0/622896048 shutdown_connections 2024-09-16T07:33:28.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.167+0000 7f5b1be00640 1 -- 172.21.15.71:0/622896048 wait complete. 2024-09-16T07:33:28.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.169+0000 7f5b1be00640 1 Processor -- start 2024-09-16T07:33:28.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.169+0000 7f5b1be00640 1 -- start start 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b14073fb0 0x7f5b141f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b140ccf00 0x7f5b14159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b1be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1415f5a0 0x7f5b1415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b1be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1406f2e0 con 0x7f5b1415f5a0 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b1be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1406f160 con 0x7f5b14073fb0 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b1be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1406f460 con 0x7f5b140ccf00 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1415f5a0 0x7f5b1415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1415f5a0 0x7f5b1415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47646/0 (socket says 172.21.15.71:47646) 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b14073fb0 0x7f5b141f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19e00640 1 -- 172.21.15.71:0/2716623235 learned_addr learned my addr 172.21.15.71:0/2716623235 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b18a00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b140ccf00 0x7f5b14159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19e00640 1 -- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b140ccf00 msgr2=0x7f5b14159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19e00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b140ccf00 0x7f5b14159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19e00640 1 -- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b14073fb0 msgr2=0x7f5b141f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19e00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b14073fb0 0x7f5b141f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.170+0000 7f5b19e00640 1 -- 172.21.15.71:0/2716623235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1415c200 con 0x7f5b1415f5a0 2024-09-16T07:33:28.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.171+0000 7f5b19e00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1415f5a0 0x7f5b1415bcc0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e970 tx=0x7f5b0c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:28.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.171+0000 7f5b11600640 1 -- 172.21.15.71:0/2716623235 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00ccc0 con 0x7f5b1415f5a0 2024-09-16T07:33:28.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.171+0000 7f5b11600640 1 -- 172.21.15.71:0/2716623235 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b0c00ce60 con 0x7f5b1415f5a0 2024-09-16T07:33:28.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.171+0000 7f5b11600640 1 -- 172.21.15.71:0/2716623235 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c005280 con 0x7f5b1415f5a0 2024-09-16T07:33:28.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.171+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1415c4c0 con 0x7f5b1415f5a0 2024-09-16T07:33:28.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.171+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b140cf020 con 0x7f5b1415f5a0 2024-09-16T07:33:28.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.172+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b140ceb30 con 0x7f5b1415f5a0 2024-09-16T07:33:28.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.173+0000 7f5b11600640 1 -- 172.21.15.71:0/2716623235 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f5b0c005420 con 0x7f5b1415f5a0 2024-09-16T07:33:28.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.174+0000 7f5b11600640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af4079e40 0x7f5af407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.174+0000 7f5b11600640 1 -- 172.21.15.71:0/2716623235 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4885+0+0 (secure 0 0 0) 0x7f5b0c09a3c0 con 0x7f5b1415f5a0 2024-09-16T07:33:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.175+0000 7f5b19400640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af4079e40 0x7f5af407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.175+0000 7f5b19400640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af4079e40 0x7f5af407c300 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5b14156410 tx=0x7f5b08009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:28.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.178+0000 7f5b11600640 1 -- 172.21.15.71:0/2716623235 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5b0c063a80 con 0x7f5b1415f5a0 2024-09-16T07:33:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.432+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5b1406f990 con 0x7f5b1415f5a0 2024-09-16T07:33:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.432+0000 7f5b11600640 1 -- 172.21.15.71:0/2716623235 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f5b0c0684b0 con 0x7f5b1415f5a0 2024-09-16T07:33:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:28.432 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:33:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:33:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:33:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:33:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:33:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:33:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:33:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:33:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:33:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:33:28.450 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:33:28.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af4079e40 msgr2=0x7f5af407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af4079e40 0x7f5af407c300 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5b14156410 tx=0x7f5b08009040 comp rx=0 tx=0).stop 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1415f5a0 msgr2=0x7f5b1415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1415f5a0 0x7f5b1415bcc0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e970 tx=0x7f5b0c00ed70 comp rx=0 tx=0).stop 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 shutdown_connections 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5af4079e40 0x7f5af407c300 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b1415f5a0 0x7f5b1415bcc0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b140ccf00 0x7f5b14159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 --2- 172.21.15.71:0/2716623235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b14073fb0 0x7f5b141f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 >> 172.21.15.71:0/2716623235 conn(0x7f5b140b1950 msgr2=0x7f5b1415e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 shutdown_connections 2024-09-16T07:33:28.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.435+0000 7f5b1be00640 1 -- 172.21.15.71:0/2716623235 wait complete. 2024-09-16T07:33:28.453 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:33:28.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.610+0000 7f2060600640 1 -- 172.21.15.71:0/548863497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20580ccf00 msgr2=0x7f20580b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.610+0000 7f2060600640 1 --2- 172.21.15.71:0/548863497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20580ccf00 0x7f20580b2440 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f205806dac0 tx=0x7f204402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.610+0000 7f2060600640 1 -- 172.21.15.71:0/548863497 shutdown_connections 2024-09-16T07:33:28.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.610+0000 7f2060600640 1 --2- 172.21.15.71:0/548863497 >> v2:172.21.15.177:3300/0 conn(0x7f2058073fb0 0x7f2058074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.610+0000 7f2060600640 1 --2- 172.21.15.71:0/548863497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20580ccf00 0x7f20580b2440 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.610+0000 7f2060600640 1 -- 172.21.15.71:0/548863497 >> 172.21.15.71:0/548863497 conn(0x7f20580b1950 msgr2=0x7f20580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:28.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.610+0000 7f2060600640 1 -- 172.21.15.71:0/548863497 shutdown_connections 2024-09-16T07:33:28.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.610+0000 7f2060600640 1 -- 172.21.15.71:0/548863497 wait complete. 2024-09-16T07:33:28.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.612+0000 7f2060600640 1 Processor -- start 2024-09-16T07:33:28.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.612+0000 7f2060600640 1 -- start start 2024-09-16T07:33:28.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f2060600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2058073fb0 0x7f2058158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f2060600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20580ccf00 0x7f205814c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f2060600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f205815f5a0 0x7f205815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f2060600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f205806f2e0 con 0x7f2058073fb0 2024-09-16T07:33:28.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f2060600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f205806f160 con 0x7f205815f5a0 2024-09-16T07:33:28.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f2060600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f205806f460 con 0x7f20580ccf00 2024-09-16T07:33:28.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f205dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2058073fb0 0x7f2058158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f205dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2058073fb0 0x7f2058158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47662/0 (socket says 172.21.15.71:47662) 2024-09-16T07:33:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f205dc00640 1 -- 172.21.15.71:0/1484619347 learned_addr learned my addr 172.21.15.71:0/1484619347 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f205e600640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f205815f5a0 0x7f205815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.613+0000 7f205d200640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20580ccf00 0x7f205814c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f205dc00640 1 -- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20580ccf00 msgr2=0x7f205814c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f205dc00640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20580ccf00 0x7f205814c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f205dc00640 1 -- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f205815f5a0 msgr2=0x7f205815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f205dc00640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f205815f5a0 0x7f205815aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f205dc00640 1 -- 172.21.15.71:0/1484619347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20581f7170 con 0x7f2058073fb0 2024-09-16T07:33:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f205dc00640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2058073fb0 0x7f2058158c90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f204c00ba50 tx=0x7f204c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f2055600640 1 -- 172.21.15.71:0/1484619347 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c019070 con 0x7f2058073fb0 2024-09-16T07:33:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f2055600640 1 -- 172.21.15.71:0/1484619347 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f204c00cd10 con 0x7f2058073fb0 2024-09-16T07:33:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20581f63b0 con 0x7f2058073fb0 2024-09-16T07:33:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.614+0000 7f2055600640 1 -- 172.21.15.71:0/1484619347 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c0146a0 con 0x7f2058073fb0 2024-09-16T07:33:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.615+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20581f6970 con 0x7f2058073fb0 2024-09-16T07:33:28.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.615+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205806aaf0 con 0x7f2058073fb0 2024-09-16T07:33:28.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.617+0000 7f2055600640 1 -- 172.21.15.71:0/1484619347 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f204c01c070 con 0x7f2058073fb0 2024-09-16T07:33:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.618+0000 7f2055600640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2038079e70 0x7f203807c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.618+0000 7f2055600640 1 -- 172.21.15.71:0/1484619347 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4885+0+0 (secure 0 0 0) 0x7f204c099750 con 0x7f2058073fb0 2024-09-16T07:33:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.618+0000 7f205d200640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2038079e70 0x7f203807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.619+0000 7f205d200640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2038079e70 0x7f203807c330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2044033000 tx=0x7f2044004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:28.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.622+0000 7f2055600640 1 -- 172.21.15.71:0/1484619347 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f205806aaf0 con 0x7f2058073fb0 2024-09-16T07:33:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.808+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f205806f9f0 con 0x7f2058073fb0 2024-09-16T07:33:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.809+0000 7f2055600640 1 -- 172.21.15.71:0/1484619347 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) ==== 84+0+65 (secure 0 0 0) 0x7f204c099bf0 con 0x7f2058073fb0 2024-09-16T07:33:28.808 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:28.809 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:33:28.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:28.809 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:33:28.809 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:33:28.809 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:28.809 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:33:28.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2038079e70 msgr2=0x7f203807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2038079e70 0x7f203807c330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2044033000 tx=0x7f2044004730 comp rx=0 tx=0).stop 2024-09-16T07:33:28.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2058073fb0 msgr2=0x7f2058158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2058073fb0 0x7f2058158c90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f204c00ba50 tx=0x7f204c00bf10 comp rx=0 tx=0).stop 2024-09-16T07:33:28.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 shutdown_connections 2024-09-16T07:33:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2038079e70 0x7f203807c330 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f205815f5a0 0x7f205815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20580ccf00 0x7f205814c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 --2- 172.21.15.71:0/1484619347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2058073fb0 0x7f2058158c90 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 >> 172.21.15.71:0/1484619347 conn(0x7f20580b1950 msgr2=0x7f205809caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 shutdown_connections 2024-09-16T07:33:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.811+0000 7f2060600640 1 -- 172.21.15.71:0/1484619347 wait complete. 2024-09-16T07:33:28.827 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:33:28.827 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:33:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.991+0000 7f8ef1400640 1 -- 172.21.15.71:0/2909192456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0b4de0 msgr2=0x7f8eec0d4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.991+0000 7f8ef1400640 1 --2- 172.21.15.71:0/2909192456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0b4de0 0x7f8eec0d4b20 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009830 tx=0x7f8ed402a480 comp rx=0 tx=0).stop 2024-09-16T07:33:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.991+0000 7f8ef1400640 1 -- 172.21.15.71:0/2909192456 shutdown_connections 2024-09-16T07:33:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.991+0000 7f8ef1400640 1 --2- 172.21.15.71:0/2909192456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eec161270 0x7f8eec161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.991+0000 7f8ef1400640 1 --2- 172.21.15.71:0/2909192456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0b4de0 0x7f8eec0d4b20 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.991+0000 7f8ef1400640 1 -- 172.21.15.71:0/2909192456 >> 172.21.15.71:0/2909192456 conn(0x7f8eec0d4000 msgr2=0x7f8eec0d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:28.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.991+0000 7f8ef1400640 1 -- 172.21.15.71:0/2909192456 shutdown_connections 2024-09-16T07:33:28.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.991+0000 7f8ef1400640 1 -- 172.21.15.71:0/2909192456 wait complete. 2024-09-16T07:33:28.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.993+0000 7f8ef1400640 1 Processor -- start 2024-09-16T07:33:28.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.994+0000 7f8ef1400640 1 -- start start 2024-09-16T07:33:28.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.994+0000 7f8ef1400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8eec0b4de0 0x7f8eec0a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.994+0000 7f8ef1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0d0ab0 0x7f8eec150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.994+0000 7f8ef1400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eec161270 0x7f8eec0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:28.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ef1400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec0d24a0 con 0x7f8eec0d0ab0 2024-09-16T07:33:28.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ef1400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec0d2320 con 0x7f8eec0b4de0 2024-09-16T07:33:28.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ef1400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8eec0d2620 con 0x7f8eec161270 2024-09-16T07:33:28.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ee3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0d0ab0 0x7f8eec150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ee3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0d0ab0 0x7f8eec150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47678/0 (socket says 172.21.15.71:47678) 2024-09-16T07:33:28.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ee3e00640 1 -- 172.21.15.71:0/3241858699 learned_addr learned my addr 172.21.15.71:0/3241858699 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:28.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8eea800640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8eec0b4de0 0x7f8eec0a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:28.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8eeb200640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eec161270 0x7f8eec0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:29.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ee3e00640 1 -- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eec161270 msgr2=0x7f8eec0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:29.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ee3e00640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eec161270 0x7f8eec0ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:29.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ee3e00640 1 -- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8eec0b4de0 msgr2=0x7f8eec0a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:29.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ee3e00640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8eec0b4de0 0x7f8eec0a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:29.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8ee3e00640 1 -- 172.21.15.71:0/3241858699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec151480 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.995+0000 7f8eeb200640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eec161270 0x7f8eec0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:29.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.996+0000 7f8ee3e00640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0d0ab0 0x7f8eec150f40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4000c00 tx=0x7f8ed4032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:29.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.996+0000 7f8ee2a00640 1 -- 172.21.15.71:0/3241858699 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4040070 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.996+0000 7f8ee2a00640 1 -- 172.21.15.71:0/3241858699 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ed4032e70 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.996+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec153db0 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.996+0000 7f8ee2a00640 1 -- 172.21.15.71:0/3241858699 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4003d90 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.996+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec0b83d0 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.997+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eec0d4b20 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.998+0000 7f8ee2a00640 1 -- 172.21.15.71:0/3241858699 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f8ed4039040 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.999+0000 7f8ee2a00640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ec8079e40 0x7f8ec807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:29.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:28.999+0000 7f8ee2a00640 1 -- 172.21.15.71:0/3241858699 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4885+0+0 (secure 0 0 0) 0x7f8ed40baf00 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:29.000+0000 7f8eea800640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ec8079e40 0x7f8ec807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:29.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:29.001+0000 7f8eea800640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ec8079e40 0x7f8ec807c300 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0b2e70 tx=0x7f8edc007e90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:29.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:29.003+0000 7f8ee2a00640 1 -- 172.21.15.71:0/3241858699 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ed4084c10 con 0x7f8eec0d0ab0 2024-09-16T07:33:29.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:29.189+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8eec160a90 con 0x7f8eec0d0ab0 2024-09-16T07:33:30.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.007+0000 7f8ee2a00640 1 -- 172.21.15.71:0/3241858699 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) ==== 138+0+0 (secure 0 0 0) 0x7f8ed4084510 con 0x7f8eec0d0ab0 2024-09-16T07:33:30.007 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:33:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ec8079e40 msgr2=0x7f8ec807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ec8079e40 0x7f8ec807c300 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0b2e70 tx=0x7f8edc007e90 comp rx=0 tx=0).stop 2024-09-16T07:33:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0d0ab0 msgr2=0x7f8eec150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0d0ab0 0x7f8eec150f40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4000c00 tx=0x7f8ed4032c80 comp rx=0 tx=0).stop 2024-09-16T07:33:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 shutdown_connections 2024-09-16T07:33:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ec8079e40 0x7f8ec807c300 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eec161270 0x7f8eec0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:30.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0d0ab0 0x7f8eec150f40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:30.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 --2- 172.21.15.71:0/3241858699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8eec0b4de0 0x7f8eec0a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:30.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.011+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 >> 172.21.15.71:0/3241858699 conn(0x7f8eec0d4000 msgr2=0x7f8eec06e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:30.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.012+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 shutdown_connections 2024-09-16T07:33:30.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.012+0000 7f8ef1400640 1 -- 172.21.15.71:0/3241858699 wait complete. 2024-09-16T07:33:30.031 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:33:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.199+0000 7f7104200640 1 -- 172.21.15.71:0/1347361833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 msgr2=0x7f70e4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.199+0000 7f7104200640 1 --2- 172.21.15.71:0/1347361833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 0x7f70e4003dc0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c640 tx=0x7f70e4005830 comp rx=0 tx=0).stop 2024-09-16T07:33:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.199+0000 7f7102200640 1 -- 172.21.15.71:0/1347361833 reap_dead start 2024-09-16T07:33:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.199+0000 7f7104200640 1 -- 172.21.15.71:0/1347361833 shutdown_connections 2024-09-16T07:33:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.199+0000 7f7104200640 1 -- 172.21.15.71:0/1347361833 >> 172.21.15.71:0/1347361833 conn(0x7f70fc0b1840 msgr2=0x7f70fc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.199+0000 7f7104200640 1 -- 172.21.15.71:0/1347361833 shutdown_connections 2024-09-16T07:33:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.199+0000 7f7104200640 1 -- 172.21.15.71:0/1347361833 wait complete. 2024-09-16T07:33:30.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.201+0000 7f7104200640 1 Processor -- start 2024-09-16T07:33:30.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.201+0000 7f7104200640 1 -- start start 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7104200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 0x7f70fc1f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7104200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70fc152500 0x7f70fc1f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7104200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70fc152b30 0x7f70fc141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7104200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc0737d0 con 0x7f70e40039e0 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7104200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc073650 con 0x7f70fc152b30 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7104200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f70fc0ce2c0 con 0x7f70fc152500 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7101800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 0x7f70fc1f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7101800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 0x7f70fc1f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47688/0 (socket says 172.21.15.71:47688) 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7101800640 1 -- 172.21.15.71:0/390090913 learned_addr learned my addr 172.21.15.71:0/390090913 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7100e00640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70fc152500 0x7f70fc1f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7102200640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70fc152b30 0x7f70fc141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7101800640 1 -- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70fc152500 msgr2=0x7f70fc1f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7101800640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70fc152500 0x7f70fc1f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7101800640 1 -- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70fc152b30 msgr2=0x7f70fc141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7101800640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70fc152b30 0x7f70fc141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.202+0000 7f7101800640 1 -- 172.21.15.71:0/390090913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc150360 con 0x7f70e40039e0 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.203+0000 7f7101800640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 0x7f70fc1f3820 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c760 tx=0x7f70f0004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.203+0000 7f70f9600640 1 -- 172.21.15.71:0/390090913 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0039070 con 0x7f70e40039e0 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.203+0000 7f70f9600640 1 -- 172.21.15.71:0/390090913 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70f002bcc0 con 0x7f70e40039e0 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.203+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70fc150640 con 0x7f70e40039e0 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.203+0000 7f70f9600640 1 -- 172.21.15.71:0/390090913 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0001450 con 0x7f70e40039e0 2024-09-16T07:33:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.203+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70fc150c00 con 0x7f70e40039e0 2024-09-16T07:33:30.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.204+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70fc0cdbb0 con 0x7f70e40039e0 2024-09-16T07:33:30.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.206+0000 7f70f9600640 1 -- 172.21.15.71:0/390090913 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f70f0006290 con 0x7f70e40039e0 2024-09-16T07:33:30.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.207+0000 7f70f9600640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70dc079e70 0x7f70dc07c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:30.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.207+0000 7f70f9600640 1 -- 172.21.15.71:0/390090913 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5255+0+0 (secure 0 0 0) 0x7f70f00b9d20 con 0x7f70e40039e0 2024-09-16T07:33:30.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.207+0000 7f7100e00640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70dc079e70 0x7f70dc07c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:30.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.208+0000 7f7100e00640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70dc079e70 0x7f70dc07c330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0afda0 tx=0x7f70e8005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:30.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.210+0000 7f70f9600640 1 -- 172.21.15.71:0/390090913 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f70f0004e20 con 0x7f70e40039e0 2024-09-16T07:33:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:30.395+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f70fc0afaf0 con 0x7f70e40039e0 2024-09-16T07:33:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.014+0000 7f70f9600640 1 -- 172.21.15.71:0/390090913 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) ==== 170+0+0 (secure 0 0 0) 0x7f70f0083380 con 0x7f70e40039e0 2024-09-16T07:33:31.014 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:33:31.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70dc079e70 msgr2=0x7f70dc07c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:31.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70dc079e70 0x7f70dc07c330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0afda0 tx=0x7f70e8005f90 comp rx=0 tx=0).stop 2024-09-16T07:33:31.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 msgr2=0x7f70fc1f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:31.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 0x7f70fc1f3820 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c760 tx=0x7f70f0004c30 comp rx=0 tx=0).stop 2024-09-16T07:33:31.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 shutdown_connections 2024-09-16T07:33:31.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70dc079e70 0x7f70dc07c330 secure :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0afda0 tx=0x7f70e8005f90 comp rx=0 tx=0).stop 2024-09-16T07:33:31.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70fc152b30 0x7f70fc141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:31.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70fc152500 0x7f70fc1f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:31.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 --2- 172.21.15.71:0/390090913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e40039e0 0x7f70fc1f3820 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:31.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.017+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 >> 172.21.15.71:0/390090913 conn(0x7f70fc0b1840 msgr2=0x7f70fc0aa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:31.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.018+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 shutdown_connections 2024-09-16T07:33:31.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.018+0000 7f7104200640 1 -- 172.21.15.71:0/390090913 wait complete. 2024-09-16T07:33:31.047 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:33:31.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.207+0000 7f266da00640 1 -- 172.21.15.71:0/2189448921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2668161460 msgr2=0x7f26680d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:31.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.207+0000 7f266da00640 1 --2- 172.21.15.71:0/2189448921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2668161460 0x7f26680d22f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2658009830 tx=0x7f265802a480 comp rx=0 tx=0).stop 2024-09-16T07:33:31.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.207+0000 7f266da00640 1 -- 172.21.15.71:0/2189448921 shutdown_connections 2024-09-16T07:33:31.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.207+0000 7f266da00640 1 --2- 172.21.15.71:0/2189448921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26680b43c0 0x7f26680d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:31.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.207+0000 7f266da00640 1 --2- 172.21.15.71:0/2189448921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2668161460 0x7f26680d22f0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:31.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.207+0000 7f266da00640 1 -- 172.21.15.71:0/2189448921 >> 172.21.15.71:0/2189448921 conn(0x7f266809d0b0 msgr2=0x7f26680d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:31.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.207+0000 7f266da00640 1 -- 172.21.15.71:0/2189448921 shutdown_connections 2024-09-16T07:33:31.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.207+0000 7f266da00640 1 -- 172.21.15.71:0/2189448921 wait complete. 2024-09-16T07:33:31.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.209+0000 7f266da00640 1 Processor -- start 2024-09-16T07:33:31.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.209+0000 7f266da00640 1 -- start start 2024-09-16T07:33:31.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f266da00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26680b43c0 0x7f266815e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:31.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f266da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26680b5480 0x7f26681e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:31.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f266da00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2668161460 0x7f26681e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:31.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f266da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f26680cef30 con 0x7f26680b5480 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f266da00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26680b5210 con 0x7f2668161460 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f266da00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26680cf0b0 con 0x7f26680b43c0 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26680b5480 0x7f26681e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26680b5480 0x7f26681e91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47716/0 (socket says 172.21.15.71:47716) 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667400640 1 -- 172.21.15.71:0/3920703062 learned_addr learned my addr 172.21.15.71:0/3920703062 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667e00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26680b43c0 0x7f266815e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f266d000640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2668161460 0x7f26681e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667400640 1 -- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26680b43c0 msgr2=0x7f266815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667400640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26680b43c0 0x7f266815e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667400640 1 -- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2668161460 msgr2=0x7f26681e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667400640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2668161460 0x7f26681e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.210+0000 7f2667400640 1 -- 172.21.15.71:0/3920703062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26681f4ae0 con 0x7f26680b5480 2024-09-16T07:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.211+0000 7f2667400640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26680b5480 0x7f26681e91a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f265400c430 tx=0x7f265400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.211+0000 7f265fe00640 1 -- 172.21.15.71:0/3920703062 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654017070 con 0x7f26680b5480 2024-09-16T07:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.211+0000 7f265fe00640 1 -- 172.21.15.71:0/3920703062 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26540035f0 con 0x7f26680b5480 2024-09-16T07:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.211+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26681f4dc0 con 0x7f26680b5480 2024-09-16T07:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.211+0000 7f265fe00640 1 -- 172.21.15.71:0/3920703062 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654003ac0 con 0x7f26680b5480 2024-09-16T07:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.211+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681f5380 con 0x7f26680b5480 2024-09-16T07:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.212+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26680cf560 con 0x7f26680b5480 2024-09-16T07:33:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.213+0000 7f265fe00640 1 -- 172.21.15.71:0/3920703062 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f265401a070 con 0x7f26680b5480 2024-09-16T07:33:31.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.214+0000 7f265fe00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2648079e70 0x7f264807c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:31.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.214+0000 7f265fe00640 1 -- 172.21.15.71:0/3920703062 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5637+0+0 (secure 0 0 0) 0x7f2654098d90 con 0x7f26680b5480 2024-09-16T07:33:31.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.215+0000 7f2667e00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2648079e70 0x7f264807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:31.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.215+0000 7f2667e00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2648079e70 0x7f264807c330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2658034000 tx=0x7f2658004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.219+0000 7f265fe00640 1 -- 172.21.15.71:0/3920703062 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2654062370 con 0x7f26680b5480 2024-09-16T07:33:31.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:31.440+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f26680b4180 con 0x7f26680b5480 2024-09-16T07:33:32.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.017+0000 7f265fe00640 1 -- 172.21.15.71:0/3920703062 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v62) ==== 174+0+0 (secure 0 0 0) 0x7f2654066da0 con 0x7f26680b5480 2024-09-16T07:33:32.016 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 14 and data pool 15 2024-09-16T07:33:32.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.020+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2648079e70 msgr2=0x7f264807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.020+0000 7f266da00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2648079e70 0x7f264807c330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2658034000 tx=0x7f2658004730 comp rx=0 tx=0).stop 2024-09-16T07:33:32.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.020+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26680b5480 msgr2=0x7f26681e91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.020+0000 7f266da00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26680b5480 0x7f26681e91a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f265400c430 tx=0x7f265400c8f0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.021+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 shutdown_connections 2024-09-16T07:33:32.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.021+0000 7f266da00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2648079e70 0x7f264807c330 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.021+0000 7f266da00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2668161460 0x7f26681e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.021+0000 7f266da00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26680b5480 0x7f26681e91a0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.021+0000 7f266da00640 1 --2- 172.21.15.71:0/3920703062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26680b43c0 0x7f266815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.021+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 >> 172.21.15.71:0/3920703062 conn(0x7f266809d0b0 msgr2=0x7f26680caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:32.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.021+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 shutdown_connections 2024-09-16T07:33:32.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.022+0000 7f266da00640 1 -- 172.21.15.71:0/3920703062 wait complete. 2024-09-16T07:33:32.052 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:33:32.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.211+0000 7fb51b200640 1 -- 172.21.15.71:0/3825220784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5141616b0 msgr2=0x7fb5140d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.211+0000 7fb51b200640 1 --2- 172.21.15.71:0/3825220784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5141616b0 0x7fb5140d21a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb5140c7390 tx=0x7fb51002a8b0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.212+0000 7fb51b200640 1 -- 172.21.15.71:0/3825220784 shutdown_connections 2024-09-16T07:33:32.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.212+0000 7fb51b200640 1 --2- 172.21.15.71:0/3825220784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5141616b0 0x7fb5140d21a0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.212+0000 7fb51b200640 1 --2- 172.21.15.71:0/3825220784 >> v2:172.21.15.177:3300/0 conn(0x7fb5140d4980 0x7fb5140d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.212+0000 7fb51b200640 1 -- 172.21.15.71:0/3825220784 >> 172.21.15.71:0/3825220784 conn(0x7fb5140d3df0 msgr2=0x7fb5140d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:32.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.212+0000 7fb51b200640 1 -- 172.21.15.71:0/3825220784 shutdown_connections 2024-09-16T07:33:32.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.212+0000 7fb51b200640 1 -- 172.21.15.71:0/3825220784 wait complete. 2024-09-16T07:33:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.213+0000 7fb51b200640 1 Processor -- start 2024-09-16T07:33:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.213+0000 7fb51b200640 1 -- start start 2024-09-16T07:33:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.214+0000 7fb51b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5140d4980 0x7fb5141f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.214+0000 7fb51b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5141612d0 0x7fb514157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.214+0000 7fb51b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5141616b0 0x7fb5140cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.214+0000 7fb51b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140b1b60 con 0x7fb5140d4980 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.214+0000 7fb51b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140b19e0 con 0x7fb5141612d0 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.214+0000 7fb51b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb5140b1ce0 con 0x7fb5141616b0 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.214+0000 7fb519e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5140d4980 0x7fb5141f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5140d4980 0x7fb5141f7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47740/0 (socket says 172.21.15.71:47740) 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519e00640 1 -- 172.21.15.71:0/1443582216 learned_addr learned my addr 172.21.15.71:0/1443582216 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519400640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5141612d0 0x7fb514157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb51a800640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5141616b0 0x7fb5140cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519e00640 1 -- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5141616b0 msgr2=0x7fb5140cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519e00640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5141616b0 0x7fb5140cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519e00640 1 -- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5141612d0 msgr2=0x7fb514157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519e00640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5141612d0 0x7fb514157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519e00640 1 -- 172.21.15.71:0/1443582216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5141f8350 con 0x7fb5140d4980 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.215+0000 7fb519e00640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5140d4980 0x7fb5141f7d70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c730 tx=0x7fb50800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.216+0000 7fb50e000640 1 -- 172.21.15.71:0/1443582216 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508017070 con 0x7fb5140d4980 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.216+0000 7fb50e000640 1 -- 172.21.15.71:0/1443582216 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb508005070 con 0x7fb5140d4980 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.216+0000 7fb50e000640 1 -- 172.21.15.71:0/1443582216 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50800a640 con 0x7fb5140d4980 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.216+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5141f8630 con 0x7fb5140d4980 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.216+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5140cdfc0 con 0x7fb5140d4980 2024-09-16T07:33:32.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.217+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5140b2260 con 0x7fb5140d4980 2024-09-16T07:33:32.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.218+0000 7fb50e000640 1 -- 172.21.15.71:0/1443582216 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fb50801a070 con 0x7fb5140d4980 2024-09-16T07:33:32.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.219+0000 7fb50e000640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb4f8079e40 0x7fb4f807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.219+0000 7fb50e000640 1 -- 172.21.15.71:0/1443582216 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5737+0+0 (secure 0 0 0) 0x7fb508099180 con 0x7fb5140d4980 2024-09-16T07:33:32.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.220+0000 7fb519400640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb4f8079e40 0x7fb4f807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.221+0000 7fb519400640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb4f8079e40 0x7fb4f807c300 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb51415b410 tx=0x7fb504009110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:32.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.223+0000 7fb50e000640 1 -- 172.21.15.71:0/1443582216 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb508021020 con 0x7fb5140d4980 2024-09-16T07:33:32.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.409+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb51415fb90 con 0x7fb5140d4980 2024-09-16T07:33:32.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.411+0000 7fb50e000640 1 -- 172.21.15.71:0/1443582216 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16747 (secure 0 0 0) 0x7fb5140d2cc0 con 0x7fb5140d4980 2024-09-16T07:33:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:32.411 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":104,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:33:31.441538+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:33:29.200091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T07:33:30.405982+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:33:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb4f8079e40 msgr2=0x7fb4f807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb4f8079e40 0x7fb4f807c300 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb51415b410 tx=0x7fb504009110 comp rx=0 tx=0).stop 2024-09-16T07:33:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5140d4980 msgr2=0x7fb5141f7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5140d4980 0x7fb5141f7d70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c730 tx=0x7fb50800cb30 comp rx=0 tx=0).stop 2024-09-16T07:33:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 shutdown_connections 2024-09-16T07:33:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb4f8079e40 0x7fb4f807c300 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5141616b0 0x7fb5140cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5141612d0 0x7fb514157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 --2- 172.21.15.71:0/1443582216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5140d4980 0x7fb5141f7d70 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 >> 172.21.15.71:0/1443582216 conn(0x7fb5140d3df0 msgr2=0x7fb5140d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 shutdown_connections 2024-09-16T07:33:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.413+0000 7fb51b200640 1 -- 172.21.15.71:0/1443582216 wait complete. 2024-09-16T07:33:32.429 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:33:32.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.588+0000 7fb258600640 1 -- 172.21.15.71:0/3675756333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 msgr2=0x7fb250074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.588+0000 7fb258600640 1 --2- 172.21.15.71:0/3675756333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 0x7fb250074440 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb248005fd0 tx=0x7fb24802aa10 comp rx=0 tx=0).stop 2024-09-16T07:33:32.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.588+0000 7fb258600640 1 -- 172.21.15.71:0/3675756333 shutdown_connections 2024-09-16T07:33:32.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.588+0000 7fb258600640 1 --2- 172.21.15.71:0/3675756333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2500ccf00 0x7fb2500b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.588+0000 7fb258600640 1 --2- 172.21.15.71:0/3675756333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 0x7fb250074440 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.588+0000 7fb258600640 1 -- 172.21.15.71:0/3675756333 >> 172.21.15.71:0/3675756333 conn(0x7fb2500b1950 msgr2=0x7fb2500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:32.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.588+0000 7fb258600640 1 -- 172.21.15.71:0/3675756333 shutdown_connections 2024-09-16T07:33:32.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.588+0000 7fb258600640 1 -- 172.21.15.71:0/3675756333 wait complete. 2024-09-16T07:33:32.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.590+0000 7fb258600640 1 Processor -- start 2024-09-16T07:33:32.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.590+0000 7fb258600640 1 -- start start 2024-09-16T07:33:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb258600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 0x7fb25014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb258600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2500ccf00 0x7fb250158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb258600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb25015f5a0 0x7fb2500b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb258600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb25006f2e0 con 0x7fb250073fb0 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb258600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb25006f160 con 0x7fb2500ccf00 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb258600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb25006f460 con 0x7fb25015f5a0 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 0x7fb25014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 0x7fb25014cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47772/0 (socket says 172.21.15.71:47772) 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 -- 172.21.15.71:0/2233560981 learned_addr learned my addr 172.21.15.71:0/2233560981 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 -- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb25015f5a0 msgr2=0x7fb2500b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb25015f5a0 0x7fb2500b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 -- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2500ccf00 msgr2=0x7fb250158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2500ccf00 0x7fb250158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 -- 172.21.15.71:0/2233560981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb250158e60 con 0x7fb250073fb0 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.591+0000 7fb255c00640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 0x7fb25014cde0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb24400c3a0 tx=0x7fb24400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.592+0000 7fb24d600640 1 -- 172.21.15.71:0/2233560981 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244017070 con 0x7fb250073fb0 2024-09-16T07:33:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.592+0000 7fb24d600640 1 -- 172.21.15.71:0/2233560981 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2440053c0 con 0x7fb250073fb0 2024-09-16T07:33:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.592+0000 7fb24d600640 1 -- 172.21.15.71:0/2233560981 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244003850 con 0x7fb250073fb0 2024-09-16T07:33:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.592+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2501f7eb0 con 0x7fb250073fb0 2024-09-16T07:33:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.592+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2501f8470 con 0x7fb250073fb0 2024-09-16T07:33:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.593+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25006f9f0 con 0x7fb250073fb0 2024-09-16T07:33:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.594+0000 7fb24d600640 1 -- 172.21.15.71:0/2233560981 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fb24401a070 con 0x7fb250073fb0 2024-09-16T07:33:32.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.595+0000 7fb24d600640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb230079e40 0x7fb23007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.595+0000 7fb24d600640 1 -- 172.21.15.71:0/2233560981 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5737+0+0 (secure 0 0 0) 0x7fb244098dd0 con 0x7fb250073fb0 2024-09-16T07:33:32.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.596+0000 7fb255200640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb230079e40 0x7fb23007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.597+0000 7fb255200640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb230079e40 0x7fb23007c300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0058c0 tx=0x7fb23c005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:32.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.599+0000 7fb24d600640 1 -- 172.21.15.71:0/2233560981 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb244062a80 con 0x7fb250073fb0 2024-09-16T07:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.783+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2500ad010 con 0x7fb250073fb0 2024-09-16T07:33:32.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.785+0000 7fb24d600640 1 -- 172.21.15.71:0/2233560981 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16747 (secure 0 0 0) 0x7fb244062380 con 0x7fb250073fb0 2024-09-16T07:33:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:32.785 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":104,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:33:31.441538+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:33:29.200091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T07:33:30.405982+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:33:32.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb230079e40 msgr2=0x7fb23007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb230079e40 0x7fb23007c300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0058c0 tx=0x7fb23c005850 comp rx=0 tx=0).stop 2024-09-16T07:33:32.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 msgr2=0x7fb25014cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 0x7fb25014cde0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb24400c3a0 tx=0x7fb24400c860 comp rx=0 tx=0).stop 2024-09-16T07:33:32.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 shutdown_connections 2024-09-16T07:33:32.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb230079e40 0x7fb23007c300 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb25015f5a0 0x7fb2500b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2500ccf00 0x7fb250158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 --2- 172.21.15.71:0/2233560981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb250073fb0 0x7fb25014cde0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 >> 172.21.15.71:0/2233560981 conn(0x7fb2500b1950 msgr2=0x7fb2500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:32.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 shutdown_connections 2024-09-16T07:33:32.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.787+0000 7fb258600640 1 -- 172.21.15.71:0/2233560981 wait complete. 2024-09-16T07:33:32.803 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:33:32.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.966+0000 7fd607600640 1 -- 172.21.15.71:0/380335655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6000cccd0 msgr2=0x7fd6000cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.966+0000 7fd607600640 1 --2- 172.21.15.71:0/380335655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6000cccd0 0x7fd6000cdbb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005fd0 tx=0x7fd5f802aa10 comp rx=0 tx=0).stop 2024-09-16T07:33:32.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.966+0000 7fd607600640 1 -- 172.21.15.71:0/380335655 shutdown_connections 2024-09-16T07:33:32.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.966+0000 7fd607600640 1 --2- 172.21.15.71:0/380335655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6000cccd0 0x7fd6000cdbb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.966+0000 7fd607600640 1 --2- 172.21.15.71:0/380335655 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600145750 0x7fd600073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.966+0000 7fd607600640 1 -- 172.21.15.71:0/380335655 >> 172.21.15.71:0/380335655 conn(0x7fd6000b1840 msgr2=0x7fd6000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:32.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.966+0000 7fd607600640 1 -- 172.21.15.71:0/380335655 shutdown_connections 2024-09-16T07:33:32.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.966+0000 7fd607600640 1 -- 172.21.15.71:0/380335655 wait complete. 2024-09-16T07:33:32.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.967+0000 7fd607600640 1 Processor -- start 2024-09-16T07:33:32.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.967+0000 7fd607600640 1 -- start start 2024-09-16T07:33:32.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.968+0000 7fd607600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd60006efa0 0x7fd60013e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.968+0000 7fd607600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6000cccd0 0x7fd6001e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.968+0000 7fd607600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600145750 0x7fd6001e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.968+0000 7fd607600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000ce480 con 0x7fd60006efa0 2024-09-16T07:33:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.968+0000 7fd607600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000ce300 con 0x7fd6000cccd0 2024-09-16T07:33:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.968+0000 7fd607600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd6000ce600 con 0x7fd600145750 2024-09-16T07:33:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd60006efa0 0x7fd60013e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd60006efa0 0x7fd60013e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47786/0 (socket says 172.21.15.71:47786) 2024-09-16T07:33:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 -- 172.21.15.71:0/1347166651 learned_addr learned my addr 172.21.15.71:0/1347166651 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd605600640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600145750 0x7fd6001e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd5f7e00640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6000cccd0 0x7fd6001e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 -- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600145750 msgr2=0x7fd6001e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600145750 0x7fd6001e7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 -- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6000cccd0 msgr2=0x7fd6001e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6000cccd0 0x7fd6001e7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 -- 172.21.15.71:0/1347166651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60013e960 con 0x7fd60006efa0 2024-09-16T07:33:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.969+0000 7fd604c00640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd60006efa0 0x7fd60013e420 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000e9f0 tx=0x7fd5f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.970+0000 7fd5fd600640 1 -- 172.21.15.71:0/1347166651 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000cd80 con 0x7fd60006efa0 2024-09-16T07:33:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.970+0000 7fd5fd600640 1 -- 172.21.15.71:0/1347166651 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5f0004020 con 0x7fd60006efa0 2024-09-16T07:33:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.970+0000 7fd5fd600640 1 -- 172.21.15.71:0/1347166651 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0004470 con 0x7fd60006efa0 2024-09-16T07:33:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.970+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd600140e50 con 0x7fd60006efa0 2024-09-16T07:33:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.970+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd600141410 con 0x7fd60006efa0 2024-09-16T07:33:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.971+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6000cdbb0 con 0x7fd60006efa0 2024-09-16T07:33:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.972+0000 7fd5fd600640 1 -- 172.21.15.71:0/1347166651 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fd5f00035f0 con 0x7fd60006efa0 2024-09-16T07:33:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.973+0000 7fd5fd600640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd5dc079e40 0x7fd5dc07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.973+0000 7fd5fd600640 1 -- 172.21.15.71:0/1347166651 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5737+0+0 (secure 0 0 0) 0x7fd5f009a480 con 0x7fd60006efa0 2024-09-16T07:33:32.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.973+0000 7fd5f7e00640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd5dc079e40 0x7fd5dc07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:32.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.974+0000 7fd5f7e00640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd5dc079e40 0x7fd5dc07c300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8007fd0 tx=0x7fd5e8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:32.977+0000 7fd5fd600640 1 -- 172.21.15.71:0/1347166651 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd5f009a920 con 0x7fd60006efa0 2024-09-16T07:33:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:33.202+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd6000b3e60 con 0x7fd60006efa0 2024-09-16T07:33:33.994 INFO:tasks.ceph.mds.a.smithi071.stderr:2024-09-16T07:33:33.995+0000 7f248c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:33:34.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.035+0000 7fd5fd600640 1 -- 172.21.15.71:0/1347166651 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) ==== 118+0+0 (secure 0 0 0) 0x7fd5f0063ae0 con 0x7fd60006efa0 2024-09-16T07:33:34.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd5dc079e40 msgr2=0x7fd5dc07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:34.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd5dc079e40 0x7fd5dc07c300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8007fd0 tx=0x7fd5e8009040 comp rx=0 tx=0).stop 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd60006efa0 msgr2=0x7fd60013e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd60006efa0 0x7fd60013e420 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000e9f0 tx=0x7fd5f000eeb0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 shutdown_connections 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd5dc079e40 0x7fd5dc07c300 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600145750 0x7fd6001e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6000cccd0 0x7fd6001e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 --2- 172.21.15.71:0/1347166651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd60006efa0 0x7fd60013e420 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 >> 172.21.15.71:0/1347166651 conn(0x7fd6000b1840 msgr2=0x7fd6000d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 shutdown_connections 2024-09-16T07:33:34.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.039+0000 7fd607600640 1 -- 172.21.15.71:0/1347166651 wait complete. 2024-09-16T07:33:34.057 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:33:34.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:33:34.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.221+0000 7fe414200640 1 -- 172.21.15.71:0/3213022971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c0cccd0 msgr2=0x7fe40c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:34.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.221+0000 7fe414200640 1 --2- 172.21.15.71:0/3213022971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c0cccd0 0x7fe40c073440 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009830 tx=0x7fe3f802a480 comp rx=0 tx=0).stop 2024-09-16T07:33:34.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.221+0000 7fe414200640 1 -- 172.21.15.71:0/3213022971 shutdown_connections 2024-09-16T07:33:34.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.221+0000 7fe414200640 1 --2- 172.21.15.71:0/3213022971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe40c06efa0 0x7fe40c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.221+0000 7fe414200640 1 --2- 172.21.15.71:0/3213022971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c0cccd0 0x7fe40c073440 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.221+0000 7fe414200640 1 -- 172.21.15.71:0/3213022971 >> 172.21.15.71:0/3213022971 conn(0x7fe40c0b1840 msgr2=0x7fe40c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:34.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.221+0000 7fe414200640 1 -- 172.21.15.71:0/3213022971 shutdown_connections 2024-09-16T07:33:34.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.221+0000 7fe414200640 1 -- 172.21.15.71:0/3213022971 wait complete. 2024-09-16T07:33:34.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.223+0000 7fe414200640 1 Processor -- start 2024-09-16T07:33:34.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.223+0000 7fe414200640 1 -- start start 2024-09-16T07:33:34.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe414200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c06efa0 0x7fe40c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:34.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe414200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe40c0cccd0 0x7fe40c13d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:34.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe414200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe40c145750 0x7fe40c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:34.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe414200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c0ce480 con 0x7fe40c06efa0 2024-09-16T07:33:34.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe414200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c0ce300 con 0x7fe40c0cccd0 2024-09-16T07:33:34.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe414200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe40c0ce600 con 0x7fe40c145750 2024-09-16T07:33:34.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe411800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c06efa0 0x7fe40c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:34.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe411800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c06efa0 0x7fe40c15e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47804/0 (socket says 172.21.15.71:47804) 2024-09-16T07:33:34.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe411800640 1 -- 172.21.15.71:0/1383029705 learned_addr learned my addr 172.21.15.71:0/1383029705 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:34.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe412200640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe40c145750 0x7fe40c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:34.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe410e00640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe40c0cccd0 0x7fe40c13d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:34.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe411800640 1 -- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe40c145750 msgr2=0x7fe40c0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:34.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe411800640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe40c145750 0x7fe40c0c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe411800640 1 -- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe40c0cccd0 msgr2=0x7fe40c13d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:34.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe411800640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe40c0cccd0 0x7fe40c13d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:34.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.224+0000 7fe411800640 1 -- 172.21.15.71:0/1383029705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c1f85a0 con 0x7fe40c06efa0 2024-09-16T07:33:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.225+0000 7fe410e00640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe40c0cccd0 0x7fe40c13d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.225+0000 7fe411800640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c06efa0 0x7fe40c15e6f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c3c0 tx=0x7fe40000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.225+0000 7fe409600640 1 -- 172.21.15.71:0/1383029705 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400017070 con 0x7fe40c06efa0 2024-09-16T07:33:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.225+0000 7fe409600640 1 -- 172.21.15.71:0/1383029705 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4000035f0 con 0x7fe40c06efa0 2024-09-16T07:33:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.225+0000 7fe409600640 1 -- 172.21.15.71:0/1383029705 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400003a80 con 0x7fe40c06efa0 2024-09-16T07:33:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.225+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c1f8880 con 0x7fe40c06efa0 2024-09-16T07:33:34.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.225+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c1f8e40 con 0x7fe40c06efa0 2024-09-16T07:33:34.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.226+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40c0cdbb0 con 0x7fe40c06efa0 2024-09-16T07:33:34.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.227+0000 7fe409600640 1 -- 172.21.15.71:0/1383029705 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fe40001a070 con 0x7fe40c06efa0 2024-09-16T07:33:34.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.228+0000 7fe409600640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3ec079e40 0x7fe3ec07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:34.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.228+0000 7fe409600640 1 -- 172.21.15.71:0/1383029705 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fe400098cd0 con 0x7fe40c06efa0 2024-09-16T07:33:34.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.229+0000 7fe410e00640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3ec079e40 0x7fe3ec07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:34.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.230+0000 7fe410e00640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3ec079e40 0x7fe3ec07c300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8034000 tx=0x7fe3f8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:34.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.233+0000 7fe409600640 1 -- 172.21.15.71:0/1383029705 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe400062330 con 0x7fe40c06efa0 2024-09-16T07:33:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:34.453+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe40c06ece0 con 0x7fe40c06efa0 2024-09-16T07:33:35.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.048+0000 7fe409600640 1 -- 172.21.15.71:0/1383029705 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) ==== 122+0+0 (secure 0 0 0) 0x7fe400066d60 con 0x7fe40c06efa0 2024-09-16T07:33:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3ec079e40 msgr2=0x7fe3ec07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3ec079e40 0x7fe3ec07c300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8034000 tx=0x7fe3f8004730 comp rx=0 tx=0).stop 2024-09-16T07:33:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c06efa0 msgr2=0x7fe40c15e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c06efa0 0x7fe40c15e6f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c3c0 tx=0x7fe40000c880 comp rx=0 tx=0).stop 2024-09-16T07:33:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 shutdown_connections 2024-09-16T07:33:35.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe3ec079e40 0x7fe3ec07c300 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe40c145750 0x7fe40c0c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe40c0cccd0 0x7fe40c13d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 --2- 172.21.15.71:0/1383029705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40c06efa0 0x7fe40c15e6f0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 >> 172.21.15.71:0/1383029705 conn(0x7fe40c0b1840 msgr2=0x7fe40c0aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 shutdown_connections 2024-09-16T07:33:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.051+0000 7fe414200640 1 -- 172.21.15.71:0/1383029705 wait complete. 2024-09-16T07:33:35.068 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:33:35.069 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:33:35.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 -- 172.21.15.71:0/2825889703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f119415f5a0 msgr2=0x7f11940b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 --2- 172.21.15.71:0/2825889703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f119415f5a0 0x7f11940b2440 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f11940cfbb0 tx=0x7f118402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 -- 172.21.15.71:0/2825889703 shutdown_connections 2024-09-16T07:33:35.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 --2- 172.21.15.71:0/2825889703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11940ad270 0x7f1194073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 --2- 172.21.15.71:0/2825889703 >> v2:172.21.15.177:3300/0 conn(0x7f11940ccf00 0x7f11940acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 --2- 172.21.15.71:0/2825889703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f119415f5a0 0x7f11940b2440 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 -- 172.21.15.71:0/2825889703 >> 172.21.15.71:0/2825889703 conn(0x7f11940b1950 msgr2=0x7f11940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:35.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 -- 172.21.15.71:0/2825889703 shutdown_connections 2024-09-16T07:33:35.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.227+0000 7f119a600640 1 -- 172.21.15.71:0/2825889703 wait complete. 2024-09-16T07:33:35.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.229+0000 7f119a600640 1 Processor -- start 2024-09-16T07:33:35.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.229+0000 7f119a600640 1 -- start start 2024-09-16T07:33:35.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f119a600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11940ad270 0x7f11941464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:35.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f119a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11940ccf00 0x7f11940b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f119a600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f119415f5a0 0x7f11940b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f119a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1194074880 con 0x7f11940ccf00 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f119a600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1194074700 con 0x7f11940ad270 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f119a600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f119406f160 con 0x7f119415f5a0 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f1192a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11940ccf00 0x7f11940b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f1192a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11940ccf00 0x7f11940b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42204/0 (socket says 172.21.15.71:42204) 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f1192a00640 1 -- 172.21.15.71:0/4175711623 learned_addr learned my addr 172.21.15.71:0/4175711623 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f1193e00640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f119415f5a0 0x7f11940b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.230+0000 7f1193400640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11940ad270 0x7f11941464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f1192a00640 1 -- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f119415f5a0 msgr2=0x7f11940b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f1192a00640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f119415f5a0 0x7f11940b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f1192a00640 1 -- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11940ad270 msgr2=0x7f11941464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f1192a00640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11940ad270 0x7f11941464a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f1192a00640 1 -- 172.21.15.71:0/4175711623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1194150070 con 0x7f11940ccf00 2024-09-16T07:33:35.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f1192a00640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11940ccf00 0x7f11940b2d50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f118000d950 tx=0x7f118000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:35.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f118b400640 1 -- 172.21.15.71:0/4175711623 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180017070 con 0x7f11940ccf00 2024-09-16T07:33:35.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f118b400640 1 -- 172.21.15.71:0/4175711623 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1180009d20 con 0x7f11940ccf00 2024-09-16T07:33:35.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f118b400640 1 -- 172.21.15.71:0/4175711623 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800106c0 con 0x7f11940ccf00 2024-09-16T07:33:35.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.231+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119414d6b0 con 0x7f11940ccf00 2024-09-16T07:33:35.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.232+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1194150600 con 0x7f11940ccf00 2024-09-16T07:33:35.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.232+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1194073fb0 con 0x7f11940ccf00 2024-09-16T07:33:35.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.234+0000 7f118b400640 1 -- 172.21.15.71:0/4175711623 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f1180010860 con 0x7f11940ccf00 2024-09-16T07:33:35.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.234+0000 7f118b400640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1174079e70 0x7f117407c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:35.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.235+0000 7f118b400640 1 -- 172.21.15.71:0/4175711623 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f1180099e10 con 0x7f11940ccf00 2024-09-16T07:33:35.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.235+0000 7f1193400640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1174079e70 0x7f117407c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:35.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.236+0000 7f1193400640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1174079e70 0x7f117407c330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1184033000 tx=0x7f1184004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:35.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.238+0000 7f118b400640 1 -- 172.21.15.71:0/4175711623 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1180063470 con 0x7f11940ccf00 2024-09-16T07:33:35.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.459+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11940afb80 con 0x7f11940ccf00 2024-09-16T07:33:35.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.459+0000 7f118b400640 1 -- 172.21.15.71:0/4175711623 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5231 (secure 0 0 0) 0x7f1180067ea0 con 0x7f11940ccf00 2024-09-16T07:33:35.459 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:35.459 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":66,"btime":"2024-09-16T07:33:35:043973+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3180037555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3180037555},{"type":"v1","addr":"172.21.15.71:6835","nonce":3180037555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3241229928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3241229928},{"type":"v1","addr":"172.21.15.177:6836","nonce":3241229928}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1569295783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1569295783},{"type":"v1","addr":"172.21.15.177:6837","nonce":1569295783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:33:31.441505+0000","modified":"2024-09-16T07:33:34.454314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/4054989006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4054989006},{"type":"v1","addr":"172.21.15.71:6833","nonce":4054989006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-09-16T07:33:35.459 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 66 2024-09-16T07:33:35.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1174079e70 msgr2=0x7f117407c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1174079e70 0x7f117407c330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1184033000 tx=0x7f1184004730 comp rx=0 tx=0).stop 2024-09-16T07:33:35.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11940ccf00 msgr2=0x7f11940b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11940ccf00 0x7f11940b2d50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f118000d950 tx=0x7f118000de10 comp rx=0 tx=0).stop 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 shutdown_connections 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1174079e70 0x7f117407c330 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f119415f5a0 0x7f11940b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11940ccf00 0x7f11940b2d50 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.461+0000 7f119a600640 1 --2- 172.21.15.71:0/4175711623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11940ad270 0x7f11941464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.462+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 >> 172.21.15.71:0/4175711623 conn(0x7f11940b1950 msgr2=0x7f119409c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.462+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 shutdown_connections 2024-09-16T07:33:35.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.462+0000 7f119a600640 1 -- 172.21.15.71:0/4175711623 wait complete. 2024-09-16T07:33:35.477 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:33:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.633+0000 7f9c2d800640 1 -- 172.21.15.71:0/2102809054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c280ccf00 msgr2=0x7f9c280b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.633+0000 7f9c2d800640 1 --2- 172.21.15.71:0/2102809054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c280ccf00 0x7f9c280b2440 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009830 tx=0x7f9c1402a530 comp rx=0 tx=0).stop 2024-09-16T07:33:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.633+0000 7f9c2d800640 1 -- 172.21.15.71:0/2102809054 shutdown_connections 2024-09-16T07:33:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.633+0000 7f9c2d800640 1 --2- 172.21.15.71:0/2102809054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c280ccf00 0x7f9c280b2440 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.633+0000 7f9c2d800640 1 -- 172.21.15.71:0/2102809054 >> 172.21.15.71:0/2102809054 conn(0x7f9c280b1950 msgr2=0x7f9c280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.633+0000 7f9c2d800640 1 -- 172.21.15.71:0/2102809054 shutdown_connections 2024-09-16T07:33:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.633+0000 7f9c2d800640 1 -- 172.21.15.71:0/2102809054 wait complete. 2024-09-16T07:33:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.635+0000 7f9c2d800640 1 Processor -- start 2024-09-16T07:33:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.635+0000 7f9c2d800640 1 -- start start 2024-09-16T07:33:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c2d800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c28073fb0 0x7f9c280b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c2d800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c280ccf00 0x7f9c2809cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c2d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2815f5a0 0x7f9c280b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c2d800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2806f9f0 con 0x7f9c2815f5a0 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c2d800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2806f870 con 0x7f9c280ccf00 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c2d800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9c28074420 con 0x7f9c28073fb0 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c27400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2815f5a0 0x7f9c280b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c27400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2815f5a0 0x7f9c280b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42224/0 (socket says 172.21.15.71:42224) 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c27400640 1 -- 172.21.15.71:0/1015901008 learned_addr learned my addr 172.21.15.71:0/1015901008 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c26000640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c280ccf00 0x7f9c2809cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c26a00640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c28073fb0 0x7f9c280b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c27400640 1 -- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c28073fb0 msgr2=0x7f9c280b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c27400640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c28073fb0 0x7f9c280b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c27400640 1 -- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c280ccf00 msgr2=0x7f9c2809cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c27400640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c280ccf00 0x7f9c2809cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c27400640 1 -- 172.21.15.71:0/1015901008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c281f7df0 con 0x7f9c2815f5a0 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.636+0000 7f9c26a00640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c28073fb0 0x7f9c280b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.637+0000 7f9c27400640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2815f5a0 0x7f9c280b4570 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000c7a0 tx=0x7f9c2000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.637+0000 7f9c1ea00640 1 -- 172.21.15.71:0/1015901008 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20003b70 con 0x7f9c2815f5a0 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.637+0000 7f9c1ea00640 1 -- 172.21.15.71:0/1015901008 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c20003d10 con 0x7f9c2815f5a0 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.637+0000 7f9c1ea00640 1 -- 172.21.15.71:0/1015901008 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2000aab0 con 0x7f9c2815f5a0 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.637+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c281f80d0 con 0x7f9c2815f5a0 2024-09-16T07:33:35.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.637+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c281f8690 con 0x7f9c2815f5a0 2024-09-16T07:33:35.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.638+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c280b2440 con 0x7f9c2815f5a0 2024-09-16T07:33:35.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.639+0000 7f9c1ea00640 1 -- 172.21.15.71:0/1015901008 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f9c2000ac50 con 0x7f9c2815f5a0 2024-09-16T07:33:35.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.640+0000 7f9c1ea00640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c08079e40 0x7f9c0807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:35.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.640+0000 7f9c1ea00640 1 -- 172.21.15.71:0/1015901008 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f9c20099870 con 0x7f9c2815f5a0 2024-09-16T07:33:35.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.641+0000 7f9c26a00640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c08079e40 0x7f9c0807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:35.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.641+0000 7f9c26a00640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c08079e40 0x7f9c0807c300 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9c280b14f0 tx=0x7f9c18009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:35.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.644+0000 7f9c1ea00640 1 -- 172.21.15.71:0/1015901008 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c20063520 con 0x7f9c2815f5a0 2024-09-16T07:33:35.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.831+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c281f4620 con 0x7f9c2815f5a0 2024-09-16T07:33:35.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.832+0000 7f9c1ea00640 1 -- 172.21.15.71:0/1015901008 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16734 (secure 0 0 0) 0x7f9c20062e20 con 0x7f9c2815f5a0 2024-09-16T07:33:35.832 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:35.832 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":105,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:33:33.017724+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:33:29.200091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T07:33:30.405982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:33:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c08079e40 msgr2=0x7f9c0807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c08079e40 0x7f9c0807c300 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9c280b14f0 tx=0x7f9c18009040 comp rx=0 tx=0).stop 2024-09-16T07:33:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2815f5a0 msgr2=0x7f9c280b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2815f5a0 0x7f9c280b4570 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000c7a0 tx=0x7f9c2000cc60 comp rx=0 tx=0).stop 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 shutdown_connections 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c08079e40 0x7f9c0807c300 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2815f5a0 0x7f9c280b4570 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c280ccf00 0x7f9c2809cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.834+0000 7f9c2d800640 1 --2- 172.21.15.71:0/1015901008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c28073fb0 0x7f9c280b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.835+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 >> 172.21.15.71:0/1015901008 conn(0x7f9c280b1950 msgr2=0x7f9c28075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.835+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 shutdown_connections 2024-09-16T07:33:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:35.835+0000 7f9c2d800640 1 -- 172.21.15.71:0/1015901008 wait complete. 2024-09-16T07:33:35.850 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:33:35.851 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:33:36.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 -- 172.21.15.71:0/4265371061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f40ad270 msgr2=0x7f97f4073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 --2- 172.21.15.71:0/4265371061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f40ad270 0x7f97f4073fb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f97f4075130 tx=0x7f97ec02a980 comp rx=0 tx=0).stop 2024-09-16T07:33:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 -- 172.21.15.71:0/4265371061 shutdown_connections 2024-09-16T07:33:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 --2- 172.21.15.71:0/4265371061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f40ad270 0x7f97f4073fb0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 --2- 172.21.15.71:0/4265371061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97f40ccf00 0x7f97f40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 --2- 172.21.15.71:0/4265371061 >> v2:172.21.15.177:3300/0 conn(0x7f97f415f5a0 0x7f97f40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 -- 172.21.15.71:0/4265371061 >> 172.21.15.71:0/4265371061 conn(0x7f97f40b1950 msgr2=0x7f97f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 -- 172.21.15.71:0/4265371061 shutdown_connections 2024-09-16T07:33:36.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.007+0000 7f97f8a00640 1 -- 172.21.15.71:0/4265371061 wait complete. 2024-09-16T07:33:36.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.009+0000 7f97f8a00640 1 Processor -- start 2024-09-16T07:33:36.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.009+0000 7f97f8a00640 1 -- start start 2024-09-16T07:33:36.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.010+0000 7f97f8a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97f40ad270 0x7f97f41f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.010+0000 7f97f8a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97f40ccf00 0x7f97f41e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.010+0000 7f97f8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f415f5a0 0x7f97f41441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.010+0000 7f97f8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f97f406f160 con 0x7f97f415f5a0 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.010+0000 7f97f8a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4074780 con 0x7f97f40ccf00 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.010+0000 7f97f8a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f97f406f2e0 con 0x7f97f40ad270 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.010+0000 7f97f2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f415f5a0 0x7f97f41441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f415f5a0 0x7f97f41441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42240/0 (socket says 172.21.15.71:42240) 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f2800640 1 -- 172.21.15.71:0/1730559234 learned_addr learned my addr 172.21.15.71:0/1730559234 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f1400640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97f40ccf00 0x7f97f41e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f1e00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97f40ad270 0x7f97f41f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f2800640 1 -- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97f40ad270 msgr2=0x7f97f41f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f2800640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97f40ad270 0x7f97f41f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f2800640 1 -- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97f40ccf00 msgr2=0x7f97f41e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f2800640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97f40ccf00 0x7f97f41e8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f2800640 1 -- 172.21.15.71:0/1730559234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f40b35c0 con 0x7f97f415f5a0 2024-09-16T07:33:36.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97f2800640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f415f5a0 0x7f97f41441a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009d60 tx=0x7f97ec003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:36.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.011+0000 7f97ea000640 1 -- 172.21.15.71:0/1730559234 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec034050 con 0x7f97f415f5a0 2024-09-16T07:33:36.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.012+0000 7f97ea000640 1 -- 172.21.15.71:0/1730559234 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97ec004450 con 0x7f97f415f5a0 2024-09-16T07:33:36.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.012+0000 7f97ea000640 1 -- 172.21.15.71:0/1730559234 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec043440 con 0x7f97f415f5a0 2024-09-16T07:33:36.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.012+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f40b38a0 con 0x7f97f415f5a0 2024-09-16T07:33:36.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.012+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f40b3e60 con 0x7f97f415f5a0 2024-09-16T07:33:36.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.013+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f415b680 con 0x7f97f415f5a0 2024-09-16T07:33:36.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.014+0000 7f97ea000640 1 -- 172.21.15.71:0/1730559234 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f97ec037020 con 0x7f97f415f5a0 2024-09-16T07:33:36.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.015+0000 7f97ea000640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97d4079e40 0x7f97d407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.015+0000 7f97ea000640 1 -- 172.21.15.71:0/1730559234 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f97ec0ba850 con 0x7f97f415f5a0 2024-09-16T07:33:36.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.015+0000 7f97f1e00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97d4079e40 0x7f97d407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.016+0000 7f97f1e00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97d4079e40 0x7f97d407c300 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009b30 tx=0x7f97e4005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:36.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.019+0000 7f97ea000640 1 -- 172.21.15.71:0/1730559234 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f97f415b680 con 0x7f97f415f5a0 2024-09-16T07:33:36.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.204+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97f406f810 con 0x7f97d4079e40 2024-09-16T07:33:36.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.210+0000 7f97ea000640 1 -- 172.21.15.71:0/1730559234 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251762 (secure 0 0 0) 0x7f97f406f810 con 0x7f97d4079e40 2024-09-16T07:33:36.212 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:36.214 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:33:36.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97d4079e40 msgr2=0x7f97d407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97d4079e40 0x7f97d407c300 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009b30 tx=0x7f97e4005f90 comp rx=0 tx=0).stop 2024-09-16T07:33:36.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f415f5a0 msgr2=0x7f97f41441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f415f5a0 0x7f97f41441a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009d60 tx=0x7f97ec003f10 comp rx=0 tx=0).stop 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 shutdown_connections 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f97d4079e40 0x7f97d407c300 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f415f5a0 0x7f97f41441a0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97f40ccf00 0x7f97f41e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 --2- 172.21.15.71:0/1730559234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97f40ad270 0x7f97f41f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 >> 172.21.15.71:0/1730559234 conn(0x7f97f40b1950 msgr2=0x7f97f409cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 shutdown_connections 2024-09-16T07:33:36.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.214+0000 7f97f8a00640 1 -- 172.21.15.71:0/1730559234 wait complete. 2024-09-16T07:33:36.230 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-16T07:33:34.580625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":964060,"kb_used_data":11172,"kb_used_omap":12,"kb_used_meta":952691,"kb_avail":754010660,"statfs":{"total":773094113280,"available":772106915840,"internally_reserved":0,"allocated":11440128,"data_stored":5612724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":975556144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.570999"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025017+0000","last_change":"2024-09-16T07:33:32.045068+0000","last_active":"2024-09-16T07:33:33.025017+0000","last_peered":"2024-09-16T07:33:33.025017+0000","last_clean":"2024-09-16T07:33:33.025017+0000","last_became_active":"2024-09-16T07:33:32.044772+0000","last_became_peered":"2024-09-16T07:33:32.044772+0000","last_unstale":"2024-09-16T07:33:33.025017+0000","last_undegraded":"2024-09-16T07:33:33.025017+0000","last_fullsized":"2024-09-16T07:33:33.025017+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024784+0000","last_change":"2024-09-16T07:33:32.049940+0000","last_active":"2024-09-16T07:33:33.024784+0000","last_peered":"2024-09-16T07:33:33.024784+0000","last_clean":"2024-09-16T07:33:33.024784+0000","last_became_active":"2024-09-16T07:33:32.049646+0000","last_became_peered":"2024-09-16T07:33:32.049646+0000","last_unstale":"2024-09-16T07:33:33.024784+0000","last_undegraded":"2024-09-16T07:33:33.024784+0000","last_fullsized":"2024-09-16T07:33:33.024784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.031577+0000","last_change":"2024-09-16T07:33:31.031577+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.031577+0000","last_undegraded":"2024-09-16T07:33:31.031577+0000","last_fullsized":"2024-09-16T07:33:31.031577+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024064+0000","last_change":"2024-09-16T07:33:32.043883+0000","last_active":"2024-09-16T07:33:33.024064+0000","last_peered":"2024-09-16T07:33:33.024064+0000","last_clean":"2024-09-16T07:33:33.024064+0000","last_became_active":"2024-09-16T07:33:32.043622+0000","last_became_peered":"2024-09-16T07:33:32.043622+0000","last_unstale":"2024-09-16T07:33:33.024064+0000","last_undegraded":"2024-09-16T07:33:33.024064+0000","last_fullsized":"2024-09-16T07:33:33.024064+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:01:49.570122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357403+0000","last_change":"2024-09-16T07:33:32.043899+0000","last_active":"2024-09-16T07:33:33.357403+0000","last_peered":"2024-09-16T07:33:33.357403+0000","last_clean":"2024-09-16T07:33:33.357403+0000","last_became_active":"2024-09-16T07:33:32.043611+0000","last_became_peered":"2024-09-16T07:33:32.043611+0000","last_unstale":"2024-09-16T07:33:33.357403+0000","last_undegraded":"2024-09-16T07:33:33.357403+0000","last_fullsized":"2024-09-16T07:33:33.357403+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:22.213386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357942+0000","last_change":"2024-09-16T07:33:32.054833+0000","last_active":"2024-09-16T07:33:33.357942+0000","last_peered":"2024-09-16T07:33:33.357942+0000","last_clean":"2024-09-16T07:33:33.357942+0000","last_became_active":"2024-09-16T07:33:32.054513+0000","last_became_peered":"2024-09-16T07:33:32.054513+0000","last_unstale":"2024-09-16T07:33:33.357942+0000","last_undegraded":"2024-09-16T07:33:33.357942+0000","last_fullsized":"2024-09-16T07:33:33.357942+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:03.590298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357327+0000","last_change":"2024-09-16T07:33:32.037656+0000","last_active":"2024-09-16T07:33:33.357327+0000","last_peered":"2024-09-16T07:33:33.357327+0000","last_clean":"2024-09-16T07:33:33.357327+0000","last_became_active":"2024-09-16T07:33:32.036949+0000","last_became_peered":"2024-09-16T07:33:32.036949+0000","last_unstale":"2024-09-16T07:33:33.357327+0000","last_undegraded":"2024-09-16T07:33:33.357327+0000","last_fullsized":"2024-09-16T07:33:33.357327+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:18.915383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.025834+0000","last_change":"2024-09-16T07:33:31.025834+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.025834+0000","last_undegraded":"2024-09-16T07:33:31.025834+0000","last_fullsized":"2024-09-16T07:33:31.025834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357756+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.357756+0000","last_peered":"2024-09-16T07:33:33.357756+0000","last_clean":"2024-09-16T07:33:33.357756+0000","last_became_active":"2024-09-16T07:33:32.035287+0000","last_became_peered":"2024-09-16T07:33:32.035287+0000","last_unstale":"2024-09-16T07:33:33.357756+0000","last_undegraded":"2024-09-16T07:33:33.357756+0000","last_fullsized":"2024-09-16T07:33:33.357756+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:52.049650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021843+0000","last_change":"2024-09-16T07:33:32.049217+0000","last_active":"2024-09-16T07:33:33.021843+0000","last_peered":"2024-09-16T07:33:33.021843+0000","last_clean":"2024-09-16T07:33:33.021843+0000","last_became_active":"2024-09-16T07:33:32.048953+0000","last_became_peered":"2024-09-16T07:33:32.048953+0000","last_unstale":"2024-09-16T07:33:33.021843+0000","last_undegraded":"2024-09-16T07:33:33.021843+0000","last_fullsized":"2024-09-16T07:33:33.021843+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357715+0000","last_change":"2024-09-16T07:33:32.054961+0000","last_active":"2024-09-16T07:33:33.357715+0000","last_peered":"2024-09-16T07:33:33.357715+0000","last_clean":"2024-09-16T07:33:33.357715+0000","last_became_active":"2024-09-16T07:33:32.054463+0000","last_became_peered":"2024-09-16T07:33:32.054463+0000","last_unstale":"2024-09-16T07:33:33.357715+0000","last_undegraded":"2024-09-16T07:33:33.357715+0000","last_fullsized":"2024-09-16T07:33:33.357715+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:57.790799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358257+0000","last_change":"2024-09-16T07:33:32.040365+0000","last_active":"2024-09-16T07:33:33.358257+0000","last_peered":"2024-09-16T07:33:33.358257+0000","last_clean":"2024-09-16T07:33:33.358257+0000","last_became_active":"2024-09-16T07:33:32.037953+0000","last_became_peered":"2024-09-16T07:33:32.037953+0000","last_unstale":"2024-09-16T07:33:33.358257+0000","last_undegraded":"2024-09-16T07:33:33.358257+0000","last_fullsized":"2024-09-16T07:33:33.358257+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:32.112813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.059035+0000","last_change":"2024-09-16T07:33:32.056887+0000","last_active":"2024-09-16T07:33:32.059035+0000","last_peered":"2024-09-16T07:33:32.059035+0000","last_clean":"2024-09-16T07:33:32.059035+0000","last_became_active":"2024-09-16T07:33:32.056276+0000","last_became_peered":"2024-09-16T07:33:32.056276+0000","last_unstale":"2024-09-16T07:33:32.059035+0000","last_undegraded":"2024-09-16T07:33:32.059035+0000","last_fullsized":"2024-09-16T07:33:32.059035+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:23.723158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023495+0000","last_change":"2024-09-16T07:33:32.054384+0000","last_active":"2024-09-16T07:33:33.023495+0000","last_peered":"2024-09-16T07:33:33.023495+0000","last_clean":"2024-09-16T07:33:33.023495+0000","last_became_active":"2024-09-16T07:33:32.051888+0000","last_became_peered":"2024-09-16T07:33:32.051888+0000","last_unstale":"2024-09-16T07:33:33.023495+0000","last_undegraded":"2024-09-16T07:33:33.023495+0000","last_fullsized":"2024-09-16T07:33:33.023495+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:19.796812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023506+0000","last_change":"2024-09-16T07:33:32.040802+0000","last_active":"2024-09-16T07:33:33.023506+0000","last_peered":"2024-09-16T07:33:33.023506+0000","last_clean":"2024-09-16T07:33:33.023506+0000","last_became_active":"2024-09-16T07:33:32.040510+0000","last_became_peered":"2024-09-16T07:33:32.040510+0000","last_unstale":"2024-09-16T07:33:33.023506+0000","last_undegraded":"2024-09-16T07:33:33.023506+0000","last_fullsized":"2024-09-16T07:33:33.023506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:53:27.647859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357155+0000","last_change":"2024-09-16T07:33:32.039107+0000","last_active":"2024-09-16T07:33:33.357155+0000","last_peered":"2024-09-16T07:33:33.357155+0000","last_clean":"2024-09-16T07:33:33.357155+0000","last_became_active":"2024-09-16T07:33:32.038828+0000","last_became_peered":"2024-09-16T07:33:32.038828+0000","last_unstale":"2024-09-16T07:33:33.357155+0000","last_undegraded":"2024-09-16T07:33:33.357155+0000","last_fullsized":"2024-09-16T07:33:33.357155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:15.415755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025462+0000","last_change":"2024-09-16T07:33:32.053433+0000","last_active":"2024-09-16T07:33:33.025462+0000","last_peered":"2024-09-16T07:33:33.025462+0000","last_clean":"2024-09-16T07:33:33.025462+0000","last_became_active":"2024-09-16T07:33:32.052843+0000","last_became_peered":"2024-09-16T07:33:32.052843+0000","last_unstale":"2024-09-16T07:33:33.025462+0000","last_undegraded":"2024-09-16T07:33:33.025462+0000","last_fullsized":"2024-09-16T07:33:33.025462+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:13.265002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025375+0000","last_change":"2024-09-16T07:33:32.047389+0000","last_active":"2024-09-16T07:33:33.025375+0000","last_peered":"2024-09-16T07:33:33.025375+0000","last_clean":"2024-09-16T07:33:33.025375+0000","last_became_active":"2024-09-16T07:33:32.047043+0000","last_became_peered":"2024-09-16T07:33:32.047043+0000","last_unstale":"2024-09-16T07:33:33.025375+0000","last_undegraded":"2024-09-16T07:33:33.025375+0000","last_fullsized":"2024-09-16T07:33:33.025375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024761+0000","last_change":"2024-09-16T07:33:32.040875+0000","last_active":"2024-09-16T07:33:33.024761+0000","last_peered":"2024-09-16T07:33:33.024761+0000","last_clean":"2024-09-16T07:33:33.024761+0000","last_became_active":"2024-09-16T07:33:32.040636+0000","last_became_peered":"2024-09-16T07:33:32.040636+0000","last_unstale":"2024-09-16T07:33:33.024761+0000","last_undegraded":"2024-09-16T07:33:33.024761+0000","last_fullsized":"2024-09-16T07:33:33.024761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:06.934545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358353+0000","last_change":"2024-09-16T07:33:32.053936+0000","last_active":"2024-09-16T07:33:33.358353+0000","last_peered":"2024-09-16T07:33:33.358353+0000","last_clean":"2024-09-16T07:33:33.358353+0000","last_became_active":"2024-09-16T07:33:32.053546+0000","last_became_peered":"2024-09-16T07:33:32.053546+0000","last_unstale":"2024-09-16T07:33:33.358353+0000","last_undegraded":"2024-09-16T07:33:33.358353+0000","last_fullsized":"2024-09-16T07:33:33.358353+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.032081+0000","last_change":"2024-09-16T07:33:31.032081+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.032081+0000","last_undegraded":"2024-09-16T07:33:31.032081+0000","last_fullsized":"2024-09-16T07:33:31.032081+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027983+0000","last_change":"2024-09-16T07:33:31.027983+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027983+0000","last_undegraded":"2024-09-16T07:33:31.027983+0000","last_fullsized":"2024-09-16T07:33:31.027983+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023549+0000","last_change":"2024-09-16T07:33:32.055235+0000","last_active":"2024-09-16T07:33:33.023549+0000","last_peered":"2024-09-16T07:33:33.023549+0000","last_clean":"2024-09-16T07:33:33.023549+0000","last_became_active":"2024-09-16T07:33:32.052675+0000","last_became_peered":"2024-09-16T07:33:32.052675+0000","last_unstale":"2024-09-16T07:33:33.023549+0000","last_undegraded":"2024-09-16T07:33:33.023549+0000","last_fullsized":"2024-09-16T07:33:33.023549+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:31.315404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357746+0000","last_change":"2024-09-16T07:33:32.050486+0000","last_active":"2024-09-16T07:33:33.357746+0000","last_peered":"2024-09-16T07:33:33.357746+0000","last_clean":"2024-09-16T07:33:33.357746+0000","last_became_active":"2024-09-16T07:33:32.050108+0000","last_became_peered":"2024-09-16T07:33:32.050108+0000","last_unstale":"2024-09-16T07:33:33.357746+0000","last_undegraded":"2024-09-16T07:33:33.357746+0000","last_fullsized":"2024-09-16T07:33:33.357746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021955+0000","last_change":"2024-09-16T07:33:32.028117+0000","last_active":"2024-09-16T07:33:33.021955+0000","last_peered":"2024-09-16T07:33:33.021955+0000","last_clean":"2024-09-16T07:33:33.021955+0000","last_became_active":"2024-09-16T07:33:32.027826+0000","last_became_peered":"2024-09-16T07:33:32.027826+0000","last_unstale":"2024-09-16T07:33:33.021955+0000","last_undegraded":"2024-09-16T07:33:33.021955+0000","last_fullsized":"2024-09-16T07:33:33.021955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:35.757466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.055382+0000","last_change":"2024-09-16T07:33:32.049150+0000","last_active":"2024-09-16T07:33:32.055382+0000","last_peered":"2024-09-16T07:33:32.055382+0000","last_clean":"2024-09-16T07:33:32.055382+0000","last_became_active":"2024-09-16T07:33:32.048893+0000","last_became_peered":"2024-09-16T07:33:32.048893+0000","last_unstale":"2024-09-16T07:33:32.055382+0000","last_undegraded":"2024-09-16T07:33:32.055382+0000","last_fullsized":"2024-09-16T07:33:32.055382+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:37.326467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358162+0000","last_change":"2024-09-16T07:33:32.041462+0000","last_active":"2024-09-16T07:33:33.358162+0000","last_peered":"2024-09-16T07:33:33.358162+0000","last_clean":"2024-09-16T07:33:33.358162+0000","last_became_active":"2024-09-16T07:33:32.041198+0000","last_became_peered":"2024-09-16T07:33:32.041198+0000","last_unstale":"2024-09-16T07:33:33.358162+0000","last_undegraded":"2024-09-16T07:33:33.358162+0000","last_fullsized":"2024-09-16T07:33:33.358162+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358295+0000","last_change":"2024-09-16T07:33:32.037402+0000","last_active":"2024-09-16T07:33:33.358295+0000","last_peered":"2024-09-16T07:33:33.358295+0000","last_clean":"2024-09-16T07:33:33.358295+0000","last_became_active":"2024-09-16T07:33:32.037131+0000","last_became_peered":"2024-09-16T07:33:32.037131+0000","last_unstale":"2024-09-16T07:33:33.358295+0000","last_undegraded":"2024-09-16T07:33:33.358295+0000","last_fullsized":"2024-09-16T07:33:33.358295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.052439+0000","last_change":"2024-09-16T07:33:32.043318+0000","last_active":"2024-09-16T07:33:32.052439+0000","last_peered":"2024-09-16T07:33:32.052439+0000","last_clean":"2024-09-16T07:33:32.052439+0000","last_became_active":"2024-09-16T07:33:32.043039+0000","last_became_peered":"2024-09-16T07:33:32.043039+0000","last_unstale":"2024-09-16T07:33:32.052439+0000","last_undegraded":"2024-09-16T07:33:32.052439+0000","last_fullsized":"2024-09-16T07:33:32.052439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:45.700538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.059020+0000","last_change":"2024-09-16T07:33:32.056817+0000","last_active":"2024-09-16T07:33:32.059020+0000","last_peered":"2024-09-16T07:33:32.059020+0000","last_clean":"2024-09-16T07:33:32.059020+0000","last_became_active":"2024-09-16T07:33:32.056535+0000","last_became_peered":"2024-09-16T07:33:32.056535+0000","last_unstale":"2024-09-16T07:33:32.059020+0000","last_undegraded":"2024-09-16T07:33:32.059020+0000","last_fullsized":"2024-09-16T07:33:32.059020+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:35.876240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027968+0000","last_change":"2024-09-16T07:33:31.027968+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027968+0000","last_undegraded":"2024-09-16T07:33:31.027968+0000","last_fullsized":"2024-09-16T07:33:31.027968+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023806+0000","last_change":"2024-09-16T07:33:32.036224+0000","last_active":"2024-09-16T07:33:33.023806+0000","last_peered":"2024-09-16T07:33:33.023806+0000","last_clean":"2024-09-16T07:33:33.023806+0000","last_became_active":"2024-09-16T07:33:32.035020+0000","last_became_peered":"2024-09-16T07:33:32.035020+0000","last_unstale":"2024-09-16T07:33:33.023806+0000","last_undegraded":"2024-09-16T07:33:33.023806+0000","last_fullsized":"2024-09-16T07:33:33.023806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:00.997013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025689+0000","last_change":"2024-09-16T07:33:32.046309+0000","last_active":"2024-09-16T07:33:33.025689+0000","last_peered":"2024-09-16T07:33:33.025689+0000","last_clean":"2024-09-16T07:33:33.025689+0000","last_became_active":"2024-09-16T07:33:32.045999+0000","last_became_peered":"2024-09-16T07:33:32.045999+0000","last_unstale":"2024-09-16T07:33:33.025689+0000","last_undegraded":"2024-09-16T07:33:33.025689+0000","last_fullsized":"2024-09-16T07:33:33.025689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:09.445884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.053925+0000","last_change":"2024-09-16T07:33:31.045574+0000","last_active":"2024-09-16T07:33:32.053925+0000","last_peered":"2024-09-16T07:33:32.053925+0000","last_clean":"2024-09-16T07:33:32.053925+0000","last_became_active":"2024-09-16T07:33:31.045014+0000","last_became_peered":"2024-09-16T07:33:31.045014+0000","last_unstale":"2024-09-16T07:33:32.053925+0000","last_undegraded":"2024-09-16T07:33:32.053925+0000","last_fullsized":"2024-09-16T07:33:32.053925+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.581864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.052698+0000","last_change":"2024-09-16T07:33:32.046389+0000","last_active":"2024-09-16T07:33:32.052698+0000","last_peered":"2024-09-16T07:33:32.052698+0000","last_clean":"2024-09-16T07:33:32.052698+0000","last_became_active":"2024-09-16T07:33:32.046041+0000","last_became_peered":"2024-09-16T07:33:32.046041+0000","last_unstale":"2024-09-16T07:33:32.052698+0000","last_undegraded":"2024-09-16T07:33:32.052698+0000","last_fullsized":"2024-09-16T07:33:32.052698+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:20.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357662+0000","last_change":"2024-09-16T07:33:31.041205+0000","last_active":"2024-09-16T07:33:33.357662+0000","last_peered":"2024-09-16T07:33:33.357662+0000","last_clean":"2024-09-16T07:33:33.357662+0000","last_became_active":"2024-09-16T07:33:31.040923+0000","last_became_peered":"2024-09-16T07:33:31.040923+0000","last_unstale":"2024-09-16T07:33:33.357662+0000","last_undegraded":"2024-09-16T07:33:33.357662+0000","last_fullsized":"2024-09-16T07:33:33.357662+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:56.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024526+0000","last_change":"2024-09-16T07:33:32.039150+0000","last_active":"2024-09-16T07:33:33.024526+0000","last_peered":"2024-09-16T07:33:33.024526+0000","last_clean":"2024-09-16T07:33:33.024526+0000","last_became_active":"2024-09-16T07:33:32.038837+0000","last_became_peered":"2024-09-16T07:33:32.038837+0000","last_unstale":"2024-09-16T07:33:33.024526+0000","last_undegraded":"2024-09-16T07:33:33.024526+0000","last_fullsized":"2024-09-16T07:33:33.024526+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:52.257876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.046999+0000","last_change":"2024-09-16T07:33:31.045183+0000","last_active":"2024-09-16T07:33:31.046999+0000","last_peered":"2024-09-16T07:33:31.046999+0000","last_clean":"2024-09-16T07:33:31.046999+0000","last_became_active":"2024-09-16T07:33:31.044253+0000","last_became_peered":"2024-09-16T07:33:31.044253+0000","last_unstale":"2024-09-16T07:33:31.046999+0000","last_undegraded":"2024-09-16T07:33:31.046999+0000","last_fullsized":"2024-09-16T07:33:31.046999+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:15:28.325154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021838+0000","last_change":"2024-09-16T07:33:32.043094+0000","last_active":"2024-09-16T07:33:33.021838+0000","last_peered":"2024-09-16T07:33:33.021838+0000","last_clean":"2024-09-16T07:33:33.021838+0000","last_became_active":"2024-09-16T07:33:32.042832+0000","last_became_peered":"2024-09-16T07:33:32.042832+0000","last_unstale":"2024-09-16T07:33:33.021838+0000","last_undegraded":"2024-09-16T07:33:33.021838+0000","last_fullsized":"2024-09-16T07:33:33.021838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:24.827070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024550+0000","last_change":"2024-09-16T07:33:31.042458+0000","last_active":"2024-09-16T07:33:33.024550+0000","last_peered":"2024-09-16T07:33:33.024550+0000","last_clean":"2024-09-16T07:33:33.024550+0000","last_became_active":"2024-09-16T07:33:31.042226+0000","last_became_peered":"2024-09-16T07:33:31.042226+0000","last_unstale":"2024-09-16T07:33:33.024550+0000","last_undegraded":"2024-09-16T07:33:33.024550+0000","last_fullsized":"2024-09-16T07:33:33.024550+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:20.182053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357806+0000","last_change":"2024-09-16T07:33:32.039748+0000","last_active":"2024-09-16T07:33:33.357806+0000","last_peered":"2024-09-16T07:33:33.357806+0000","last_clean":"2024-09-16T07:33:33.357806+0000","last_became_active":"2024-09-16T07:33:32.038397+0000","last_became_peered":"2024-09-16T07:33:32.038397+0000","last_unstale":"2024-09-16T07:33:33.357806+0000","last_undegraded":"2024-09-16T07:33:33.357806+0000","last_fullsized":"2024-09-16T07:33:33.357806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:15.732874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023864+0000","last_change":"2024-09-16T07:33:31.033141+0000","last_active":"2024-09-16T07:33:33.023864+0000","last_peered":"2024-09-16T07:33:33.023864+0000","last_clean":"2024-09-16T07:33:33.023864+0000","last_became_active":"2024-09-16T07:33:31.032570+0000","last_became_peered":"2024-09-16T07:33:31.032570+0000","last_unstale":"2024-09-16T07:33:33.023864+0000","last_undegraded":"2024-09-16T07:33:33.023864+0000","last_fullsized":"2024-09-16T07:33:33.023864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:26.450637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357619+0000","last_change":"2024-09-16T07:33:32.044965+0000","last_active":"2024-09-16T07:33:33.357619+0000","last_peered":"2024-09-16T07:33:33.357619+0000","last_clean":"2024-09-16T07:33:33.357619+0000","last_became_active":"2024-09-16T07:33:32.044705+0000","last_became_peered":"2024-09-16T07:33:32.044705+0000","last_unstale":"2024-09-16T07:33:33.357619+0000","last_undegraded":"2024-09-16T07:33:33.357619+0000","last_fullsized":"2024-09-16T07:33:33.357619+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:50.022797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.040511+0000","last_change":"2024-09-16T07:33:31.035805+0000","last_active":"2024-09-16T07:33:31.040511+0000","last_peered":"2024-09-16T07:33:31.040511+0000","last_clean":"2024-09-16T07:33:31.040511+0000","last_became_active":"2024-09-16T07:33:31.035545+0000","last_became_peered":"2024-09-16T07:33:31.035545+0000","last_unstale":"2024-09-16T07:33:31.040511+0000","last_undegraded":"2024-09-16T07:33:31.040511+0000","last_fullsized":"2024-09-16T07:33:31.040511+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:16.651576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.053801+0000","last_change":"2024-09-16T07:33:32.051200+0000","last_active":"2024-09-16T07:33:32.053801+0000","last_peered":"2024-09-16T07:33:32.053801+0000","last_clean":"2024-09-16T07:33:32.053801+0000","last_became_active":"2024-09-16T07:33:32.047036+0000","last_became_peered":"2024-09-16T07:33:32.047036+0000","last_unstale":"2024-09-16T07:33:32.053801+0000","last_undegraded":"2024-09-16T07:33:32.053801+0000","last_fullsized":"2024-09-16T07:33:32.053801+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:23.177353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357424+0000","last_change":"2024-09-16T07:33:31.034485+0000","last_active":"2024-09-16T07:33:33.357424+0000","last_peered":"2024-09-16T07:33:33.357424+0000","last_clean":"2024-09-16T07:33:33.357424+0000","last_became_active":"2024-09-16T07:33:31.034236+0000","last_became_peered":"2024-09-16T07:33:31.034236+0000","last_unstale":"2024-09-16T07:33:33.357424+0000","last_undegraded":"2024-09-16T07:33:33.357424+0000","last_fullsized":"2024-09-16T07:33:33.357424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:52.133642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357028+0000","last_change":"2024-09-16T07:33:32.039686+0000","last_active":"2024-09-16T07:33:33.357028+0000","last_peered":"2024-09-16T07:33:33.357028+0000","last_clean":"2024-09-16T07:33:33.357028+0000","last_became_active":"2024-09-16T07:33:32.038343+0000","last_became_peered":"2024-09-16T07:33:32.038343+0000","last_unstale":"2024-09-16T07:33:33.357028+0000","last_undegraded":"2024-09-16T07:33:33.357028+0000","last_fullsized":"2024-09-16T07:33:33.357028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:00.455428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024348+0000","last_change":"2024-09-16T07:33:31.039439+0000","last_active":"2024-09-16T07:33:33.024348+0000","last_peered":"2024-09-16T07:33:33.024348+0000","last_clean":"2024-09-16T07:33:33.024348+0000","last_became_active":"2024-09-16T07:33:31.039029+0000","last_became_peered":"2024-09-16T07:33:31.039029+0000","last_unstale":"2024-09-16T07:33:33.024348+0000","last_undegraded":"2024-09-16T07:33:33.024348+0000","last_fullsized":"2024-09-16T07:33:33.024348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:21.209706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025089+0000","last_change":"2024-09-16T07:33:32.045355+0000","last_active":"2024-09-16T07:33:33.025089+0000","last_peered":"2024-09-16T07:33:33.025089+0000","last_clean":"2024-09-16T07:33:33.025089+0000","last_became_active":"2024-09-16T07:33:32.045070+0000","last_became_peered":"2024-09-16T07:33:32.045070+0000","last_unstale":"2024-09-16T07:33:33.025089+0000","last_undegraded":"2024-09-16T07:33:33.025089+0000","last_fullsized":"2024-09-16T07:33:33.025089+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.042006+0000","last_change":"2024-09-16T07:33:31.039400+0000","last_active":"2024-09-16T07:33:31.042006+0000","last_peered":"2024-09-16T07:33:31.042006+0000","last_clean":"2024-09-16T07:33:31.042006+0000","last_became_active":"2024-09-16T07:33:31.039119+0000","last_became_peered":"2024-09-16T07:33:31.039119+0000","last_unstale":"2024-09-16T07:33:31.042006+0000","last_undegraded":"2024-09-16T07:33:31.042006+0000","last_fullsized":"2024-09-16T07:33:31.042006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:34.147476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358517+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.358517+0000","last_peered":"2024-09-16T07:33:33.358517+0000","last_clean":"2024-09-16T07:33:33.358517+0000","last_became_active":"2024-09-16T07:33:32.035316+0000","last_became_peered":"2024-09-16T07:33:32.035316+0000","last_unstale":"2024-09-16T07:33:33.358517+0000","last_undegraded":"2024-09-16T07:33:33.358517+0000","last_fullsized":"2024-09-16T07:33:33.358517+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:06.190971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025197+0000","last_change":"2024-09-16T07:33:31.037265+0000","last_active":"2024-09-16T07:33:33.025197+0000","last_peered":"2024-09-16T07:33:33.025197+0000","last_clean":"2024-09-16T07:33:33.025197+0000","last_became_active":"2024-09-16T07:33:31.036981+0000","last_became_peered":"2024-09-16T07:33:31.036981+0000","last_unstale":"2024-09-16T07:33:33.025197+0000","last_undegraded":"2024-09-16T07:33:33.025197+0000","last_fullsized":"2024-09-16T07:33:33.025197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:08.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357328+0000","last_change":"2024-09-16T07:33:32.042815+0000","last_active":"2024-09-16T07:33:33.357328+0000","last_peered":"2024-09-16T07:33:33.357328+0000","last_clean":"2024-09-16T07:33:33.357328+0000","last_became_active":"2024-09-16T07:33:32.042470+0000","last_became_peered":"2024-09-16T07:33:32.042470+0000","last_unstale":"2024-09-16T07:33:33.357328+0000","last_undegraded":"2024-09-16T07:33:33.357328+0000","last_fullsized":"2024-09-16T07:33:33.357328+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:39.595734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357447+0000","last_change":"2024-09-16T07:33:31.032993+0000","last_active":"2024-09-16T07:33:33.357447+0000","last_peered":"2024-09-16T07:33:33.357447+0000","last_clean":"2024-09-16T07:33:33.357447+0000","last_became_active":"2024-09-16T07:33:31.032741+0000","last_became_peered":"2024-09-16T07:33:31.032741+0000","last_unstale":"2024-09-16T07:33:33.357447+0000","last_undegraded":"2024-09-16T07:33:33.357447+0000","last_fullsized":"2024-09-16T07:33:33.357447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:03.607602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357971+0000","last_change":"2024-09-16T07:33:32.035426+0000","last_active":"2024-09-16T07:33:33.357971+0000","last_peered":"2024-09-16T07:33:33.357971+0000","last_clean":"2024-09-16T07:33:33.357971+0000","last_became_active":"2024-09-16T07:33:32.034929+0000","last_became_peered":"2024-09-16T07:33:32.034929+0000","last_unstale":"2024-09-16T07:33:33.357971+0000","last_undegraded":"2024-09-16T07:33:33.357971+0000","last_fullsized":"2024-09-16T07:33:33.357971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":21,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358359+0000","last_change":"2024-09-16T07:33:31.041833+0000","last_active":"2024-09-16T07:33:33.358359+0000","last_peered":"2024-09-16T07:33:33.358359+0000","last_clean":"2024-09-16T07:33:33.358359+0000","last_became_active":"2024-09-16T07:33:31.041034+0000","last_became_peered":"2024-09-16T07:33:31.041034+0000","last_unstale":"2024-09-16T07:33:33.358359+0000","last_undegraded":"2024-09-16T07:33:33.358359+0000","last_fullsized":"2024-09-16T07:33:33.358359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:25.618706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024155+0000","last_change":"2024-09-16T07:33:32.042155+0000","last_active":"2024-09-16T07:33:33.024155+0000","last_peered":"2024-09-16T07:33:33.024155+0000","last_clean":"2024-09-16T07:33:33.024155+0000","last_became_active":"2024-09-16T07:33:32.041955+0000","last_became_peered":"2024-09-16T07:33:32.041955+0000","last_unstale":"2024-09-16T07:33:33.024155+0000","last_undegraded":"2024-09-16T07:33:33.024155+0000","last_fullsized":"2024-09-16T07:33:33.024155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:45.546271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025124+0000","last_change":"2024-09-16T07:33:31.036453+0000","last_active":"2024-09-16T07:33:33.025124+0000","last_peered":"2024-09-16T07:33:33.025124+0000","last_clean":"2024-09-16T07:33:33.025124+0000","last_became_active":"2024-09-16T07:33:31.034972+0000","last_became_peered":"2024-09-16T07:33:31.034972+0000","last_unstale":"2024-09-16T07:33:33.025124+0000","last_undegraded":"2024-09-16T07:33:33.025124+0000","last_fullsized":"2024-09-16T07:33:33.025124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:49.504210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.031515+0000","last_change":"2024-09-16T07:33:31.031515+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.031515+0000","last_undegraded":"2024-09-16T07:33:31.031515+0000","last_fullsized":"2024-09-16T07:33:31.031515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024241+0000","last_change":"2024-09-16T07:33:31.042034+0000","last_active":"2024-09-16T07:33:33.024241+0000","last_peered":"2024-09-16T07:33:33.024241+0000","last_clean":"2024-09-16T07:33:33.024241+0000","last_became_active":"2024-09-16T07:33:31.041059+0000","last_became_peered":"2024-09-16T07:33:31.041059+0000","last_unstale":"2024-09-16T07:33:33.024241+0000","last_undegraded":"2024-09-16T07:33:33.024241+0000","last_fullsized":"2024-09-16T07:33:33.024241+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:59.775582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357991+0000","last_change":"2024-09-16T07:33:32.046065+0000","last_active":"2024-09-16T07:33:33.357991+0000","last_peered":"2024-09-16T07:33:33.357991+0000","last_clean":"2024-09-16T07:33:33.357991+0000","last_became_active":"2024-09-16T07:33:32.045750+0000","last_became_peered":"2024-09-16T07:33:32.045750+0000","last_unstale":"2024-09-16T07:33:33.357991+0000","last_undegraded":"2024-09-16T07:33:33.357991+0000","last_fullsized":"2024-09-16T07:33:33.357991+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:26.414491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357741+0000","last_change":"2024-09-16T07:33:31.031995+0000","last_active":"2024-09-16T07:33:33.357741+0000","last_peered":"2024-09-16T07:33:33.357741+0000","last_clean":"2024-09-16T07:33:33.357741+0000","last_became_active":"2024-09-16T07:33:31.031131+0000","last_became_peered":"2024-09-16T07:33:31.031131+0000","last_unstale":"2024-09-16T07:33:33.357741+0000","last_undegraded":"2024-09-16T07:33:33.357741+0000","last_fullsized":"2024-09-16T07:33:33.357741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:28.800839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027506+0000","last_change":"2024-09-16T07:33:31.027506+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027506+0000","last_undegraded":"2024-09-16T07:33:31.027506+0000","last_fullsized":"2024-09-16T07:33:31.027506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023950+0000","last_change":"2024-09-16T07:33:31.030808+0000","last_active":"2024-09-16T07:33:33.023950+0000","last_peered":"2024-09-16T07:33:33.023950+0000","last_clean":"2024-09-16T07:33:33.023950+0000","last_became_active":"2024-09-16T07:33:31.028772+0000","last_became_peered":"2024-09-16T07:33:31.028772+0000","last_unstale":"2024-09-16T07:33:33.023950+0000","last_undegraded":"2024-09-16T07:33:33.023950+0000","last_fullsized":"2024-09-16T07:33:33.023950+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:53.744425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358706+0000","last_change":"2024-09-16T07:33:32.045394+0000","last_active":"2024-09-16T07:33:33.358706+0000","last_peered":"2024-09-16T07:33:33.358706+0000","last_clean":"2024-09-16T07:33:33.358706+0000","last_became_active":"2024-09-16T07:33:32.045106+0000","last_became_peered":"2024-09-16T07:33:32.045106+0000","last_unstale":"2024-09-16T07:33:33.358706+0000","last_undegraded":"2024-09-16T07:33:33.358706+0000","last_fullsized":"2024-09-16T07:33:33.358706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.537036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358438+0000","last_change":"2024-09-16T07:33:31.041342+0000","last_active":"2024-09-16T07:33:33.358438+0000","last_peered":"2024-09-16T07:33:33.358438+0000","last_clean":"2024-09-16T07:33:33.358438+0000","last_became_active":"2024-09-16T07:33:31.041056+0000","last_became_peered":"2024-09-16T07:33:31.041056+0000","last_unstale":"2024-09-16T07:33:33.358438+0000","last_undegraded":"2024-09-16T07:33:33.358438+0000","last_fullsized":"2024-09-16T07:33:33.358438+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:39.097159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358460+0000","last_change":"2024-09-16T07:33:32.044491+0000","last_active":"2024-09-16T07:33:33.358460+0000","last_peered":"2024-09-16T07:33:33.358460+0000","last_clean":"2024-09-16T07:33:33.358460+0000","last_became_active":"2024-09-16T07:33:32.044228+0000","last_became_peered":"2024-09-16T07:33:32.044228+0000","last_unstale":"2024-09-16T07:33:33.358460+0000","last_undegraded":"2024-09-16T07:33:33.358460+0000","last_fullsized":"2024-09-16T07:33:33.358460+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:14.023175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357548+0000","last_change":"2024-09-16T07:33:31.034256+0000","last_active":"2024-09-16T07:33:33.357548+0000","last_peered":"2024-09-16T07:33:33.357548+0000","last_clean":"2024-09-16T07:33:33.357548+0000","last_became_active":"2024-09-16T07:33:31.033825+0000","last_became_peered":"2024-09-16T07:33:31.033825+0000","last_unstale":"2024-09-16T07:33:33.357548+0000","last_undegraded":"2024-09-16T07:33:33.357548+0000","last_fullsized":"2024-09-16T07:33:33.357548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:44.857125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358285+0000","last_change":"2024-09-16T07:33:32.041248+0000","last_active":"2024-09-16T07:33:33.358285+0000","last_peered":"2024-09-16T07:33:33.358285+0000","last_clean":"2024-09-16T07:33:33.358285+0000","last_became_active":"2024-09-16T07:33:32.040998+0000","last_became_peered":"2024-09-16T07:33:32.040998+0000","last_unstale":"2024-09-16T07:33:33.358285+0000","last_undegraded":"2024-09-16T07:33:33.358285+0000","last_fullsized":"2024-09-16T07:33:33.358285+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:06.696198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025145+0000","last_change":"2024-09-16T07:33:31.035273+0000","last_active":"2024-09-16T07:33:33.025145+0000","last_peered":"2024-09-16T07:33:33.025145+0000","last_clean":"2024-09-16T07:33:33.025145+0000","last_became_active":"2024-09-16T07:33:31.034957+0000","last_became_peered":"2024-09-16T07:33:31.034957+0000","last_unstale":"2024-09-16T07:33:33.025145+0000","last_undegraded":"2024-09-16T07:33:33.025145+0000","last_fullsized":"2024-09-16T07:33:33.025145+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:17.685015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025196+0000","last_change":"2024-09-16T07:33:32.033426+0000","last_active":"2024-09-16T07:33:33.025196+0000","last_peered":"2024-09-16T07:33:33.025196+0000","last_clean":"2024-09-16T07:33:33.025196+0000","last_became_active":"2024-09-16T07:33:32.032989+0000","last_became_peered":"2024-09-16T07:33:32.032989+0000","last_unstale":"2024-09-16T07:33:33.025196+0000","last_undegraded":"2024-09-16T07:33:33.025196+0000","last_fullsized":"2024-09-16T07:33:33.025196+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:06.277092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023949+0000","last_change":"2024-09-16T07:33:31.038931+0000","last_active":"2024-09-16T07:33:33.023949+0000","last_peered":"2024-09-16T07:33:33.023949+0000","last_clean":"2024-09-16T07:33:33.023949+0000","last_became_active":"2024-09-16T07:33:31.038639+0000","last_became_peered":"2024-09-16T07:33:31.038639+0000","last_unstale":"2024-09-16T07:33:33.023949+0000","last_undegraded":"2024-09-16T07:33:33.023949+0000","last_fullsized":"2024-09-16T07:33:33.023949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:37.191040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025027+0000","last_change":"2024-09-16T07:33:32.050892+0000","last_active":"2024-09-16T07:33:33.025027+0000","last_peered":"2024-09-16T07:33:33.025027+0000","last_clean":"2024-09-16T07:33:33.025027+0000","last_became_active":"2024-09-16T07:33:32.050449+0000","last_became_peered":"2024-09-16T07:33:32.050449+0000","last_unstale":"2024-09-16T07:33:33.025027+0000","last_undegraded":"2024-09-16T07:33:33.025027+0000","last_fullsized":"2024-09-16T07:33:33.025027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:53.956082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358694+0000","last_change":"2024-09-16T07:33:31.035777+0000","last_active":"2024-09-16T07:33:33.358694+0000","last_peered":"2024-09-16T07:33:33.358694+0000","last_clean":"2024-09-16T07:33:33.358694+0000","last_became_active":"2024-09-16T07:33:31.034448+0000","last_became_peered":"2024-09-16T07:33:31.034448+0000","last_unstale":"2024-09-16T07:33:33.358694+0000","last_undegraded":"2024-09-16T07:33:33.358694+0000","last_fullsized":"2024-09-16T07:33:33.358694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:45.713854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025244+0000","last_change":"2024-09-16T07:33:32.052617+0000","last_active":"2024-09-16T07:33:33.025244+0000","last_peered":"2024-09-16T07:33:33.025244+0000","last_clean":"2024-09-16T07:33:33.025244+0000","last_became_active":"2024-09-16T07:33:32.052089+0000","last_became_peered":"2024-09-16T07:33:32.052089+0000","last_unstale":"2024-09-16T07:33:33.025244+0000","last_undegraded":"2024-09-16T07:33:33.025244+0000","last_fullsized":"2024-09-16T07:33:33.025244+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:49.702884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021703+0000","last_change":"2024-09-16T07:33:31.036417+0000","last_active":"2024-09-16T07:33:33.021703+0000","last_peered":"2024-09-16T07:33:33.021703+0000","last_clean":"2024-09-16T07:33:33.021703+0000","last_became_active":"2024-09-16T07:33:31.036118+0000","last_became_peered":"2024-09-16T07:33:31.036118+0000","last_unstale":"2024-09-16T07:33:33.021703+0000","last_undegraded":"2024-09-16T07:33:33.021703+0000","last_fullsized":"2024-09-16T07:33:33.021703+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:50.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358156+0000","last_change":"2024-09-16T07:33:32.039339+0000","last_active":"2024-09-16T07:33:33.358156+0000","last_peered":"2024-09-16T07:33:33.358156+0000","last_clean":"2024-09-16T07:33:33.358156+0000","last_became_active":"2024-09-16T07:33:32.037162+0000","last_became_peered":"2024-09-16T07:33:32.037162+0000","last_unstale":"2024-09-16T07:33:33.358156+0000","last_undegraded":"2024-09-16T07:33:33.358156+0000","last_fullsized":"2024-09-16T07:33:33.358156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:42.850047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045296+0000","last_change":"2024-09-16T07:33:31.042541+0000","last_active":"2024-09-16T07:33:31.045296+0000","last_peered":"2024-09-16T07:33:31.045296+0000","last_clean":"2024-09-16T07:33:31.045296+0000","last_became_active":"2024-09-16T07:33:31.042235+0000","last_became_peered":"2024-09-16T07:33:31.042235+0000","last_unstale":"2024-09-16T07:33:31.045296+0000","last_undegraded":"2024-09-16T07:33:31.045296+0000","last_fullsized":"2024-09-16T07:33:31.045296+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:20.784004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025623+0000","last_change":"2024-09-16T07:33:32.045670+0000","last_active":"2024-09-16T07:33:33.025623+0000","last_peered":"2024-09-16T07:33:33.025623+0000","last_clean":"2024-09-16T07:33:33.025623+0000","last_became_active":"2024-09-16T07:33:32.045383+0000","last_became_peered":"2024-09-16T07:33:32.045383+0000","last_unstale":"2024-09-16T07:33:33.025623+0000","last_undegraded":"2024-09-16T07:33:33.025623+0000","last_fullsized":"2024-09-16T07:33:33.025623+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:56.570489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358500+0000","last_change":"2024-09-16T07:33:31.030651+0000","last_active":"2024-09-16T07:33:33.358500+0000","last_peered":"2024-09-16T07:33:33.358500+0000","last_clean":"2024-09-16T07:33:33.358500+0000","last_became_active":"2024-09-16T07:33:31.030312+0000","last_became_peered":"2024-09-16T07:33:31.030312+0000","last_unstale":"2024-09-16T07:33:33.358500+0000","last_undegraded":"2024-09-16T07:33:33.358500+0000","last_fullsized":"2024-09-16T07:33:33.358500+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:57.124363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025464+0000","last_change":"2024-09-16T07:33:32.045722+0000","last_active":"2024-09-16T07:33:33.025464+0000","last_peered":"2024-09-16T07:33:33.025464+0000","last_clean":"2024-09-16T07:33:33.025464+0000","last_became_active":"2024-09-16T07:33:32.045414+0000","last_became_peered":"2024-09-16T07:33:32.045414+0000","last_unstale":"2024-09-16T07:33:33.025464+0000","last_undegraded":"2024-09-16T07:33:33.025464+0000","last_fullsized":"2024-09-16T07:33:33.025464+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:57.552859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025686+0000","last_change":"2024-09-16T07:33:31.040312+0000","last_active":"2024-09-16T07:33:33.025686+0000","last_peered":"2024-09-16T07:33:33.025686+0000","last_clean":"2024-09-16T07:33:33.025686+0000","last_became_active":"2024-09-16T07:33:31.040009+0000","last_became_peered":"2024-09-16T07:33:31.040009+0000","last_unstale":"2024-09-16T07:33:33.025686+0000","last_undegraded":"2024-09-16T07:33:33.025686+0000","last_fullsized":"2024-09-16T07:33:33.025686+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:56.393574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.032016+0000","last_change":"2024-09-16T07:33:31.032016+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.032016+0000","last_undegraded":"2024-09-16T07:33:31.032016+0000","last_fullsized":"2024-09-16T07:33:31.032016+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025287+0000","last_change":"2024-09-16T07:33:31.033696+0000","last_active":"2024-09-16T07:33:33.025287+0000","last_peered":"2024-09-16T07:33:33.025287+0000","last_clean":"2024-09-16T07:33:33.025287+0000","last_became_active":"2024-09-16T07:33:31.033430+0000","last_became_peered":"2024-09-16T07:33:31.033430+0000","last_unstale":"2024-09-16T07:33:33.025287+0000","last_undegraded":"2024-09-16T07:33:33.025287+0000","last_fullsized":"2024-09-16T07:33:33.025287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:55.604351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357346+0000","last_change":"2024-09-16T07:33:32.042206+0000","last_active":"2024-09-16T07:33:33.357346+0000","last_peered":"2024-09-16T07:33:33.357346+0000","last_clean":"2024-09-16T07:33:33.357346+0000","last_became_active":"2024-09-16T07:33:32.041947+0000","last_became_peered":"2024-09-16T07:33:32.041947+0000","last_unstale":"2024-09-16T07:33:33.357346+0000","last_undegraded":"2024-09-16T07:33:33.357346+0000","last_fullsized":"2024-09-16T07:33:33.357346+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:37:06.473975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358556+0000","last_change":"2024-09-16T07:33:31.032898+0000","last_active":"2024-09-16T07:33:33.358556+0000","last_peered":"2024-09-16T07:33:33.358556+0000","last_clean":"2024-09-16T07:33:33.358556+0000","last_became_active":"2024-09-16T07:33:31.032344+0000","last_became_peered":"2024-09-16T07:33:31.032344+0000","last_unstale":"2024-09-16T07:33:33.358556+0000","last_undegraded":"2024-09-16T07:33:33.358556+0000","last_fullsized":"2024-09-16T07:33:33.358556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:26.380802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025119+0000","last_change":"2024-09-16T07:33:32.038999+0000","last_active":"2024-09-16T07:33:33.025119+0000","last_peered":"2024-09-16T07:33:33.025119+0000","last_clean":"2024-09-16T07:33:33.025119+0000","last_became_active":"2024-09-16T07:33:32.038709+0000","last_became_peered":"2024-09-16T07:33:32.038709+0000","last_unstale":"2024-09-16T07:33:33.025119+0000","last_undegraded":"2024-09-16T07:33:33.025119+0000","last_fullsized":"2024-09-16T07:33:33.025119+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:02.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045928+0000","last_change":"2024-09-16T07:33:31.044406+0000","last_active":"2024-09-16T07:33:31.045928+0000","last_peered":"2024-09-16T07:33:31.045928+0000","last_clean":"2024-09-16T07:33:31.045928+0000","last_became_active":"2024-09-16T07:33:31.044017+0000","last_became_peered":"2024-09-16T07:33:31.044017+0000","last_unstale":"2024-09-16T07:33:31.045928+0000","last_undegraded":"2024-09-16T07:33:31.045928+0000","last_fullsized":"2024-09-16T07:33:31.045928+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:37.852077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357714+0000","last_change":"2024-09-16T07:33:32.046842+0000","last_active":"2024-09-16T07:33:33.357714+0000","last_peered":"2024-09-16T07:33:33.357714+0000","last_clean":"2024-09-16T07:33:33.357714+0000","last_became_active":"2024-09-16T07:33:32.046569+0000","last_became_peered":"2024-09-16T07:33:32.046569+0000","last_unstale":"2024-09-16T07:33:33.357714+0000","last_undegraded":"2024-09-16T07:33:33.357714+0000","last_fullsized":"2024-09-16T07:33:33.357714+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:01.321702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358757+0000","last_change":"2024-09-16T07:33:31.037028+0000","last_active":"2024-09-16T07:33:33.358757+0000","last_peered":"2024-09-16T07:33:33.358757+0000","last_clean":"2024-09-16T07:33:33.358757+0000","last_became_active":"2024-09-16T07:33:31.036714+0000","last_became_peered":"2024-09-16T07:33:31.036714+0000","last_unstale":"2024-09-16T07:33:33.358757+0000","last_undegraded":"2024-09-16T07:33:33.358757+0000","last_fullsized":"2024-09-16T07:33:33.358757+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:01.872929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024139+0000","last_change":"2024-09-16T07:33:32.036865+0000","last_active":"2024-09-16T07:33:33.024139+0000","last_peered":"2024-09-16T07:33:33.024139+0000","last_clean":"2024-09-16T07:33:33.024139+0000","last_became_active":"2024-09-16T07:33:32.036613+0000","last_became_peered":"2024-09-16T07:33:32.036613+0000","last_unstale":"2024-09-16T07:33:33.024139+0000","last_undegraded":"2024-09-16T07:33:33.024139+0000","last_fullsized":"2024-09-16T07:33:33.024139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:29:18.832918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.040193+0000","last_change":"2024-09-16T07:33:31.044776+0000","last_active":"2024-09-16T07:33:32.040193+0000","last_peered":"2024-09-16T07:33:32.040193+0000","last_clean":"2024-09-16T07:33:32.040193+0000","last_became_active":"2024-09-16T07:33:31.044453+0000","last_became_peered":"2024-09-16T07:33:31.044453+0000","last_unstale":"2024-09-16T07:33:32.040193+0000","last_undegraded":"2024-09-16T07:33:32.040193+0000","last_fullsized":"2024-09-16T07:33:32.040193+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:06.219751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358395+0000","last_change":"2024-09-16T07:33:32.043818+0000","last_active":"2024-09-16T07:33:33.358395+0000","last_peered":"2024-09-16T07:33:33.358395+0000","last_clean":"2024-09-16T07:33:33.358395+0000","last_became_active":"2024-09-16T07:33:32.043573+0000","last_became_peered":"2024-09-16T07:33:32.043573+0000","last_unstale":"2024-09-16T07:33:33.358395+0000","last_undegraded":"2024-09-16T07:33:33.358395+0000","last_fullsized":"2024-09-16T07:33:33.358395+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:10.405842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045274+0000","last_change":"2024-09-16T07:33:31.042397+0000","last_active":"2024-09-16T07:33:31.045274+0000","last_peered":"2024-09-16T07:33:31.045274+0000","last_clean":"2024-09-16T07:33:31.045274+0000","last_became_active":"2024-09-16T07:33:31.041205+0000","last_became_peered":"2024-09-16T07:33:31.041205+0000","last_unstale":"2024-09-16T07:33:31.045274+0000","last_undegraded":"2024-09-16T07:33:31.045274+0000","last_fullsized":"2024-09-16T07:33:31.045274+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:13.919862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.030279+0000","last_change":"2024-09-16T07:33:31.030279+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.030279+0000","last_undegraded":"2024-09-16T07:33:31.030279+0000","last_fullsized":"2024-09-16T07:33:31.030279+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358516+0000","last_change":"2024-09-16T07:33:31.041709+0000","last_active":"2024-09-16T07:33:33.358516+0000","last_peered":"2024-09-16T07:33:33.358516+0000","last_clean":"2024-09-16T07:33:33.358516+0000","last_became_active":"2024-09-16T07:33:31.040991+0000","last_became_peered":"2024-09-16T07:33:31.040991+0000","last_unstale":"2024-09-16T07:33:33.358516+0000","last_undegraded":"2024-09-16T07:33:33.358516+0000","last_fullsized":"2024-09-16T07:33:33.358516+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:32.562633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":265,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357091+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:33:33.357091+0000","last_peered":"2024-09-16T07:33:33.357091+0000","last_clean":"2024-09-16T07:33:33.357091+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:33:33.357091+0000","last_undegraded":"2024-09-16T07:33:33.357091+0000","last_fullsized":"2024-09-16T07:33:33.357091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122632,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249208,"statfs":{"total":96636764160,"available":96511188992,"internally_reserved":0,"allocated":1298432,"data_stored":560047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122612,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249228,"statfs":{"total":96636764160,"available":96511209472,"internally_reserved":0,"allocated":1277952,"data_stored":553595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119056,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252784,"statfs":{"total":96636764160,"available":96514850816,"internally_reserved":0,"allocated":1888256,"data_stored":1145617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100400,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271440,"statfs":{"total":96636764160,"available":96533954560,"internally_reserved":0,"allocated":1273856,"data_stored":553525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100412,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271428,"statfs":{"total":96636764160,"available":96533942272,"internally_reserved":0,"allocated":1277952,"data_stored":554107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":101513674},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148984,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222856,"statfs":{"total":96636764160,"available":96484204544,"internally_reserved":0,"allocated":1273856,"data_stored":553525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509575,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127380,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244460,"statfs":{"total":96636764160,"available":96506327040,"internally_reserved":0,"allocated":1900544,"data_stored":1151889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542277,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122584,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249256,"statfs":{"total":96636764160,"available":96511238144,"internally_reserved":0,"allocated":1249280,"data_stored":540419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:33:36.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:33:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.427+0000 7fead2800640 1 -- 172.21.15.71:0/597112347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 msgr2=0x7feacc074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.427+0000 7fead2800640 1 --2- 172.21.15.71:0/597112347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 0x7feacc074440 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7feac4005fd0 tx=0x7feac402a970 comp rx=0 tx=0).stop 2024-09-16T07:33:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.427+0000 7fead2800640 1 -- 172.21.15.71:0/597112347 shutdown_connections 2024-09-16T07:33:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.427+0000 7fead2800640 1 --2- 172.21.15.71:0/597112347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feacc0ccf00 0x7feacc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.427+0000 7fead2800640 1 --2- 172.21.15.71:0/597112347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 0x7feacc074440 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.427+0000 7fead2800640 1 -- 172.21.15.71:0/597112347 >> 172.21.15.71:0/597112347 conn(0x7feacc0b1950 msgr2=0x7feacc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.427+0000 7fead2800640 1 -- 172.21.15.71:0/597112347 shutdown_connections 2024-09-16T07:33:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.427+0000 7fead2800640 1 -- 172.21.15.71:0/597112347 wait complete. 2024-09-16T07:33:36.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.429+0000 7fead2800640 1 Processor -- start 2024-09-16T07:33:36.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.429+0000 7fead2800640 1 -- start start 2024-09-16T07:33:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7fead2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 0x7feacc14bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7fead2800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feacc0ccf00 0x7feacc0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7fead2800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feacc15f5a0 0x7feacc1f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7fead2800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feacc06f2e0 con 0x7feacc073fb0 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7fead2800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feacc06f160 con 0x7feacc0ccf00 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7fead2800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feacc06f460 con 0x7feacc15f5a0 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 0x7feacc14bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 0x7feacc14bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42266/0 (socket says 172.21.15.71:42266) 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacb400640 1 -- 172.21.15.71:0/2378245450 learned_addr learned my addr 172.21.15.71:0/2378245450 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacaa00640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feacc0ccf00 0x7feacc0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacbe00640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feacc15f5a0 0x7feacc1f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacb400640 1 -- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feacc15f5a0 msgr2=0x7feacc1f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacb400640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feacc15f5a0 0x7feacc1f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacb400640 1 -- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feacc0ccf00 msgr2=0x7feacc0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacb400640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feacc0ccf00 0x7feacc0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.430+0000 7feacb400640 1 -- 172.21.15.71:0/2378245450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc14c1b0 con 0x7feacc073fb0 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.431+0000 7feacb400640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 0x7feacc14bc70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7feabc00b3f0 tx=0x7feabc00b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.431+0000 7feac3400640 1 -- 172.21.15.71:0/2378245450 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc015020 con 0x7feacc073fb0 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.431+0000 7feac3400640 1 -- 172.21.15.71:0/2378245450 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feabc011920 con 0x7feacc073fb0 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.431+0000 7feac3400640 1 -- 172.21.15.71:0/2378245450 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc011db0 con 0x7feacc073fb0 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.431+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feacc0cc1b0 con 0x7feacc073fb0 2024-09-16T07:33:36.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.431+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feacc0cf390 con 0x7feacc073fb0 2024-09-16T07:33:36.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.432+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feacc0b4350 con 0x7feacc073fb0 2024-09-16T07:33:36.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.433+0000 7feac3400640 1 -- 172.21.15.71:0/2378245450 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7feabc018020 con 0x7feacc073fb0 2024-09-16T07:33:36.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.434+0000 7feac3400640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaac079e40 0x7feaac07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.434+0000 7feac3400640 1 -- 172.21.15.71:0/2378245450 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7feabc099620 con 0x7feacc073fb0 2024-09-16T07:33:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.435+0000 7feacaa00640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaac079e40 0x7feaac07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.436+0000 7feacaa00640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaac079e40 0x7feaac07c300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7feacc155fb0 tx=0x7feab800a2d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.438+0000 7feac3400640 1 -- 172.21.15.71:0/2378245450 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feacc0b4350 con 0x7feacc073fb0 2024-09-16T07:33:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.620+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feacc0b08d0 con 0x7feaac079e40 2024-09-16T07:33:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.625+0000 7feac3400640 1 -- 172.21.15.71:0/2378245450 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251762 (secure 0 0 0) 0x7feacc0b08d0 con 0x7feaac079e40 2024-09-16T07:33:36.627 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:33:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaac079e40 msgr2=0x7feaac07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaac079e40 0x7feaac07c300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7feacc155fb0 tx=0x7feab800a2d0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 msgr2=0x7feacc14bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 0x7feacc14bc70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7feabc00b3f0 tx=0x7feabc00b7f0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 shutdown_connections 2024-09-16T07:33:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feaac079e40 0x7feaac07c300 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feacc15f5a0 0x7feacc1f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feacc0ccf00 0x7feacc0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 --2- 172.21.15.71:0/2378245450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feacc073fb0 0x7feacc14bc70 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 >> 172.21.15.71:0/2378245450 conn(0x7feacc0b1950 msgr2=0x7feacc0b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 shutdown_connections 2024-09-16T07:33:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.629+0000 7fead2800640 1 -- 172.21.15.71:0/2378245450 wait complete. 2024-09-16T07:33:36.645 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-16T07:33:36.581316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":964060,"kb_used_data":11172,"kb_used_omap":12,"kb_used_meta":952691,"kb_avail":754010660,"statfs":{"total":773094113280,"available":772106915840,"internally_reserved":0,"allocated":11440128,"data_stored":5612724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":975556144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001607"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025017+0000","last_change":"2024-09-16T07:33:32.045068+0000","last_active":"2024-09-16T07:33:33.025017+0000","last_peered":"2024-09-16T07:33:33.025017+0000","last_clean":"2024-09-16T07:33:33.025017+0000","last_became_active":"2024-09-16T07:33:32.044772+0000","last_became_peered":"2024-09-16T07:33:32.044772+0000","last_unstale":"2024-09-16T07:33:33.025017+0000","last_undegraded":"2024-09-16T07:33:33.025017+0000","last_fullsized":"2024-09-16T07:33:33.025017+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024784+0000","last_change":"2024-09-16T07:33:32.049940+0000","last_active":"2024-09-16T07:33:33.024784+0000","last_peered":"2024-09-16T07:33:33.024784+0000","last_clean":"2024-09-16T07:33:33.024784+0000","last_became_active":"2024-09-16T07:33:32.049646+0000","last_became_peered":"2024-09-16T07:33:32.049646+0000","last_unstale":"2024-09-16T07:33:33.024784+0000","last_undegraded":"2024-09-16T07:33:33.024784+0000","last_fullsized":"2024-09-16T07:33:33.024784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.031577+0000","last_change":"2024-09-16T07:33:31.031577+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.031577+0000","last_undegraded":"2024-09-16T07:33:31.031577+0000","last_fullsized":"2024-09-16T07:33:31.031577+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024064+0000","last_change":"2024-09-16T07:33:32.043883+0000","last_active":"2024-09-16T07:33:33.024064+0000","last_peered":"2024-09-16T07:33:33.024064+0000","last_clean":"2024-09-16T07:33:33.024064+0000","last_became_active":"2024-09-16T07:33:32.043622+0000","last_became_peered":"2024-09-16T07:33:32.043622+0000","last_unstale":"2024-09-16T07:33:33.024064+0000","last_undegraded":"2024-09-16T07:33:33.024064+0000","last_fullsized":"2024-09-16T07:33:33.024064+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:01:49.570122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357403+0000","last_change":"2024-09-16T07:33:32.043899+0000","last_active":"2024-09-16T07:33:33.357403+0000","last_peered":"2024-09-16T07:33:33.357403+0000","last_clean":"2024-09-16T07:33:33.357403+0000","last_became_active":"2024-09-16T07:33:32.043611+0000","last_became_peered":"2024-09-16T07:33:32.043611+0000","last_unstale":"2024-09-16T07:33:33.357403+0000","last_undegraded":"2024-09-16T07:33:33.357403+0000","last_fullsized":"2024-09-16T07:33:33.357403+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:22.213386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357942+0000","last_change":"2024-09-16T07:33:32.054833+0000","last_active":"2024-09-16T07:33:33.357942+0000","last_peered":"2024-09-16T07:33:33.357942+0000","last_clean":"2024-09-16T07:33:33.357942+0000","last_became_active":"2024-09-16T07:33:32.054513+0000","last_became_peered":"2024-09-16T07:33:32.054513+0000","last_unstale":"2024-09-16T07:33:33.357942+0000","last_undegraded":"2024-09-16T07:33:33.357942+0000","last_fullsized":"2024-09-16T07:33:33.357942+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:03.590298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357327+0000","last_change":"2024-09-16T07:33:32.037656+0000","last_active":"2024-09-16T07:33:33.357327+0000","last_peered":"2024-09-16T07:33:33.357327+0000","last_clean":"2024-09-16T07:33:33.357327+0000","last_became_active":"2024-09-16T07:33:32.036949+0000","last_became_peered":"2024-09-16T07:33:32.036949+0000","last_unstale":"2024-09-16T07:33:33.357327+0000","last_undegraded":"2024-09-16T07:33:33.357327+0000","last_fullsized":"2024-09-16T07:33:33.357327+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:18.915383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.025834+0000","last_change":"2024-09-16T07:33:31.025834+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.025834+0000","last_undegraded":"2024-09-16T07:33:31.025834+0000","last_fullsized":"2024-09-16T07:33:31.025834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357756+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.357756+0000","last_peered":"2024-09-16T07:33:33.357756+0000","last_clean":"2024-09-16T07:33:33.357756+0000","last_became_active":"2024-09-16T07:33:32.035287+0000","last_became_peered":"2024-09-16T07:33:32.035287+0000","last_unstale":"2024-09-16T07:33:33.357756+0000","last_undegraded":"2024-09-16T07:33:33.357756+0000","last_fullsized":"2024-09-16T07:33:33.357756+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:52.049650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021843+0000","last_change":"2024-09-16T07:33:32.049217+0000","last_active":"2024-09-16T07:33:33.021843+0000","last_peered":"2024-09-16T07:33:33.021843+0000","last_clean":"2024-09-16T07:33:33.021843+0000","last_became_active":"2024-09-16T07:33:32.048953+0000","last_became_peered":"2024-09-16T07:33:32.048953+0000","last_unstale":"2024-09-16T07:33:33.021843+0000","last_undegraded":"2024-09-16T07:33:33.021843+0000","last_fullsized":"2024-09-16T07:33:33.021843+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357715+0000","last_change":"2024-09-16T07:33:32.054961+0000","last_active":"2024-09-16T07:33:33.357715+0000","last_peered":"2024-09-16T07:33:33.357715+0000","last_clean":"2024-09-16T07:33:33.357715+0000","last_became_active":"2024-09-16T07:33:32.054463+0000","last_became_peered":"2024-09-16T07:33:32.054463+0000","last_unstale":"2024-09-16T07:33:33.357715+0000","last_undegraded":"2024-09-16T07:33:33.357715+0000","last_fullsized":"2024-09-16T07:33:33.357715+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:57.790799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358257+0000","last_change":"2024-09-16T07:33:32.040365+0000","last_active":"2024-09-16T07:33:33.358257+0000","last_peered":"2024-09-16T07:33:33.358257+0000","last_clean":"2024-09-16T07:33:33.358257+0000","last_became_active":"2024-09-16T07:33:32.037953+0000","last_became_peered":"2024-09-16T07:33:32.037953+0000","last_unstale":"2024-09-16T07:33:33.358257+0000","last_undegraded":"2024-09-16T07:33:33.358257+0000","last_fullsized":"2024-09-16T07:33:33.358257+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:32.112813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.059035+0000","last_change":"2024-09-16T07:33:32.056887+0000","last_active":"2024-09-16T07:33:32.059035+0000","last_peered":"2024-09-16T07:33:32.059035+0000","last_clean":"2024-09-16T07:33:32.059035+0000","last_became_active":"2024-09-16T07:33:32.056276+0000","last_became_peered":"2024-09-16T07:33:32.056276+0000","last_unstale":"2024-09-16T07:33:32.059035+0000","last_undegraded":"2024-09-16T07:33:32.059035+0000","last_fullsized":"2024-09-16T07:33:32.059035+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:23.723158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023495+0000","last_change":"2024-09-16T07:33:32.054384+0000","last_active":"2024-09-16T07:33:33.023495+0000","last_peered":"2024-09-16T07:33:33.023495+0000","last_clean":"2024-09-16T07:33:33.023495+0000","last_became_active":"2024-09-16T07:33:32.051888+0000","last_became_peered":"2024-09-16T07:33:32.051888+0000","last_unstale":"2024-09-16T07:33:33.023495+0000","last_undegraded":"2024-09-16T07:33:33.023495+0000","last_fullsized":"2024-09-16T07:33:33.023495+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:19.796812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023506+0000","last_change":"2024-09-16T07:33:32.040802+0000","last_active":"2024-09-16T07:33:33.023506+0000","last_peered":"2024-09-16T07:33:33.023506+0000","last_clean":"2024-09-16T07:33:33.023506+0000","last_became_active":"2024-09-16T07:33:32.040510+0000","last_became_peered":"2024-09-16T07:33:32.040510+0000","last_unstale":"2024-09-16T07:33:33.023506+0000","last_undegraded":"2024-09-16T07:33:33.023506+0000","last_fullsized":"2024-09-16T07:33:33.023506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:53:27.647859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357155+0000","last_change":"2024-09-16T07:33:32.039107+0000","last_active":"2024-09-16T07:33:33.357155+0000","last_peered":"2024-09-16T07:33:33.357155+0000","last_clean":"2024-09-16T07:33:33.357155+0000","last_became_active":"2024-09-16T07:33:32.038828+0000","last_became_peered":"2024-09-16T07:33:32.038828+0000","last_unstale":"2024-09-16T07:33:33.357155+0000","last_undegraded":"2024-09-16T07:33:33.357155+0000","last_fullsized":"2024-09-16T07:33:33.357155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:15.415755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025462+0000","last_change":"2024-09-16T07:33:32.053433+0000","last_active":"2024-09-16T07:33:33.025462+0000","last_peered":"2024-09-16T07:33:33.025462+0000","last_clean":"2024-09-16T07:33:33.025462+0000","last_became_active":"2024-09-16T07:33:32.052843+0000","last_became_peered":"2024-09-16T07:33:32.052843+0000","last_unstale":"2024-09-16T07:33:33.025462+0000","last_undegraded":"2024-09-16T07:33:33.025462+0000","last_fullsized":"2024-09-16T07:33:33.025462+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:13.265002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025375+0000","last_change":"2024-09-16T07:33:32.047389+0000","last_active":"2024-09-16T07:33:33.025375+0000","last_peered":"2024-09-16T07:33:33.025375+0000","last_clean":"2024-09-16T07:33:33.025375+0000","last_became_active":"2024-09-16T07:33:32.047043+0000","last_became_peered":"2024-09-16T07:33:32.047043+0000","last_unstale":"2024-09-16T07:33:33.025375+0000","last_undegraded":"2024-09-16T07:33:33.025375+0000","last_fullsized":"2024-09-16T07:33:33.025375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024761+0000","last_change":"2024-09-16T07:33:32.040875+0000","last_active":"2024-09-16T07:33:33.024761+0000","last_peered":"2024-09-16T07:33:33.024761+0000","last_clean":"2024-09-16T07:33:33.024761+0000","last_became_active":"2024-09-16T07:33:32.040636+0000","last_became_peered":"2024-09-16T07:33:32.040636+0000","last_unstale":"2024-09-16T07:33:33.024761+0000","last_undegraded":"2024-09-16T07:33:33.024761+0000","last_fullsized":"2024-09-16T07:33:33.024761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:06.934545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358353+0000","last_change":"2024-09-16T07:33:32.053936+0000","last_active":"2024-09-16T07:33:33.358353+0000","last_peered":"2024-09-16T07:33:33.358353+0000","last_clean":"2024-09-16T07:33:33.358353+0000","last_became_active":"2024-09-16T07:33:32.053546+0000","last_became_peered":"2024-09-16T07:33:32.053546+0000","last_unstale":"2024-09-16T07:33:33.358353+0000","last_undegraded":"2024-09-16T07:33:33.358353+0000","last_fullsized":"2024-09-16T07:33:33.358353+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.032081+0000","last_change":"2024-09-16T07:33:31.032081+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.032081+0000","last_undegraded":"2024-09-16T07:33:31.032081+0000","last_fullsized":"2024-09-16T07:33:31.032081+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027983+0000","last_change":"2024-09-16T07:33:31.027983+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027983+0000","last_undegraded":"2024-09-16T07:33:31.027983+0000","last_fullsized":"2024-09-16T07:33:31.027983+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023549+0000","last_change":"2024-09-16T07:33:32.055235+0000","last_active":"2024-09-16T07:33:33.023549+0000","last_peered":"2024-09-16T07:33:33.023549+0000","last_clean":"2024-09-16T07:33:33.023549+0000","last_became_active":"2024-09-16T07:33:32.052675+0000","last_became_peered":"2024-09-16T07:33:32.052675+0000","last_unstale":"2024-09-16T07:33:33.023549+0000","last_undegraded":"2024-09-16T07:33:33.023549+0000","last_fullsized":"2024-09-16T07:33:33.023549+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:31.315404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357746+0000","last_change":"2024-09-16T07:33:32.050486+0000","last_active":"2024-09-16T07:33:33.357746+0000","last_peered":"2024-09-16T07:33:33.357746+0000","last_clean":"2024-09-16T07:33:33.357746+0000","last_became_active":"2024-09-16T07:33:32.050108+0000","last_became_peered":"2024-09-16T07:33:32.050108+0000","last_unstale":"2024-09-16T07:33:33.357746+0000","last_undegraded":"2024-09-16T07:33:33.357746+0000","last_fullsized":"2024-09-16T07:33:33.357746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021955+0000","last_change":"2024-09-16T07:33:32.028117+0000","last_active":"2024-09-16T07:33:33.021955+0000","last_peered":"2024-09-16T07:33:33.021955+0000","last_clean":"2024-09-16T07:33:33.021955+0000","last_became_active":"2024-09-16T07:33:32.027826+0000","last_became_peered":"2024-09-16T07:33:32.027826+0000","last_unstale":"2024-09-16T07:33:33.021955+0000","last_undegraded":"2024-09-16T07:33:33.021955+0000","last_fullsized":"2024-09-16T07:33:33.021955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:35.757466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.055382+0000","last_change":"2024-09-16T07:33:32.049150+0000","last_active":"2024-09-16T07:33:32.055382+0000","last_peered":"2024-09-16T07:33:32.055382+0000","last_clean":"2024-09-16T07:33:32.055382+0000","last_became_active":"2024-09-16T07:33:32.048893+0000","last_became_peered":"2024-09-16T07:33:32.048893+0000","last_unstale":"2024-09-16T07:33:32.055382+0000","last_undegraded":"2024-09-16T07:33:32.055382+0000","last_fullsized":"2024-09-16T07:33:32.055382+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:37.326467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358162+0000","last_change":"2024-09-16T07:33:32.041462+0000","last_active":"2024-09-16T07:33:33.358162+0000","last_peered":"2024-09-16T07:33:33.358162+0000","last_clean":"2024-09-16T07:33:33.358162+0000","last_became_active":"2024-09-16T07:33:32.041198+0000","last_became_peered":"2024-09-16T07:33:32.041198+0000","last_unstale":"2024-09-16T07:33:33.358162+0000","last_undegraded":"2024-09-16T07:33:33.358162+0000","last_fullsized":"2024-09-16T07:33:33.358162+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358295+0000","last_change":"2024-09-16T07:33:32.037402+0000","last_active":"2024-09-16T07:33:33.358295+0000","last_peered":"2024-09-16T07:33:33.358295+0000","last_clean":"2024-09-16T07:33:33.358295+0000","last_became_active":"2024-09-16T07:33:32.037131+0000","last_became_peered":"2024-09-16T07:33:32.037131+0000","last_unstale":"2024-09-16T07:33:33.358295+0000","last_undegraded":"2024-09-16T07:33:33.358295+0000","last_fullsized":"2024-09-16T07:33:33.358295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.052439+0000","last_change":"2024-09-16T07:33:32.043318+0000","last_active":"2024-09-16T07:33:32.052439+0000","last_peered":"2024-09-16T07:33:32.052439+0000","last_clean":"2024-09-16T07:33:32.052439+0000","last_became_active":"2024-09-16T07:33:32.043039+0000","last_became_peered":"2024-09-16T07:33:32.043039+0000","last_unstale":"2024-09-16T07:33:32.052439+0000","last_undegraded":"2024-09-16T07:33:32.052439+0000","last_fullsized":"2024-09-16T07:33:32.052439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:45.700538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.059020+0000","last_change":"2024-09-16T07:33:32.056817+0000","last_active":"2024-09-16T07:33:32.059020+0000","last_peered":"2024-09-16T07:33:32.059020+0000","last_clean":"2024-09-16T07:33:32.059020+0000","last_became_active":"2024-09-16T07:33:32.056535+0000","last_became_peered":"2024-09-16T07:33:32.056535+0000","last_unstale":"2024-09-16T07:33:32.059020+0000","last_undegraded":"2024-09-16T07:33:32.059020+0000","last_fullsized":"2024-09-16T07:33:32.059020+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:35.876240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027968+0000","last_change":"2024-09-16T07:33:31.027968+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027968+0000","last_undegraded":"2024-09-16T07:33:31.027968+0000","last_fullsized":"2024-09-16T07:33:31.027968+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023806+0000","last_change":"2024-09-16T07:33:32.036224+0000","last_active":"2024-09-16T07:33:33.023806+0000","last_peered":"2024-09-16T07:33:33.023806+0000","last_clean":"2024-09-16T07:33:33.023806+0000","last_became_active":"2024-09-16T07:33:32.035020+0000","last_became_peered":"2024-09-16T07:33:32.035020+0000","last_unstale":"2024-09-16T07:33:33.023806+0000","last_undegraded":"2024-09-16T07:33:33.023806+0000","last_fullsized":"2024-09-16T07:33:33.023806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:00.997013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025689+0000","last_change":"2024-09-16T07:33:32.046309+0000","last_active":"2024-09-16T07:33:33.025689+0000","last_peered":"2024-09-16T07:33:33.025689+0000","last_clean":"2024-09-16T07:33:33.025689+0000","last_became_active":"2024-09-16T07:33:32.045999+0000","last_became_peered":"2024-09-16T07:33:32.045999+0000","last_unstale":"2024-09-16T07:33:33.025689+0000","last_undegraded":"2024-09-16T07:33:33.025689+0000","last_fullsized":"2024-09-16T07:33:33.025689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:09.445884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.053925+0000","last_change":"2024-09-16T07:33:31.045574+0000","last_active":"2024-09-16T07:33:32.053925+0000","last_peered":"2024-09-16T07:33:32.053925+0000","last_clean":"2024-09-16T07:33:32.053925+0000","last_became_active":"2024-09-16T07:33:31.045014+0000","last_became_peered":"2024-09-16T07:33:31.045014+0000","last_unstale":"2024-09-16T07:33:32.053925+0000","last_undegraded":"2024-09-16T07:33:32.053925+0000","last_fullsized":"2024-09-16T07:33:32.053925+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.581864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.052698+0000","last_change":"2024-09-16T07:33:32.046389+0000","last_active":"2024-09-16T07:33:32.052698+0000","last_peered":"2024-09-16T07:33:32.052698+0000","last_clean":"2024-09-16T07:33:32.052698+0000","last_became_active":"2024-09-16T07:33:32.046041+0000","last_became_peered":"2024-09-16T07:33:32.046041+0000","last_unstale":"2024-09-16T07:33:32.052698+0000","last_undegraded":"2024-09-16T07:33:32.052698+0000","last_fullsized":"2024-09-16T07:33:32.052698+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:20.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357662+0000","last_change":"2024-09-16T07:33:31.041205+0000","last_active":"2024-09-16T07:33:33.357662+0000","last_peered":"2024-09-16T07:33:33.357662+0000","last_clean":"2024-09-16T07:33:33.357662+0000","last_became_active":"2024-09-16T07:33:31.040923+0000","last_became_peered":"2024-09-16T07:33:31.040923+0000","last_unstale":"2024-09-16T07:33:33.357662+0000","last_undegraded":"2024-09-16T07:33:33.357662+0000","last_fullsized":"2024-09-16T07:33:33.357662+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:56.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024526+0000","last_change":"2024-09-16T07:33:32.039150+0000","last_active":"2024-09-16T07:33:33.024526+0000","last_peered":"2024-09-16T07:33:33.024526+0000","last_clean":"2024-09-16T07:33:33.024526+0000","last_became_active":"2024-09-16T07:33:32.038837+0000","last_became_peered":"2024-09-16T07:33:32.038837+0000","last_unstale":"2024-09-16T07:33:33.024526+0000","last_undegraded":"2024-09-16T07:33:33.024526+0000","last_fullsized":"2024-09-16T07:33:33.024526+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:52.257876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.046999+0000","last_change":"2024-09-16T07:33:31.045183+0000","last_active":"2024-09-16T07:33:31.046999+0000","last_peered":"2024-09-16T07:33:31.046999+0000","last_clean":"2024-09-16T07:33:31.046999+0000","last_became_active":"2024-09-16T07:33:31.044253+0000","last_became_peered":"2024-09-16T07:33:31.044253+0000","last_unstale":"2024-09-16T07:33:31.046999+0000","last_undegraded":"2024-09-16T07:33:31.046999+0000","last_fullsized":"2024-09-16T07:33:31.046999+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:15:28.325154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021838+0000","last_change":"2024-09-16T07:33:32.043094+0000","last_active":"2024-09-16T07:33:33.021838+0000","last_peered":"2024-09-16T07:33:33.021838+0000","last_clean":"2024-09-16T07:33:33.021838+0000","last_became_active":"2024-09-16T07:33:32.042832+0000","last_became_peered":"2024-09-16T07:33:32.042832+0000","last_unstale":"2024-09-16T07:33:33.021838+0000","last_undegraded":"2024-09-16T07:33:33.021838+0000","last_fullsized":"2024-09-16T07:33:33.021838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:24.827070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024550+0000","last_change":"2024-09-16T07:33:31.042458+0000","last_active":"2024-09-16T07:33:33.024550+0000","last_peered":"2024-09-16T07:33:33.024550+0000","last_clean":"2024-09-16T07:33:33.024550+0000","last_became_active":"2024-09-16T07:33:31.042226+0000","last_became_peered":"2024-09-16T07:33:31.042226+0000","last_unstale":"2024-09-16T07:33:33.024550+0000","last_undegraded":"2024-09-16T07:33:33.024550+0000","last_fullsized":"2024-09-16T07:33:33.024550+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:20.182053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357806+0000","last_change":"2024-09-16T07:33:32.039748+0000","last_active":"2024-09-16T07:33:33.357806+0000","last_peered":"2024-09-16T07:33:33.357806+0000","last_clean":"2024-09-16T07:33:33.357806+0000","last_became_active":"2024-09-16T07:33:32.038397+0000","last_became_peered":"2024-09-16T07:33:32.038397+0000","last_unstale":"2024-09-16T07:33:33.357806+0000","last_undegraded":"2024-09-16T07:33:33.357806+0000","last_fullsized":"2024-09-16T07:33:33.357806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:15.732874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023864+0000","last_change":"2024-09-16T07:33:31.033141+0000","last_active":"2024-09-16T07:33:33.023864+0000","last_peered":"2024-09-16T07:33:33.023864+0000","last_clean":"2024-09-16T07:33:33.023864+0000","last_became_active":"2024-09-16T07:33:31.032570+0000","last_became_peered":"2024-09-16T07:33:31.032570+0000","last_unstale":"2024-09-16T07:33:33.023864+0000","last_undegraded":"2024-09-16T07:33:33.023864+0000","last_fullsized":"2024-09-16T07:33:33.023864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:26.450637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357619+0000","last_change":"2024-09-16T07:33:32.044965+0000","last_active":"2024-09-16T07:33:33.357619+0000","last_peered":"2024-09-16T07:33:33.357619+0000","last_clean":"2024-09-16T07:33:33.357619+0000","last_became_active":"2024-09-16T07:33:32.044705+0000","last_became_peered":"2024-09-16T07:33:32.044705+0000","last_unstale":"2024-09-16T07:33:33.357619+0000","last_undegraded":"2024-09-16T07:33:33.357619+0000","last_fullsized":"2024-09-16T07:33:33.357619+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:50.022797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.040511+0000","last_change":"2024-09-16T07:33:31.035805+0000","last_active":"2024-09-16T07:33:31.040511+0000","last_peered":"2024-09-16T07:33:31.040511+0000","last_clean":"2024-09-16T07:33:31.040511+0000","last_became_active":"2024-09-16T07:33:31.035545+0000","last_became_peered":"2024-09-16T07:33:31.035545+0000","last_unstale":"2024-09-16T07:33:31.040511+0000","last_undegraded":"2024-09-16T07:33:31.040511+0000","last_fullsized":"2024-09-16T07:33:31.040511+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:16.651576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.053801+0000","last_change":"2024-09-16T07:33:32.051200+0000","last_active":"2024-09-16T07:33:32.053801+0000","last_peered":"2024-09-16T07:33:32.053801+0000","last_clean":"2024-09-16T07:33:32.053801+0000","last_became_active":"2024-09-16T07:33:32.047036+0000","last_became_peered":"2024-09-16T07:33:32.047036+0000","last_unstale":"2024-09-16T07:33:32.053801+0000","last_undegraded":"2024-09-16T07:33:32.053801+0000","last_fullsized":"2024-09-16T07:33:32.053801+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:23.177353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357424+0000","last_change":"2024-09-16T07:33:31.034485+0000","last_active":"2024-09-16T07:33:33.357424+0000","last_peered":"2024-09-16T07:33:33.357424+0000","last_clean":"2024-09-16T07:33:33.357424+0000","last_became_active":"2024-09-16T07:33:31.034236+0000","last_became_peered":"2024-09-16T07:33:31.034236+0000","last_unstale":"2024-09-16T07:33:33.357424+0000","last_undegraded":"2024-09-16T07:33:33.357424+0000","last_fullsized":"2024-09-16T07:33:33.357424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:52.133642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357028+0000","last_change":"2024-09-16T07:33:32.039686+0000","last_active":"2024-09-16T07:33:33.357028+0000","last_peered":"2024-09-16T07:33:33.357028+0000","last_clean":"2024-09-16T07:33:33.357028+0000","last_became_active":"2024-09-16T07:33:32.038343+0000","last_became_peered":"2024-09-16T07:33:32.038343+0000","last_unstale":"2024-09-16T07:33:33.357028+0000","last_undegraded":"2024-09-16T07:33:33.357028+0000","last_fullsized":"2024-09-16T07:33:33.357028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:00.455428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024348+0000","last_change":"2024-09-16T07:33:31.039439+0000","last_active":"2024-09-16T07:33:33.024348+0000","last_peered":"2024-09-16T07:33:33.024348+0000","last_clean":"2024-09-16T07:33:33.024348+0000","last_became_active":"2024-09-16T07:33:31.039029+0000","last_became_peered":"2024-09-16T07:33:31.039029+0000","last_unstale":"2024-09-16T07:33:33.024348+0000","last_undegraded":"2024-09-16T07:33:33.024348+0000","last_fullsized":"2024-09-16T07:33:33.024348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:21.209706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025089+0000","last_change":"2024-09-16T07:33:32.045355+0000","last_active":"2024-09-16T07:33:33.025089+0000","last_peered":"2024-09-16T07:33:33.025089+0000","last_clean":"2024-09-16T07:33:33.025089+0000","last_became_active":"2024-09-16T07:33:32.045070+0000","last_became_peered":"2024-09-16T07:33:32.045070+0000","last_unstale":"2024-09-16T07:33:33.025089+0000","last_undegraded":"2024-09-16T07:33:33.025089+0000","last_fullsized":"2024-09-16T07:33:33.025089+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.042006+0000","last_change":"2024-09-16T07:33:31.039400+0000","last_active":"2024-09-16T07:33:31.042006+0000","last_peered":"2024-09-16T07:33:31.042006+0000","last_clean":"2024-09-16T07:33:31.042006+0000","last_became_active":"2024-09-16T07:33:31.039119+0000","last_became_peered":"2024-09-16T07:33:31.039119+0000","last_unstale":"2024-09-16T07:33:31.042006+0000","last_undegraded":"2024-09-16T07:33:31.042006+0000","last_fullsized":"2024-09-16T07:33:31.042006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:34.147476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358517+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.358517+0000","last_peered":"2024-09-16T07:33:33.358517+0000","last_clean":"2024-09-16T07:33:33.358517+0000","last_became_active":"2024-09-16T07:33:32.035316+0000","last_became_peered":"2024-09-16T07:33:32.035316+0000","last_unstale":"2024-09-16T07:33:33.358517+0000","last_undegraded":"2024-09-16T07:33:33.358517+0000","last_fullsized":"2024-09-16T07:33:33.358517+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:06.190971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025197+0000","last_change":"2024-09-16T07:33:31.037265+0000","last_active":"2024-09-16T07:33:33.025197+0000","last_peered":"2024-09-16T07:33:33.025197+0000","last_clean":"2024-09-16T07:33:33.025197+0000","last_became_active":"2024-09-16T07:33:31.036981+0000","last_became_peered":"2024-09-16T07:33:31.036981+0000","last_unstale":"2024-09-16T07:33:33.025197+0000","last_undegraded":"2024-09-16T07:33:33.025197+0000","last_fullsized":"2024-09-16T07:33:33.025197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:08.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357328+0000","last_change":"2024-09-16T07:33:32.042815+0000","last_active":"2024-09-16T07:33:33.357328+0000","last_peered":"2024-09-16T07:33:33.357328+0000","last_clean":"2024-09-16T07:33:33.357328+0000","last_became_active":"2024-09-16T07:33:32.042470+0000","last_became_peered":"2024-09-16T07:33:32.042470+0000","last_unstale":"2024-09-16T07:33:33.357328+0000","last_undegraded":"2024-09-16T07:33:33.357328+0000","last_fullsized":"2024-09-16T07:33:33.357328+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:39.595734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357447+0000","last_change":"2024-09-16T07:33:31.032993+0000","last_active":"2024-09-16T07:33:33.357447+0000","last_peered":"2024-09-16T07:33:33.357447+0000","last_clean":"2024-09-16T07:33:33.357447+0000","last_became_active":"2024-09-16T07:33:31.032741+0000","last_became_peered":"2024-09-16T07:33:31.032741+0000","last_unstale":"2024-09-16T07:33:33.357447+0000","last_undegraded":"2024-09-16T07:33:33.357447+0000","last_fullsized":"2024-09-16T07:33:33.357447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:03.607602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357971+0000","last_change":"2024-09-16T07:33:32.035426+0000","last_active":"2024-09-16T07:33:33.357971+0000","last_peered":"2024-09-16T07:33:33.357971+0000","last_clean":"2024-09-16T07:33:33.357971+0000","last_became_active":"2024-09-16T07:33:32.034929+0000","last_became_peered":"2024-09-16T07:33:32.034929+0000","last_unstale":"2024-09-16T07:33:33.357971+0000","last_undegraded":"2024-09-16T07:33:33.357971+0000","last_fullsized":"2024-09-16T07:33:33.357971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":21,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358359+0000","last_change":"2024-09-16T07:33:31.041833+0000","last_active":"2024-09-16T07:33:33.358359+0000","last_peered":"2024-09-16T07:33:33.358359+0000","last_clean":"2024-09-16T07:33:33.358359+0000","last_became_active":"2024-09-16T07:33:31.041034+0000","last_became_peered":"2024-09-16T07:33:31.041034+0000","last_unstale":"2024-09-16T07:33:33.358359+0000","last_undegraded":"2024-09-16T07:33:33.358359+0000","last_fullsized":"2024-09-16T07:33:33.358359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:25.618706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024155+0000","last_change":"2024-09-16T07:33:32.042155+0000","last_active":"2024-09-16T07:33:33.024155+0000","last_peered":"2024-09-16T07:33:33.024155+0000","last_clean":"2024-09-16T07:33:33.024155+0000","last_became_active":"2024-09-16T07:33:32.041955+0000","last_became_peered":"2024-09-16T07:33:32.041955+0000","last_unstale":"2024-09-16T07:33:33.024155+0000","last_undegraded":"2024-09-16T07:33:33.024155+0000","last_fullsized":"2024-09-16T07:33:33.024155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:45.546271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025124+0000","last_change":"2024-09-16T07:33:31.036453+0000","last_active":"2024-09-16T07:33:33.025124+0000","last_peered":"2024-09-16T07:33:33.025124+0000","last_clean":"2024-09-16T07:33:33.025124+0000","last_became_active":"2024-09-16T07:33:31.034972+0000","last_became_peered":"2024-09-16T07:33:31.034972+0000","last_unstale":"2024-09-16T07:33:33.025124+0000","last_undegraded":"2024-09-16T07:33:33.025124+0000","last_fullsized":"2024-09-16T07:33:33.025124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:49.504210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.031515+0000","last_change":"2024-09-16T07:33:31.031515+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.031515+0000","last_undegraded":"2024-09-16T07:33:31.031515+0000","last_fullsized":"2024-09-16T07:33:31.031515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024241+0000","last_change":"2024-09-16T07:33:31.042034+0000","last_active":"2024-09-16T07:33:33.024241+0000","last_peered":"2024-09-16T07:33:33.024241+0000","last_clean":"2024-09-16T07:33:33.024241+0000","last_became_active":"2024-09-16T07:33:31.041059+0000","last_became_peered":"2024-09-16T07:33:31.041059+0000","last_unstale":"2024-09-16T07:33:33.024241+0000","last_undegraded":"2024-09-16T07:33:33.024241+0000","last_fullsized":"2024-09-16T07:33:33.024241+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:59.775582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357991+0000","last_change":"2024-09-16T07:33:32.046065+0000","last_active":"2024-09-16T07:33:33.357991+0000","last_peered":"2024-09-16T07:33:33.357991+0000","last_clean":"2024-09-16T07:33:33.357991+0000","last_became_active":"2024-09-16T07:33:32.045750+0000","last_became_peered":"2024-09-16T07:33:32.045750+0000","last_unstale":"2024-09-16T07:33:33.357991+0000","last_undegraded":"2024-09-16T07:33:33.357991+0000","last_fullsized":"2024-09-16T07:33:33.357991+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:26.414491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357741+0000","last_change":"2024-09-16T07:33:31.031995+0000","last_active":"2024-09-16T07:33:33.357741+0000","last_peered":"2024-09-16T07:33:33.357741+0000","last_clean":"2024-09-16T07:33:33.357741+0000","last_became_active":"2024-09-16T07:33:31.031131+0000","last_became_peered":"2024-09-16T07:33:31.031131+0000","last_unstale":"2024-09-16T07:33:33.357741+0000","last_undegraded":"2024-09-16T07:33:33.357741+0000","last_fullsized":"2024-09-16T07:33:33.357741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:28.800839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027506+0000","last_change":"2024-09-16T07:33:31.027506+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027506+0000","last_undegraded":"2024-09-16T07:33:31.027506+0000","last_fullsized":"2024-09-16T07:33:31.027506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023950+0000","last_change":"2024-09-16T07:33:31.030808+0000","last_active":"2024-09-16T07:33:33.023950+0000","last_peered":"2024-09-16T07:33:33.023950+0000","last_clean":"2024-09-16T07:33:33.023950+0000","last_became_active":"2024-09-16T07:33:31.028772+0000","last_became_peered":"2024-09-16T07:33:31.028772+0000","last_unstale":"2024-09-16T07:33:33.023950+0000","last_undegraded":"2024-09-16T07:33:33.023950+0000","last_fullsized":"2024-09-16T07:33:33.023950+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:53.744425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358706+0000","last_change":"2024-09-16T07:33:32.045394+0000","last_active":"2024-09-16T07:33:33.358706+0000","last_peered":"2024-09-16T07:33:33.358706+0000","last_clean":"2024-09-16T07:33:33.358706+0000","last_became_active":"2024-09-16T07:33:32.045106+0000","last_became_peered":"2024-09-16T07:33:32.045106+0000","last_unstale":"2024-09-16T07:33:33.358706+0000","last_undegraded":"2024-09-16T07:33:33.358706+0000","last_fullsized":"2024-09-16T07:33:33.358706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.537036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358438+0000","last_change":"2024-09-16T07:33:31.041342+0000","last_active":"2024-09-16T07:33:33.358438+0000","last_peered":"2024-09-16T07:33:33.358438+0000","last_clean":"2024-09-16T07:33:33.358438+0000","last_became_active":"2024-09-16T07:33:31.041056+0000","last_became_peered":"2024-09-16T07:33:31.041056+0000","last_unstale":"2024-09-16T07:33:33.358438+0000","last_undegraded":"2024-09-16T07:33:33.358438+0000","last_fullsized":"2024-09-16T07:33:33.358438+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:39.097159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358460+0000","last_change":"2024-09-16T07:33:32.044491+0000","last_active":"2024-09-16T07:33:33.358460+0000","last_peered":"2024-09-16T07:33:33.358460+0000","last_clean":"2024-09-16T07:33:33.358460+0000","last_became_active":"2024-09-16T07:33:32.044228+0000","last_became_peered":"2024-09-16T07:33:32.044228+0000","last_unstale":"2024-09-16T07:33:33.358460+0000","last_undegraded":"2024-09-16T07:33:33.358460+0000","last_fullsized":"2024-09-16T07:33:33.358460+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:14.023175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357548+0000","last_change":"2024-09-16T07:33:31.034256+0000","last_active":"2024-09-16T07:33:33.357548+0000","last_peered":"2024-09-16T07:33:33.357548+0000","last_clean":"2024-09-16T07:33:33.357548+0000","last_became_active":"2024-09-16T07:33:31.033825+0000","last_became_peered":"2024-09-16T07:33:31.033825+0000","last_unstale":"2024-09-16T07:33:33.357548+0000","last_undegraded":"2024-09-16T07:33:33.357548+0000","last_fullsized":"2024-09-16T07:33:33.357548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:44.857125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358285+0000","last_change":"2024-09-16T07:33:32.041248+0000","last_active":"2024-09-16T07:33:33.358285+0000","last_peered":"2024-09-16T07:33:33.358285+0000","last_clean":"2024-09-16T07:33:33.358285+0000","last_became_active":"2024-09-16T07:33:32.040998+0000","last_became_peered":"2024-09-16T07:33:32.040998+0000","last_unstale":"2024-09-16T07:33:33.358285+0000","last_undegraded":"2024-09-16T07:33:33.358285+0000","last_fullsized":"2024-09-16T07:33:33.358285+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:06.696198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025145+0000","last_change":"2024-09-16T07:33:31.035273+0000","last_active":"2024-09-16T07:33:33.025145+0000","last_peered":"2024-09-16T07:33:33.025145+0000","last_clean":"2024-09-16T07:33:33.025145+0000","last_became_active":"2024-09-16T07:33:31.034957+0000","last_became_peered":"2024-09-16T07:33:31.034957+0000","last_unstale":"2024-09-16T07:33:33.025145+0000","last_undegraded":"2024-09-16T07:33:33.025145+0000","last_fullsized":"2024-09-16T07:33:33.025145+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:17.685015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025196+0000","last_change":"2024-09-16T07:33:32.033426+0000","last_active":"2024-09-16T07:33:33.025196+0000","last_peered":"2024-09-16T07:33:33.025196+0000","last_clean":"2024-09-16T07:33:33.025196+0000","last_became_active":"2024-09-16T07:33:32.032989+0000","last_became_peered":"2024-09-16T07:33:32.032989+0000","last_unstale":"2024-09-16T07:33:33.025196+0000","last_undegraded":"2024-09-16T07:33:33.025196+0000","last_fullsized":"2024-09-16T07:33:33.025196+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:06.277092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023949+0000","last_change":"2024-09-16T07:33:31.038931+0000","last_active":"2024-09-16T07:33:33.023949+0000","last_peered":"2024-09-16T07:33:33.023949+0000","last_clean":"2024-09-16T07:33:33.023949+0000","last_became_active":"2024-09-16T07:33:31.038639+0000","last_became_peered":"2024-09-16T07:33:31.038639+0000","last_unstale":"2024-09-16T07:33:33.023949+0000","last_undegraded":"2024-09-16T07:33:33.023949+0000","last_fullsized":"2024-09-16T07:33:33.023949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:37.191040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025027+0000","last_change":"2024-09-16T07:33:32.050892+0000","last_active":"2024-09-16T07:33:33.025027+0000","last_peered":"2024-09-16T07:33:33.025027+0000","last_clean":"2024-09-16T07:33:33.025027+0000","last_became_active":"2024-09-16T07:33:32.050449+0000","last_became_peered":"2024-09-16T07:33:32.050449+0000","last_unstale":"2024-09-16T07:33:33.025027+0000","last_undegraded":"2024-09-16T07:33:33.025027+0000","last_fullsized":"2024-09-16T07:33:33.025027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:53.956082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358694+0000","last_change":"2024-09-16T07:33:31.035777+0000","last_active":"2024-09-16T07:33:33.358694+0000","last_peered":"2024-09-16T07:33:33.358694+0000","last_clean":"2024-09-16T07:33:33.358694+0000","last_became_active":"2024-09-16T07:33:31.034448+0000","last_became_peered":"2024-09-16T07:33:31.034448+0000","last_unstale":"2024-09-16T07:33:33.358694+0000","last_undegraded":"2024-09-16T07:33:33.358694+0000","last_fullsized":"2024-09-16T07:33:33.358694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:45.713854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025244+0000","last_change":"2024-09-16T07:33:32.052617+0000","last_active":"2024-09-16T07:33:33.025244+0000","last_peered":"2024-09-16T07:33:33.025244+0000","last_clean":"2024-09-16T07:33:33.025244+0000","last_became_active":"2024-09-16T07:33:32.052089+0000","last_became_peered":"2024-09-16T07:33:32.052089+0000","last_unstale":"2024-09-16T07:33:33.025244+0000","last_undegraded":"2024-09-16T07:33:33.025244+0000","last_fullsized":"2024-09-16T07:33:33.025244+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:49.702884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021703+0000","last_change":"2024-09-16T07:33:31.036417+0000","last_active":"2024-09-16T07:33:33.021703+0000","last_peered":"2024-09-16T07:33:33.021703+0000","last_clean":"2024-09-16T07:33:33.021703+0000","last_became_active":"2024-09-16T07:33:31.036118+0000","last_became_peered":"2024-09-16T07:33:31.036118+0000","last_unstale":"2024-09-16T07:33:33.021703+0000","last_undegraded":"2024-09-16T07:33:33.021703+0000","last_fullsized":"2024-09-16T07:33:33.021703+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:50.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358156+0000","last_change":"2024-09-16T07:33:32.039339+0000","last_active":"2024-09-16T07:33:33.358156+0000","last_peered":"2024-09-16T07:33:33.358156+0000","last_clean":"2024-09-16T07:33:33.358156+0000","last_became_active":"2024-09-16T07:33:32.037162+0000","last_became_peered":"2024-09-16T07:33:32.037162+0000","last_unstale":"2024-09-16T07:33:33.358156+0000","last_undegraded":"2024-09-16T07:33:33.358156+0000","last_fullsized":"2024-09-16T07:33:33.358156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:42.850047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045296+0000","last_change":"2024-09-16T07:33:31.042541+0000","last_active":"2024-09-16T07:33:31.045296+0000","last_peered":"2024-09-16T07:33:31.045296+0000","last_clean":"2024-09-16T07:33:31.045296+0000","last_became_active":"2024-09-16T07:33:31.042235+0000","last_became_peered":"2024-09-16T07:33:31.042235+0000","last_unstale":"2024-09-16T07:33:31.045296+0000","last_undegraded":"2024-09-16T07:33:31.045296+0000","last_fullsized":"2024-09-16T07:33:31.045296+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:20.784004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025623+0000","last_change":"2024-09-16T07:33:32.045670+0000","last_active":"2024-09-16T07:33:33.025623+0000","last_peered":"2024-09-16T07:33:33.025623+0000","last_clean":"2024-09-16T07:33:33.025623+0000","last_became_active":"2024-09-16T07:33:32.045383+0000","last_became_peered":"2024-09-16T07:33:32.045383+0000","last_unstale":"2024-09-16T07:33:33.025623+0000","last_undegraded":"2024-09-16T07:33:33.025623+0000","last_fullsized":"2024-09-16T07:33:33.025623+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:56.570489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358500+0000","last_change":"2024-09-16T07:33:31.030651+0000","last_active":"2024-09-16T07:33:33.358500+0000","last_peered":"2024-09-16T07:33:33.358500+0000","last_clean":"2024-09-16T07:33:33.358500+0000","last_became_active":"2024-09-16T07:33:31.030312+0000","last_became_peered":"2024-09-16T07:33:31.030312+0000","last_unstale":"2024-09-16T07:33:33.358500+0000","last_undegraded":"2024-09-16T07:33:33.358500+0000","last_fullsized":"2024-09-16T07:33:33.358500+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:57.124363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025464+0000","last_change":"2024-09-16T07:33:32.045722+0000","last_active":"2024-09-16T07:33:33.025464+0000","last_peered":"2024-09-16T07:33:33.025464+0000","last_clean":"2024-09-16T07:33:33.025464+0000","last_became_active":"2024-09-16T07:33:32.045414+0000","last_became_peered":"2024-09-16T07:33:32.045414+0000","last_unstale":"2024-09-16T07:33:33.025464+0000","last_undegraded":"2024-09-16T07:33:33.025464+0000","last_fullsized":"2024-09-16T07:33:33.025464+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:57.552859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025686+0000","last_change":"2024-09-16T07:33:31.040312+0000","last_active":"2024-09-16T07:33:33.025686+0000","last_peered":"2024-09-16T07:33:33.025686+0000","last_clean":"2024-09-16T07:33:33.025686+0000","last_became_active":"2024-09-16T07:33:31.040009+0000","last_became_peered":"2024-09-16T07:33:31.040009+0000","last_unstale":"2024-09-16T07:33:33.025686+0000","last_undegraded":"2024-09-16T07:33:33.025686+0000","last_fullsized":"2024-09-16T07:33:33.025686+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:56.393574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.032016+0000","last_change":"2024-09-16T07:33:31.032016+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.032016+0000","last_undegraded":"2024-09-16T07:33:31.032016+0000","last_fullsized":"2024-09-16T07:33:31.032016+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025287+0000","last_change":"2024-09-16T07:33:31.033696+0000","last_active":"2024-09-16T07:33:33.025287+0000","last_peered":"2024-09-16T07:33:33.025287+0000","last_clean":"2024-09-16T07:33:33.025287+0000","last_became_active":"2024-09-16T07:33:31.033430+0000","last_became_peered":"2024-09-16T07:33:31.033430+0000","last_unstale":"2024-09-16T07:33:33.025287+0000","last_undegraded":"2024-09-16T07:33:33.025287+0000","last_fullsized":"2024-09-16T07:33:33.025287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:55.604351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357346+0000","last_change":"2024-09-16T07:33:32.042206+0000","last_active":"2024-09-16T07:33:33.357346+0000","last_peered":"2024-09-16T07:33:33.357346+0000","last_clean":"2024-09-16T07:33:33.357346+0000","last_became_active":"2024-09-16T07:33:32.041947+0000","last_became_peered":"2024-09-16T07:33:32.041947+0000","last_unstale":"2024-09-16T07:33:33.357346+0000","last_undegraded":"2024-09-16T07:33:33.357346+0000","last_fullsized":"2024-09-16T07:33:33.357346+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:37:06.473975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358556+0000","last_change":"2024-09-16T07:33:31.032898+0000","last_active":"2024-09-16T07:33:33.358556+0000","last_peered":"2024-09-16T07:33:33.358556+0000","last_clean":"2024-09-16T07:33:33.358556+0000","last_became_active":"2024-09-16T07:33:31.032344+0000","last_became_peered":"2024-09-16T07:33:31.032344+0000","last_unstale":"2024-09-16T07:33:33.358556+0000","last_undegraded":"2024-09-16T07:33:33.358556+0000","last_fullsized":"2024-09-16T07:33:33.358556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:26.380802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025119+0000","last_change":"2024-09-16T07:33:32.038999+0000","last_active":"2024-09-16T07:33:33.025119+0000","last_peered":"2024-09-16T07:33:33.025119+0000","last_clean":"2024-09-16T07:33:33.025119+0000","last_became_active":"2024-09-16T07:33:32.038709+0000","last_became_peered":"2024-09-16T07:33:32.038709+0000","last_unstale":"2024-09-16T07:33:33.025119+0000","last_undegraded":"2024-09-16T07:33:33.025119+0000","last_fullsized":"2024-09-16T07:33:33.025119+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:02.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045928+0000","last_change":"2024-09-16T07:33:31.044406+0000","last_active":"2024-09-16T07:33:31.045928+0000","last_peered":"2024-09-16T07:33:31.045928+0000","last_clean":"2024-09-16T07:33:31.045928+0000","last_became_active":"2024-09-16T07:33:31.044017+0000","last_became_peered":"2024-09-16T07:33:31.044017+0000","last_unstale":"2024-09-16T07:33:31.045928+0000","last_undegraded":"2024-09-16T07:33:31.045928+0000","last_fullsized":"2024-09-16T07:33:31.045928+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:37.852077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357714+0000","last_change":"2024-09-16T07:33:32.046842+0000","last_active":"2024-09-16T07:33:33.357714+0000","last_peered":"2024-09-16T07:33:33.357714+0000","last_clean":"2024-09-16T07:33:33.357714+0000","last_became_active":"2024-09-16T07:33:32.046569+0000","last_became_peered":"2024-09-16T07:33:32.046569+0000","last_unstale":"2024-09-16T07:33:33.357714+0000","last_undegraded":"2024-09-16T07:33:33.357714+0000","last_fullsized":"2024-09-16T07:33:33.357714+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:01.321702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358757+0000","last_change":"2024-09-16T07:33:31.037028+0000","last_active":"2024-09-16T07:33:33.358757+0000","last_peered":"2024-09-16T07:33:33.358757+0000","last_clean":"2024-09-16T07:33:33.358757+0000","last_became_active":"2024-09-16T07:33:31.036714+0000","last_became_peered":"2024-09-16T07:33:31.036714+0000","last_unstale":"2024-09-16T07:33:33.358757+0000","last_undegraded":"2024-09-16T07:33:33.358757+0000","last_fullsized":"2024-09-16T07:33:33.358757+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:01.872929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024139+0000","last_change":"2024-09-16T07:33:32.036865+0000","last_active":"2024-09-16T07:33:33.024139+0000","last_peered":"2024-09-16T07:33:33.024139+0000","last_clean":"2024-09-16T07:33:33.024139+0000","last_became_active":"2024-09-16T07:33:32.036613+0000","last_became_peered":"2024-09-16T07:33:32.036613+0000","last_unstale":"2024-09-16T07:33:33.024139+0000","last_undegraded":"2024-09-16T07:33:33.024139+0000","last_fullsized":"2024-09-16T07:33:33.024139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:29:18.832918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.040193+0000","last_change":"2024-09-16T07:33:31.044776+0000","last_active":"2024-09-16T07:33:32.040193+0000","last_peered":"2024-09-16T07:33:32.040193+0000","last_clean":"2024-09-16T07:33:32.040193+0000","last_became_active":"2024-09-16T07:33:31.044453+0000","last_became_peered":"2024-09-16T07:33:31.044453+0000","last_unstale":"2024-09-16T07:33:32.040193+0000","last_undegraded":"2024-09-16T07:33:32.040193+0000","last_fullsized":"2024-09-16T07:33:32.040193+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:06.219751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358395+0000","last_change":"2024-09-16T07:33:32.043818+0000","last_active":"2024-09-16T07:33:33.358395+0000","last_peered":"2024-09-16T07:33:33.358395+0000","last_clean":"2024-09-16T07:33:33.358395+0000","last_became_active":"2024-09-16T07:33:32.043573+0000","last_became_peered":"2024-09-16T07:33:32.043573+0000","last_unstale":"2024-09-16T07:33:33.358395+0000","last_undegraded":"2024-09-16T07:33:33.358395+0000","last_fullsized":"2024-09-16T07:33:33.358395+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:10.405842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045274+0000","last_change":"2024-09-16T07:33:31.042397+0000","last_active":"2024-09-16T07:33:31.045274+0000","last_peered":"2024-09-16T07:33:31.045274+0000","last_clean":"2024-09-16T07:33:31.045274+0000","last_became_active":"2024-09-16T07:33:31.041205+0000","last_became_peered":"2024-09-16T07:33:31.041205+0000","last_unstale":"2024-09-16T07:33:31.045274+0000","last_undegraded":"2024-09-16T07:33:31.045274+0000","last_fullsized":"2024-09-16T07:33:31.045274+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:13.919862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.030279+0000","last_change":"2024-09-16T07:33:31.030279+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.030279+0000","last_undegraded":"2024-09-16T07:33:31.030279+0000","last_fullsized":"2024-09-16T07:33:31.030279+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358516+0000","last_change":"2024-09-16T07:33:31.041709+0000","last_active":"2024-09-16T07:33:33.358516+0000","last_peered":"2024-09-16T07:33:33.358516+0000","last_clean":"2024-09-16T07:33:33.358516+0000","last_became_active":"2024-09-16T07:33:31.040991+0000","last_became_peered":"2024-09-16T07:33:31.040991+0000","last_unstale":"2024-09-16T07:33:33.358516+0000","last_undegraded":"2024-09-16T07:33:33.358516+0000","last_fullsized":"2024-09-16T07:33:33.358516+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:32.562633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":265,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357091+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:33:33.357091+0000","last_peered":"2024-09-16T07:33:33.357091+0000","last_clean":"2024-09-16T07:33:33.357091+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:33:33.357091+0000","last_undegraded":"2024-09-16T07:33:33.357091+0000","last_fullsized":"2024-09-16T07:33:33.357091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122632,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249208,"statfs":{"total":96636764160,"available":96511188992,"internally_reserved":0,"allocated":1298432,"data_stored":560047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122612,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249228,"statfs":{"total":96636764160,"available":96511209472,"internally_reserved":0,"allocated":1277952,"data_stored":553595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119056,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252784,"statfs":{"total":96636764160,"available":96514850816,"internally_reserved":0,"allocated":1888256,"data_stored":1145617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100400,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271440,"statfs":{"total":96636764160,"available":96533954560,"internally_reserved":0,"allocated":1273856,"data_stored":553525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100412,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271428,"statfs":{"total":96636764160,"available":96533942272,"internally_reserved":0,"allocated":1277952,"data_stored":554107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":101513674},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148984,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222856,"statfs":{"total":96636764160,"available":96484204544,"internally_reserved":0,"allocated":1273856,"data_stored":553525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509575,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127380,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244460,"statfs":{"total":96636764160,"available":96506327040,"internally_reserved":0,"allocated":1900544,"data_stored":1151889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542277,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122584,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249256,"statfs":{"total":96636764160,"available":96511238144,"internally_reserved":0,"allocated":1249280,"data_stored":540419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:33:36.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:33:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.844+0000 7f53c3000640 1 -- 172.21.15.71:0/1620752252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53bc1612d0 msgr2=0x7f53bc0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.844+0000 7f53c3000640 1 --2- 172.21.15.71:0/1620752252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53bc1612d0 0x7f53bc0b4e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009860 tx=0x7f53b002a5a0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.844+0000 7f53c3000640 1 -- 172.21.15.71:0/1620752252 shutdown_connections 2024-09-16T07:33:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.844+0000 7f53c3000640 1 --2- 172.21.15.71:0/1620752252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53bc1612d0 0x7f53bc0b4e00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.844+0000 7f53c3000640 1 -- 172.21.15.71:0/1620752252 >> 172.21.15.71:0/1620752252 conn(0x7f53bc0d3df0 msgr2=0x7f53bc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:36.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.844+0000 7f53c3000640 1 -- 172.21.15.71:0/1620752252 shutdown_connections 2024-09-16T07:33:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.844+0000 7f53c3000640 1 -- 172.21.15.71:0/1620752252 wait complete. 2024-09-16T07:33:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.846+0000 7f53c3000640 1 Processor -- start 2024-09-16T07:33:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.846+0000 7f53c3000640 1 -- start start 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c3000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53bc0d4980 0x7f53bc15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c3000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53bc1612d0 0x7f53bc0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc1616b0 0x7f53bc0c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc0b1ee0 con 0x7f53bc1616b0 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c3000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc0b1d60 con 0x7f53bc1612d0 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c3000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f53bc0b2060 con 0x7f53bc0d4980 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c2600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc1616b0 0x7f53bc0c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c2600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc1616b0 0x7f53bc0c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42276/0 (socket says 172.21.15.71:42276) 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c2600640 1 -- 172.21.15.71:0/1606138454 learned_addr learned my addr 172.21.15.71:0/1606138454 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c1c00640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53bc0d4980 0x7f53bc15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.847+0000 7f53c1200640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53bc1612d0 0x7f53bc0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c2600640 1 -- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53bc0d4980 msgr2=0x7f53bc15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c2600640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53bc0d4980 0x7f53bc15b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c2600640 1 -- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53bc1612d0 msgr2=0x7f53bc0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c2600640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53bc1612d0 0x7f53bc0c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c2600640 1 -- 172.21.15.71:0/1606138454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc0c74f0 con 0x7f53bc1616b0 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c1200640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53bc1612d0 0x7f53bc0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c2600640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc1616b0 0x7f53bc0c93d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f53b400c700 tx=0x7f53b400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53b9600640 1 -- 172.21.15.71:0/1606138454 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4003b70 con 0x7f53bc1616b0 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53b9600640 1 -- 172.21.15.71:0/1606138454 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53b4003d10 con 0x7f53bc1616b0 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53b9600640 1 -- 172.21.15.71:0/1606138454 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b400aab0 con 0x7f53bc1616b0 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc0c77d0 con 0x7f53bc1616b0 2024-09-16T07:33:36.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.848+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc0c7d90 con 0x7f53bc1616b0 2024-09-16T07:33:36.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.849+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc0b2580 con 0x7f53bc1616b0 2024-09-16T07:33:36.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.851+0000 7f53b9600640 1 -- 172.21.15.71:0/1606138454 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f53b400ac50 con 0x7f53bc1616b0 2024-09-16T07:33:36.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.852+0000 7f53b9600640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f539c079e40 0x7f539c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:36.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.852+0000 7f53b9600640 1 -- 172.21.15.71:0/1606138454 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f53b4099800 con 0x7f53bc1616b0 2024-09-16T07:33:36.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.853+0000 7f53c1c00640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f539c079e40 0x7f539c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:36.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.854+0000 7f53c1c00640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f539c079e40 0x7f539c07c300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f53b0034000 tx=0x7f53b0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:36.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:36.857+0000 7f53b9600640 1 -- 172.21.15.71:0/1606138454 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f53b40634b0 con 0x7f53bc1616b0 2024-09-16T07:33:37.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.040+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53bc06e790 con 0x7f539c079e40 2024-09-16T07:33:37.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.045+0000 7f53b9600640 1 -- 172.21.15.71:0/1606138454 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251762 (secure 0 0 0) 0x7f53bc06e790 con 0x7f539c079e40 2024-09-16T07:33:37.047 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:37.048 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:33:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f539c079e40 msgr2=0x7f539c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f539c079e40 0x7f539c07c300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f53b0034000 tx=0x7f53b0004730 comp rx=0 tx=0).stop 2024-09-16T07:33:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc1616b0 msgr2=0x7f53bc0c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc1616b0 0x7f53bc0c93d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f53b400c700 tx=0x7f53b400cbc0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 shutdown_connections 2024-09-16T07:33:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f539c079e40 0x7f539c07c300 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53bc1616b0 0x7f53bc0c93d0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53bc1612d0 0x7f53bc0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 --2- 172.21.15.71:0/1606138454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53bc0d4980 0x7f53bc15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.049+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 >> 172.21.15.71:0/1606138454 conn(0x7f53bc0d3df0 msgr2=0x7f53bc160570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.050+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 shutdown_connections 2024-09-16T07:33:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.050+0000 7f53c3000640 1 -- 172.21.15.71:0/1606138454 wait complete. 2024-09-16T07:33:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-16T07:33:36.581316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":964060,"kb_used_data":11172,"kb_used_omap":12,"kb_used_meta":952691,"kb_avail":754010660,"statfs":{"total":773094113280,"available":772106915840,"internally_reserved":0,"allocated":11440128,"data_stored":5612724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":975556144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001607"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025017+0000","last_change":"2024-09-16T07:33:32.045068+0000","last_active":"2024-09-16T07:33:33.025017+0000","last_peered":"2024-09-16T07:33:33.025017+0000","last_clean":"2024-09-16T07:33:33.025017+0000","last_became_active":"2024-09-16T07:33:32.044772+0000","last_became_peered":"2024-09-16T07:33:32.044772+0000","last_unstale":"2024-09-16T07:33:33.025017+0000","last_undegraded":"2024-09-16T07:33:33.025017+0000","last_fullsized":"2024-09-16T07:33:33.025017+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024784+0000","last_change":"2024-09-16T07:33:32.049940+0000","last_active":"2024-09-16T07:33:33.024784+0000","last_peered":"2024-09-16T07:33:33.024784+0000","last_clean":"2024-09-16T07:33:33.024784+0000","last_became_active":"2024-09-16T07:33:32.049646+0000","last_became_peered":"2024-09-16T07:33:32.049646+0000","last_unstale":"2024-09-16T07:33:33.024784+0000","last_undegraded":"2024-09-16T07:33:33.024784+0000","last_fullsized":"2024-09-16T07:33:33.024784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.031577+0000","last_change":"2024-09-16T07:33:31.031577+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.031577+0000","last_undegraded":"2024-09-16T07:33:31.031577+0000","last_fullsized":"2024-09-16T07:33:31.031577+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024064+0000","last_change":"2024-09-16T07:33:32.043883+0000","last_active":"2024-09-16T07:33:33.024064+0000","last_peered":"2024-09-16T07:33:33.024064+0000","last_clean":"2024-09-16T07:33:33.024064+0000","last_became_active":"2024-09-16T07:33:32.043622+0000","last_became_peered":"2024-09-16T07:33:32.043622+0000","last_unstale":"2024-09-16T07:33:33.024064+0000","last_undegraded":"2024-09-16T07:33:33.024064+0000","last_fullsized":"2024-09-16T07:33:33.024064+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:01:49.570122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357403+0000","last_change":"2024-09-16T07:33:32.043899+0000","last_active":"2024-09-16T07:33:33.357403+0000","last_peered":"2024-09-16T07:33:33.357403+0000","last_clean":"2024-09-16T07:33:33.357403+0000","last_became_active":"2024-09-16T07:33:32.043611+0000","last_became_peered":"2024-09-16T07:33:32.043611+0000","last_unstale":"2024-09-16T07:33:33.357403+0000","last_undegraded":"2024-09-16T07:33:33.357403+0000","last_fullsized":"2024-09-16T07:33:33.357403+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:22.213386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357942+0000","last_change":"2024-09-16T07:33:32.054833+0000","last_active":"2024-09-16T07:33:33.357942+0000","last_peered":"2024-09-16T07:33:33.357942+0000","last_clean":"2024-09-16T07:33:33.357942+0000","last_became_active":"2024-09-16T07:33:32.054513+0000","last_became_peered":"2024-09-16T07:33:32.054513+0000","last_unstale":"2024-09-16T07:33:33.357942+0000","last_undegraded":"2024-09-16T07:33:33.357942+0000","last_fullsized":"2024-09-16T07:33:33.357942+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:03.590298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357327+0000","last_change":"2024-09-16T07:33:32.037656+0000","last_active":"2024-09-16T07:33:33.357327+0000","last_peered":"2024-09-16T07:33:33.357327+0000","last_clean":"2024-09-16T07:33:33.357327+0000","last_became_active":"2024-09-16T07:33:32.036949+0000","last_became_peered":"2024-09-16T07:33:32.036949+0000","last_unstale":"2024-09-16T07:33:33.357327+0000","last_undegraded":"2024-09-16T07:33:33.357327+0000","last_fullsized":"2024-09-16T07:33:33.357327+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:18.915383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.025834+0000","last_change":"2024-09-16T07:33:31.025834+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.025834+0000","last_undegraded":"2024-09-16T07:33:31.025834+0000","last_fullsized":"2024-09-16T07:33:31.025834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357756+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.357756+0000","last_peered":"2024-09-16T07:33:33.357756+0000","last_clean":"2024-09-16T07:33:33.357756+0000","last_became_active":"2024-09-16T07:33:32.035287+0000","last_became_peered":"2024-09-16T07:33:32.035287+0000","last_unstale":"2024-09-16T07:33:33.357756+0000","last_undegraded":"2024-09-16T07:33:33.357756+0000","last_fullsized":"2024-09-16T07:33:33.357756+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:52.049650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021843+0000","last_change":"2024-09-16T07:33:32.049217+0000","last_active":"2024-09-16T07:33:33.021843+0000","last_peered":"2024-09-16T07:33:33.021843+0000","last_clean":"2024-09-16T07:33:33.021843+0000","last_became_active":"2024-09-16T07:33:32.048953+0000","last_became_peered":"2024-09-16T07:33:32.048953+0000","last_unstale":"2024-09-16T07:33:33.021843+0000","last_undegraded":"2024-09-16T07:33:33.021843+0000","last_fullsized":"2024-09-16T07:33:33.021843+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357715+0000","last_change":"2024-09-16T07:33:32.054961+0000","last_active":"2024-09-16T07:33:33.357715+0000","last_peered":"2024-09-16T07:33:33.357715+0000","last_clean":"2024-09-16T07:33:33.357715+0000","last_became_active":"2024-09-16T07:33:32.054463+0000","last_became_peered":"2024-09-16T07:33:32.054463+0000","last_unstale":"2024-09-16T07:33:33.357715+0000","last_undegraded":"2024-09-16T07:33:33.357715+0000","last_fullsized":"2024-09-16T07:33:33.357715+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:57.790799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358257+0000","last_change":"2024-09-16T07:33:32.040365+0000","last_active":"2024-09-16T07:33:33.358257+0000","last_peered":"2024-09-16T07:33:33.358257+0000","last_clean":"2024-09-16T07:33:33.358257+0000","last_became_active":"2024-09-16T07:33:32.037953+0000","last_became_peered":"2024-09-16T07:33:32.037953+0000","last_unstale":"2024-09-16T07:33:33.358257+0000","last_undegraded":"2024-09-16T07:33:33.358257+0000","last_fullsized":"2024-09-16T07:33:33.358257+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:32.112813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.059035+0000","last_change":"2024-09-16T07:33:32.056887+0000","last_active":"2024-09-16T07:33:32.059035+0000","last_peered":"2024-09-16T07:33:32.059035+0000","last_clean":"2024-09-16T07:33:32.059035+0000","last_became_active":"2024-09-16T07:33:32.056276+0000","last_became_peered":"2024-09-16T07:33:32.056276+0000","last_unstale":"2024-09-16T07:33:32.059035+0000","last_undegraded":"2024-09-16T07:33:32.059035+0000","last_fullsized":"2024-09-16T07:33:32.059035+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:23.723158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023495+0000","last_change":"2024-09-16T07:33:32.054384+0000","last_active":"2024-09-16T07:33:33.023495+0000","last_peered":"2024-09-16T07:33:33.023495+0000","last_clean":"2024-09-16T07:33:33.023495+0000","last_became_active":"2024-09-16T07:33:32.051888+0000","last_became_peered":"2024-09-16T07:33:32.051888+0000","last_unstale":"2024-09-16T07:33:33.023495+0000","last_undegraded":"2024-09-16T07:33:33.023495+0000","last_fullsized":"2024-09-16T07:33:33.023495+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:19.796812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023506+0000","last_change":"2024-09-16T07:33:32.040802+0000","last_active":"2024-09-16T07:33:33.023506+0000","last_peered":"2024-09-16T07:33:33.023506+0000","last_clean":"2024-09-16T07:33:33.023506+0000","last_became_active":"2024-09-16T07:33:32.040510+0000","last_became_peered":"2024-09-16T07:33:32.040510+0000","last_unstale":"2024-09-16T07:33:33.023506+0000","last_undegraded":"2024-09-16T07:33:33.023506+0000","last_fullsized":"2024-09-16T07:33:33.023506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:53:27.647859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357155+0000","last_change":"2024-09-16T07:33:32.039107+0000","last_active":"2024-09-16T07:33:33.357155+0000","last_peered":"2024-09-16T07:33:33.357155+0000","last_clean":"2024-09-16T07:33:33.357155+0000","last_became_active":"2024-09-16T07:33:32.038828+0000","last_became_peered":"2024-09-16T07:33:32.038828+0000","last_unstale":"2024-09-16T07:33:33.357155+0000","last_undegraded":"2024-09-16T07:33:33.357155+0000","last_fullsized":"2024-09-16T07:33:33.357155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:15.415755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025462+0000","last_change":"2024-09-16T07:33:32.053433+0000","last_active":"2024-09-16T07:33:33.025462+0000","last_peered":"2024-09-16T07:33:33.025462+0000","last_clean":"2024-09-16T07:33:33.025462+0000","last_became_active":"2024-09-16T07:33:32.052843+0000","last_became_peered":"2024-09-16T07:33:32.052843+0000","last_unstale":"2024-09-16T07:33:33.025462+0000","last_undegraded":"2024-09-16T07:33:33.025462+0000","last_fullsized":"2024-09-16T07:33:33.025462+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:13.265002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025375+0000","last_change":"2024-09-16T07:33:32.047389+0000","last_active":"2024-09-16T07:33:33.025375+0000","last_peered":"2024-09-16T07:33:33.025375+0000","last_clean":"2024-09-16T07:33:33.025375+0000","last_became_active":"2024-09-16T07:33:32.047043+0000","last_became_peered":"2024-09-16T07:33:32.047043+0000","last_unstale":"2024-09-16T07:33:33.025375+0000","last_undegraded":"2024-09-16T07:33:33.025375+0000","last_fullsized":"2024-09-16T07:33:33.025375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024761+0000","last_change":"2024-09-16T07:33:32.040875+0000","last_active":"2024-09-16T07:33:33.024761+0000","last_peered":"2024-09-16T07:33:33.024761+0000","last_clean":"2024-09-16T07:33:33.024761+0000","last_became_active":"2024-09-16T07:33:32.040636+0000","last_became_peered":"2024-09-16T07:33:32.040636+0000","last_unstale":"2024-09-16T07:33:33.024761+0000","last_undegraded":"2024-09-16T07:33:33.024761+0000","last_fullsized":"2024-09-16T07:33:33.024761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:06.934545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358353+0000","last_change":"2024-09-16T07:33:32.053936+0000","last_active":"2024-09-16T07:33:33.358353+0000","last_peered":"2024-09-16T07:33:33.358353+0000","last_clean":"2024-09-16T07:33:33.358353+0000","last_became_active":"2024-09-16T07:33:32.053546+0000","last_became_peered":"2024-09-16T07:33:32.053546+0000","last_unstale":"2024-09-16T07:33:33.358353+0000","last_undegraded":"2024-09-16T07:33:33.358353+0000","last_fullsized":"2024-09-16T07:33:33.358353+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.032081+0000","last_change":"2024-09-16T07:33:31.032081+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.032081+0000","last_undegraded":"2024-09-16T07:33:31.032081+0000","last_fullsized":"2024-09-16T07:33:31.032081+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027983+0000","last_change":"2024-09-16T07:33:31.027983+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027983+0000","last_undegraded":"2024-09-16T07:33:31.027983+0000","last_fullsized":"2024-09-16T07:33:31.027983+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023549+0000","last_change":"2024-09-16T07:33:32.055235+0000","last_active":"2024-09-16T07:33:33.023549+0000","last_peered":"2024-09-16T07:33:33.023549+0000","last_clean":"2024-09-16T07:33:33.023549+0000","last_became_active":"2024-09-16T07:33:32.052675+0000","last_became_peered":"2024-09-16T07:33:32.052675+0000","last_unstale":"2024-09-16T07:33:33.023549+0000","last_undegraded":"2024-09-16T07:33:33.023549+0000","last_fullsized":"2024-09-16T07:33:33.023549+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:31.315404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357746+0000","last_change":"2024-09-16T07:33:32.050486+0000","last_active":"2024-09-16T07:33:33.357746+0000","last_peered":"2024-09-16T07:33:33.357746+0000","last_clean":"2024-09-16T07:33:33.357746+0000","last_became_active":"2024-09-16T07:33:32.050108+0000","last_became_peered":"2024-09-16T07:33:32.050108+0000","last_unstale":"2024-09-16T07:33:33.357746+0000","last_undegraded":"2024-09-16T07:33:33.357746+0000","last_fullsized":"2024-09-16T07:33:33.357746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021955+0000","last_change":"2024-09-16T07:33:32.028117+0000","last_active":"2024-09-16T07:33:33.021955+0000","last_peered":"2024-09-16T07:33:33.021955+0000","last_clean":"2024-09-16T07:33:33.021955+0000","last_became_active":"2024-09-16T07:33:32.027826+0000","last_became_peered":"2024-09-16T07:33:32.027826+0000","last_unstale":"2024-09-16T07:33:33.021955+0000","last_undegraded":"2024-09-16T07:33:33.021955+0000","last_fullsized":"2024-09-16T07:33:33.021955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:35.757466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.055382+0000","last_change":"2024-09-16T07:33:32.049150+0000","last_active":"2024-09-16T07:33:32.055382+0000","last_peered":"2024-09-16T07:33:32.055382+0000","last_clean":"2024-09-16T07:33:32.055382+0000","last_became_active":"2024-09-16T07:33:32.048893+0000","last_became_peered":"2024-09-16T07:33:32.048893+0000","last_unstale":"2024-09-16T07:33:32.055382+0000","last_undegraded":"2024-09-16T07:33:32.055382+0000","last_fullsized":"2024-09-16T07:33:32.055382+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:37.326467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358162+0000","last_change":"2024-09-16T07:33:32.041462+0000","last_active":"2024-09-16T07:33:33.358162+0000","last_peered":"2024-09-16T07:33:33.358162+0000","last_clean":"2024-09-16T07:33:33.358162+0000","last_became_active":"2024-09-16T07:33:32.041198+0000","last_became_peered":"2024-09-16T07:33:32.041198+0000","last_unstale":"2024-09-16T07:33:33.358162+0000","last_undegraded":"2024-09-16T07:33:33.358162+0000","last_fullsized":"2024-09-16T07:33:33.358162+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358295+0000","last_change":"2024-09-16T07:33:32.037402+0000","last_active":"2024-09-16T07:33:33.358295+0000","last_peered":"2024-09-16T07:33:33.358295+0000","last_clean":"2024-09-16T07:33:33.358295+0000","last_became_active":"2024-09-16T07:33:32.037131+0000","last_became_peered":"2024-09-16T07:33:32.037131+0000","last_unstale":"2024-09-16T07:33:33.358295+0000","last_undegraded":"2024-09-16T07:33:33.358295+0000","last_fullsized":"2024-09-16T07:33:33.358295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.052439+0000","last_change":"2024-09-16T07:33:32.043318+0000","last_active":"2024-09-16T07:33:32.052439+0000","last_peered":"2024-09-16T07:33:32.052439+0000","last_clean":"2024-09-16T07:33:32.052439+0000","last_became_active":"2024-09-16T07:33:32.043039+0000","last_became_peered":"2024-09-16T07:33:32.043039+0000","last_unstale":"2024-09-16T07:33:32.052439+0000","last_undegraded":"2024-09-16T07:33:32.052439+0000","last_fullsized":"2024-09-16T07:33:32.052439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:45.700538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.059020+0000","last_change":"2024-09-16T07:33:32.056817+0000","last_active":"2024-09-16T07:33:32.059020+0000","last_peered":"2024-09-16T07:33:32.059020+0000","last_clean":"2024-09-16T07:33:32.059020+0000","last_became_active":"2024-09-16T07:33:32.056535+0000","last_became_peered":"2024-09-16T07:33:32.056535+0000","last_unstale":"2024-09-16T07:33:32.059020+0000","last_undegraded":"2024-09-16T07:33:32.059020+0000","last_fullsized":"2024-09-16T07:33:32.059020+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:35.876240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027968+0000","last_change":"2024-09-16T07:33:31.027968+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027968+0000","last_undegraded":"2024-09-16T07:33:31.027968+0000","last_fullsized":"2024-09-16T07:33:31.027968+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023806+0000","last_change":"2024-09-16T07:33:32.036224+0000","last_active":"2024-09-16T07:33:33.023806+0000","last_peered":"2024-09-16T07:33:33.023806+0000","last_clean":"2024-09-16T07:33:33.023806+0000","last_became_active":"2024-09-16T07:33:32.035020+0000","last_became_peered":"2024-09-16T07:33:32.035020+0000","last_unstale":"2024-09-16T07:33:33.023806+0000","last_undegraded":"2024-09-16T07:33:33.023806+0000","last_fullsized":"2024-09-16T07:33:33.023806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:00.997013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025689+0000","last_change":"2024-09-16T07:33:32.046309+0000","last_active":"2024-09-16T07:33:33.025689+0000","last_peered":"2024-09-16T07:33:33.025689+0000","last_clean":"2024-09-16T07:33:33.025689+0000","last_became_active":"2024-09-16T07:33:32.045999+0000","last_became_peered":"2024-09-16T07:33:32.045999+0000","last_unstale":"2024-09-16T07:33:33.025689+0000","last_undegraded":"2024-09-16T07:33:33.025689+0000","last_fullsized":"2024-09-16T07:33:33.025689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:09.445884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.053925+0000","last_change":"2024-09-16T07:33:31.045574+0000","last_active":"2024-09-16T07:33:32.053925+0000","last_peered":"2024-09-16T07:33:32.053925+0000","last_clean":"2024-09-16T07:33:32.053925+0000","last_became_active":"2024-09-16T07:33:31.045014+0000","last_became_peered":"2024-09-16T07:33:31.045014+0000","last_unstale":"2024-09-16T07:33:32.053925+0000","last_undegraded":"2024-09-16T07:33:32.053925+0000","last_fullsized":"2024-09-16T07:33:32.053925+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.581864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.052698+0000","last_change":"2024-09-16T07:33:32.046389+0000","last_active":"2024-09-16T07:33:32.052698+0000","last_peered":"2024-09-16T07:33:32.052698+0000","last_clean":"2024-09-16T07:33:32.052698+0000","last_became_active":"2024-09-16T07:33:32.046041+0000","last_became_peered":"2024-09-16T07:33:32.046041+0000","last_unstale":"2024-09-16T07:33:32.052698+0000","last_undegraded":"2024-09-16T07:33:32.052698+0000","last_fullsized":"2024-09-16T07:33:32.052698+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:20.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357662+0000","last_change":"2024-09-16T07:33:31.041205+0000","last_active":"2024-09-16T07:33:33.357662+0000","last_peered":"2024-09-16T07:33:33.357662+0000","last_clean":"2024-09-16T07:33:33.357662+0000","last_became_active":"2024-09-16T07:33:31.040923+0000","last_became_peered":"2024-09-16T07:33:31.040923+0000","last_unstale":"2024-09-16T07:33:33.357662+0000","last_undegraded":"2024-09-16T07:33:33.357662+0000","last_fullsized":"2024-09-16T07:33:33.357662+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:56.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024526+0000","last_change":"2024-09-16T07:33:32.039150+0000","last_active":"2024-09-16T07:33:33.024526+0000","last_peered":"2024-09-16T07:33:33.024526+0000","last_clean":"2024-09-16T07:33:33.024526+0000","last_became_active":"2024-09-16T07:33:32.038837+0000","last_became_peered":"2024-09-16T07:33:32.038837+0000","last_unstale":"2024-09-16T07:33:33.024526+0000","last_undegraded":"2024-09-16T07:33:33.024526+0000","last_fullsized":"2024-09-16T07:33:33.024526+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:52.257876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.046999+0000","last_change":"2024-09-16T07:33:31.045183+0000","last_active":"2024-09-16T07:33:31.046999+0000","last_peered":"2024-09-16T07:33:31.046999+0000","last_clean":"2024-09-16T07:33:31.046999+0000","last_became_active":"2024-09-16T07:33:31.044253+0000","last_became_peered":"2024-09-16T07:33:31.044253+0000","last_unstale":"2024-09-16T07:33:31.046999+0000","last_undegraded":"2024-09-16T07:33:31.046999+0000","last_fullsized":"2024-09-16T07:33:31.046999+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:15:28.325154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021838+0000","last_change":"2024-09-16T07:33:32.043094+0000","last_active":"2024-09-16T07:33:33.021838+0000","last_peered":"2024-09-16T07:33:33.021838+0000","last_clean":"2024-09-16T07:33:33.021838+0000","last_became_active":"2024-09-16T07:33:32.042832+0000","last_became_peered":"2024-09-16T07:33:32.042832+0000","last_unstale":"2024-09-16T07:33:33.021838+0000","last_undegraded":"2024-09-16T07:33:33.021838+0000","last_fullsized":"2024-09-16T07:33:33.021838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:24.827070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024550+0000","last_change":"2024-09-16T07:33:31.042458+0000","last_active":"2024-09-16T07:33:33.024550+0000","last_peered":"2024-09-16T07:33:33.024550+0000","last_clean":"2024-09-16T07:33:33.024550+0000","last_became_active":"2024-09-16T07:33:31.042226+0000","last_became_peered":"2024-09-16T07:33:31.042226+0000","last_unstale":"2024-09-16T07:33:33.024550+0000","last_undegraded":"2024-09-16T07:33:33.024550+0000","last_fullsized":"2024-09-16T07:33:33.024550+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:20.182053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357806+0000","last_change":"2024-09-16T07:33:32.039748+0000","last_active":"2024-09-16T07:33:33.357806+0000","last_peered":"2024-09-16T07:33:33.357806+0000","last_clean":"2024-09-16T07:33:33.357806+0000","last_became_active":"2024-09-16T07:33:32.038397+0000","last_became_peered":"2024-09-16T07:33:32.038397+0000","last_unstale":"2024-09-16T07:33:33.357806+0000","last_undegraded":"2024-09-16T07:33:33.357806+0000","last_fullsized":"2024-09-16T07:33:33.357806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:15.732874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023864+0000","last_change":"2024-09-16T07:33:31.033141+0000","last_active":"2024-09-16T07:33:33.023864+0000","last_peered":"2024-09-16T07:33:33.023864+0000","last_clean":"2024-09-16T07:33:33.023864+0000","last_became_active":"2024-09-16T07:33:31.032570+0000","last_became_peered":"2024-09-16T07:33:31.032570+0000","last_unstale":"2024-09-16T07:33:33.023864+0000","last_undegraded":"2024-09-16T07:33:33.023864+0000","last_fullsized":"2024-09-16T07:33:33.023864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:26.450637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357619+0000","last_change":"2024-09-16T07:33:32.044965+0000","last_active":"2024-09-16T07:33:33.357619+0000","last_peered":"2024-09-16T07:33:33.357619+0000","last_clean":"2024-09-16T07:33:33.357619+0000","last_became_active":"2024-09-16T07:33:32.044705+0000","last_became_peered":"2024-09-16T07:33:32.044705+0000","last_unstale":"2024-09-16T07:33:33.357619+0000","last_undegraded":"2024-09-16T07:33:33.357619+0000","last_fullsized":"2024-09-16T07:33:33.357619+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:50.022797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.040511+0000","last_change":"2024-09-16T07:33:31.035805+0000","last_active":"2024-09-16T07:33:31.040511+0000","last_peered":"2024-09-16T07:33:31.040511+0000","last_clean":"2024-09-16T07:33:31.040511+0000","last_became_active":"2024-09-16T07:33:31.035545+0000","last_became_peered":"2024-09-16T07:33:31.035545+0000","last_unstale":"2024-09-16T07:33:31.040511+0000","last_undegraded":"2024-09-16T07:33:31.040511+0000","last_fullsized":"2024-09-16T07:33:31.040511+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:16.651576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.053801+0000","last_change":"2024-09-16T07:33:32.051200+0000","last_active":"2024-09-16T07:33:32.053801+0000","last_peered":"2024-09-16T07:33:32.053801+0000","last_clean":"2024-09-16T07:33:32.053801+0000","last_became_active":"2024-09-16T07:33:32.047036+0000","last_became_peered":"2024-09-16T07:33:32.047036+0000","last_unstale":"2024-09-16T07:33:32.053801+0000","last_undegraded":"2024-09-16T07:33:32.053801+0000","last_fullsized":"2024-09-16T07:33:32.053801+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:23.177353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357424+0000","last_change":"2024-09-16T07:33:31.034485+0000","last_active":"2024-09-16T07:33:33.357424+0000","last_peered":"2024-09-16T07:33:33.357424+0000","last_clean":"2024-09-16T07:33:33.357424+0000","last_became_active":"2024-09-16T07:33:31.034236+0000","last_became_peered":"2024-09-16T07:33:31.034236+0000","last_unstale":"2024-09-16T07:33:33.357424+0000","last_undegraded":"2024-09-16T07:33:33.357424+0000","last_fullsized":"2024-09-16T07:33:33.357424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:52.133642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357028+0000","last_change":"2024-09-16T07:33:32.039686+0000","last_active":"2024-09-16T07:33:33.357028+0000","last_peered":"2024-09-16T07:33:33.357028+0000","last_clean":"2024-09-16T07:33:33.357028+0000","last_became_active":"2024-09-16T07:33:32.038343+0000","last_became_peered":"2024-09-16T07:33:32.038343+0000","last_unstale":"2024-09-16T07:33:33.357028+0000","last_undegraded":"2024-09-16T07:33:33.357028+0000","last_fullsized":"2024-09-16T07:33:33.357028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:00.455428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024348+0000","last_change":"2024-09-16T07:33:31.039439+0000","last_active":"2024-09-16T07:33:33.024348+0000","last_peered":"2024-09-16T07:33:33.024348+0000","last_clean":"2024-09-16T07:33:33.024348+0000","last_became_active":"2024-09-16T07:33:31.039029+0000","last_became_peered":"2024-09-16T07:33:31.039029+0000","last_unstale":"2024-09-16T07:33:33.024348+0000","last_undegraded":"2024-09-16T07:33:33.024348+0000","last_fullsized":"2024-09-16T07:33:33.024348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:21.209706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025089+0000","last_change":"2024-09-16T07:33:32.045355+0000","last_active":"2024-09-16T07:33:33.025089+0000","last_peered":"2024-09-16T07:33:33.025089+0000","last_clean":"2024-09-16T07:33:33.025089+0000","last_became_active":"2024-09-16T07:33:32.045070+0000","last_became_peered":"2024-09-16T07:33:32.045070+0000","last_unstale":"2024-09-16T07:33:33.025089+0000","last_undegraded":"2024-09-16T07:33:33.025089+0000","last_fullsized":"2024-09-16T07:33:33.025089+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.042006+0000","last_change":"2024-09-16T07:33:31.039400+0000","last_active":"2024-09-16T07:33:31.042006+0000","last_peered":"2024-09-16T07:33:31.042006+0000","last_clean":"2024-09-16T07:33:31.042006+0000","last_became_active":"2024-09-16T07:33:31.039119+0000","last_became_peered":"2024-09-16T07:33:31.039119+0000","last_unstale":"2024-09-16T07:33:31.042006+0000","last_undegraded":"2024-09-16T07:33:31.042006+0000","last_fullsized":"2024-09-16T07:33:31.042006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:34.147476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358517+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.358517+0000","last_peered":"2024-09-16T07:33:33.358517+0000","last_clean":"2024-09-16T07:33:33.358517+0000","last_became_active":"2024-09-16T07:33:32.035316+0000","last_became_peered":"2024-09-16T07:33:32.035316+0000","last_unstale":"2024-09-16T07:33:33.358517+0000","last_undegraded":"2024-09-16T07:33:33.358517+0000","last_fullsized":"2024-09-16T07:33:33.358517+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:06.190971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025197+0000","last_change":"2024-09-16T07:33:31.037265+0000","last_active":"2024-09-16T07:33:33.025197+0000","last_peered":"2024-09-16T07:33:33.025197+0000","last_clean":"2024-09-16T07:33:33.025197+0000","last_became_active":"2024-09-16T07:33:31.036981+0000","last_became_peered":"2024-09-16T07:33:31.036981+0000","last_unstale":"2024-09-16T07:33:33.025197+0000","last_undegraded":"2024-09-16T07:33:33.025197+0000","last_fullsized":"2024-09-16T07:33:33.025197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:08.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357328+0000","last_change":"2024-09-16T07:33:32.042815+0000","last_active":"2024-09-16T07:33:33.357328+0000","last_peered":"2024-09-16T07:33:33.357328+0000","last_clean":"2024-09-16T07:33:33.357328+0000","last_became_active":"2024-09-16T07:33:32.042470+0000","last_became_peered":"2024-09-16T07:33:32.042470+0000","last_unstale":"2024-09-16T07:33:33.357328+0000","last_undegraded":"2024-09-16T07:33:33.357328+0000","last_fullsized":"2024-09-16T07:33:33.357328+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:39.595734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357447+0000","last_change":"2024-09-16T07:33:31.032993+0000","last_active":"2024-09-16T07:33:33.357447+0000","last_peered":"2024-09-16T07:33:33.357447+0000","last_clean":"2024-09-16T07:33:33.357447+0000","last_became_active":"2024-09-16T07:33:31.032741+0000","last_became_peered":"2024-09-16T07:33:31.032741+0000","last_unstale":"2024-09-16T07:33:33.357447+0000","last_undegraded":"2024-09-16T07:33:33.357447+0000","last_fullsized":"2024-09-16T07:33:33.357447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:03.607602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357971+0000","last_change":"2024-09-16T07:33:32.035426+0000","last_active":"2024-09-16T07:33:33.357971+0000","last_peered":"2024-09-16T07:33:33.357971+0000","last_clean":"2024-09-16T07:33:33.357971+0000","last_became_active":"2024-09-16T07:33:32.034929+0000","last_became_peered":"2024-09-16T07:33:32.034929+0000","last_unstale":"2024-09-16T07:33:33.357971+0000","last_undegraded":"2024-09-16T07:33:33.357971+0000","last_fullsized":"2024-09-16T07:33:33.357971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":21,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358359+0000","last_change":"2024-09-16T07:33:31.041833+0000","last_active":"2024-09-16T07:33:33.358359+0000","last_peered":"2024-09-16T07:33:33.358359+0000","last_clean":"2024-09-16T07:33:33.358359+0000","last_became_active":"2024-09-16T07:33:31.041034+0000","last_became_peered":"2024-09-16T07:33:31.041034+0000","last_unstale":"2024-09-16T07:33:33.358359+0000","last_undegraded":"2024-09-16T07:33:33.358359+0000","last_fullsized":"2024-09-16T07:33:33.358359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:25.618706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024155+0000","last_change":"2024-09-16T07:33:32.042155+0000","last_active":"2024-09-16T07:33:33.024155+0000","last_peered":"2024-09-16T07:33:33.024155+0000","last_clean":"2024-09-16T07:33:33.024155+0000","last_became_active":"2024-09-16T07:33:32.041955+0000","last_became_peered":"2024-09-16T07:33:32.041955+0000","last_unstale":"2024-09-16T07:33:33.024155+0000","last_undegraded":"2024-09-16T07:33:33.024155+0000","last_fullsized":"2024-09-16T07:33:33.024155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:45.546271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025124+0000","last_change":"2024-09-16T07:33:31.036453+0000","last_active":"2024-09-16T07:33:33.025124+0000","last_peered":"2024-09-16T07:33:33.025124+0000","last_clean":"2024-09-16T07:33:33.025124+0000","last_became_active":"2024-09-16T07:33:31.034972+0000","last_became_peered":"2024-09-16T07:33:31.034972+0000","last_unstale":"2024-09-16T07:33:33.025124+0000","last_undegraded":"2024-09-16T07:33:33.025124+0000","last_fullsized":"2024-09-16T07:33:33.025124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:49.504210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.031515+0000","last_change":"2024-09-16T07:33:31.031515+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.031515+0000","last_undegraded":"2024-09-16T07:33:31.031515+0000","last_fullsized":"2024-09-16T07:33:31.031515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024241+0000","last_change":"2024-09-16T07:33:31.042034+0000","last_active":"2024-09-16T07:33:33.024241+0000","last_peered":"2024-09-16T07:33:33.024241+0000","last_clean":"2024-09-16T07:33:33.024241+0000","last_became_active":"2024-09-16T07:33:31.041059+0000","last_became_peered":"2024-09-16T07:33:31.041059+0000","last_unstale":"2024-09-16T07:33:33.024241+0000","last_undegraded":"2024-09-16T07:33:33.024241+0000","last_fullsized":"2024-09-16T07:33:33.024241+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:59.775582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357991+0000","last_change":"2024-09-16T07:33:32.046065+0000","last_active":"2024-09-16T07:33:33.357991+0000","last_peered":"2024-09-16T07:33:33.357991+0000","last_clean":"2024-09-16T07:33:33.357991+0000","last_became_active":"2024-09-16T07:33:32.045750+0000","last_became_peered":"2024-09-16T07:33:32.045750+0000","last_unstale":"2024-09-16T07:33:33.357991+0000","last_undegraded":"2024-09-16T07:33:33.357991+0000","last_fullsized":"2024-09-16T07:33:33.357991+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:26.414491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357741+0000","last_change":"2024-09-16T07:33:31.031995+0000","last_active":"2024-09-16T07:33:33.357741+0000","last_peered":"2024-09-16T07:33:33.357741+0000","last_clean":"2024-09-16T07:33:33.357741+0000","last_became_active":"2024-09-16T07:33:31.031131+0000","last_became_peered":"2024-09-16T07:33:31.031131+0000","last_unstale":"2024-09-16T07:33:33.357741+0000","last_undegraded":"2024-09-16T07:33:33.357741+0000","last_fullsized":"2024-09-16T07:33:33.357741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:28.800839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027506+0000","last_change":"2024-09-16T07:33:31.027506+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027506+0000","last_undegraded":"2024-09-16T07:33:31.027506+0000","last_fullsized":"2024-09-16T07:33:31.027506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023950+0000","last_change":"2024-09-16T07:33:31.030808+0000","last_active":"2024-09-16T07:33:33.023950+0000","last_peered":"2024-09-16T07:33:33.023950+0000","last_clean":"2024-09-16T07:33:33.023950+0000","last_became_active":"2024-09-16T07:33:31.028772+0000","last_became_peered":"2024-09-16T07:33:31.028772+0000","last_unstale":"2024-09-16T07:33:33.023950+0000","last_undegraded":"2024-09-16T07:33:33.023950+0000","last_fullsized":"2024-09-16T07:33:33.023950+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:53.744425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358706+0000","last_change":"2024-09-16T07:33:32.045394+0000","last_active":"2024-09-16T07:33:33.358706+0000","last_peered":"2024-09-16T07:33:33.358706+0000","last_clean":"2024-09-16T07:33:33.358706+0000","last_became_active":"2024-09-16T07:33:32.045106+0000","last_became_peered":"2024-09-16T07:33:32.045106+0000","last_unstale":"2024-09-16T07:33:33.358706+0000","last_undegraded":"2024-09-16T07:33:33.358706+0000","last_fullsized":"2024-09-16T07:33:33.358706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.537036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358438+0000","last_change":"2024-09-16T07:33:31.041342+0000","last_active":"2024-09-16T07:33:33.358438+0000","last_peered":"2024-09-16T07:33:33.358438+0000","last_clean":"2024-09-16T07:33:33.358438+0000","last_became_active":"2024-09-16T07:33:31.041056+0000","last_became_peered":"2024-09-16T07:33:31.041056+0000","last_unstale":"2024-09-16T07:33:33.358438+0000","last_undegraded":"2024-09-16T07:33:33.358438+0000","last_fullsized":"2024-09-16T07:33:33.358438+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:39.097159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358460+0000","last_change":"2024-09-16T07:33:32.044491+0000","last_active":"2024-09-16T07:33:33.358460+0000","last_peered":"2024-09-16T07:33:33.358460+0000","last_clean":"2024-09-16T07:33:33.358460+0000","last_became_active":"2024-09-16T07:33:32.044228+0000","last_became_peered":"2024-09-16T07:33:32.044228+0000","last_unstale":"2024-09-16T07:33:33.358460+0000","last_undegraded":"2024-09-16T07:33:33.358460+0000","last_fullsized":"2024-09-16T07:33:33.358460+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:14.023175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357548+0000","last_change":"2024-09-16T07:33:31.034256+0000","last_active":"2024-09-16T07:33:33.357548+0000","last_peered":"2024-09-16T07:33:33.357548+0000","last_clean":"2024-09-16T07:33:33.357548+0000","last_became_active":"2024-09-16T07:33:31.033825+0000","last_became_peered":"2024-09-16T07:33:31.033825+0000","last_unstale":"2024-09-16T07:33:33.357548+0000","last_undegraded":"2024-09-16T07:33:33.357548+0000","last_fullsized":"2024-09-16T07:33:33.357548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:44.857125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358285+0000","last_change":"2024-09-16T07:33:32.041248+0000","last_active":"2024-09-16T07:33:33.358285+0000","last_peered":"2024-09-16T07:33:33.358285+0000","last_clean":"2024-09-16T07:33:33.358285+0000","last_became_active":"2024-09-16T07:33:32.040998+0000","last_became_peered":"2024-09-16T07:33:32.040998+0000","last_unstale":"2024-09-16T07:33:33.358285+0000","last_undegraded":"2024-09-16T07:33:33.358285+0000","last_fullsized":"2024-09-16T07:33:33.358285+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:06.696198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025145+0000","last_change":"2024-09-16T07:33:31.035273+0000","last_active":"2024-09-16T07:33:33.025145+0000","last_peered":"2024-09-16T07:33:33.025145+0000","last_clean":"2024-09-16T07:33:33.025145+0000","last_became_active":"2024-09-16T07:33:31.034957+0000","last_became_peered":"2024-09-16T07:33:31.034957+0000","last_unstale":"2024-09-16T07:33:33.025145+0000","last_undegraded":"2024-09-16T07:33:33.025145+0000","last_fullsized":"2024-09-16T07:33:33.025145+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:17.685015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025196+0000","last_change":"2024-09-16T07:33:32.033426+0000","last_active":"2024-09-16T07:33:33.025196+0000","last_peered":"2024-09-16T07:33:33.025196+0000","last_clean":"2024-09-16T07:33:33.025196+0000","last_became_active":"2024-09-16T07:33:32.032989+0000","last_became_peered":"2024-09-16T07:33:32.032989+0000","last_unstale":"2024-09-16T07:33:33.025196+0000","last_undegraded":"2024-09-16T07:33:33.025196+0000","last_fullsized":"2024-09-16T07:33:33.025196+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:06.277092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023949+0000","last_change":"2024-09-16T07:33:31.038931+0000","last_active":"2024-09-16T07:33:33.023949+0000","last_peered":"2024-09-16T07:33:33.023949+0000","last_clean":"2024-09-16T07:33:33.023949+0000","last_became_active":"2024-09-16T07:33:31.038639+0000","last_became_peered":"2024-09-16T07:33:31.038639+0000","last_unstale":"2024-09-16T07:33:33.023949+0000","last_undegraded":"2024-09-16T07:33:33.023949+0000","last_fullsized":"2024-09-16T07:33:33.023949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:37.191040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025027+0000","last_change":"2024-09-16T07:33:32.050892+0000","last_active":"2024-09-16T07:33:33.025027+0000","last_peered":"2024-09-16T07:33:33.025027+0000","last_clean":"2024-09-16T07:33:33.025027+0000","last_became_active":"2024-09-16T07:33:32.050449+0000","last_became_peered":"2024-09-16T07:33:32.050449+0000","last_unstale":"2024-09-16T07:33:33.025027+0000","last_undegraded":"2024-09-16T07:33:33.025027+0000","last_fullsized":"2024-09-16T07:33:33.025027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:53.956082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358694+0000","last_change":"2024-09-16T07:33:31.035777+0000","last_active":"2024-09-16T07:33:33.358694+0000","last_peered":"2024-09-16T07:33:33.358694+0000","last_clean":"2024-09-16T07:33:33.358694+0000","last_became_active":"2024-09-16T07:33:31.034448+0000","last_became_peered":"2024-09-16T07:33:31.034448+0000","last_unstale":"2024-09-16T07:33:33.358694+0000","last_undegraded":"2024-09-16T07:33:33.358694+0000","last_fullsized":"2024-09-16T07:33:33.358694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:45.713854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025244+0000","last_change":"2024-09-16T07:33:32.052617+0000","last_active":"2024-09-16T07:33:33.025244+0000","last_peered":"2024-09-16T07:33:33.025244+0000","last_clean":"2024-09-16T07:33:33.025244+0000","last_became_active":"2024-09-16T07:33:32.052089+0000","last_became_peered":"2024-09-16T07:33:32.052089+0000","last_unstale":"2024-09-16T07:33:33.025244+0000","last_undegraded":"2024-09-16T07:33:33.025244+0000","last_fullsized":"2024-09-16T07:33:33.025244+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:49.702884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021703+0000","last_change":"2024-09-16T07:33:31.036417+0000","last_active":"2024-09-16T07:33:33.021703+0000","last_peered":"2024-09-16T07:33:33.021703+0000","last_clean":"2024-09-16T07:33:33.021703+0000","last_became_active":"2024-09-16T07:33:31.036118+0000","last_became_peered":"2024-09-16T07:33:31.036118+0000","last_unstale":"2024-09-16T07:33:33.021703+0000","last_undegraded":"2024-09-16T07:33:33.021703+0000","last_fullsized":"2024-09-16T07:33:33.021703+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:50.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358156+0000","last_change":"2024-09-16T07:33:32.039339+0000","last_active":"2024-09-16T07:33:33.358156+0000","last_peered":"2024-09-16T07:33:33.358156+0000","last_clean":"2024-09-16T07:33:33.358156+0000","last_became_active":"2024-09-16T07:33:32.037162+0000","last_became_peered":"2024-09-16T07:33:32.037162+0000","last_unstale":"2024-09-16T07:33:33.358156+0000","last_undegraded":"2024-09-16T07:33:33.358156+0000","last_fullsized":"2024-09-16T07:33:33.358156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:42.850047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045296+0000","last_change":"2024-09-16T07:33:31.042541+0000","last_active":"2024-09-16T07:33:31.045296+0000","last_peered":"2024-09-16T07:33:31.045296+0000","last_clean":"2024-09-16T07:33:31.045296+0000","last_became_active":"2024-09-16T07:33:31.042235+0000","last_became_peered":"2024-09-16T07:33:31.042235+0000","last_unstale":"2024-09-16T07:33:31.045296+0000","last_undegraded":"2024-09-16T07:33:31.045296+0000","last_fullsized":"2024-09-16T07:33:31.045296+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:20.784004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025623+0000","last_change":"2024-09-16T07:33:32.045670+0000","last_active":"2024-09-16T07:33:33.025623+0000","last_peered":"2024-09-16T07:33:33.025623+0000","last_clean":"2024-09-16T07:33:33.025623+0000","last_became_active":"2024-09-16T07:33:32.045383+0000","last_became_peered":"2024-09-16T07:33:32.045383+0000","last_unstale":"2024-09-16T07:33:33.025623+0000","last_undegraded":"2024-09-16T07:33:33.025623+0000","last_fullsized":"2024-09-16T07:33:33.025623+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:56.570489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358500+0000","last_change":"2024-09-16T07:33:31.030651+0000","last_active":"2024-09-16T07:33:33.358500+0000","last_peered":"2024-09-16T07:33:33.358500+0000","last_clean":"2024-09-16T07:33:33.358500+0000","last_became_active":"2024-09-16T07:33:31.030312+0000","last_became_peered":"2024-09-16T07:33:31.030312+0000","last_unstale":"2024-09-16T07:33:33.358500+0000","last_undegraded":"2024-09-16T07:33:33.358500+0000","last_fullsized":"2024-09-16T07:33:33.358500+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:57.124363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025464+0000","last_change":"2024-09-16T07:33:32.045722+0000","last_active":"2024-09-16T07:33:33.025464+0000","last_peered":"2024-09-16T07:33:33.025464+0000","last_clean":"2024-09-16T07:33:33.025464+0000","last_became_active":"2024-09-16T07:33:32.045414+0000","last_became_peered":"2024-09-16T07:33:32.045414+0000","last_unstale":"2024-09-16T07:33:33.025464+0000","last_undegraded":"2024-09-16T07:33:33.025464+0000","last_fullsized":"2024-09-16T07:33:33.025464+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:57.552859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025686+0000","last_change":"2024-09-16T07:33:31.040312+0000","last_active":"2024-09-16T07:33:33.025686+0000","last_peered":"2024-09-16T07:33:33.025686+0000","last_clean":"2024-09-16T07:33:33.025686+0000","last_became_active":"2024-09-16T07:33:31.040009+0000","last_became_peered":"2024-09-16T07:33:31.040009+0000","last_unstale":"2024-09-16T07:33:33.025686+0000","last_undegraded":"2024-09-16T07:33:33.025686+0000","last_fullsized":"2024-09-16T07:33:33.025686+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:56.393574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.032016+0000","last_change":"2024-09-16T07:33:31.032016+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.032016+0000","last_undegraded":"2024-09-16T07:33:31.032016+0000","last_fullsized":"2024-09-16T07:33:31.032016+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025287+0000","last_change":"2024-09-16T07:33:31.033696+0000","last_active":"2024-09-16T07:33:33.025287+0000","last_peered":"2024-09-16T07:33:33.025287+0000","last_clean":"2024-09-16T07:33:33.025287+0000","last_became_active":"2024-09-16T07:33:31.033430+0000","last_became_peered":"2024-09-16T07:33:31.033430+0000","last_unstale":"2024-09-16T07:33:33.025287+0000","last_undegraded":"2024-09-16T07:33:33.025287+0000","last_fullsized":"2024-09-16T07:33:33.025287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:55.604351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357346+0000","last_change":"2024-09-16T07:33:32.042206+0000","last_active":"2024-09-16T07:33:33.357346+0000","last_peered":"2024-09-16T07:33:33.357346+0000","last_clean":"2024-09-16T07:33:33.357346+0000","last_became_active":"2024-09-16T07:33:32.041947+0000","last_became_peered":"2024-09-16T07:33:32.041947+0000","last_unstale":"2024-09-16T07:33:33.357346+0000","last_undegraded":"2024-09-16T07:33:33.357346+0000","last_fullsized":"2024-09-16T07:33:33.357346+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:37:06.473975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358556+0000","last_change":"2024-09-16T07:33:31.032898+0000","last_active":"2024-09-16T07:33:33.358556+0000","last_peered":"2024-09-16T07:33:33.358556+0000","last_clean":"2024-09-16T07:33:33.358556+0000","last_became_active":"2024-09-16T07:33:31.032344+0000","last_became_peered":"2024-09-16T07:33:31.032344+0000","last_unstale":"2024-09-16T07:33:33.358556+0000","last_undegraded":"2024-09-16T07:33:33.358556+0000","last_fullsized":"2024-09-16T07:33:33.358556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:26.380802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025119+0000","last_change":"2024-09-16T07:33:32.038999+0000","last_active":"2024-09-16T07:33:33.025119+0000","last_peered":"2024-09-16T07:33:33.025119+0000","last_clean":"2024-09-16T07:33:33.025119+0000","last_became_active":"2024-09-16T07:33:32.038709+0000","last_became_peered":"2024-09-16T07:33:32.038709+0000","last_unstale":"2024-09-16T07:33:33.025119+0000","last_undegraded":"2024-09-16T07:33:33.025119+0000","last_fullsized":"2024-09-16T07:33:33.025119+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:02.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045928+0000","last_change":"2024-09-16T07:33:31.044406+0000","last_active":"2024-09-16T07:33:31.045928+0000","last_peered":"2024-09-16T07:33:31.045928+0000","last_clean":"2024-09-16T07:33:31.045928+0000","last_became_active":"2024-09-16T07:33:31.044017+0000","last_became_peered":"2024-09-16T07:33:31.044017+0000","last_unstale":"2024-09-16T07:33:31.045928+0000","last_undegraded":"2024-09-16T07:33:31.045928+0000","last_fullsized":"2024-09-16T07:33:31.045928+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:37.852077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357714+0000","last_change":"2024-09-16T07:33:32.046842+0000","last_active":"2024-09-16T07:33:33.357714+0000","last_peered":"2024-09-16T07:33:33.357714+0000","last_clean":"2024-09-16T07:33:33.357714+0000","last_became_active":"2024-09-16T07:33:32.046569+0000","last_became_peered":"2024-09-16T07:33:32.046569+0000","last_unstale":"2024-09-16T07:33:33.357714+0000","last_undegraded":"2024-09-16T07:33:33.357714+0000","last_fullsized":"2024-09-16T07:33:33.357714+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:01.321702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358757+0000","last_change":"2024-09-16T07:33:31.037028+0000","last_active":"2024-09-16T07:33:33.358757+0000","last_peered":"2024-09-16T07:33:33.358757+0000","last_clean":"2024-09-16T07:33:33.358757+0000","last_became_active":"2024-09-16T07:33:31.036714+0000","last_became_peered":"2024-09-16T07:33:31.036714+0000","last_unstale":"2024-09-16T07:33:33.358757+0000","last_undegraded":"2024-09-16T07:33:33.358757+0000","last_fullsized":"2024-09-16T07:33:33.358757+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:01.872929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024139+0000","last_change":"2024-09-16T07:33:32.036865+0000","last_active":"2024-09-16T07:33:33.024139+0000","last_peered":"2024-09-16T07:33:33.024139+0000","last_clean":"2024-09-16T07:33:33.024139+0000","last_became_active":"2024-09-16T07:33:32.036613+0000","last_became_peered":"2024-09-16T07:33:32.036613+0000","last_unstale":"2024-09-16T07:33:33.024139+0000","last_undegraded":"2024-09-16T07:33:33.024139+0000","last_fullsized":"2024-09-16T07:33:33.024139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:29:18.832918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.040193+0000","last_change":"2024-09-16T07:33:31.044776+0000","last_active":"2024-09-16T07:33:32.040193+0000","last_peered":"2024-09-16T07:33:32.040193+0000","last_clean":"2024-09-16T07:33:32.040193+0000","last_became_active":"2024-09-16T07:33:31.044453+0000","last_became_peered":"2024-09-16T07:33:31.044453+0000","last_unstale":"2024-09-16T07:33:32.040193+0000","last_undegraded":"2024-09-16T07:33:32.040193+0000","last_fullsized":"2024-09-16T07:33:32.040193+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:06.219751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358395+0000","last_change":"2024-09-16T07:33:32.043818+0000","last_active":"2024-09-16T07:33:33.358395+0000","last_peered":"2024-09-16T07:33:33.358395+0000","last_clean":"2024-09-16T07:33:33.358395+0000","last_became_active":"2024-09-16T07:33:32.043573+0000","last_became_peered":"2024-09-16T07:33:32.043573+0000","last_unstale":"2024-09-16T07:33:33.358395+0000","last_undegraded":"2024-09-16T07:33:33.358395+0000","last_fullsized":"2024-09-16T07:33:33.358395+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:10.405842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045274+0000","last_change":"2024-09-16T07:33:31.042397+0000","last_active":"2024-09-16T07:33:31.045274+0000","last_peered":"2024-09-16T07:33:31.045274+0000","last_clean":"2024-09-16T07:33:31.045274+0000","last_became_active":"2024-09-16T07:33:31.041205+0000","last_became_peered":"2024-09-16T07:33:31.041205+0000","last_unstale":"2024-09-16T07:33:31.045274+0000","last_undegraded":"2024-09-16T07:33:31.045274+0000","last_fullsized":"2024-09-16T07:33:31.045274+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:13.919862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.030279+0000","last_change":"2024-09-16T07:33:31.030279+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.030279+0000","last_undegraded":"2024-09-16T07:33:31.030279+0000","last_fullsized":"2024-09-16T07:33:31.030279+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358516+0000","last_change":"2024-09-16T07:33:31.041709+0000","last_active":"2024-09-16T07:33:33.358516+0000","last_peered":"2024-09-16T07:33:33.358516+0000","last_clean":"2024-09-16T07:33:33.358516+0000","last_became_active":"2024-09-16T07:33:31.040991+0000","last_became_peered":"2024-09-16T07:33:31.040991+0000","last_unstale":"2024-09-16T07:33:33.358516+0000","last_undegraded":"2024-09-16T07:33:33.358516+0000","last_fullsized":"2024-09-16T07:33:33.358516+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:32.562633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":265,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357091+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:33:33.357091+0000","last_peered":"2024-09-16T07:33:33.357091+0000","last_clean":"2024-09-16T07:33:33.357091+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:33:33.357091+0000","last_undegraded":"2024-09-16T07:33:33.357091+0000","last_fullsized":"2024-09-16T07:33:33.357091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122632,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249208,"statfs":{"total":96636764160,"available":96511188992,"internally_reserved":0,"allocated":1298432,"data_stored":560047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122612,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249228,"statfs":{"total":96636764160,"available":96511209472,"internally_reserved":0,"allocated":1277952,"data_stored":553595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119056,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252784,"statfs":{"total":96636764160,"available":96514850816,"internally_reserved":0,"allocated":1888256,"data_stored":1145617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100400,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271440,"statfs":{"total":96636764160,"available":96533954560,"internally_reserved":0,"allocated":1273856,"data_stored":553525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100412,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271428,"statfs":{"total":96636764160,"available":96533942272,"internally_reserved":0,"allocated":1277952,"data_stored":554107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":101513674},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148984,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222856,"statfs":{"total":96636764160,"available":96484204544,"internally_reserved":0,"allocated":1273856,"data_stored":553525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509575,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127380,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244460,"statfs":{"total":96636764160,"available":96506327040,"internally_reserved":0,"allocated":1900544,"data_stored":1151889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542277,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122584,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249256,"statfs":{"total":96636764160,"available":96511238144,"internally_reserved":0,"allocated":1249280,"data_stored":540419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3d is not active+clean 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.031577+0000', 'last_change': '2024-09-16T07:33:31.031577+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.031577+0000', 'last_undegraded': '2024-09-16T07:33:31.031577+0000', 'last_fullsized': '2024-09-16T07:33:31.031577+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:PG 15.38 is not active+clean 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.025834+0000', 'last_change': '2024-09-16T07:33:31.025834+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.025834+0000', 'last_undegraded': '2024-09-16T07:33:31.025834+0000', 'last_fullsized': '2024-09-16T07:33:31.025834+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2b is not active+clean 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.032081+0000', 'last_change': '2024-09-16T07:33:31.032081+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.032081+0000', 'last_undegraded': '2024-09-16T07:33:31.032081+0000', 'last_fullsized': '2024-09-16T07:33:31.032081+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2a is not active+clean 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.027983+0000', 'last_change': '2024-09-16T07:33:31.027983+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.027983+0000', 'last_undegraded': '2024-09-16T07:33:31.027983+0000', 'last_fullsized': '2024-09-16T07:33:31.027983+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:PG 15.21 is not active+clean 2024-09-16T07:33:37.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.027968+0000', 'last_change': '2024-09-16T07:33:31.027968+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.027968+0000', 'last_undegraded': '2024-09-16T07:33:31.027968+0000', 'last_fullsized': '2024-09-16T07:33:31.027968+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.071 INFO:tasks.ceph.ceph_manager.ceph:PG 15.13 is not active+clean 2024-09-16T07:33:37.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.031515+0000', 'last_change': '2024-09-16T07:33:31.031515+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.031515+0000', 'last_undegraded': '2024-09-16T07:33:31.031515+0000', 'last_fullsized': '2024-09-16T07:33:31.031515+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.071 INFO:tasks.ceph.ceph_manager.ceph:PG 15.11 is not active+clean 2024-09-16T07:33:37.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.027506+0000', 'last_change': '2024-09-16T07:33:31.027506+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.027506+0000', 'last_undegraded': '2024-09-16T07:33:31.027506+0000', 'last_fullsized': '2024-09-16T07:33:31.027506+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.071 INFO:tasks.ceph.ceph_manager.ceph:PG 15.4 is not active+clean 2024-09-16T07:33:37.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.032016+0000', 'last_change': '2024-09-16T07:33:31.032016+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.032016+0000', 'last_undegraded': '2024-09-16T07:33:31.032016+0000', 'last_fullsized': '2024-09-16T07:33:31.032016+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.071 INFO:tasks.ceph.ceph_manager.ceph:PG 15.e is not active+clean 2024-09-16T07:33:37.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:33:31.030279+0000', 'last_change': '2024-09-16T07:33:31.030279+0000', 'last_active': '2024-09-16T07:33:31.008881+0000', 'last_peered': '2024-09-16T07:33:31.008881+0000', 'last_clean': '2024-09-16T07:33:31.008881+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:33:31.030279+0000', 'last_undegraded': '2024-09-16T07:33:31.030279+0000', 'last_fullsized': '2024-09-16T07:33:31.030279+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'last_clean_scrub_stamp': '2024-09-16T07:33:31.008881+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T07:33:37.072 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T07:33:37.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.263+0000 7f5109000640 1 -- 172.21.15.71:0/2344110086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f510406e9b0 msgr2=0x7f51040cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.263+0000 7f5109000640 1 --2- 172.21.15.71:0/2344110086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f510406e9b0 0x7f51040cc660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f510406fc20 tx=0x7f50f002a5d0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.263+0000 7f5109000640 1 -- 172.21.15.71:0/2344110086 shutdown_connections 2024-09-16T07:33:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.263+0000 7f5109000640 1 --2- 172.21.15.71:0/2344110086 >> v2:172.21.15.177:3300/0 conn(0x7f51040ccba0 0x7f5104073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.263+0000 7f5109000640 1 --2- 172.21.15.71:0/2344110086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f510406e9b0 0x7f51040cc660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.263+0000 7f5109000640 1 -- 172.21.15.71:0/2344110086 >> 172.21.15.71:0/2344110086 conn(0x7f51040b18e0 msgr2=0x7f51040b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.263+0000 7f5109000640 1 -- 172.21.15.71:0/2344110086 shutdown_connections 2024-09-16T07:33:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.263+0000 7f5109000640 1 -- 172.21.15.71:0/2344110086 wait complete. 2024-09-16T07:33:37.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.264+0000 7f5109000640 1 Processor -- start 2024-09-16T07:33:37.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.265+0000 7f5109000640 1 -- start start 2024-09-16T07:33:37.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5109000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f510406e9b0 0x7f5104155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:37.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5109000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51040ccba0 0x7f51040b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5109000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51041563a0 0x7f51041f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5109000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5104073bd0 con 0x7f510406e9b0 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5109000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5104073a50 con 0x7f51041563a0 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5109000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5104073d50 con 0x7f51040ccba0 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f510406e9b0 0x7f5104155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f510406e9b0 0x7f5104155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42286/0 (socket says 172.21.15.71:42286) 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103400640 1 -- 172.21.15.71:0/2703735853 learned_addr learned my addr 172.21.15.71:0/2703735853 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103e00640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51041563a0 0x7f51041f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5102a00640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51040ccba0 0x7f51040b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103400640 1 -- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51040ccba0 msgr2=0x7f51040b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103400640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51040ccba0 0x7f51040b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103400640 1 -- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51041563a0 msgr2=0x7f51041f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103400640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51041563a0 0x7f51041f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5103400640 1 -- 172.21.15.71:0/2703735853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5104155a60 con 0x7f510406e9b0 2024-09-16T07:33:37.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.266+0000 7f5102a00640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51040ccba0 0x7f51040b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:37.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.267+0000 7f5103400640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f510406e9b0 0x7f5104155520 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f50f400c730 tx=0x7f50f400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:37.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.267+0000 7f50fb400640 1 -- 172.21.15.71:0/2703735853 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4003b70 con 0x7f510406e9b0 2024-09-16T07:33:37.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.267+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51041f5380 con 0x7f510406e9b0 2024-09-16T07:33:37.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.267+0000 7f50fb400640 1 -- 172.21.15.71:0/2703735853 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50f4003d30 con 0x7f510406e9b0 2024-09-16T07:33:37.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.267+0000 7f50fb400640 1 -- 172.21.15.71:0/2703735853 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4005330 con 0x7f510406e9b0 2024-09-16T07:33:37.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.267+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51040c9100 con 0x7f510406e9b0 2024-09-16T07:33:37.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.268+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51041f58a0 con 0x7f510406e9b0 2024-09-16T07:33:37.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.269+0000 7f50fb400640 1 -- 172.21.15.71:0/2703735853 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f50f40054d0 con 0x7f510406e9b0 2024-09-16T07:33:37.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.270+0000 7f50fb400640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50e4079e40 0x7f50e407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:37.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.270+0000 7f50fb400640 1 -- 172.21.15.71:0/2703735853 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f50f4099800 con 0x7f510406e9b0 2024-09-16T07:33:37.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.271+0000 7f5102a00640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50e4079e40 0x7f50e407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:37.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.271+0000 7f5102a00640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50e4079e40 0x7f50e407c300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f50f0033000 tx=0x7f50f0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.274+0000 7f50fb400640 1 -- 172.21.15.71:0/2703735853 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51041f58a0 con 0x7f510406e9b0 2024-09-16T07:33:37.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.527+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5104074270 con 0x7f510406e9b0 2024-09-16T07:33:37.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.528+0000 7f50fb400640 1 -- 172.21.15.71:0/2703735853 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1143 (secure 0 0 0) 0x7f50f4063430 con 0x7f510406e9b0 2024-09-16T07:33:37.528 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:37.528 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":674,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":105,"num_osds":8,"num_up_osds":8,"osd_up_since":1726471357,"num_in_osds":8,"osd_in_since":1726471347,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":88},{"state_name":"creating+peering","count":9}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592778,"bytes_used":987197440,"bytes_avail":772106915840,"bytes_total":773094113280,"inactive_pgs_ratio":0.092783503234386444,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":66,"btime":"2024-09-16T07:33:35:043973+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7863}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-09-16T07:32:18.525085+0000","services":{}},"progress_events":{}} 2024-09-16T07:33:37.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50e4079e40 msgr2=0x7f50e407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50e4079e40 0x7f50e407c300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f50f0033000 tx=0x7f50f0004730 comp rx=0 tx=0).stop 2024-09-16T07:33:37.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f510406e9b0 msgr2=0x7f5104155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f510406e9b0 0x7f5104155520 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f50f400c730 tx=0x7f50f400cb30 comp rx=0 tx=0).stop 2024-09-16T07:33:37.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 shutdown_connections 2024-09-16T07:33:37.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f50e4079e40 0x7f50e407c300 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51041563a0 0x7f51041f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51040ccba0 0x7f51040b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 --2- 172.21.15.71:0/2703735853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f510406e9b0 0x7f5104155520 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.530+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 >> 172.21.15.71:0/2703735853 conn(0x7f51040b18e0 msgr2=0x7f510415cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:37.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.531+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 shutdown_connections 2024-09-16T07:33:37.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.531+0000 7f5109000640 1 -- 172.21.15.71:0/2703735853 wait complete. 2024-09-16T07:33:37.546 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T07:33:37.546 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:33:37.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.702+0000 7f1ba4a00640 1 -- 172.21.15.71:0/1490309919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00d1620 msgr2=0x7f1ba015cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.702+0000 7f1ba4a00640 1 --2- 172.21.15.71:0/1490309919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00d1620 0x7f1ba015cb40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005fd0 tx=0x7f1b8c02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.703+0000 7f1ba4a00640 1 -- 172.21.15.71:0/1490309919 shutdown_connections 2024-09-16T07:33:37.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.703+0000 7f1ba4a00640 1 --2- 172.21.15.71:0/1490309919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00d1620 0x7f1ba015cb40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.703+0000 7f1ba4a00640 1 -- 172.21.15.71:0/1490309919 >> 172.21.15.71:0/1490309919 conn(0x7f1ba00d0b30 msgr2=0x7f1ba00d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:37.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.703+0000 7f1ba4a00640 1 -- 172.21.15.71:0/1490309919 shutdown_connections 2024-09-16T07:33:37.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.703+0000 7f1ba4a00640 1 -- 172.21.15.71:0/1490309919 wait complete. 2024-09-16T07:33:37.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.704+0000 7f1ba4a00640 1 Processor -- start 2024-09-16T07:33:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.704+0000 7f1ba4a00640 1 -- start start 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.705+0000 7f1ba4a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba0073ee0 0x7f1ba00cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.705+0000 7f1ba4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00cc8d0 0x7f1ba01f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.705+0000 7f1ba4a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba00d1620 0x7f1ba01e6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.705+0000 7f1ba4a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba006ffd0 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.705+0000 7f1ba4a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba006fe50 con 0x7f1ba00d1620 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.705+0000 7f1ba4a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba0152ac0 con 0x7f1ba0073ee0 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.705+0000 7f1b9de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00cc8d0 0x7f1ba01f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00cc8d0 0x7f1ba01f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42300/0 (socket says 172.21.15.71:42300) 2024-09-16T07:33:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9de00640 1 -- 172.21.15.71:0/3974314908 learned_addr learned my addr 172.21.15.71:0/3974314908 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9e800640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba0073ee0 0x7f1ba00cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9f200640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba00d1620 0x7f1ba01e6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9de00640 1 -- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba0073ee0 msgr2=0x7f1ba00cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9de00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba0073ee0 0x7f1ba00cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9de00640 1 -- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba00d1620 msgr2=0x7f1ba01e6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9de00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba00d1620 0x7f1ba01e6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9de00640 1 -- 172.21.15.71:0/3974314908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba01f8970 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b9de00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00cc8d0 0x7f1ba01f8000 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c034040 tx=0x7f1b8c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.706+0000 7f1b96a00640 1 -- 172.21.15.71:0/3974314908 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c03e030 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.707+0000 7f1b96a00640 1 -- 172.21.15.71:0/3974314908 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b8c003ca0 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.707+0000 7f1b96a00640 1 -- 172.21.15.71:0/3974314908 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c035040 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.707+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba01f8c50 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.707+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba00ceb80 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.708+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba01f8e90 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.709+0000 7f1b96a00640 1 -- 172.21.15.71:0/3974314908 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f1b8c041030 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.710+0000 7f1b96a00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1b80079e40 0x7f1b8007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.710+0000 7f1b96a00640 1 -- 172.21.15.71:0/3974314908 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f1b8c0b9fd0 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.710+0000 7f1b9e800640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1b80079e40 0x7f1b8007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:37.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.711+0000 7f1b9e800640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1b80079e40 0x7f1b8007c300 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1b90001680 tx=0x7f1b90009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.714+0000 7f1b96a00640 1 -- 172.21.15.71:0/3974314908 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1ba01f8e90 con 0x7f1ba00cc8d0 2024-09-16T07:33:37.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.896+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ba015cb40 con 0x7f1b80079e40 2024-09-16T07:33:37.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.901+0000 7f1b96a00640 1 -- 172.21.15.71:0/3974314908 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251762 (secure 0 0 0) 0x7f1ba015cb40 con 0x7f1b80079e40 2024-09-16T07:33:37.904 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:37.905 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.904+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1b80079e40 msgr2=0x7f1b8007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.904+0000 7f1ba4a00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1b80079e40 0x7f1b8007c300 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1b90001680 tx=0x7f1b90009040 comp rx=0 tx=0).stop 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00cc8d0 msgr2=0x7f1ba01f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00cc8d0 0x7f1ba01f8000 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c034040 tx=0x7f1b8c0038a0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 shutdown_connections 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1b80079e40 0x7f1b8007c300 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba00d1620 0x7f1ba01e6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba00cc8d0 0x7f1ba01f8000 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 --2- 172.21.15.71:0/3974314908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba0073ee0 0x7f1ba00cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 >> 172.21.15.71:0/3974314908 conn(0x7f1ba00d0b30 msgr2=0x7f1ba00cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 shutdown_connections 2024-09-16T07:33:37.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:37.905+0000 7f1ba4a00640 1 -- 172.21.15.71:0/3974314908 wait complete. 2024-09-16T07:33:37.920 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-16T07:33:36.581316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":964060,"kb_used_data":11172,"kb_used_omap":12,"kb_used_meta":952691,"kb_avail":754010660,"statfs":{"total":773094113280,"available":772106915840,"internally_reserved":0,"allocated":11440128,"data_stored":5612724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":975556144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001607"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025017+0000","last_change":"2024-09-16T07:33:32.045068+0000","last_active":"2024-09-16T07:33:33.025017+0000","last_peered":"2024-09-16T07:33:33.025017+0000","last_clean":"2024-09-16T07:33:33.025017+0000","last_became_active":"2024-09-16T07:33:32.044772+0000","last_became_peered":"2024-09-16T07:33:32.044772+0000","last_unstale":"2024-09-16T07:33:33.025017+0000","last_undegraded":"2024-09-16T07:33:33.025017+0000","last_fullsized":"2024-09-16T07:33:33.025017+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024784+0000","last_change":"2024-09-16T07:33:32.049940+0000","last_active":"2024-09-16T07:33:33.024784+0000","last_peered":"2024-09-16T07:33:33.024784+0000","last_clean":"2024-09-16T07:33:33.024784+0000","last_became_active":"2024-09-16T07:33:32.049646+0000","last_became_peered":"2024-09-16T07:33:32.049646+0000","last_unstale":"2024-09-16T07:33:33.024784+0000","last_undegraded":"2024-09-16T07:33:33.024784+0000","last_fullsized":"2024-09-16T07:33:33.024784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.031577+0000","last_change":"2024-09-16T07:33:31.031577+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.031577+0000","last_undegraded":"2024-09-16T07:33:31.031577+0000","last_fullsized":"2024-09-16T07:33:31.031577+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024064+0000","last_change":"2024-09-16T07:33:32.043883+0000","last_active":"2024-09-16T07:33:33.024064+0000","last_peered":"2024-09-16T07:33:33.024064+0000","last_clean":"2024-09-16T07:33:33.024064+0000","last_became_active":"2024-09-16T07:33:32.043622+0000","last_became_peered":"2024-09-16T07:33:32.043622+0000","last_unstale":"2024-09-16T07:33:33.024064+0000","last_undegraded":"2024-09-16T07:33:33.024064+0000","last_fullsized":"2024-09-16T07:33:33.024064+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:01:49.570122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357403+0000","last_change":"2024-09-16T07:33:32.043899+0000","last_active":"2024-09-16T07:33:33.357403+0000","last_peered":"2024-09-16T07:33:33.357403+0000","last_clean":"2024-09-16T07:33:33.357403+0000","last_became_active":"2024-09-16T07:33:32.043611+0000","last_became_peered":"2024-09-16T07:33:32.043611+0000","last_unstale":"2024-09-16T07:33:33.357403+0000","last_undegraded":"2024-09-16T07:33:33.357403+0000","last_fullsized":"2024-09-16T07:33:33.357403+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:22.213386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357942+0000","last_change":"2024-09-16T07:33:32.054833+0000","last_active":"2024-09-16T07:33:33.357942+0000","last_peered":"2024-09-16T07:33:33.357942+0000","last_clean":"2024-09-16T07:33:33.357942+0000","last_became_active":"2024-09-16T07:33:32.054513+0000","last_became_peered":"2024-09-16T07:33:32.054513+0000","last_unstale":"2024-09-16T07:33:33.357942+0000","last_undegraded":"2024-09-16T07:33:33.357942+0000","last_fullsized":"2024-09-16T07:33:33.357942+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:03.590298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357327+0000","last_change":"2024-09-16T07:33:32.037656+0000","last_active":"2024-09-16T07:33:33.357327+0000","last_peered":"2024-09-16T07:33:33.357327+0000","last_clean":"2024-09-16T07:33:33.357327+0000","last_became_active":"2024-09-16T07:33:32.036949+0000","last_became_peered":"2024-09-16T07:33:32.036949+0000","last_unstale":"2024-09-16T07:33:33.357327+0000","last_undegraded":"2024-09-16T07:33:33.357327+0000","last_fullsized":"2024-09-16T07:33:33.357327+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:18.915383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.025834+0000","last_change":"2024-09-16T07:33:31.025834+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.025834+0000","last_undegraded":"2024-09-16T07:33:31.025834+0000","last_fullsized":"2024-09-16T07:33:31.025834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357756+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.357756+0000","last_peered":"2024-09-16T07:33:33.357756+0000","last_clean":"2024-09-16T07:33:33.357756+0000","last_became_active":"2024-09-16T07:33:32.035287+0000","last_became_peered":"2024-09-16T07:33:32.035287+0000","last_unstale":"2024-09-16T07:33:33.357756+0000","last_undegraded":"2024-09-16T07:33:33.357756+0000","last_fullsized":"2024-09-16T07:33:33.357756+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:52.049650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021843+0000","last_change":"2024-09-16T07:33:32.049217+0000","last_active":"2024-09-16T07:33:33.021843+0000","last_peered":"2024-09-16T07:33:33.021843+0000","last_clean":"2024-09-16T07:33:33.021843+0000","last_became_active":"2024-09-16T07:33:32.048953+0000","last_became_peered":"2024-09-16T07:33:32.048953+0000","last_unstale":"2024-09-16T07:33:33.021843+0000","last_undegraded":"2024-09-16T07:33:33.021843+0000","last_fullsized":"2024-09-16T07:33:33.021843+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357715+0000","last_change":"2024-09-16T07:33:32.054961+0000","last_active":"2024-09-16T07:33:33.357715+0000","last_peered":"2024-09-16T07:33:33.357715+0000","last_clean":"2024-09-16T07:33:33.357715+0000","last_became_active":"2024-09-16T07:33:32.054463+0000","last_became_peered":"2024-09-16T07:33:32.054463+0000","last_unstale":"2024-09-16T07:33:33.357715+0000","last_undegraded":"2024-09-16T07:33:33.357715+0000","last_fullsized":"2024-09-16T07:33:33.357715+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:57.790799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358257+0000","last_change":"2024-09-16T07:33:32.040365+0000","last_active":"2024-09-16T07:33:33.358257+0000","last_peered":"2024-09-16T07:33:33.358257+0000","last_clean":"2024-09-16T07:33:33.358257+0000","last_became_active":"2024-09-16T07:33:32.037953+0000","last_became_peered":"2024-09-16T07:33:32.037953+0000","last_unstale":"2024-09-16T07:33:33.358257+0000","last_undegraded":"2024-09-16T07:33:33.358257+0000","last_fullsized":"2024-09-16T07:33:33.358257+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:32.112813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.059035+0000","last_change":"2024-09-16T07:33:32.056887+0000","last_active":"2024-09-16T07:33:32.059035+0000","last_peered":"2024-09-16T07:33:32.059035+0000","last_clean":"2024-09-16T07:33:32.059035+0000","last_became_active":"2024-09-16T07:33:32.056276+0000","last_became_peered":"2024-09-16T07:33:32.056276+0000","last_unstale":"2024-09-16T07:33:32.059035+0000","last_undegraded":"2024-09-16T07:33:32.059035+0000","last_fullsized":"2024-09-16T07:33:32.059035+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:23.723158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023495+0000","last_change":"2024-09-16T07:33:32.054384+0000","last_active":"2024-09-16T07:33:33.023495+0000","last_peered":"2024-09-16T07:33:33.023495+0000","last_clean":"2024-09-16T07:33:33.023495+0000","last_became_active":"2024-09-16T07:33:32.051888+0000","last_became_peered":"2024-09-16T07:33:32.051888+0000","last_unstale":"2024-09-16T07:33:33.023495+0000","last_undegraded":"2024-09-16T07:33:33.023495+0000","last_fullsized":"2024-09-16T07:33:33.023495+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:19.796812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023506+0000","last_change":"2024-09-16T07:33:32.040802+0000","last_active":"2024-09-16T07:33:33.023506+0000","last_peered":"2024-09-16T07:33:33.023506+0000","last_clean":"2024-09-16T07:33:33.023506+0000","last_became_active":"2024-09-16T07:33:32.040510+0000","last_became_peered":"2024-09-16T07:33:32.040510+0000","last_unstale":"2024-09-16T07:33:33.023506+0000","last_undegraded":"2024-09-16T07:33:33.023506+0000","last_fullsized":"2024-09-16T07:33:33.023506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:53:27.647859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357155+0000","last_change":"2024-09-16T07:33:32.039107+0000","last_active":"2024-09-16T07:33:33.357155+0000","last_peered":"2024-09-16T07:33:33.357155+0000","last_clean":"2024-09-16T07:33:33.357155+0000","last_became_active":"2024-09-16T07:33:32.038828+0000","last_became_peered":"2024-09-16T07:33:32.038828+0000","last_unstale":"2024-09-16T07:33:33.357155+0000","last_undegraded":"2024-09-16T07:33:33.357155+0000","last_fullsized":"2024-09-16T07:33:33.357155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:15.415755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025462+0000","last_change":"2024-09-16T07:33:32.053433+0000","last_active":"2024-09-16T07:33:33.025462+0000","last_peered":"2024-09-16T07:33:33.025462+0000","last_clean":"2024-09-16T07:33:33.025462+0000","last_became_active":"2024-09-16T07:33:32.052843+0000","last_became_peered":"2024-09-16T07:33:32.052843+0000","last_unstale":"2024-09-16T07:33:33.025462+0000","last_undegraded":"2024-09-16T07:33:33.025462+0000","last_fullsized":"2024-09-16T07:33:33.025462+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:13.265002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025375+0000","last_change":"2024-09-16T07:33:32.047389+0000","last_active":"2024-09-16T07:33:33.025375+0000","last_peered":"2024-09-16T07:33:33.025375+0000","last_clean":"2024-09-16T07:33:33.025375+0000","last_became_active":"2024-09-16T07:33:32.047043+0000","last_became_peered":"2024-09-16T07:33:32.047043+0000","last_unstale":"2024-09-16T07:33:33.025375+0000","last_undegraded":"2024-09-16T07:33:33.025375+0000","last_fullsized":"2024-09-16T07:33:33.025375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024761+0000","last_change":"2024-09-16T07:33:32.040875+0000","last_active":"2024-09-16T07:33:33.024761+0000","last_peered":"2024-09-16T07:33:33.024761+0000","last_clean":"2024-09-16T07:33:33.024761+0000","last_became_active":"2024-09-16T07:33:32.040636+0000","last_became_peered":"2024-09-16T07:33:32.040636+0000","last_unstale":"2024-09-16T07:33:33.024761+0000","last_undegraded":"2024-09-16T07:33:33.024761+0000","last_fullsized":"2024-09-16T07:33:33.024761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:06.934545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358353+0000","last_change":"2024-09-16T07:33:32.053936+0000","last_active":"2024-09-16T07:33:33.358353+0000","last_peered":"2024-09-16T07:33:33.358353+0000","last_clean":"2024-09-16T07:33:33.358353+0000","last_became_active":"2024-09-16T07:33:32.053546+0000","last_became_peered":"2024-09-16T07:33:32.053546+0000","last_unstale":"2024-09-16T07:33:33.358353+0000","last_undegraded":"2024-09-16T07:33:33.358353+0000","last_fullsized":"2024-09-16T07:33:33.358353+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.032081+0000","last_change":"2024-09-16T07:33:31.032081+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.032081+0000","last_undegraded":"2024-09-16T07:33:31.032081+0000","last_fullsized":"2024-09-16T07:33:31.032081+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027983+0000","last_change":"2024-09-16T07:33:31.027983+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027983+0000","last_undegraded":"2024-09-16T07:33:31.027983+0000","last_fullsized":"2024-09-16T07:33:31.027983+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023549+0000","last_change":"2024-09-16T07:33:32.055235+0000","last_active":"2024-09-16T07:33:33.023549+0000","last_peered":"2024-09-16T07:33:33.023549+0000","last_clean":"2024-09-16T07:33:33.023549+0000","last_became_active":"2024-09-16T07:33:32.052675+0000","last_became_peered":"2024-09-16T07:33:32.052675+0000","last_unstale":"2024-09-16T07:33:33.023549+0000","last_undegraded":"2024-09-16T07:33:33.023549+0000","last_fullsized":"2024-09-16T07:33:33.023549+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:31.315404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357746+0000","last_change":"2024-09-16T07:33:32.050486+0000","last_active":"2024-09-16T07:33:33.357746+0000","last_peered":"2024-09-16T07:33:33.357746+0000","last_clean":"2024-09-16T07:33:33.357746+0000","last_became_active":"2024-09-16T07:33:32.050108+0000","last_became_peered":"2024-09-16T07:33:32.050108+0000","last_unstale":"2024-09-16T07:33:33.357746+0000","last_undegraded":"2024-09-16T07:33:33.357746+0000","last_fullsized":"2024-09-16T07:33:33.357746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021955+0000","last_change":"2024-09-16T07:33:32.028117+0000","last_active":"2024-09-16T07:33:33.021955+0000","last_peered":"2024-09-16T07:33:33.021955+0000","last_clean":"2024-09-16T07:33:33.021955+0000","last_became_active":"2024-09-16T07:33:32.027826+0000","last_became_peered":"2024-09-16T07:33:32.027826+0000","last_unstale":"2024-09-16T07:33:33.021955+0000","last_undegraded":"2024-09-16T07:33:33.021955+0000","last_fullsized":"2024-09-16T07:33:33.021955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:35.757466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.055382+0000","last_change":"2024-09-16T07:33:32.049150+0000","last_active":"2024-09-16T07:33:32.055382+0000","last_peered":"2024-09-16T07:33:32.055382+0000","last_clean":"2024-09-16T07:33:32.055382+0000","last_became_active":"2024-09-16T07:33:32.048893+0000","last_became_peered":"2024-09-16T07:33:32.048893+0000","last_unstale":"2024-09-16T07:33:32.055382+0000","last_undegraded":"2024-09-16T07:33:32.055382+0000","last_fullsized":"2024-09-16T07:33:32.055382+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:37.326467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358162+0000","last_change":"2024-09-16T07:33:32.041462+0000","last_active":"2024-09-16T07:33:33.358162+0000","last_peered":"2024-09-16T07:33:33.358162+0000","last_clean":"2024-09-16T07:33:33.358162+0000","last_became_active":"2024-09-16T07:33:32.041198+0000","last_became_peered":"2024-09-16T07:33:32.041198+0000","last_unstale":"2024-09-16T07:33:33.358162+0000","last_undegraded":"2024-09-16T07:33:33.358162+0000","last_fullsized":"2024-09-16T07:33:33.358162+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358295+0000","last_change":"2024-09-16T07:33:32.037402+0000","last_active":"2024-09-16T07:33:33.358295+0000","last_peered":"2024-09-16T07:33:33.358295+0000","last_clean":"2024-09-16T07:33:33.358295+0000","last_became_active":"2024-09-16T07:33:32.037131+0000","last_became_peered":"2024-09-16T07:33:32.037131+0000","last_unstale":"2024-09-16T07:33:33.358295+0000","last_undegraded":"2024-09-16T07:33:33.358295+0000","last_fullsized":"2024-09-16T07:33:33.358295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.052439+0000","last_change":"2024-09-16T07:33:32.043318+0000","last_active":"2024-09-16T07:33:32.052439+0000","last_peered":"2024-09-16T07:33:32.052439+0000","last_clean":"2024-09-16T07:33:32.052439+0000","last_became_active":"2024-09-16T07:33:32.043039+0000","last_became_peered":"2024-09-16T07:33:32.043039+0000","last_unstale":"2024-09-16T07:33:32.052439+0000","last_undegraded":"2024-09-16T07:33:32.052439+0000","last_fullsized":"2024-09-16T07:33:32.052439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:45.700538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.059020+0000","last_change":"2024-09-16T07:33:32.056817+0000","last_active":"2024-09-16T07:33:32.059020+0000","last_peered":"2024-09-16T07:33:32.059020+0000","last_clean":"2024-09-16T07:33:32.059020+0000","last_became_active":"2024-09-16T07:33:32.056535+0000","last_became_peered":"2024-09-16T07:33:32.056535+0000","last_unstale":"2024-09-16T07:33:32.059020+0000","last_undegraded":"2024-09-16T07:33:32.059020+0000","last_fullsized":"2024-09-16T07:33:32.059020+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:00:35.876240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027968+0000","last_change":"2024-09-16T07:33:31.027968+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027968+0000","last_undegraded":"2024-09-16T07:33:31.027968+0000","last_fullsized":"2024-09-16T07:33:31.027968+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023806+0000","last_change":"2024-09-16T07:33:32.036224+0000","last_active":"2024-09-16T07:33:33.023806+0000","last_peered":"2024-09-16T07:33:33.023806+0000","last_clean":"2024-09-16T07:33:33.023806+0000","last_became_active":"2024-09-16T07:33:32.035020+0000","last_became_peered":"2024-09-16T07:33:32.035020+0000","last_unstale":"2024-09-16T07:33:33.023806+0000","last_undegraded":"2024-09-16T07:33:33.023806+0000","last_fullsized":"2024-09-16T07:33:33.023806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:00.997013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025689+0000","last_change":"2024-09-16T07:33:32.046309+0000","last_active":"2024-09-16T07:33:33.025689+0000","last_peered":"2024-09-16T07:33:33.025689+0000","last_clean":"2024-09-16T07:33:33.025689+0000","last_became_active":"2024-09-16T07:33:32.045999+0000","last_became_peered":"2024-09-16T07:33:32.045999+0000","last_unstale":"2024-09-16T07:33:33.025689+0000","last_undegraded":"2024-09-16T07:33:33.025689+0000","last_fullsized":"2024-09-16T07:33:33.025689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:09.445884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.053925+0000","last_change":"2024-09-16T07:33:31.045574+0000","last_active":"2024-09-16T07:33:32.053925+0000","last_peered":"2024-09-16T07:33:32.053925+0000","last_clean":"2024-09-16T07:33:32.053925+0000","last_became_active":"2024-09-16T07:33:31.045014+0000","last_became_peered":"2024-09-16T07:33:31.045014+0000","last_unstale":"2024-09-16T07:33:32.053925+0000","last_undegraded":"2024-09-16T07:33:32.053925+0000","last_fullsized":"2024-09-16T07:33:32.053925+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.581864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.052698+0000","last_change":"2024-09-16T07:33:32.046389+0000","last_active":"2024-09-16T07:33:32.052698+0000","last_peered":"2024-09-16T07:33:32.052698+0000","last_clean":"2024-09-16T07:33:32.052698+0000","last_became_active":"2024-09-16T07:33:32.046041+0000","last_became_peered":"2024-09-16T07:33:32.046041+0000","last_unstale":"2024-09-16T07:33:32.052698+0000","last_undegraded":"2024-09-16T07:33:32.052698+0000","last_fullsized":"2024-09-16T07:33:32.052698+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:20.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357662+0000","last_change":"2024-09-16T07:33:31.041205+0000","last_active":"2024-09-16T07:33:33.357662+0000","last_peered":"2024-09-16T07:33:33.357662+0000","last_clean":"2024-09-16T07:33:33.357662+0000","last_became_active":"2024-09-16T07:33:31.040923+0000","last_became_peered":"2024-09-16T07:33:31.040923+0000","last_unstale":"2024-09-16T07:33:33.357662+0000","last_undegraded":"2024-09-16T07:33:33.357662+0000","last_fullsized":"2024-09-16T07:33:33.357662+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:56.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024526+0000","last_change":"2024-09-16T07:33:32.039150+0000","last_active":"2024-09-16T07:33:33.024526+0000","last_peered":"2024-09-16T07:33:33.024526+0000","last_clean":"2024-09-16T07:33:33.024526+0000","last_became_active":"2024-09-16T07:33:32.038837+0000","last_became_peered":"2024-09-16T07:33:32.038837+0000","last_unstale":"2024-09-16T07:33:33.024526+0000","last_undegraded":"2024-09-16T07:33:33.024526+0000","last_fullsized":"2024-09-16T07:33:33.024526+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:52.257876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.046999+0000","last_change":"2024-09-16T07:33:31.045183+0000","last_active":"2024-09-16T07:33:31.046999+0000","last_peered":"2024-09-16T07:33:31.046999+0000","last_clean":"2024-09-16T07:33:31.046999+0000","last_became_active":"2024-09-16T07:33:31.044253+0000","last_became_peered":"2024-09-16T07:33:31.044253+0000","last_unstale":"2024-09-16T07:33:31.046999+0000","last_undegraded":"2024-09-16T07:33:31.046999+0000","last_fullsized":"2024-09-16T07:33:31.046999+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:15:28.325154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021838+0000","last_change":"2024-09-16T07:33:32.043094+0000","last_active":"2024-09-16T07:33:33.021838+0000","last_peered":"2024-09-16T07:33:33.021838+0000","last_clean":"2024-09-16T07:33:33.021838+0000","last_became_active":"2024-09-16T07:33:32.042832+0000","last_became_peered":"2024-09-16T07:33:32.042832+0000","last_unstale":"2024-09-16T07:33:33.021838+0000","last_undegraded":"2024-09-16T07:33:33.021838+0000","last_fullsized":"2024-09-16T07:33:33.021838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:24.827070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024550+0000","last_change":"2024-09-16T07:33:31.042458+0000","last_active":"2024-09-16T07:33:33.024550+0000","last_peered":"2024-09-16T07:33:33.024550+0000","last_clean":"2024-09-16T07:33:33.024550+0000","last_became_active":"2024-09-16T07:33:31.042226+0000","last_became_peered":"2024-09-16T07:33:31.042226+0000","last_unstale":"2024-09-16T07:33:33.024550+0000","last_undegraded":"2024-09-16T07:33:33.024550+0000","last_fullsized":"2024-09-16T07:33:33.024550+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:20.182053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357806+0000","last_change":"2024-09-16T07:33:32.039748+0000","last_active":"2024-09-16T07:33:33.357806+0000","last_peered":"2024-09-16T07:33:33.357806+0000","last_clean":"2024-09-16T07:33:33.357806+0000","last_became_active":"2024-09-16T07:33:32.038397+0000","last_became_peered":"2024-09-16T07:33:32.038397+0000","last_unstale":"2024-09-16T07:33:33.357806+0000","last_undegraded":"2024-09-16T07:33:33.357806+0000","last_fullsized":"2024-09-16T07:33:33.357806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:15.732874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023864+0000","last_change":"2024-09-16T07:33:31.033141+0000","last_active":"2024-09-16T07:33:33.023864+0000","last_peered":"2024-09-16T07:33:33.023864+0000","last_clean":"2024-09-16T07:33:33.023864+0000","last_became_active":"2024-09-16T07:33:31.032570+0000","last_became_peered":"2024-09-16T07:33:31.032570+0000","last_unstale":"2024-09-16T07:33:33.023864+0000","last_undegraded":"2024-09-16T07:33:33.023864+0000","last_fullsized":"2024-09-16T07:33:33.023864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:26.450637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357619+0000","last_change":"2024-09-16T07:33:32.044965+0000","last_active":"2024-09-16T07:33:33.357619+0000","last_peered":"2024-09-16T07:33:33.357619+0000","last_clean":"2024-09-16T07:33:33.357619+0000","last_became_active":"2024-09-16T07:33:32.044705+0000","last_became_peered":"2024-09-16T07:33:32.044705+0000","last_unstale":"2024-09-16T07:33:33.357619+0000","last_undegraded":"2024-09-16T07:33:33.357619+0000","last_fullsized":"2024-09-16T07:33:33.357619+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:50.022797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.040511+0000","last_change":"2024-09-16T07:33:31.035805+0000","last_active":"2024-09-16T07:33:31.040511+0000","last_peered":"2024-09-16T07:33:31.040511+0000","last_clean":"2024-09-16T07:33:31.040511+0000","last_became_active":"2024-09-16T07:33:31.035545+0000","last_became_peered":"2024-09-16T07:33:31.035545+0000","last_unstale":"2024-09-16T07:33:31.040511+0000","last_undegraded":"2024-09-16T07:33:31.040511+0000","last_fullsized":"2024-09-16T07:33:31.040511+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:16.651576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.053801+0000","last_change":"2024-09-16T07:33:32.051200+0000","last_active":"2024-09-16T07:33:32.053801+0000","last_peered":"2024-09-16T07:33:32.053801+0000","last_clean":"2024-09-16T07:33:32.053801+0000","last_became_active":"2024-09-16T07:33:32.047036+0000","last_became_peered":"2024-09-16T07:33:32.047036+0000","last_unstale":"2024-09-16T07:33:32.053801+0000","last_undegraded":"2024-09-16T07:33:32.053801+0000","last_fullsized":"2024-09-16T07:33:32.053801+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:23.177353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357424+0000","last_change":"2024-09-16T07:33:31.034485+0000","last_active":"2024-09-16T07:33:33.357424+0000","last_peered":"2024-09-16T07:33:33.357424+0000","last_clean":"2024-09-16T07:33:33.357424+0000","last_became_active":"2024-09-16T07:33:31.034236+0000","last_became_peered":"2024-09-16T07:33:31.034236+0000","last_unstale":"2024-09-16T07:33:33.357424+0000","last_undegraded":"2024-09-16T07:33:33.357424+0000","last_fullsized":"2024-09-16T07:33:33.357424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:52.133642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357028+0000","last_change":"2024-09-16T07:33:32.039686+0000","last_active":"2024-09-16T07:33:33.357028+0000","last_peered":"2024-09-16T07:33:33.357028+0000","last_clean":"2024-09-16T07:33:33.357028+0000","last_became_active":"2024-09-16T07:33:32.038343+0000","last_became_peered":"2024-09-16T07:33:32.038343+0000","last_unstale":"2024-09-16T07:33:33.357028+0000","last_undegraded":"2024-09-16T07:33:33.357028+0000","last_fullsized":"2024-09-16T07:33:33.357028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:00.455428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024348+0000","last_change":"2024-09-16T07:33:31.039439+0000","last_active":"2024-09-16T07:33:33.024348+0000","last_peered":"2024-09-16T07:33:33.024348+0000","last_clean":"2024-09-16T07:33:33.024348+0000","last_became_active":"2024-09-16T07:33:31.039029+0000","last_became_peered":"2024-09-16T07:33:31.039029+0000","last_unstale":"2024-09-16T07:33:33.024348+0000","last_undegraded":"2024-09-16T07:33:33.024348+0000","last_fullsized":"2024-09-16T07:33:33.024348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:21.209706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025089+0000","last_change":"2024-09-16T07:33:32.045355+0000","last_active":"2024-09-16T07:33:33.025089+0000","last_peered":"2024-09-16T07:33:33.025089+0000","last_clean":"2024-09-16T07:33:33.025089+0000","last_became_active":"2024-09-16T07:33:32.045070+0000","last_became_peered":"2024-09-16T07:33:32.045070+0000","last_unstale":"2024-09-16T07:33:33.025089+0000","last_undegraded":"2024-09-16T07:33:33.025089+0000","last_fullsized":"2024-09-16T07:33:33.025089+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.042006+0000","last_change":"2024-09-16T07:33:31.039400+0000","last_active":"2024-09-16T07:33:31.042006+0000","last_peered":"2024-09-16T07:33:31.042006+0000","last_clean":"2024-09-16T07:33:31.042006+0000","last_became_active":"2024-09-16T07:33:31.039119+0000","last_became_peered":"2024-09-16T07:33:31.039119+0000","last_unstale":"2024-09-16T07:33:31.042006+0000","last_undegraded":"2024-09-16T07:33:31.042006+0000","last_fullsized":"2024-09-16T07:33:31.042006+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:34.147476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358517+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.358517+0000","last_peered":"2024-09-16T07:33:33.358517+0000","last_clean":"2024-09-16T07:33:33.358517+0000","last_became_active":"2024-09-16T07:33:32.035316+0000","last_became_peered":"2024-09-16T07:33:32.035316+0000","last_unstale":"2024-09-16T07:33:33.358517+0000","last_undegraded":"2024-09-16T07:33:33.358517+0000","last_fullsized":"2024-09-16T07:33:33.358517+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:06.190971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025197+0000","last_change":"2024-09-16T07:33:31.037265+0000","last_active":"2024-09-16T07:33:33.025197+0000","last_peered":"2024-09-16T07:33:33.025197+0000","last_clean":"2024-09-16T07:33:33.025197+0000","last_became_active":"2024-09-16T07:33:31.036981+0000","last_became_peered":"2024-09-16T07:33:31.036981+0000","last_unstale":"2024-09-16T07:33:33.025197+0000","last_undegraded":"2024-09-16T07:33:33.025197+0000","last_fullsized":"2024-09-16T07:33:33.025197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:08.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357328+0000","last_change":"2024-09-16T07:33:32.042815+0000","last_active":"2024-09-16T07:33:33.357328+0000","last_peered":"2024-09-16T07:33:33.357328+0000","last_clean":"2024-09-16T07:33:33.357328+0000","last_became_active":"2024-09-16T07:33:32.042470+0000","last_became_peered":"2024-09-16T07:33:32.042470+0000","last_unstale":"2024-09-16T07:33:33.357328+0000","last_undegraded":"2024-09-16T07:33:33.357328+0000","last_fullsized":"2024-09-16T07:33:33.357328+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:39.595734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357447+0000","last_change":"2024-09-16T07:33:31.032993+0000","last_active":"2024-09-16T07:33:33.357447+0000","last_peered":"2024-09-16T07:33:33.357447+0000","last_clean":"2024-09-16T07:33:33.357447+0000","last_became_active":"2024-09-16T07:33:31.032741+0000","last_became_peered":"2024-09-16T07:33:31.032741+0000","last_unstale":"2024-09-16T07:33:33.357447+0000","last_undegraded":"2024-09-16T07:33:33.357447+0000","last_fullsized":"2024-09-16T07:33:33.357447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:03.607602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357971+0000","last_change":"2024-09-16T07:33:32.035426+0000","last_active":"2024-09-16T07:33:33.357971+0000","last_peered":"2024-09-16T07:33:33.357971+0000","last_clean":"2024-09-16T07:33:33.357971+0000","last_became_active":"2024-09-16T07:33:32.034929+0000","last_became_peered":"2024-09-16T07:33:32.034929+0000","last_unstale":"2024-09-16T07:33:33.357971+0000","last_undegraded":"2024-09-16T07:33:33.357971+0000","last_fullsized":"2024-09-16T07:33:33.357971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":21,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358359+0000","last_change":"2024-09-16T07:33:31.041833+0000","last_active":"2024-09-16T07:33:33.358359+0000","last_peered":"2024-09-16T07:33:33.358359+0000","last_clean":"2024-09-16T07:33:33.358359+0000","last_became_active":"2024-09-16T07:33:31.041034+0000","last_became_peered":"2024-09-16T07:33:31.041034+0000","last_unstale":"2024-09-16T07:33:33.358359+0000","last_undegraded":"2024-09-16T07:33:33.358359+0000","last_fullsized":"2024-09-16T07:33:33.358359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:25.618706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024155+0000","last_change":"2024-09-16T07:33:32.042155+0000","last_active":"2024-09-16T07:33:33.024155+0000","last_peered":"2024-09-16T07:33:33.024155+0000","last_clean":"2024-09-16T07:33:33.024155+0000","last_became_active":"2024-09-16T07:33:32.041955+0000","last_became_peered":"2024-09-16T07:33:32.041955+0000","last_unstale":"2024-09-16T07:33:33.024155+0000","last_undegraded":"2024-09-16T07:33:33.024155+0000","last_fullsized":"2024-09-16T07:33:33.024155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:45.546271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025124+0000","last_change":"2024-09-16T07:33:31.036453+0000","last_active":"2024-09-16T07:33:33.025124+0000","last_peered":"2024-09-16T07:33:33.025124+0000","last_clean":"2024-09-16T07:33:33.025124+0000","last_became_active":"2024-09-16T07:33:31.034972+0000","last_became_peered":"2024-09-16T07:33:31.034972+0000","last_unstale":"2024-09-16T07:33:33.025124+0000","last_undegraded":"2024-09-16T07:33:33.025124+0000","last_fullsized":"2024-09-16T07:33:33.025124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:49.504210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.031515+0000","last_change":"2024-09-16T07:33:31.031515+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.031515+0000","last_undegraded":"2024-09-16T07:33:31.031515+0000","last_fullsized":"2024-09-16T07:33:31.031515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024241+0000","last_change":"2024-09-16T07:33:31.042034+0000","last_active":"2024-09-16T07:33:33.024241+0000","last_peered":"2024-09-16T07:33:33.024241+0000","last_clean":"2024-09-16T07:33:33.024241+0000","last_became_active":"2024-09-16T07:33:31.041059+0000","last_became_peered":"2024-09-16T07:33:31.041059+0000","last_unstale":"2024-09-16T07:33:33.024241+0000","last_undegraded":"2024-09-16T07:33:33.024241+0000","last_fullsized":"2024-09-16T07:33:33.024241+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:59.775582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357991+0000","last_change":"2024-09-16T07:33:32.046065+0000","last_active":"2024-09-16T07:33:33.357991+0000","last_peered":"2024-09-16T07:33:33.357991+0000","last_clean":"2024-09-16T07:33:33.357991+0000","last_became_active":"2024-09-16T07:33:32.045750+0000","last_became_peered":"2024-09-16T07:33:32.045750+0000","last_unstale":"2024-09-16T07:33:33.357991+0000","last_undegraded":"2024-09-16T07:33:33.357991+0000","last_fullsized":"2024-09-16T07:33:33.357991+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:26.414491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357741+0000","last_change":"2024-09-16T07:33:31.031995+0000","last_active":"2024-09-16T07:33:33.357741+0000","last_peered":"2024-09-16T07:33:33.357741+0000","last_clean":"2024-09-16T07:33:33.357741+0000","last_became_active":"2024-09-16T07:33:31.031131+0000","last_became_peered":"2024-09-16T07:33:31.031131+0000","last_unstale":"2024-09-16T07:33:33.357741+0000","last_undegraded":"2024-09-16T07:33:33.357741+0000","last_fullsized":"2024-09-16T07:33:33.357741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:28.800839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.027506+0000","last_change":"2024-09-16T07:33:31.027506+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.027506+0000","last_undegraded":"2024-09-16T07:33:31.027506+0000","last_fullsized":"2024-09-16T07:33:31.027506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023950+0000","last_change":"2024-09-16T07:33:31.030808+0000","last_active":"2024-09-16T07:33:33.023950+0000","last_peered":"2024-09-16T07:33:33.023950+0000","last_clean":"2024-09-16T07:33:33.023950+0000","last_became_active":"2024-09-16T07:33:31.028772+0000","last_became_peered":"2024-09-16T07:33:31.028772+0000","last_unstale":"2024-09-16T07:33:33.023950+0000","last_undegraded":"2024-09-16T07:33:33.023950+0000","last_fullsized":"2024-09-16T07:33:33.023950+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:53.744425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358706+0000","last_change":"2024-09-16T07:33:32.045394+0000","last_active":"2024-09-16T07:33:33.358706+0000","last_peered":"2024-09-16T07:33:33.358706+0000","last_clean":"2024-09-16T07:33:33.358706+0000","last_became_active":"2024-09-16T07:33:32.045106+0000","last_became_peered":"2024-09-16T07:33:32.045106+0000","last_unstale":"2024-09-16T07:33:33.358706+0000","last_undegraded":"2024-09-16T07:33:33.358706+0000","last_fullsized":"2024-09-16T07:33:33.358706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.537036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358438+0000","last_change":"2024-09-16T07:33:31.041342+0000","last_active":"2024-09-16T07:33:33.358438+0000","last_peered":"2024-09-16T07:33:33.358438+0000","last_clean":"2024-09-16T07:33:33.358438+0000","last_became_active":"2024-09-16T07:33:31.041056+0000","last_became_peered":"2024-09-16T07:33:31.041056+0000","last_unstale":"2024-09-16T07:33:33.358438+0000","last_undegraded":"2024-09-16T07:33:33.358438+0000","last_fullsized":"2024-09-16T07:33:33.358438+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:39.097159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358460+0000","last_change":"2024-09-16T07:33:32.044491+0000","last_active":"2024-09-16T07:33:33.358460+0000","last_peered":"2024-09-16T07:33:33.358460+0000","last_clean":"2024-09-16T07:33:33.358460+0000","last_became_active":"2024-09-16T07:33:32.044228+0000","last_became_peered":"2024-09-16T07:33:32.044228+0000","last_unstale":"2024-09-16T07:33:33.358460+0000","last_undegraded":"2024-09-16T07:33:33.358460+0000","last_fullsized":"2024-09-16T07:33:33.358460+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:14.023175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357548+0000","last_change":"2024-09-16T07:33:31.034256+0000","last_active":"2024-09-16T07:33:33.357548+0000","last_peered":"2024-09-16T07:33:33.357548+0000","last_clean":"2024-09-16T07:33:33.357548+0000","last_became_active":"2024-09-16T07:33:31.033825+0000","last_became_peered":"2024-09-16T07:33:31.033825+0000","last_unstale":"2024-09-16T07:33:33.357548+0000","last_undegraded":"2024-09-16T07:33:33.357548+0000","last_fullsized":"2024-09-16T07:33:33.357548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:44.857125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358285+0000","last_change":"2024-09-16T07:33:32.041248+0000","last_active":"2024-09-16T07:33:33.358285+0000","last_peered":"2024-09-16T07:33:33.358285+0000","last_clean":"2024-09-16T07:33:33.358285+0000","last_became_active":"2024-09-16T07:33:32.040998+0000","last_became_peered":"2024-09-16T07:33:32.040998+0000","last_unstale":"2024-09-16T07:33:33.358285+0000","last_undegraded":"2024-09-16T07:33:33.358285+0000","last_fullsized":"2024-09-16T07:33:33.358285+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:06.696198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025145+0000","last_change":"2024-09-16T07:33:31.035273+0000","last_active":"2024-09-16T07:33:33.025145+0000","last_peered":"2024-09-16T07:33:33.025145+0000","last_clean":"2024-09-16T07:33:33.025145+0000","last_became_active":"2024-09-16T07:33:31.034957+0000","last_became_peered":"2024-09-16T07:33:31.034957+0000","last_unstale":"2024-09-16T07:33:33.025145+0000","last_undegraded":"2024-09-16T07:33:33.025145+0000","last_fullsized":"2024-09-16T07:33:33.025145+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:17.685015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025196+0000","last_change":"2024-09-16T07:33:32.033426+0000","last_active":"2024-09-16T07:33:33.025196+0000","last_peered":"2024-09-16T07:33:33.025196+0000","last_clean":"2024-09-16T07:33:33.025196+0000","last_became_active":"2024-09-16T07:33:32.032989+0000","last_became_peered":"2024-09-16T07:33:32.032989+0000","last_unstale":"2024-09-16T07:33:33.025196+0000","last_undegraded":"2024-09-16T07:33:33.025196+0000","last_fullsized":"2024-09-16T07:33:33.025196+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:06.277092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023949+0000","last_change":"2024-09-16T07:33:31.038931+0000","last_active":"2024-09-16T07:33:33.023949+0000","last_peered":"2024-09-16T07:33:33.023949+0000","last_clean":"2024-09-16T07:33:33.023949+0000","last_became_active":"2024-09-16T07:33:31.038639+0000","last_became_peered":"2024-09-16T07:33:31.038639+0000","last_unstale":"2024-09-16T07:33:33.023949+0000","last_undegraded":"2024-09-16T07:33:33.023949+0000","last_fullsized":"2024-09-16T07:33:33.023949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:37.191040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025027+0000","last_change":"2024-09-16T07:33:32.050892+0000","last_active":"2024-09-16T07:33:33.025027+0000","last_peered":"2024-09-16T07:33:33.025027+0000","last_clean":"2024-09-16T07:33:33.025027+0000","last_became_active":"2024-09-16T07:33:32.050449+0000","last_became_peered":"2024-09-16T07:33:32.050449+0000","last_unstale":"2024-09-16T07:33:33.025027+0000","last_undegraded":"2024-09-16T07:33:33.025027+0000","last_fullsized":"2024-09-16T07:33:33.025027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:53.956082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358694+0000","last_change":"2024-09-16T07:33:31.035777+0000","last_active":"2024-09-16T07:33:33.358694+0000","last_peered":"2024-09-16T07:33:33.358694+0000","last_clean":"2024-09-16T07:33:33.358694+0000","last_became_active":"2024-09-16T07:33:31.034448+0000","last_became_peered":"2024-09-16T07:33:31.034448+0000","last_unstale":"2024-09-16T07:33:33.358694+0000","last_undegraded":"2024-09-16T07:33:33.358694+0000","last_fullsized":"2024-09-16T07:33:33.358694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:45.713854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025244+0000","last_change":"2024-09-16T07:33:32.052617+0000","last_active":"2024-09-16T07:33:33.025244+0000","last_peered":"2024-09-16T07:33:33.025244+0000","last_clean":"2024-09-16T07:33:33.025244+0000","last_became_active":"2024-09-16T07:33:32.052089+0000","last_became_peered":"2024-09-16T07:33:32.052089+0000","last_unstale":"2024-09-16T07:33:33.025244+0000","last_undegraded":"2024-09-16T07:33:33.025244+0000","last_fullsized":"2024-09-16T07:33:33.025244+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:49.702884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021703+0000","last_change":"2024-09-16T07:33:31.036417+0000","last_active":"2024-09-16T07:33:33.021703+0000","last_peered":"2024-09-16T07:33:33.021703+0000","last_clean":"2024-09-16T07:33:33.021703+0000","last_became_active":"2024-09-16T07:33:31.036118+0000","last_became_peered":"2024-09-16T07:33:31.036118+0000","last_unstale":"2024-09-16T07:33:33.021703+0000","last_undegraded":"2024-09-16T07:33:33.021703+0000","last_fullsized":"2024-09-16T07:33:33.021703+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:50.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358156+0000","last_change":"2024-09-16T07:33:32.039339+0000","last_active":"2024-09-16T07:33:33.358156+0000","last_peered":"2024-09-16T07:33:33.358156+0000","last_clean":"2024-09-16T07:33:33.358156+0000","last_became_active":"2024-09-16T07:33:32.037162+0000","last_became_peered":"2024-09-16T07:33:32.037162+0000","last_unstale":"2024-09-16T07:33:33.358156+0000","last_undegraded":"2024-09-16T07:33:33.358156+0000","last_fullsized":"2024-09-16T07:33:33.358156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:42.850047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045296+0000","last_change":"2024-09-16T07:33:31.042541+0000","last_active":"2024-09-16T07:33:31.045296+0000","last_peered":"2024-09-16T07:33:31.045296+0000","last_clean":"2024-09-16T07:33:31.045296+0000","last_became_active":"2024-09-16T07:33:31.042235+0000","last_became_peered":"2024-09-16T07:33:31.042235+0000","last_unstale":"2024-09-16T07:33:31.045296+0000","last_undegraded":"2024-09-16T07:33:31.045296+0000","last_fullsized":"2024-09-16T07:33:31.045296+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:27:20.784004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025623+0000","last_change":"2024-09-16T07:33:32.045670+0000","last_active":"2024-09-16T07:33:33.025623+0000","last_peered":"2024-09-16T07:33:33.025623+0000","last_clean":"2024-09-16T07:33:33.025623+0000","last_became_active":"2024-09-16T07:33:32.045383+0000","last_became_peered":"2024-09-16T07:33:32.045383+0000","last_unstale":"2024-09-16T07:33:33.025623+0000","last_undegraded":"2024-09-16T07:33:33.025623+0000","last_fullsized":"2024-09-16T07:33:33.025623+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:56.570489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358500+0000","last_change":"2024-09-16T07:33:31.030651+0000","last_active":"2024-09-16T07:33:33.358500+0000","last_peered":"2024-09-16T07:33:33.358500+0000","last_clean":"2024-09-16T07:33:33.358500+0000","last_became_active":"2024-09-16T07:33:31.030312+0000","last_became_peered":"2024-09-16T07:33:31.030312+0000","last_unstale":"2024-09-16T07:33:33.358500+0000","last_undegraded":"2024-09-16T07:33:33.358500+0000","last_fullsized":"2024-09-16T07:33:33.358500+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:57.124363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025464+0000","last_change":"2024-09-16T07:33:32.045722+0000","last_active":"2024-09-16T07:33:33.025464+0000","last_peered":"2024-09-16T07:33:33.025464+0000","last_clean":"2024-09-16T07:33:33.025464+0000","last_became_active":"2024-09-16T07:33:32.045414+0000","last_became_peered":"2024-09-16T07:33:32.045414+0000","last_unstale":"2024-09-16T07:33:33.025464+0000","last_undegraded":"2024-09-16T07:33:33.025464+0000","last_fullsized":"2024-09-16T07:33:33.025464+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:57.552859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025686+0000","last_change":"2024-09-16T07:33:31.040312+0000","last_active":"2024-09-16T07:33:33.025686+0000","last_peered":"2024-09-16T07:33:33.025686+0000","last_clean":"2024-09-16T07:33:33.025686+0000","last_became_active":"2024-09-16T07:33:31.040009+0000","last_became_peered":"2024-09-16T07:33:31.040009+0000","last_unstale":"2024-09-16T07:33:33.025686+0000","last_undegraded":"2024-09-16T07:33:33.025686+0000","last_fullsized":"2024-09-16T07:33:33.025686+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:56.393574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.032016+0000","last_change":"2024-09-16T07:33:31.032016+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.032016+0000","last_undegraded":"2024-09-16T07:33:31.032016+0000","last_fullsized":"2024-09-16T07:33:31.032016+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025287+0000","last_change":"2024-09-16T07:33:31.033696+0000","last_active":"2024-09-16T07:33:33.025287+0000","last_peered":"2024-09-16T07:33:33.025287+0000","last_clean":"2024-09-16T07:33:33.025287+0000","last_became_active":"2024-09-16T07:33:31.033430+0000","last_became_peered":"2024-09-16T07:33:31.033430+0000","last_unstale":"2024-09-16T07:33:33.025287+0000","last_undegraded":"2024-09-16T07:33:33.025287+0000","last_fullsized":"2024-09-16T07:33:33.025287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:55.604351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357346+0000","last_change":"2024-09-16T07:33:32.042206+0000","last_active":"2024-09-16T07:33:33.357346+0000","last_peered":"2024-09-16T07:33:33.357346+0000","last_clean":"2024-09-16T07:33:33.357346+0000","last_became_active":"2024-09-16T07:33:32.041947+0000","last_became_peered":"2024-09-16T07:33:32.041947+0000","last_unstale":"2024-09-16T07:33:33.357346+0000","last_undegraded":"2024-09-16T07:33:33.357346+0000","last_fullsized":"2024-09-16T07:33:33.357346+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:37:06.473975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358556+0000","last_change":"2024-09-16T07:33:31.032898+0000","last_active":"2024-09-16T07:33:33.358556+0000","last_peered":"2024-09-16T07:33:33.358556+0000","last_clean":"2024-09-16T07:33:33.358556+0000","last_became_active":"2024-09-16T07:33:31.032344+0000","last_became_peered":"2024-09-16T07:33:31.032344+0000","last_unstale":"2024-09-16T07:33:33.358556+0000","last_undegraded":"2024-09-16T07:33:33.358556+0000","last_fullsized":"2024-09-16T07:33:33.358556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:26.380802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025119+0000","last_change":"2024-09-16T07:33:32.038999+0000","last_active":"2024-09-16T07:33:33.025119+0000","last_peered":"2024-09-16T07:33:33.025119+0000","last_clean":"2024-09-16T07:33:33.025119+0000","last_became_active":"2024-09-16T07:33:32.038709+0000","last_became_peered":"2024-09-16T07:33:32.038709+0000","last_unstale":"2024-09-16T07:33:33.025119+0000","last_undegraded":"2024-09-16T07:33:33.025119+0000","last_fullsized":"2024-09-16T07:33:33.025119+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:02.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045928+0000","last_change":"2024-09-16T07:33:31.044406+0000","last_active":"2024-09-16T07:33:31.045928+0000","last_peered":"2024-09-16T07:33:31.045928+0000","last_clean":"2024-09-16T07:33:31.045928+0000","last_became_active":"2024-09-16T07:33:31.044017+0000","last_became_peered":"2024-09-16T07:33:31.044017+0000","last_unstale":"2024-09-16T07:33:31.045928+0000","last_undegraded":"2024-09-16T07:33:31.045928+0000","last_fullsized":"2024-09-16T07:33:31.045928+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:37.852077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357714+0000","last_change":"2024-09-16T07:33:32.046842+0000","last_active":"2024-09-16T07:33:33.357714+0000","last_peered":"2024-09-16T07:33:33.357714+0000","last_clean":"2024-09-16T07:33:33.357714+0000","last_became_active":"2024-09-16T07:33:32.046569+0000","last_became_peered":"2024-09-16T07:33:32.046569+0000","last_unstale":"2024-09-16T07:33:33.357714+0000","last_undegraded":"2024-09-16T07:33:33.357714+0000","last_fullsized":"2024-09-16T07:33:33.357714+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:01.321702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358757+0000","last_change":"2024-09-16T07:33:31.037028+0000","last_active":"2024-09-16T07:33:33.358757+0000","last_peered":"2024-09-16T07:33:33.358757+0000","last_clean":"2024-09-16T07:33:33.358757+0000","last_became_active":"2024-09-16T07:33:31.036714+0000","last_became_peered":"2024-09-16T07:33:31.036714+0000","last_unstale":"2024-09-16T07:33:33.358757+0000","last_undegraded":"2024-09-16T07:33:33.358757+0000","last_fullsized":"2024-09-16T07:33:33.358757+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:01.872929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024139+0000","last_change":"2024-09-16T07:33:32.036865+0000","last_active":"2024-09-16T07:33:33.024139+0000","last_peered":"2024-09-16T07:33:33.024139+0000","last_clean":"2024-09-16T07:33:33.024139+0000","last_became_active":"2024-09-16T07:33:32.036613+0000","last_became_peered":"2024-09-16T07:33:32.036613+0000","last_unstale":"2024-09-16T07:33:33.024139+0000","last_undegraded":"2024-09-16T07:33:33.024139+0000","last_fullsized":"2024-09-16T07:33:33.024139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:29:18.832918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T07:33:32.040193+0000","last_change":"2024-09-16T07:33:31.044776+0000","last_active":"2024-09-16T07:33:32.040193+0000","last_peered":"2024-09-16T07:33:32.040193+0000","last_clean":"2024-09-16T07:33:32.040193+0000","last_became_active":"2024-09-16T07:33:31.044453+0000","last_became_peered":"2024-09-16T07:33:31.044453+0000","last_unstale":"2024-09-16T07:33:32.040193+0000","last_undegraded":"2024-09-16T07:33:32.040193+0000","last_fullsized":"2024-09-16T07:33:32.040193+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:06.219751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358395+0000","last_change":"2024-09-16T07:33:32.043818+0000","last_active":"2024-09-16T07:33:33.358395+0000","last_peered":"2024-09-16T07:33:33.358395+0000","last_clean":"2024-09-16T07:33:33.358395+0000","last_became_active":"2024-09-16T07:33:32.043573+0000","last_became_peered":"2024-09-16T07:33:32.043573+0000","last_unstale":"2024-09-16T07:33:33.358395+0000","last_undegraded":"2024-09-16T07:33:33.358395+0000","last_fullsized":"2024-09-16T07:33:33.358395+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:10.405842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T07:33:31.045274+0000","last_change":"2024-09-16T07:33:31.042397+0000","last_active":"2024-09-16T07:33:31.045274+0000","last_peered":"2024-09-16T07:33:31.045274+0000","last_clean":"2024-09-16T07:33:31.045274+0000","last_became_active":"2024-09-16T07:33:31.041205+0000","last_became_peered":"2024-09-16T07:33:31.041205+0000","last_unstale":"2024-09-16T07:33:31.045274+0000","last_undegraded":"2024-09-16T07:33:31.045274+0000","last_fullsized":"2024-09-16T07:33:31.045274+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:13.919862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-09-16T07:33:31.030279+0000","last_change":"2024-09-16T07:33:31.030279+0000","last_active":"2024-09-16T07:33:31.008881+0000","last_peered":"2024-09-16T07:33:31.008881+0000","last_clean":"2024-09-16T07:33:31.008881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:33:31.030279+0000","last_undegraded":"2024-09-16T07:33:31.030279+0000","last_fullsized":"2024-09-16T07:33:31.030279+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358516+0000","last_change":"2024-09-16T07:33:31.041709+0000","last_active":"2024-09-16T07:33:33.358516+0000","last_peered":"2024-09-16T07:33:33.358516+0000","last_clean":"2024-09-16T07:33:33.358516+0000","last_became_active":"2024-09-16T07:33:31.040991+0000","last_became_peered":"2024-09-16T07:33:31.040991+0000","last_unstale":"2024-09-16T07:33:33.358516+0000","last_undegraded":"2024-09-16T07:33:33.358516+0000","last_fullsized":"2024-09-16T07:33:33.358516+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:32.562633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":265,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357091+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:33:33.357091+0000","last_peered":"2024-09-16T07:33:33.357091+0000","last_clean":"2024-09-16T07:33:33.357091+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:33:33.357091+0000","last_undegraded":"2024-09-16T07:33:33.357091+0000","last_fullsized":"2024-09-16T07:33:33.357091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476872,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122632,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249208,"statfs":{"total":96636764160,"available":96511188992,"internally_reserved":0,"allocated":1298432,"data_stored":560047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122612,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249228,"statfs":{"total":96636764160,"available":96511209472,"internally_reserved":0,"allocated":1277952,"data_stored":553595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119056,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252784,"statfs":{"total":96636764160,"available":96514850816,"internally_reserved":0,"allocated":1888256,"data_stored":1145617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100400,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271440,"statfs":{"total":96636764160,"available":96533954560,"internally_reserved":0,"allocated":1273856,"data_stored":553525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100412,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271428,"statfs":{"total":96636764160,"available":96533942272,"internally_reserved":0,"allocated":1277952,"data_stored":554107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":101513674},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148984,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222856,"statfs":{"total":96636764160,"available":96484204544,"internally_reserved":0,"allocated":1273856,"data_stored":553525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509575,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127380,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244460,"statfs":{"total":96636764160,"available":96506327040,"internally_reserved":0,"allocated":1900544,"data_stored":1151889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542277,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122584,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249256,"statfs":{"total":96636764160,"available":96511238144,"internally_reserved":0,"allocated":1249280,"data_stored":540419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:33:40.925 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:33:41.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.087+0000 7fb48b200640 1 -- 172.21.15.71:0/2163620344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 msgr2=0x7fb48406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.087+0000 7fb48b200640 1 --2- 172.21.15.71:0/2163620344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 0x7fb48406f740 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fb480005fd0 tx=0x7fb48002aaa0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.088+0000 7fb48b200640 1 -- 172.21.15.71:0/2163620344 shutdown_connections 2024-09-16T07:33:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.088+0000 7fb48b200640 1 --2- 172.21.15.71:0/2163620344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 0x7fb48406f740 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.088+0000 7fb48b200640 1 --2- 172.21.15.71:0/2163620344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4840cc8d0 0x7fb48415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.088+0000 7fb48b200640 1 -- 172.21.15.71:0/2163620344 >> 172.21.15.71:0/2163620344 conn(0x7fb4840d0b30 msgr2=0x7fb4840d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.088+0000 7fb48b200640 1 -- 172.21.15.71:0/2163620344 shutdown_connections 2024-09-16T07:33:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.088+0000 7fb48b200640 1 -- 172.21.15.71:0/2163620344 wait complete. 2024-09-16T07:33:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.089+0000 7fb48b200640 1 Processor -- start 2024-09-16T07:33:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.089+0000 7fb48b200640 1 -- start start 2024-09-16T07:33:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb48b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 0x7fb48414a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb48b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4840cc8d0 0x7fb4840c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb48b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4840d1620 0x7fb48414f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb48b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb48406fc80 con 0x7fb484073ee0 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb48b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4840745a0 con 0x7fb4840d1620 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb48b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb48406fe00 con 0x7fb4840cc8d0 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb489e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 0x7fb48414a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb489e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 0x7fb48414a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42308/0 (socket says 172.21.15.71:42308) 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.090+0000 7fb489e00640 1 -- 172.21.15.71:0/3392584799 learned_addr learned my addr 172.21.15.71:0/3392584799 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb489400640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4840cc8d0 0x7fb4840c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb489e00640 1 -- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4840cc8d0 msgr2=0x7fb4840c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb489e00640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4840cc8d0 0x7fb4840c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb489e00640 1 -- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4840d1620 msgr2=0x7fb48414f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb489e00640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4840d1620 0x7fb48414f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb489e00640 1 -- 172.21.15.71:0/3392584799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48414e280 con 0x7fb484073ee0 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb489400640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4840cc8d0 0x7fb4840c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb489e00640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 0x7fb48414a0f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00d980 tx=0x7fb47c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:41.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb47a000640 1 -- 172.21.15.71:0/3392584799 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c017070 con 0x7fb484073ee0 2024-09-16T07:33:41.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb47a000640 1 -- 172.21.15.71:0/3392584799 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb47c009d20 con 0x7fb484073ee0 2024-09-16T07:33:41.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb47a000640 1 -- 172.21.15.71:0/3392584799 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0106c0 con 0x7fb484073ee0 2024-09-16T07:33:41.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48414e560 con 0x7fb484073ee0 2024-09-16T07:33:41.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.091+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48414a630 con 0x7fb484073ee0 2024-09-16T07:33:41.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.092+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4840c78c0 con 0x7fb484073ee0 2024-09-16T07:33:41.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.094+0000 7fb47a000640 1 -- 172.21.15.71:0/3392584799 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fb47c010860 con 0x7fb484073ee0 2024-09-16T07:33:41.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.095+0000 7fb47a000640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb468079e40 0x7fb46807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.095+0000 7fb47a000640 1 -- 172.21.15.71:0/3392584799 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fb47c099f00 con 0x7fb484073ee0 2024-09-16T07:33:41.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.095+0000 7fb489400640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb468079e40 0x7fb46807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.096+0000 7fb489400640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb468079e40 0x7fb46807c300 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb474007fd0 tx=0x7fb474009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:41.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.099+0000 7fb47a000640 1 -- 172.21.15.71:0/3392584799 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb4840c78c0 con 0x7fb484073ee0 2024-09-16T07:33:41.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.282+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4841ea790 con 0x7fb468079e40 2024-09-16T07:33:41.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.286+0000 7fb47a000640 1 -- 172.21.15.71:0/3392584799 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252419 (secure 0 0 0) 0x7fb4841ea790 con 0x7fb468079e40 2024-09-16T07:33:41.290 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:41.291 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb468079e40 msgr2=0x7fb46807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb468079e40 0x7fb46807c300 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb474007fd0 tx=0x7fb474009040 comp rx=0 tx=0).stop 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 msgr2=0x7fb48414a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 0x7fb48414a0f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00d980 tx=0x7fb47c00de40 comp rx=0 tx=0).stop 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 shutdown_connections 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb468079e40 0x7fb46807c300 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4840d1620 0x7fb48414f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4840cc8d0 0x7fb4840c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 --2- 172.21.15.71:0/3392584799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb484073ee0 0x7fb48414a0f0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 >> 172.21.15.71:0/3392584799 conn(0x7fb4840d0b30 msgr2=0x7fb4840b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:41.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 shutdown_connections 2024-09-16T07:33:41.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.291+0000 7fb48b200640 1 -- 172.21.15.71:0/3392584799 wait complete. 2024-09-16T07:33:41.306 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-16T07:33:40.582623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":964144,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":952691,"kb_avail":754010576,"statfs":{"total":773094113280,"available":772106829824,"internally_reserved":0,"allocated":11542528,"data_stored":5664380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":975556144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.562998"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025017+0000","last_change":"2024-09-16T07:33:32.045068+0000","last_active":"2024-09-16T07:33:33.025017+0000","last_peered":"2024-09-16T07:33:33.025017+0000","last_clean":"2024-09-16T07:33:33.025017+0000","last_became_active":"2024-09-16T07:33:32.044772+0000","last_became_peered":"2024-09-16T07:33:32.044772+0000","last_unstale":"2024-09-16T07:33:33.025017+0000","last_undegraded":"2024-09-16T07:33:33.025017+0000","last_fullsized":"2024-09-16T07:33:33.025017+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024784+0000","last_change":"2024-09-16T07:33:32.049940+0000","last_active":"2024-09-16T07:33:33.024784+0000","last_peered":"2024-09-16T07:33:33.024784+0000","last_clean":"2024-09-16T07:33:33.024784+0000","last_became_active":"2024-09-16T07:33:32.049646+0000","last_became_peered":"2024-09-16T07:33:32.049646+0000","last_unstale":"2024-09-16T07:33:33.024784+0000","last_undegraded":"2024-09-16T07:33:33.024784+0000","last_fullsized":"2024-09-16T07:33:33.024784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357750+0000","last_change":"2024-09-16T07:33:32.055290+0000","last_active":"2024-09-16T07:33:33.357750+0000","last_peered":"2024-09-16T07:33:33.357750+0000","last_clean":"2024-09-16T07:33:33.357750+0000","last_became_active":"2024-09-16T07:33:32.055031+0000","last_became_peered":"2024-09-16T07:33:32.055031+0000","last_unstale":"2024-09-16T07:33:33.357750+0000","last_undegraded":"2024-09-16T07:33:33.357750+0000","last_fullsized":"2024-09-16T07:33:33.357750+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:43.744160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024064+0000","last_change":"2024-09-16T07:33:32.043883+0000","last_active":"2024-09-16T07:33:33.024064+0000","last_peered":"2024-09-16T07:33:33.024064+0000","last_clean":"2024-09-16T07:33:33.024064+0000","last_became_active":"2024-09-16T07:33:32.043622+0000","last_became_peered":"2024-09-16T07:33:32.043622+0000","last_unstale":"2024-09-16T07:33:33.024064+0000","last_undegraded":"2024-09-16T07:33:33.024064+0000","last_fullsized":"2024-09-16T07:33:33.024064+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:01:49.570122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357403+0000","last_change":"2024-09-16T07:33:32.043899+0000","last_active":"2024-09-16T07:33:33.357403+0000","last_peered":"2024-09-16T07:33:33.357403+0000","last_clean":"2024-09-16T07:33:33.357403+0000","last_became_active":"2024-09-16T07:33:32.043611+0000","last_became_peered":"2024-09-16T07:33:32.043611+0000","last_unstale":"2024-09-16T07:33:33.357403+0000","last_undegraded":"2024-09-16T07:33:33.357403+0000","last_fullsized":"2024-09-16T07:33:33.357403+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:22.213386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357942+0000","last_change":"2024-09-16T07:33:32.054833+0000","last_active":"2024-09-16T07:33:33.357942+0000","last_peered":"2024-09-16T07:33:33.357942+0000","last_clean":"2024-09-16T07:33:33.357942+0000","last_became_active":"2024-09-16T07:33:32.054513+0000","last_became_peered":"2024-09-16T07:33:32.054513+0000","last_unstale":"2024-09-16T07:33:33.357942+0000","last_undegraded":"2024-09-16T07:33:33.357942+0000","last_fullsized":"2024-09-16T07:33:33.357942+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:03.590298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357327+0000","last_change":"2024-09-16T07:33:32.037656+0000","last_active":"2024-09-16T07:33:33.357327+0000","last_peered":"2024-09-16T07:33:33.357327+0000","last_clean":"2024-09-16T07:33:33.357327+0000","last_became_active":"2024-09-16T07:33:32.036949+0000","last_became_peered":"2024-09-16T07:33:32.036949+0000","last_unstale":"2024-09-16T07:33:33.357327+0000","last_undegraded":"2024-09-16T07:33:33.357327+0000","last_fullsized":"2024-09-16T07:33:33.357327+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:18.915383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.356860+0000","last_change":"2024-09-16T07:33:32.044180+0000","last_active":"2024-09-16T07:33:33.356860+0000","last_peered":"2024-09-16T07:33:33.356860+0000","last_clean":"2024-09-16T07:33:33.356860+0000","last_became_active":"2024-09-16T07:33:32.043893+0000","last_became_peered":"2024-09-16T07:33:32.043893+0000","last_unstale":"2024-09-16T07:33:33.356860+0000","last_undegraded":"2024-09-16T07:33:33.356860+0000","last_fullsized":"2024-09-16T07:33:33.356860+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:20.855593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357756+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.357756+0000","last_peered":"2024-09-16T07:33:33.357756+0000","last_clean":"2024-09-16T07:33:33.357756+0000","last_became_active":"2024-09-16T07:33:32.035287+0000","last_became_peered":"2024-09-16T07:33:32.035287+0000","last_unstale":"2024-09-16T07:33:33.357756+0000","last_undegraded":"2024-09-16T07:33:33.357756+0000","last_fullsized":"2024-09-16T07:33:33.357756+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:52.049650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021843+0000","last_change":"2024-09-16T07:33:32.049217+0000","last_active":"2024-09-16T07:33:33.021843+0000","last_peered":"2024-09-16T07:33:33.021843+0000","last_clean":"2024-09-16T07:33:33.021843+0000","last_became_active":"2024-09-16T07:33:32.048953+0000","last_became_peered":"2024-09-16T07:33:32.048953+0000","last_unstale":"2024-09-16T07:33:33.021843+0000","last_undegraded":"2024-09-16T07:33:33.021843+0000","last_fullsized":"2024-09-16T07:33:33.021843+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357715+0000","last_change":"2024-09-16T07:33:32.054961+0000","last_active":"2024-09-16T07:33:33.357715+0000","last_peered":"2024-09-16T07:33:33.357715+0000","last_clean":"2024-09-16T07:33:33.357715+0000","last_became_active":"2024-09-16T07:33:32.054463+0000","last_became_peered":"2024-09-16T07:33:32.054463+0000","last_unstale":"2024-09-16T07:33:33.357715+0000","last_undegraded":"2024-09-16T07:33:33.357715+0000","last_fullsized":"2024-09-16T07:33:33.357715+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:57.790799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358257+0000","last_change":"2024-09-16T07:33:32.040365+0000","last_active":"2024-09-16T07:33:33.358257+0000","last_peered":"2024-09-16T07:33:33.358257+0000","last_clean":"2024-09-16T07:33:33.358257+0000","last_became_active":"2024-09-16T07:33:32.037953+0000","last_became_peered":"2024-09-16T07:33:32.037953+0000","last_unstale":"2024-09-16T07:33:33.358257+0000","last_undegraded":"2024-09-16T07:33:33.358257+0000","last_fullsized":"2024-09-16T07:33:33.358257+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:32.112813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357230+0000","last_change":"2024-09-16T07:33:32.056887+0000","last_active":"2024-09-16T07:33:33.357230+0000","last_peered":"2024-09-16T07:33:33.357230+0000","last_clean":"2024-09-16T07:33:33.357230+0000","last_became_active":"2024-09-16T07:33:32.056276+0000","last_became_peered":"2024-09-16T07:33:32.056276+0000","last_unstale":"2024-09-16T07:33:33.357230+0000","last_undegraded":"2024-09-16T07:33:33.357230+0000","last_fullsized":"2024-09-16T07:33:33.357230+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:42.401994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023495+0000","last_change":"2024-09-16T07:33:32.054384+0000","last_active":"2024-09-16T07:33:33.023495+0000","last_peered":"2024-09-16T07:33:33.023495+0000","last_clean":"2024-09-16T07:33:33.023495+0000","last_became_active":"2024-09-16T07:33:32.051888+0000","last_became_peered":"2024-09-16T07:33:32.051888+0000","last_unstale":"2024-09-16T07:33:33.023495+0000","last_undegraded":"2024-09-16T07:33:33.023495+0000","last_fullsized":"2024-09-16T07:33:33.023495+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:19.796812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023506+0000","last_change":"2024-09-16T07:33:32.040802+0000","last_active":"2024-09-16T07:33:33.023506+0000","last_peered":"2024-09-16T07:33:33.023506+0000","last_clean":"2024-09-16T07:33:33.023506+0000","last_became_active":"2024-09-16T07:33:32.040510+0000","last_became_peered":"2024-09-16T07:33:32.040510+0000","last_unstale":"2024-09-16T07:33:33.023506+0000","last_undegraded":"2024-09-16T07:33:33.023506+0000","last_fullsized":"2024-09-16T07:33:33.023506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:53:27.647859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357155+0000","last_change":"2024-09-16T07:33:32.039107+0000","last_active":"2024-09-16T07:33:33.357155+0000","last_peered":"2024-09-16T07:33:33.357155+0000","last_clean":"2024-09-16T07:33:33.357155+0000","last_became_active":"2024-09-16T07:33:32.038828+0000","last_became_peered":"2024-09-16T07:33:32.038828+0000","last_unstale":"2024-09-16T07:33:33.357155+0000","last_undegraded":"2024-09-16T07:33:33.357155+0000","last_fullsized":"2024-09-16T07:33:33.357155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:15.415755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025462+0000","last_change":"2024-09-16T07:33:32.053433+0000","last_active":"2024-09-16T07:33:33.025462+0000","last_peered":"2024-09-16T07:33:33.025462+0000","last_clean":"2024-09-16T07:33:33.025462+0000","last_became_active":"2024-09-16T07:33:32.052843+0000","last_became_peered":"2024-09-16T07:33:32.052843+0000","last_unstale":"2024-09-16T07:33:33.025462+0000","last_undegraded":"2024-09-16T07:33:33.025462+0000","last_fullsized":"2024-09-16T07:33:33.025462+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:06:13.265002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025375+0000","last_change":"2024-09-16T07:33:32.047389+0000","last_active":"2024-09-16T07:33:33.025375+0000","last_peered":"2024-09-16T07:33:33.025375+0000","last_clean":"2024-09-16T07:33:33.025375+0000","last_became_active":"2024-09-16T07:33:32.047043+0000","last_became_peered":"2024-09-16T07:33:32.047043+0000","last_unstale":"2024-09-16T07:33:33.025375+0000","last_undegraded":"2024-09-16T07:33:33.025375+0000","last_fullsized":"2024-09-16T07:33:33.025375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024761+0000","last_change":"2024-09-16T07:33:32.040875+0000","last_active":"2024-09-16T07:33:33.024761+0000","last_peered":"2024-09-16T07:33:33.024761+0000","last_clean":"2024-09-16T07:33:33.024761+0000","last_became_active":"2024-09-16T07:33:32.040636+0000","last_became_peered":"2024-09-16T07:33:32.040636+0000","last_unstale":"2024-09-16T07:33:33.024761+0000","last_undegraded":"2024-09-16T07:33:33.024761+0000","last_fullsized":"2024-09-16T07:33:33.024761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:06.934545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358353+0000","last_change":"2024-09-16T07:33:32.053936+0000","last_active":"2024-09-16T07:33:33.358353+0000","last_peered":"2024-09-16T07:33:33.358353+0000","last_clean":"2024-09-16T07:33:33.358353+0000","last_became_active":"2024-09-16T07:33:32.053546+0000","last_became_peered":"2024-09-16T07:33:32.053546+0000","last_unstale":"2024-09-16T07:33:33.358353+0000","last_undegraded":"2024-09-16T07:33:33.358353+0000","last_fullsized":"2024-09-16T07:33:33.358353+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:24.364353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357403+0000","last_change":"2024-09-16T07:33:32.036138+0000","last_active":"2024-09-16T07:33:33.357403+0000","last_peered":"2024-09-16T07:33:33.357403+0000","last_clean":"2024-09-16T07:33:33.357403+0000","last_became_active":"2024-09-16T07:33:32.035544+0000","last_became_peered":"2024-09-16T07:33:32.035544+0000","last_unstale":"2024-09-16T07:33:33.357403+0000","last_undegraded":"2024-09-16T07:33:33.357403+0000","last_fullsized":"2024-09-16T07:33:33.357403+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:10:21.071908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357641+0000","last_change":"2024-09-16T07:33:32.040994+0000","last_active":"2024-09-16T07:33:33.357641+0000","last_peered":"2024-09-16T07:33:33.357641+0000","last_clean":"2024-09-16T07:33:33.357641+0000","last_became_active":"2024-09-16T07:33:32.040735+0000","last_became_peered":"2024-09-16T07:33:32.040735+0000","last_unstale":"2024-09-16T07:33:33.357641+0000","last_undegraded":"2024-09-16T07:33:33.357641+0000","last_fullsized":"2024-09-16T07:33:33.357641+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:21.796391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023549+0000","last_change":"2024-09-16T07:33:32.055235+0000","last_active":"2024-09-16T07:33:33.023549+0000","last_peered":"2024-09-16T07:33:33.023549+0000","last_clean":"2024-09-16T07:33:33.023549+0000","last_became_active":"2024-09-16T07:33:32.052675+0000","last_became_peered":"2024-09-16T07:33:32.052675+0000","last_unstale":"2024-09-16T07:33:33.023549+0000","last_undegraded":"2024-09-16T07:33:33.023549+0000","last_fullsized":"2024-09-16T07:33:33.023549+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:31.315404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357746+0000","last_change":"2024-09-16T07:33:32.050486+0000","last_active":"2024-09-16T07:33:33.357746+0000","last_peered":"2024-09-16T07:33:33.357746+0000","last_clean":"2024-09-16T07:33:33.357746+0000","last_became_active":"2024-09-16T07:33:32.050108+0000","last_became_peered":"2024-09-16T07:33:32.050108+0000","last_unstale":"2024-09-16T07:33:33.357746+0000","last_undegraded":"2024-09-16T07:33:33.357746+0000","last_fullsized":"2024-09-16T07:33:33.357746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:54.780782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021955+0000","last_change":"2024-09-16T07:33:32.028117+0000","last_active":"2024-09-16T07:33:33.021955+0000","last_peered":"2024-09-16T07:33:33.021955+0000","last_clean":"2024-09-16T07:33:33.021955+0000","last_became_active":"2024-09-16T07:33:32.027826+0000","last_became_peered":"2024-09-16T07:33:32.027826+0000","last_unstale":"2024-09-16T07:33:33.021955+0000","last_undegraded":"2024-09-16T07:33:33.021955+0000","last_fullsized":"2024-09-16T07:33:33.021955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:35.757466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357604+0000","last_change":"2024-09-16T07:33:32.049150+0000","last_active":"2024-09-16T07:33:33.357604+0000","last_peered":"2024-09-16T07:33:33.357604+0000","last_clean":"2024-09-16T07:33:33.357604+0000","last_became_active":"2024-09-16T07:33:32.048893+0000","last_became_peered":"2024-09-16T07:33:32.048893+0000","last_unstale":"2024-09-16T07:33:33.357604+0000","last_undegraded":"2024-09-16T07:33:33.357604+0000","last_fullsized":"2024-09-16T07:33:33.357604+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:21.184947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358162+0000","last_change":"2024-09-16T07:33:32.041462+0000","last_active":"2024-09-16T07:33:33.358162+0000","last_peered":"2024-09-16T07:33:33.358162+0000","last_clean":"2024-09-16T07:33:33.358162+0000","last_became_active":"2024-09-16T07:33:32.041198+0000","last_became_peered":"2024-09-16T07:33:32.041198+0000","last_unstale":"2024-09-16T07:33:33.358162+0000","last_undegraded":"2024-09-16T07:33:33.358162+0000","last_fullsized":"2024-09-16T07:33:33.358162+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358295+0000","last_change":"2024-09-16T07:33:32.037402+0000","last_active":"2024-09-16T07:33:33.358295+0000","last_peered":"2024-09-16T07:33:33.358295+0000","last_clean":"2024-09-16T07:33:33.358295+0000","last_became_active":"2024-09-16T07:33:32.037131+0000","last_became_peered":"2024-09-16T07:33:32.037131+0000","last_unstale":"2024-09-16T07:33:33.358295+0000","last_undegraded":"2024-09-16T07:33:33.358295+0000","last_fullsized":"2024-09-16T07:33:33.358295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:02.945216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357279+0000","last_change":"2024-09-16T07:33:32.043318+0000","last_active":"2024-09-16T07:33:33.357279+0000","last_peered":"2024-09-16T07:33:33.357279+0000","last_clean":"2024-09-16T07:33:33.357279+0000","last_became_active":"2024-09-16T07:33:32.043039+0000","last_became_peered":"2024-09-16T07:33:32.043039+0000","last_unstale":"2024-09-16T07:33:33.357279+0000","last_undegraded":"2024-09-16T07:33:33.357279+0000","last_fullsized":"2024-09-16T07:33:33.357279+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:54.747318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357439+0000","last_change":"2024-09-16T07:33:32.056817+0000","last_active":"2024-09-16T07:33:33.357439+0000","last_peered":"2024-09-16T07:33:33.357439+0000","last_clean":"2024-09-16T07:33:33.357439+0000","last_became_active":"2024-09-16T07:33:32.056535+0000","last_became_peered":"2024-09-16T07:33:32.056535+0000","last_unstale":"2024-09-16T07:33:33.357439+0000","last_undegraded":"2024-09-16T07:33:33.357439+0000","last_fullsized":"2024-09-16T07:33:33.357439+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:38.193933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357145+0000","last_change":"2024-09-16T07:33:32.032388+0000","last_active":"2024-09-16T07:33:33.357145+0000","last_peered":"2024-09-16T07:33:33.357145+0000","last_clean":"2024-09-16T07:33:33.357145+0000","last_became_active":"2024-09-16T07:33:32.032016+0000","last_became_peered":"2024-09-16T07:33:32.032016+0000","last_unstale":"2024-09-16T07:33:33.357145+0000","last_undegraded":"2024-09-16T07:33:33.357145+0000","last_fullsized":"2024-09-16T07:33:33.357145+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:53.061371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023806+0000","last_change":"2024-09-16T07:33:32.036224+0000","last_active":"2024-09-16T07:33:33.023806+0000","last_peered":"2024-09-16T07:33:33.023806+0000","last_clean":"2024-09-16T07:33:33.023806+0000","last_became_active":"2024-09-16T07:33:32.035020+0000","last_became_peered":"2024-09-16T07:33:32.035020+0000","last_unstale":"2024-09-16T07:33:33.023806+0000","last_undegraded":"2024-09-16T07:33:33.023806+0000","last_fullsized":"2024-09-16T07:33:33.023806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:00.997013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025689+0000","last_change":"2024-09-16T07:33:32.046309+0000","last_active":"2024-09-16T07:33:33.025689+0000","last_peered":"2024-09-16T07:33:33.025689+0000","last_clean":"2024-09-16T07:33:33.025689+0000","last_became_active":"2024-09-16T07:33:32.045999+0000","last_became_peered":"2024-09-16T07:33:32.045999+0000","last_unstale":"2024-09-16T07:33:33.025689+0000","last_undegraded":"2024-09-16T07:33:33.025689+0000","last_fullsized":"2024-09-16T07:33:33.025689+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:09.445884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357884+0000","last_change":"2024-09-16T07:33:31.045574+0000","last_active":"2024-09-16T07:33:33.357884+0000","last_peered":"2024-09-16T07:33:33.357884+0000","last_clean":"2024-09-16T07:33:33.357884+0000","last_became_active":"2024-09-16T07:33:31.045014+0000","last_became_peered":"2024-09-16T07:33:31.045014+0000","last_unstale":"2024-09-16T07:33:33.357884+0000","last_undegraded":"2024-09-16T07:33:33.357884+0000","last_fullsized":"2024-09-16T07:33:33.357884+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.581864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357842+0000","last_change":"2024-09-16T07:33:32.046389+0000","last_active":"2024-09-16T07:33:33.357842+0000","last_peered":"2024-09-16T07:33:33.357842+0000","last_clean":"2024-09-16T07:33:33.357842+0000","last_became_active":"2024-09-16T07:33:32.046041+0000","last_became_peered":"2024-09-16T07:33:32.046041+0000","last_unstale":"2024-09-16T07:33:33.357842+0000","last_undegraded":"2024-09-16T07:33:33.357842+0000","last_fullsized":"2024-09-16T07:33:33.357842+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:27.453530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357662+0000","last_change":"2024-09-16T07:33:31.041205+0000","last_active":"2024-09-16T07:33:33.357662+0000","last_peered":"2024-09-16T07:33:33.357662+0000","last_clean":"2024-09-16T07:33:33.357662+0000","last_became_active":"2024-09-16T07:33:31.040923+0000","last_became_peered":"2024-09-16T07:33:31.040923+0000","last_unstale":"2024-09-16T07:33:33.357662+0000","last_undegraded":"2024-09-16T07:33:33.357662+0000","last_fullsized":"2024-09-16T07:33:33.357662+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:56.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024526+0000","last_change":"2024-09-16T07:33:32.039150+0000","last_active":"2024-09-16T07:33:33.024526+0000","last_peered":"2024-09-16T07:33:33.024526+0000","last_clean":"2024-09-16T07:33:33.024526+0000","last_became_active":"2024-09-16T07:33:32.038837+0000","last_became_peered":"2024-09-16T07:33:32.038837+0000","last_unstale":"2024-09-16T07:33:33.024526+0000","last_undegraded":"2024-09-16T07:33:33.024526+0000","last_fullsized":"2024-09-16T07:33:33.024526+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:52.257876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357916+0000","last_change":"2024-09-16T07:33:31.045183+0000","last_active":"2024-09-16T07:33:33.357916+0000","last_peered":"2024-09-16T07:33:33.357916+0000","last_clean":"2024-09-16T07:33:33.357916+0000","last_became_active":"2024-09-16T07:33:31.044253+0000","last_became_peered":"2024-09-16T07:33:31.044253+0000","last_unstale":"2024-09-16T07:33:33.357916+0000","last_undegraded":"2024-09-16T07:33:33.357916+0000","last_fullsized":"2024-09-16T07:33:33.357916+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:48.114900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021838+0000","last_change":"2024-09-16T07:33:32.043094+0000","last_active":"2024-09-16T07:33:33.021838+0000","last_peered":"2024-09-16T07:33:33.021838+0000","last_clean":"2024-09-16T07:33:33.021838+0000","last_became_active":"2024-09-16T07:33:32.042832+0000","last_became_peered":"2024-09-16T07:33:32.042832+0000","last_unstale":"2024-09-16T07:33:33.021838+0000","last_undegraded":"2024-09-16T07:33:33.021838+0000","last_fullsized":"2024-09-16T07:33:33.021838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:24.827070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024550+0000","last_change":"2024-09-16T07:33:31.042458+0000","last_active":"2024-09-16T07:33:33.024550+0000","last_peered":"2024-09-16T07:33:33.024550+0000","last_clean":"2024-09-16T07:33:33.024550+0000","last_became_active":"2024-09-16T07:33:31.042226+0000","last_became_peered":"2024-09-16T07:33:31.042226+0000","last_unstale":"2024-09-16T07:33:33.024550+0000","last_undegraded":"2024-09-16T07:33:33.024550+0000","last_fullsized":"2024-09-16T07:33:33.024550+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:20.182053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357806+0000","last_change":"2024-09-16T07:33:32.039748+0000","last_active":"2024-09-16T07:33:33.357806+0000","last_peered":"2024-09-16T07:33:33.357806+0000","last_clean":"2024-09-16T07:33:33.357806+0000","last_became_active":"2024-09-16T07:33:32.038397+0000","last_became_peered":"2024-09-16T07:33:32.038397+0000","last_unstale":"2024-09-16T07:33:33.357806+0000","last_undegraded":"2024-09-16T07:33:33.357806+0000","last_fullsized":"2024-09-16T07:33:33.357806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:15.732874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023864+0000","last_change":"2024-09-16T07:33:31.033141+0000","last_active":"2024-09-16T07:33:33.023864+0000","last_peered":"2024-09-16T07:33:33.023864+0000","last_clean":"2024-09-16T07:33:33.023864+0000","last_became_active":"2024-09-16T07:33:31.032570+0000","last_became_peered":"2024-09-16T07:33:31.032570+0000","last_unstale":"2024-09-16T07:33:33.023864+0000","last_undegraded":"2024-09-16T07:33:33.023864+0000","last_fullsized":"2024-09-16T07:33:33.023864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:26.450637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357619+0000","last_change":"2024-09-16T07:33:32.044965+0000","last_active":"2024-09-16T07:33:33.357619+0000","last_peered":"2024-09-16T07:33:33.357619+0000","last_clean":"2024-09-16T07:33:33.357619+0000","last_became_active":"2024-09-16T07:33:32.044705+0000","last_became_peered":"2024-09-16T07:33:32.044705+0000","last_unstale":"2024-09-16T07:33:33.357619+0000","last_undegraded":"2024-09-16T07:33:33.357619+0000","last_fullsized":"2024-09-16T07:33:33.357619+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:50.022797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357790+0000","last_change":"2024-09-16T07:33:31.035805+0000","last_active":"2024-09-16T07:33:33.357790+0000","last_peered":"2024-09-16T07:33:33.357790+0000","last_clean":"2024-09-16T07:33:33.357790+0000","last_became_active":"2024-09-16T07:33:31.035545+0000","last_became_peered":"2024-09-16T07:33:31.035545+0000","last_unstale":"2024-09-16T07:33:33.357790+0000","last_undegraded":"2024-09-16T07:33:33.357790+0000","last_fullsized":"2024-09-16T07:33:33.357790+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:40.945804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.356916+0000","last_change":"2024-09-16T07:33:32.051200+0000","last_active":"2024-09-16T07:33:33.356916+0000","last_peered":"2024-09-16T07:33:33.356916+0000","last_clean":"2024-09-16T07:33:33.356916+0000","last_became_active":"2024-09-16T07:33:32.047036+0000","last_became_peered":"2024-09-16T07:33:32.047036+0000","last_unstale":"2024-09-16T07:33:33.356916+0000","last_undegraded":"2024-09-16T07:33:33.356916+0000","last_fullsized":"2024-09-16T07:33:33.356916+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:15.416732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357424+0000","last_change":"2024-09-16T07:33:31.034485+0000","last_active":"2024-09-16T07:33:33.357424+0000","last_peered":"2024-09-16T07:33:33.357424+0000","last_clean":"2024-09-16T07:33:33.357424+0000","last_became_active":"2024-09-16T07:33:31.034236+0000","last_became_peered":"2024-09-16T07:33:31.034236+0000","last_unstale":"2024-09-16T07:33:33.357424+0000","last_undegraded":"2024-09-16T07:33:33.357424+0000","last_fullsized":"2024-09-16T07:33:33.357424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:52.133642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357028+0000","last_change":"2024-09-16T07:33:32.039686+0000","last_active":"2024-09-16T07:33:33.357028+0000","last_peered":"2024-09-16T07:33:33.357028+0000","last_clean":"2024-09-16T07:33:33.357028+0000","last_became_active":"2024-09-16T07:33:32.038343+0000","last_became_peered":"2024-09-16T07:33:32.038343+0000","last_unstale":"2024-09-16T07:33:33.357028+0000","last_undegraded":"2024-09-16T07:33:33.357028+0000","last_fullsized":"2024-09-16T07:33:33.357028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:13:00.455428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024348+0000","last_change":"2024-09-16T07:33:31.039439+0000","last_active":"2024-09-16T07:33:33.024348+0000","last_peered":"2024-09-16T07:33:33.024348+0000","last_clean":"2024-09-16T07:33:33.024348+0000","last_became_active":"2024-09-16T07:33:31.039029+0000","last_became_peered":"2024-09-16T07:33:31.039029+0000","last_unstale":"2024-09-16T07:33:33.024348+0000","last_undegraded":"2024-09-16T07:33:33.024348+0000","last_fullsized":"2024-09-16T07:33:33.024348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:21.209706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025089+0000","last_change":"2024-09-16T07:33:32.045355+0000","last_active":"2024-09-16T07:33:33.025089+0000","last_peered":"2024-09-16T07:33:33.025089+0000","last_clean":"2024-09-16T07:33:33.025089+0000","last_became_active":"2024-09-16T07:33:32.045070+0000","last_became_peered":"2024-09-16T07:33:32.045070+0000","last_unstale":"2024-09-16T07:33:33.025089+0000","last_undegraded":"2024-09-16T07:33:33.025089+0000","last_fullsized":"2024-09-16T07:33:33.025089+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:15.919118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358191+0000","last_change":"2024-09-16T07:33:31.039400+0000","last_active":"2024-09-16T07:33:33.358191+0000","last_peered":"2024-09-16T07:33:33.358191+0000","last_clean":"2024-09-16T07:33:33.358191+0000","last_became_active":"2024-09-16T07:33:31.039119+0000","last_became_peered":"2024-09-16T07:33:31.039119+0000","last_unstale":"2024-09-16T07:33:33.358191+0000","last_undegraded":"2024-09-16T07:33:33.358191+0000","last_fullsized":"2024-09-16T07:33:33.358191+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:56:40.448104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358517+0000","last_change":"2024-09-16T07:33:32.035552+0000","last_active":"2024-09-16T07:33:33.358517+0000","last_peered":"2024-09-16T07:33:33.358517+0000","last_clean":"2024-09-16T07:33:33.358517+0000","last_became_active":"2024-09-16T07:33:32.035316+0000","last_became_peered":"2024-09-16T07:33:32.035316+0000","last_unstale":"2024-09-16T07:33:33.358517+0000","last_undegraded":"2024-09-16T07:33:33.358517+0000","last_fullsized":"2024-09-16T07:33:33.358517+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:06.190971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025197+0000","last_change":"2024-09-16T07:33:31.037265+0000","last_active":"2024-09-16T07:33:33.025197+0000","last_peered":"2024-09-16T07:33:33.025197+0000","last_clean":"2024-09-16T07:33:33.025197+0000","last_became_active":"2024-09-16T07:33:31.036981+0000","last_became_peered":"2024-09-16T07:33:31.036981+0000","last_unstale":"2024-09-16T07:33:33.025197+0000","last_undegraded":"2024-09-16T07:33:33.025197+0000","last_fullsized":"2024-09-16T07:33:33.025197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:08.818149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357328+0000","last_change":"2024-09-16T07:33:32.042815+0000","last_active":"2024-09-16T07:33:33.357328+0000","last_peered":"2024-09-16T07:33:33.357328+0000","last_clean":"2024-09-16T07:33:33.357328+0000","last_became_active":"2024-09-16T07:33:32.042470+0000","last_became_peered":"2024-09-16T07:33:32.042470+0000","last_unstale":"2024-09-16T07:33:33.357328+0000","last_undegraded":"2024-09-16T07:33:33.357328+0000","last_fullsized":"2024-09-16T07:33:33.357328+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:39.595734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357447+0000","last_change":"2024-09-16T07:33:31.032993+0000","last_active":"2024-09-16T07:33:33.357447+0000","last_peered":"2024-09-16T07:33:33.357447+0000","last_clean":"2024-09-16T07:33:33.357447+0000","last_became_active":"2024-09-16T07:33:31.032741+0000","last_became_peered":"2024-09-16T07:33:31.032741+0000","last_unstale":"2024-09-16T07:33:33.357447+0000","last_undegraded":"2024-09-16T07:33:33.357447+0000","last_fullsized":"2024-09-16T07:33:33.357447+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:03.607602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357971+0000","last_change":"2024-09-16T07:33:32.035426+0000","last_active":"2024-09-16T07:33:33.357971+0000","last_peered":"2024-09-16T07:33:33.357971+0000","last_clean":"2024-09-16T07:33:33.357971+0000","last_became_active":"2024-09-16T07:33:32.034929+0000","last_became_peered":"2024-09-16T07:33:32.034929+0000","last_unstale":"2024-09-16T07:33:33.357971+0000","last_undegraded":"2024-09-16T07:33:33.357971+0000","last_fullsized":"2024-09-16T07:33:33.357971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:57:07.162730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":21,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358359+0000","last_change":"2024-09-16T07:33:31.041833+0000","last_active":"2024-09-16T07:33:33.358359+0000","last_peered":"2024-09-16T07:33:33.358359+0000","last_clean":"2024-09-16T07:33:33.358359+0000","last_became_active":"2024-09-16T07:33:31.041034+0000","last_became_peered":"2024-09-16T07:33:31.041034+0000","last_unstale":"2024-09-16T07:33:33.358359+0000","last_undegraded":"2024-09-16T07:33:33.358359+0000","last_fullsized":"2024-09-16T07:33:33.358359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:16:25.618706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024155+0000","last_change":"2024-09-16T07:33:32.042155+0000","last_active":"2024-09-16T07:33:33.024155+0000","last_peered":"2024-09-16T07:33:33.024155+0000","last_clean":"2024-09-16T07:33:33.024155+0000","last_became_active":"2024-09-16T07:33:32.041955+0000","last_became_peered":"2024-09-16T07:33:32.041955+0000","last_unstale":"2024-09-16T07:33:33.024155+0000","last_undegraded":"2024-09-16T07:33:33.024155+0000","last_fullsized":"2024-09-16T07:33:33.024155+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:45.546271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025124+0000","last_change":"2024-09-16T07:33:31.036453+0000","last_active":"2024-09-16T07:33:33.025124+0000","last_peered":"2024-09-16T07:33:33.025124+0000","last_clean":"2024-09-16T07:33:33.025124+0000","last_became_active":"2024-09-16T07:33:31.034972+0000","last_became_peered":"2024-09-16T07:33:31.034972+0000","last_unstale":"2024-09-16T07:33:33.025124+0000","last_undegraded":"2024-09-16T07:33:33.025124+0000","last_fullsized":"2024-09-16T07:33:33.025124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:49.504210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357444+0000","last_change":"2024-09-16T07:33:32.036463+0000","last_active":"2024-09-16T07:33:33.357444+0000","last_peered":"2024-09-16T07:33:33.357444+0000","last_clean":"2024-09-16T07:33:33.357444+0000","last_became_active":"2024-09-16T07:33:32.035843+0000","last_became_peered":"2024-09-16T07:33:32.035843+0000","last_unstale":"2024-09-16T07:33:33.357444+0000","last_undegraded":"2024-09-16T07:33:33.357444+0000","last_fullsized":"2024-09-16T07:33:33.357444+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:44.761435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024241+0000","last_change":"2024-09-16T07:33:31.042034+0000","last_active":"2024-09-16T07:33:33.024241+0000","last_peered":"2024-09-16T07:33:33.024241+0000","last_clean":"2024-09-16T07:33:33.024241+0000","last_became_active":"2024-09-16T07:33:31.041059+0000","last_became_peered":"2024-09-16T07:33:31.041059+0000","last_unstale":"2024-09-16T07:33:33.024241+0000","last_undegraded":"2024-09-16T07:33:33.024241+0000","last_fullsized":"2024-09-16T07:33:33.024241+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:59.775582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357991+0000","last_change":"2024-09-16T07:33:32.046065+0000","last_active":"2024-09-16T07:33:33.357991+0000","last_peered":"2024-09-16T07:33:33.357991+0000","last_clean":"2024-09-16T07:33:33.357991+0000","last_became_active":"2024-09-16T07:33:32.045750+0000","last_became_peered":"2024-09-16T07:33:32.045750+0000","last_unstale":"2024-09-16T07:33:33.357991+0000","last_undegraded":"2024-09-16T07:33:33.357991+0000","last_fullsized":"2024-09-16T07:33:33.357991+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:24:26.414491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357741+0000","last_change":"2024-09-16T07:33:31.031995+0000","last_active":"2024-09-16T07:33:33.357741+0000","last_peered":"2024-09-16T07:33:33.357741+0000","last_clean":"2024-09-16T07:33:33.357741+0000","last_became_active":"2024-09-16T07:33:31.031131+0000","last_became_peered":"2024-09-16T07:33:31.031131+0000","last_unstale":"2024-09-16T07:33:33.357741+0000","last_undegraded":"2024-09-16T07:33:33.357741+0000","last_fullsized":"2024-09-16T07:33:33.357741+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:28.800839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358051+0000","last_change":"2024-09-16T07:33:32.037836+0000","last_active":"2024-09-16T07:33:33.358051+0000","last_peered":"2024-09-16T07:33:33.358051+0000","last_clean":"2024-09-16T07:33:33.358051+0000","last_became_active":"2024-09-16T07:33:32.037536+0000","last_became_peered":"2024-09-16T07:33:32.037536+0000","last_unstale":"2024-09-16T07:33:33.358051+0000","last_undegraded":"2024-09-16T07:33:33.358051+0000","last_fullsized":"2024-09-16T07:33:33.358051+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:10:04.873568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023950+0000","last_change":"2024-09-16T07:33:31.030808+0000","last_active":"2024-09-16T07:33:33.023950+0000","last_peered":"2024-09-16T07:33:33.023950+0000","last_clean":"2024-09-16T07:33:33.023950+0000","last_became_active":"2024-09-16T07:33:31.028772+0000","last_became_peered":"2024-09-16T07:33:31.028772+0000","last_unstale":"2024-09-16T07:33:33.023950+0000","last_undegraded":"2024-09-16T07:33:33.023950+0000","last_fullsized":"2024-09-16T07:33:33.023950+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:29:53.744425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358706+0000","last_change":"2024-09-16T07:33:32.045394+0000","last_active":"2024-09-16T07:33:33.358706+0000","last_peered":"2024-09-16T07:33:33.358706+0000","last_clean":"2024-09-16T07:33:33.358706+0000","last_became_active":"2024-09-16T07:33:32.045106+0000","last_became_peered":"2024-09-16T07:33:32.045106+0000","last_unstale":"2024-09-16T07:33:33.358706+0000","last_undegraded":"2024-09-16T07:33:33.358706+0000","last_fullsized":"2024-09-16T07:33:33.358706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.537036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358438+0000","last_change":"2024-09-16T07:33:31.041342+0000","last_active":"2024-09-16T07:33:33.358438+0000","last_peered":"2024-09-16T07:33:33.358438+0000","last_clean":"2024-09-16T07:33:33.358438+0000","last_became_active":"2024-09-16T07:33:31.041056+0000","last_became_peered":"2024-09-16T07:33:31.041056+0000","last_unstale":"2024-09-16T07:33:33.358438+0000","last_undegraded":"2024-09-16T07:33:33.358438+0000","last_fullsized":"2024-09-16T07:33:33.358438+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:39.097159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358460+0000","last_change":"2024-09-16T07:33:32.044491+0000","last_active":"2024-09-16T07:33:33.358460+0000","last_peered":"2024-09-16T07:33:33.358460+0000","last_clean":"2024-09-16T07:33:33.358460+0000","last_became_active":"2024-09-16T07:33:32.044228+0000","last_became_peered":"2024-09-16T07:33:32.044228+0000","last_unstale":"2024-09-16T07:33:33.358460+0000","last_undegraded":"2024-09-16T07:33:33.358460+0000","last_fullsized":"2024-09-16T07:33:33.358460+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:14.023175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357548+0000","last_change":"2024-09-16T07:33:31.034256+0000","last_active":"2024-09-16T07:33:33.357548+0000","last_peered":"2024-09-16T07:33:33.357548+0000","last_clean":"2024-09-16T07:33:33.357548+0000","last_became_active":"2024-09-16T07:33:31.033825+0000","last_became_peered":"2024-09-16T07:33:31.033825+0000","last_unstale":"2024-09-16T07:33:33.357548+0000","last_undegraded":"2024-09-16T07:33:33.357548+0000","last_fullsized":"2024-09-16T07:33:33.357548+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:44.857125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358285+0000","last_change":"2024-09-16T07:33:32.041248+0000","last_active":"2024-09-16T07:33:33.358285+0000","last_peered":"2024-09-16T07:33:33.358285+0000","last_clean":"2024-09-16T07:33:33.358285+0000","last_became_active":"2024-09-16T07:33:32.040998+0000","last_became_peered":"2024-09-16T07:33:32.040998+0000","last_unstale":"2024-09-16T07:33:33.358285+0000","last_undegraded":"2024-09-16T07:33:33.358285+0000","last_fullsized":"2024-09-16T07:33:33.358285+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:06.696198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025145+0000","last_change":"2024-09-16T07:33:31.035273+0000","last_active":"2024-09-16T07:33:33.025145+0000","last_peered":"2024-09-16T07:33:33.025145+0000","last_clean":"2024-09-16T07:33:33.025145+0000","last_became_active":"2024-09-16T07:33:31.034957+0000","last_became_peered":"2024-09-16T07:33:31.034957+0000","last_unstale":"2024-09-16T07:33:33.025145+0000","last_undegraded":"2024-09-16T07:33:33.025145+0000","last_fullsized":"2024-09-16T07:33:33.025145+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:17.685015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025196+0000","last_change":"2024-09-16T07:33:32.033426+0000","last_active":"2024-09-16T07:33:33.025196+0000","last_peered":"2024-09-16T07:33:33.025196+0000","last_clean":"2024-09-16T07:33:33.025196+0000","last_became_active":"2024-09-16T07:33:32.032989+0000","last_became_peered":"2024-09-16T07:33:32.032989+0000","last_unstale":"2024-09-16T07:33:33.025196+0000","last_undegraded":"2024-09-16T07:33:33.025196+0000","last_fullsized":"2024-09-16T07:33:33.025196+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:06.277092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.023949+0000","last_change":"2024-09-16T07:33:31.038931+0000","last_active":"2024-09-16T07:33:33.023949+0000","last_peered":"2024-09-16T07:33:33.023949+0000","last_clean":"2024-09-16T07:33:33.023949+0000","last_became_active":"2024-09-16T07:33:31.038639+0000","last_became_peered":"2024-09-16T07:33:31.038639+0000","last_unstale":"2024-09-16T07:33:33.023949+0000","last_undegraded":"2024-09-16T07:33:33.023949+0000","last_fullsized":"2024-09-16T07:33:33.023949+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:37.191040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025027+0000","last_change":"2024-09-16T07:33:32.050892+0000","last_active":"2024-09-16T07:33:33.025027+0000","last_peered":"2024-09-16T07:33:33.025027+0000","last_clean":"2024-09-16T07:33:33.025027+0000","last_became_active":"2024-09-16T07:33:32.050449+0000","last_became_peered":"2024-09-16T07:33:32.050449+0000","last_unstale":"2024-09-16T07:33:33.025027+0000","last_undegraded":"2024-09-16T07:33:33.025027+0000","last_fullsized":"2024-09-16T07:33:33.025027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:53.956082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358694+0000","last_change":"2024-09-16T07:33:31.035777+0000","last_active":"2024-09-16T07:33:33.358694+0000","last_peered":"2024-09-16T07:33:33.358694+0000","last_clean":"2024-09-16T07:33:33.358694+0000","last_became_active":"2024-09-16T07:33:31.034448+0000","last_became_peered":"2024-09-16T07:33:31.034448+0000","last_unstale":"2024-09-16T07:33:33.358694+0000","last_undegraded":"2024-09-16T07:33:33.358694+0000","last_fullsized":"2024-09-16T07:33:33.358694+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:45.713854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025244+0000","last_change":"2024-09-16T07:33:32.052617+0000","last_active":"2024-09-16T07:33:33.025244+0000","last_peered":"2024-09-16T07:33:33.025244+0000","last_clean":"2024-09-16T07:33:33.025244+0000","last_became_active":"2024-09-16T07:33:32.052089+0000","last_became_peered":"2024-09-16T07:33:32.052089+0000","last_unstale":"2024-09-16T07:33:33.025244+0000","last_undegraded":"2024-09-16T07:33:33.025244+0000","last_fullsized":"2024-09-16T07:33:33.025244+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:49.702884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.021703+0000","last_change":"2024-09-16T07:33:31.036417+0000","last_active":"2024-09-16T07:33:33.021703+0000","last_peered":"2024-09-16T07:33:33.021703+0000","last_clean":"2024-09-16T07:33:33.021703+0000","last_became_active":"2024-09-16T07:33:31.036118+0000","last_became_peered":"2024-09-16T07:33:31.036118+0000","last_unstale":"2024-09-16T07:33:33.021703+0000","last_undegraded":"2024-09-16T07:33:33.021703+0000","last_fullsized":"2024-09-16T07:33:33.021703+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:39:50.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358156+0000","last_change":"2024-09-16T07:33:32.039339+0000","last_active":"2024-09-16T07:33:33.358156+0000","last_peered":"2024-09-16T07:33:33.358156+0000","last_clean":"2024-09-16T07:33:33.358156+0000","last_became_active":"2024-09-16T07:33:32.037162+0000","last_became_peered":"2024-09-16T07:33:32.037162+0000","last_unstale":"2024-09-16T07:33:33.358156+0000","last_undegraded":"2024-09-16T07:33:33.358156+0000","last_fullsized":"2024-09-16T07:33:33.358156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:42.850047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.356816+0000","last_change":"2024-09-16T07:33:31.042541+0000","last_active":"2024-09-16T07:33:33.356816+0000","last_peered":"2024-09-16T07:33:33.356816+0000","last_clean":"2024-09-16T07:33:33.356816+0000","last_became_active":"2024-09-16T07:33:31.042235+0000","last_became_peered":"2024-09-16T07:33:31.042235+0000","last_unstale":"2024-09-16T07:33:33.356816+0000","last_undegraded":"2024-09-16T07:33:33.356816+0000","last_fullsized":"2024-09-16T07:33:33.356816+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:41:18.078744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025623+0000","last_change":"2024-09-16T07:33:32.045670+0000","last_active":"2024-09-16T07:33:33.025623+0000","last_peered":"2024-09-16T07:33:33.025623+0000","last_clean":"2024-09-16T07:33:33.025623+0000","last_became_active":"2024-09-16T07:33:32.045383+0000","last_became_peered":"2024-09-16T07:33:32.045383+0000","last_unstale":"2024-09-16T07:33:33.025623+0000","last_undegraded":"2024-09-16T07:33:33.025623+0000","last_fullsized":"2024-09-16T07:33:33.025623+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:32:56.570489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358500+0000","last_change":"2024-09-16T07:33:31.030651+0000","last_active":"2024-09-16T07:33:33.358500+0000","last_peered":"2024-09-16T07:33:33.358500+0000","last_clean":"2024-09-16T07:33:33.358500+0000","last_became_active":"2024-09-16T07:33:31.030312+0000","last_became_peered":"2024-09-16T07:33:31.030312+0000","last_unstale":"2024-09-16T07:33:33.358500+0000","last_undegraded":"2024-09-16T07:33:33.358500+0000","last_fullsized":"2024-09-16T07:33:33.358500+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:21:57.124363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025464+0000","last_change":"2024-09-16T07:33:32.045722+0000","last_active":"2024-09-16T07:33:33.025464+0000","last_peered":"2024-09-16T07:33:33.025464+0000","last_clean":"2024-09-16T07:33:33.025464+0000","last_became_active":"2024-09-16T07:33:32.045414+0000","last_became_peered":"2024-09-16T07:33:32.045414+0000","last_unstale":"2024-09-16T07:33:33.025464+0000","last_undegraded":"2024-09-16T07:33:33.025464+0000","last_fullsized":"2024-09-16T07:33:33.025464+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:57.552859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"105'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.324194+0000","last_change":"2024-09-16T07:33:31.040312+0000","last_active":"2024-09-16T07:33:33.324194+0000","last_peered":"2024-09-16T07:33:33.324194+0000","last_clean":"2024-09-16T07:33:33.324194+0000","last_became_active":"2024-09-16T07:33:31.040009+0000","last_became_peered":"2024-09-16T07:33:31.040009+0000","last_unstale":"2024-09-16T07:33:33.324194+0000","last_undegraded":"2024-09-16T07:33:33.324194+0000","last_fullsized":"2024-09-16T07:33:33.324194+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:56.393574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357488+0000","last_change":"2024-09-16T07:33:32.054578+0000","last_active":"2024-09-16T07:33:33.357488+0000","last_peered":"2024-09-16T07:33:33.357488+0000","last_clean":"2024-09-16T07:33:33.357488+0000","last_became_active":"2024-09-16T07:33:32.054325+0000","last_became_peered":"2024-09-16T07:33:32.054325+0000","last_unstale":"2024-09-16T07:33:33.357488+0000","last_undegraded":"2024-09-16T07:33:33.357488+0000","last_fullsized":"2024-09-16T07:33:33.357488+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:48.975097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025287+0000","last_change":"2024-09-16T07:33:31.033696+0000","last_active":"2024-09-16T07:33:33.025287+0000","last_peered":"2024-09-16T07:33:33.025287+0000","last_clean":"2024-09-16T07:33:33.025287+0000","last_became_active":"2024-09-16T07:33:31.033430+0000","last_became_peered":"2024-09-16T07:33:31.033430+0000","last_unstale":"2024-09-16T07:33:33.025287+0000","last_undegraded":"2024-09-16T07:33:33.025287+0000","last_fullsized":"2024-09-16T07:33:33.025287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:55.604351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357346+0000","last_change":"2024-09-16T07:33:32.042206+0000","last_active":"2024-09-16T07:33:33.357346+0000","last_peered":"2024-09-16T07:33:33.357346+0000","last_clean":"2024-09-16T07:33:33.357346+0000","last_became_active":"2024-09-16T07:33:32.041947+0000","last_became_peered":"2024-09-16T07:33:32.041947+0000","last_unstale":"2024-09-16T07:33:33.357346+0000","last_undegraded":"2024-09-16T07:33:33.357346+0000","last_fullsized":"2024-09-16T07:33:33.357346+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:37:06.473975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358556+0000","last_change":"2024-09-16T07:33:31.032898+0000","last_active":"2024-09-16T07:33:33.358556+0000","last_peered":"2024-09-16T07:33:33.358556+0000","last_clean":"2024-09-16T07:33:33.358556+0000","last_became_active":"2024-09-16T07:33:31.032344+0000","last_became_peered":"2024-09-16T07:33:31.032344+0000","last_unstale":"2024-09-16T07:33:33.358556+0000","last_undegraded":"2024-09-16T07:33:33.358556+0000","last_fullsized":"2024-09-16T07:33:33.358556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:26.380802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.025119+0000","last_change":"2024-09-16T07:33:32.038999+0000","last_active":"2024-09-16T07:33:33.025119+0000","last_peered":"2024-09-16T07:33:33.025119+0000","last_clean":"2024-09-16T07:33:33.025119+0000","last_became_active":"2024-09-16T07:33:32.038709+0000","last_became_peered":"2024-09-16T07:33:32.038709+0000","last_unstale":"2024-09-16T07:33:33.025119+0000","last_undegraded":"2024-09-16T07:33:33.025119+0000","last_fullsized":"2024-09-16T07:33:33.025119+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:02.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357641+0000","last_change":"2024-09-16T07:33:31.044406+0000","last_active":"2024-09-16T07:33:33.357641+0000","last_peered":"2024-09-16T07:33:33.357641+0000","last_clean":"2024-09-16T07:33:33.357641+0000","last_became_active":"2024-09-16T07:33:31.044017+0000","last_became_peered":"2024-09-16T07:33:31.044017+0000","last_unstale":"2024-09-16T07:33:33.357641+0000","last_undegraded":"2024-09-16T07:33:33.357641+0000","last_fullsized":"2024-09-16T07:33:33.357641+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:06.481250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357714+0000","last_change":"2024-09-16T07:33:32.046842+0000","last_active":"2024-09-16T07:33:33.357714+0000","last_peered":"2024-09-16T07:33:33.357714+0000","last_clean":"2024-09-16T07:33:33.357714+0000","last_became_active":"2024-09-16T07:33:32.046569+0000","last_became_peered":"2024-09-16T07:33:32.046569+0000","last_unstale":"2024-09-16T07:33:33.357714+0000","last_undegraded":"2024-09-16T07:33:33.357714+0000","last_fullsized":"2024-09-16T07:33:33.357714+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:01.321702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358757+0000","last_change":"2024-09-16T07:33:31.037028+0000","last_active":"2024-09-16T07:33:33.358757+0000","last_peered":"2024-09-16T07:33:33.358757+0000","last_clean":"2024-09-16T07:33:33.358757+0000","last_became_active":"2024-09-16T07:33:31.036714+0000","last_became_peered":"2024-09-16T07:33:31.036714+0000","last_unstale":"2024-09-16T07:33:33.358757+0000","last_undegraded":"2024-09-16T07:33:33.358757+0000","last_fullsized":"2024-09-16T07:33:33.358757+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:01.872929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.024139+0000","last_change":"2024-09-16T07:33:32.036865+0000","last_active":"2024-09-16T07:33:33.024139+0000","last_peered":"2024-09-16T07:33:33.024139+0000","last_clean":"2024-09-16T07:33:33.024139+0000","last_became_active":"2024-09-16T07:33:32.036613+0000","last_became_peered":"2024-09-16T07:33:32.036613+0000","last_unstale":"2024-09-16T07:33:33.024139+0000","last_undegraded":"2024-09-16T07:33:33.024139+0000","last_fullsized":"2024-09-16T07:33:33.024139+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:29:18.832918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.356952+0000","last_change":"2024-09-16T07:33:31.044776+0000","last_active":"2024-09-16T07:33:33.356952+0000","last_peered":"2024-09-16T07:33:33.356952+0000","last_clean":"2024-09-16T07:33:33.356952+0000","last_became_active":"2024-09-16T07:33:31.044453+0000","last_became_peered":"2024-09-16T07:33:31.044453+0000","last_unstale":"2024-09-16T07:33:33.356952+0000","last_undegraded":"2024-09-16T07:33:33.356952+0000","last_fullsized":"2024-09-16T07:33:33.356952+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:06.219751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358395+0000","last_change":"2024-09-16T07:33:32.043818+0000","last_active":"2024-09-16T07:33:33.358395+0000","last_peered":"2024-09-16T07:33:33.358395+0000","last_clean":"2024-09-16T07:33:33.358395+0000","last_became_active":"2024-09-16T07:33:32.043573+0000","last_became_peered":"2024-09-16T07:33:32.043573+0000","last_unstale":"2024-09-16T07:33:33.358395+0000","last_undegraded":"2024-09-16T07:33:33.358395+0000","last_fullsized":"2024-09-16T07:33:33.358395+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:10.405842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.356930+0000","last_change":"2024-09-16T07:33:31.042397+0000","last_active":"2024-09-16T07:33:33.356930+0000","last_peered":"2024-09-16T07:33:33.356930+0000","last_clean":"2024-09-16T07:33:33.356930+0000","last_became_active":"2024-09-16T07:33:31.041205+0000","last_became_peered":"2024-09-16T07:33:31.041205+0000","last_unstale":"2024-09-16T07:33:33.356930+0000","last_undegraded":"2024-09-16T07:33:33.356930+0000","last_fullsized":"2024-09-16T07:33:33.356930+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:51:12.255215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357893+0000","last_change":"2024-09-16T07:33:32.044597+0000","last_active":"2024-09-16T07:33:33.357893+0000","last_peered":"2024-09-16T07:33:33.357893+0000","last_clean":"2024-09-16T07:33:33.357893+0000","last_became_active":"2024-09-16T07:33:32.044298+0000","last_became_peered":"2024-09-16T07:33:32.044298+0000","last_unstale":"2024-09-16T07:33:33.357893+0000","last_undegraded":"2024-09-16T07:33:33.357893+0000","last_fullsized":"2024-09-16T07:33:33.357893+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:31.008881+0000","last_clean_scrub_stamp":"2024-09-16T07:33:31.008881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:46.980763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.358516+0000","last_change":"2024-09-16T07:33:31.041709+0000","last_active":"2024-09-16T07:33:33.358516+0000","last_peered":"2024-09-16T07:33:33.358516+0000","last_clean":"2024-09-16T07:33:33.358516+0000","last_became_active":"2024-09-16T07:33:31.040991+0000","last_became_peered":"2024-09-16T07:33:31.040991+0000","last_unstale":"2024-09-16T07:33:33.358516+0000","last_undegraded":"2024-09-16T07:33:33.358516+0000","last_fullsized":"2024-09-16T07:33:33.358516+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:33:30.005987+0000","last_clean_scrub_stamp":"2024-09-16T07:33:30.005987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:32.562633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":265,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T07:33:33.357091+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:33:33.357091+0000","last_peered":"2024-09-16T07:33:33.357091+0000","last_clean":"2024-09-16T07:33:33.357091+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:33:33.357091+0000","last_undegraded":"2024-09-16T07:33:33.357091+0000","last_fullsized":"2024-09-16T07:33:33.357091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476873,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122632,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249208,"statfs":{"total":96636764160,"available":96511188992,"internally_reserved":0,"allocated":1298432,"data_stored":560047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122628,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249212,"statfs":{"total":96636764160,"available":96511193088,"internally_reserved":0,"allocated":1290240,"data_stored":559923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119060,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252780,"statfs":{"total":96636764160,"available":96514846720,"internally_reserved":0,"allocated":1900544,"data_stored":1151945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476872,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100416,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271424,"statfs":{"total":96636764160,"available":96533938176,"internally_reserved":0,"allocated":1286144,"data_stored":559853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509574,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100416,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94271424,"statfs":{"total":96636764160,"available":96533938176,"internally_reserved":0,"allocated":1290240,"data_stored":560435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":101513674},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509576,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148988,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222852,"statfs":{"total":96636764160,"available":96484200448,"internally_reserved":0,"allocated":1286144,"data_stored":559853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509576,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127380,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244460,"statfs":{"total":96636764160,"available":96506327040,"internally_reserved":0,"allocated":1900544,"data_stored":1151889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542278,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122624,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249216,"statfs":{"total":96636764160,"available":96511197184,"internally_reserved":0,"allocated":1290240,"data_stored":560435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:33:41.310 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:33:41.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:33:41.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.508+0000 7f3838c00640 1 -- 172.21.15.71:0/39497786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340b34d0 msgr2=0x7f38340d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.508+0000 7f3838c00640 1 --2- 172.21.15.71:0/39497786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340b34d0 0x7f38340d05c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f381c005fd0 tx=0x7f381c02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.509+0000 7f3838c00640 1 -- 172.21.15.71:0/39497786 shutdown_connections 2024-09-16T07:33:41.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.509+0000 7f3838c00640 1 --2- 172.21.15.71:0/39497786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38340d0b00 0x7f38340c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.509+0000 7f3838c00640 1 --2- 172.21.15.71:0/39497786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340b34d0 0x7f38340d05c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.509+0000 7f3838c00640 1 -- 172.21.15.71:0/39497786 >> 172.21.15.71:0/39497786 conn(0x7f38340d1e30 msgr2=0x7f38340d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:41.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.509+0000 7f3838c00640 1 -- 172.21.15.71:0/39497786 shutdown_connections 2024-09-16T07:33:41.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.509+0000 7f3838c00640 1 -- 172.21.15.71:0/39497786 wait complete. 2024-09-16T07:33:41.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.510+0000 7f3838c00640 1 Processor -- start 2024-09-16T07:33:41.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.510+0000 7f3838c00640 1 -- start start 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3838c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38340b34d0 0x7f38341f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3838c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340d0b00 0x7f38340cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3838c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f383415f530 0x7f38340b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3838c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f38340c6280 con 0x7f38340d0b00 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3838c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f38340cf7b0 con 0x7f38340b34d0 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3838c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f38340c6400 con 0x7f383415f530 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3831400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340d0b00 0x7f38340cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3831400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340d0b00 0x7f38340cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42334/0 (socket says 172.21.15.71:42334) 2024-09-16T07:33:41.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.511+0000 7f3831400640 1 -- 172.21.15.71:0/3580458407 learned_addr learned my addr 172.21.15.71:0/3580458407 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f3831400640 1 -- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f383415f530 msgr2=0x7f38340b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f3831400640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f383415f530 0x7f38340b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f3831400640 1 -- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38340b34d0 msgr2=0x7f38341f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f3831400640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38340b34d0 0x7f38341f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f3831400640 1 -- 172.21.15.71:0/3580458407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38340c90a0 con 0x7f38340d0b00 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f3831400640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340d0b00 0x7f38340cad70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f381c000c00 tx=0x7f381c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f382a000640 1 -- 172.21.15.71:0/3580458407 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c03f070 con 0x7f38340d0b00 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f382a000640 1 -- 172.21.15.71:0/3580458407 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f381c0047a0 con 0x7f38340d0b00 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f382a000640 1 -- 172.21.15.71:0/3580458407 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c035030 con 0x7f38340d0b00 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38340c9380 con 0x7f38340d0b00 2024-09-16T07:33:41.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.512+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38340c9940 con 0x7f38340d0b00 2024-09-16T07:33:41.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.514+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38340c5b10 con 0x7f38340d0b00 2024-09-16T07:33:41.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.515+0000 7f382a000640 1 -- 172.21.15.71:0/3580458407 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f381c042070 con 0x7f38340d0b00 2024-09-16T07:33:41.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.516+0000 7f382a000640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3810079e70 0x7f381007c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.516+0000 7f382a000640 1 -- 172.21.15.71:0/3580458407 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f381c0ba740 con 0x7f38340d0b00 2024-09-16T07:33:41.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.516+0000 7f3831e00640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3810079e70 0x7f381007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.517+0000 7f3831e00640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3810079e70 0x7f381007c330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f38240012e0 tx=0x7f3824007d30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.520+0000 7f382a000640 1 -- 172.21.15.71:0/3580458407 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f381c0843f0 con 0x7f38340d0b00 2024-09-16T07:33:41.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.776+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3834070980 con 0x7f38340d0b00 2024-09-16T07:33:41.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.778+0000 7f382a000640 1 -- 172.21.15.71:0/3580458407 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f381c083cf0 con 0x7f38340d0b00 2024-09-16T07:33:41.780 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:33:41.780 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:33:41.780 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:33:41.781 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:33:41.781 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3810079e70 msgr2=0x7f381007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3810079e70 0x7f381007c330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f38240012e0 tx=0x7f3824007d30 comp rx=0 tx=0).stop 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340d0b00 msgr2=0x7f38340cad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340d0b00 0x7f38340cad70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f381c000c00 tx=0x7f381c0038a0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 shutdown_connections 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3810079e70 0x7f381007c330 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f383415f530 0x7f38340b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38340d0b00 0x7f38340cad70 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 --2- 172.21.15.71:0/3580458407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38340b34d0 0x7f38341f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 >> 172.21.15.71:0/3580458407 conn(0x7f38340d1e30 msgr2=0x7f38340cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:41.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 shutdown_connections 2024-09-16T07:33:41.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.781+0000 7f3838c00640 1 -- 172.21.15.71:0/3580458407 wait complete. 2024-09-16T07:33:41.798 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:33:41.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.954+0000 7f8b87000640 1 -- 172.21.15.71:0/3233995381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 msgr2=0x7f8b800b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.954+0000 7f8b87000640 1 --2- 172.21.15.71:0/3233995381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 0x7f8b800b28f0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009830 tx=0x7f8b7802a510 comp rx=0 tx=0).stop 2024-09-16T07:33:41.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.954+0000 7f8b87000640 1 -- 172.21.15.71:0/3233995381 shutdown_connections 2024-09-16T07:33:41.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.954+0000 7f8b87000640 1 --2- 172.21.15.71:0/3233995381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 0x7f8b800b28f0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.954+0000 7f8b87000640 1 -- 172.21.15.71:0/3233995381 >> 172.21.15.71:0/3233995381 conn(0x7f8b800cfa20 msgr2=0x7f8b800cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:41.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.955+0000 7f8b87000640 1 -- 172.21.15.71:0/3233995381 shutdown_connections 2024-09-16T07:33:41.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.955+0000 7f8b87000640 1 -- 172.21.15.71:0/3233995381 wait complete. 2024-09-16T07:33:41.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.956+0000 7f8b87000640 1 Processor -- start 2024-09-16T07:33:41.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.956+0000 7f8b87000640 1 -- start start 2024-09-16T07:33:41.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b87000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 0x7f8b80143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b87000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8015bc30 0x7f8b80150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b87000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b80151060 0x7f8b80155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b87000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b80074660 con 0x7f8b800d0540 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b87000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8006fa30 con 0x7f8b8015bc30 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b87000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b800c8460 con 0x7f8b80151060 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b85c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 0x7f8b80143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b85c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 0x7f8b80143060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42340/0 (socket says 172.21.15.71:42340) 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.957+0000 7f8b85c00640 1 -- 172.21.15.71:0/731827940 learned_addr learned my addr 172.21.15.71:0/731827940 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b85200640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8015bc30 0x7f8b80150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b86600640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b80151060 0x7f8b80155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b85c00640 1 -- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b80151060 msgr2=0x7f8b80155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b85c00640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b80151060 0x7f8b80155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b85c00640 1 -- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8015bc30 msgr2=0x7f8b80150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b85c00640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8015bc30 0x7f8b80150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b85c00640 1 -- 172.21.15.71:0/731827940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b801f2f80 con 0x7f8b800d0540 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b85c00640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 0x7f8b80143060 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400c7a0 tx=0x7f8b7400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b7d600640 1 -- 172.21.15.71:0/731827940 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74017070 con 0x7f8b800d0540 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b7d600640 1 -- 172.21.15.71:0/731827940 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b74005070 con 0x7f8b800d0540 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b7d600640 1 -- 172.21.15.71:0/731827940 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b740054a0 con 0x7f8b800d0540 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b800c7e10 con 0x7f8b800d0540 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.958+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b80156370 con 0x7f8b800d0540 2024-09-16T07:33:41.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.960+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b80151470 con 0x7f8b800d0540 2024-09-16T07:33:41.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.961+0000 7f8b7d600640 1 -- 172.21.15.71:0/731827940 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f8b7401a070 con 0x7f8b800d0540 2024-09-16T07:33:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.962+0000 7f8b7d600640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b60079e70 0x7f8b6007c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.962+0000 7f8b7d600640 1 -- 172.21.15.71:0/731827940 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f8b74099070 con 0x7f8b800d0540 2024-09-16T07:33:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.962+0000 7f8b85200640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b60079e70 0x7f8b6007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:41.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.963+0000 7f8b85200640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b60079e70 0x7f8b6007c330 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8b801efe10 tx=0x7f8b6c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:41.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:41.966+0000 7f8b7d600640 1 -- 172.21.15.71:0/731827940 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b74062d80 con 0x7f8b800d0540 2024-09-16T07:33:42.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.186+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b800b28f0 con 0x7f8b800d0540 2024-09-16T07:33:42.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.187+0000 7f8b7d600640 1 -- 172.21.15.71:0/731827940 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5231 (secure 0 0 0) 0x7f8b74062680 con 0x7f8b800d0540 2024-09-16T07:33:42.187 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:33:42.187 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":66,"btime":"2024-09-16T07:33:35:043973+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3180037555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3180037555},{"type":"v1","addr":"172.21.15.71:6835","nonce":3180037555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3241229928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3241229928},{"type":"v1","addr":"172.21.15.177:6836","nonce":3241229928}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1569295783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1569295783},{"type":"v1","addr":"172.21.15.177:6837","nonce":1569295783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:33:31.441505+0000","modified":"2024-09-16T07:33:34.454314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/4054989006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4054989006},{"type":"v1","addr":"172.21.15.71:6833","nonce":4054989006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-09-16T07:33:42.187 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 66 2024-09-16T07:33:42.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b60079e70 msgr2=0x7f8b6007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b60079e70 0x7f8b6007c330 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8b801efe10 tx=0x7f8b6c009040 comp rx=0 tx=0).stop 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 msgr2=0x7f8b80143060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 0x7f8b80143060 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400c7a0 tx=0x7f8b7400cc60 comp rx=0 tx=0).stop 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 shutdown_connections 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b60079e70 0x7f8b6007c330 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b80151060 0x7f8b80155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8015bc30 0x7f8b80150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 --2- 172.21.15.71:0/731827940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b800d0540 0x7f8b80143060 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 >> 172.21.15.71:0/731827940 conn(0x7f8b800cfa20 msgr2=0x7f8b80156cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 shutdown_connections 2024-09-16T07:33:42.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.189+0000 7f8b87000640 1 -- 172.21.15.71:0/731827940 wait complete. 2024-09-16T07:33:42.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:33:31.441505+0000', 'modified': '2024-09-16T07:33:34.454314+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7863': {'gid': 7863, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6833/4054989006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 4054989006}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 4054989006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7863, 'qdb_cluster': [7863]} 2024-09-16T07:33:42.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:33:42.609 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-16T07:33:42.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.769+0000 7f6c2c400640 1 -- 172.21.15.71:0/2188975560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c240ccf00 msgr2=0x7f6c240b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.769+0000 7f6c2c400640 1 --2- 172.21.15.71:0/2188975560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c240ccf00 0x7f6c240b2440 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6c10005fd0 tx=0x7f6c1002a540 comp rx=0 tx=0).stop 2024-09-16T07:33:42.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.769+0000 7f6c2c400640 1 -- 172.21.15.71:0/2188975560 shutdown_connections 2024-09-16T07:33:42.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.769+0000 7f6c2c400640 1 --2- 172.21.15.71:0/2188975560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c24073fb0 0x7f6c24074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.769+0000 7f6c2c400640 1 --2- 172.21.15.71:0/2188975560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c240ccf00 0x7f6c240b2440 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.769+0000 7f6c2c400640 1 -- 172.21.15.71:0/2188975560 >> 172.21.15.71:0/2188975560 conn(0x7f6c240b1950 msgr2=0x7f6c240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.769+0000 7f6c2c400640 1 -- 172.21.15.71:0/2188975560 shutdown_connections 2024-09-16T07:33:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.769+0000 7f6c2c400640 1 -- 172.21.15.71:0/2188975560 wait complete. 2024-09-16T07:33:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.771+0000 7f6c2c400640 1 Processor -- start 2024-09-16T07:33:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.771+0000 7f6c2c400640 1 -- start start 2024-09-16T07:33:42.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2c400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c24073fb0 0x7f6c2414cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2c400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c240ccf00 0x7f6c240b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2c400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2415f5a0 0x7f6c2414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2c400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2406f2e0 con 0x7f6c2415f5a0 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2c400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2406f160 con 0x7f6c24073fb0 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2c400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2406f460 con 0x7f6c240ccf00 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2415f5a0 0x7f6c2414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2415f5a0 0x7f6c2414bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42358/0 (socket says 172.21.15.71:42358) 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c2a400640 1 -- 172.21.15.71:0/289019970 learned_addr learned my addr 172.21.15.71:0/289019970 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c29000640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c240ccf00 0x7f6c240b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.772+0000 7f6c29a00640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c24073fb0 0x7f6c2414cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c2a400640 1 -- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c240ccf00 msgr2=0x7f6c240b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c2a400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c240ccf00 0x7f6c240b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c2a400640 1 -- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c24073fb0 msgr2=0x7f6c2414cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c2a400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c24073fb0 0x7f6c2414cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c2a400640 1 -- 172.21.15.71:0/289019970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c24150500 con 0x7f6c2415f5a0 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c29000640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c240ccf00 0x7f6c240b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c29a00640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c24073fb0 0x7f6c2414cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c2a400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2415f5a0 0x7f6c2414bfd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d780 tx=0x7f6c1c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c21600640 1 -- 172.21.15.71:0/289019970 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c017070 con 0x7f6c2415f5a0 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c21600640 1 -- 172.21.15.71:0/289019970 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c1c00de90 con 0x7f6c2415f5a0 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c21600640 1 -- 172.21.15.71:0/289019970 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c005020 con 0x7f6c2415f5a0 2024-09-16T07:33:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c2c400640 1 -- 172.21.15.71:0/289019970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c241507e0 con 0x7f6c2415f5a0 2024-09-16T07:33:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.773+0000 7f6c2c400640 1 -- 172.21.15.71:0/289019970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c240ce790 con 0x7f6c2415f5a0 2024-09-16T07:33:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.776+0000 7f6c21600640 1 -- 172.21.15.71:0/289019970 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f6c1c0051c0 con 0x7f6c2415f5a0 2024-09-16T07:33:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.776+0000 7f6c21600640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6c04079e40 0x7f6c0407c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c21600640 1 -- 172.21.15.71:0/289019970 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f6c1c099ba0 con 0x7f6c2415f5a0 2024-09-16T07:33:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c2f3c1740 1 Processor -- start 2024-09-16T07:33:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c2f3c1740 1 -- start start 2024-09-16T07:33:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c29a00640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6c04079e40 0x7f6c0407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 0x55de16991c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de1694ca20 0x55de16990390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 0x55de169908d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55de168cf430 con 0x55de1694ca20 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55de168bf5d0 con 0x55de1694cfa0 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.777+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55de168aea00 con 0x55de169918b0 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de1694ca20 0x55de16990390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29a00640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6c04079e40 0x7f6c0407c300 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6c18001da0 tx=0x7f6c18009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de1694ca20 0x55de16990390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42374/0 (socket says 172.21.15.71:42374) 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29000640 1 -- 172.21.15.71:0/2043438663 learned_addr learned my addr 172.21.15.71:0/2043438663 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c2a400640 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 0x55de169908d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29a00640 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 0x55de16991c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29000640 1 -- 172.21.15.71:0/2043438663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 msgr2=0x55de16991c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:33:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29000640 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 0x55de16991c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29000640 1 -- 172.21.15.71:0/2043438663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 msgr2=0x55de169908d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:33:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29000640 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 0x55de169908d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.778+0000 7f6c29000640 1 -- 172.21.15.71:0/2043438663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de16877520 con 0x55de1694ca20 2024-09-16T07:33:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.779+0000 7f6c29000640 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de1694ca20 0x55de16990390 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6c100099f0 tx=0x7f6c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761a70c2ab5a22c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.779+0000 7f6c16000640 1 -- 172.21.15.71:0/2043438663 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1003f070 con 0x55de1694ca20 2024-09-16T07:33:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.779+0000 7f6c16000640 1 -- 172.21.15.71:0/2043438663 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c100035f0 con 0x55de1694ca20 2024-09-16T07:33:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.779+0000 7f6c16000640 1 -- 172.21.15.71:0/2043438663 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10035070 con 0x55de1694ca20 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.779+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/2043438663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de1694ca20 msgr2=0x55de16990390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.779+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de1694ca20 0x55de16990390 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6c100099f0 tx=0x7f6c100046c0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.780+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/2043438663 shutdown_connections 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.780+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 0x55de169908d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.780+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de1694ca20 0x55de16990390 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.780+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/2043438663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 0x55de16991c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.780+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/2043438663 >> 172.21.15.71:0/2043438663 conn(0x55de16947ff0 msgr2=0x55de1694ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.780+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/2043438663 shutdown_connections 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.780+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/2043438663 wait complete. 2024-09-16T07:33:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.780+0000 7f6c2f3c1740 1 -- start start 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.781+0000 7f6c2f3c1740 1 Processor -- start 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 0x55de1698f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 0x55de169a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16975200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55de16855330 con 0x55de169a1360 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55de168b2810 con 0x55de1694cfa0 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55de1661f620 con 0x55de169918b0 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2f3c1740 10 client.0 _ll_register_callbacks cb 0x55de1685a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16975200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16975200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42386/0 (socket says 172.21.15.71:42386) 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 -- 172.21.15.71:0/113098674 learned_addr learned my addr 172.21.15.71:0/113098674 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c29a00640 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 0x55de1698f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c29000640 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 0x55de169a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 -- 172.21.15.71:0/113098674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 msgr2=0x55de169a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 0x55de169a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 -- 172.21.15.71:0/113098674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 msgr2=0x55de1698f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 0x55de1698f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 -- 172.21.15.71:0/113098674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de16610850 con 0x55de169a1360 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c2a400640 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16975200 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c062e50 tx=0x7f6c1c05f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d83267b24c91c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.782+0000 7f6c0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c0aa00640 1 -- 172.21.15.71:0/113098674 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c089070 con 0x55de169a1360 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/113098674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c100035f0 con 0x55de169a1360 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c0aa00640 1 -- 172.21.15.71:0/113098674 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f6c1c06c650 con 0x55de169a1360 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c0aa00640 1 -- 172.21.15.71:0/113098674 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f6c1c020040 con 0x55de169a1360 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 10 client.8025 fetch_fsmap learned FSMap version 66 2024-09-16T07:33:42.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/113098674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55de166876b0 con 0x55de169a1360 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c0aa00640 1 -- 172.21.15.71:0/113098674 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c100035f0 con 0x55de169a1360 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c0aa00640 1 -- 172.21.15.71:0/113098674 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7f6c1c07d080 con 0x55de169a1360 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 10 client.8025 fetch_fsmap finished waiting for FSMap version 66 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 10 client.8025 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 20 client.8025 populate_metadata read hostname 'smithi071' 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] conn(0x55de169a48e0 0x55de169a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 4 client.8025 mds_command: new command op to 7863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.783+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/113098674 --> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de168c1520 con 0x55de169a48e0 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.784+0000 7f6c29a00640 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] conn(0x55de169a48e0 0x55de169a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.784+0000 7f6c29a00640 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] conn(0x55de169a48e0 0x55de169a4cc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb052271d01f6a0b server_cookie=f2f80a1c28e30b74 in_seq=0 out_seq=0 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.784+0000 7f6c0aa00640 10 client.8025 ms_handle_connect on v2:172.21.15.71:6832/4054989006 2024-09-16T07:33:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c0aa00640 1 -- 172.21.15.71:0/113098674 <== mds.0 v2:172.21.15.71:6832/4054989006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de168c1520 con 0x55de169a48e0 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c0aa00640 10 client.8025 handle_command_reply: tid=0 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 client.8025 shutdown 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/113098674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 msgr2=0x55de16975200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16975200 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c062e50 tx=0x7f6c1c05f690 comp rx=0 tx=0).stop 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/113098674 shutdown_connections 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] conn(0x55de169a48e0 0x55de169a4cc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16975200 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de169918b0 0x55de169a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/113098674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de1694cfa0 0x55de1698f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c0aa00640 0 client.8025 ms_handle_reset on v2:172.21.15.71:6832/4054989006 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/113098674 >> 172.21.15.71:0/113098674 conn(0x55de1694ca20 msgr2=0x55de1698a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.785+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/113098674 shutdown_connections 2024-09-16T07:33:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.786+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/113098674 wait complete. 2024-09-16T07:33:42.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.786+0000 7f6c2f3c1740 20 client.8025 trim_cache size 0 max 16384 2024-09-16T07:33:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.805+0000 7f6c2f3c1740 1 Processor -- start 2024-09-16T07:33:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.805+0000 7f6c2f3c1740 1 -- start start 2024-09-16T07:33:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.805+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de169b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169918b0 0x55de1694b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169a1360 0x55de16a0c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24204640 con 0x55de169918b0 2024-09-16T07:33:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241fcd70 con 0x55de169a1360 2024-09-16T07:33:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6c240cfa20 con 0x55de1694cfa0 2024-09-16T07:33:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169918b0 0x55de1694b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169918b0 0x55de1694b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42390/0 (socket says 172.21.15.71:42390) 2024-09-16T07:33:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29000640 1 -- 172.21.15.71:0/921542187 learned_addr learned my addr 172.21.15.71:0/921542187 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29a00640 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de169b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c2a400640 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169a1360 0x55de16a0c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29000640 1 -- 172.21.15.71:0/921542187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 msgr2=0x55de169b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:33:42.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29000640 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de169b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29000640 1 -- 172.21.15.71:0/921542187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169a1360 msgr2=0x55de16a0c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:33:42.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29000640 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169a1360 0x55de16a0c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.806+0000 7f6c29000640 1 -- 172.21.15.71:0/921542187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de16993320 con 0x55de169918b0 2024-09-16T07:33:42.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.807+0000 7f6c29000640 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169918b0 0x55de1694b410 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f6c100099c0 tx=0x7f6c10009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8526bbdf4f8292e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:42.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.807+0000 7f6c15600640 1 -- 172.21.15.71:0/921542187 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1004b020 con 0x55de169918b0 2024-09-16T07:33:42.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.807+0000 7f6c15600640 1 -- 172.21.15.71:0/921542187 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c100037f0 con 0x55de169918b0 2024-09-16T07:33:42.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.807+0000 7f6c15600640 1 -- 172.21.15.71:0/921542187 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1003f040 con 0x55de169918b0 2024-09-16T07:33:42.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.807+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/921542187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169918b0 msgr2=0x55de1694b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.807+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169918b0 0x55de1694b410 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f6c100099c0 tx=0x7f6c10009260 comp rx=0 tx=0).stop 2024-09-16T07:33:42.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/921542187 shutdown_connections 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169a1360 0x55de16a0c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169918b0 0x55de1694b410 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/921542187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de169b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/921542187 >> 172.21.15.71:0/921542187 conn(0x55de1694ca20 msgr2=0x55de169a0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/921542187 shutdown_connections 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/921542187 wait complete. 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 -- start start 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.808+0000 7f6c2f3c1740 1 Processor -- start 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.809+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de16a0b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.809+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169918b0 0x55de16944e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.809+0000 7f6c2f3c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16945380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.809+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55de1698ba20 con 0x55de169a1360 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.809+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55de169642d0 con 0x55de169918b0 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.809+0000 7f6c2f3c1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55de1698b230 con 0x55de1694cfa0 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.809+0000 7f6c2f3c1740 10 client.0 _ll_register_callbacks cb 0x7f6c1c062cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16945380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16945380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42396/0 (socket says 172.21.15.71:42396) 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 -- 172.21.15.71:0/3863508354 learned_addr learned my addr 172.21.15.71:0/3863508354 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c29000640 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169918b0 0x55de16944e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c29a00640 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de16a0b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 -- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 msgr2=0x55de16a0b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de16a0b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 -- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169918b0 msgr2=0x55de16944e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169918b0 0x55de16944e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 -- 172.21.15.71:0/3863508354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de16945d50 con 0x55de169a1360 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c29a00640 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de16a0b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c2a400640 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16945380 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c062e50 tx=0x7f6c1c063910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923064560557645b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.810+0000 7f6c0aa00640 1 -- 172.21.15.71:0/3863508354 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c089040 con 0x55de169a1360 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c0aa00640 1 -- 172.21.15.71:0/3863508354 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f6c1c0623b0 con 0x55de169a1360 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/3863508354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c100037f0 con 0x55de169a1360 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c0aa00640 1 -- 172.21.15.71:0/3863508354 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f6c1c067c80 con 0x55de169a1360 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 10 client.8031 fetch_fsmap learned FSMap version 66 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/3863508354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55de16a0baf0 con 0x55de169a1360 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c0aa00640 1 -- 172.21.15.71:0/3863508354 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c100037f0 con 0x55de169a1360 2024-09-16T07:33:42.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c0aa00640 1 -- 172.21.15.71:0/3863508354 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7f6c1c05eaa0 con 0x55de169a1360 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 10 client.8031 fetch_fsmap finished waiting for FSMap version 66 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 10 client.8031 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 20 client.8031 populate_metadata read hostname 'smithi071' 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] conn(0x55de169a8f10 0x55de169a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 4 client.8031 mds_command: new command op to 7863 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/3863508354 --> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] -- command(tid 0: {"prefix": "status"}) -- 0x55de168bcfd0 con 0x55de169a8f10 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.811+0000 7f6c29a00640 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] conn(0x55de169a8f10 0x55de169a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c29a00640 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] conn(0x55de169a8f10 0x55de169a92f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69e81c5663c5c1a server_cookie=a09846b2d8347904 in_seq=0 out_seq=0 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c0aa00640 10 client.8031 ms_handle_connect on v2:172.21.15.71:6832/4054989006 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c0aa00640 1 -- 172.21.15.71:0/3863508354 <== mds.0 v2:172.21.15.71:6832/4054989006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55de168bcfd0 con 0x55de169a8f10 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c0aa00640 10 client.8031 handle_command_reply: tid=0 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 client.8031 shutdown 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/3863508354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 msgr2=0x55de16945380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16945380 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c062e50 tx=0x7f6c1c063910 comp rx=0 tx=0).stop 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/3863508354 shutdown_connections 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.71:6832/4054989006,v1:172.21.15.71:6833/4054989006] conn(0x55de169a8f10 0x55de169a92f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de169a1360 0x55de16945380 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55de169918b0 0x55de16944e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c0aa00640 0 client.8031 ms_handle_reset on v2:172.21.15.71:6832/4054989006 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 --2- 172.21.15.71:0/3863508354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55de1694cfa0 0x55de16a0b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/3863508354 >> 172.21.15.71:0/3863508354 conn(0x55de1694ca20 msgr2=0x55de1699d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.812+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/3863508354 shutdown_connections 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2f3c1740 1 -- 172.21.15.71:0/3863508354 wait complete. 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2f3c1740 20 client.8031 trim_cache size 0 max 16384 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:33:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7863, 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 10.795743626, 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 66, 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 105, 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 105, 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 15.121599872999999 2024-09-16T07:33:42.820 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:33:42.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 -- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6c04079e40 msgr2=0x7f6c0407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6c04079e40 0x7f6c0407c300 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6c18001da0 tx=0x7f6c18009040 comp rx=0 tx=0).stop 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 -- 172.21.15.71:0/289019970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2415f5a0 msgr2=0x7f6c2414bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2415f5a0 0x7f6c2414bfd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d780 tx=0x7f6c1c00dc40 comp rx=0 tx=0).stop 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 -- 172.21.15.71:0/289019970 shutdown_connections 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6c04079e40 0x7f6c0407c300 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2415f5a0 0x7f6c2414bfd0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c240ccf00 0x7f6c240b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 --2- 172.21.15.71:0/289019970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c24073fb0 0x7f6c2414cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 -- 172.21.15.71:0/289019970 >> 172.21.15.71:0/289019970 conn(0x7f6c240b1950 msgr2=0x7f6c2409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 -- 172.21.15.71:0/289019970 shutdown_connections 2024-09-16T07:33:42.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.813+0000 7f6c2c400640 1 -- 172.21.15.71:0/289019970 wait complete. 2024-09-16T07:33:42.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:33:42.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.984+0000 7f0208e00640 1 -- 172.21.15.71:0/1045652910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 msgr2=0x7f01e8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.984+0000 7f0208e00640 1 --2- 172.21.15.71:0/1045652910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 0x7f01e8004010 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009b80 tx=0x7f01e8005830 comp rx=0 tx=0).stop 2024-09-16T07:33:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.984+0000 7f0203e00640 1 -- 172.21.15.71:0/1045652910 reap_dead start 2024-09-16T07:33:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.985+0000 7f0208e00640 1 -- 172.21.15.71:0/1045652910 shutdown_connections 2024-09-16T07:33:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.985+0000 7f0208e00640 1 -- 172.21.15.71:0/1045652910 >> 172.21.15.71:0/1045652910 conn(0x7f02040d1c40 msgr2=0x7f02040d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.985+0000 7f0208e00640 1 -- 172.21.15.71:0/1045652910 shutdown_connections 2024-09-16T07:33:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.985+0000 7f0208e00640 1 -- 172.21.15.71:0/1045652910 wait complete. 2024-09-16T07:33:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.986+0000 7f0208e00640 1 Processor -- start 2024-09-16T07:33:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.986+0000 7f0208e00640 1 -- start start 2024-09-16T07:33:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.987+0000 7f0208e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 0x7f02041e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.987+0000 7f0208e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0204071550 0x7f02040719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.987+0000 7f0208e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204150bc0 0x7f0204151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.987+0000 7f0208e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f02040d28e0 con 0x7f01e8003c10 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.987+0000 7f0208e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02040d2760 con 0x7f0204150bc0 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.987+0000 7f0208e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f02040d2a60 con 0x7f0204071550 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 0x7f02041e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 0x7f02041e54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42410/0 (socket says 172.21.15.71:42410) 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 -- 172.21.15.71:0/909083427 learned_addr learned my addr 172.21.15.71:0/909083427 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0202a00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0204071550 0x7f02040719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203e00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204150bc0 0x7f0204151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 -- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0204071550 msgr2=0x7f02040719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0204071550 0x7f02040719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 -- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204150bc0 msgr2=0x7f0204151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204150bc0 0x7f0204151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 -- 172.21.15.71:0/909083427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0204159520 con 0x7f01e8003c10 2024-09-16T07:33:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.988+0000 7f0203400640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 0x7f02041e54a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f01f4036040 tx=0x7f01f4033910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.989+0000 7f01fb400640 1 -- 172.21.15.71:0/909083427 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4038040 con 0x7f01e8003c10 2024-09-16T07:33:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.989+0000 7f01fb400640 1 -- 172.21.15.71:0/909083427 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01f4033b00 con 0x7f01e8003c10 2024-09-16T07:33:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.989+0000 7f01fb400640 1 -- 172.21.15.71:0/909083427 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4004020 con 0x7f01e8003c10 2024-09-16T07:33:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.989+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0204072130 con 0x7f01e8003c10 2024-09-16T07:33:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.989+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02040c6e20 con 0x7f01e8003c10 2024-09-16T07:33:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.990+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02040b1de0 con 0x7f01e8003c10 2024-09-16T07:33:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.991+0000 7f01fb400640 1 -- 172.21.15.71:0/909083427 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f01f4045020 con 0x7f01e8003c10 2024-09-16T07:33:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.992+0000 7f01fb400640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01e4079e70 0x7f01e407c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.992+0000 7f01fb400640 1 -- 172.21.15.71:0/909083427 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f01f40ba1d0 con 0x7f01e8003c10 2024-09-16T07:33:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.992+0000 7f0202a00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01e4079e70 0x7f01e407c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:42.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.993+0000 7f0202a00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01e4079e70 0x7f01e407c330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f02040c52c0 tx=0x7f01f0006dd0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:42.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:42.996+0000 7f01fb400640 1 -- 172.21.15.71:0/909083427 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0204159520 con 0x7f01e8003c10 2024-09-16T07:33:43.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.218+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f02040cde20 con 0x7f01e8003c10 2024-09-16T07:33:43.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.218+0000 7f01fb400640 1 -- 172.21.15.71:0/909083427 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f01f4083ee0 con 0x7f01e8003c10 2024-09-16T07:33:43.219 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:33:43.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01e4079e70 msgr2=0x7f01e407c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:43.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01e4079e70 0x7f01e407c330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f02040c52c0 tx=0x7f01f0006dd0 comp rx=0 tx=0).stop 2024-09-16T07:33:43.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 msgr2=0x7f02041e54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 0x7f02041e54a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f01f4036040 tx=0x7f01f4033910 comp rx=0 tx=0).stop 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 shutdown_connections 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f01e4079e70 0x7f01e407c330 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204150bc0 0x7f0204151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0204071550 0x7f02040719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 --2- 172.21.15.71:0/909083427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01e8003c10 0x7f02041e54a0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 >> 172.21.15.71:0/909083427 conn(0x7f02040d1c40 msgr2=0x7f02040d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 shutdown_connections 2024-09-16T07:33:43.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43.221+0000 7f0208e00640 1 -- 172.21.15.71:0/909083427 wait complete. 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:33:43.237 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:33:43.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:43.238 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:33:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:33:43.273 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:33:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42054sec preferred_lft 42054sec 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:33:43.275 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:33:43.276 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:33:43.276 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:33:43.276 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:33:43.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:43.276 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:33:43.367 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:33:43.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:43.370 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:33:43.406 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:33:43.406 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:33:43.406 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:33:43.406 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:33:43.406 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:33:43.407 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:33:43.407 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:33:43.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:33:43.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:33:43.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:33:43.407 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:33:43.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:33:43.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:33:43.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:43.602 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:33:43.637 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:33:43.638 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:33:43.638 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:33:43.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:43.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:33:43.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:33:43.735 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:33:43.770 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:33:43.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:33:43.771 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:33:43.845 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:33:43.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:43.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:33:43.974 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:33:43.974 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:33:44.002 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:33:44.002 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:33:44.002 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:33:44.002 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:33:44.002 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:33:44.004 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:33:44.004 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:33:44.004 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:33:44.005 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:33:44.005 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:33:44.007 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:33:44.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:44.008 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:33:44.064 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:33:44.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:33:44.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:33:44.162 INFO:teuthology.orchestra.run.smithi071.stdout:client7300 2024-09-16T07:33:44.162 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:33:44.162 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:44.198 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 -- 172.21.15.71:0/4226445769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb015bb90 msgr2=0x7f0eb00b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 --2- 172.21.15.71:0/4226445769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb015bb90 0x7f0eb00b2810 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0eb006ff40 tx=0x7f0ea002a540 comp rx=0 tx=0).stop 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 -- 172.21.15.71:0/4226445769 shutdown_connections 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 --2- 172.21.15.71:0/4226445769 >> v2:172.21.15.177:3300/0 conn(0x7f0eb00cf3e0 0x7f0eb0074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 --2- 172.21.15.71:0/4226445769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb00cefc0 0x7f0eb0073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 --2- 172.21.15.71:0/4226445769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb015bb90 0x7f0eb00b2810 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 -- 172.21.15.71:0/4226445769 >> 172.21.15.71:0/4226445769 conn(0x7f0eb00b1510 msgr2=0x7f0eb00b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 -- 172.21.15.71:0/4226445769 shutdown_connections 2024-09-16T07:33:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.399+0000 7f0eb5800640 1 -- 172.21.15.71:0/4226445769 wait complete. 2024-09-16T07:33:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.401+0000 7f0eb5800640 1 Processor -- start 2024-09-16T07:33:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.401+0000 7f0eb5800640 1 -- start start 2024-09-16T07:33:44.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eb5800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb00cefc0 0x7f0eb01e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eb5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb00cf3e0 0x7f0eb014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eb5800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb015bb90 0x7f0eb014de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eb5800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb006eee0 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eb5800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb006ed60 con 0x7f0eb00cefc0 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eb5800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb006f060 con 0x7f0eb015bb90 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eaf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb00cf3e0 0x7f0eb014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eaf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb00cf3e0 0x7f0eb014d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42426/0 (socket says 172.21.15.71:42426) 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eaf400640 1 -- 172.21.15.71:0/2056894435 learned_addr learned my addr 172.21.15.71:0/2056894435 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eafe00640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb00cefc0 0x7f0eb01e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eb4e00640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb015bb90 0x7f0eb014de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eaf400640 1 -- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb015bb90 msgr2=0x7f0eb014de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eaf400640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb015bb90 0x7f0eb014de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eaf400640 1 -- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb00cefc0 msgr2=0x7f0eb01e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eaf400640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb00cefc0 0x7f0eb01e45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.402+0000 7f0eaf400640 1 -- 172.21.15.71:0/2056894435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb015e1a0 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.403+0000 7f0eaf400640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb00cf3e0 0x7f0eb014d940 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b4a0 tx=0x7f0e9c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.403+0000 7f0ea7e00640 1 -- 172.21.15.71:0/2056894435 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c014070 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.403+0000 7f0ea7e00640 1 -- 172.21.15.71:0/2056894435 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e9c010700 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.403+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb015e480 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.403+0000 7f0ea7e00640 1 -- 172.21.15.71:0/2056894435 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c010b50 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.403+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb015ea40 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.404+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb00b2810 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.405+0000 7f0ea7e00640 1 -- 172.21.15.71:0/2056894435 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f0e9c017070 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.406+0000 7f0ea7e00640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e90079e40 0x7f0e9007c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.406+0000 7f0ea7e00640 1 -- 172.21.15.71:0/2056894435 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f0e9c099500 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.407+0000 7f0eafe00640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e90079e40 0x7f0e9007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:44.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.407+0000 7f0eafe00640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e90079e40 0x7f0e9007c300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0033000 tx=0x7f0ea0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:44.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.411+0000 7f0ea7e00640 1 -- 172.21.15.71:0/2056894435 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0e9c063210 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.634+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f0eb0069d20 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.787+0000 7f0ea7e00640 1 -- 172.21.15.71:0/2056894435 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v67) ==== 136+0+0 (secure 0 0 0) 0x7f0e9c062b10 con 0x7f0eb00cf3e0 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e90079e40 msgr2=0x7f0e9007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e90079e40 0x7f0e9007c300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0033000 tx=0x7f0ea0004730 comp rx=0 tx=0).stop 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb00cf3e0 msgr2=0x7f0eb014d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb00cf3e0 0x7f0eb014d940 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b4a0 tx=0x7f0e9c00b960 comp rx=0 tx=0).stop 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 shutdown_connections 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e90079e40 0x7f0e9007c300 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb015bb90 0x7f0eb014de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb00cf3e0 0x7f0eb014d940 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 --2- 172.21.15.71:0/2056894435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb00cefc0 0x7f0eb01e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 >> 172.21.15.71:0/2056894435 conn(0x7f0eb00b1510 msgr2=0x7f0eb00b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:44.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 shutdown_connections 2024-09-16T07:33:44.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.790+0000 7f0eb5800640 1 -- 172.21.15.71:0/2056894435 wait complete. 2024-09-16T07:33:44.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:33:44.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.964+0000 7f5b80a00640 1 -- 172.21.15.71:0/1730585528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c0746a0 msgr2=0x7f5b7c0732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:44.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.964+0000 7f5b80a00640 1 --2- 172.21.15.71:0/1730585528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c0746a0 0x7f5b7c0732b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400acf0 tx=0x7f5b7402bae0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.964+0000 7f5b80a00640 1 -- 172.21.15.71:0/1730585528 shutdown_connections 2024-09-16T07:33:44.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.964+0000 7f5b80a00640 1 --2- 172.21.15.71:0/1730585528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c0746a0 0x7f5b7c0732b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.964+0000 7f5b80a00640 1 -- 172.21.15.71:0/1730585528 >> 172.21.15.71:0/1730585528 conn(0x7f5b7c0b1950 msgr2=0x7f5b7c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:44.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.964+0000 7f5b80a00640 1 -- 172.21.15.71:0/1730585528 shutdown_connections 2024-09-16T07:33:44.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.964+0000 7f5b80a00640 1 -- 172.21.15.71:0/1730585528 wait complete. 2024-09-16T07:33:44.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.966+0000 7f5b80a00640 1 Processor -- start 2024-09-16T07:33:44.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.966+0000 7f5b80a00640 1 -- start start 2024-09-16T07:33:44.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b80a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c074150 0x7f5b7c1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b80a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b7c0746a0 0x7f5b7c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b80a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b7c0b43f0 0x7f5b7c158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b80a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c073970 con 0x7f5b7c074150 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b80a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c0737f0 con 0x7f5b7c0746a0 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b80a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7c073af0 con 0x7f5b7c0b43f0 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c074150 0x7f5b7c1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c074150 0x7f5b7c1516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46602/0 (socket says 172.21.15.71:46602) 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79e00640 1 -- 172.21.15.71:0/3366615029 learned_addr learned my addr 172.21.15.71:0/3366615029 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79400640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b7c0746a0 0x7f5b7c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79e00640 1 -- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b7c0b43f0 msgr2=0x7f5b7c158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:44.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79e00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b7c0b43f0 0x7f5b7c158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79e00640 1 -- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b7c0746a0 msgr2=0x7f5b7c0c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79e00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b7c0746a0 0x7f5b7c0c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79e00640 1 -- 172.21.15.71:0/3366615029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c1f93d0 con 0x7f5b7c074150 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.967+0000 7f5b79400640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b7c0746a0 0x7f5b7c0c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.968+0000 7f5b79e00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c074150 0x7f5b7c1516a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c730 tx=0x7f5b6c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.968+0000 7f5b72000640 1 -- 172.21.15.71:0/3366615029 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c016020 con 0x7f5b7c074150 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.968+0000 7f5b72000640 1 -- 172.21.15.71:0/3366615029 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b6c00a950 con 0x7f5b7c074150 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.968+0000 7f5b72000640 1 -- 172.21.15.71:0/3366615029 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00ada0 con 0x7f5b7c074150 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.968+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c09be40 con 0x7f5b7c074150 2024-09-16T07:33:44.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.968+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c09c0e0 con 0x7f5b7c074150 2024-09-16T07:33:44.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.969+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c0b47d0 con 0x7f5b7c074150 2024-09-16T07:33:44.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.970+0000 7f5b72000640 1 -- 172.21.15.71:0/3366615029 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f5b6c019020 con 0x7f5b7c074150 2024-09-16T07:33:44.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.971+0000 7f5b72000640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b5c079e40 0x7f5b5c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:44.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.971+0000 7f5b72000640 1 -- 172.21.15.71:0/3366615029 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f5b6c099660 con 0x7f5b7c074150 2024-09-16T07:33:44.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.972+0000 7f5b79400640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b5c079e40 0x7f5b5c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.972+0000 7f5b79400640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b5c079e40 0x7f5b5c07c300 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5b68003eb0 tx=0x7f5b68009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:44.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:44.975+0000 7f5b72000640 1 -- 172.21.15.71:0/3366615029 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5b6c020030 con 0x7f5b7c074150 2024-09-16T07:33:45.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.182+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5b7c151c40 con 0x7f5b5c079e40 2024-09-16T07:33:45.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.185+0000 7f5b72000640 1 -- 172.21.15.71:0/3366615029 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5b7c151c40 con 0x7f5b5c079e40 2024-09-16T07:33:45.185 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:33:45.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:33:45.185 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:33:45.185 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:33:45.185 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b5c079e40 msgr2=0x7f5b5c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b5c079e40 0x7f5b5c07c300 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5b68003eb0 tx=0x7f5b68009040 comp rx=0 tx=0).stop 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c074150 msgr2=0x7f5b7c1516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c074150 0x7f5b7c1516a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c730 tx=0x7f5b6c00cbf0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 shutdown_connections 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5b5c079e40 0x7f5b5c07c300 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b7c0b43f0 0x7f5b7c158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b7c0746a0 0x7f5b7c0c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 --2- 172.21.15.71:0/3366615029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b7c074150 0x7f5b7c1516a0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 >> 172.21.15.71:0/3366615029 conn(0x7f5b7c0b1950 msgr2=0x7f5b7c0ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:45.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 shutdown_connections 2024-09-16T07:33:45.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.187+0000 7f5b80a00640 1 -- 172.21.15.71:0/3366615029 wait complete. 2024-09-16T07:33:45.203 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:33:45.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.363+0000 7f2856200640 1 -- 172.21.15.71:0/3752170941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 msgr2=0x7f28500cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.363+0000 7f2856200640 1 --2- 172.21.15.71:0/3752170941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 0x7f28500cdbb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f28500af780 tx=0x7f284802a820 comp rx=0 tx=0).stop 2024-09-16T07:33:45.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.363+0000 7f2856200640 1 -- 172.21.15.71:0/3752170941 shutdown_connections 2024-09-16T07:33:45.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.363+0000 7f2856200640 1 --2- 172.21.15.71:0/3752170941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 0x7f28500cdbb0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.363+0000 7f2856200640 1 --2- 172.21.15.71:0/3752170941 >> v2:172.21.15.177:3300/0 conn(0x7f28500cccd0 0x7f2850073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.363+0000 7f2856200640 1 -- 172.21.15.71:0/3752170941 >> 172.21.15.71:0/3752170941 conn(0x7f28500b1840 msgr2=0x7f28500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:45.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.363+0000 7f2856200640 1 -- 172.21.15.71:0/3752170941 shutdown_connections 2024-09-16T07:33:45.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.363+0000 7f2856200640 1 -- 172.21.15.71:0/3752170941 wait complete. 2024-09-16T07:33:45.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.365+0000 7f2856200640 1 Processor -- start 2024-09-16T07:33:45.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.365+0000 7f2856200640 1 -- start start 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f2856200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 0x7f28501f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f2856200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28500cccd0 0x7f28500b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f2856200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2850145750 0x7f28500c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f2856200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28500ce480 con 0x7f285006efa0 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f2856200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28500ce300 con 0x7f2850145750 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f2856200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f28500ce600 con 0x7f28500cccd0 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 0x7f28501f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 0x7f28501f3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46624/0 (socket says 172.21.15.71:46624) 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284f400640 1 -- 172.21.15.71:0/3255389276 learned_addr learned my addr 172.21.15.71:0/3255389276 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284fe00640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2850145750 0x7f28500c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:45.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f2847e00640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28500cccd0 0x7f28500b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:45.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284f400640 1 -- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28500cccd0 msgr2=0x7f28500b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284f400640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28500cccd0 0x7f28500b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284f400640 1 -- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2850145750 msgr2=0x7f28500c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284f400640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2850145750 0x7f28500c4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.366+0000 7f284f400640 1 -- 172.21.15.71:0/3255389276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28500b3fc0 con 0x7f285006efa0 2024-09-16T07:33:45.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.367+0000 7f284f400640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 0x7f28501f3e60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f284000c3d0 tx=0x7f284000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:45.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.367+0000 7f2847400640 1 -- 172.21.15.71:0/3255389276 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840011790 con 0x7f285006efa0 2024-09-16T07:33:45.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.367+0000 7f2847400640 1 -- 172.21.15.71:0/3255389276 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2840017070 con 0x7f285006efa0 2024-09-16T07:33:45.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.367+0000 7f2847400640 1 -- 172.21.15.71:0/3255389276 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840011cc0 con 0x7f285006efa0 2024-09-16T07:33:45.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.367+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28500b42a0 con 0x7f285006efa0 2024-09-16T07:33:45.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.367+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285013da70 con 0x7f285006efa0 2024-09-16T07:33:45.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.368+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28500cdbb0 con 0x7f285006efa0 2024-09-16T07:33:45.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.369+0000 7f2847400640 1 -- 172.21.15.71:0/3255389276 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f284001c460 con 0x7f285006efa0 2024-09-16T07:33:45.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.371+0000 7f2847400640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f282c079e40 0x7f282c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:45.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.371+0000 7f2847400640 1 -- 172.21.15.71:0/3255389276 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f28400996b0 con 0x7f285006efa0 2024-09-16T07:33:45.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.371+0000 7f2847e00640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f282c079e40 0x7f282c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:45.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.372+0000 7f2847e00640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f282c079e40 0x7f282c07c300 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f285013bff0 tx=0x7f283800b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:45.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.375+0000 7f2847400640 1 -- 172.21.15.71:0/3255389276 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2840099b50 con 0x7f285006efa0 2024-09-16T07:33:45.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.560+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f28500b12b0 con 0x7f285006efa0 2024-09-16T07:33:45.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.560+0000 7f2847400640 1 -- 172.21.15.71:0/3255389276 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f2840062d10 con 0x7f285006efa0 2024-09-16T07:33:45.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f282c079e40 msgr2=0x7f282c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f282c079e40 0x7f282c07c300 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f285013bff0 tx=0x7f283800b390 comp rx=0 tx=0).stop 2024-09-16T07:33:45.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 msgr2=0x7f28501f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 0x7f28501f3e60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f284000c3d0 tx=0x7f284000c890 comp rx=0 tx=0).stop 2024-09-16T07:33:45.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 shutdown_connections 2024-09-16T07:33:45.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f282c079e40 0x7f282c07c300 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2850145750 0x7f28500c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28500cccd0 0x7f28500b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 --2- 172.21.15.71:0/3255389276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f285006efa0 0x7f28501f3e60 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.562+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 >> 172.21.15.71:0/3255389276 conn(0x7f28500b1840 msgr2=0x7f285015eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:45.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.563+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 shutdown_connections 2024-09-16T07:33:45.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.563+0000 7f2856200640 1 -- 172.21.15.71:0/3255389276 wait complete. 2024-09-16T07:33:45.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:33:45.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.739+0000 7fd9ac200640 1 -- 172.21.15.71:0/3722558014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd98c003c10 msgr2=0x7fd98c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.739+0000 7fd9ac200640 1 --2- 172.21.15.71:0/3722558014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd98c003c10 0x7fd98c004010 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd9a406db90 tx=0x7fd98c005830 comp rx=0 tx=0).stop 2024-09-16T07:33:45.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.739+0000 7fd9aa200640 1 -- 172.21.15.71:0/3722558014 reap_dead start 2024-09-16T07:33:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.739+0000 7fd9ac200640 1 -- 172.21.15.71:0/3722558014 shutdown_connections 2024-09-16T07:33:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.739+0000 7fd9ac200640 1 -- 172.21.15.71:0/3722558014 >> 172.21.15.71:0/3722558014 conn(0x7fd9a40b1840 msgr2=0x7fd9a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.740+0000 7fd9ac200640 1 -- 172.21.15.71:0/3722558014 shutdown_connections 2024-09-16T07:33:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.740+0000 7fd9ac200640 1 -- 172.21.15.71:0/3722558014 wait complete. 2024-09-16T07:33:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.741+0000 7fd9ac200640 1 Processor -- start 2024-09-16T07:33:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.741+0000 7fd9ac200640 1 -- start start 2024-09-16T07:33:45.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9ac200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd98c003c10 0x7fd9a414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9ac200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a414e690 0x7fd9a41f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9ac200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9a40cb080 0x7fd9a40733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9ac200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a40ce620 con 0x7fd9a414e690 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9ac200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a40ce4a0 con 0x7fd98c003c10 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9ac200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a40ce7a0 con 0x7fd9a40cb080 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a414e690 0x7fd9a41f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a414e690 0x7fd9a41f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46630/0 (socket says 172.21.15.71:46630) 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.742+0000 7fd9a8e00640 1 -- 172.21.15.71:0/2676869440 learned_addr learned my addr 172.21.15.71:0/2676869440 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a9800640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd98c003c10 0x7fd9a414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9aa200640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9a40cb080 0x7fd9a40733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:45.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a8e00640 1 -- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9a40cb080 msgr2=0x7fd9a40733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a8e00640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9a40cb080 0x7fd9a40733e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a8e00640 1 -- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd98c003c10 msgr2=0x7fd9a414bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a8e00640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd98c003c10 0x7fd9a414bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a8e00640 1 -- 172.21.15.71:0/2676869440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a414c310 con 0x7fd9a414e690 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a8e00640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a414e690 0x7fd9a41f52b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd990004d40 tx=0x7fd990001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a1600640 1 -- 172.21.15.71:0/2676869440 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900107d0 con 0x7fd9a414e690 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a1600640 1 -- 172.21.15.71:0/2676869440 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd990010a80 con 0x7fd9a414e690 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9a1600640 1 -- 172.21.15.71:0/2676869440 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990011460 con 0x7fd9a414e690 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a414ebc0 con 0x7fd9a414e690 2024-09-16T07:33:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.743+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a414f180 con 0x7fd9a414e690 2024-09-16T07:33:45.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.744+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a4162c20 con 0x7fd9a414e690 2024-09-16T07:33:45.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.745+0000 7fd9a1600640 1 -- 172.21.15.71:0/2676869440 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fd990011600 con 0x7fd9a414e690 2024-09-16T07:33:45.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.746+0000 7fd9a1600640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd984079e70 0x7fd98407c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.746+0000 7fd9a1600640 1 -- 172.21.15.71:0/2676869440 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fd99009a440 con 0x7fd9a414e690 2024-09-16T07:33:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.747+0000 7fd9a9800640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd984079e70 0x7fd98407c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:45.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.747+0000 7fd9a9800640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd984079e70 0x7fd98407c330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd998035000 tx=0x7fd99802d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:45.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.750+0000 7fd9a1600640 1 -- 172.21.15.71:0/2676869440 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd990064150 con 0x7fd9a414e690 2024-09-16T07:33:45.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.938+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fd9a40b1140 con 0x7fd9a414e690 2024-09-16T07:33:45.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.939+0000 7fd9a1600640 1 -- 172.21.15.71:0/2676869440 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v45) ==== 116+0+2 (secure 0 0 0) 0x7fd990063a50 con 0x7fd9a414e690 2024-09-16T07:33:45.940 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd984079e70 msgr2=0x7fd98407c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd984079e70 0x7fd98407c330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd998035000 tx=0x7fd99802d040 comp rx=0 tx=0).stop 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a414e690 msgr2=0x7fd9a41f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a414e690 0x7fd9a41f52b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd990004d40 tx=0x7fd990001800 comp rx=0 tx=0).stop 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 shutdown_connections 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd984079e70 0x7fd98407c330 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9a40cb080 0x7fd9a40733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a414e690 0x7fd9a41f52b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 --2- 172.21.15.71:0/2676869440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd98c003c10 0x7fd9a414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:45.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 >> 172.21.15.71:0/2676869440 conn(0x7fd9a40b1840 msgr2=0x7fd9a40cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:45.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 shutdown_connections 2024-09-16T07:33:45.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:45.941+0000 7fd9ac200640 1 -- 172.21.15.71:0/2676869440 wait complete. 2024-09-16T07:33:45.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:33:46.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.117+0000 7f8f4d400640 1 -- 172.21.15.71:0/3230036558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b4e70 msgr2=0x7f8f481625b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.117+0000 7f8f4d400640 1 --2- 172.21.15.71:0/3230036558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b4e70 0x7f8f481625b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009ca0 tx=0x7f8f4002ab50 comp rx=0 tx=0).stop 2024-09-16T07:33:46.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.118+0000 7f8f4d400640 1 -- 172.21.15.71:0/3230036558 shutdown_connections 2024-09-16T07:33:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.118+0000 7f8f4d400640 1 --2- 172.21.15.71:0/3230036558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b4e70 0x7f8f481625b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.118+0000 7f8f4d400640 1 --2- 172.21.15.71:0/3230036558 >> v2:172.21.15.177:3300/0 conn(0x7f8f480d2000 0x7f8f480d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.118+0000 7f8f4d400640 1 -- 172.21.15.71:0/3230036558 >> 172.21.15.71:0/3230036558 conn(0x7f8f480d3e20 msgr2=0x7f8f480d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.118+0000 7f8f4d400640 1 -- 172.21.15.71:0/3230036558 shutdown_connections 2024-09-16T07:33:46.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.118+0000 7f8f4d400640 1 -- 172.21.15.71:0/3230036558 wait complete. 2024-09-16T07:33:46.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.119+0000 7f8f4d400640 1 Processor -- start 2024-09-16T07:33:46.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.119+0000 7f8f4d400640 1 -- start start 2024-09-16T07:33:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.120+0000 7f8f4d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b4e70 0x7f8f4815fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.120+0000 7f8f4d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f480d2000 0x7f8f480b9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.120+0000 7f8f4d400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480d2920 0x7f8f481593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.120+0000 7f8f4d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f480b1a10 con 0x7f8f480b4e70 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.120+0000 7f8f4d400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f480b1890 con 0x7f8f480d2000 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.120+0000 7f8f4d400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8f480b1b90 con 0x7f8f480d2920 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b4e70 0x7f8f4815fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b4e70 0x7f8f4815fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46642/0 (socket says 172.21.15.71:46642) 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 -- 172.21.15.71:0/1664828751 learned_addr learned my addr 172.21.15.71:0/1664828751 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f480d2000 0x7f8f480b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f4ca00640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480d2920 0x7f8f481593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 -- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480d2920 msgr2=0x7f8f481593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480d2920 0x7f8f481593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 -- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f480d2000 msgr2=0x7f8f480b9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f480d2000 0x7f8f480b9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 -- 172.21.15.71:0/1664828751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4815a1a0 con 0x7f8f480b4e70 2024-09-16T07:33:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.121+0000 7f8f47e00640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b4e70 0x7f8f4815fed0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800e9f0 tx=0x7f8f3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.122+0000 7f8f3fe00640 1 -- 172.21.15.71:0/1664828751 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3800cd80 con 0x7f8f480b4e70 2024-09-16T07:33:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.122+0000 7f8f3fe00640 1 -- 172.21.15.71:0/1664828751 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f38010430 con 0x7f8f480b4e70 2024-09-16T07:33:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.122+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4815a480 con 0x7f8f480b4e70 2024-09-16T07:33:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.122+0000 7f8f3fe00640 1 -- 172.21.15.71:0/1664828751 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38010840 con 0x7f8f480b4e70 2024-09-16T07:33:46.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.122+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f480b6df0 con 0x7f8f480b4e70 2024-09-16T07:33:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.123+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f480d23e0 con 0x7f8f480b4e70 2024-09-16T07:33:46.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.124+0000 7f8f3fe00640 1 -- 172.21.15.71:0/1664828751 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7f8f380109e0 con 0x7f8f480b4e70 2024-09-16T07:33:46.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.125+0000 7f8f3fe00640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f28079e40 0x7f8f2807c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.126+0000 7f8f3fe00640 1 -- 172.21.15.71:0/1664828751 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f8f3809a5a0 con 0x7f8f480b4e70 2024-09-16T07:33:46.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.126+0000 7f8f47400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f28079e40 0x7f8f2807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.127+0000 7f8f47400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f28079e40 0x7f8f2807c300 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8f480d28d0 tx=0x7f8f34009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:46.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.130+0000 7f8f3fe00640 1 -- 172.21.15.71:0/1664828751 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8f3809aa40 con 0x7f8f480b4e70 2024-09-16T07:33:46.182 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:33:46.187 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:33:46.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.325+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f8f480b3c00 con 0x7f8f480b4e70 2024-09-16T07:33:46.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.329+0000 7f8f3fe00640 1 -- 172.21.15.71:0/1664828751 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7f8f38063cc0 con 0x7f8f480b4e70 2024-09-16T07:33:46.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f28079e40 msgr2=0x7f8f2807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f28079e40 0x7f8f2807c300 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8f480d28d0 tx=0x7f8f34009040 comp rx=0 tx=0).stop 2024-09-16T07:33:46.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b4e70 msgr2=0x7f8f4815fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b4e70 0x7f8f4815fed0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800e9f0 tx=0x7f8f3800eeb0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 shutdown_connections 2024-09-16T07:33:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f28079e40 0x7f8f2807c300 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480d2920 0x7f8f481593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f480d2000 0x7f8f480b9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 --2- 172.21.15.71:0/1664828751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b4e70 0x7f8f4815fed0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 >> 172.21.15.71:0/1664828751 conn(0x7f8f480d3e20 msgr2=0x7f8f4806e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:46.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 shutdown_connections 2024-09-16T07:33:46.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.332+0000 7f8f4d400640 1 -- 172.21.15.71:0/1664828751 wait complete. 2024-09-16T07:33:46.349 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.349+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.349 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.349+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.349 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.349+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.349 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.349+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.349 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.349+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.351 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T07:33:46.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.511+0000 7ff841400640 1 -- 172.21.15.71:0/2403443563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c158900 msgr2=0x7ff83c15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.511+0000 7ff841400640 1 --2- 172.21.15.71:0/2403443563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c158900 0x7ff83c15b1c0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff82c005fd0 tx=0x7ff82c02aa90 comp rx=0 tx=0).stop 2024-09-16T07:33:46.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.511+0000 7ff841400640 1 -- 172.21.15.71:0/2403443563 shutdown_connections 2024-09-16T07:33:46.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.511+0000 7ff841400640 1 --2- 172.21.15.71:0/2403443563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff83c15b700 0x7ff83c0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.511+0000 7ff841400640 1 --2- 172.21.15.71:0/2403443563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c158900 0x7ff83c15b1c0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.511+0000 7ff841400640 1 -- 172.21.15.71:0/2403443563 >> 172.21.15.71:0/2403443563 conn(0x7ff83c0b18e0 msgr2=0x7ff83c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:46.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.511+0000 7ff841400640 1 -- 172.21.15.71:0/2403443563 shutdown_connections 2024-09-16T07:33:46.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.511+0000 7ff841400640 1 -- 172.21.15.71:0/2403443563 wait complete. 2024-09-16T07:33:46.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.513+0000 7ff841400640 1 Processor -- start 2024-09-16T07:33:46.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.513+0000 7ff841400640 1 -- start start 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff841400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff83c0cd1e0 0x7ff83c1f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff841400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff83c158900 0x7ff83c09c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff841400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c15b700 0x7ff83c09d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff841400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0ced60 con 0x7ff83c15b700 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff841400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0cebe0 con 0x7ff83c158900 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff841400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c0ceee0 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff83b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c15b700 0x7ff83c09d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff83a800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff83c0cd1e0 0x7ff83c1f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff83a800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff83c0cd1e0 0x7ff83c1f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:57834/0 (socket says 172.21.15.71:57834) 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff83b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c15b700 0x7ff83c09d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46664/0 (socket says 172.21.15.71:46664) 2024-09-16T07:33:46.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff83a800640 1 -- 172.21.15.71:0/738868236 learned_addr learned my addr 172.21.15.71:0/738868236 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.514+0000 7ff839e00640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff83c158900 0x7ff83c09c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff83a800640 1 -- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff83c158900 msgr2=0x7ff83c09c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff83a800640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff83c158900 0x7ff83c09c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff83a800640 1 -- 172.21.15.71:0/738868236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c15b700 msgr2=0x7ff83c09d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff83a800640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c15b700 0x7ff83c09d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff83a800640 1 -- 172.21.15.71:0/738868236 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c09db10 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff83b200640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c15b700 0x7ff83c09d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff83a800640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff83c0cd1e0 0x7ff83c1f2350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000f80 tx=0x7ff82c003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff832a00640 1 -- 172.21.15.71:0/738868236 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c046070 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff832a00640 1 -- 172.21.15.71:0/738868236 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff82c004250 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff832a00640 1 -- 172.21.15.71:0/738868236 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c004660 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c09ddf0 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.515+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c09beb0 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.516+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c069360 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.518+0000 7ff832a00640 1 -- 172.21.15.71:0/738868236 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7ff82c049070 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.519+0000 7ff832a00640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff81c079e40 0x7ff81c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.519+0000 7ff832a00640 1 -- 172.21.15.71:0/738868236 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7ff82c0ba860 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.519+0000 7ff839e00640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff81c079e40 0x7ff81c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.520+0000 7ff839e00640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff81c079e40 0x7ff81c07c300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff83c14bf40 tx=0x7ff828005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:46.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.523+0000 7ff832a00640 1 -- 172.21.15.71:0/738868236 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff82c032030 con 0x7ff83c0cd1e0 2024-09-16T07:33:46.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.734+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7ff83c0b3d60 con 0x7ff81c079e40 2024-09-16T07:33:46.782 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.745+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.782 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.745+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.782 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.745+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.782 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.745+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.782 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:33:46.745+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:33:46.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.791+0000 7ff832a00640 1 -- 172.21.15.71:0/738868236 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff83c0b3d60 con 0x7ff81c079e40 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff81c079e40 msgr2=0x7ff81c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff81c079e40 0x7ff81c07c300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff83c14bf40 tx=0x7ff828005f90 comp rx=0 tx=0).stop 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff83c0cd1e0 msgr2=0x7ff83c1f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff83c0cd1e0 0x7ff83c1f2350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff82c000f80 tx=0x7ff82c003ce0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 shutdown_connections 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff81c079e40 0x7ff81c07c300 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff83c15b700 0x7ff83c09d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff83c158900 0x7ff83c09c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 --2- 172.21.15.71:0/738868236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff83c0cd1e0 0x7ff83c1f2350 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 >> 172.21.15.71:0/738868236 conn(0x7ff83c0b18e0 msgr2=0x7ff83c158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 shutdown_connections 2024-09-16T07:33:46.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.794+0000 7ff841400640 1 -- 172.21.15.71:0/738868236 wait complete. 2024-09-16T07:33:46.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:33:46.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.970+0000 7fe914a00640 1 -- 172.21.15.71:0/2372259076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9100b2810 msgr2=0x7fe910073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.970+0000 7fe914a00640 1 --2- 172.21.15.71:0/2372259076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9100b2810 0x7fe910073d40 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8005fd0 tx=0x7fe8f802a5d0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.970+0000 7fe914a00640 1 -- 172.21.15.71:0/2372259076 shutdown_connections 2024-09-16T07:33:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.970+0000 7fe914a00640 1 --2- 172.21.15.71:0/2372259076 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe910074280 0x7fe91006ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.970+0000 7fe914a00640 1 --2- 172.21.15.71:0/2372259076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9100b2810 0x7fe910073d40 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.970+0000 7fe914a00640 1 -- 172.21.15.71:0/2372259076 >> 172.21.15.71:0/2372259076 conn(0x7fe9100b1510 msgr2=0x7fe9100b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.970+0000 7fe914a00640 1 -- 172.21.15.71:0/2372259076 shutdown_connections 2024-09-16T07:33:46.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.970+0000 7fe914a00640 1 -- 172.21.15.71:0/2372259076 wait complete. 2024-09-16T07:33:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.971+0000 7fe914a00640 1 Processor -- start 2024-09-16T07:33:46.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.971+0000 7fe914a00640 1 -- start start 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.972+0000 7fe914a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe910074280 0x7fe91015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.972+0000 7fe914a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9100b2810 0x7fe9100c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.972+0000 7fe914a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe91015bb90 0x7fe91015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.972+0000 7fe914a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe91006f2a0 con 0x7fe91015bb90 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.972+0000 7fe914a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe91015bf70 con 0x7fe9100b2810 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.972+0000 7fe914a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe91006f420 con 0x7fe910074280 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe91015bb90 0x7fe91015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe91015bb90 0x7fe91015e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46684/0 (socket says 172.21.15.71:46684) 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 -- 172.21.15.71:0/347537084 learned_addr learned my addr 172.21.15.71:0/347537084 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe907e00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9100b2810 0x7fe9100c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90f400640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe910074280 0x7fe91015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 -- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe910074280 msgr2=0x7fe91015a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe910074280 0x7fe91015a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 -- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9100b2810 msgr2=0x7fe9100c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9100b2810 0x7fe9100c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 -- 172.21.15.71:0/347537084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91015aff0 con 0x7fe91015bb90 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90f400640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe910074280 0x7fe91015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.973+0000 7fe90fe00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe91015bb90 0x7fe91015e5e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe90800eb40 tx=0x7fe90800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.974+0000 7fe907400640 1 -- 172.21.15.71:0/347537084 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908004b20 con 0x7fe91015bb90 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.974+0000 7fe907400640 1 -- 172.21.15.71:0/347537084 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe908004ce0 con 0x7fe91015bb90 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.974+0000 7fe907400640 1 -- 172.21.15.71:0/347537084 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908005280 con 0x7fe91015bb90 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.974+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91015a860 con 0x7fe91015bb90 2024-09-16T07:33:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.974+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9100c4290 con 0x7fe91015bb90 2024-09-16T07:33:46.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.975+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91006f940 con 0x7fe91015bb90 2024-09-16T07:33:46.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.976+0000 7fe907400640 1 -- 172.21.15.71:0/347537084 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 37) ==== 103276+0+0 (secure 0 0 0) 0x7fe908005420 con 0x7fe91015bb90 2024-09-16T07:33:46.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.977+0000 7fe907400640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8ec079e40 0x7fe8ec07c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:33:46.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.977+0000 7fe907400640 1 -- 172.21.15.71:0/347537084 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fe90809a700 con 0x7fe91015bb90 2024-09-16T07:33:46.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.977+0000 7fe90f400640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8ec079e40 0x7fe8ec07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:33:46.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.978+0000 7fe90f400640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8ec079e40 0x7fe8ec07c300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe9100c6470 tx=0x7fe900009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:33:46.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:46.981+0000 7fe907400640 1 -- 172.21.15.71:0/347537084 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe908064410 con 0x7fe91015bb90 2024-09-16T07:33:47.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.193+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe910069d20 con 0x7fe8ec079e40 2024-09-16T07:33:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.199+0000 7fe907400640 1 -- 172.21.15.71:0/347537084 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fe910069d20 con 0x7fe8ec079e40 2024-09-16T07:33:47.200 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:33:47.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.201+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8ec079e40 msgr2=0x7fe8ec07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:47.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.201+0000 7fe914a00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8ec079e40 0x7fe8ec07c300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe9100c6470 tx=0x7fe900009040 comp rx=0 tx=0).stop 2024-09-16T07:33:47.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.201+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe91015bb90 msgr2=0x7fe91015e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:33:47.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.201+0000 7fe914a00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe91015bb90 0x7fe91015e5e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe90800eb40 tx=0x7fe90800c2d0 comp rx=0 tx=0).stop 2024-09-16T07:33:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.202+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 shutdown_connections 2024-09-16T07:33:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.202+0000 7fe914a00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe8ec079e40 0x7fe8ec07c300 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.202+0000 7fe914a00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe91015bb90 0x7fe91015e5e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.202+0000 7fe914a00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9100b2810 0x7fe9100c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.202+0000 7fe914a00640 1 --2- 172.21.15.71:0/347537084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe910074280 0x7fe91015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:33:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.202+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 >> 172.21.15.71:0/347537084 conn(0x7fe9100b1510 msgr2=0x7fe9100b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:33:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.202+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 shutdown_connections 2024-09-16T07:33:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:33:47.202+0000 7fe914a00640 1 -- 172.21.15.71:0/347537084 wait complete. 2024-09-16T07:33:47.217 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:33:47.218 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:33:47.218 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:47.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:47.254 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:33:54.737 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:33:54.738 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:33:54.738 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.39609 s, 145 MB/s 2024-09-16T07:33:54.739 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:33:54.739 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:33:54.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:33:54.775 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:34:02.254 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:34:02.254 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:34:02.254 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.40249 s, 145 MB/s 2024-09-16T07:34:02.255 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:34:02.255 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:02.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:02.292 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:34:09.649 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:34:09.649 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:34:09.650 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.28029 s, 147 MB/s 2024-09-16T07:34:09.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T07:34:09.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.814+0000 7f879b800640 1 -- 172.21.15.71:0/3981296512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879415b070 msgr2=0x7f87940727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:09.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.814+0000 7f879b800640 1 --2- 172.21.15.71:0/3981296512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879415b070 0x7f87940727c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f87940cc510 tx=0x7f878402aaa0 comp rx=0 tx=0).stop 2024-09-16T07:34:09.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.814+0000 7f879b800640 1 -- 172.21.15.71:0/3981296512 shutdown_connections 2024-09-16T07:34:09.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.814+0000 7f879b800640 1 --2- 172.21.15.71:0/3981296512 >> v2:172.21.15.177:3300/0 conn(0x7f87940b2be0 0x7f87940b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:09.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.814+0000 7f879b800640 1 --2- 172.21.15.71:0/3981296512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879415b070 0x7f87940727c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:09.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.814+0000 7f879b800640 1 -- 172.21.15.71:0/3981296512 >> 172.21.15.71:0/3981296512 conn(0x7f87940b18e0 msgr2=0x7f87940b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:09.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.814+0000 7f879b800640 1 -- 172.21.15.71:0/3981296512 shutdown_connections 2024-09-16T07:34:09.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.814+0000 7f879b800640 1 -- 172.21.15.71:0/3981296512 wait complete. 2024-09-16T07:34:09.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.816+0000 7f879b800640 1 Processor -- start 2024-09-16T07:34:09.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.816+0000 7f879b800640 1 -- start start 2024-09-16T07:34:09.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f879b800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87940b2be0 0x7f87940cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f879b800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87940b4b80 0x7f87940acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f879b800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879414f8b0 0x7f87940ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f879b800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f879406ea80 con 0x7f879414f8b0 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f879b800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f879406e900 con 0x7f87940b4b80 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f879b800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f879406ec00 con 0x7f87940b2be0 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f8799800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879414f8b0 0x7f87940ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f8799800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879414f8b0 0x7f87940ad270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51520/0 (socket says 172.21.15.71:51520) 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.817+0000 7f8799800640 1 -- 172.21.15.71:0/3466117671 learned_addr learned my addr 172.21.15.71:0/3466117671 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8793e00640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87940b4b80 0x7f87940acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8798e00640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87940b2be0 0x7f87940cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8799800640 1 -- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87940b2be0 msgr2=0x7f87940cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:09.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8799800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87940b2be0 0x7f87940cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8799800640 1 -- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87940b4b80 msgr2=0x7f87940acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8799800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87940b4b80 0x7f87940acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8799800640 1 -- 172.21.15.71:0/3466117671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87940ce320 con 0x7f879414f8b0 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8799800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879414f8b0 0x7f87940ad270 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f878c009fd0 tx=0x7f878c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8790c00640 1 -- 172.21.15.71:0/3466117671 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c011830 con 0x7f879414f8b0 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8790c00640 1 -- 172.21.15.71:0/3466117671 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f878c016070 con 0x7f879414f8b0 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f8790c00640 1 -- 172.21.15.71:0/3466117671 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c011d40 con 0x7f879414f8b0 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87940ca930 con 0x7f879414f8b0 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.818+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87940caef0 con 0x7f879414f8b0 2024-09-16T07:34:09.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.820+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87940cc010 con 0x7f879414f8b0 2024-09-16T07:34:09.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.821+0000 7f8790c00640 1 -- 172.21.15.71:0/3466117671 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 39) ==== 103378+0+0 (secure 0 0 0) 0x7f878c01b460 con 0x7f879414f8b0 2024-09-16T07:34:09.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.822+0000 7f8790c00640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8774079f60 0x7f877407c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:09.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.822+0000 7f8790c00640 1 -- 172.21.15.71:0/3466117671 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f878c099400 con 0x7f879414f8b0 2024-09-16T07:34:09.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.822+0000 7f8798e00640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8774079f60 0x7f877407c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:09.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.823+0000 7f8798e00640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8774079f60 0x7f877407c420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8784035000 tx=0x7f8784038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:09.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:09.826+0000 7f8790c00640 1 -- 172.21.15.71:0/3466117671 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f878c0634d0 con 0x7f879414f8b0 2024-09-16T07:34:10.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.037+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f8794069a50 con 0x7f8774079f60 2024-09-16T07:34:10.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.051+0000 7f8790c00640 1 -- 172.21.15.71:0/3466117671 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8794069a50 con 0x7f8774079f60 2024-09-16T07:34:10.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8774079f60 msgr2=0x7f877407c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:10.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8774079f60 0x7f877407c420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8784035000 tx=0x7f8784038040 comp rx=0 tx=0).stop 2024-09-16T07:34:10.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879414f8b0 msgr2=0x7f87940ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:10.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879414f8b0 0x7f87940ad270 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f878c009fd0 tx=0x7f878c00b8e0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 shutdown_connections 2024-09-16T07:34:10.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8774079f60 0x7f877407c420 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879414f8b0 0x7f87940ad270 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87940b4b80 0x7f87940acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 --2- 172.21.15.71:0/3466117671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87940b2be0 0x7f87940cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 >> 172.21.15.71:0/3466117671 conn(0x7f87940b18e0 msgr2=0x7f8794155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:10.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 shutdown_connections 2024-09-16T07:34:10.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.054+0000 7f879b800640 1 -- 172.21.15.71:0/3466117671 wait complete. 2024-09-16T07:34:10.069 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:34:10.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 -- 172.21.15.71:0/100886867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 msgr2=0x7f37c00cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:10.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 --2- 172.21.15.71:0/100886867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 0x7f37c00cc5c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f37c00cc550 tx=0x7f37ac02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 -- 172.21.15.71:0/100886867 shutdown_connections 2024-09-16T07:34:10.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 --2- 172.21.15.71:0/100886867 >> v2:172.21.15.177:3300/0 conn(0x7f37c00ccb00 0x7f37c015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 --2- 172.21.15.71:0/100886867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 0x7f37c00cc5c0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 --2- 172.21.15.71:0/100886867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c0145750 0x7f37c0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 -- 172.21.15.71:0/100886867 >> 172.21.15.71:0/100886867 conn(0x7f37c00b1840 msgr2=0x7f37c00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:10.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 -- 172.21.15.71:0/100886867 shutdown_connections 2024-09-16T07:34:10.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.230+0000 7f37c8600640 1 -- 172.21.15.71:0/100886867 wait complete. 2024-09-16T07:34:10.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.232+0000 7f37c8600640 1 Processor -- start 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.232+0000 7f37c8600640 1 -- start start 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c8600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 0x7f37c00c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c8600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c00ccb00 0x7f37c01f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c8600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c0145750 0x7f37c01f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c8600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01600b0 con 0x7f37c006efa0 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c8600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37c015ff30 con 0x7f37c0145750 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c8600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f37c0160230 con 0x7f37c00ccb00 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 0x7f37c00c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 0x7f37c00c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51532/0 (socket says 172.21.15.71:51532) 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5c00640 1 -- 172.21.15.71:0/1066837361 learned_addr learned my addr 172.21.15.71:0/1066837361 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5200640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c00ccb00 0x7f37c01f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:10.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c6600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c0145750 0x7f37c01f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5c00640 1 -- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c00ccb00 msgr2=0x7f37c01f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5c00640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c00ccb00 0x7f37c01f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5c00640 1 -- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c0145750 msgr2=0x7f37c01f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5c00640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c0145750 0x7f37c01f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c5c00640 1 -- 172.21.15.71:0/1066837361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c0151da0 con 0x7f37c006efa0 2024-09-16T07:34:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.233+0000 7f37c6600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c0145750 0x7f37c01f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.234+0000 7f37c5c00640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 0x7f37c00c7300 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f37b400e7f0 tx=0x7f37b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.234+0000 7f37bd600640 1 -- 172.21.15.71:0/1066837361 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b400ccc0 con 0x7f37c006efa0 2024-09-16T07:34:10.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.234+0000 7f37bd600640 1 -- 172.21.15.71:0/1066837361 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37b400ce80 con 0x7f37c006efa0 2024-09-16T07:34:10.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.234+0000 7f37bd600640 1 -- 172.21.15.71:0/1066837361 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4005280 con 0x7f37c006efa0 2024-09-16T07:34:10.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.234+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c0152080 con 0x7f37c006efa0 2024-09-16T07:34:10.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.234+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c00ac9b0 con 0x7f37c006efa0 2024-09-16T07:34:10.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.235+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0160710 con 0x7f37c006efa0 2024-09-16T07:34:10.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.236+0000 7f37bd600640 1 -- 172.21.15.71:0/1066837361 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 39) ==== 103378+0+0 (secure 0 0 0) 0x7f37b4005420 con 0x7f37c006efa0 2024-09-16T07:34:10.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.238+0000 7f37bd600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f37a0079f60 0x7f37a007c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:10.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.238+0000 7f37bd600640 1 -- 172.21.15.71:0/1066837361 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f37b4099fe0 con 0x7f37c006efa0 2024-09-16T07:34:10.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.238+0000 7f37c5200640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f37a0079f60 0x7f37a007c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:10.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.239+0000 7f37c5200640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f37a0079f60 0x7f37a007c420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f37ac033000 tx=0x7f37ac004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:10.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.242+0000 7f37bd600640 1 -- 172.21.15.71:0/1066837361 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f37b40640b0 con 0x7f37c006efa0 2024-09-16T07:34:10.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.454+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f37c0069a10 con 0x7f37a0079f60 2024-09-16T07:34:10.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.459+0000 7f37bd600640 1 -- 172.21.15.71:0/1066837361 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f37c0069a10 con 0x7f37a0079f60 2024-09-16T07:34:10.459 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:34:10.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.461+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f37a0079f60 msgr2=0x7f37a007c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:10.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.461+0000 7f37c8600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f37a0079f60 0x7f37a007c420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f37ac033000 tx=0x7f37ac004730 comp rx=0 tx=0).stop 2024-09-16T07:34:10.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.461+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 msgr2=0x7f37c00c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:10.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.461+0000 7f37c8600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 0x7f37c00c7300 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f37b400e7f0 tx=0x7f37b400ecb0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.462+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 shutdown_connections 2024-09-16T07:34:10.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.462+0000 7f37c8600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f37a0079f60 0x7f37a007c420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.462+0000 7f37c8600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c0145750 0x7f37c01f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.462+0000 7f37c8600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c00ccb00 0x7f37c01f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.462+0000 7f37c8600640 1 --2- 172.21.15.71:0/1066837361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c006efa0 0x7f37c00c7300 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:10.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.462+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 >> 172.21.15.71:0/1066837361 conn(0x7f37c00b1840 msgr2=0x7f37c0142050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:10.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.462+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 shutdown_connections 2024-09-16T07:34:10.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:10.462+0000 7f37c8600640 1 -- 172.21.15.71:0/1066837361 wait complete. 2024-09-16T07:34:10.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:10.477 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:34:10.478 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:34:10.513 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:34:10.513 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T07:34:10.513 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:10.514 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T07:34:12.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:12.515 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:34:12.515 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:34:12.555 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:34:12.556 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T07:34:12.556 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:12.557 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T07:34:14.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:14.557 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:34:14.557 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:34:14.598 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/cfe33f48-4355-4eac-a6c1-df7f7cda873d 2024-09-16T07:34:14.598 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T07:34:14.598 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:14.599 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T07:34:14.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-16T07:34:14.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.810+0000 7fd972200640 1 -- 172.21.15.71:0/1386335361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 msgr2=0x7fd96c0c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.810+0000 7fd972200640 1 --2- 172.21.15.71:0/1386335361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 0x7fd96c0c6430 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd96c072300 tx=0x7fd96002a540 comp rx=0 tx=0).stop 2024-09-16T07:34:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.810+0000 7fd972200640 1 -- 172.21.15.71:0/1386335361 shutdown_connections 2024-09-16T07:34:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.810+0000 7fd972200640 1 --2- 172.21.15.71:0/1386335361 >> v2:172.21.15.177:3300/0 conn(0x7fd96c0c5730 0x7fd96c15fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.810+0000 7fd972200640 1 --2- 172.21.15.71:0/1386335361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 0x7fd96c0c6430 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.810+0000 7fd972200640 1 -- 172.21.15.71:0/1386335361 >> 172.21.15.71:0/1386335361 conn(0x7fd96c0cc490 msgr2=0x7fd96c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.810+0000 7fd972200640 1 -- 172.21.15.71:0/1386335361 shutdown_connections 2024-09-16T07:34:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.810+0000 7fd972200640 1 -- 172.21.15.71:0/1386335361 wait complete. 2024-09-16T07:34:14.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.811+0000 7fd972200640 1 Processor -- start 2024-09-16T07:34:14.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.811+0000 7fd972200640 1 -- start start 2024-09-16T07:34:14.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.812+0000 7fd972200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c0c5730 0x7fd96c0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:14.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.812+0000 7fd972200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd96c15c0f0 0x7fd96c14f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:14.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.812+0000 7fd972200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 0x7fd96c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.812+0000 7fd972200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c0cd2c0 con 0x7fd96c15f670 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.812+0000 7fd972200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c0cd140 con 0x7fd96c0c5730 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.812+0000 7fd972200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c0ca370 con 0x7fd96c15c0f0 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 0x7fd96c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 0x7fd96c153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51458/0 (socket says 172.21.15.71:51458) 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 -- 172.21.15.71:0/1402112349 learned_addr learned my addr 172.21.15.71:0/1402112349 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd970e00640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c0c5730 0x7fd96c0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd96be00640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd96c15c0f0 0x7fd96c14f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 -- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd96c15c0f0 msgr2=0x7fd96c14f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd96c15c0f0 0x7fd96c14f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:14.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 -- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c0c5730 msgr2=0x7fd96c0c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:14.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c0c5730 0x7fd96c0c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:14.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 -- 172.21.15.71:0/1402112349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c0b1da0 con 0x7fd96c15f670 2024-09-16T07:34:14.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd971800640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 0x7fd96c153b20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd96400cbf0 tx=0x7fd96400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:14.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.813+0000 7fd968c00640 1 -- 172.21.15.71:0/1402112349 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964016020 con 0x7fd96c15f670 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.814+0000 7fd968c00640 1 -- 172.21.15.71:0/1402112349 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd96400add0 con 0x7fd96c15f670 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.814+0000 7fd968c00640 1 -- 172.21.15.71:0/1402112349 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964012680 con 0x7fd96c15f670 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.814+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c0b2040 con 0x7fd96c15f670 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.814+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c0ce240 con 0x7fd96c15f670 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.815+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96c15ce00 con 0x7fd96c15f670 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.816+0000 7fd968c00640 1 -- 172.21.15.71:0/1402112349 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 39) ==== 103378+0+0 (secure 0 0 0) 0x7fd964019020 con 0x7fd96c15f670 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.817+0000 7fd968c00640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd94c079f60 0x7fd94c07c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.817+0000 7fd968c00640 1 -- 172.21.15.71:0/1402112349 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fd964099d00 con 0x7fd96c15f670 2024-09-16T07:34:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.817+0000 7fd970e00640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd94c079f60 0x7fd94c07c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:14.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.819+0000 7fd970e00640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd94c079f60 0x7fd94c07c420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd960034000 tx=0x7fd960004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:14.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:14.821+0000 7fd968c00640 1 -- 172.21.15.71:0/1402112349 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd964020020 con 0x7fd96c15f670 2024-09-16T07:34:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.038+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd96c0ce480 con 0x7fd94c079f60 2024-09-16T07:34:15.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.106+0000 7fd968c00640 1 -- 172.21.15.71:0/1402112349 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd9600044d0 con 0x7fd94c079f60 2024-09-16T07:34:15.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd94c079f60 msgr2=0x7fd94c07c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd94c079f60 0x7fd94c07c420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd960034000 tx=0x7fd960004730 comp rx=0 tx=0).stop 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 msgr2=0x7fd96c153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 0x7fd96c153b20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd96400cbf0 tx=0x7fd96400a430 comp rx=0 tx=0).stop 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 shutdown_connections 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd94c079f60 0x7fd94c07c420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd96c15f670 0x7fd96c153b20 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd96c15c0f0 0x7fd96c14f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 --2- 172.21.15.71:0/1402112349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c0c5730 0x7fd96c0c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 >> 172.21.15.71:0/1402112349 conn(0x7fd96c0cc490 msgr2=0x7fd96c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 shutdown_connections 2024-09-16T07:34:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.109+0000 7fd972200640 1 -- 172.21.15.71:0/1402112349 wait complete. 2024-09-16T07:34:15.114 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.115+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.115 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.115+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.115 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.115+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.115 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.115+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.115 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.115+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.125 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T07:34:15.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.133+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.133+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.133+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.133+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.133 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:15.133+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.290+0000 7f4194e00640 1 -- 172.21.15.71:0/2620460608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 msgr2=0x7f4174004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.290+0000 7f4194e00640 1 --2- 172.21.15.71:0/2620460608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 0x7f4174004c80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c260 tx=0x7f4174005830 comp rx=0 tx=0).stop 2024-09-16T07:34:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.290+0000 7f418e800640 1 -- 172.21.15.71:0/2620460608 reap_dead start 2024-09-16T07:34:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.290+0000 7f4194e00640 1 -- 172.21.15.71:0/2620460608 shutdown_connections 2024-09-16T07:34:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.290+0000 7f4194e00640 1 -- 172.21.15.71:0/2620460608 >> 172.21.15.71:0/2620460608 conn(0x7f41900b1840 msgr2=0x7f41900b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.290+0000 7f4194e00640 1 -- 172.21.15.71:0/2620460608 shutdown_connections 2024-09-16T07:34:15.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.290+0000 7f4194e00640 1 -- 172.21.15.71:0/2620460608 wait complete. 2024-09-16T07:34:15.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.292+0000 7f4194e00640 1 Processor -- start 2024-09-16T07:34:15.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.292+0000 7f4194e00640 1 -- start start 2024-09-16T07:34:15.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f4194e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 0x7f419013f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:15.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f4194e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41901f5cf0 0x7f41901375e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f4194e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419013eb40 0x7f4190072780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f4194e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f419006e9e0 con 0x7f4174004800 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f4194e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f419006e860 con 0x7f419013eb40 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f4194e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f419006eb60 con 0x7f41901f5cf0 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 0x7f419013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 0x7f419013f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51484/0 (socket says 172.21.15.71:51484) 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418de00640 1 -- 172.21.15.71:0/305377850 learned_addr learned my addr 172.21.15.71:0/305377850 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418d400640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41901f5cf0 0x7f41901375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418e800640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419013eb40 0x7f4190072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418de00640 1 -- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41901f5cf0 msgr2=0x7f41901375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418de00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41901f5cf0 0x7f41901375e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:15.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418de00640 1 -- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419013eb40 msgr2=0x7f4190072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418de00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419013eb40 0x7f4190072780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418de00640 1 -- 172.21.15.71:0/305377850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4190137b20 con 0x7f4174004800 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418e800640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419013eb40 0x7f4190072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.293+0000 7f418d400640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41901f5cf0 0x7f41901375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.294+0000 7f418de00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 0x7f419013f3c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f418000ebc0 tx=0x7f418000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.294+0000 7f4186000640 1 -- 172.21.15.71:0/305377850 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180004d20 con 0x7f4174004800 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.294+0000 7f4186000640 1 -- 172.21.15.71:0/305377850 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4180005020 con 0x7f4174004800 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.294+0000 7f4186000640 1 -- 172.21.15.71:0/305377850 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180005450 con 0x7f4174004800 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.294+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419013b4b0 con 0x7f4174004800 2024-09-16T07:34:15.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.294+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41901f64d0 con 0x7f4174004800 2024-09-16T07:34:15.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.295+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41900cdad0 con 0x7f4174004800 2024-09-16T07:34:15.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.296+0000 7f4186000640 1 -- 172.21.15.71:0/305377850 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 39) ==== 103378+0+0 (secure 0 0 0) 0x7f41800055f0 con 0x7f4174004800 2024-09-16T07:34:15.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.298+0000 7f4186000640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4170079f90 0x7f417007c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:15.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.298+0000 7f4186000640 1 -- 172.21.15.71:0/305377850 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f418009a600 con 0x7f4174004800 2024-09-16T07:34:15.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.298+0000 7f418d400640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4170079f90 0x7f417007c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.299+0000 7f418d400640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4170079f90 0x7f417007c450 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f417c033000 tx=0x7f41900cefa0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:15.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.302+0000 7f4186000640 1 -- 172.21.15.71:0/305377850 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4190137b20 con 0x7f4174004800 2024-09-16T07:34:15.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:15.520+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f41900cd570 con 0x7f4170079f90 2024-09-16T07:34:16.110 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:16.111+0000 7fe6fcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:16.111 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:16.111+0000 7fe6fcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:16.111 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:16.111+0000 7fe6fcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:16.111 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:16.111+0000 7fe6fcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:16.111 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:16.111+0000 7fe6fcc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:16.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.143+0000 7f4186000640 1 -- 172.21.15.71:0/305377850 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f41900cd570 con 0x7f4170079f90 2024-09-16T07:34:16.142 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "in-progress", 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_report": { 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "percentage cloned": "0.0%", 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "amount cloned": "184/3.0G", 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: "files cloned": "3/4" 2024-09-16T07:34:16.143 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:16.144 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:16.144 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:16.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4170079f90 msgr2=0x7f417007c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4170079f90 0x7f417007c450 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f417c033000 tx=0x7f41900cefa0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 msgr2=0x7f419013f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 0x7f419013f3c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f418000ebc0 tx=0x7f418000c6a0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 shutdown_connections 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4170079f90 0x7f417007c450 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f419013eb40 0x7f4190072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41901f5cf0 0x7f41901375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 --2- 172.21.15.71:0/305377850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4174004800 0x7f419013f3c0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 >> 172.21.15.71:0/305377850 conn(0x7f41900b1840 msgr2=0x7f419015cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 shutdown_connections 2024-09-16T07:34:16.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.146+0000 7f4194e00640 1 -- 172.21.15.71:0/305377850 wait complete. 2024-09-16T07:34:16.163 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T07:34:16.163 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T07:34:16.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:34:16.283 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:34:16.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.391+0000 7f4b5c800640 1 -- 172.21.15.71:0/2519544863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54152e30 msgr2=0x7f4b5415cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:16.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.391+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2519544863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54152e30 0x7f4b5415cde0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4b540ccd40 tx=0x7f4b4402a580 comp rx=0 tx=0).stop 2024-09-16T07:34:16.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.391+0000 7f4b5c800640 1 -- 172.21.15.71:0/2519544863 shutdown_connections 2024-09-16T07:34:16.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.391+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2519544863 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b54073030 0x7f4b54073450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.391+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2519544863 >> v2:172.21.15.177:3300/0 conn(0x7f4b540afcb0 0x7f4b540cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.391+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2519544863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54152e30 0x7f4b5415cde0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.391+0000 7f4b5c800640 1 -- 172.21.15.71:0/2519544863 >> 172.21.15.71:0/2519544863 conn(0x7f4b540b2130 msgr2=0x7f4b540b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:16.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.391+0000 7f4b5c800640 1 -- 172.21.15.71:0/2519544863 shutdown_connections 2024-09-16T07:34:16.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.392+0000 7f4b5c800640 1 -- 172.21.15.71:0/2519544863 wait complete. 2024-09-16T07:34:16.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.394+0000 7f4b5c800640 1 Processor -- start 2024-09-16T07:34:16.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.394+0000 7f4b5c800640 1 -- start start 2024-09-16T07:34:16.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b5c800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54073030 0x7f4b5414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:16.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b5c800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b54152e30 0x7f4b54158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:16.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b5c800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b541590b0 0x7f4b540ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:16.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b5c800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5415fd00 con 0x7f4b54073030 2024-09-16T07:34:16.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b5c800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5415fb80 con 0x7f4b54152e30 2024-09-16T07:34:16.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b5c800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5415fe80 con 0x7f4b541590b0 2024-09-16T07:34:16.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b59e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54073030 0x7f4b5414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:16.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b59e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54073030 0x7f4b5414fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51510/0 (socket says 172.21.15.71:51510) 2024-09-16T07:34:16.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.396+0000 7f4b59e00640 1 -- 172.21.15.71:0/2147331073 learned_addr learned my addr 172.21.15.71:0/2147331073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:16.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.397+0000 7f4b59e00640 1 -- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b541590b0 msgr2=0x7f4b540ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:16.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.397+0000 7f4b59e00640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b541590b0 0x7f4b540ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.397+0000 7f4b59e00640 1 -- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b54152e30 msgr2=0x7f4b54158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:16.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.397+0000 7f4b59e00640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b54152e30 0x7f4b54158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.397+0000 7f4b59e00640 1 -- 172.21.15.71:0/2147331073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b541596d0 con 0x7f4b54073030 2024-09-16T07:34:16.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.397+0000 7f4b59e00640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54073030 0x7f4b5414fc90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4b44034040 tx=0x7f4b440038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:16.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.398+0000 7f4b4e000640 1 -- 172.21.15.71:0/2147331073 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4403e030 con 0x7f4b54073030 2024-09-16T07:34:16.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.398+0000 7f4b4e000640 1 -- 172.21.15.71:0/2147331073 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b44003d90 con 0x7f4b54073030 2024-09-16T07:34:16.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.398+0000 7f4b4e000640 1 -- 172.21.15.71:0/2147331073 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44035040 con 0x7f4b54073030 2024-09-16T07:34:16.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.398+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b54156a80 con 0x7f4b54073030 2024-09-16T07:34:16.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.398+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b54157560 con 0x7f4b54073030 2024-09-16T07:34:16.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.400+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b541603a0 con 0x7f4b54073030 2024-09-16T07:34:16.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.402+0000 7f4b4e000640 1 -- 172.21.15.71:0/2147331073 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 39) ==== 103378+0+0 (secure 0 0 0) 0x7f4b44041030 con 0x7f4b54073030 2024-09-16T07:34:16.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.404+0000 7f4b4e000640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4b38079f60 0x7f4b3807c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:16.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.404+0000 7f4b4e000640 1 -- 172.21.15.71:0/2147331073 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f4b440b9fd0 con 0x7f4b54073030 2024-09-16T07:34:16.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.406+0000 7f4b59400640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4b38079f60 0x7f4b3807c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:16.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.409+0000 7f4b59400640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4b38079f60 0x7f4b3807c420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4b480058c0 tx=0x7f4b48005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:16.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.411+0000 7f4b4e000640 1 -- 172.21.15.71:0/2147331073 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4b44084140 con 0x7f4b54073030 2024-09-16T07:34:16.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.745+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f4b5415a110 con 0x7f4b38079f60 2024-09-16T07:34:16.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.801+0000 7f4b4e000640 1 -- 172.21.15.71:0/2147331073 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f4b5415a110 con 0x7f4b38079f60 2024-09-16T07:34:16.801 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:34:16.801 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "in-progress", 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_report": { 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "percentage cloned": "0.0%", 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "amount cloned": "184/3.0G", 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: "files cloned": "4/4" 2024-09-16T07:34:16.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:16.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:16.803 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4b38079f60 msgr2=0x7f4b3807c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4b38079f60 0x7f4b3807c420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4b480058c0 tx=0x7f4b48005850 comp rx=0 tx=0).stop 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54073030 msgr2=0x7f4b5414fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54073030 0x7f4b5414fc90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4b44034040 tx=0x7f4b440038a0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 shutdown_connections 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4b38079f60 0x7f4b3807c420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b541590b0 0x7f4b540ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b54152e30 0x7f4b54158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 --2- 172.21.15.71:0/2147331073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b54073030 0x7f4b5414fc90 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:16.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.804+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 >> 172.21.15.71:0/2147331073 conn(0x7f4b540b2130 msgr2=0x7f4b5406cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:16.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.805+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 shutdown_connections 2024-09-16T07:34:16.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:16.805+0000 7f4b5c800640 1 -- 172.21.15.71:0/2147331073 wait complete. 2024-09-16T07:34:16.836 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 0.0% 2024-09-16T07:34:17.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T07:34:18.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.079+0000 7fd4a8c00640 1 -- 172.21.15.71:0/1665316458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4a406efa0 msgr2=0x7fd4a40cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:18.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.079+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/1665316458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4a406efa0 0x7fd4a40cc5c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd490009830 tx=0x7fd49002a480 comp rx=0 tx=0).stop 2024-09-16T07:34:18.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.079+0000 7fd4a8c00640 1 -- 172.21.15.71:0/1665316458 shutdown_connections 2024-09-16T07:34:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.079+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/1665316458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4a406efa0 0x7fd4a40cc5c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.079+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/1665316458 >> v2:172.21.15.177:3300/0 conn(0x7fd4a4145750 0x7fd4a4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.079+0000 7fd4a8c00640 1 -- 172.21.15.71:0/1665316458 >> 172.21.15.71:0/1665316458 conn(0x7fd4a40b1840 msgr2=0x7fd4a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.080+0000 7fd4a8c00640 1 -- 172.21.15.71:0/1665316458 shutdown_connections 2024-09-16T07:34:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.080+0000 7fd4a8c00640 1 -- 172.21.15.71:0/1665316458 wait complete. 2024-09-16T07:34:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.081+0000 7fd4a8c00640 1 Processor -- start 2024-09-16T07:34:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.081+0000 7fd4a8c00640 1 -- start start 2024-09-16T07:34:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.082+0000 7fd4a8c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4a406efa0 0x7fd4a40c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.082+0000 7fd4a8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a40ccb00 0x7fd4a415edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.082+0000 7fd4a8c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4a4145750 0x7fd4a4145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.082+0000 7fd4a8c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a40b26b0 con 0x7fd4a40ccb00 2024-09-16T07:34:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.082+0000 7fd4a8c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a40cedd0 con 0x7fd4a406efa0 2024-09-16T07:34:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.082+0000 7fd4a8c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a4160100 con 0x7fd4a4145750 2024-09-16T07:34:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a40ccb00 0x7fd4a415edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a40ccb00 0x7fd4a415edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51524/0 (socket says 172.21.15.71:51524) 2024-09-16T07:34:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1400640 1 -- 172.21.15.71:0/628291183 learned_addr learned my addr 172.21.15.71:0/628291183 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1e00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4a406efa0 0x7fd4a40c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a2800640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4a4145750 0x7fd4a4145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1e00640 1 -- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4a4145750 msgr2=0x7fd4a4145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1e00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4a4145750 0x7fd4a4145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1e00640 1 -- 172.21.15.71:0/628291183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a40ccb00 msgr2=0x7fd4a415edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1e00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a40ccb00 0x7fd4a415edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.083+0000 7fd4a1e00640 1 -- 172.21.15.71:0/628291183 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a4136c20 con 0x7fd4a406efa0 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.084+0000 7fd4a1e00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4a406efa0 0x7fd4a40c98b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd494005fd0 tx=0x7fd49400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.084+0000 7fd49a000640 1 -- 172.21.15.71:0/628291183 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49400cc40 con 0x7fd4a406efa0 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.084+0000 7fd49a000640 1 -- 172.21.15.71:0/628291183 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd49400cde0 con 0x7fd4a406efa0 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.084+0000 7fd49a000640 1 -- 172.21.15.71:0/628291183 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494004260 con 0x7fd4a406efa0 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.084+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a4136ec0 con 0x7fd4a406efa0 2024-09-16T07:34:18.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.084+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a4137480 con 0x7fd4a406efa0 2024-09-16T07:34:18.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.085+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a40b1660 con 0x7fd4a406efa0 2024-09-16T07:34:18.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.087+0000 7fd49a000640 1 -- 172.21.15.71:0/628291183 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7fd4940035f0 con 0x7fd4a406efa0 2024-09-16T07:34:18.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.088+0000 7fd49a000640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd48407a140 0x7fd48407c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:18.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.088+0000 7fd49a000640 1 -- 172.21.15.71:0/628291183 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fd494099f10 con 0x7fd4a406efa0 2024-09-16T07:34:18.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.088+0000 7fd4a1400640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd48407a140 0x7fd48407c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:18.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.089+0000 7fd4a1400640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd48407a140 0x7fd48407c600 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40b4e20 tx=0x7fd490004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.093+0000 7fd49a000640 1 -- 172.21.15.71:0/628291183 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd4a40b1660 con 0x7fd4a406efa0 2024-09-16T07:34:18.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.406+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd4a41605b0 con 0x7fd48407a140 2024-09-16T07:34:18.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.504+0000 7fd49a000640 1 -- 172.21.15.71:0/628291183 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7fd4a41605b0 con 0x7fd48407a140 2024-09-16T07:34:18.504 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:18.504 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:34:18.504 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "in-progress", 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_report": { 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: "percentage cloned": "23.958%", 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: "amount cloned": "736M/3.0G", 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: "files cloned": "4/4" 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:18.505 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:18.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd48407a140 msgr2=0x7fd48407c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:18.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd48407a140 0x7fd48407c600 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40b4e20 tx=0x7fd490004730 comp rx=0 tx=0).stop 2024-09-16T07:34:18.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4a406efa0 msgr2=0x7fd4a40c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4a406efa0 0x7fd4a40c98b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd494005fd0 tx=0x7fd49400ee50 comp rx=0 tx=0).stop 2024-09-16T07:34:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 shutdown_connections 2024-09-16T07:34:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd48407a140 0x7fd48407c600 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4a4145750 0x7fd4a4145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a40ccb00 0x7fd4a415edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 --2- 172.21.15.71:0/628291183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4a406efa0 0x7fd4a40c98b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:18.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 >> 172.21.15.71:0/628291183 conn(0x7fd4a40b1840 msgr2=0x7fd4a40af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:18.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 shutdown_connections 2024-09-16T07:34:18.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:18.510+0000 7fd4a8c00640 1 -- 172.21.15.71:0/628291183 wait complete. 2024-09-16T07:34:18.529 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 23.958% 2024-09-16T07:34:19.530 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T07:34:19.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.754+0000 7fa5a6400640 1 -- 172.21.15.71:0/3094699509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a00afcb0 msgr2=0x7fa5a00cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:19.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.754+0000 7fa5a6400640 1 --2- 172.21.15.71:0/3094699509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a00afcb0 0x7fa5a00cceb0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa588009830 tx=0x7fa58802a510 comp rx=0 tx=0).stop 2024-09-16T07:34:19.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.754+0000 7fa5a6400640 1 -- 172.21.15.71:0/3094699509 shutdown_connections 2024-09-16T07:34:19.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.754+0000 7fa5a6400640 1 --2- 172.21.15.71:0/3094699509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a00afcb0 0x7fa5a00cceb0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:19.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.754+0000 7fa5a6400640 1 --2- 172.21.15.71:0/3094699509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5a0152e30 0x7fa5a015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:19.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.754+0000 7fa5a6400640 1 -- 172.21.15.71:0/3094699509 >> 172.21.15.71:0/3094699509 conn(0x7fa5a00b2130 msgr2=0x7fa5a00b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:19.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.754+0000 7fa5a6400640 1 -- 172.21.15.71:0/3094699509 shutdown_connections 2024-09-16T07:34:19.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.754+0000 7fa5a6400640 1 -- 172.21.15.71:0/3094699509 wait complete. 2024-09-16T07:34:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.756+0000 7fa5a6400640 1 Processor -- start 2024-09-16T07:34:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.757+0000 7fa5a6400640 1 -- start start 2024-09-16T07:34:19.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.758+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a0073030 0x7fa5a013e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:19.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.758+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5a0152e30 0x7fa5a013c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:19.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.758+0000 7fa5a6400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5a013cd40 0x7fa5a013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:19.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.758+0000 7fa5a6400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a015ff40 con 0x7fa5a0073030 2024-09-16T07:34:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.758+0000 7fa5a6400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0073940 con 0x7fa5a0152e30 2024-09-16T07:34:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.758+0000 7fa5a6400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a01600c0 con 0x7fa5a013cd40 2024-09-16T07:34:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.758+0000 7fa59f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a0073030 0x7fa5a013e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a0073030 0x7fa5a013e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51544/0 (socket says 172.21.15.71:51544) 2024-09-16T07:34:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59f400640 1 -- 172.21.15.71:0/2803580438 learned_addr learned my addr 172.21.15.71:0/2803580438 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59f400640 1 -- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5a013cd40 msgr2=0x7fa5a013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59fe00640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5a013cd40 0x7fa5a013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59f400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5a013cd40 0x7fa5a013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59f400640 1 -- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5a0152e30 msgr2=0x7fa5a013c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59f400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5a0152e30 0x7fa5a013c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59f400640 1 -- 172.21.15.71:0/2803580438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a013d720 con 0x7fa5a0073030 2024-09-16T07:34:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.759+0000 7fa59fe00640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5a013cd40 0x7fa5a013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.760+0000 7fa59f400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a0073030 0x7fa5a013e7c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa59000b440 tx=0x7fa59000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.760+0000 7fa597400640 1 -- 172.21.15.71:0/2803580438 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590015020 con 0x7fa5a0073030 2024-09-16T07:34:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.760+0000 7fa597400640 1 -- 172.21.15.71:0/2803580438 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa590011980 con 0x7fa5a0073030 2024-09-16T07:34:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.760+0000 7fa597400640 1 -- 172.21.15.71:0/2803580438 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590011dd0 con 0x7fa5a0073030 2024-09-16T07:34:19.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.760+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a013da00 con 0x7fa5a0073030 2024-09-16T07:34:19.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.760+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a013dfc0 con 0x7fa5a0073030 2024-09-16T07:34:19.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.762+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a01605e0 con 0x7fa5a0073030 2024-09-16T07:34:19.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.763+0000 7fa597400640 1 -- 172.21.15.71:0/2803580438 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7fa590018020 con 0x7fa5a0073030 2024-09-16T07:34:19.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.764+0000 7fa597400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa57c07a110 0x7fa57c07c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:19.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.765+0000 7fa597400640 1 -- 172.21.15.71:0/2803580438 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fa590099900 con 0x7fa5a0073030 2024-09-16T07:34:19.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.765+0000 7fa597e00640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa57c07a110 0x7fa57c07c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:19.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.766+0000 7fa597e00640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa57c07a110 0x7fa57c07c5d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa588034000 tx=0x7fa5880046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:19.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:19.770+0000 7fa597400640 1 -- 172.21.15.71:0/2803580438 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa59001f020 con 0x7fa5a0073030 2024-09-16T07:34:20.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.112+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa5a01525e0 con 0x7fa57c07a110 2024-09-16T07:34:20.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.164+0000 7fa597400640 1 -- 172.21.15.71:0/2803580438 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7fa5a01525e0 con 0x7fa57c07a110 2024-09-16T07:34:20.164 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:20.164 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:34:20.164 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "in-progress", 2024-09-16T07:34:20.164 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:34:20.164 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_report": { 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: "percentage cloned": "64.323%", 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: "amount cloned": "1.9G/3.0G", 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: "files cloned": "5/4" 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:20.165 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa57c07a110 msgr2=0x7fa57c07c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa57c07a110 0x7fa57c07c5d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa588034000 tx=0x7fa5880046c0 comp rx=0 tx=0).stop 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a0073030 msgr2=0x7fa5a013e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a0073030 0x7fa5a013e7c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa59000b440 tx=0x7fa59000b900 comp rx=0 tx=0).stop 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 shutdown_connections 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa57c07a110 0x7fa57c07c5d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5a013cd40 0x7fa5a013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5a0152e30 0x7fa5a013c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 --2- 172.21.15.71:0/2803580438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a0073030 0x7fa5a013e7c0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:20.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 >> 172.21.15.71:0/2803580438 conn(0x7fa5a00b2130 msgr2=0x7fa5a015f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:20.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 shutdown_connections 2024-09-16T07:34:20.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:20.167+0000 7fa5a6400640 1 -- 172.21.15.71:0/2803580438 wait complete. 2024-09-16T07:34:20.189 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 64.323% 2024-09-16T07:34:21.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T07:34:21.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.431+0000 7f7937200640 1 -- 172.21.15.71:0/1559581335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 msgr2=0x7f7918004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:21.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.431+0000 7f7937200640 1 --2- 172.21.15.71:0/1559581335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 0x7f7918004c80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7924006f90 tx=0x7f7918005830 comp rx=0 tx=0).stop 2024-09-16T07:34:21.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.431+0000 7f7937200640 1 -- 172.21.15.71:0/1559581335 shutdown_connections 2024-09-16T07:34:21.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.431+0000 7f7937200640 1 --2- 172.21.15.71:0/1559581335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 0x7f7918004c80 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:21.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.431+0000 7f7937200640 1 -- 172.21.15.71:0/1559581335 >> 172.21.15.71:0/1559581335 conn(0x7f79300d3d20 msgr2=0x7f79300d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:21.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.431+0000 7f7937200640 1 -- 172.21.15.71:0/1559581335 shutdown_connections 2024-09-16T07:34:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.432+0000 7f7937200640 1 -- 172.21.15.71:0/1559581335 wait complete. 2024-09-16T07:34:21.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.434+0000 7f7937200640 1 Processor -- start 2024-09-16T07:34:21.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.434+0000 7f7937200640 1 -- start start 2024-09-16T07:34:21.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.435+0000 7f7937200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 0x7f79301f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:21.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.435+0000 7f7937200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79301f8980 0x7f793006d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:21.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.435+0000 7f7937200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f793015c690 0x7f79300b9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:21.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.435+0000 7f7937200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79300cebc0 con 0x7f7918004800 2024-09-16T07:34:21.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.435+0000 7f7937200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f79300cea40 con 0x7f793015c690 2024-09-16T07:34:21.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.435+0000 7f7937200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f79300d18f0 con 0x7f79301f8980 2024-09-16T07:34:21.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.436+0000 7f7935e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 0x7f79301f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:21.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.436+0000 7f7935e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 0x7f79301f4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51560/0 (socket says 172.21.15.71:51560) 2024-09-16T07:34:21.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.436+0000 7f7935e00640 1 -- 172.21.15.71:0/1215436733 learned_addr learned my addr 172.21.15.71:0/1215436733 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:21.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.436+0000 7f7935e00640 1 -- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79301f8980 msgr2=0x7f793006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:21.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.436+0000 7f7935e00640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79301f8980 0x7f793006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:21.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.436+0000 7f7935e00640 1 -- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f793015c690 msgr2=0x7f79300b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:21.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.436+0000 7f7935e00640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f793015c690 0x7f79300b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:21.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.436+0000 7f7935e00640 1 -- 172.21.15.71:0/1215436733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79301f5140 con 0x7f7918004800 2024-09-16T07:34:21.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.437+0000 7f7935e00640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 0x7f79301f4880 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f792000b460 tx=0x7f792000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:21.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.437+0000 7f792a000640 1 -- 172.21.15.71:0/1215436733 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920015020 con 0x7f7918004800 2024-09-16T07:34:21.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.437+0000 7f792a000640 1 -- 172.21.15.71:0/1215436733 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7920011980 con 0x7f7918004800 2024-09-16T07:34:21.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.437+0000 7f792a000640 1 -- 172.21.15.71:0/1215436733 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920011dd0 con 0x7f7918004800 2024-09-16T07:34:21.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.437+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79301f5420 con 0x7f7918004800 2024-09-16T07:34:21.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.438+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793015be90 con 0x7f7918004800 2024-09-16T07:34:21.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.439+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79300d0680 con 0x7f7918004800 2024-09-16T07:34:21.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.442+0000 7f792a000640 1 -- 172.21.15.71:0/1215436733 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7f7920018020 con 0x7f7918004800 2024-09-16T07:34:21.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.443+0000 7f792a000640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791407a160 0x7f791407c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:21.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.443+0000 7f792a000640 1 -- 172.21.15.71:0/1215436733 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f7920099740 con 0x7f7918004800 2024-09-16T07:34:21.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.444+0000 7f7935400640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791407a160 0x7f791407c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:21.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.445+0000 7f7935400640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791407a160 0x7f791407c620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7924033000 tx=0x7f79300cf200 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:21.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.447+0000 7f792a000640 1 -- 172.21.15.71:0/1215436733 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f792001f020 con 0x7f7918004800 2024-09-16T07:34:21.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.768+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f79300b3aa0 con 0x7f791407a160 2024-09-16T07:34:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.810+0000 7f792a000640 1 -- 172.21.15.71:0/1215436733 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f79300b3aa0 con 0x7f791407a160 2024-09-16T07:34:21.810 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:21.810 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:34:21.810 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "in-progress", 2024-09-16T07:34:21.810 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:34:21.810 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_report": { 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: "percentage cloned": "82.552%", 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: "amount cloned": "2.4G/3.0G", 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: "files cloned": "6/4" 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:21.811 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791407a160 msgr2=0x7f791407c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791407a160 0x7f791407c620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7924033000 tx=0x7f79300cf200 comp rx=0 tx=0).stop 2024-09-16T07:34:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 msgr2=0x7f79301f4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 0x7f79301f4880 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f792000b460 tx=0x7f792000b920 comp rx=0 tx=0).stop 2024-09-16T07:34:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 shutdown_connections 2024-09-16T07:34:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791407a160 0x7f791407c620 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f793015c690 0x7f79300b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79301f8980 0x7f793006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 --2- 172.21.15.71:0/1215436733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918004800 0x7f79301f4880 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 >> 172.21.15.71:0/1215436733 conn(0x7f79300d3d20 msgr2=0x7f793006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 shutdown_connections 2024-09-16T07:34:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:21.814+0000 7f7937200640 1 -- 172.21.15.71:0/1215436733 wait complete. 2024-09-16T07:34:21.831 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 82.552% 2024-09-16T07:34:22.833 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T07:34:22.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 -- 172.21.15.71:0/554329337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161670 msgr2=0x7fd1cc0d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:22.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 --2- 172.21.15.71:0/554329337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161670 0x7fd1cc0d18f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc08c1c0 tx=0x7fd1b802a560 comp rx=0 tx=0).stop 2024-09-16T07:34:22.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 -- 172.21.15.71:0/554329337 shutdown_connections 2024-09-16T07:34:22.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 --2- 172.21.15.71:0/554329337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1cc0d1e30 0x7fd1cc0d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:22.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 --2- 172.21.15.71:0/554329337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161670 0x7fd1cc0d18f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:22.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 --2- 172.21.15.71:0/554329337 >> v2:172.21.15.177:3300/0 conn(0x7fd1cc161290 0x7fd1cc0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:22.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 -- 172.21.15.71:0/554329337 >> 172.21.15.71:0/554329337 conn(0x7fd1cc0d3d20 msgr2=0x7fd1cc0d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:22.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 -- 172.21.15.71:0/554329337 shutdown_connections 2024-09-16T07:34:22.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.995+0000 7fd1d1600640 1 -- 172.21.15.71:0/554329337 wait complete. 2024-09-16T07:34:22.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.996+0000 7fd1d1600640 1 Processor -- start 2024-09-16T07:34:22.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.996+0000 7fd1d1600640 1 -- start start 2024-09-16T07:34:22.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.997+0000 7fd1d1600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1cc0d1e30 0x7fd1cc15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.997+0000 7fd1d1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161290 0x7fd1cc0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.997+0000 7fd1d1600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1cc161670 0x7fd1cc0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.997+0000 7fd1d1600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc0d0d70 con 0x7fd1cc161290 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.997+0000 7fd1d1600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc0d0bf0 con 0x7fd1cc161670 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.997+0000 7fd1d1600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1cc0d0ef0 con 0x7fd1cc0d1e30 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161290 0x7fd1cc0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161290 0x7fd1cc0cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51576/0 (socket says 172.21.15.71:51576) 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 -- 172.21.15.71:0/3811425949 learned_addr learned my addr 172.21.15.71:0/3811425949 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cbe00640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1cc0d1e30 0x7fd1cc15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1d0c00640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1cc161670 0x7fd1cc0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 -- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1cc0d1e30 msgr2=0x7fd1cc15eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1cc0d1e30 0x7fd1cc15eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:22.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 -- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1cc161670 msgr2=0x7fd1cc0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:22.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1cc161670 0x7fd1cc0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:22.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 -- 172.21.15.71:0/3811425949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc0cb160 con 0x7fd1cc161290 2024-09-16T07:34:22.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.998+0000 7fd1cb400640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161290 0x7fd1cc0cc200 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8034040 tx=0x7fd1b80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.999+0000 7fd1c3e00640 1 -- 172.21.15.71:0/3811425949 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b803e030 con 0x7fd1cc161290 2024-09-16T07:34:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.999+0000 7fd1c3e00640 1 -- 172.21.15.71:0/3811425949 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1b80047a0 con 0x7fd1cc161290 2024-09-16T07:34:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.999+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc0cb440 con 0x7fd1cc161290 2024-09-16T07:34:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.999+0000 7fd1c3e00640 1 -- 172.21.15.71:0/3811425949 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8035040 con 0x7fd1cc161290 2024-09-16T07:34:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:22.999+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc0cbce0 con 0x7fd1cc161290 2024-09-16T07:34:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:23.000+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc0cb830 con 0x7fd1cc161290 2024-09-16T07:34:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:23.001+0000 7fd1c3e00640 1 -- 172.21.15.71:0/3811425949 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7fd1b8041030 con 0x7fd1cc161290 2024-09-16T07:34:23.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:23.002+0000 7fd1c3e00640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd1ac07a110 0x7fd1ac07c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:23.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:23.002+0000 7fd1c3e00640 1 -- 172.21.15.71:0/3811425949 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fd1b80b9d30 con 0x7fd1cc161290 2024-09-16T07:34:23.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:23.002+0000 7fd1cbe00640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd1ac07a110 0x7fd1ac07c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:23.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:23.003+0000 7fd1cbe00640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd1ac07a110 0x7fd1ac07c5d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00bbe0 tx=0x7fd1bc00b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:23.006+0000 7fd1c3e00640 1 -- 172.21.15.71:0/3811425949 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd1b80837b0 con 0x7fd1cc161290 2024-09-16T07:34:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:23.227+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd1cc0b23e0 con 0x7fd1ac07a110 2024-09-16T07:34:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.706+0000 7fd1c3e00640 1 -- 172.21.15.71:0/3811425949 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fd1cc0b23e0 con 0x7fd1ac07a110 2024-09-16T07:34:27.706 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:27.707 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:34:27.707 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "complete" 2024-09-16T07:34:27.707 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:27.707 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:27.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd1ac07a110 msgr2=0x7fd1ac07c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:27.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd1ac07a110 0x7fd1ac07c5d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00bbe0 tx=0x7fd1bc00b390 comp rx=0 tx=0).stop 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161290 msgr2=0x7fd1cc0cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161290 0x7fd1cc0cc200 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8034040 tx=0x7fd1b80039e0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 shutdown_connections 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd1ac07a110 0x7fd1ac07c5d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1cc161670 0x7fd1cc0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1cc161290 0x7fd1cc0cc200 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 --2- 172.21.15.71:0/3811425949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1cc0d1e30 0x7fd1cc15eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 >> 172.21.15.71:0/3811425949 conn(0x7fd1cc0d3d20 msgr2=0x7fd1cc06e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 shutdown_connections 2024-09-16T07:34:27.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.710+0000 7fd1d1600640 1 -- 172.21.15.71:0/3811425949 wait complete. 2024-09-16T07:34:27.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T07:34:27.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.892+0000 7f793fe00640 1 -- 172.21.15.71:0/3225557218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793806efa0 msgr2=0x7f79380cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:27.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.892+0000 7f793fe00640 1 --2- 172.21.15.71:0/3225557218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793806efa0 0x7f79380cdbb0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f79380af780 tx=0x7f793002a820 comp rx=0 tx=0).stop 2024-09-16T07:34:27.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.892+0000 7f793fe00640 1 -- 172.21.15.71:0/3225557218 shutdown_connections 2024-09-16T07:34:27.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.892+0000 7f793fe00640 1 --2- 172.21.15.71:0/3225557218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793806efa0 0x7f79380cdbb0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.892+0000 7f793fe00640 1 --2- 172.21.15.71:0/3225557218 >> v2:172.21.15.177:3300/0 conn(0x7f79380cccd0 0x7f7938073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.892+0000 7f793fe00640 1 -- 172.21.15.71:0/3225557218 >> 172.21.15.71:0/3225557218 conn(0x7f79380b1840 msgr2=0x7f79380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:27.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.892+0000 7f793fe00640 1 -- 172.21.15.71:0/3225557218 shutdown_connections 2024-09-16T07:34:27.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.892+0000 7f793fe00640 1 -- 172.21.15.71:0/3225557218 wait complete. 2024-09-16T07:34:27.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.893+0000 7f793fe00640 1 Processor -- start 2024-09-16T07:34:27.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.894+0000 7f793fe00640 1 -- start start 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f793806efa0 0x7f7938156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79380cccd0 0x7f79381f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7938145750 0x7f79381f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79380ce480 con 0x7f7938145750 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f79380ce300 con 0x7f79380cccd0 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f79380ce600 con 0x7f793806efa0 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7938145750 0x7f79381f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7938145750 0x7f79381f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41358/0 (socket says 172.21.15.71:41358) 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793de00640 1 -- 172.21.15.71:0/1500208594 learned_addr learned my addr 172.21.15.71:0/1500208594 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793ca00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79380cccd0 0x7f79381f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:27.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793d400640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f793806efa0 0x7f7938156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:27.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793de00640 1 -- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f793806efa0 msgr2=0x7f7938156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:27.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793de00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f793806efa0 0x7f7938156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793de00640 1 -- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79380cccd0 msgr2=0x7f79381f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:27.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793de00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79380cccd0 0x7f79381f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:27.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.895+0000 7f793de00640 1 -- 172.21.15.71:0/1500208594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79381f35e0 con 0x7f7938145750 2024-09-16T07:34:27.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.896+0000 7f793de00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7938145750 0x7f79381f6000 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7930009ca0 tx=0x7f7930032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:27.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.896+0000 7f7935600640 1 -- 172.21.15.71:0/1500208594 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930037040 con 0x7f7938145750 2024-09-16T07:34:27.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.896+0000 7f7935600640 1 -- 172.21.15.71:0/1500208594 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7930042070 con 0x7f7938145750 2024-09-16T07:34:27.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.896+0000 7f7935600640 1 -- 172.21.15.71:0/1500208594 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300042a0 con 0x7f7938145750 2024-09-16T07:34:27.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.896+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79381f3850 con 0x7f7938145750 2024-09-16T07:34:27.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.896+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79381f3e10 con 0x7f7938145750 2024-09-16T07:34:27.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.897+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79380cdbb0 con 0x7f7938145750 2024-09-16T07:34:27.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.898+0000 7f7935600640 1 -- 172.21.15.71:0/1500208594 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7f7930004440 con 0x7f7938145750 2024-09-16T07:34:27.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.900+0000 7f7935600640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791807a110 0x7f791807c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:27.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.900+0000 7f7935600640 1 -- 172.21.15.71:0/1500208594 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f79300ba4f0 con 0x7f7938145750 2024-09-16T07:34:27.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.900+0000 7f793d400640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791807a110 0x7f791807c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:27.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.901+0000 7f793d400640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791807a110 0x7f791807c5d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f792c007fd0 tx=0x7f792c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:27.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:27.904+0000 7f7935600640 1 -- 172.21.15.71:0/1500208594 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f79300845c0 con 0x7f7938145750 2024-09-16T07:34:28.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.129+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f7938071090 con 0x7f791807a110 2024-09-16T07:34:28.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.136+0000 7f7935600640 1 -- 172.21.15.71:0/1500208594 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f7938071090 con 0x7f791807a110 2024-09-16T07:34:28.135 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:28.135 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:34:28.135 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "complete" 2024-09-16T07:34:28.135 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:28.136 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791807a110 msgr2=0x7f791807c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791807a110 0x7f791807c5d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f792c007fd0 tx=0x7f792c009040 comp rx=0 tx=0).stop 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7938145750 msgr2=0x7f79381f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7938145750 0x7f79381f6000 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7930009ca0 tx=0x7f7930032d10 comp rx=0 tx=0).stop 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 shutdown_connections 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f791807a110 0x7f791807c5d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7938145750 0x7f79381f6000 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79380cccd0 0x7f79381f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 --2- 172.21.15.71:0/1500208594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f793806efa0 0x7f7938156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 >> 172.21.15.71:0/1500208594 conn(0x7f79380b1840 msgr2=0x7f79380aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 shutdown_connections 2024-09-16T07:34:28.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.138+0000 7f793fe00640 1 -- 172.21.15.71:0/1500208594 wait complete. 2024-09-16T07:34:28.154 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-16T07:34:28.154 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:34:28.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.313+0000 7f06b5200640 1 -- 172.21.15.71:0/1081636167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06940052e0 msgr2=0x7f069400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.313+0000 7f06b5200640 1 --2- 172.21.15.71:0/1081636167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06940052e0 0x7f069400b9c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f06a8001d90 tx=0x7f0694005830 comp rx=0 tx=0).stop 2024-09-16T07:34:28.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.313+0000 7f06b5200640 1 -- 172.21.15.71:0/1081636167 shutdown_connections 2024-09-16T07:34:28.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.313+0000 7f06b5200640 1 --2- 172.21.15.71:0/1081636167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06940052e0 0x7f069400b9c0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.313+0000 7f06b5200640 1 -- 172.21.15.71:0/1081636167 >> 172.21.15.71:0/1081636167 conn(0x7f06b00b1950 msgr2=0x7f06b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:28.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.313+0000 7f06b5200640 1 -- 172.21.15.71:0/1081636167 shutdown_connections 2024-09-16T07:34:28.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.313+0000 7f06b5200640 1 -- 172.21.15.71:0/1081636167 wait complete. 2024-09-16T07:34:28.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.315+0000 7f06b5200640 1 Processor -- start 2024-09-16T07:34:28.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.315+0000 7f06b5200640 1 -- start start 2024-09-16T07:34:28.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06b5200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06940052e0 0x7f06b015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:28.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06b5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06b015a550 0x7f06b00b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:28.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06b5200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06b015a9b0 0x7f06b00b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:28.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06b5200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06b0074420 con 0x7f06b015a550 2024-09-16T07:34:28.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06b5200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f06b00b2700 con 0x7f06940052e0 2024-09-16T07:34:28.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06b5200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f06b00745a0 con 0x7f06b015a9b0 2024-09-16T07:34:28.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06b015a550 0x7f06b00b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:28.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06b015a550 0x7f06b00b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41384/0 (socket says 172.21.15.71:41384) 2024-09-16T07:34:28.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06a7e00640 1 -- 172.21.15.71:0/2570580456 learned_addr learned my addr 172.21.15.71:0/2570580456 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:28.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06ae800640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06b015a9b0 0x7f06b00b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:28.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06ade00640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06940052e0 0x7f06b015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:28.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06a7e00640 1 -- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06b015a9b0 msgr2=0x7f06b00b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06a7e00640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06b015a9b0 0x7f06b00b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06a7e00640 1 -- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06940052e0 msgr2=0x7f06b015c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06a7e00640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06940052e0 0x7f06b015c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.316+0000 7f06a7e00640 1 -- 172.21.15.71:0/2570580456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b00b2ef0 con 0x7f06b015a550 2024-09-16T07:34:28.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.317+0000 7f06a7e00640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06b015a550 0x7f06b00b3c90 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f069800e630 tx=0x7f069800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:28.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.317+0000 7f06a6000640 1 -- 172.21.15.71:0/2570580456 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698015020 con 0x7f06b015a550 2024-09-16T07:34:28.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.317+0000 7f06a6000640 1 -- 172.21.15.71:0/2570580456 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f069800ca30 con 0x7f06b015a550 2024-09-16T07:34:28.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.317+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b00b31d0 con 0x7f06b015a550 2024-09-16T07:34:28.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.317+0000 7f06a6000640 1 -- 172.21.15.71:0/2570580456 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069800ce80 con 0x7f06b015a550 2024-09-16T07:34:28.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.317+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b00b3790 con 0x7f06b015a550 2024-09-16T07:34:28.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.318+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b00acd30 con 0x7f06b015a550 2024-09-16T07:34:28.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.319+0000 7f06a6000640 1 -- 172.21.15.71:0/2570580456 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7f0698018020 con 0x7f06b015a550 2024-09-16T07:34:28.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.320+0000 7f06a6000640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07a140 0x7f068c07c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:28.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.320+0000 7f06a6000640 1 -- 172.21.15.71:0/2570580456 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f0698099970 con 0x7f06b015a550 2024-09-16T07:34:28.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.321+0000 7f06ade00640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07a140 0x7f068c07c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:28.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.321+0000 7f06ade00640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07a140 0x7f068c07c600 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009a20 tx=0x7f06a0005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:28.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.325+0000 7f06a6000640 1 -- 172.21.15.71:0/2570580456 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f069801f030 con 0x7f06b015a550 2024-09-16T07:34:28.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.530+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f06b00b4710 con 0x7f068c07a140 2024-09-16T07:34:28.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.532+0000 7f06a6000640 1 -- 172.21.15.71:0/2570580456 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f06a0013ae0 con 0x7f068c07a140 2024-09-16T07:34:28.531 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:34:28.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:28.531 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:34:28.532 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:28.532 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:34:28.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07a140 msgr2=0x7f068c07c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07a140 0x7f068c07c600 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009a20 tx=0x7f06a0005f90 comp rx=0 tx=0).stop 2024-09-16T07:34:28.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06b015a550 msgr2=0x7f06b00b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06b015a550 0x7f06b00b3c90 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f069800e630 tx=0x7f069800eaf0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 shutdown_connections 2024-09-16T07:34:28.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f068c07a140 0x7f068c07c600 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06b015a9b0 0x7f06b00b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06b015a550 0x7f06b00b3c90 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 --2- 172.21.15.71:0/2570580456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06940052e0 0x7f06b015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 >> 172.21.15.71:0/2570580456 conn(0x7f06b00b1950 msgr2=0x7f06b015e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:28.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 shutdown_connections 2024-09-16T07:34:28.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.534+0000 7f06b5200640 1 -- 172.21.15.71:0/2570580456 wait complete. 2024-09-16T07:34:28.549 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:34:28.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.712+0000 7facf6c00640 1 -- 172.21.15.71:0/611115268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd8004d40 msgr2=0x7facd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.712+0000 7facf6c00640 1 --2- 172.21.15.71:0/611115268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd8004d40 0x7facd80051c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7face800c610 tx=0x7facd8005830 comp rx=0 tx=0).stop 2024-09-16T07:34:28.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.712+0000 7facf6200640 1 -- 172.21.15.71:0/611115268 reap_dead start 2024-09-16T07:34:28.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.712+0000 7facf6c00640 1 -- 172.21.15.71:0/611115268 shutdown_connections 2024-09-16T07:34:28.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.712+0000 7facf6c00640 1 -- 172.21.15.71:0/611115268 >> 172.21.15.71:0/611115268 conn(0x7facf00cf9f0 msgr2=0x7facf00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:28.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.712+0000 7facf6c00640 1 -- 172.21.15.71:0/611115268 shutdown_connections 2024-09-16T07:34:28.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.712+0000 7facf6c00640 1 -- 172.21.15.71:0/611115268 wait complete. 2024-09-16T07:34:28.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.713+0000 7facf6c00640 1 Processor -- start 2024-09-16T07:34:28.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.714+0000 7facf6c00640 1 -- start start 2024-09-16T07:34:28.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.714+0000 7facf6c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facd8004d40 0x7facf00b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf6c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00b38d0 0x7facf0156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf6c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf00c8e40 0x7facf01e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf6c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7facf015ec50 con 0x7facf00b38d0 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf6c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7facf00ca700 con 0x7facf00c8e40 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf6c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7facf015edd0 con 0x7facd8004d40 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00b38d0 0x7facf0156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00b38d0 0x7facf0156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41402/0 (socket says 172.21.15.71:41402) 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 -- 172.21.15.71:0/3466073184 learned_addr learned my addr 172.21.15.71:0/3466073184 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf5800640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facd8004d40 0x7facf00b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf6200640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf00c8e40 0x7facf01e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 -- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facd8004d40 msgr2=0x7facf00b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facd8004d40 0x7facf00b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 -- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf00c8e40 msgr2=0x7facf01e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf00c8e40 0x7facf01e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 -- 172.21.15.71:0/3466073184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf00b1320 con 0x7facf00b38d0 2024-09-16T07:34:28.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.715+0000 7facf4e00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00b38d0 0x7facf0156110 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7facdc011b10 tx=0x7facdc009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:28.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.716+0000 7faced600640 1 -- 172.21.15.71:0/3466073184 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc01e070 con 0x7facf00b38d0 2024-09-16T07:34:28.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.716+0000 7faced600640 1 -- 172.21.15.71:0/3466073184 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facdc004050 con 0x7facf00b38d0 2024-09-16T07:34:28.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.716+0000 7faced600640 1 -- 172.21.15.71:0/3466073184 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc00fea0 con 0x7facf00b38d0 2024-09-16T07:34:28.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.716+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf00c7de0 con 0x7facf00b38d0 2024-09-16T07:34:28.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.716+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf00c8360 con 0x7facf00b38d0 2024-09-16T07:34:28.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.717+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf00ae5e0 con 0x7facf00b38d0 2024-09-16T07:34:28.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.718+0000 7faced600640 1 -- 172.21.15.71:0/3466073184 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7facdc021070 con 0x7facf00b38d0 2024-09-16T07:34:28.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.720+0000 7faced600640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007a110 0x7facd007c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:28.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.720+0000 7faced600640 1 -- 172.21.15.71:0/3466073184 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7facdc09a1d0 con 0x7facf00b38d0 2024-09-16T07:34:28.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.720+0000 7facf5800640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007a110 0x7facd007c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:28.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.721+0000 7facf5800640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007a110 0x7facd007c5d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7face4009c10 tx=0x7face4005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:28.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.724+0000 7faced600640 1 -- 172.21.15.71:0/3466073184 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7facdc0044a0 con 0x7facf00b38d0 2024-09-16T07:34:28.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.932+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facf00d0710 con 0x7facd007a110 2024-09-16T07:34:28.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.938+0000 7faced600640 1 -- 172.21.15.71:0/3466073184 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7facf00d0710 con 0x7facd007a110 2024-09-16T07:34:28.937 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:28.937 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:34:28.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:28.938 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sv1" 2024-09-16T07:34:28.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:28.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:28.938 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1clone1" 2024-09-16T07:34:28.938 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:28.938 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:34:28.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007a110 msgr2=0x7facd007c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007a110 0x7facd007c5d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7face4009c10 tx=0x7face4005f90 comp rx=0 tx=0).stop 2024-09-16T07:34:28.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00b38d0 msgr2=0x7facf0156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:28.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00b38d0 0x7facf0156110 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7facdc011b10 tx=0x7facdc009e90 comp rx=0 tx=0).stop 2024-09-16T07:34:28.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 shutdown_connections 2024-09-16T07:34:28.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007a110 0x7facd007c5d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf00c8e40 0x7facf01e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00b38d0 0x7facf0156110 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 --2- 172.21.15.71:0/3466073184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facd8004d40 0x7facf00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:28.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.940+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 >> 172.21.15.71:0/3466073184 conn(0x7facf00cf9f0 msgr2=0x7facf00ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:28.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.941+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 shutdown_connections 2024-09-16T07:34:28.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:28.941+0000 7facf6c00640 1 -- 172.21.15.71:0/3466073184 wait complete. 2024-09-16T07:34:28.956 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T07:34:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.118+0000 7f83a9c00640 1 -- 172.21.15.71:0/3116290113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a415b700 msgr2=0x7f83a40ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.118+0000 7f83a9c00640 1 --2- 172.21.15.71:0/3116290113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a415b700 0x7f83a40ccca0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f83a406aa00 tx=0x7f838c02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.119+0000 7f83a9c00640 1 -- 172.21.15.71:0/3116290113 shutdown_connections 2024-09-16T07:34:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.119+0000 7f83a9c00640 1 --2- 172.21.15.71:0/3116290113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a415b700 0x7f83a40ccca0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.119+0000 7f83a9c00640 1 --2- 172.21.15.71:0/3116290113 >> v2:172.21.15.177:3300/0 conn(0x7f83a4158900 0x7f83a415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.119+0000 7f83a9c00640 1 -- 172.21.15.71:0/3116290113 >> 172.21.15.71:0/3116290113 conn(0x7f83a40b18e0 msgr2=0x7f83a40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.119+0000 7f83a9c00640 1 -- 172.21.15.71:0/3116290113 shutdown_connections 2024-09-16T07:34:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.119+0000 7f83a9c00640 1 -- 172.21.15.71:0/3116290113 wait complete. 2024-09-16T07:34:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.120+0000 7f83a9c00640 1 Processor -- start 2024-09-16T07:34:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.120+0000 7f83a9c00640 1 -- start start 2024-09-16T07:34:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.121+0000 7f83a9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a40cd1e0 0x7f83a41577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.121+0000 7f83a9c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83a4158900 0x7f83a409d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.121+0000 7f83a9c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83a415b700 0x7f83a409b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.121+0000 7f83a9c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83a40ced60 con 0x7f83a40cd1e0 2024-09-16T07:34:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.121+0000 7f83a9c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f83a40cebe0 con 0x7f83a4158900 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.121+0000 7f83a9c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f83a40ceee0 con 0x7f83a415b700 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.121+0000 7f83a2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a40cd1e0 0x7f83a41577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a40cd1e0 0x7f83a41577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41430/0 (socket says 172.21.15.71:41430) 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a2a00640 1 -- 172.21.15.71:0/1282744061 learned_addr learned my addr 172.21.15.71:0/1282744061 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f839be00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83a4158900 0x7f83a409d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a3400640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83a415b700 0x7f83a409b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a2a00640 1 -- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83a415b700 msgr2=0x7f83a409b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a2a00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83a415b700 0x7f83a409b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a2a00640 1 -- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83a4158900 msgr2=0x7f83a409d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a2a00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83a4158900 0x7f83a409d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a2a00640 1 -- 172.21.15.71:0/1282744061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a4074660 con 0x7f83a40cd1e0 2024-09-16T07:34:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.122+0000 7f83a2a00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a40cd1e0 0x7f83a41577b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f839400e9f0 tx=0x7f839400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.123+0000 7f839aa00640 1 -- 172.21.15.71:0/1282744061 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839400cd80 con 0x7f83a40cd1e0 2024-09-16T07:34:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.123+0000 7f839aa00640 1 -- 172.21.15.71:0/1282744061 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8394010430 con 0x7f83a40cd1e0 2024-09-16T07:34:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.123+0000 7f839aa00640 1 -- 172.21.15.71:0/1282744061 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394010840 con 0x7f83a40cd1e0 2024-09-16T07:34:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.123+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a4157cf0 con 0x7f83a40cd1e0 2024-09-16T07:34:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.123+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a40740a0 con 0x7f83a40cd1e0 2024-09-16T07:34:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.124+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a4069360 con 0x7f83a40cd1e0 2024-09-16T07:34:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.125+0000 7f839aa00640 1 -- 172.21.15.71:0/1282744061 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7f83940109e0 con 0x7f83a40cd1e0 2024-09-16T07:34:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.126+0000 7f839aa00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f838007a110 0x7f838007c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.126+0000 7f839aa00640 1 -- 172.21.15.71:0/1282744061 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f839409a270 con 0x7f83a40cd1e0 2024-09-16T07:34:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.126+0000 7f839be00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f838007a110 0x7f838007c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.127+0000 7f839be00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f838007a110 0x7f838007c5d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f838c033000 tx=0x7f838c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:29.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.130+0000 7f839aa00640 1 -- 172.21.15.71:0/1282744061 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f83a4157cf0 con 0x7f83a40cd1e0 2024-09-16T07:34:29.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.343+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83a40cf470 con 0x7f838007a110 2024-09-16T07:34:29.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.357+0000 7f839aa00640 1 -- 172.21.15.71:0/1282744061 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f83a40cf470 con 0x7f838007a110 2024-09-16T07:34:29.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:29.357 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:34:29.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:29.357 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1" 2024-09-16T07:34:29.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:29.357 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:34:29.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.359+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f838007a110 msgr2=0x7f838007c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.359+0000 7f83a9c00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f838007a110 0x7f838007c5d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f838c033000 tx=0x7f838c004730 comp rx=0 tx=0).stop 2024-09-16T07:34:29.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.359+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a40cd1e0 msgr2=0x7f83a41577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.359+0000 7f83a9c00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a40cd1e0 0x7f83a41577b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f839400e9f0 tx=0x7f839400eeb0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.360+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 shutdown_connections 2024-09-16T07:34:29.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.360+0000 7f83a9c00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f838007a110 0x7f838007c5d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.360+0000 7f83a9c00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83a415b700 0x7f83a409b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.360+0000 7f83a9c00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83a4158900 0x7f83a409d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.360+0000 7f83a9c00640 1 --2- 172.21.15.71:0/1282744061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a40cd1e0 0x7f83a41577b0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.360+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 >> 172.21.15.71:0/1282744061 conn(0x7f83a40b18e0 msgr2=0x7f83a4158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:29.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.360+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 shutdown_connections 2024-09-16T07:34:29.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.360+0000 7f83a9c00640 1 -- 172.21.15.71:0/1282744061 wait complete. 2024-09-16T07:34:29.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T07:34:29.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.537+0000 7f45c9400640 1 -- 172.21.15.71:0/1253056245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a8004d40 msgr2=0x7f45a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.537+0000 7f45c9400640 1 --2- 172.21.15.71:0/1253056245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c640 tx=0x7f45a8005830 comp rx=0 tx=0).stop 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.537+0000 7f45c3200640 1 -- 172.21.15.71:0/1253056245 reap_dead start 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.538+0000 7f45c9400640 1 -- 172.21.15.71:0/1253056245 shutdown_connections 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.538+0000 7f45c9400640 1 -- 172.21.15.71:0/1253056245 >> 172.21.15.71:0/1253056245 conn(0x7f45c40b1840 msgr2=0x7f45c40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.538+0000 7f45c9400640 1 -- 172.21.15.71:0/1253056245 shutdown_connections 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.538+0000 7f45c9400640 1 -- 172.21.15.71:0/1253056245 wait complete. 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.539+0000 7f45c9400640 1 Processor -- start 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.539+0000 7f45c9400640 1 -- start start 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c9400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45a8004d40 0x7f45c41350b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c9400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45c413b030 0x7f45c4153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45c41f61a0 0x7f45c4155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c9400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f45c40cdf10 con 0x7f45c41f61a0 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c9400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f45c40cdd90 con 0x7f45a8004d40 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c9400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f45c40ce090 con 0x7f45c413b030 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45c41f61a0 0x7f45c4155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45c41f61a0 0x7f45c4155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41448/0 (socket says 172.21.15.71:41448) 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.540+0000 7f45c3200640 1 -- 172.21.15.71:0/2572882040 learned_addr learned my addr 172.21.15.71:0/2572882040 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45bbe00640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45c413b030 0x7f45c4153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45c2800640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45a8004d40 0x7f45c41350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45c3200640 1 -- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45c413b030 msgr2=0x7f45c4153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45c3200640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45c413b030 0x7f45c4153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45c3200640 1 -- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45a8004d40 msgr2=0x7f45c41350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45c3200640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45a8004d40 0x7f45c41350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45c3200640 1 -- 172.21.15.71:0/2572882040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c41358b0 con 0x7f45c41f61a0 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45bbe00640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45c413b030 0x7f45c4153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45c3200640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45c41f61a0 0x7f45c4155370 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c730 tx=0x7f45bc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45baa00640 1 -- 172.21.15.71:0/2572882040 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc039070 con 0x7f45c41f61a0 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45baa00640 1 -- 172.21.15.71:0/2572882040 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45bc02ba10 con 0x7f45c41f61a0 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45baa00640 1 -- 172.21.15.71:0/2572882040 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc001fb0 con 0x7f45c41f61a0 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.541+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c41355f0 con 0x7f45c41f61a0 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.542+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c41f6d10 con 0x7f45c41f61a0 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.543+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c406e860 con 0x7f45c41f61a0 2024-09-16T07:34:29.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.544+0000 7f45baa00640 1 -- 172.21.15.71:0/2572882040 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7f45bc034680 con 0x7f45c41f61a0 2024-09-16T07:34:29.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.545+0000 7f45baa00640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f45a007a110 0x7f45a007c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.545+0000 7f45baa00640 1 -- 172.21.15.71:0/2572882040 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f45bc0b9770 con 0x7f45c41f61a0 2024-09-16T07:34:29.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.545+0000 7f45c2800640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f45a007a110 0x7f45a007c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.546+0000 7f45c2800640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f45a007a110 0x7f45a007c5d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f45b400acc0 tx=0x7f45b4005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:29.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.549+0000 7f45baa00640 1 -- 172.21.15.71:0/2572882040 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f45bc083900 con 0x7f45c41f61a0 2024-09-16T07:34:29.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.765+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45c406ec70 con 0x7f45a007a110 2024-09-16T07:34:29.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.795+0000 7f45baa00640 1 -- 172.21.15.71:0/2572882040 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f45c406ec70 con 0x7f45a007a110 2024-09-16T07:34:29.795 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:29.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.797+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f45a007a110 msgr2=0x7f45a007c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.797+0000 7f45c9400640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f45a007a110 0x7f45a007c5d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f45b400acc0 tx=0x7f45b4005f90 comp rx=0 tx=0).stop 2024-09-16T07:34:29.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.797+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45c41f61a0 msgr2=0x7f45c4155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.797+0000 7f45c9400640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45c41f61a0 0x7f45c4155370 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c730 tx=0x7f45bc003ba0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.798+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 shutdown_connections 2024-09-16T07:34:29.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.798+0000 7f45c9400640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f45a007a110 0x7f45a007c5d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.798+0000 7f45c9400640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45c41f61a0 0x7f45c4155370 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.798+0000 7f45c9400640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45c413b030 0x7f45c4153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.798+0000 7f45c9400640 1 --2- 172.21.15.71:0/2572882040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45a8004d40 0x7f45c41350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.798+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 >> 172.21.15.71:0/2572882040 conn(0x7f45c40b1840 msgr2=0x7f45c40cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:29.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.798+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 shutdown_connections 2024-09-16T07:34:29.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.798+0000 7f45c9400640 1 -- 172.21.15.71:0/2572882040 wait complete. 2024-09-16T07:34:29.814 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T07:34:29.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.973+0000 7f4f22e00640 1 -- 172.21.15.71:0/3125296985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cd5c0 msgr2=0x7f4f1c0ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.973+0000 7f4f22e00640 1 --2- 172.21.15.71:0/3125296985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cd5c0 0x7f4f1c0ca250 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0cd4d0 tx=0x7f4f0802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.974+0000 7f4f22e00640 1 -- 172.21.15.71:0/3125296985 shutdown_connections 2024-09-16T07:34:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.974+0000 7f4f22e00640 1 --2- 172.21.15.71:0/3125296985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cd5c0 0x7f4f1c0ca250 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.974+0000 7f4f22e00640 1 --2- 172.21.15.71:0/3125296985 >> v2:172.21.15.177:3300/0 conn(0x7f4f1c0cf5a0 0x7f4f1c06f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.974+0000 7f4f22e00640 1 -- 172.21.15.71:0/3125296985 >> 172.21.15.71:0/3125296985 conn(0x7f4f1c0cf9f0 msgr2=0x7f4f1c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.974+0000 7f4f22e00640 1 -- 172.21.15.71:0/3125296985 shutdown_connections 2024-09-16T07:34:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.974+0000 7f4f22e00640 1 -- 172.21.15.71:0/3125296985 wait complete. 2024-09-16T07:34:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.975+0000 7f4f22e00640 1 Processor -- start 2024-09-16T07:34:29.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.975+0000 7f4f22e00640 1 -- start start 2024-09-16T07:34:29.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.976+0000 7f4f22e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c0aeb30 0x7f4f1c1e02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.976+0000 7f4f22e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0cd5c0 0x7f4f1c0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.976+0000 7f4f22e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cf5a0 0x7f4f1c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.976+0000 7f4f22e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c0cab20 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.976+0000 7f4f22e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c0ca9a0 con 0x7f4f1c0aeb30 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.976+0000 7f4f22e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1c06e5a0 con 0x7f4f1c0cd5c0 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cf5a0 0x7f4f1c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cf5a0 0x7f4f1c152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41456/0 (socket says 172.21.15.71:41456) 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 -- 172.21.15.71:0/2087377943 learned_addr learned my addr 172.21.15.71:0/2087377943 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f21a00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c0aeb30 0x7f4f1c1e02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f21000640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0cd5c0 0x7f4f1c0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 -- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0cd5c0 msgr2=0x7f4f1c0b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0cd5c0 0x7f4f1c0b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 -- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c0aeb30 msgr2=0x7f4f1c1e02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:29.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c0aeb30 0x7f4f1c1e02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:29.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 -- 172.21.15.71:0/2087377943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c0aff60 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f21000640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0cd5c0 0x7f4f1c0b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:29.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.977+0000 7f4f22400640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cf5a0 0x7f4f1c152d90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400c410 tx=0x7f4f1400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:29.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.978+0000 7f4f19600640 1 -- 172.21.15.71:0/2087377943 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14011790 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.978+0000 7f4f19600640 1 -- 172.21.15.71:0/2087377943 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f14017070 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.978+0000 7f4f19600640 1 -- 172.21.15.71:0/2087377943 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14011d00 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.978+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c1e0b40 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.978+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c0b04e0 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.979+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c06eac0 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.980+0000 7f4f19600640 1 -- 172.21.15.71:0/2087377943 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7f4f1401c460 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.982+0000 7f4f19600640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4efc07a140 0x7f4efc07c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:29.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.982+0000 7f4f19600640 1 -- 172.21.15.71:0/2087377943 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f4f14099470 con 0x7f4f1c0cf5a0 2024-09-16T07:34:29.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.982+0000 7f4f21a00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4efc07a140 0x7f4efc07c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:29.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.983+0000 7f4f21a00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4efc07a140 0x7f4efc07c600 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4f10005fd0 tx=0x7f4f1000a670 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:29.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:29.986+0000 7f4f19600640 1 -- 172.21.15.71:0/2087377943 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4f140635e0 con 0x7f4f1c0cf5a0 2024-09-16T07:34:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.198+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4f1c0c3090 con 0x7f4efc07a140 2024-09-16T07:34:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.220+0000 7f4f19600640 1 -- 172.21.15.71:0/2087377943 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4f1c0c3090 con 0x7f4efc07a140 2024-09-16T07:34:30.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.222+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4efc07a140 msgr2=0x7f4efc07c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:30.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.222+0000 7f4f22e00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4efc07a140 0x7f4efc07c600 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4f10005fd0 tx=0x7f4f1000a670 comp rx=0 tx=0).stop 2024-09-16T07:34:30.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.222+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cf5a0 msgr2=0x7f4f1c152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:30.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.222+0000 7f4f22e00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cf5a0 0x7f4f1c152d90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400c410 tx=0x7f4f1400c8d0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.223+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 shutdown_connections 2024-09-16T07:34:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.223+0000 7f4f22e00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4efc07a140 0x7f4efc07c600 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.223+0000 7f4f22e00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f1c0cf5a0 0x7f4f1c152d90 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.223+0000 7f4f22e00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0cd5c0 0x7f4f1c0b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.223+0000 7f4f22e00640 1 --2- 172.21.15.71:0/2087377943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c0aeb30 0x7f4f1c1e02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.223+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 >> 172.21.15.71:0/2087377943 conn(0x7f4f1c0cf9f0 msgr2=0x7f4f1c0b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.223+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 shutdown_connections 2024-09-16T07:34:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.223+0000 7f4f22e00640 1 -- 172.21.15.71:0/2087377943 wait complete. 2024-09-16T07:34:30.236 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.236+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.236 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.236+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.236 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.236+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.236 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.236+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.236 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.236+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.240 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 -- 172.21.15.71:0/2701163456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 msgr2=0x7f3394074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 --2- 172.21.15.71:0/2701163456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 0x7f3394074280 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f3394070740 tx=0x7f338c02aa30 comp rx=0 tx=0).stop 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 -- 172.21.15.71:0/2701163456 shutdown_connections 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 --2- 172.21.15.71:0/2701163456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 0x7f3394074280 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 --2- 172.21.15.71:0/2701163456 >> v2:172.21.15.177:3300/0 conn(0x7f33940cefc0 0x7f3394073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 --2- 172.21.15.71:0/2701163456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f339415bb90 0x7f33940b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 -- 172.21.15.71:0/2701163456 >> 172.21.15.71:0/2701163456 conn(0x7f33940b1510 msgr2=0x7f33940b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 -- 172.21.15.71:0/2701163456 shutdown_connections 2024-09-16T07:34:30.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.409+0000 7f339a800640 1 -- 172.21.15.71:0/2701163456 wait complete. 2024-09-16T07:34:30.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.411+0000 7f339a800640 1 Processor -- start 2024-09-16T07:34:30.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.411+0000 7f339a800640 1 -- start start 2024-09-16T07:34:30.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f339a800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33940cefc0 0x7f33940c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f339a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 0x7f33940c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f339a800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f339415bb90 0x7f339415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f339a800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f339406eee0 con 0x7f33940cf3e0 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f339a800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f339406ed60 con 0x7f339415bb90 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f339a800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f339406f060 con 0x7f33940cefc0 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3398a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 0x7f33940c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3398a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 0x7f33940c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41468/0 (socket says 172.21.15.71:41468) 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3398a00640 1 -- 172.21.15.71:0/345038163 learned_addr learned my addr 172.21.15.71:0/345038163 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3399e00640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f339415bb90 0x7f339415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3398a00640 1 -- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33940cefc0 msgr2=0x7f33940c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3398a00640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33940cefc0 0x7f33940c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3398a00640 1 -- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f339415bb90 msgr2=0x7f339415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3398a00640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f339415bb90 0x7f339415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.412+0000 7f3398a00640 1 -- 172.21.15.71:0/345038163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33940c6bd0 con 0x7f33940cf3e0 2024-09-16T07:34:30.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.413+0000 7f3398a00640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 0x7f33940c7950 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f338000e9c0 tx=0x7f338000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:30.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.413+0000 7f3391600640 1 -- 172.21.15.71:0/345038163 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338000cd80 con 0x7f33940cf3e0 2024-09-16T07:34:30.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.413+0000 7f3391600640 1 -- 172.21.15.71:0/345038163 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3380004020 con 0x7f33940cf3e0 2024-09-16T07:34:30.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.413+0000 7f3391600640 1 -- 172.21.15.71:0/345038163 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380004450 con 0x7f33940cf3e0 2024-09-16T07:34:30.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.413+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33940c6e70 con 0x7f33940cf3e0 2024-09-16T07:34:30.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.413+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33940c7430 con 0x7f33940cf3e0 2024-09-16T07:34:30.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.414+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339406a110 con 0x7f33940cf3e0 2024-09-16T07:34:30.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.415+0000 7f3391600640 1 -- 172.21.15.71:0/345038163 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7f33800035f0 con 0x7f33940cf3e0 2024-09-16T07:34:30.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.416+0000 7f3391600640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f337407a110 0x7f337407c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:30.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.416+0000 7f3391600640 1 -- 172.21.15.71:0/345038163 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7f338009a1d0 con 0x7f33940cf3e0 2024-09-16T07:34:30.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.417+0000 7f3399400640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f337407a110 0x7f337407c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.417+0000 7f3399400640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f337407a110 0x7f337407c5d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3388009a30 tx=0x7f33880094f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.420+0000 7f3391600640 1 -- 172.21.15.71:0/345038163 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f339406a110 con 0x7f33940cf3e0 2024-09-16T07:34:30.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.643+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f33940b2810 con 0x7f337407a110 2024-09-16T07:34:30.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.657+0000 7f3391600640 1 -- 172.21.15.71:0/345038163 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33940b2810 con 0x7f337407a110 2024-09-16T07:34:30.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f337407a110 msgr2=0x7f337407c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:30.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f337407a110 0x7f337407c5d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3388009a30 tx=0x7f33880094f0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 msgr2=0x7f33940c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:30.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 0x7f33940c7950 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f338000e9c0 tx=0x7f338000ee80 comp rx=0 tx=0).stop 2024-09-16T07:34:30.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 shutdown_connections 2024-09-16T07:34:30.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f337407a110 0x7f337407c5d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f339415bb90 0x7f339415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33940cf3e0 0x7f33940c7950 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 --2- 172.21.15.71:0/345038163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33940cefc0 0x7f33940c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 >> 172.21.15.71:0/345038163 conn(0x7f33940b1510 msgr2=0x7f33940b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 shutdown_connections 2024-09-16T07:34:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30.660+0000 7f339a800640 1 -- 172.21.15.71:0/345038163 wait complete. 2024-09-16T07:34:30.673 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.673+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.673 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.673+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.673 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.673+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.673 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.673+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.673 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:34:30.673+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:34:30.677 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:34:30.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:34:30.982 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:34:30.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:34:30.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:34:31.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.151+0000 7fc1db200640 1 -- 172.21.15.71:0/637978639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc0b1af0 msgr2=0x7fc1dc0d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:31.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.151+0000 7fc1db200640 1 --2- 172.21.15.71:0/637978639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc0b1af0 0x7fc1dc0d0d90 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4005fd0 tx=0x7fc1d402a940 comp rx=0 tx=0).stop 2024-09-16T07:34:31.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.151+0000 7fc1db200640 1 -- 172.21.15.71:0/637978639 shutdown_connections 2024-09-16T07:34:31.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.151+0000 7fc1db200640 1 --2- 172.21.15.71:0/637978639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc0b1af0 0x7fc1dc0d0d90 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.151+0000 7fc1db200640 1 -- 172.21.15.71:0/637978639 >> 172.21.15.71:0/637978639 conn(0x7fc1dc0d3d20 msgr2=0x7fc1dc0d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:31.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.151+0000 7fc1db200640 1 -- 172.21.15.71:0/637978639 shutdown_connections 2024-09-16T07:34:31.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.151+0000 7fc1db200640 1 -- 172.21.15.71:0/637978639 wait complete. 2024-09-16T07:34:31.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.152+0000 7fc1db200640 1 Processor -- start 2024-09-16T07:34:31.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.152+0000 7fc1db200640 1 -- start start 2024-09-16T07:34:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.153+0000 7fc1db200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1dc0b1af0 0x7fc1dc149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.153+0000 7fc1db200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc161290 0x7fc1dc1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.153+0000 7fc1db200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1dc161670 0x7fc1dc15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.153+0000 7fc1db200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc169070 con 0x7fc1dc161290 2024-09-16T07:34:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.153+0000 7fc1db200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc168ef0 con 0x7fc1dc161670 2024-09-16T07:34:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.153+0000 7fc1db200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1dc1691f0 con 0x7fc1dc0b1af0 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.153+0000 7fc1d3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc161290 0x7fc1dc1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc161290 0x7fc1dc1574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41488/0 (socket says 172.21.15.71:41488) 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d3e00640 1 -- 172.21.15.71:0/2689239240 learned_addr learned my addr 172.21.15.71:0/2689239240 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1da800640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1dc161670 0x7fc1dc15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d9e00640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1dc0b1af0 0x7fc1dc149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d3e00640 1 -- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1dc0b1af0 msgr2=0x7fc1dc149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d3e00640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1dc0b1af0 0x7fc1dc149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d3e00640 1 -- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1dc161670 msgr2=0x7fc1dc15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d3e00640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1dc161670 0x7fc1dc15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d3e00640 1 -- 172.21.15.71:0/2689239240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc14a210 con 0x7fc1dc161290 2024-09-16T07:34:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d9e00640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1dc0b1af0 0x7fc1dc149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.154+0000 7fc1d3e00640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc161290 0x7fc1dc1574f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c3a0 tx=0x7fc1c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.155+0000 7fc1d2000640 1 -- 172.21.15.71:0/2689239240 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4017070 con 0x7fc1dc161290 2024-09-16T07:34:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.155+0000 7fc1d2000640 1 -- 172.21.15.71:0/2689239240 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1c40053c0 con 0x7fc1dc161290 2024-09-16T07:34:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.155+0000 7fc1d2000640 1 -- 172.21.15.71:0/2689239240 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4003850 con 0x7fc1dc161290 2024-09-16T07:34:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.155+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc0c8fa0 con 0x7fc1dc161290 2024-09-16T07:34:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.155+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc0c9560 con 0x7fc1dc161290 2024-09-16T07:34:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.156+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc1f5b60 con 0x7fc1dc161290 2024-09-16T07:34:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.157+0000 7fc1d2000640 1 -- 172.21.15.71:0/2689239240 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 40) ==== 103531+0+0 (secure 0 0 0) 0x7fc1c401a070 con 0x7fc1dc161290 2024-09-16T07:34:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.158+0000 7fc1d2000640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1b807a110 0x7fc1b807c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.158+0000 7fc1d2000640 1 -- 172.21.15.71:0/2689239240 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5737+0+0 (secure 0 0 0) 0x7fc1c4098a50 con 0x7fc1dc161290 2024-09-16T07:34:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.158+0000 7fc1d9e00640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1b807a110 0x7fc1b807c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.159+0000 7fc1d9e00640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1b807a110 0x7fc1b807c5d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc007a50 tx=0x7fc1cc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:31.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.162+0000 7fc1d2000640 1 -- 172.21.15.71:0/2689239240 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc1dc1f5b60 con 0x7fc1dc161290 2024-09-16T07:34:31.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.347+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fc1dc0ce830 con 0x7fc1dc161290 2024-09-16T07:34:31.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.348+0000 7fc1d2000640 1 -- 172.21.15.71:0/2689239240 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7fc1c40624d0 con 0x7fc1dc161290 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1b807a110 msgr2=0x7fc1b807c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1b807a110 0x7fc1b807c5d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc007a50 tx=0x7fc1cc009040 comp rx=0 tx=0).stop 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc161290 msgr2=0x7fc1dc1574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc161290 0x7fc1dc1574f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c3a0 tx=0x7fc1c400c860 comp rx=0 tx=0).stop 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 shutdown_connections 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc1b807a110 0x7fc1b807c5d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1dc161670 0x7fc1dc15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1dc161290 0x7fc1dc1574f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 --2- 172.21.15.71:0/2689239240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1dc0b1af0 0x7fc1dc149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 >> 172.21.15.71:0/2689239240 conn(0x7fc1dc0d3d20 msgr2=0x7fc1dc15c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 shutdown_connections 2024-09-16T07:34:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.350+0000 7fc1db200640 1 -- 172.21.15.71:0/2689239240 wait complete. 2024-09-16T07:34:31.366 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:34:31.366 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:34:31.455 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:34:31.544 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:34:31.544 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:34:31.544 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:34:31.544 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:31.580 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:34:31.581 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:31.656 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:34:31.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.855+0000 7f2795800640 1 -- 172.21.15.71:0/385595493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f279015bc30 msgr2=0x7f27900b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:31.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.855+0000 7f2795800640 1 --2- 172.21.15.71:0/385595493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f279015bc30 0x7f27900b28f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2780005fd0 tx=0x7f278002a5d0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.855+0000 7f2795800640 1 -- 172.21.15.71:0/385595493 shutdown_connections 2024-09-16T07:34:31.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.855+0000 7f2795800640 1 --2- 172.21.15.71:0/385595493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27900d0540 0x7f279006f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.855+0000 7f2795800640 1 --2- 172.21.15.71:0/385595493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f279015bc30 0x7f27900b28f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.855+0000 7f2795800640 1 -- 172.21.15.71:0/385595493 >> 172.21.15.71:0/385595493 conn(0x7f27900cfa20 msgr2=0x7f27900cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:31.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.855+0000 7f2795800640 1 -- 172.21.15.71:0/385595493 shutdown_connections 2024-09-16T07:34:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.855+0000 7f2795800640 1 -- 172.21.15.71:0/385595493 wait complete. 2024-09-16T07:34:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.856+0000 7f2795800640 1 Processor -- start 2024-09-16T07:34:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.857+0000 7f2795800640 1 -- start start 2024-09-16T07:34:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2795800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27900d0540 0x7f2790145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2795800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f279015bc30 0x7f27901485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2795800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27901f7c30 0x7f27901f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2795800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2790074660 con 0x7f27901f7c30 2024-09-16T07:34:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2795800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f279006f860 con 0x7f27900d0540 2024-09-16T07:34:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2795800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f27900c7d10 con 0x7f279015bc30 2024-09-16T07:34:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2794e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27901f7c30 0x7f27901f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2794e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27901f7c30 0x7f27901f80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41496/0 (socket says 172.21.15.71:41496) 2024-09-16T07:34:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2794e00640 1 -- 172.21.15.71:0/2607060262 learned_addr learned my addr 172.21.15.71:0/2607060262 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f278f400640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f279015bc30 0x7f27901485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f278fe00640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27900d0540 0x7f2790145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2794e00640 1 -- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f279015bc30 msgr2=0x7f27901485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2794e00640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f279015bc30 0x7f27901485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2794e00640 1 -- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27900d0540 msgr2=0x7f2790145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2794e00640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27900d0540 0x7f2790145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.858+0000 7f2794e00640 1 -- 172.21.15.71:0/2607060262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27900c7f80 con 0x7f27901f7c30 2024-09-16T07:34:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.859+0000 7f2794e00640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27901f7c30 0x7f27901f80e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f278800eb40 tx=0x7f278800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.859+0000 7f2787e00640 1 -- 172.21.15.71:0/2607060262 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788004b20 con 0x7f27901f7c30 2024-09-16T07:34:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.859+0000 7f2787e00640 1 -- 172.21.15.71:0/2607060262 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2788004ce0 con 0x7f27901f7c30 2024-09-16T07:34:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.859+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27900c8260 con 0x7f27901f7c30 2024-09-16T07:34:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.859+0000 7f2787e00640 1 -- 172.21.15.71:0/2607060262 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788005280 con 0x7f27901f7c30 2024-09-16T07:34:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.859+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901460c0 con 0x7f27901f7c30 2024-09-16T07:34:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.860+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2790148b60 con 0x7f27901f7c30 2024-09-16T07:34:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.861+0000 7f2787e00640 1 -- 172.21.15.71:0/2607060262 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f2788005420 con 0x7f27901f7c30 2024-09-16T07:34:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.862+0000 7f2787e00640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f277007a230 0x7f277007c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:31.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.862+0000 7f2787e00640 1 -- 172.21.15.71:0/2607060262 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5795+0+0 (secure 0 0 0) 0x7f278809a360 con 0x7f27901f7c30 2024-09-16T07:34:31.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.863+0000 7f278fe00640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f277007a230 0x7f277007c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:31.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.863+0000 7f278fe00640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f277007a230 0x7f277007c6f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2780033000 tx=0x7f2780004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:31.867+0000 7f2787e00640 1 -- 172.21.15.71:0/2607060262 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2790148b60 con 0x7f27901f7c30 2024-09-16T07:34:32.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.064+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f279006fdf0 con 0x7f27901f7c30 2024-09-16T07:34:32.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.064+0000 7f2787e00640 1 -- 172.21.15.71:0/2607060262 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f2788063e80 con 0x7f27901f7c30 2024-09-16T07:34:32.064 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:34:32.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f277007a230 msgr2=0x7f277007c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f277007a230 0x7f277007c6f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2780033000 tx=0x7f2780004730 comp rx=0 tx=0).stop 2024-09-16T07:34:32.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27901f7c30 msgr2=0x7f27901f80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27901f7c30 0x7f27901f80e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f278800eb40 tx=0x7f278800c2d0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 shutdown_connections 2024-09-16T07:34:32.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f277007a230 0x7f277007c6f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27901f7c30 0x7f27901f80e0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f279015bc30 0x7f27901485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 --2- 172.21.15.71:0/2607060262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27900d0540 0x7f2790145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 >> 172.21.15.71:0/2607060262 conn(0x7f27900cfa20 msgr2=0x7f27900b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:32.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 shutdown_connections 2024-09-16T07:34:32.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.067+0000 7f2795800640 1 -- 172.21.15.71:0/2607060262 wait complete. 2024-09-16T07:34:32.084 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:34:32.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:32.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:34:32.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:34:32.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:32.169 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:34:32.217 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:34:32.218 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:34:32.218 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:32.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:34:32.254 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:34:32.254 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:34:32.254 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:32.293 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:34:32.294 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:34:32.294 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:34:32.294 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:32.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:34:32.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:34:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.491+0000 7f5135000640 1 -- 172.21.15.71:0/436596924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f513015f5a0 msgr2=0x7f51300b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.491+0000 7f5135000640 1 --2- 172.21.15.71:0/436596924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f513015f5a0 0x7f51300b2440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5120005fd0 tx=0x7f512002a540 comp rx=0 tx=0).stop 2024-09-16T07:34:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.491+0000 7f5135000640 1 -- 172.21.15.71:0/436596924 shutdown_connections 2024-09-16T07:34:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.491+0000 7f5135000640 1 --2- 172.21.15.71:0/436596924 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51300ccf00 0x7f51300acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.491+0000 7f5135000640 1 --2- 172.21.15.71:0/436596924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f513015f5a0 0x7f51300b2440 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.491+0000 7f5135000640 1 -- 172.21.15.71:0/436596924 >> 172.21.15.71:0/436596924 conn(0x7f51300b1950 msgr2=0x7f51300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.491+0000 7f5135000640 1 -- 172.21.15.71:0/436596924 shutdown_connections 2024-09-16T07:34:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.491+0000 7f5135000640 1 -- 172.21.15.71:0/436596924 wait complete. 2024-09-16T07:34:32.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.493+0000 7f5135000640 1 Processor -- start 2024-09-16T07:34:32.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.493+0000 7f5135000640 1 -- start start 2024-09-16T07:34:32.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f5135000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51300ad270 0x7f513014e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:32.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f5135000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51300ccf00 0x7f51300c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:32.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f5135000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513015f5a0 0x7f51300cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:32.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f5135000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f513006f2e0 con 0x7f51300ccf00 2024-09-16T07:34:32.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f5135000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f513006f160 con 0x7f51300ad270 2024-09-16T07:34:32.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f5135000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f513006f460 con 0x7f513015f5a0 2024-09-16T07:34:32.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51300ccf00 0x7f51300c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:32.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51300ccf00 0x7f51300c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41524/0 (socket says 172.21.15.71:41524) 2024-09-16T07:34:32.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512d400640 1 -- 172.21.15.71:0/2349992652 learned_addr learned my addr 172.21.15.71:0/2349992652 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:32.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512de00640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51300ad270 0x7f513014e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:32.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512e800640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513015f5a0 0x7f51300cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:32.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512d400640 1 -- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513015f5a0 msgr2=0x7f51300cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512d400640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513015f5a0 0x7f51300cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512d400640 1 -- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51300ad270 msgr2=0x7f513014e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512d400640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51300ad270 0x7f513014e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.494+0000 7f512d400640 1 -- 172.21.15.71:0/2349992652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5130156f80 con 0x7f51300ccf00 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.495+0000 7f512d400640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51300ccf00 0x7f51300c99b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f511c00d950 tx=0x7f511c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.495+0000 7f5126000640 1 -- 172.21.15.71:0/2349992652 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c017070 con 0x7f51300ccf00 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.495+0000 7f5126000640 1 -- 172.21.15.71:0/2349992652 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f511c009d20 con 0x7f51300ccf00 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.495+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51300ca5e0 con 0x7f51300ccf00 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.495+0000 7f5126000640 1 -- 172.21.15.71:0/2349992652 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0106c0 con 0x7f51300ccf00 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.495+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513014eaf0 con 0x7f51300ccf00 2024-09-16T07:34:32.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.496+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513006f990 con 0x7f51300ccf00 2024-09-16T07:34:32.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.497+0000 7f5126000640 1 -- 172.21.15.71:0/2349992652 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f511c010860 con 0x7f51300ccf00 2024-09-16T07:34:32.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.499+0000 7f5126000640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f511007a230 0x7f511007c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:32.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.499+0000 7f5126000640 1 -- 172.21.15.71:0/2349992652 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5795+0+0 (secure 0 0 0) 0x7f511c099bb0 con 0x7f51300ccf00 2024-09-16T07:34:32.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.499+0000 7f512de00640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f511007a230 0x7f511007c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:32.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.500+0000 7f512de00640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f511007a230 0x7f511007c6f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5120033000 tx=0x7f5120004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:32.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.502+0000 7f5126000640 1 -- 172.21.15.71:0/2349992652 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f511c0635b0 con 0x7f51300ccf00 2024-09-16T07:34:32.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.733+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f513009d8a0 con 0x7f51300ccf00 2024-09-16T07:34:32.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.734+0000 7f5126000640 1 -- 172.21.15.71:0/2349992652 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5231 (secure 0 0 0) 0x7f511c067fe0 con 0x7f51300ccf00 2024-09-16T07:34:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:32.734 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":67,"btime":"2024-09-16T07:33:44:783893+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3180037555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3180037555},{"type":"v1","addr":"172.21.15.71:6835","nonce":3180037555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3241229928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3241229928},{"type":"v1","addr":"172.21.15.177:6836","nonce":3241229928}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1569295783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1569295783},{"type":"v1","addr":"172.21.15.177:6837","nonce":1569295783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:33:31.441505+0000","modified":"2024-09-16T07:33:34.454314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/4054989006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4054989006},{"type":"v1","addr":"172.21.15.71:6833","nonce":4054989006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-09-16T07:34:32.734 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 67 2024-09-16T07:34:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f511007a230 msgr2=0x7f511007c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f511007a230 0x7f511007c6f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5120033000 tx=0x7f5120004730 comp rx=0 tx=0).stop 2024-09-16T07:34:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51300ccf00 msgr2=0x7f51300c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51300ccf00 0x7f51300c99b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f511c00d950 tx=0x7f511c00de10 comp rx=0 tx=0).stop 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 shutdown_connections 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f511007a230 0x7f511007c6f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513015f5a0 0x7f51300cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51300ccf00 0x7f51300c99b0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 --2- 172.21.15.71:0/2349992652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51300ad270 0x7f513014e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.736+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 >> 172.21.15.71:0/2349992652 conn(0x7f51300b1950 msgr2=0x7f513009d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.737+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 shutdown_connections 2024-09-16T07:34:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.737+0000 7f5135000640 1 -- 172.21.15.71:0/2349992652 wait complete. 2024-09-16T07:34:32.752 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:34:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.911+0000 7f9f3f200640 1 -- 172.21.15.71:0/1578335978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f400b4100 msgr2=0x7f9f400b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.911+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1578335978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f400b4100 0x7f9f400b1770 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9f400ce3d0 tx=0x7f9f3802a560 comp rx=0 tx=0).stop 2024-09-16T07:34:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.912+0000 7f9f3f200640 1 -- 172.21.15.71:0/1578335978 shutdown_connections 2024-09-16T07:34:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.912+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1578335978 >> v2:172.21.15.177:3300/0 conn(0x7f9f400b5370 0x7f9f400b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.912+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1578335978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f400b4100 0x7f9f400b1770 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.912+0000 7f9f3f200640 1 -- 172.21.15.71:0/1578335978 >> 172.21.15.71:0/1578335978 conn(0x7f9f400d3df0 msgr2=0x7f9f400d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:32.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.912+0000 7f9f3f200640 1 -- 172.21.15.71:0/1578335978 shutdown_connections 2024-09-16T07:34:32.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.912+0000 7f9f3f200640 1 -- 172.21.15.71:0/1578335978 wait complete. 2024-09-16T07:34:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.913+0000 7f9f3f200640 1 Processor -- start 2024-09-16T07:34:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.913+0000 7f9f3f200640 1 -- start start 2024-09-16T07:34:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f400b4100 0x7f9f4006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3f200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f400b5370 0x7f9f4015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40161440 0x7f9f4015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9f400b2040 con 0x7f9f40161440 2024-09-16T07:34:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f400b1ec0 con 0x7f9f400b4100 2024-09-16T07:34:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3f200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9f400b21c0 con 0x7f9f400b5370 2024-09-16T07:34:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40161440 0x7f9f4015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40161440 0x7f9f4015ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41532/0 (socket says 172.21.15.71:41532) 2024-09-16T07:34:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3e800640 1 -- 172.21.15.71:0/1454903494 learned_addr learned my addr 172.21.15.71:0/1454903494 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.914+0000 7f9f3de00640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f400b4100 0x7f9f4006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f3d400640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f400b5370 0x7f9f4015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f3e800640 1 -- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f400b5370 msgr2=0x7f9f4015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f3e800640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f400b5370 0x7f9f4015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f3e800640 1 -- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f400b4100 msgr2=0x7f9f4006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f3e800640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f400b4100 0x7f9f4006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f3e800640 1 -- 172.21.15.71:0/1454903494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4015b790 con 0x7f9f40161440 2024-09-16T07:34:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f3e800640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40161440 0x7f9f4015ca00 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009230 tx=0x7f9f38003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f36000640 1 -- 172.21.15.71:0/1454903494 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3803f070 con 0x7f9f40161440 2024-09-16T07:34:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f36000640 1 -- 172.21.15.71:0/1454903494 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f380047a0 con 0x7f9f40161440 2024-09-16T07:34:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f36000640 1 -- 172.21.15.71:0/1454903494 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38035030 con 0x7f9f40161440 2024-09-16T07:34:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.915+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4015ba70 con 0x7f9f40161440 2024-09-16T07:34:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.916+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f400d0c90 con 0x7f9f40161440 2024-09-16T07:34:32.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.916+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f400b1770 con 0x7f9f40161440 2024-09-16T07:34:32.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.918+0000 7f9f36000640 1 -- 172.21.15.71:0/1454903494 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f9f38042070 con 0x7f9f40161440 2024-09-16T07:34:32.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.919+0000 7f9f36000640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f2007a230 0x7f9f2007c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:32.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.919+0000 7f9f36000640 1 -- 172.21.15.71:0/1454903494 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5851+0+0 (secure 0 0 0) 0x7f9f380ba850 con 0x7f9f40161440 2024-09-16T07:34:32.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.919+0000 7f9f3de00640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f2007a230 0x7f9f2007c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:32.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.920+0000 7f9f3de00640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f2007a230 0x7f9f2007c6f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f30001e20 tx=0x7f9f30009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:32.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:32.923+0000 7f9f36000640 1 -- 172.21.15.71:0/1454903494 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9f380842d0 con 0x7f9f40161440 2024-09-16T07:34:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.202+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f4015d2a0 con 0x7f9f40161440 2024-09-16T07:34:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.203+0000 7f9f36000640 1 -- 172.21.15.71:0/1454903494 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5231 (secure 0 0 0) 0x7f9f38088d00 con 0x7f9f40161440 2024-09-16T07:34:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:33.203 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":67,"btime":"2024-09-16T07:33:44:783893+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3180037555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3180037555},{"type":"v1","addr":"172.21.15.71:6835","nonce":3180037555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3241229928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3241229928},{"type":"v1","addr":"172.21.15.177:6836","nonce":3241229928}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1569295783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1569295783},{"type":"v1","addr":"172.21.15.177:6837","nonce":1569295783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:33:31.441505+0000","modified":"2024-09-16T07:33:34.454314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/4054989006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4054989006},{"type":"v1","addr":"172.21.15.71:6833","nonce":4054989006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-09-16T07:34:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 67 2024-09-16T07:34:33.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f2007a230 msgr2=0x7f9f2007c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:33.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f2007a230 0x7f9f2007c6f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f30001e20 tx=0x7f9f30009040 comp rx=0 tx=0).stop 2024-09-16T07:34:33.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40161440 msgr2=0x7f9f4015ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:33.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40161440 0x7f9f4015ca00 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009230 tx=0x7f9f38003820 comp rx=0 tx=0).stop 2024-09-16T07:34:33.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 shutdown_connections 2024-09-16T07:34:33.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9f2007a230 0x7f9f2007c6f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f40161440 0x7f9f4015ca00 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f400b5370 0x7f9f4015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.207+0000 7f9f3f200640 1 --2- 172.21.15.71:0/1454903494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f400b4100 0x7f9f4006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.208+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 >> 172.21.15.71:0/1454903494 conn(0x7f9f400d3df0 msgr2=0x7f9f401600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:33.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.208+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 shutdown_connections 2024-09-16T07:34:33.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.208+0000 7f9f3f200640 1 -- 172.21.15.71:0/1454903494 wait complete. 2024-09-16T07:34:33.234 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:34:33.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.487+0000 7f6e49400640 1 -- 172.21.15.71:0/499831231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 msgr2=0x7f6e4406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:33.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.487+0000 7f6e49400640 1 --2- 172.21.15.71:0/499831231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 0x7f6e4406ff60 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f6e300082a0 tx=0x7f6e3002c5a0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.487+0000 7f6e49400640 1 -- 172.21.15.71:0/499831231 shutdown_connections 2024-09-16T07:34:33.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.487+0000 7f6e49400640 1 --2- 172.21.15.71:0/499831231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 0x7f6e4406ff60 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.487+0000 7f6e49400640 1 -- 172.21.15.71:0/499831231 >> 172.21.15.71:0/499831231 conn(0x7f6e440b1950 msgr2=0x7f6e440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:33.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.487+0000 7f6e49400640 1 -- 172.21.15.71:0/499831231 shutdown_connections 2024-09-16T07:34:33.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.488+0000 7f6e49400640 1 -- 172.21.15.71:0/499831231 wait complete. 2024-09-16T07:34:33.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.490+0000 7f6e49400640 1 Processor -- start 2024-09-16T07:34:33.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.491+0000 7f6e49400640 1 -- start start 2024-09-16T07:34:33.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.493+0000 7f6e49400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e44070920 0x7f6e4413a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:33.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.493+0000 7f6e49400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e440b3f00 0x7f6e4415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:33.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.493+0000 7f6e49400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 0x7f6e44138f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:33.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.493+0000 7f6e49400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6e440b4ee0 con 0x7f6e441438b0 2024-09-16T07:34:33.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.493+0000 7f6e49400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e440b4d60 con 0x7f6e44070920 2024-09-16T07:34:33.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.493+0000 7f6e49400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6e440cc5f0 con 0x7f6e440b3f00 2024-09-16T07:34:33.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.494+0000 7f6e42800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e44070920 0x7f6e4413a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:33.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.494+0000 7f6e42800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e44070920 0x7f6e4413a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:47648/0 (socket says 172.21.15.71:47648) 2024-09-16T07:34:33.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.494+0000 7f6e42800640 1 -- 172.21.15.71:0/3837213674 learned_addr learned my addr 172.21.15.71:0/3837213674 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:33.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.494+0000 7f6e41e00640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e440b3f00 0x7f6e4415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:33.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.494+0000 7f6e43200640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 0x7f6e44138f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:33.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.494+0000 7f6e42800640 1 -- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e440b3f00 msgr2=0x7f6e4415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:33.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.494+0000 7f6e42800640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e440b3f00 0x7f6e4415abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.495+0000 7f6e42800640 1 -- 172.21.15.71:0/3837213674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 msgr2=0x7f6e44138f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:33.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.495+0000 7f6e42800640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 0x7f6e44138f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.495+0000 7f6e42800640 1 -- 172.21.15.71:0/3837213674 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4413b100 con 0x7f6e44070920 2024-09-16T07:34:33.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.495+0000 7f6e43200640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 0x7f6e44138f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:33.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.495+0000 7f6e42800640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e44070920 0x7f6e4413a970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00cbd0 tx=0x7f6e3c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:33.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.496+0000 7f6e3aa00640 1 -- 172.21.15.71:0/3837213674 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c016020 con 0x7f6e44070920 2024-09-16T07:34:33.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.496+0000 7f6e3aa00640 1 -- 172.21.15.71:0/3837213674 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e3c00ae30 con 0x7f6e44070920 2024-09-16T07:34:33.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.496+0000 7f6e3aa00640 1 -- 172.21.15.71:0/3837213674 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c012660 con 0x7f6e44070920 2024-09-16T07:34:33.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.497+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e44138880 con 0x7f6e44070920 2024-09-16T07:34:33.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.497+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e44072a40 con 0x7f6e44070920 2024-09-16T07:34:33.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.499+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e44072830 con 0x7f6e44070920 2024-09-16T07:34:33.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.502+0000 7f6e3aa00640 1 -- 172.21.15.71:0/3837213674 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f6e3c019020 con 0x7f6e44070920 2024-09-16T07:34:33.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.504+0000 7f6e3aa00640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e2407a280 0x7f6e2407c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:33.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.504+0000 7f6e3aa00640 1 -- 172.21.15.71:0/3837213674 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5851+0+0 (secure 0 0 0) 0x7f6e3c09a0d0 con 0x7f6e44070920 2024-09-16T07:34:33.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.505+0000 7f6e41e00640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e2407a280 0x7f6e2407c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:33.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.506+0000 7f6e41e00640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e2407a280 0x7f6e2407c740 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6e30034000 tx=0x7f6e30004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:33.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.510+0000 7f6e3aa00640 1 -- 172.21.15.71:0/3837213674 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6e3c020020 con 0x7f6e44070920 2024-09-16T07:34:33.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.822+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e440ccb80 con 0x7f6e44070920 2024-09-16T07:34:33.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.823+0000 7f6e3aa00640 1 -- 172.21.15.71:0/3837213674 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16943 (secure 0 0 0) 0x7f6e440cdae0 con 0x7f6e44070920 2024-09-16T07:34:33.823 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:33.823 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":107,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:34:32.674714+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:33:29.200091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T07:33:30.405982+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.c","mappings":[{"from":5,"to":0}]},{"pgid":"14.1e","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:34:33.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.825+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e2407a280 msgr2=0x7f6e2407c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:33.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.825+0000 7f6e49400640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e2407a280 0x7f6e2407c740 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6e30034000 tx=0x7f6e30004730 comp rx=0 tx=0).stop 2024-09-16T07:34:33.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e44070920 msgr2=0x7f6e4413a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e44070920 0x7f6e4413a970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00cbd0 tx=0x7f6e3c00a430 comp rx=0 tx=0).stop 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 shutdown_connections 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e2407a280 0x7f6e2407c740 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e441438b0 0x7f6e44138f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e440b3f00 0x7f6e4415abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 --2- 172.21.15.71:0/3837213674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e44070920 0x7f6e4413a970 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 >> 172.21.15.71:0/3837213674 conn(0x7f6e440b1950 msgr2=0x7f6e4415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 shutdown_connections 2024-09-16T07:34:33.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:33.826+0000 7f6e49400640 1 -- 172.21.15.71:0/3837213674 wait complete. 2024-09-16T07:34:33.842 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:34:33.842 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:34:34.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.001+0000 7fbb4d800640 1 -- 172.21.15.71:0/2209648909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb4813efa0 msgr2=0x7fbb480cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.001+0000 7fbb4d800640 1 --2- 172.21.15.71:0/2209648909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb4813efa0 0x7fbb480cd1d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fbb480d03a0 tx=0x7fbb3802a600 comp rx=0 tx=0).stop 2024-09-16T07:34:34.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.001+0000 7fbb4d800640 1 -- 172.21.15.71:0/2209648909 shutdown_connections 2024-09-16T07:34:34.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.001+0000 7fbb4d800640 1 --2- 172.21.15.71:0/2209648909 >> v2:172.21.15.177:3300/0 conn(0x7fbb480cff70 0x7fbb4806d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.001+0000 7fbb4d800640 1 --2- 172.21.15.71:0/2209648909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb4813efa0 0x7fbb480cd1d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.001+0000 7fbb4d800640 1 -- 172.21.15.71:0/2209648909 >> 172.21.15.71:0/2209648909 conn(0x7fbb480b2230 msgr2=0x7fbb480b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:34.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.001+0000 7fbb4d800640 1 -- 172.21.15.71:0/2209648909 shutdown_connections 2024-09-16T07:34:34.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.001+0000 7fbb4d800640 1 -- 172.21.15.71:0/2209648909 wait complete. 2024-09-16T07:34:34.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.003+0000 7fbb4d800640 1 Processor -- start 2024-09-16T07:34:34.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.003+0000 7fbb4d800640 1 -- start start 2024-09-16T07:34:34.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb4d800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb4806cb70 0x7fbb480b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb4d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb480cff70 0x7fbb4813b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb4d800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb4813efa0 0x7fbb4813bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb4d800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbb480cd890 con 0x7fbb480cff70 2024-09-16T07:34:34.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb4d800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb480cd710 con 0x7fbb4813efa0 2024-09-16T07:34:34.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb4d800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb480737d0 con 0x7fbb4806cb70 2024-09-16T07:34:34.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb480cff70 0x7fbb4813b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb480cff70 0x7fbb4813b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41576/0 (socket says 172.21.15.71:41576) 2024-09-16T07:34:34.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46000640 1 -- 172.21.15.71:0/1226366351 learned_addr learned my addr 172.21.15.71:0/1226366351 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:34.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46a00640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb4806cb70 0x7fbb480b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb47400640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb4813efa0 0x7fbb4813bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46000640 1 -- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb4806cb70 msgr2=0x7fbb480b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46000640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb4806cb70 0x7fbb480b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46000640 1 -- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb4813efa0 msgr2=0x7fbb4813bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46000640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb4813efa0 0x7fbb4813bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46000640 1 -- 172.21.15.71:0/1226366351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb481f7ca0 con 0x7fbb480cff70 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb46a00640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb4806cb70 0x7fbb480b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.004+0000 7fbb47400640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb4813efa0 0x7fbb4813bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.005+0000 7fbb46000640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb480cff70 0x7fbb4813b4b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400c530 tx=0x7fbb3400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.005+0000 7fbb3ea00640 1 -- 172.21.15.71:0/1226366351 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb340052d0 con 0x7fbb480cff70 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.005+0000 7fbb3ea00640 1 -- 172.21.15.71:0/1226366351 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb34005470 con 0x7fbb480cff70 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.005+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb481f7f80 con 0x7fbb480cff70 2024-09-16T07:34:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.005+0000 7fbb3ea00640 1 -- 172.21.15.71:0/1226366351 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34012870 con 0x7fbb480cff70 2024-09-16T07:34:34.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.005+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb481f8540 con 0x7fbb480cff70 2024-09-16T07:34:34.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.006+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb48073bf0 con 0x7fbb480cff70 2024-09-16T07:34:34.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.007+0000 7fbb3ea00640 1 -- 172.21.15.71:0/1226366351 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fbb340035f0 con 0x7fbb480cff70 2024-09-16T07:34:34.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.009+0000 7fbb3ea00640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb2807a230 0x7fbb2807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.009+0000 7fbb3ea00640 1 -- 172.21.15.71:0/1226366351 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5851+0+0 (secure 0 0 0) 0x7fbb34099360 con 0x7fbb480cff70 2024-09-16T07:34:34.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.009+0000 7fbb46a00640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb2807a230 0x7fbb2807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.010+0000 7fbb46a00640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb2807a230 0x7fbb2807c6f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbb38033000 tx=0x7fbb38004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:34.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.013+0000 7fbb3ea00640 1 -- 172.21.15.71:0/1226366351 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb340634b0 con 0x7fbb480cff70 2024-09-16T07:34:34.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.243+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb481f8710 con 0x7fbb480cff70 2024-09-16T07:34:34.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.244+0000 7fbb3ea00640 1 -- 172.21.15.71:0/1226366351 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5231 (secure 0 0 0) 0x7fbb34062db0 con 0x7fbb480cff70 2024-09-16T07:34:34.243 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:34.243 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":67,"btime":"2024-09-16T07:33:44:783893+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3180037555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3180037555},{"type":"v1","addr":"172.21.15.71:6835","nonce":3180037555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3241229928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3241229928},{"type":"v1","addr":"172.21.15.177:6836","nonce":3241229928}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1569295783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1569295783},{"type":"v1","addr":"172.21.15.177:6837","nonce":1569295783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:33:31.441505+0000","modified":"2024-09-16T07:33:34.454314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/4054989006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4054989006},{"type":"v1","addr":"172.21.15.71:6833","nonce":4054989006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-09-16T07:34:34.244 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 67 2024-09-16T07:34:34.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb2807a230 msgr2=0x7fbb2807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb2807a230 0x7fbb2807c6f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbb38033000 tx=0x7fbb38004730 comp rx=0 tx=0).stop 2024-09-16T07:34:34.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb480cff70 msgr2=0x7fbb4813b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb480cff70 0x7fbb4813b4b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400c530 tx=0x7fbb3400c9f0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 shutdown_connections 2024-09-16T07:34:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb2807a230 0x7fbb2807c6f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb4813efa0 0x7fbb4813bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb480cff70 0x7fbb4813b4b0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 --2- 172.21.15.71:0/1226366351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb4806cb70 0x7fbb480b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 >> 172.21.15.71:0/1226366351 conn(0x7fbb480b2230 msgr2=0x7fbb4815e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 shutdown_connections 2024-09-16T07:34:34.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.246+0000 7fbb4d800640 1 -- 172.21.15.71:0/1226366351 wait complete. 2024-09-16T07:34:34.263 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:34:34.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.421+0000 7f9edd000640 1 -- 172.21.15.71:0/433733274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b2780 msgr2=0x7f9ed8168c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.421+0000 7f9edd000640 1 --2- 172.21.15.71:0/433733274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b2780 0x7f9ed8168c30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80d4ea0 tx=0x7f9ed002a510 comp rx=0 tx=0).stop 2024-09-16T07:34:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.421+0000 7f9edd000640 1 -- 172.21.15.71:0/433733274 shutdown_connections 2024-09-16T07:34:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.421+0000 7f9edd000640 1 --2- 172.21.15.71:0/433733274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b2780 0x7f9ed8168c30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.421+0000 7f9edd000640 1 -- 172.21.15.71:0/433733274 >> 172.21.15.71:0/433733274 conn(0x7f9ed80d3df0 msgr2=0x7f9ed80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.421+0000 7f9edd000640 1 -- 172.21.15.71:0/433733274 shutdown_connections 2024-09-16T07:34:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.421+0000 7f9edd000640 1 -- 172.21.15.71:0/433733274 wait complete. 2024-09-16T07:34:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.422+0000 7f9edd000640 1 Processor -- start 2024-09-16T07:34:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.422+0000 7f9edd000640 1 -- start start 2024-09-16T07:34:34.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.423+0000 7f9edd000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed80b2780 0x7f9ed814dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.423+0000 7f9edd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b5370 0x7f9ed815b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.423+0000 7f9edd000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ed8161440 0x7f9ed80b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.423+0000 7f9edd000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81692f0 con 0x7f9ed8161440 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.423+0000 7f9edd000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8169170 con 0x7f9ed80b5370 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.423+0000 7f9edd000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed8169470 con 0x7f9ed80b2780 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.423+0000 7f9ed7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ed8161440 0x7f9ed80b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.423+0000 7f9ed7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ed8161440 0x7f9ed80b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41582/0 (socket says 172.21.15.71:41582) 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed7e00640 1 -- 172.21.15.71:0/1581119174 learned_addr learned my addr 172.21.15.71:0/1581119174 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed6a00640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b5370 0x7f9ed815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed7400640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed80b2780 0x7f9ed814dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed7e00640 1 -- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed80b2780 msgr2=0x7f9ed814dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed7e00640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed80b2780 0x7f9ed814dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed7e00640 1 -- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b5370 msgr2=0x7f9ed815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed7e00640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b5370 0x7f9ed815b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed7e00640 1 -- 172.21.15.71:0/1581119174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed8159d90 con 0x7f9ed8161440 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed6a00640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b5370 0x7f9ed815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ed7e00640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ed8161440 0x7f9ed80b6c20 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009920 tx=0x7f9ed0032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ecf400640 1 -- 172.21.15.71:0/1581119174 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0040070 con 0x7f9ed8161440 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.424+0000 7f9ecf400640 1 -- 172.21.15.71:0/1581119174 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ed0005310 con 0x7f9ed8161440 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.425+0000 7f9ecf400640 1 -- 172.21.15.71:0/1581119174 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0005c90 con 0x7f9ed8161440 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.425+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed815a8d0 con 0x7f9ed8161440 2024-09-16T07:34:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.425+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed815ae90 con 0x7f9ed8161440 2024-09-16T07:34:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.426+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed80b34b0 con 0x7f9ed8161440 2024-09-16T07:34:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.427+0000 7f9ecf400640 1 -- 172.21.15.71:0/1581119174 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f9ed0039040 con 0x7f9ed8161440 2024-09-16T07:34:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.428+0000 7f9ecf400640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9eb407a230 0x7f9eb407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.428+0000 7f9ecf400640 1 -- 172.21.15.71:0/1581119174 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5851+0+0 (secure 0 0 0) 0x7f9ed00ba790 con 0x7f9ed8161440 2024-09-16T07:34:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.428+0000 7f9ed7400640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9eb407a230 0x7f9eb407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.429+0000 7f9ed7400640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9eb407a230 0x7f9eb407c6f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80014a0 tx=0x7f9ec8009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.432+0000 7f9ecf400640 1 -- 172.21.15.71:0/1581119174 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9ed0084900 con 0x7f9ed8161440 2024-09-16T07:34:34.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.651+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ed80cac80 con 0x7f9ed8161440 2024-09-16T07:34:34.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.652+0000 7f9ecf400640 1 -- 172.21.15.71:0/1581119174 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5231 (secure 0 0 0) 0x7f9ed0084200 con 0x7f9ed8161440 2024-09-16T07:34:34.652 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:34.652 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":67,"btime":"2024-09-16T07:33:44:783893+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3180037555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3180037555},{"type":"v1","addr":"172.21.15.71:6835","nonce":3180037555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3241229928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3241229928},{"type":"v1","addr":"172.21.15.177:6836","nonce":3241229928}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1569295783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1569295783},{"type":"v1","addr":"172.21.15.177:6837","nonce":1569295783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:33:31.441505+0000","modified":"2024-09-16T07:33:34.454314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/4054989006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4054989006},{"type":"v1","addr":"172.21.15.71:6833","nonce":4054989006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-09-16T07:34:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 67 2024-09-16T07:34:34.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9eb407a230 msgr2=0x7f9eb407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9eb407a230 0x7f9eb407c6f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80014a0 tx=0x7f9ec8009040 comp rx=0 tx=0).stop 2024-09-16T07:34:34.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ed8161440 msgr2=0x7f9ed80b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ed8161440 0x7f9ed80b6c20 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009920 tx=0x7f9ed0032b90 comp rx=0 tx=0).stop 2024-09-16T07:34:34.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 shutdown_connections 2024-09-16T07:34:34.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9eb407a230 0x7f9eb407c6f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ed8161440 0x7f9ed80b6c20 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b5370 0x7f9ed815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 --2- 172.21.15.71:0/1581119174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed80b2780 0x7f9ed814dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 >> 172.21.15.71:0/1581119174 conn(0x7f9ed80d3df0 msgr2=0x7f9ed815fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:34.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.654+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 shutdown_connections 2024-09-16T07:34:34.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.655+0000 7f9edd000640 1 -- 172.21.15.71:0/1581119174 wait complete. 2024-09-16T07:34:34.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:34:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.828+0000 7f36fb200640 1 -- 172.21.15.71:0/3919416301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f4073fb0 msgr2=0x7f36f4074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.828+0000 7f36fb200640 1 --2- 172.21.15.71:0/3919416301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f4073fb0 0x7f36f4074460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009830 tx=0x7f36ec02a480 comp rx=0 tx=0).stop 2024-09-16T07:34:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.828+0000 7f36fb200640 1 -- 172.21.15.71:0/3919416301 shutdown_connections 2024-09-16T07:34:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.828+0000 7f36fb200640 1 --2- 172.21.15.71:0/3919416301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f4073fb0 0x7f36f4074460 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.828+0000 7f36fb200640 1 --2- 172.21.15.71:0/3919416301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f415f5a0 0x7f36f40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.828+0000 7f36fb200640 1 -- 172.21.15.71:0/3919416301 >> 172.21.15.71:0/3919416301 conn(0x7f36f40b1950 msgr2=0x7f36f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:34.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.828+0000 7f36fb200640 1 -- 172.21.15.71:0/3919416301 shutdown_connections 2024-09-16T07:34:34.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.828+0000 7f36fb200640 1 -- 172.21.15.71:0/3919416301 wait complete. 2024-09-16T07:34:34.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.830+0000 7f36fb200640 1 Processor -- start 2024-09-16T07:34:34.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.830+0000 7f36fb200640 1 -- start start 2024-09-16T07:34:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36fb200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4073fb0 0x7f36f4149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36fb200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f40ccf00 0x7f36f40d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36fb200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f415f5a0 0x7f36f409ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36fb200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36f406f2e0 con 0x7f36f4073fb0 2024-09-16T07:34:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36fb200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36f406f160 con 0x7f36f415f5a0 2024-09-16T07:34:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36fb200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36f406f460 con 0x7f36f40ccf00 2024-09-16T07:34:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4073fb0 0x7f36f4149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4073fb0 0x7f36f4149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60678/0 (socket says 172.21.15.71:60678) 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3e00640 1 -- 172.21.15.71:0/1431844107 learned_addr learned my addr 172.21.15.71:0/1431844107 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f9200640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f415f5a0 0x7f36f409ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3400640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f40ccf00 0x7f36f40d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3e00640 1 -- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f40ccf00 msgr2=0x7f36f40d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3e00640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f40ccf00 0x7f36f40d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3e00640 1 -- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f415f5a0 msgr2=0x7f36f409ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3e00640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f415f5a0 0x7f36f409ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.831+0000 7f36f3e00640 1 -- 172.21.15.71:0/1431844107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f4149bd0 con 0x7f36f4073fb0 2024-09-16T07:34:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.832+0000 7f36f3e00640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4073fb0 0x7f36f4149690 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f36e400ef60 tx=0x7f36e400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.832+0000 7f36ebe00640 1 -- 172.21.15.71:0/1431844107 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003970 con 0x7f36f4073fb0 2024-09-16T07:34:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.832+0000 7f36ebe00640 1 -- 172.21.15.71:0/1431844107 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36e4003c00 con 0x7f36f4073fb0 2024-09-16T07:34:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.832+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f41f82b0 con 0x7f36f4073fb0 2024-09-16T07:34:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.832+0000 7f36ebe00640 1 -- 172.21.15.71:0/1431844107 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4004020 con 0x7f36f4073fb0 2024-09-16T07:34:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.832+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f41f8870 con 0x7f36f4073fb0 2024-09-16T07:34:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.833+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f406f9f0 con 0x7f36f4073fb0 2024-09-16T07:34:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.834+0000 7f36ebe00640 1 -- 172.21.15.71:0/1431844107 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f36e401d020 con 0x7f36f4073fb0 2024-09-16T07:34:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.835+0000 7f36ebe00640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36d407a230 0x7f36d407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.835+0000 7f36ebe00640 1 -- 172.21.15.71:0/1431844107 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5851+0+0 (secure 0 0 0) 0x7f36e4099cc0 con 0x7f36f4073fb0 2024-09-16T07:34:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.836+0000 7f36f3400640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36d407a230 0x7f36d407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.837+0000 7f36f3400640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36d407a230 0x7f36d407c6f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f36f41f4b60 tx=0x7f36e0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:34.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:34.839+0000 7f36ebe00640 1 -- 172.21.15.71:0/1431844107 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f36f4149bd0 con 0x7f36f4073fb0 2024-09-16T07:34:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.022+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36f415dea0 con 0x7f36f4073fb0 2024-09-16T07:34:35.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.023+0000 7f36ebe00640 1 -- 172.21.15.71:0/1431844107 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16943 (secure 0 0 0) 0x7f36e4063e30 con 0x7f36f4073fb0 2024-09-16T07:34:35.023 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:35.024 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":107,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:34:32.674714+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:33:29.200091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T07:33:30.405982+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.c","mappings":[{"from":5,"to":0}]},{"pgid":"14.1e","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:34:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36d407a230 msgr2=0x7f36d407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36d407a230 0x7f36d407c6f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f36f41f4b60 tx=0x7f36e0009040 comp rx=0 tx=0).stop 2024-09-16T07:34:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4073fb0 msgr2=0x7f36f4149690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4073fb0 0x7f36f4149690 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f36e400ef60 tx=0x7f36e400ef90 comp rx=0 tx=0).stop 2024-09-16T07:34:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 shutdown_connections 2024-09-16T07:34:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36d407a230 0x7f36d407c6f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f415f5a0 0x7f36f409ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f40ccf00 0x7f36f40d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 --2- 172.21.15.71:0/1431844107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4073fb0 0x7f36f4149690 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 >> 172.21.15.71:0/1431844107 conn(0x7f36f40b1950 msgr2=0x7f36f415e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 shutdown_connections 2024-09-16T07:34:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.026+0000 7f36fb200640 1 -- 172.21.15.71:0/1431844107 wait complete. 2024-09-16T07:34:35.044 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 -- 172.21.15.71:0/2701381299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ad270 msgr2=0x7f8b7c073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 --2- 172.21.15.71:0/2701381299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ad270 0x7f8b7c073fb0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c075130 tx=0x7f8b7402a980 comp rx=0 tx=0).stop 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 -- 172.21.15.71:0/2701381299 shutdown_connections 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 --2- 172.21.15.71:0/2701381299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ad270 0x7f8b7c073fb0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 --2- 172.21.15.71:0/2701381299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c0ccf00 0x7f8b7c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 --2- 172.21.15.71:0/2701381299 >> v2:172.21.15.177:3300/0 conn(0x7f8b7c15f5a0 0x7f8b7c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 -- 172.21.15.71:0/2701381299 >> 172.21.15.71:0/2701381299 conn(0x7f8b7c0b1950 msgr2=0x7f8b7c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 -- 172.21.15.71:0/2701381299 shutdown_connections 2024-09-16T07:34:35.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.202+0000 7f8b81200640 1 -- 172.21.15.71:0/2701381299 wait complete. 2024-09-16T07:34:35.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.203+0000 7f8b81200640 1 Processor -- start 2024-09-16T07:34:35.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.204+0000 7f8b81200640 1 -- start start 2024-09-16T07:34:35.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.204+0000 7f8b81200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c0ad270 0x7f8b7c1f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.204+0000 7f8b81200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ccf00 0x7f8b7c1e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b81200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c15f5a0 0x7f8b7c1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b81200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c06f160 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b81200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c074780 con 0x7f8b7c15f5a0 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b81200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c06f2e0 con 0x7f8b7c0ad270 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ccf00 0x7f8b7c1e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ccf00 0x7f8b7c1e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60702/0 (socket says 172.21.15.71:60702) 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79400640 1 -- 172.21.15.71:0/1005994208 learned_addr learned my addr 172.21.15.71:0/1005994208 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79e00640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c0ad270 0x7f8b7c1f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b7a800640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c15f5a0 0x7f8b7c1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79400640 1 -- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c0ad270 msgr2=0x7f8b7c1f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79400640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c0ad270 0x7f8b7c1f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79400640 1 -- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c15f5a0 msgr2=0x7f8b7c1441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79400640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c15f5a0 0x7f8b7c1441a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.205+0000 7f8b79400640 1 -- 172.21.15.71:0/1005994208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c0b35c0 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.206+0000 7f8b79400640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ccf00 0x7f8b7c1e8db0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800e9f0 tx=0x7f8b6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:35.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.206+0000 7f8b72000640 1 -- 172.21.15.71:0/1005994208 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800ccc0 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.206+0000 7f8b72000640 1 -- 172.21.15.71:0/1005994208 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b6800ce60 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.206+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7c0b38a0 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.206+0000 7f8b72000640 1 -- 172.21.15.71:0/1005994208 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68005260 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.206+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7c0b3e60 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.207+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7c06f810 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.208+0000 7f8b72000640 1 -- 172.21.15.71:0/1005994208 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f8b68005400 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.210+0000 7f8b72000640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b5c07a230 0x7f8b5c07c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.211+0000 7f8b72000640 1 -- 172.21.15.71:0/1005994208 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5851+0+0 (secure 0 0 0) 0x7f8b6809a110 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.211+0000 7f8b79e00640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b5c07a230 0x7f8b5c07c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:35.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.212+0000 7f8b79e00640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b5c07a230 0x7f8b5c07c6f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009b30 tx=0x7f8b6c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:35.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.214+0000 7f8b72000640 1 -- 172.21.15.71:0/1005994208 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b68063c10 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.434+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8b7c15db40 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.434 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:34:35.434+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:34:35.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.436+0000 7f8b72000640 1 -- 172.21.15.71:0/1005994208 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f8b68068640 con 0x7f8b7c0ccf00 2024-09-16T07:34:35.436 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:34:35.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.439+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b5c07a230 msgr2=0x7f8b5c07c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.439+0000 7f8b81200640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b5c07a230 0x7f8b5c07c6f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009b30 tx=0x7f8b6c005f90 comp rx=0 tx=0).stop 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.439+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ccf00 msgr2=0x7f8b7c1e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.439+0000 7f8b81200640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ccf00 0x7f8b7c1e8db0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800e9f0 tx=0x7f8b6800eeb0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.440+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 shutdown_connections 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.440+0000 7f8b81200640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8b5c07a230 0x7f8b5c07c6f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.440+0000 7f8b81200640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c15f5a0 0x7f8b7c1441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.440+0000 7f8b81200640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b7c0ccf00 0x7f8b7c1e8db0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.440+0000 7f8b81200640 1 --2- 172.21.15.71:0/1005994208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c0ad270 0x7f8b7c1f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.440+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 >> 172.21.15.71:0/1005994208 conn(0x7f8b7c0b1950 msgr2=0x7f8b7c09cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.440+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 shutdown_connections 2024-09-16T07:34:35.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.440+0000 7f8b81200640 1 -- 172.21.15.71:0/1005994208 wait complete. 2024-09-16T07:34:35.456 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:34:35.509 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:34:35.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.623+0000 7f0ba9400640 1 -- 172.21.15.71:0/473025150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 msgr2=0x7f0b88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.623+0000 7f0ba9400640 1 --2- 172.21.15.71:0/473025150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 0x7f0b88004c80 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009b50 tx=0x7f0b88005830 comp rx=0 tx=0).stop 2024-09-16T07:34:35.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.623+0000 7f0ba3200640 1 -- 172.21.15.71:0/473025150 reap_dead start 2024-09-16T07:34:35.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.623+0000 7f0ba9400640 1 -- 172.21.15.71:0/473025150 shutdown_connections 2024-09-16T07:34:35.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.623+0000 7f0ba9400640 1 -- 172.21.15.71:0/473025150 >> 172.21.15.71:0/473025150 conn(0x7f0ba40b1840 msgr2=0x7f0ba40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:35.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.623+0000 7f0ba9400640 1 -- 172.21.15.71:0/473025150 shutdown_connections 2024-09-16T07:34:35.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.623+0000 7f0ba9400640 1 -- 172.21.15.71:0/473025150 wait complete. 2024-09-16T07:34:35.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.625+0000 7f0ba9400640 1 Processor -- start 2024-09-16T07:34:35.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.625+0000 7f0ba9400640 1 -- start start 2024-09-16T07:34:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 0x7f0ba41f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba9400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba4142890 0x7f0ba40b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba9400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba40b3840 0x7f0ba4140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba9400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40cdf40 con 0x7f0b88004800 2024-09-16T07:34:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba9400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40cddc0 con 0x7f0ba4142890 2024-09-16T07:34:35.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba9400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba40ce0c0 con 0x7f0ba40b3840 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 0x7f0ba41f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 0x7f0ba41f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60728/0 (socket says 172.21.15.71:60728) 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 -- 172.21.15.71:0/1845138837 learned_addr learned my addr 172.21.15.71:0/1845138837 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba1e00640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba4142890 0x7f0ba40b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba3200640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba40b3840 0x7f0ba4140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 -- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba40b3840 msgr2=0x7f0ba4140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba40b3840 0x7f0ba4140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 -- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba4142890 msgr2=0x7f0ba40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba4142890 0x7f0ba40b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 -- 172.21.15.71:0/1845138837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba41f4fc0 con 0x7f0b88004800 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.626+0000 7f0ba2800640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 0x7f0ba41f48e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0b94001be0 tx=0x7f0b940067a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.627+0000 7f0b9aa00640 1 -- 172.21.15.71:0/1845138837 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94015070 con 0x7f0b88004800 2024-09-16T07:34:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.627+0000 7f0b9aa00640 1 -- 172.21.15.71:0/1845138837 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b9400fd40 con 0x7f0b88004800 2024-09-16T07:34:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.627+0000 7f0b9aa00640 1 -- 172.21.15.71:0/1845138837 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94011690 con 0x7f0b88004800 2024-09-16T07:34:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.627+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba41f52a0 con 0x7f0b88004800 2024-09-16T07:34:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.627+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba40b2de0 con 0x7f0b88004800 2024-09-16T07:34:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.628+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba40d0ae0 con 0x7f0b88004800 2024-09-16T07:34:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.629+0000 7f0b9aa00640 1 -- 172.21.15.71:0/1845138837 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f0b94011830 con 0x7f0b88004800 2024-09-16T07:34:35.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.630+0000 7f0b9aa00640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b8407a230 0x7f0b8407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:35.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.630+0000 7f0b9aa00640 1 -- 172.21.15.71:0/1845138837 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5909+0+0 (secure 0 0 0) 0x7f0b9409a680 con 0x7f0b88004800 2024-09-16T07:34:35.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.630+0000 7f0ba1e00640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b8407a230 0x7f0b8407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:35.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.631+0000 7f0ba1e00640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b8407a230 0x7f0b8407c6f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0b90035000 tx=0x7f0b9002d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.634+0000 7f0b9aa00640 1 -- 172.21.15.71:0/1845138837 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0b940647f0 con 0x7f0b88004800 2024-09-16T07:34:35.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:35.857+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0ba4073440 con 0x7f0b88004800 2024-09-16T07:34:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.438+0000 7f0b9aa00640 1 -- 172.21.15.71:0/1845138837 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7f0b940640f0 con 0x7f0b88004800 2024-09-16T07:34:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b8407a230 msgr2=0x7f0b8407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b8407a230 0x7f0b8407c6f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0b90035000 tx=0x7f0b9002d040 comp rx=0 tx=0).stop 2024-09-16T07:34:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 msgr2=0x7f0ba41f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 0x7f0ba41f48e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0b94001be0 tx=0x7f0b940067a0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 shutdown_connections 2024-09-16T07:34:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0b8407a230 0x7f0b8407c6f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba40b3840 0x7f0ba4140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba4142890 0x7f0ba40b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 --2- 172.21.15.71:0/1845138837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88004800 0x7f0ba41f48e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 >> 172.21.15.71:0/1845138837 conn(0x7f0ba40b1840 msgr2=0x7f0ba4142fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 shutdown_connections 2024-09-16T07:34:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.442+0000 7f0ba9400640 1 -- 172.21.15.71:0/1845138837 wait complete. 2024-09-16T07:34:36.459 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:34:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.624+0000 7f2a95200640 1 -- 172.21.15.71:0/2308468391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a9015f9a0 msgr2=0x7f2a9015fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.624+0000 7f2a95200640 1 --2- 172.21.15.71:0/2308468391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a9015f9a0 0x7f2a9015fd80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005fd0 tx=0x7f2a8002a540 comp rx=0 tx=0).stop 2024-09-16T07:34:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.624+0000 7f2a95200640 1 -- 172.21.15.71:0/2308468391 shutdown_connections 2024-09-16T07:34:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.624+0000 7f2a95200640 1 --2- 172.21.15.71:0/2308468391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a9006e6f0 0x7f2a9006eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.624+0000 7f2a95200640 1 --2- 172.21.15.71:0/2308468391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a9015f9a0 0x7f2a9015fd80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.624+0000 7f2a95200640 1 -- 172.21.15.71:0/2308468391 >> 172.21.15.71:0/2308468391 conn(0x7f2a900b18e0 msgr2=0x7f2a900b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.624+0000 7f2a95200640 1 -- 172.21.15.71:0/2308468391 shutdown_connections 2024-09-16T07:34:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.624+0000 7f2a95200640 1 -- 172.21.15.71:0/2308468391 wait complete. 2024-09-16T07:34:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.625+0000 7f2a95200640 1 Processor -- start 2024-09-16T07:34:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.626+0000 7f2a95200640 1 -- start start 2024-09-16T07:34:36.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.626+0000 7f2a95200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a9006e6f0 0x7f2a901559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:36.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.626+0000 7f2a95200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a90073de0 0x7f2a900d0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:36.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a95200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a9015f9a0 0x7f2a900d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a95200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9006f0b0 con 0x7f2a9015f9a0 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a95200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a900cd5e0 con 0x7f2a90073de0 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a95200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9006f230 con 0x7f2a9006e6f0 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a9015f9a0 0x7f2a900d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a9015f9a0 0x7f2a900d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60744/0 (socket says 172.21.15.71:60744) 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8e800640 1 -- 172.21.15.71:0/1723466708 learned_addr learned my addr 172.21.15.71:0/1723466708 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8de00640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a9006e6f0 0x7f2a901559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8d400640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a90073de0 0x7f2a900d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8e800640 1 -- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a9006e6f0 msgr2=0x7f2a901559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8e800640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a9006e6f0 0x7f2a901559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8e800640 1 -- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a90073de0 msgr2=0x7f2a900d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8e800640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a90073de0 0x7f2a900d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8e800640 1 -- 172.21.15.71:0/1723466708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a90153600 con 0x7f2a9015f9a0 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.627+0000 7f2a8de00640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a9006e6f0 0x7f2a901559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.628+0000 7f2a8e800640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a9015f9a0 0x7f2a900d0560 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009cf0 tx=0x7f2a8800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.628+0000 7f2a86000640 1 -- 172.21.15.71:0/1723466708 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88003b70 con 0x7f2a9015f9a0 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.628+0000 7f2a86000640 1 -- 172.21.15.71:0/1723466708 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a88003e00 con 0x7f2a9015f9a0 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.628+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a901538e0 con 0x7f2a9015f9a0 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.628+0000 7f2a86000640 1 -- 172.21.15.71:0/1723466708 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88004280 con 0x7f2a9015f9a0 2024-09-16T07:34:36.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.628+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a90153ea0 con 0x7f2a9015f9a0 2024-09-16T07:34:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.629+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a90069360 con 0x7f2a9015f9a0 2024-09-16T07:34:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.630+0000 7f2a86000640 1 -- 172.21.15.71:0/1723466708 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f2a88004420 con 0x7f2a9015f9a0 2024-09-16T07:34:36.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.631+0000 7f2a86000640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a7007a230 0x7f2a7007c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.631+0000 7f2a86000640 1 -- 172.21.15.71:0/1723466708 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5909+0+0 (secure 0 0 0) 0x7f2a8809a7a0 con 0x7f2a9015f9a0 2024-09-16T07:34:36.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.632+0000 7f2a8de00640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a7007a230 0x7f2a7007c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:36.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.632+0000 7f2a8de00640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a7007a230 0x7f2a7007c6f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2a80033000 tx=0x7f2a80004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:36.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.635+0000 7f2a86000640 1 -- 172.21.15.71:0/1723466708 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2a88064870 con 0x7f2a9015f9a0 2024-09-16T07:34:36.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:36.819+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2a9006eb70 con 0x7f2a9015f9a0 2024-09-16T07:34:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.695+0000 7f2a86000640 1 -- 172.21.15.71:0/1723466708 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7f2a88064170 con 0x7f2a9015f9a0 2024-09-16T07:34:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:34:37.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.699+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a7007a230 msgr2=0x7f2a7007c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:37.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.699+0000 7f2a95200640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a7007a230 0x7f2a7007c6f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2a80033000 tx=0x7f2a80004730 comp rx=0 tx=0).stop 2024-09-16T07:34:37.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.699+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a9015f9a0 msgr2=0x7f2a900d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:37.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.699+0000 7f2a95200640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a9015f9a0 0x7f2a900d0560 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009cf0 tx=0x7f2a8800c6a0 comp rx=0 tx=0).stop 2024-09-16T07:34:37.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.699+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 shutdown_connections 2024-09-16T07:34:37.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.699+0000 7f2a95200640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2a7007a230 0x7f2a7007c6f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:37.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.700+0000 7f2a95200640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a9015f9a0 0x7f2a900d0560 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:37.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.700+0000 7f2a95200640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a90073de0 0x7f2a900d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:37.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.700+0000 7f2a95200640 1 --2- 172.21.15.71:0/1723466708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a9006e6f0 0x7f2a901559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:37.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.700+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 >> 172.21.15.71:0/1723466708 conn(0x7f2a900b18e0 msgr2=0x7f2a900caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:37.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.700+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 shutdown_connections 2024-09-16T07:34:37.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.700+0000 7f2a95200640 1 -- 172.21.15.71:0/1723466708 wait complete. 2024-09-16T07:34:37.719 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:34:37.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.883+0000 7f6328e00640 1 -- 172.21.15.71:0/3487077854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632406e910 msgr2=0x7f63240cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:37.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.883+0000 7f6328e00640 1 --2- 172.21.15.71:0/3487077854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632406e910 0x7f63240cdad0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f6310009830 tx=0x7f631002a510 comp rx=0 tx=0).stop 2024-09-16T07:34:37.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.883+0000 7f6328e00640 1 -- 172.21.15.71:0/3487077854 shutdown_connections 2024-09-16T07:34:37.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.883+0000 7f6328e00640 1 --2- 172.21.15.71:0/3487077854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f632406e910 0x7f63240cdad0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:37.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.883+0000 7f6328e00640 1 -- 172.21.15.71:0/3487077854 >> 172.21.15.71:0/3487077854 conn(0x7f63240b18e0 msgr2=0x7f63240b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:37.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.883+0000 7f6328e00640 1 -- 172.21.15.71:0/3487077854 shutdown_connections 2024-09-16T07:34:37.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.883+0000 7f6328e00640 1 -- 172.21.15.71:0/3487077854 wait complete. 2024-09-16T07:34:37.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.885+0000 7f6328e00640 1 Processor -- start 2024-09-16T07:34:37.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.885+0000 7f6328e00640 1 -- start start 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6328e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632406e910 0x7f632414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6328e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6324074460 0x7f632409ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6328e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632415bd60 0x7f63241f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6328e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f63240cd050 con 0x7f6324074460 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6328e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f63240cced0 con 0x7f632406e910 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6328e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63240cd1d0 con 0x7f632415bd60 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6321400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6324074460 0x7f632409ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6321400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6324074460 0x7f632409ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60748/0 (socket says 172.21.15.71:60748) 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.886+0000 7f6321400640 1 -- 172.21.15.71:0/4036462654 learned_addr learned my addr 172.21.15.71:0/4036462654 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:37.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f6321e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632406e910 0x7f632414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:37.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f6322800640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632415bd60 0x7f63241f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:37.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f6321400640 1 -- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632415bd60 msgr2=0x7f63241f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f6321400640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632415bd60 0x7f63241f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f6321400640 1 -- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632406e910 msgr2=0x7f632414fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f6321400640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632406e910 0x7f632414fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f6321400640 1 -- 172.21.15.71:0/4036462654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632409d6e0 con 0x7f6324074460 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f6321400640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6324074460 0x7f632409ad30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6310000c00 tx=0x7f6310032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f631a000640 1 -- 172.21.15.71:0/4036462654 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310040070 con 0x7f6324074460 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.887+0000 7f631a000640 1 -- 172.21.15.71:0/4036462654 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6310003af0 con 0x7f6324074460 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.888+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632409b6a0 con 0x7f6324074460 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.888+0000 7f631a000640 1 -- 172.21.15.71:0/4036462654 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310045460 con 0x7f6324074460 2024-09-16T07:34:37.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.888+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63241f93f0 con 0x7f6324074460 2024-09-16T07:34:37.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.889+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6324069360 con 0x7f6324074460 2024-09-16T07:34:37.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.890+0000 7f631a000640 1 -- 172.21.15.71:0/4036462654 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f6310039040 con 0x7f6324074460 2024-09-16T07:34:37.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.891+0000 7f631a000640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630407a230 0x7f630407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:37.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.891+0000 7f631a000640 1 -- 172.21.15.71:0/4036462654 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5413+0+0 (secure 0 0 0) 0x7f63100bac80 con 0x7f6324074460 2024-09-16T07:34:37.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.892+0000 7f6321e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630407a230 0x7f630407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:37.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.892+0000 7f6321e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630407a230 0x7f630407c6f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6314002200 tx=0x7f63140019f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:37.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:37.895+0000 7f631a000640 1 -- 172.21.15.71:0/4036462654 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6310084700 con 0x7f6324074460 2024-09-16T07:34:38.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.079+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63240cdad0 con 0x7f6324074460 2024-09-16T07:34:38.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.701+0000 7f631a000640 1 -- 172.21.15.71:0/4036462654 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7f6310089130 con 0x7f6324074460 2024-09-16T07:34:38.701 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:34:38.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.705+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630407a230 msgr2=0x7f630407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:38.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.705+0000 7f6328e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630407a230 0x7f630407c6f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6314002200 tx=0x7f63140019f0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.705+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6324074460 msgr2=0x7f632409ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:38.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.705+0000 7f6328e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6324074460 0x7f632409ad30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f6310000c00 tx=0x7f6310032d50 comp rx=0 tx=0).stop 2024-09-16T07:34:38.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.706+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 shutdown_connections 2024-09-16T07:34:38.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.706+0000 7f6328e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f630407a230 0x7f630407c6f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.706+0000 7f6328e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f632415bd60 0x7f63241f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.706+0000 7f6328e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6324074460 0x7f632409ad30 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.706+0000 7f6328e00640 1 --2- 172.21.15.71:0/4036462654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f632406e910 0x7f632414fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.706+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 >> 172.21.15.71:0/4036462654 conn(0x7f63240b18e0 msgr2=0x7f63240b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:38.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.706+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 shutdown_connections 2024-09-16T07:34:38.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.706+0000 7f6328e00640 1 -- 172.21.15.71:0/4036462654 wait complete. 2024-09-16T07:34:38.725 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:34:38.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.883+0000 7f721e200640 1 -- 172.21.15.71:0/2234613951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181612d0 msgr2=0x7f72180b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:38.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.883+0000 7f721e200640 1 --2- 172.21.15.71:0/2234613951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181612d0 0x7f72180b4e00 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7208005fd0 tx=0x7f720802a8b0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.884+0000 7f721e200640 1 -- 172.21.15.71:0/2234613951 shutdown_connections 2024-09-16T07:34:38.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.884+0000 7f721e200640 1 --2- 172.21.15.71:0/2234613951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181612d0 0x7f72180b4e00 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.884+0000 7f721e200640 1 -- 172.21.15.71:0/2234613951 >> 172.21.15.71:0/2234613951 conn(0x7f72180d3df0 msgr2=0x7f72180d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:38.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.884+0000 7f721e200640 1 -- 172.21.15.71:0/2234613951 shutdown_connections 2024-09-16T07:34:38.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.884+0000 7f721e200640 1 -- 172.21.15.71:0/2234613951 wait complete. 2024-09-16T07:34:38.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.885+0000 7f721e200640 1 Processor -- start 2024-09-16T07:34:38.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.885+0000 7f721e200640 1 -- start start 2024-09-16T07:34:38.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.886+0000 7f721e200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72180d4980 0x7f721815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:38.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.886+0000 7f721e200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72181612d0 0x7f72181502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:38.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.886+0000 7f721e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181616b0 0x7f72181f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:38.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.886+0000 7f721e200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f72180b1e60 con 0x7f72181616b0 2024-09-16T07:34:38.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.886+0000 7f721e200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f72180b1ce0 con 0x7f72181612d0 2024-09-16T07:34:38.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.886+0000 7f721e200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f72180b1fe0 con 0x7f72180d4980 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181616b0 0x7f72181f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181616b0 0x7f72181f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60770/0 (socket says 172.21.15.71:60770) 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 -- 172.21.15.71:0/1918216713 learned_addr learned my addr 172.21.15.71:0/1918216713 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f7217e00640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72181612d0 0x7f72181502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721ce00640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72180d4980 0x7f721815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 -- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72180d4980 msgr2=0x7f721815aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72180d4980 0x7f721815aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 -- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72181612d0 msgr2=0x7f72181502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72181612d0 0x7f72181502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 -- 172.21.15.71:0/1918216713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721815b170 con 0x7f72181616b0 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.887+0000 7f721d800640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181616b0 0x7f72181f4e50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f721000c3c0 tx=0x7f721000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:38.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.888+0000 7f7214c00640 1 -- 172.21.15.71:0/1918216713 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210017070 con 0x7f72181616b0 2024-09-16T07:34:38.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.888+0000 7f7214c00640 1 -- 172.21.15.71:0/1918216713 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72100035f0 con 0x7f72181616b0 2024-09-16T07:34:38.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.888+0000 7f7214c00640 1 -- 172.21.15.71:0/1918216713 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210003a40 con 0x7f72181616b0 2024-09-16T07:34:38.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.888+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721815b450 con 0x7f72181616b0 2024-09-16T07:34:38.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.888+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7218150800 con 0x7f72181616b0 2024-09-16T07:34:38.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.889+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72180b25f0 con 0x7f72181616b0 2024-09-16T07:34:38.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.890+0000 7f7214c00640 1 -- 172.21.15.71:0/1918216713 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f721001a070 con 0x7f72181616b0 2024-09-16T07:34:38.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.891+0000 7f7214c00640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f71f807a230 0x7f71f807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:38.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.891+0000 7f7214c00640 1 -- 172.21.15.71:0/1918216713 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4999+0+0 (secure 0 0 0) 0x7f7210098870 con 0x7f72181616b0 2024-09-16T07:34:38.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.891+0000 7f721ce00640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f71f807a230 0x7f71f807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:38.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.892+0000 7f721ce00640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f71f807a230 0x7f71f807c6f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7208034000 tx=0x7f7208004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:38.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:38.895+0000 7f7214c00640 1 -- 172.21.15.71:0/1918216713 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f72180d2cc0 con 0x7f72181616b0 2024-09-16T07:34:39.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.081+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f72180d0f10 con 0x7f72181616b0 2024-09-16T07:34:39.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.085+0000 7f7214c00640 1 -- 172.21.15.71:0/1918216713 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7f7210062390 con 0x7f72181616b0 2024-09-16T07:34:39.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f71f807a230 msgr2=0x7f71f807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f71f807a230 0x7f71f807c6f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7208034000 tx=0x7f7208004ab0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181616b0 msgr2=0x7f72181f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181616b0 0x7f72181f4e50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f721000c3c0 tx=0x7f721000c880 comp rx=0 tx=0).stop 2024-09-16T07:34:39.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 shutdown_connections 2024-09-16T07:34:39.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f71f807a230 0x7f71f807c6f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72181616b0 0x7f72181f4e50 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72181612d0 0x7f72181502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 --2- 172.21.15.71:0/1918216713 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72180d4980 0x7f721815aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.087+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 >> 172.21.15.71:0/1918216713 conn(0x7f72180d3df0 msgr2=0x7f72180ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:39.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.088+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 shutdown_connections 2024-09-16T07:34:39.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.088+0000 7f721e200640 1 -- 172.21.15.71:0/1918216713 wait complete. 2024-09-16T07:34:39.104 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-16T07:34:39.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.267+0000 7f0bed000640 1 -- 172.21.15.71:0/695779188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc003a50 msgr2=0x7f0bcc003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.267+0000 7f0bed000640 1 --2- 172.21.15.71:0/695779188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc003a50 0x7f0bcc003e90 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80016c0 tx=0x7f0bcc005830 comp rx=0 tx=0).stop 2024-09-16T07:34:39.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.267+0000 7f0bed000640 1 -- 172.21.15.71:0/695779188 shutdown_connections 2024-09-16T07:34:39.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.267+0000 7f0bed000640 1 --2- 172.21.15.71:0/695779188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc003a50 0x7f0bcc003e90 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.267+0000 7f0bed000640 1 -- 172.21.15.71:0/695779188 >> 172.21.15.71:0/695779188 conn(0x7f0be80d3df0 msgr2=0x7f0be80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:39.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.267+0000 7f0bed000640 1 -- 172.21.15.71:0/695779188 shutdown_connections 2024-09-16T07:34:39.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.267+0000 7f0bed000640 1 -- 172.21.15.71:0/695779188 wait complete. 2024-09-16T07:34:39.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.268+0000 7f0bed000640 1 Processor -- start 2024-09-16T07:34:39.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.268+0000 7f0bed000640 1 -- start start 2024-09-16T07:34:39.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.269+0000 7f0bed000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc003a50 0x7f0be8155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:39.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.269+0000 7f0bed000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be815ff20 0x7f0be81603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:39.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.269+0000 7f0bed000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be815cec0 0x7f0be815d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:39.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.269+0000 7f0bed000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0be80b1b00 con 0x7f0be815cec0 2024-09-16T07:34:39.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.269+0000 7f0bed000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0be80b1980 con 0x7f0bcc003a50 2024-09-16T07:34:39.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.269+0000 7f0bed000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0be80b1c80 con 0x7f0be815ff20 2024-09-16T07:34:39.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be815cec0 0x7f0be815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:39.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be815cec0 0x7f0be815d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60774/0 (socket says 172.21.15.71:60774) 2024-09-16T07:34:39.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 -- 172.21.15.71:0/2702656782 learned_addr learned my addr 172.21.15.71:0/2702656782 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:39.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 -- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be815ff20 msgr2=0x7f0be81603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be815ff20 0x7f0be81603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 -- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc003a50 msgr2=0x7f0be8155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc003a50 0x7f0be8155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 -- 172.21.15.71:0/2702656782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be8160c80 con 0x7f0be815cec0 2024-09-16T07:34:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0be7e00640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be815cec0 0x7f0be815d370 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0be80c7470 tx=0x7f0be0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0bdf400640 1 -- 172.21.15.71:0/2702656782 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0003970 con 0x7f0be815cec0 2024-09-16T07:34:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0bdf400640 1 -- 172.21.15.71:0/2702656782 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0be0015070 con 0x7f0be815cec0 2024-09-16T07:34:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be81609a0 con 0x7f0be815cec0 2024-09-16T07:34:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.270+0000 7f0bdf400640 1 -- 172.21.15.71:0/2702656782 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0004030 con 0x7f0be815cec0 2024-09-16T07:34:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.271+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be80b81b0 con 0x7f0be815cec0 2024-09-16T07:34:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.272+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be80b2150 con 0x7f0be815cec0 2024-09-16T07:34:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.273+0000 7f0bdf400640 1 -- 172.21.15.71:0/2702656782 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f0be00041d0 con 0x7f0be815cec0 2024-09-16T07:34:39.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.274+0000 7f0bdf400640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc807a230 0x7f0bc807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:39.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.274+0000 7f0bdf400640 1 -- 172.21.15.71:0/2702656782 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4999+0+0 (secure 0 0 0) 0x7f0be009a260 con 0x7f0be815cec0 2024-09-16T07:34:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.275+0000 7f0be7400640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc807a230 0x7f0bc807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.276+0000 7f0be7400640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc807a230 0x7f0bc807c6f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8032000 tx=0x7f0bd8001120 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.278+0000 7f0bdf400640 1 -- 172.21.15.71:0/2702656782 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0be8160c80 con 0x7f0be815cec0 2024-09-16T07:34:39.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.531+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f0be8155810 con 0x7f0be815cec0 2024-09-16T07:34:39.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.703+0000 7f0bdf400640 1 -- 172.21.15.71:0/2702656782 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v552) ==== 178+0+0 (secure 0 0 0) 0x7f0be0064330 con 0x7f0be815cec0 2024-09-16T07:34:39.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc807a230 msgr2=0x7f0bc807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc807a230 0x7f0bc807c6f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8032000 tx=0x7f0bd8001120 comp rx=0 tx=0).stop 2024-09-16T07:34:39.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be815cec0 msgr2=0x7f0be815d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be815cec0 0x7f0be815d370 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0be80c7470 tx=0x7f0be0010f40 comp rx=0 tx=0).stop 2024-09-16T07:34:39.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 shutdown_connections 2024-09-16T07:34:39.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0bc807a230 0x7f0bc807c6f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be815cec0 0x7f0be815d370 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be815ff20 0x7f0be81603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 --2- 172.21.15.71:0/2702656782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bcc003a50 0x7f0be8155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.705+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 >> 172.21.15.71:0/2702656782 conn(0x7f0be80d3df0 msgr2=0x7f0be80d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:39.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.706+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 shutdown_connections 2024-09-16T07:34:39.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.706+0000 7f0bed000640 1 -- 172.21.15.71:0/2702656782 wait complete. 2024-09-16T07:34:39.721 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-09-16T07:34:39.721 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-09-16T07:34:39.721 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:34:39.722 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:34:39.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.877+0000 7f9107e00640 1 -- 172.21.15.71:0/3252797372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080ad880 msgr2=0x7f91080adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.877+0000 7f9107e00640 1 --2- 172.21.15.71:0/3252797372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080ad880 0x7f91080adcd0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f91080cd570 tx=0x7f90f002a560 comp rx=0 tx=0).stop 2024-09-16T07:34:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.878+0000 7f9107e00640 1 -- 172.21.15.71:0/3252797372 shutdown_connections 2024-09-16T07:34:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.878+0000 7f9107e00640 1 --2- 172.21.15.71:0/3252797372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91080cd5f0 0x7f91080cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.878+0000 7f9107e00640 1 --2- 172.21.15.71:0/3252797372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080ad880 0x7f91080adcd0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.878+0000 7f9107e00640 1 --2- 172.21.15.71:0/3252797372 >> v2:172.21.15.177:3300/0 conn(0x7f910806fb90 0x7f910806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.878+0000 7f9107e00640 1 -- 172.21.15.71:0/3252797372 >> 172.21.15.71:0/3252797372 conn(0x7f91080cfa20 msgr2=0x7f91080cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.878+0000 7f9107e00640 1 -- 172.21.15.71:0/3252797372 shutdown_connections 2024-09-16T07:34:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.878+0000 7f9107e00640 1 -- 172.21.15.71:0/3252797372 wait complete. 2024-09-16T07:34:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.879+0000 7f9107e00640 1 Processor -- start 2024-09-16T07:34:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.879+0000 7f9107e00640 1 -- start start 2024-09-16T07:34:39.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.880+0000 7f9107e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910806fb90 0x7f91080b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.880+0000 7f9107e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91080ad880 0x7f9108159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.880+0000 7f9107e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080cd5f0 0x7f91081f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.880+0000 7f9107e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f910815ebf0 con 0x7f91080cd5f0 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.880+0000 7f9107e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f910815ea70 con 0x7f910806fb90 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.880+0000 7f9107e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f910815ed70 con 0x7f91080ad880 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9107400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080cd5f0 0x7f91081f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9107400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080cd5f0 0x7f91081f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60794/0 (socket says 172.21.15.71:60794) 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9107400640 1 -- 172.21.15.71:0/82466355 learned_addr learned my addr 172.21.15.71:0/82466355 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f90fe000640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91080ad880 0x7f9108159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9106a00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910806fb90 0x7f91080b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9107400640 1 -- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91080ad880 msgr2=0x7f9108159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9107400640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91080ad880 0x7f9108159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9107400640 1 -- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910806fb90 msgr2=0x7f91080b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9107400640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910806fb90 0x7f91080b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.881+0000 7f9107400640 1 -- 172.21.15.71:0/82466355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91080c35f0 con 0x7f91080cd5f0 2024-09-16T07:34:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.882+0000 7f9107400640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080cd5f0 0x7f91081f73c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f910000b9a0 tx=0x7f910000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.882+0000 7f90ffe00640 1 -- 172.21.15.71:0/82466355 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100019070 con 0x7f91080cd5f0 2024-09-16T07:34:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.882+0000 7f90ffe00640 1 -- 172.21.15.71:0/82466355 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f910000cbb0 con 0x7f91080cd5f0 2024-09-16T07:34:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.882+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91081598a0 con 0x7f91080cd5f0 2024-09-16T07:34:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.882+0000 7f90ffe00640 1 -- 172.21.15.71:0/82466355 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100014440 con 0x7f91080cd5f0 2024-09-16T07:34:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.882+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91080c3b40 con 0x7f91080cd5f0 2024-09-16T07:34:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.883+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910815d2c0 con 0x7f91080cd5f0 2024-09-16T07:34:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.884+0000 7f90ffe00640 1 -- 172.21.15.71:0/82466355 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f910001c070 con 0x7f91080cd5f0 2024-09-16T07:34:39.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.885+0000 7f90ffe00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e407a230 0x7f90e407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:39.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.885+0000 7f90ffe00640 1 -- 172.21.15.71:0/82466355 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4999+0+0 (secure 0 0 0) 0x7f91000993e0 con 0x7f91080cd5f0 2024-09-16T07:34:39.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.885+0000 7f9106a00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e407a230 0x7f90e407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:39.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.886+0000 7f9106a00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e407a230 0x7f90e407c6f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9108158590 tx=0x7f90f800a2d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:39.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:39.889+0000 7f90ffe00640 1 -- 172.21.15.71:0/82466355 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f910815d2c0 con 0x7f91080cd5f0 2024-09-16T07:34:40.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.077+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91080008d0 con 0x7f91080cd5f0 2024-09-16T07:34:40.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.078+0000 7f90ffe00640 1 -- 172.21.15.71:0/82466355 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+13142 (secure 0 0 0) 0x7f9100062f00 con 0x7f91080cd5f0 2024-09-16T07:34:40.079 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:40.079 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":110,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:34:38.696788+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6833/4054989006":"2024-09-17T07:34:35.435486+0000","172.21.15.71:6832/4054989006":"2024-09-17T07:34:35.435486+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:34:40.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e407a230 msgr2=0x7f90e407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e407a230 0x7f90e407c6f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9108158590 tx=0x7f90f800a2d0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080cd5f0 msgr2=0x7f91081f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080cd5f0 0x7f91081f73c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f910000b9a0 tx=0x7f910000bda0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 shutdown_connections 2024-09-16T07:34:40.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90e407a230 0x7f90e407c6f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91080cd5f0 0x7f91081f73c0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91080ad880 0x7f9108159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 --2- 172.21.15.71:0/82466355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910806fb90 0x7f91080b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 >> 172.21.15.71:0/82466355 conn(0x7f91080cfa20 msgr2=0x7f91080729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.080+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 shutdown_connections 2024-09-16T07:34:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.081+0000 7f9107e00640 1 -- 172.21.15.71:0/82466355 wait complete. 2024-09-16T07:34:40.096 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:34:40.097 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:34:40.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.261+0000 7f3385800640 1 -- 172.21.15.71:0/3958140359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 msgr2=0x7f3364004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.261+0000 7f3385800640 1 --2- 172.21.15.71:0/3958140359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 0x7f3364004c80 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f336800c310 tx=0x7f3364005830 comp rx=0 tx=0).stop 2024-09-16T07:34:40.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.261+0000 7f337f400640 1 -- 172.21.15.71:0/3958140359 reap_dead start 2024-09-16T07:34:40.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.261+0000 7f3385800640 1 -- 172.21.15.71:0/3958140359 shutdown_connections 2024-09-16T07:34:40.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.261+0000 7f3385800640 1 -- 172.21.15.71:0/3958140359 >> 172.21.15.71:0/3958140359 conn(0x7f33800b2130 msgr2=0x7f33800b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:40.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.261+0000 7f3385800640 1 -- 172.21.15.71:0/3958140359 shutdown_connections 2024-09-16T07:34:40.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.262+0000 7f3385800640 1 -- 172.21.15.71:0/3958140359 wait complete. 2024-09-16T07:34:40.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.263+0000 7f3385800640 1 Processor -- start 2024-09-16T07:34:40.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.263+0000 7f3385800640 1 -- start start 2024-09-16T07:34:40.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.264+0000 7f3385800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 0x7f338014d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:40.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.264+0000 7f3385800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f338014a190 0x7f338015d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:40.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.264+0000 7f3385800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33801f8650 0x7f338013db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:40.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.264+0000 7f3385800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f338006f960 con 0x7f3364004800 2024-09-16T07:34:40.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.264+0000 7f3385800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f338006fae0 con 0x7f33801f8650 2024-09-16T07:34:40.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.264+0000 7f3385800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f338006fc60 con 0x7f338014a190 2024-09-16T07:34:40.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 0x7f338014d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 0x7f338014d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60802/0 (socket says 172.21.15.71:60802) 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 -- 172.21.15.71:0/4174065998 learned_addr learned my addr 172.21.15.71:0/4174065998 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337e000640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f338014a190 0x7f338015d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337f400640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33801f8650 0x7f338013db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 -- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f338014a190 msgr2=0x7f338015d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f338014a190 0x7f338015d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 -- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33801f8650 msgr2=0x7f338013db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33801f8650 0x7f338013db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 -- 172.21.15.71:0/4174065998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33801f7e90 con 0x7f3364004800 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337f400640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33801f8650 0x7f338013db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.265+0000 7f337ea00640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 0x7f338014d700 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3370010c60 tx=0x7f337000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.266+0000 7f3376a00640 1 -- 172.21.15.71:0/4174065998 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370003b70 con 0x7f3364004800 2024-09-16T07:34:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.266+0000 7f3376a00640 1 -- 172.21.15.71:0/4174065998 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3370003e20 con 0x7f3364004800 2024-09-16T07:34:40.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.266+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338014bba0 con 0x7f3364004800 2024-09-16T07:34:40.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.266+0000 7f3376a00640 1 -- 172.21.15.71:0/4174065998 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370004290 con 0x7f3364004800 2024-09-16T07:34:40.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.266+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338014c160 con 0x7f3364004800 2024-09-16T07:34:40.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.267+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3380070180 con 0x7f3364004800 2024-09-16T07:34:40.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.268+0000 7f3376a00640 1 -- 172.21.15.71:0/4174065998 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f3370004430 con 0x7f3364004800 2024-09-16T07:34:40.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.269+0000 7f3376a00640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335c07a260 0x7f335c07c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:40.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.269+0000 7f3376a00640 1 -- 172.21.15.71:0/4174065998 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4999+0+0 (secure 0 0 0) 0x7f337009a8b0 con 0x7f3364004800 2024-09-16T07:34:40.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.269+0000 7f337e000640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335c07a260 0x7f335c07c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:40.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.270+0000 7f337e000640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335c07a260 0x7f335c07c720 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3368033000 tx=0x7f3368038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:40.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.273+0000 7f3376a00640 1 -- 172.21.15.71:0/4174065998 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3370064980 con 0x7f3364004800 2024-09-16T07:34:40.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.455+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f338015bfb0 con 0x7f3364004800 2024-09-16T07:34:40.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.455+0000 7f3376a00640 1 -- 172.21.15.71:0/4174065998 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f3370064280 con 0x7f3364004800 2024-09-16T07:34:40.456 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335c07a260 msgr2=0x7f335c07c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335c07a260 0x7f335c07c720 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3368033000 tx=0x7f3368038040 comp rx=0 tx=0).stop 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 msgr2=0x7f338014d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 0x7f338014d700 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3370010c60 tx=0x7f337000e720 comp rx=0 tx=0).stop 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 shutdown_connections 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335c07a260 0x7f335c07c720 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33801f8650 0x7f338013db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f338014a190 0x7f338015d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 --2- 172.21.15.71:0/4174065998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3364004800 0x7f338014d700 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 >> 172.21.15.71:0/4174065998 conn(0x7f33800b2130 msgr2=0x7f338015afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:40.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 shutdown_connections 2024-09-16T07:34:40.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.458+0000 7f3385800640 1 -- 172.21.15.71:0/4174065998 wait complete. 2024-09-16T07:34:40.475 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-16T07:34:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.632+0000 7f1182e00640 1 -- 172.21.15.71:0/1190735864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 msgr2=0x7f1160004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.632+0000 7f1182e00640 1 --2- 172.21.15.71:0/1190735864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 0x7f1160004c80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f116400c310 tx=0x7f1160005830 comp rx=0 tx=0).stop 2024-09-16T07:34:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.632+0000 7f1182e00640 1 -- 172.21.15.71:0/1190735864 shutdown_connections 2024-09-16T07:34:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.632+0000 7f1182e00640 1 --2- 172.21.15.71:0/1190735864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 0x7f1160004c80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.632+0000 7f1182e00640 1 -- 172.21.15.71:0/1190735864 >> 172.21.15.71:0/1190735864 conn(0x7f117c0b1950 msgr2=0x7f117c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.632+0000 7f1182e00640 1 -- 172.21.15.71:0/1190735864 shutdown_connections 2024-09-16T07:34:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.632+0000 7f1182e00640 1 -- 172.21.15.71:0/1190735864 wait complete. 2024-09-16T07:34:40.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.634+0000 7f1182e00640 1 Processor -- start 2024-09-16T07:34:40.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.634+0000 7f1182e00640 1 -- start start 2024-09-16T07:34:40.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f1182e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 0x7f117c1363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f1182e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c142cd0 0x7f117c09d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f1182e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c06e8f0 0x7f117c06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f1182e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f117c0cebf0 con 0x7f1160004800 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f1182e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f117c0b4eb0 con 0x7f117c142cd0 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f1182e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f117c0ced70 con 0x7f117c06e8f0 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 0x7f117c1363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 0x7f117c1363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60814/0 (socket says 172.21.15.71:60814) 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 -- 172.21.15.71:0/3619235733 learned_addr learned my addr 172.21.15.71:0/3619235733 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 -- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c06e8f0 msgr2=0x7f117c06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c06e8f0 0x7f117c06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 -- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c142cd0 msgr2=0x7f117c09d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:40.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c142cd0 0x7f117c09d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:40.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 -- 172.21.15.71:0/3619235733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c138dd0 con 0x7f1160004800 2024-09-16T07:34:40.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.635+0000 7f117be00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 0x7f117c1363d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f116c006060 tx=0x7f116c006520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:40.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.636+0000 7f1173e00640 1 -- 172.21.15.71:0/3619235733 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c00f7b0 con 0x7f1160004800 2024-09-16T07:34:40.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.636+0000 7f1173e00640 1 -- 172.21.15.71:0/3619235733 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f116c016020 con 0x7f1160004800 2024-09-16T07:34:40.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.636+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c1390b0 con 0x7f1160004800 2024-09-16T07:34:40.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.636+0000 7f1173e00640 1 -- 172.21.15.71:0/3619235733 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c00fbf0 con 0x7f1160004800 2024-09-16T07:34:40.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.636+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c139670 con 0x7f1160004800 2024-09-16T07:34:40.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.637+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117c15e970 con 0x7f1160004800 2024-09-16T07:34:40.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.638+0000 7f1173e00640 1 -- 172.21.15.71:0/3619235733 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f116c011430 con 0x7f1160004800 2024-09-16T07:34:40.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.639+0000 7f1173e00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f115807a260 0x7f115807c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:40.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.639+0000 7f1173e00640 1 -- 172.21.15.71:0/3619235733 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4999+0+0 (secure 0 0 0) 0x7f116c09a340 con 0x7f1160004800 2024-09-16T07:34:40.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.639+0000 7f117b400640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f115807a260 0x7f115807c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:40.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.640+0000 7f117b400640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f115807a260 0x7f115807c720 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1164033000 tx=0x7f117c0b2e20 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:40.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.643+0000 7f1173e00640 1 -- 172.21.15.71:0/3619235733 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f116c020020 con 0x7f1160004800 2024-09-16T07:34:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:40.899+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7f117c070a20 con 0x7f1160004800 2024-09-16T07:34:41.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.709+0000 7f1173e00640 1 -- 172.21.15.71:0/3619235733 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v554) ==== 188+0+0 (secure 0 0 0) 0x7f116c064410 con 0x7f1160004800 2024-09-16T07:34:41.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f115807a260 msgr2=0x7f115807c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f115807a260 0x7f115807c720 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1164033000 tx=0x7f117c0b2e20 comp rx=0 tx=0).stop 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 msgr2=0x7f117c1363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 0x7f117c1363d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f116c006060 tx=0x7f116c006520 comp rx=0 tx=0).stop 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 shutdown_connections 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f115807a260 0x7f115807c720 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c06e8f0 0x7f117c06eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c142cd0 0x7f117c09d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 --2- 172.21.15.71:0/3619235733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1160004800 0x7f117c1363d0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 >> 172.21.15.71:0/3619235733 conn(0x7f117c0b1950 msgr2=0x7f117c0aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 shutdown_connections 2024-09-16T07:34:41.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.713+0000 7f1182e00640 1 -- 172.21.15.71:0/3619235733 wait complete. 2024-09-16T07:34:41.732 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:34:41.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.892+0000 7fc8f9000640 1 -- 172.21.15.71:0/4048335526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f40ccf00 msgr2=0x7fc8f40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:41.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.892+0000 7fc8f9000640 1 --2- 172.21.15.71:0/4048335526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f40ccf00 0x7fc8f40b2440 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0005fd0 tx=0x7fc8e002a540 comp rx=0 tx=0).stop 2024-09-16T07:34:41.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.892+0000 7fc8f9000640 1 -- 172.21.15.71:0/4048335526 shutdown_connections 2024-09-16T07:34:41.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.892+0000 7fc8f9000640 1 --2- 172.21.15.71:0/4048335526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f4073fb0 0x7fc8f4074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:41.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.892+0000 7fc8f9000640 1 --2- 172.21.15.71:0/4048335526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f40ccf00 0x7fc8f40b2440 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:41.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.892+0000 7fc8f9000640 1 -- 172.21.15.71:0/4048335526 >> 172.21.15.71:0/4048335526 conn(0x7fc8f40b1950 msgr2=0x7fc8f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:41.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.892+0000 7fc8f9000640 1 -- 172.21.15.71:0/4048335526 shutdown_connections 2024-09-16T07:34:41.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.892+0000 7fc8f9000640 1 -- 172.21.15.71:0/4048335526 wait complete. 2024-09-16T07:34:41.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.894+0000 7fc8f9000640 1 Processor -- start 2024-09-16T07:34:41.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.894+0000 7fc8f9000640 1 -- start start 2024-09-16T07:34:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4073fb0 0x7fc8f414cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f9000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ccf00 0x7fc8f415da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f9000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f415f5a0 0x7fc8f415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f9000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f406f2e0 con 0x7fc8f4073fb0 2024-09-16T07:34:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f9000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f406f160 con 0x7fc8f40ccf00 2024-09-16T07:34:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f9000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f406f460 con 0x7fc8f415f5a0 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4073fb0 0x7fc8f414cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4073fb0 0x7fc8f414cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60834/0 (socket says 172.21.15.71:60834) 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1e00640 1 -- 172.21.15.71:0/3123317459 learned_addr learned my addr 172.21.15.71:0/3123317459 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f2800640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f415f5a0 0x7fc8f415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1400640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ccf00 0x7fc8f415da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1e00640 1 -- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f415f5a0 msgr2=0x7fc8f415df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1e00640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f415f5a0 0x7fc8f415df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1e00640 1 -- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ccf00 msgr2=0x7fc8f415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1e00640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ccf00 0x7fc8f415da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.895+0000 7fc8f1e00640 1 -- 172.21.15.71:0/3123317459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f40b2fa0 con 0x7fc8f4073fb0 2024-09-16T07:34:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.896+0000 7fc8f1e00640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4073fb0 0x7fc8f414cb80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c730 tx=0x7fc8e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.896+0000 7fc8ea000640 1 -- 172.21.15.71:0/3123317459 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4003b70 con 0x7fc8f4073fb0 2024-09-16T07:34:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.896+0000 7fc8ea000640 1 -- 172.21.15.71:0/3123317459 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8e4003d30 con 0x7fc8f4073fb0 2024-09-16T07:34:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.896+0000 7fc8ea000640 1 -- 172.21.15.71:0/3123317459 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400aab0 con 0x7fc8f4073fb0 2024-09-16T07:34:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.896+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f40b3280 con 0x7fc8f4073fb0 2024-09-16T07:34:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.896+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f40b3840 con 0x7fc8f4073fb0 2024-09-16T07:34:41.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.897+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f406f990 con 0x7fc8f4073fb0 2024-09-16T07:34:41.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.898+0000 7fc8ea000640 1 -- 172.21.15.71:0/3123317459 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fc8e400ac50 con 0x7fc8f4073fb0 2024-09-16T07:34:41.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.899+0000 7fc8ea000640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d407a230 0x7fc8d407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:41.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.899+0000 7fc8ea000640 1 -- 172.21.15.71:0/3123317459 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4999+0+0 (secure 0 0 0) 0x7fc8e4099560 con 0x7fc8f4073fb0 2024-09-16T07:34:41.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.900+0000 7fc8f1400640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d407a230 0x7fc8d407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:41.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.901+0000 7fc8f1400640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d407a230 0x7fc8d407c6f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0033000 tx=0x7fc8e0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:41.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:41.903+0000 7fc8ea000640 1 -- 172.21.15.71:0/3123317459 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc8e40636d0 con 0x7fc8f4073fb0 2024-09-16T07:34:42.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.087+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc8f409bca0 con 0x7fc8f4073fb0 2024-09-16T07:34:42.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.088+0000 7fc8ea000640 1 -- 172.21.15.71:0/3123317459 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7fc8e4062fd0 con 0x7fc8f4073fb0 2024-09-16T07:34:42.088 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:34:42.089 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:34:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:34:42.091 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:34:42.091 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:34:42.091 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:34:42.091 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:34:42.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d407a230 msgr2=0x7fc8d407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d407a230 0x7fc8d407c6f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0033000 tx=0x7fc8e0004730 comp rx=0 tx=0).stop 2024-09-16T07:34:42.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4073fb0 msgr2=0x7fc8f414cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4073fb0 0x7fc8f414cb80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c730 tx=0x7fc8e400cbf0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 shutdown_connections 2024-09-16T07:34:42.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc8d407a230 0x7fc8d407c6f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f415f5a0 0x7fc8f415df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ccf00 0x7fc8f415da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 --2- 172.21.15.71:0/3123317459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f4073fb0 0x7fc8f414cb80 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 >> 172.21.15.71:0/3123317459 conn(0x7fc8f40b1950 msgr2=0x7fc8f415e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:42.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 shutdown_connections 2024-09-16T07:34:42.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.090+0000 7fc8f9000640 1 -- 172.21.15.71:0/3123317459 wait complete. 2024-09-16T07:34:42.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:34:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.266+0000 7f2634a00640 1 -- 172.21.15.71:0/1776891069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300cec60 msgr2=0x7f26300cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.266+0000 7f2634a00640 1 --2- 172.21.15.71:0/1776891069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300cec60 0x7f26300cf070 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f26300cfba0 tx=0x7f262802aa10 comp rx=0 tx=0).stop 2024-09-16T07:34:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.266+0000 7f2634a00640 1 -- 172.21.15.71:0/1776891069 shutdown_connections 2024-09-16T07:34:42.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.266+0000 7f2634a00640 1 --2- 172.21.15.71:0/1776891069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300cec60 0x7f26300cf070 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.266+0000 7f2634a00640 1 --2- 172.21.15.71:0/1776891069 >> v2:172.21.15.177:3300/0 conn(0x7f2630161760 0x7f26300ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.266+0000 7f2634a00640 1 -- 172.21.15.71:0/1776891069 >> 172.21.15.71:0/1776891069 conn(0x7f26300b1840 msgr2=0x7f26300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:42.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.266+0000 7f2634a00640 1 -- 172.21.15.71:0/1776891069 shutdown_connections 2024-09-16T07:34:42.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.266+0000 7f2634a00640 1 -- 172.21.15.71:0/1776891069 wait complete. 2024-09-16T07:34:42.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.267+0000 7f2634a00640 1 Processor -- start 2024-09-16T07:34:42.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.268+0000 7f2634a00640 1 -- start start 2024-09-16T07:34:42.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.268+0000 7f2634a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300af190 0x7f26301f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:42.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.268+0000 7f2634a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26300cec60 0x7f26301f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:42.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.268+0000 7f2634a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2630161760 0x7f2630072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.268+0000 7f2634a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2630162310 con 0x7f26300af190 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.268+0000 7f2634a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2630162190 con 0x7f2630161760 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.268+0000 7f2634a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2630162490 con 0x7f26300cec60 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300af190 0x7f26301f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300af190 0x7f26301f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60860/0 (socket says 172.21.15.71:60860) 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 -- 172.21.15.71:0/648462231 learned_addr learned my addr 172.21.15.71:0/648462231 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 -- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26300cec60 msgr2=0x7f26301f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26300cec60 0x7f26301f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 -- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2630161760 msgr2=0x7f2630072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2630161760 0x7f2630072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 -- 172.21.15.71:0/648462231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26301f8fa0 con 0x7f26300af190 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.269+0000 7f262de00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300af190 0x7f26301f8a60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f262000b440 tx=0x7f262000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.270+0000 7f2626000640 1 -- 172.21.15.71:0/648462231 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620015020 con 0x7f26300af190 2024-09-16T07:34:42.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.270+0000 7f2626000640 1 -- 172.21.15.71:0/648462231 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2620011980 con 0x7f26300af190 2024-09-16T07:34:42.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.270+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263015f250 con 0x7f26300af190 2024-09-16T07:34:42.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.270+0000 7f2626000640 1 -- 172.21.15.71:0/648462231 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620011dd0 con 0x7f26300af190 2024-09-16T07:34:42.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.270+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263015f810 con 0x7f26300af190 2024-09-16T07:34:42.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.271+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26300aef50 con 0x7f26300af190 2024-09-16T07:34:42.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.272+0000 7f2626000640 1 -- 172.21.15.71:0/648462231 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f2620018020 con 0x7f26300af190 2024-09-16T07:34:42.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.273+0000 7f2626000640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f261007a230 0x7f261007c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:42.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.273+0000 7f2626000640 1 -- 172.21.15.71:0/648462231 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4999+0+0 (secure 0 0 0) 0x7f2620099910 con 0x7f26300af190 2024-09-16T07:34:42.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.273+0000 7f262d400640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f261007a230 0x7f261007c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:42.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.274+0000 7f262d400640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f261007a230 0x7f261007c6f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2630073b30 tx=0x7f261c007990 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:42.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.277+0000 7f2626000640 1 -- 172.21.15.71:0/648462231 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f26300aef50 con 0x7f26300af190 2024-09-16T07:34:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.462+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2630162a70 con 0x7f26300af190 2024-09-16T07:34:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.466+0000 7f2626000640 1 -- 172.21.15.71:0/648462231 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f262001f020 con 0x7f26300af190 2024-09-16T07:34:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f261007a230 msgr2=0x7f261007c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f261007a230 0x7f261007c6f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2630073b30 tx=0x7f261c007990 comp rx=0 tx=0).stop 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300af190 msgr2=0x7f26301f8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300af190 0x7f26301f8a60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f262000b440 tx=0x7f262000b900 comp rx=0 tx=0).stop 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 shutdown_connections 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f261007a230 0x7f261007c6f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2630161760 0x7f2630072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26300cec60 0x7f26301f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 --2- 172.21.15.71:0/648462231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300af190 0x7f26301f8a60 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 >> 172.21.15.71:0/648462231 conn(0x7f26300b1840 msgr2=0x7f26300cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 shutdown_connections 2024-09-16T07:34:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.469+0000 7f2634a00640 1 -- 172.21.15.71:0/648462231 wait complete. 2024-09-16T07:34:42.486 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:34:42.486 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:34:42.578 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:34:42.672 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:34:42.672 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:34:42.672 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:42.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:34:42.708 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:34:42.708 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:42.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:34:42.746 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:34:42.746 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:42.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:34:42.783 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:34:42.784 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:34:42.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:34:42.823 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:34:42.824 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:34:42.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.980+0000 7f7f40e00640 1 -- 172.21.15.71:0/1766696213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 msgr2=0x7f7f3c0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.980+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1766696213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 0x7f7f3c0cebf0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c005fd0 tx=0x7f7f2c02a620 comp rx=0 tx=0).stop 2024-09-16T07:34:42.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.980+0000 7f7f40e00640 1 -- 172.21.15.71:0/1766696213 shutdown_connections 2024-09-16T07:34:42.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.980+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1766696213 >> v2:172.21.15.177:3300/0 conn(0x7f7f3c0b4ca0 0x7f7f3c0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.980+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1766696213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 0x7f7f3c0cebf0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.980+0000 7f7f40e00640 1 -- 172.21.15.71:0/1766696213 >> 172.21.15.71:0/1766696213 conn(0x7f7f3c0b1950 msgr2=0x7f7f3c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:42.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.980+0000 7f7f40e00640 1 -- 172.21.15.71:0/1766696213 shutdown_connections 2024-09-16T07:34:42.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.980+0000 7f7f40e00640 1 -- 172.21.15.71:0/1766696213 wait complete. 2024-09-16T07:34:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.982+0000 7f7f40e00640 1 Processor -- start 2024-09-16T07:34:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.982+0000 7f7f40e00640 1 -- start start 2024-09-16T07:34:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f40e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f3c0b4ca0 0x7f7f3c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f40e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f3c0ccfd0 0x7f7f3c1f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f40e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 0x7f7f3c1f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f40e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c15f080 con 0x7f7f3c1445d0 2024-09-16T07:34:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f40e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c073bc0 con 0x7f7f3c0ccfd0 2024-09-16T07:34:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f40e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3c15f200 con 0x7f7f3c0b4ca0 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f3a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 0x7f7f3c1f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f3a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 0x7f7f3c1f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60876/0 (socket says 172.21.15.71:60876) 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f3a800640 1 -- 172.21.15.71:0/1955348753 learned_addr learned my addr 172.21.15.71:0/1955348753 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f39e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f3c0b4ca0 0x7f7f3c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.983+0000 7f7f39400640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f3c0ccfd0 0x7f7f3c1f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f3a800640 1 -- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f3c0b4ca0 msgr2=0x7f7f3c15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f3a800640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f3c0b4ca0 0x7f7f3c15a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f3a800640 1 -- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f3c0ccfd0 msgr2=0x7f7f3c1f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f3a800640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f3c0ccfd0 0x7f7f3c1f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f3a800640 1 -- 172.21.15.71:0/1955348753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c1f2cb0 con 0x7f7f3c1445d0 2024-09-16T07:34:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f3a800640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 0x7f7f3c1f3440 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400b970 tx=0x7f7f3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f32000640 1 -- 172.21.15.71:0/1955348753 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34019070 con 0x7f7f3c1445d0 2024-09-16T07:34:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f32000640 1 -- 172.21.15.71:0/1955348753 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f3400cc50 con 0x7f7f3c1445d0 2024-09-16T07:34:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f32000640 1 -- 172.21.15.71:0/1955348753 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340146a0 con 0x7f7f3c1445d0 2024-09-16T07:34:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c1f6990 con 0x7f7f3c1445d0 2024-09-16T07:34:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.984+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c1f6f50 con 0x7f7f3c1445d0 2024-09-16T07:34:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.986+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c15e970 con 0x7f7f3c1445d0 2024-09-16T07:34:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.987+0000 7f7f32000640 1 -- 172.21.15.71:0/1955348753 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f7f3401c070 con 0x7f7f3c1445d0 2024-09-16T07:34:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.988+0000 7f7f32000640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f1c07a230 0x7f7f1c07c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.988+0000 7f7f32000640 1 -- 172.21.15.71:0/1955348753 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4999+0+0 (secure 0 0 0) 0x7f7f340993a0 con 0x7f7f3c1445d0 2024-09-16T07:34:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.988+0000 7f7f39e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f1c07a230 0x7f7f1c07c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.989+0000 7f7f39e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f1c07a230 0x7f7f1c07c6f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c033000 tx=0x7f7f2c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:42.992+0000 7f7f32000640 1 -- 172.21.15.71:0/1955348753 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7f34062ec0 con 0x7f7f3c1445d0 2024-09-16T07:34:43.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.212+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f3c070a20 con 0x7f7f3c1445d0 2024-09-16T07:34:43.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.212+0000 7f7f32000640 1 -- 172.21.15.71:0/1955348753 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3917 (secure 0 0 0) 0x7f7f340678f0 con 0x7f7f3c1445d0 2024-09-16T07:34:43.214 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:43.214 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":69,"btime":"2024-09-16T07:34:36:437563+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3180037555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3180037555},{"type":"v1","addr":"172.21.15.71:6835","nonce":3180037555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/3241229928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3241229928},{"type":"v1","addr":"172.21.15.177:6836","nonce":3241229928}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1569295783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1569295783},{"type":"v1","addr":"172.21.15.177:6837","nonce":1569295783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":8283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3383298689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3383298689},{"type":"v1","addr":"172.21.15.71:6833","nonce":3383298689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-09-16T07:34:43.214 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 69 2024-09-16T07:34:43.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f1c07a230 msgr2=0x7f7f1c07c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:43.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f1c07a230 0x7f7f1c07c6f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c033000 tx=0x7f7f2c004730 comp rx=0 tx=0).stop 2024-09-16T07:34:43.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 msgr2=0x7f7f3c1f3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:43.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 0x7f7f3c1f3440 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400b970 tx=0x7f7f3400be30 comp rx=0 tx=0).stop 2024-09-16T07:34:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 shutdown_connections 2024-09-16T07:34:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7f1c07a230 0x7f7f1c07c6f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f3c1445d0 0x7f7f3c1f3440 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f3c0ccfd0 0x7f7f3c1f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 --2- 172.21.15.71:0/1955348753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f3c0b4ca0 0x7f7f3c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 >> 172.21.15.71:0/1955348753 conn(0x7f7f3c0b1950 msgr2=0x7f7f3c142210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:43.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 shutdown_connections 2024-09-16T07:34:43.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.215+0000 7f7f40e00640 1 -- 172.21.15.71:0/1955348753 wait complete. 2024-09-16T07:34:43.231 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:34:43.231 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:34:43.232 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:34:43.232 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:34:43.232 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:34:43.232 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:34:43.232 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:34:43.232 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:34:43.233 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:34:43.233 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:34:43.236 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:34:43.236 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:34:43.236 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:34:43.236 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:34:43.236 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:34:43.236 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:34:43.250 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:34:43.250 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:34:43.251 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:34:43.251 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:34:43.252 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:34:43.253 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:34:43.254 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:34:43.254 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:34:43.294 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:34:43.295 DEBUG:teuthology.parallel:result is None 2024-09-16T07:34:43.296 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:34:43.297 DEBUG:teuthology.parallel:result is None 2024-09-16T07:34:43.297 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:34:43.297 DEBUG:teuthology.parallel:result is None 2024-09-16T07:34:43.299 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:34:43.299 DEBUG:teuthology.parallel:result is None 2024-09-16T07:34:43.299 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:34:43.425 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:34:43.427 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:34:43.429 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:34:43.429 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:34:43.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 -- 172.21.15.71:0/3459346017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f9d0 msgr2=0x7f1d2c161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:43.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 --2- 172.21.15.71:0/3459346017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f9d0 0x7f1d2c161c90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c06d9e0 tx=0x7f1d2402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 -- 172.21.15.71:0/3459346017 shutdown_connections 2024-09-16T07:34:43.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 --2- 172.21.15.71:0/3459346017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c06f9d0 0x7f1d2c161c90 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 --2- 172.21.15.71:0/3459346017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c0af3d0 0x7f1d2c06f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 --2- 172.21.15.71:0/3459346017 >> v2:172.21.15.177:3300/0 conn(0x7f1d2c161550 0x7f1d2c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 -- 172.21.15.71:0/3459346017 >> 172.21.15.71:0/3459346017 conn(0x7f1d2c0b1950 msgr2=0x7f1d2c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:43.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 -- 172.21.15.71:0/3459346017 shutdown_connections 2024-09-16T07:34:43.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.466+0000 7f1d32a00640 1 -- 172.21.15.71:0/3459346017 wait complete. 2024-09-16T07:34:43.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.468+0000 7f1d32a00640 1 Processor -- start 2024-09-16T07:34:43.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.468+0000 7f1d32a00640 1 -- start start 2024-09-16T07:34:43.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c06f9d0 0x7f1d2c0cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:43.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c0af3d0 0x7f1d2c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:43.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c161550 0x7f1d2c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:43.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d32a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c162560 con 0x7f1d2c161550 2024-09-16T07:34:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d32a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1623e0 con 0x7f1d2c0af3d0 2024-09-16T07:34:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d32a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2c1626e0 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d2be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c06f9d0 0x7f1d2c0cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d2be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c06f9d0 0x7f1d2c0cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:44812/0 (socket says 172.21.15.71:44812) 2024-09-16T07:34:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d2be00640 1 -- 172.21.15.71:0/1582628029 learned_addr learned my addr 172.21.15.71:0/1582628029 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.469+0000 7f1d30a00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c161550 0x7f1d2c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d2b400640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c0af3d0 0x7f1d2c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d2be00640 1 -- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c0af3d0 msgr2=0x7f1d2c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d2be00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c0af3d0 0x7f1d2c0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d2be00640 1 -- 172.21.15.71:0/1582628029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c161550 msgr2=0x7f1d2c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d2be00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c161550 0x7f1d2c0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d2be00640 1 -- 172.21.15.71:0/1582628029 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c0d0900 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d30a00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c161550 0x7f1d2c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d2b400640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c0af3d0 0x7f1d2c0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d2be00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c06f9d0 0x7f1d2c0cc140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00eb30 tx=0x7f1d1c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:43.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.470+0000 7f1d23e00640 1 -- 172.21.15.71:0/1582628029 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c010430 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.471+0000 7f1d23e00640 1 -- 172.21.15.71:0/1582628029 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d1c0105d0 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.471+0000 7f1d23e00640 1 -- 172.21.15.71:0/1582628029 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c010a20 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.471+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c0cc8b0 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.471+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c0cce70 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.472+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c1f5ab0 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.476+0000 7f1d23e00640 1 -- 172.21.15.71:0/1582628029 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f1d1c010bc0 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.477+0000 7f1d23e00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0807a260 0x7f1d0807c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:43.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.477+0000 7f1d23e00640 1 -- 172.21.15.71:0/1582628029 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(112..112 src has 1..112) ==== 4999+0+0 (secure 0 0 0) 0x7f1d1c09acb0 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.477+0000 7f1d2b400640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0807a260 0x7f1d0807c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:43.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.478+0000 7f1d2b400640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0807a260 0x7f1d0807c720 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0aebf0 tx=0x7f1d14005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:43.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.480+0000 7f1d23e00640 1 -- 172.21.15.71:0/1582628029 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1d1c064f00 con 0x7f1d2c06f9d0 2024-09-16T07:34:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:43.678+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1d2c161c90 con 0x7f1d2c06f9d0 2024-09-16T07:34:44.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.496+0000 7f1d23e00640 1 -- 172.21.15.71:0/1582628029 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v115) ==== 97+0+0 (secure 0 0 0) 0x7f1d1c064800 con 0x7f1d2c06f9d0 2024-09-16T07:34:44.496 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:34:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0807a260 msgr2=0x7f1d0807c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0807a260 0x7f1d0807c720 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0aebf0 tx=0x7f1d14005f90 comp rx=0 tx=0).stop 2024-09-16T07:34:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c06f9d0 msgr2=0x7f1d2c0cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c06f9d0 0x7f1d2c0cc140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00eb30 tx=0x7f1d1c00c2d0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 shutdown_connections 2024-09-16T07:34:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1d0807a260 0x7f1d0807c720 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d2c161550 0x7f1d2c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c0af3d0 0x7f1d2c0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 --2- 172.21.15.71:0/1582628029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d2c06f9d0 0x7f1d2c0cc140 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 >> 172.21.15.71:0/1582628029 conn(0x7f1d2c0b1950 msgr2=0x7f1d2c072d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 shutdown_connections 2024-09-16T07:34:44.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.498+0000 7f1d32a00640 1 -- 172.21.15.71:0/1582628029 wait complete. 2024-09-16T07:34:44.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:34:44.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.673+0000 7fda00000640 1 -- 172.21.15.71:0/3541289038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9f80af3d0 msgr2=0x7fd9f806f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:44.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.673+0000 7fda00000640 1 --2- 172.21.15.71:0/3541289038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9f80af3d0 0x7fd9f806f490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009830 tx=0x7fd9e402a480 comp rx=0 tx=0).stop 2024-09-16T07:34:44.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.674+0000 7fda00000640 1 -- 172.21.15.71:0/3541289038 shutdown_connections 2024-09-16T07:34:44.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.674+0000 7fda00000640 1 --2- 172.21.15.71:0/3541289038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9f80af3d0 0x7fd9f806f490 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.674+0000 7fda00000640 1 --2- 172.21.15.71:0/3541289038 >> v2:172.21.15.177:3300/0 conn(0x7fd9f8161550 0x7fd9f806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.674+0000 7fda00000640 1 -- 172.21.15.71:0/3541289038 >> 172.21.15.71:0/3541289038 conn(0x7fd9f80b1950 msgr2=0x7fd9f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:44.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.674+0000 7fda00000640 1 -- 172.21.15.71:0/3541289038 shutdown_connections 2024-09-16T07:34:44.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.674+0000 7fda00000640 1 -- 172.21.15.71:0/3541289038 wait complete. 2024-09-16T07:34:44.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.675+0000 7fda00000640 1 Processor -- start 2024-09-16T07:34:44.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.675+0000 7fda00000640 1 -- start start 2024-09-16T07:34:44.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.676+0000 7fda00000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f806f9d0 0x7fd9f809d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:44.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.676+0000 7fda00000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9f80af3d0 0x7fd9f815fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:44.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.676+0000 7fda00000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9f8161550 0x7fd9f81600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:44.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.676+0000 7fda00000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f81623a0 con 0x7fd9f806f9d0 2024-09-16T07:34:44.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.676+0000 7fda00000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f8074010 con 0x7fd9f8161550 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.676+0000 7fda00000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f8162520 con 0x7fd9f80af3d0 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.676+0000 7fd9fd600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f806f9d0 0x7fd9f809d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fd600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f806f9d0 0x7fd9f809d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60948/0 (socket says 172.21.15.71:60948) 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fd600640 1 -- 172.21.15.71:0/2121018154 learned_addr learned my addr 172.21.15.71:0/2121018154 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fcc00640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9f80af3d0 0x7fd9f815fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fe000640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9f8161550 0x7fd9f81600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fd600640 1 -- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9f80af3d0 msgr2=0x7fd9f815fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fd600640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9f80af3d0 0x7fd9f815fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fd600640 1 -- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9f8161550 msgr2=0x7fd9f81600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:44.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fd600640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9f8161550 0x7fd9f81600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fd600640 1 -- 172.21.15.71:0/2121018154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f81f48c0 con 0x7fd9f806f9d0 2024-09-16T07:34:44.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9fd600640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f806f9d0 0x7fd9f809d490 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00eb30 tx=0x7fd9ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:44.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9f5600640 1 -- 172.21.15.71:0/2121018154 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec010430 con 0x7fd9f806f9d0 2024-09-16T07:34:44.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9f5600640 1 -- 172.21.15.71:0/2121018154 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9ec0105d0 con 0x7fd9f806f9d0 2024-09-16T07:34:44.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fd9f5600640 1 -- 172.21.15.71:0/2121018154 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec010a20 con 0x7fd9f806f9d0 2024-09-16T07:34:44.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.677+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f809c3f0 con 0x7fd9f806f9d0 2024-09-16T07:34:44.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.678+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f809c9b0 con 0x7fd9f806f9d0 2024-09-16T07:34:44.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.679+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f80af080 con 0x7fd9f806f9d0 2024-09-16T07:34:44.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.680+0000 7fd9f5600640 1 -- 172.21.15.71:0/2121018154 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fd9ec010bc0 con 0x7fd9f806f9d0 2024-09-16T07:34:44.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.681+0000 7fd9f5600640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9d807a230 0x7fd9d807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:44.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.681+0000 7fd9f5600640 1 -- 172.21.15.71:0/2121018154 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4913+0+0 (secure 0 0 0) 0x7fd9ec09a670 con 0x7fd9f806f9d0 2024-09-16T07:34:44.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.681+0000 7fd9fcc00640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9d807a230 0x7fd9d807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:44.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.682+0000 7fd9fcc00640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9d807a230 0x7fd9d807c6f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4034000 tx=0x7fd9e4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:44.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.685+0000 7fd9f5600640 1 -- 172.21.15.71:0/2121018154 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd9f80af080 con 0x7fd9f806f9d0 2024-09-16T07:34:44.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.942+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd9f8162ab0 con 0x7fd9f806f9d0 2024-09-16T07:34:44.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.943+0000 7fd9f5600640 1 -- 172.21.15.71:0/2121018154 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fd9ec0646c0 con 0x7fd9f806f9d0 2024-09-16T07:34:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:44.944 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:34:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:34:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:34:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.944 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:34:44.945 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:34:44.947 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:34:44.948 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:34:44.950 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:34:44.951 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:34:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.953 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:34:44.954 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.956 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.957 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:44.959 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:34:44.959 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9d807a230 msgr2=0x7fd9d807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9d807a230 0x7fd9d807c6f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4034000 tx=0x7fd9e4004730 comp rx=0 tx=0).stop 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f806f9d0 msgr2=0x7fd9f809d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f806f9d0 0x7fd9f809d490 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00eb30 tx=0x7fd9ec00c2d0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 shutdown_connections 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9d807a230 0x7fd9d807c6f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9f8161550 0x7fd9f81600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9f80af3d0 0x7fd9f815fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 --2- 172.21.15.71:0/2121018154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f806f9d0 0x7fd9f809d490 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.945+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 >> 172.21.15.71:0/2121018154 conn(0x7fd9f80b1950 msgr2=0x7fd9f80ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.946+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 shutdown_connections 2024-09-16T07:34:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:44.946+0000 7fda00000640 1 -- 172.21.15.71:0/2121018154 wait complete. 2024-09-16T07:34:44.963 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:34:45.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.117+0000 7f6ba7200640 1 -- 172.21.15.71:0/2688292117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 msgr2=0x7f6ba806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:45.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.118+0000 7f6ba7200640 1 --2- 172.21.15.71:0/2688292117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 0x7f6ba806f320 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8073b00 tx=0x7f6ba002a8b0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.118+0000 7f6ba7200640 1 -- 172.21.15.71:0/2688292117 shutdown_connections 2024-09-16T07:34:45.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.118+0000 7f6ba7200640 1 --2- 172.21.15.71:0/2688292117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 0x7f6ba806f320 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.118+0000 7f6ba7200640 1 --2- 172.21.15.71:0/2688292117 >> v2:172.21.15.177:3300/0 conn(0x7f6ba80725d0 0x7f6ba80b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.118+0000 7f6ba7200640 1 -- 172.21.15.71:0/2688292117 >> 172.21.15.71:0/2688292117 conn(0x7f6ba80cfa20 msgr2=0x7f6ba80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:45.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.118+0000 7f6ba7200640 1 -- 172.21.15.71:0/2688292117 shutdown_connections 2024-09-16T07:34:45.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.118+0000 7f6ba7200640 1 -- 172.21.15.71:0/2688292117 wait complete. 2024-09-16T07:34:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.119+0000 7f6ba7200640 1 Processor -- start 2024-09-16T07:34:45.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.119+0000 7f6ba7200640 1 -- start start 2024-09-16T07:34:45.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.120+0000 7f6ba7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 0x7f6ba80c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:45.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.120+0000 7f6ba7200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ba815bc30 0x7f6ba81f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:45.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.120+0000 7f6ba7200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ba81f8170 0x7f6ba81f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:45.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.120+0000 7f6ba7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8074660 con 0x7f6ba80d0540 2024-09-16T07:34:45.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.120+0000 7f6ba7200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba806f860 con 0x7f6ba81f8170 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.120+0000 7f6ba7200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba81e97d0 con 0x7f6ba815bc30 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 0x7f6ba80c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 0x7f6ba80c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34396/0 (socket says 172.21.15.71:34396) 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 -- 172.21.15.71:0/1950976907 learned_addr learned my addr 172.21.15.71:0/1950976907 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba6800640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ba81f8170 0x7f6ba81f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5400640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ba815bc30 0x7f6ba81f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 -- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ba815bc30 msgr2=0x7f6ba81f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ba815bc30 0x7f6ba81f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 -- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ba81f8170 msgr2=0x7f6ba81f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ba81f8170 0x7f6ba81f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 -- 172.21.15.71:0/1950976907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80c6c90 con 0x7f6ba80d0540 2024-09-16T07:34:45.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5400640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ba815bc30 0x7f6ba81f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:45.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.121+0000 7f6ba5e00640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 0x7f6ba80c9560 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c430 tx=0x7f6b9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:45.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.122+0000 7f6b9e000640 1 -- 172.21.15.71:0/1950976907 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98011790 con 0x7f6ba80d0540 2024-09-16T07:34:45.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.122+0000 7f6b9e000640 1 -- 172.21.15.71:0/1950976907 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b98017070 con 0x7f6ba80d0540 2024-09-16T07:34:45.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.122+0000 7f6b9e000640 1 -- 172.21.15.71:0/1950976907 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98011ce0 con 0x7f6ba80d0540 2024-09-16T07:34:45.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.122+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba81f8b60 con 0x7f6ba80d0540 2024-09-16T07:34:45.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.122+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81e9210 con 0x7f6ba80d0540 2024-09-16T07:34:45.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.123+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba806fdf0 con 0x7f6ba80d0540 2024-09-16T07:34:45.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.124+0000 7f6b9e000640 1 -- 172.21.15.71:0/1950976907 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f6b9801c460 con 0x7f6ba80d0540 2024-09-16T07:34:45.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.125+0000 7f6b9e000640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b8807a230 0x7f6b8807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:45.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.125+0000 7f6b9e000640 1 -- 172.21.15.71:0/1950976907 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4913+0+0 (secure 0 0 0) 0x7f6b98099620 con 0x7f6ba80d0540 2024-09-16T07:34:45.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.125+0000 7f6ba5400640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b8807a230 0x7f6b8807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:45.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.126+0000 7f6ba5400640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b8807a230 0x7f6b8807c6f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6ba81f7b90 tx=0x7f6b94005e50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:45.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.129+0000 7f6b9e000640 1 -- 172.21.15.71:0/1950976907 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6b980636f0 con 0x7f6ba80d0540 2024-09-16T07:34:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.316+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6ba80c6e60 con 0x7f6ba80d0540 2024-09-16T07:34:45.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.316+0000 7f6b9e000640 1 -- 172.21.15.71:0/1950976907 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v115) ==== 84+0+65 (secure 0 0 0) 0x7f6b98062ff0 con 0x7f6ba80d0540 2024-09-16T07:34:45.317 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:45.317 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:34:45.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:34:45.317 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:34:45.318 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:34:45.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:34:45.318 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:34:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b8807a230 msgr2=0x7f6b8807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b8807a230 0x7f6b8807c6f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6ba81f7b90 tx=0x7f6b94005e50 comp rx=0 tx=0).stop 2024-09-16T07:34:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 msgr2=0x7f6ba80c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 0x7f6ba80c9560 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c430 tx=0x7f6b9800c8f0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 shutdown_connections 2024-09-16T07:34:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6b8807a230 0x7f6b8807c6f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ba81f8170 0x7f6ba81f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ba815bc30 0x7f6ba81f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 --2- 172.21.15.71:0/1950976907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba80d0540 0x7f6ba80c9560 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 >> 172.21.15.71:0/1950976907 conn(0x7f6ba80cfa20 msgr2=0x7f6ba81581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 shutdown_connections 2024-09-16T07:34:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.319+0000 7f6ba7200640 1 -- 172.21.15.71:0/1950976907 wait complete. 2024-09-16T07:34:45.337 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:34:45.337 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:34:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.498+0000 7fec83800640 1 -- 172.21.15.71:0/880034599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec60004d40 msgr2=0x7fec600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.498+0000 7fec83800640 1 --2- 172.21.15.71:0/880034599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec60004d40 0x7fec600051c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c780 tx=0x7fec60005830 comp rx=0 tx=0).stop 2024-09-16T07:34:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.498+0000 7fec81800640 1 -- 172.21.15.71:0/880034599 reap_dead start 2024-09-16T07:34:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.498+0000 7fec83800640 1 -- 172.21.15.71:0/880034599 shutdown_connections 2024-09-16T07:34:45.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.498+0000 7fec83800640 1 -- 172.21.15.71:0/880034599 >> 172.21.15.71:0/880034599 conn(0x7fec7c0b18e0 msgr2=0x7fec7c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:45.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.498+0000 7fec83800640 1 -- 172.21.15.71:0/880034599 shutdown_connections 2024-09-16T07:34:45.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.498+0000 7fec83800640 1 -- 172.21.15.71:0/880034599 wait complete. 2024-09-16T07:34:45.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.500+0000 7fec83800640 1 Processor -- start 2024-09-16T07:34:45.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.500+0000 7fec83800640 1 -- start start 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec83800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec60004d40 0x7fec7c1f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec83800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c0cae70 0x7fec7c15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec83800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0b3450 0x7fec7c0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec83800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c074720 con 0x7fec7c0cae70 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec83800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0cdd40 con 0x7fec60004d40 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec83800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c0748a0 con 0x7fec7c0b3450 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c0cae70 0x7fec7c15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c0cae70 0x7fec7c15e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34418/0 (socket says 172.21.15.71:34418) 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.501+0000 7fec7be00640 1 -- 172.21.15.71:0/2596224469 learned_addr learned my addr 172.21.15.71:0/2596224469 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec80e00640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec60004d40 0x7fec7c1f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec81800640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0b3450 0x7fec7c0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec7be00640 1 -- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0b3450 msgr2=0x7fec7c0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec7be00640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0b3450 0x7fec7c0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec7be00640 1 -- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec60004d40 msgr2=0x7fec7c1f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec7be00640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec60004d40 0x7fec7c1f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec7be00640 1 -- 172.21.15.71:0/2596224469 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c1f5ca0 con 0x7fec7c0cae70 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec81800640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0b3450 0x7fec7c0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec80e00640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec60004d40 0x7fec7c1f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec7be00640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c0cae70 0x7fec7c15e9d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fec68011ae0 tx=0x7fec68009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec78c00640 1 -- 172.21.15.71:0/2596224469 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6800fcb0 con 0x7fec7c0cae70 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec78c00640 1 -- 172.21.15.71:0/2596224469 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec6800fe50 con 0x7fec7c0cae70 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec78c00640 1 -- 172.21.15.71:0/2596224469 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68004280 con 0x7fec7c0cae70 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c0cb330 con 0x7fec7c0cae70 2024-09-16T07:34:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.502+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c0cb8f0 con 0x7fec7c0cae70 2024-09-16T07:34:45.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.503+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7c069360 con 0x7fec7c0cae70 2024-09-16T07:34:45.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.505+0000 7fec78c00640 1 -- 172.21.15.71:0/2596224469 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fec680035f0 con 0x7fec7c0cae70 2024-09-16T07:34:45.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.505+0000 7fec78c00640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec5c07a230 0x7fec5c07c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:45.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.506+0000 7fec78c00640 1 -- 172.21.15.71:0/2596224469 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4913+0+0 (secure 0 0 0) 0x7fec6809a250 con 0x7fec7c0cae70 2024-09-16T07:34:45.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.506+0000 7fec80e00640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec5c07a230 0x7fec5c07c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:45.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.507+0000 7fec80e00640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec5c07a230 0x7fec5c07c6f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009b40 tx=0x7fec6c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:45.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.510+0000 7fec78c00640 1 -- 172.21.15.71:0/2596224469 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fec7c1f5ca0 con 0x7fec7c0cae70 2024-09-16T07:34:45.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:45.695+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fec7c073d50 con 0x7fec7c0cae70 2024-09-16T07:34:46.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:34:46.442 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:34:46.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.490+0000 7fec78c00640 1 -- 172.21.15.71:0/2596224469 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v116) ==== 138+0+0 (secure 0 0 0) 0x7fec680643c0 con 0x7fec7c0cae70 2024-09-16T07:34:46.491 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:34:46.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec5c07a230 msgr2=0x7fec5c07c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:46.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec5c07a230 0x7fec5c07c6f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009b40 tx=0x7fec6c005f90 comp rx=0 tx=0).stop 2024-09-16T07:34:46.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c0cae70 msgr2=0x7fec7c15e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:46.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c0cae70 0x7fec7c15e9d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fec68011ae0 tx=0x7fec68009f10 comp rx=0 tx=0).stop 2024-09-16T07:34:46.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 shutdown_connections 2024-09-16T07:34:46.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec5c07a230 0x7fec5c07c6f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:46.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0b3450 0x7fec7c0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:46.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c0cae70 0x7fec7c15e9d0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:46.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 --2- 172.21.15.71:0/2596224469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec60004d40 0x7fec7c1f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:46.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.495+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 >> 172.21.15.71:0/2596224469 conn(0x7fec7c0b18e0 msgr2=0x7fec7c0cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:46.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.496+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 shutdown_connections 2024-09-16T07:34:46.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.496+0000 7fec83800640 1 -- 172.21.15.71:0/2596224469 wait complete. 2024-09-16T07:34:46.517 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:34:46.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.684+0000 7f70e4000640 1 -- 172.21.15.71:0/3749702950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 msgr2=0x7f70dc073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:46.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.684+0000 7f70e4000640 1 --2- 172.21.15.71:0/3749702950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 0x7f70dc073440 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f70c8005fd0 tx=0x7f70c802a540 comp rx=0 tx=0).stop 2024-09-16T07:34:46.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.684+0000 7f70e4000640 1 -- 172.21.15.71:0/3749702950 shutdown_connections 2024-09-16T07:34:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.684+0000 7f70e4000640 1 --2- 172.21.15.71:0/3749702950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70dc06efa0 0x7f70dc0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.684+0000 7f70e4000640 1 --2- 172.21.15.71:0/3749702950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 0x7f70dc073440 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.684+0000 7f70e4000640 1 -- 172.21.15.71:0/3749702950 >> 172.21.15.71:0/3749702950 conn(0x7f70dc0b1840 msgr2=0x7f70dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.684+0000 7f70e4000640 1 -- 172.21.15.71:0/3749702950 shutdown_connections 2024-09-16T07:34:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.684+0000 7f70e4000640 1 -- 172.21.15.71:0/3749702950 wait complete. 2024-09-16T07:34:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.686+0000 7f70e4000640 1 Processor -- start 2024-09-16T07:34:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.686+0000 7f70e4000640 1 -- start start 2024-09-16T07:34:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e4000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70dc06efa0 0x7f70dc142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e4000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 0x7f70dc0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e4000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70dc145750 0x7f70dc0b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e4000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0ce480 con 0x7f70dc0cccd0 2024-09-16T07:34:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e4000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0ce300 con 0x7f70dc06efa0 2024-09-16T07:34:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e4000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0ce600 con 0x7f70dc145750 2024-09-16T07:34:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 0x7f70dc0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 0x7f70dc0c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34434/0 (socket says 172.21.15.71:34434) 2024-09-16T07:34:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e0c00640 1 -- 172.21.15.71:0/3014209341 learned_addr learned my addr 172.21.15.71:0/3014209341 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e2000640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70dc145750 0x7f70dc0b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.687+0000 7f70e1600640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70dc06efa0 0x7f70dc142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70e0c00640 1 -- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70dc145750 msgr2=0x7f70dc0b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70e0c00640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70dc145750 0x7f70dc0b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70e0c00640 1 -- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70dc06efa0 msgr2=0x7f70dc142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70e0c00640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70dc06efa0 0x7f70dc142bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70e0c00640 1 -- 172.21.15.71:0/3014209341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0b3a20 con 0x7f70dc0cccd0 2024-09-16T07:34:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70e0c00640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 0x7f70dc0c5690 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f70c8000c00 tx=0x7f70c80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70d9600640 1 -- 172.21.15.71:0/3014209341 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c803f070 con 0x7f70dc0cccd0 2024-09-16T07:34:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70d9600640 1 -- 172.21.15.71:0/3014209341 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70c80047a0 con 0x7f70dc0cccd0 2024-09-16T07:34:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70d9600640 1 -- 172.21.15.71:0/3014209341 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8035030 con 0x7f70dc0cccd0 2024-09-16T07:34:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc0b3cc0 con 0x7f70dc0cccd0 2024-09-16T07:34:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.688+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc0b4280 con 0x7f70dc0cccd0 2024-09-16T07:34:46.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.689+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc0cdbb0 con 0x7f70dc0cccd0 2024-09-16T07:34:46.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.691+0000 7f70d9600640 1 -- 172.21.15.71:0/3014209341 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f70c8042070 con 0x7f70dc0cccd0 2024-09-16T07:34:46.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.692+0000 7f70d9600640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70bc07a230 0x7f70bc07c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:46.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.692+0000 7f70d9600640 1 -- 172.21.15.71:0/3014209341 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5283+0+0 (secure 0 0 0) 0x7f70c80baa90 con 0x7f70dc0cccd0 2024-09-16T07:34:46.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.693+0000 7f70e1600640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70bc07a230 0x7f70bc07c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:46.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.693+0000 7f70e1600640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70bc07a230 0x7f70bc07c6f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f70d0001e20 tx=0x7f70d0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.696+0000 7f70d9600640 1 -- 172.21.15.71:0/3014209341 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f70c80846b0 con 0x7f70dc0cccd0 2024-09-16T07:34:46.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:46.881+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f70dc0cfde0 con 0x7f70dc0cccd0 2024-09-16T07:34:47.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.499+0000 7f70d9600640 1 -- 172.21.15.71:0/3014209341 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v117) ==== 170+0+0 (secure 0 0 0) 0x7f70c80890e0 con 0x7f70dc0cccd0 2024-09-16T07:34:47.499 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:34:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.501+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70bc07a230 msgr2=0x7f70bc07c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.501+0000 7f70e4000640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70bc07a230 0x7f70bc07c6f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f70d0001e20 tx=0x7f70d0009040 comp rx=0 tx=0).stop 2024-09-16T07:34:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.501+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 msgr2=0x7f70dc0c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.501+0000 7f70e4000640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 0x7f70dc0c5690 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f70c8000c00 tx=0x7f70c80038a0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.501+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 shutdown_connections 2024-09-16T07:34:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.502+0000 7f70e4000640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70bc07a230 0x7f70bc07c6f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.502+0000 7f70e4000640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70dc145750 0x7f70dc0b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.502+0000 7f70e4000640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70dc0cccd0 0x7f70dc0c5690 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.502+0000 7f70e4000640 1 --2- 172.21.15.71:0/3014209341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70dc06efa0 0x7f70dc142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.502+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 >> 172.21.15.71:0/3014209341 conn(0x7f70dc0b1840 msgr2=0x7f70dc15f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.502+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 shutdown_connections 2024-09-16T07:34:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.502+0000 7f70e4000640 1 -- 172.21.15.71:0/3014209341 wait complete. 2024-09-16T07:34:47.532 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:34:47.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.692+0000 7fad2ca00640 1 -- 172.21.15.71:0/3378318444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad280c91e0 msgr2=0x7fad280d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:47.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.692+0000 7fad2ca00640 1 --2- 172.21.15.71:0/3378318444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad280c91e0 0x7fad280d0e50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fad10005fd0 tx=0x7fad1002a5d0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.692+0000 7fad2ca00640 1 -- 172.21.15.71:0/3378318444 shutdown_connections 2024-09-16T07:34:47.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.692+0000 7fad2ca00640 1 --2- 172.21.15.71:0/3378318444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad280c91e0 0x7fad280d0e50 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.692+0000 7fad2ca00640 1 -- 172.21.15.71:0/3378318444 >> 172.21.15.71:0/3378318444 conn(0x7fad280d3de0 msgr2=0x7fad280d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:47.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.692+0000 7fad2ca00640 1 -- 172.21.15.71:0/3378318444 shutdown_connections 2024-09-16T07:34:47.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.692+0000 7fad2ca00640 1 -- 172.21.15.71:0/3378318444 wait complete. 2024-09-16T07:34:47.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.693+0000 7fad2ca00640 1 Processor -- start 2024-09-16T07:34:47.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.694+0000 7fad2ca00640 1 -- start start 2024-09-16T07:34:47.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.694+0000 7fad2ca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280c91e0 0x7fad2814c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:47.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.694+0000 7fad2ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280d19b0 0x7fad281f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:47.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.694+0000 7fad2ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2815df60 0x7fad281f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad2ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fad280d2570 con 0x7fad2815df60 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad2ca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad280d15a0 con 0x7fad280d19b0 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad2ca00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fad280d26f0 con 0x7fad280c91e0 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad27200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2815df60 0x7fad281f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad27200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2815df60 0x7fad281f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34452/0 (socket says 172.21.15.71:34452) 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad27200640 1 -- 172.21.15.71:0/685020716 learned_addr learned my addr 172.21.15.71:0/685020716 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad1fe00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280d19b0 0x7fad281f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad26800640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280c91e0 0x7fad2814c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad27200640 1 -- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280c91e0 msgr2=0x7fad2814c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad27200640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280c91e0 0x7fad2814c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad27200640 1 -- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280d19b0 msgr2=0x7fad281f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad27200640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280d19b0 0x7fad281f8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:47.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad27200640 1 -- 172.21.15.71:0/685020716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2814c7d0 con 0x7fad2815df60 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad1fe00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280d19b0 0x7fad281f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.695+0000 7fad26800640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280c91e0 0x7fad2814c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.696+0000 7fad27200640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2815df60 0x7fad281f5430 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c7b0 tx=0x7fad2000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.696+0000 7fad1ea00640 1 -- 172.21.15.71:0/685020716 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20004b60 con 0x7fad2815df60 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.696+0000 7fad1ea00640 1 -- 172.21.15.71:0/685020716 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad20004df0 con 0x7fad2815df60 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.696+0000 7fad1ea00640 1 -- 172.21.15.71:0/685020716 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2000aa30 con 0x7fad2815df60 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.696+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad281f86d0 con 0x7fad2815df60 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.696+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad281f8f00 con 0x7fad2815df60 2024-09-16T07:34:47.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.697+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad2806e6e0 con 0x7fad2815df60 2024-09-16T07:34:47.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.699+0000 7fad1ea00640 1 -- 172.21.15.71:0/685020716 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fad2000abd0 con 0x7fad2815df60 2024-09-16T07:34:47.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.700+0000 7fad1ea00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad0407a230 0x7fad0407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:47.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.700+0000 7fad1ea00640 1 -- 172.21.15.71:0/685020716 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5665+0+0 (secure 0 0 0) 0x7fad20099a30 con 0x7fad2815df60 2024-09-16T07:34:47.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.700+0000 7fad26800640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad0407a230 0x7fad0407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:47.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.701+0000 7fad26800640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad0407a230 0x7fad0407c6f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fad18007fd0 tx=0x7fad18009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:47.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.704+0000 7fad1ea00640 1 -- 172.21.15.71:0/685020716 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fad2806e6e0 con 0x7fad2815df60 2024-09-16T07:34:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:47.924+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fad280d2b90 con 0x7fad2815df60 2024-09-16T07:34:48.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.503+0000 7fad1ea00640 1 -- 172.21.15.71:0/685020716 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v71) ==== 174+0+0 (secure 0 0 0) 0x7fad20020070 con 0x7fad2815df60 2024-09-16T07:34:48.503 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 16 and data pool 17 2024-09-16T07:34:48.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad0407a230 msgr2=0x7fad0407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad0407a230 0x7fad0407c6f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fad18007fd0 tx=0x7fad18009040 comp rx=0 tx=0).stop 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2815df60 msgr2=0x7fad281f5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2815df60 0x7fad281f5430 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c7b0 tx=0x7fad2000cc70 comp rx=0 tx=0).stop 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 shutdown_connections 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad0407a230 0x7fad0407c6f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2815df60 0x7fad281f5430 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280d19b0 0x7fad281f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 --2- 172.21.15.71:0/685020716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280c91e0 0x7fad2814c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 >> 172.21.15.71:0/685020716 conn(0x7fad280d3de0 msgr2=0x7fad2806db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 shutdown_connections 2024-09-16T07:34:48.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.506+0000 7fad2ca00640 1 -- 172.21.15.71:0/685020716 wait complete. 2024-09-16T07:34:48.540 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:34:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.698+0000 7f1801000640 1 -- 172.21.15.71:0/2175338864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb1b0 msgr2=0x7f17fc15f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.698+0000 7f1801000640 1 --2- 172.21.15.71:0/2175338864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb1b0 0x7f17fc15f160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009860 tx=0x7f17ec02a530 comp rx=0 tx=0).stop 2024-09-16T07:34:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.698+0000 7f1801000640 1 -- 172.21.15.71:0/2175338864 shutdown_connections 2024-09-16T07:34:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.698+0000 7f1801000640 1 --2- 172.21.15.71:0/2175338864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb1b0 0x7f17fc15f160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.698+0000 7f1801000640 1 -- 172.21.15.71:0/2175338864 >> 172.21.15.71:0/2175338864 conn(0x7f17fc0cffe0 msgr2=0x7f17fc0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.698+0000 7f1801000640 1 -- 172.21.15.71:0/2175338864 shutdown_connections 2024-09-16T07:34:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.698+0000 7f1801000640 1 -- 172.21.15.71:0/2175338864 wait complete. 2024-09-16T07:34:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.700+0000 7f1801000640 1 Processor -- start 2024-09-16T07:34:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.700+0000 7f1801000640 1 -- start start 2024-09-16T07:34:48.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f1801000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17fc0cb1b0 0x7f17fc0b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:48.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f1801000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb590 0x7f17fc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:48.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f1801000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc15f6a0 0x7f17fc153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:48.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f1801000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc073ff0 con 0x7f17fc15f6a0 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f1801000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc0af7b0 con 0x7f17fc0cb1b0 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f1801000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f17fc0ca630 con 0x7f17fc0cb590 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc15f6a0 0x7f17fc153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc15f6a0 0x7f17fc153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34460/0 (socket says 172.21.15.71:34460) 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17fa800640 1 -- 172.21.15.71:0/1417434444 learned_addr learned my addr 172.21.15.71:0/1417434444 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17f9e00640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17fc0cb1b0 0x7f17fc0b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17f9400640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb590 0x7f17fc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17fa800640 1 -- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb590 msgr2=0x7f17fc0b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17fa800640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb590 0x7f17fc0b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17fa800640 1 -- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17fc0cb1b0 msgr2=0x7f17fc0b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17fa800640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17fc0cb1b0 0x7f17fc0b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17fa800640 1 -- 172.21.15.71:0/1417434444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc0b5520 con 0x7f17fc15f6a0 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17f9400640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb590 0x7f17fc0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.701+0000 7f17f9e00640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17fc0cb1b0 0x7f17fc0b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.702+0000 7f17fa800640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc15f6a0 0x7f17fc153380 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c780 tx=0x7f17f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.702+0000 7f17f2000640 1 -- 172.21.15.71:0/1417434444 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4003b70 con 0x7f17fc15f6a0 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.702+0000 7f17f2000640 1 -- 172.21.15.71:0/1417434444 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17f4003d30 con 0x7f17fc15f6a0 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.702+0000 7f17f2000640 1 -- 172.21.15.71:0/1417434444 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f40052d0 con 0x7f17fc15f6a0 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.702+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc0b3950 con 0x7f17fc15f6a0 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.702+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc0b3f10 con 0x7f17fc15f6a0 2024-09-16T07:34:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.703+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17fc0af280 con 0x7f17fc15f6a0 2024-09-16T07:34:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.704+0000 7f17f2000640 1 -- 172.21.15.71:0/1417434444 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f17f4005470 con 0x7f17fc15f6a0 2024-09-16T07:34:48.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.705+0000 7f17f2000640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17dc07a230 0x7f17dc07c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:48.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.705+0000 7f17f2000640 1 -- 172.21.15.71:0/1417434444 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5765+0+0 (secure 0 0 0) 0x7f17f40995a0 con 0x7f17fc15f6a0 2024-09-16T07:34:48.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.706+0000 7f17f9e00640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17dc07a230 0x7f17dc07c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:48.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.707+0000 7f17f9e00640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17dc07a230 0x7f17dc07c6f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f17ec034000 tx=0x7f17ec004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.710+0000 7f17f2000640 1 -- 172.21.15.71:0/1417434444 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f17f40630c0 con 0x7f17fc15f6a0 2024-09-16T07:34:48.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.895+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17fc06dc30 con 0x7f17fc15f6a0 2024-09-16T07:34:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.896+0000 7f17f2000640 1 -- 172.21.15.71:0/1417434444 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7f17f4067af0 con 0x7f17fc15f6a0 2024-09-16T07:34:48.895 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:48.896 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":118,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:34:47.926470+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:34:45.706284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T07:34:46.892492+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17dc07a230 msgr2=0x7f17dc07c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17dc07a230 0x7f17dc07c6f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f17ec034000 tx=0x7f17ec004730 comp rx=0 tx=0).stop 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc15f6a0 msgr2=0x7f17fc153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc15f6a0 0x7f17fc153380 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c780 tx=0x7f17f400cc40 comp rx=0 tx=0).stop 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 shutdown_connections 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f17dc07a230 0x7f17dc07c6f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc15f6a0 0x7f17fc153380 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17fc0cb590 0x7f17fc0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 --2- 172.21.15.71:0/1417434444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17fc0cb1b0 0x7f17fc0b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 >> 172.21.15.71:0/1417434444 conn(0x7f17fc0cffe0 msgr2=0x7f17fc15e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 shutdown_connections 2024-09-16T07:34:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:48.898+0000 7f1801000640 1 -- 172.21.15.71:0/1417434444 wait complete. 2024-09-16T07:34:48.914 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:34:49.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.070+0000 7fa811200640 1 -- 172.21.15.71:0/1065108929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 msgr2=0x7fa80c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:49.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.070+0000 7fa811200640 1 --2- 172.21.15.71:0/1065108929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 0x7fa80c0b2440 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc005fd0 tx=0x7fa7fc02a540 comp rx=0 tx=0).stop 2024-09-16T07:34:49.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.070+0000 7fa811200640 1 -- 172.21.15.71:0/1065108929 shutdown_connections 2024-09-16T07:34:49.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.070+0000 7fa811200640 1 --2- 172.21.15.71:0/1065108929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa80c0ccf00 0x7fa80c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.070+0000 7fa811200640 1 --2- 172.21.15.71:0/1065108929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 0x7fa80c0b2440 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.070+0000 7fa811200640 1 -- 172.21.15.71:0/1065108929 >> 172.21.15.71:0/1065108929 conn(0x7fa80c0b1950 msgr2=0x7fa80c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:49.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.070+0000 7fa811200640 1 -- 172.21.15.71:0/1065108929 shutdown_connections 2024-09-16T07:34:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.070+0000 7fa811200640 1 -- 172.21.15.71:0/1065108929 wait complete. 2024-09-16T07:34:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.072+0000 7fa811200640 1 Processor -- start 2024-09-16T07:34:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.072+0000 7fa811200640 1 -- start start 2024-09-16T07:34:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa811200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa80c0ad270 0x7fa80c1ebf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa811200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa80c0ccf00 0x7fa80c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa811200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 0x7fa80c0d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa811200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c06f2e0 con 0x7fa80c15f5a0 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa811200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c06f160 con 0x7fa80c0ccf00 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa811200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa80c06f460 con 0x7fa80c0ad270 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa80a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 0x7fa80c0d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa80a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 0x7fa80c0d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34474/0 (socket says 172.21.15.71:34474) 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa80a800640 1 -- 172.21.15.71:0/4109851096 learned_addr learned my addr 172.21.15.71:0/4109851096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.073+0000 7fa809e00640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa80c0ad270 0x7fa80c1ebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa809400640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa80c0ccf00 0x7fa80c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa80a800640 1 -- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa80c0ad270 msgr2=0x7fa80c1ebf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:49.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa80a800640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa80c0ad270 0x7fa80c1ebf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa80a800640 1 -- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa80c0ccf00 msgr2=0x7fa80c0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa80a800640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa80c0ccf00 0x7fa80c0cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa80a800640 1 -- 172.21.15.71:0/4109851096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c0c9070 con 0x7fa80c15f5a0 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa80a800640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 0x7fa80c0d01f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c7a0 tx=0x7fa80400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa802000640 1 -- 172.21.15.71:0/4109851096 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804003b70 con 0x7fa80c15f5a0 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa802000640 1 -- 172.21.15.71:0/4109851096 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa804003d30 con 0x7fa80c15f5a0 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c0c9350 con 0x7fa80c15f5a0 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa802000640 1 -- 172.21.15.71:0/4109851096 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80400aab0 con 0x7fa80c15f5a0 2024-09-16T07:34:49.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.074+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c0c9910 con 0x7fa80c15f5a0 2024-09-16T07:34:49.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.076+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80c06ac10 con 0x7fa80c15f5a0 2024-09-16T07:34:49.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.077+0000 7fa802000640 1 -- 172.21.15.71:0/4109851096 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fa80400ac50 con 0x7fa80c15f5a0 2024-09-16T07:34:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.078+0000 7fa802000640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7ec07a260 0x7fa7ec07c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.078+0000 7fa802000640 1 -- 172.21.15.71:0/4109851096 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5765+0+0 (secure 0 0 0) 0x7fa804099540 con 0x7fa80c15f5a0 2024-09-16T07:34:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.078+0000 7fa809e00640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7ec07a260 0x7fa7ec07c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.079+0000 7fa809e00640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7ec07a260 0x7fa7ec07c720 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc033000 tx=0x7fa7fc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:49.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.082+0000 7fa802000640 1 -- 172.21.15.71:0/4109851096 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8040636b0 con 0x7fa80c15f5a0 2024-09-16T07:34:49.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.267+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa80c073f90 con 0x7fa80c15f5a0 2024-09-16T07:34:49.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.268+0000 7fa802000640 1 -- 172.21.15.71:0/4109851096 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7fa804062fb0 con 0x7fa80c15f5a0 2024-09-16T07:34:49.268 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:49.268 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":118,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:34:47.926470+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:34:45.706284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T07:34:46.892492+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:34:49.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7ec07a260 msgr2=0x7fa7ec07c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7ec07a260 0x7fa7ec07c720 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc033000 tx=0x7fa7fc004730 comp rx=0 tx=0).stop 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 msgr2=0x7fa80c0d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 0x7fa80c0d01f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c7a0 tx=0x7fa80400cc60 comp rx=0 tx=0).stop 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 shutdown_connections 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa7ec07a260 0x7fa7ec07c720 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa80c15f5a0 0x7fa80c0d01f0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa80c0ccf00 0x7fa80c0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 --2- 172.21.15.71:0/4109851096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa80c0ad270 0x7fa80c1ebf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 >> 172.21.15.71:0/4109851096 conn(0x7fa80c0b1950 msgr2=0x7fa80c0b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 shutdown_connections 2024-09-16T07:34:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.271+0000 7fa811200640 1 -- 172.21.15.71:0/4109851096 wait complete. 2024-09-16T07:34:49.287 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:34:49.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 -- 172.21.15.71:0/263563057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 msgr2=0x7fa35c15ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:49.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 --2- 172.21.15.71:0/263563057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 0x7fa35c15ee10 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa35c15b4d0 tx=0x7fa35402a540 comp rx=0 tx=0).stop 2024-09-16T07:34:49.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 -- 172.21.15.71:0/263563057 shutdown_connections 2024-09-16T07:34:49.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 --2- 172.21.15.71:0/263563057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 0x7fa35c15ee10 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 --2- 172.21.15.71:0/263563057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa35c15c080 0x7fa35c0cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 --2- 172.21.15.71:0/263563057 >> v2:172.21.15.177:3300/0 conn(0x7fa35c0b3370 0x7fa35c0cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 -- 172.21.15.71:0/263563057 >> 172.21.15.71:0/263563057 conn(0x7fa35c0b1840 msgr2=0x7fa35c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:49.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 -- 172.21.15.71:0/263563057 shutdown_connections 2024-09-16T07:34:49.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.447+0000 7fa360c00640 1 -- 172.21.15.71:0/263563057 wait complete. 2024-09-16T07:34:49.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.449+0000 7fa360c00640 1 Processor -- start 2024-09-16T07:34:49.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.449+0000 7fa360c00640 1 -- start start 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa360c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa35c0b3370 0x7fa35c1f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa360c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa35c15c080 0x7fa35c0b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa360c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 0x7fa35c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa360c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c140770 con 0x7fa35c15e980 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa360c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1405f0 con 0x7fa35c0b3370 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa360c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c1408f0 con 0x7fa35c15c080 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa35a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 0x7fa35c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa359e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa35c0b3370 0x7fa35c1f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa359400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa35c15c080 0x7fa35c0b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa35a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 0x7fa35c0b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34492/0 (socket says 172.21.15.71:34492) 2024-09-16T07:34:49.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.450+0000 7fa35a800640 1 -- 172.21.15.71:0/2248715354 learned_addr learned my addr 172.21.15.71:0/2248715354 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:49.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa35a800640 1 -- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa35c15c080 msgr2=0x7fa35c0b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa35a800640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa35c15c080 0x7fa35c0b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa35a800640 1 -- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa35c0b3370 msgr2=0x7fa35c1f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa35a800640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa35c0b3370 0x7fa35c1f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa35a800640 1 -- 172.21.15.71:0/2248715354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c13ce20 con 0x7fa35c15e980 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa35a800640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 0x7fa35c0b0e70 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa354034040 tx=0x7fa354003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa352000640 1 -- 172.21.15.71:0/2248715354 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35403e030 con 0x7fa35c15e980 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa352000640 1 -- 172.21.15.71:0/2248715354 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa354003e50 con 0x7fa35c15e980 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c1f8170 con 0x7fa35c15e980 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.451+0000 7fa352000640 1 -- 172.21.15.71:0/2248715354 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354035040 con 0x7fa35c15e980 2024-09-16T07:34:49.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.452+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c1f8730 con 0x7fa35c15e980 2024-09-16T07:34:49.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.452+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c15ee10 con 0x7fa35c15e980 2024-09-16T07:34:49.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.454+0000 7fa352000640 1 -- 172.21.15.71:0/2248715354 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fa354041030 con 0x7fa35c15e980 2024-09-16T07:34:49.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.455+0000 7fa352000640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa33c07a260 0x7fa33c07c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:49.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.455+0000 7fa352000640 1 -- 172.21.15.71:0/2248715354 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5765+0+0 (secure 0 0 0) 0x7fa3540b9da0 con 0x7fa35c15e980 2024-09-16T07:34:49.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.455+0000 7fa359e00640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa33c07a260 0x7fa33c07c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:49.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.456+0000 7fa359e00640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa33c07a260 0x7fa33c07c720 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005fd0 tx=0x7fa34c009680 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:49.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.459+0000 7fa352000640 1 -- 172.21.15.71:0/2248715354 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa354083820 con 0x7fa35c15e980 2024-09-16T07:34:49.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:49.686+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa35c13ffa0 con 0x7fa35c15e980 2024-09-16T07:34:50.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.533+0000 7fa352000640 1 -- 172.21.15.71:0/2248715354 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) ==== 118+0+0 (secure 0 0 0) 0x7fa354088250 con 0x7fa35c15e980 2024-09-16T07:34:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa33c07a260 msgr2=0x7fa33c07c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa33c07a260 0x7fa33c07c720 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005fd0 tx=0x7fa34c009680 comp rx=0 tx=0).stop 2024-09-16T07:34:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 msgr2=0x7fa35c0b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 0x7fa35c0b0e70 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa354034040 tx=0x7fa354003960 comp rx=0 tx=0).stop 2024-09-16T07:34:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 shutdown_connections 2024-09-16T07:34:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa33c07a260 0x7fa33c07c720 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c15e980 0x7fa35c0b0e70 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa35c15c080 0x7fa35c0b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 --2- 172.21.15.71:0/2248715354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa35c0b3370 0x7fa35c1f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 >> 172.21.15.71:0/2248715354 conn(0x7fa35c0b1840 msgr2=0x7fa35c0ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 shutdown_connections 2024-09-16T07:34:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.537+0000 7fa360c00640 1 -- 172.21.15.71:0/2248715354 wait complete. 2024-09-16T07:34:50.556 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:34:50.557 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:34:50.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.721+0000 7f2c24600640 1 -- 172.21.15.71:0/2241653779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c04003a50 msgr2=0x7f2c04003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:50.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.721+0000 7f2c24600640 1 --2- 172.21.15.71:0/2241653779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c04003a50 0x7f2c04003e70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2c100016c0 tx=0x7f2c04005830 comp rx=0 tx=0).stop 2024-09-16T07:34:50.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.722+0000 7f2c24600640 1 -- 172.21.15.71:0/2241653779 shutdown_connections 2024-09-16T07:34:50.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.722+0000 7f2c24600640 1 --2- 172.21.15.71:0/2241653779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c04003a50 0x7f2c04003e70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:50.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.722+0000 7f2c24600640 1 -- 172.21.15.71:0/2241653779 >> 172.21.15.71:0/2241653779 conn(0x7f2c1c0d0d40 msgr2=0x7f2c1c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:50.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.722+0000 7f2c24600640 1 -- 172.21.15.71:0/2241653779 shutdown_connections 2024-09-16T07:34:50.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.722+0000 7f2c24600640 1 -- 172.21.15.71:0/2241653779 wait complete. 2024-09-16T07:34:50.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.723+0000 7f2c24600640 1 Processor -- start 2024-09-16T07:34:50.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.723+0000 7f2c24600640 1 -- start start 2024-09-16T07:34:50.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c24600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c04003a50 0x7f2c1c154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c24600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c1c15b140 0x7f2c1c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c24600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c1c15d650 0x7f2c1c15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c24600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c0cc610 con 0x7f2c1c15b140 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c24600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c0cc490 con 0x7f2c04003a50 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c24600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1c0cc790 con 0x7f2c1c15d650 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c21200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c1c15b140 0x7f2c1c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c21200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c1c15b140 0x7f2c1c15a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34508/0 (socket says 172.21.15.71:34508) 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.724+0000 7f2c21200640 1 -- 172.21.15.71:0/1880714402 learned_addr learned my addr 172.21.15.71:0/1880714402 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c22600640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c1c15d650 0x7f2c1c15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c21c00640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c04003a50 0x7f2c1c154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c21200640 1 -- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c1c15d650 msgr2=0x7f2c1c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:50.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c21200640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c1c15d650 0x7f2c1c15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c21200640 1 -- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c04003a50 msgr2=0x7f2c1c154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c21200640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c04003a50 0x7f2c1c154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c21200640 1 -- 172.21.15.71:0/1880714402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c1f4960 con 0x7f2c1c15b140 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c21200640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c1c15b140 0x7f2c1c15a6d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006fa0 tx=0x7f2c080039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c19600640 1 -- 172.21.15.71:0/1880714402 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08011700 con 0x7f2c1c15b140 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c19600640 1 -- 172.21.15.71:0/1880714402 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c08016020 con 0x7f2c1c15b140 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c19600640 1 -- 172.21.15.71:0/1880714402 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08011b20 con 0x7f2c1c15b140 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.725+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c1f4c40 con 0x7f2c1c15b140 2024-09-16T07:34:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.726+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c1f5200 con 0x7f2c1c15b140 2024-09-16T07:34:50.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.727+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c0ae190 con 0x7f2c1c15b140 2024-09-16T07:34:50.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.728+0000 7f2c19600640 1 -- 172.21.15.71:0/1880714402 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f2c08011cc0 con 0x7f2c1c15b140 2024-09-16T07:34:50.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.729+0000 7f2c19600640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bfc07a260 0x7f2bfc07c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:50.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.729+0000 7f2c19600640 1 -- 172.21.15.71:0/1880714402 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f2c0809a180 con 0x7f2c1c15b140 2024-09-16T07:34:50.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.729+0000 7f2c21c00640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bfc07a260 0x7f2bfc07c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:50.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.730+0000 7f2c21c00640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bfc07a260 0x7f2bfc07c720 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2c10032000 tx=0x7f2c10001190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.733+0000 7f2c19600640 1 -- 172.21.15.71:0/1880714402 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c08020020 con 0x7f2c1c15b140 2024-09-16T07:34:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:50.956+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2c1c0b1280 con 0x7f2c1c15b140 2024-09-16T07:34:51.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.539+0000 7f2c19600640 1 -- 172.21.15.71:0/1880714402 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) ==== 122+0+0 (secure 0 0 0) 0x7f2c08064250 con 0x7f2c1c15b140 2024-09-16T07:34:51.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bfc07a260 msgr2=0x7f2bfc07c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:51.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bfc07a260 0x7f2bfc07c720 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2c10032000 tx=0x7f2c10001190 comp rx=0 tx=0).stop 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c1c15b140 msgr2=0x7f2c1c15a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c1c15b140 0x7f2c1c15a6d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006fa0 tx=0x7f2c080039c0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 shutdown_connections 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bfc07a260 0x7f2bfc07c720 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c1c15d650 0x7f2c1c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c1c15b140 0x7f2c1c15a6d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 --2- 172.21.15.71:0/1880714402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c04003a50 0x7f2c1c154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 >> 172.21.15.71:0/1880714402 conn(0x7f2c1c0d0d40 msgr2=0x7f2c1c06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 shutdown_connections 2024-09-16T07:34:51.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.542+0000 7f2c24600640 1 -- 172.21.15.71:0/1880714402 wait complete. 2024-09-16T07:34:51.557 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:34:51.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:34:51.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.716+0000 7febaea00640 1 -- 172.21.15.71:0/4015454575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba8161550 msgr2=0x7feba806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:51.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.716+0000 7febaea00640 1 --2- 172.21.15.71:0/4015454575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba8161550 0x7feba806ef50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7feba80cd5d0 tx=0x7feb9802a980 comp rx=0 tx=0).stop 2024-09-16T07:34:51.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.716+0000 7febaea00640 1 -- 172.21.15.71:0/4015454575 shutdown_connections 2024-09-16T07:34:51.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.716+0000 7febaea00640 1 --2- 172.21.15.71:0/4015454575 >> v2:172.21.15.177:3300/0 conn(0x7feba8072df0 0x7feba80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.716+0000 7febaea00640 1 --2- 172.21.15.71:0/4015454575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba8161550 0x7feba806ef50 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.716+0000 7febaea00640 1 -- 172.21.15.71:0/4015454575 >> 172.21.15.71:0/4015454575 conn(0x7feba80b1950 msgr2=0x7feba80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:51.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.716+0000 7febaea00640 1 -- 172.21.15.71:0/4015454575 shutdown_connections 2024-09-16T07:34:51.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.716+0000 7febaea00640 1 -- 172.21.15.71:0/4015454575 wait complete. 2024-09-16T07:34:51.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.718+0000 7febaea00640 1 Processor -- start 2024-09-16T07:34:51.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.718+0000 7febaea00640 1 -- start start 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7febaea00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feba8072df0 0x7feba8145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7febaea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba80af3d0 0x7feba8160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7febaea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feba8161550 0x7feba8142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7febaea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feba80737e0 con 0x7feba80af3d0 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7febaea00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feba806fdb0 con 0x7feba8161550 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7febaea00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feba8073960 con 0x7feba8072df0 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7feba7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba80af3d0 0x7feba8160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7feba7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba80af3d0 0x7feba8160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34530/0 (socket says 172.21.15.71:34530) 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7feba7400640 1 -- 172.21.15.71:0/1810419132 learned_addr learned my addr 172.21.15.71:0/1810419132 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7feba7e00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feba8072df0 0x7feba8145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:51.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.719+0000 7febaca00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feba8161550 0x7feba8142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feba7400640 1 -- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feba8072df0 msgr2=0x7feba8145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feba7400640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feba8072df0 0x7feba8145fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feba7400640 1 -- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feba8161550 msgr2=0x7feba8142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feba7400640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feba8161550 0x7feba8142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feba7400640 1 -- 172.21.15.71:0/1810419132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba81464e0 con 0x7feba80af3d0 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feba7400640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba80af3d0 0x7feba8160280 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c400 tx=0x7feb9400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feb9fe00640 1 -- 172.21.15.71:0/1810419132 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94017070 con 0x7feba80af3d0 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feb9fe00640 1 -- 172.21.15.71:0/1810419132 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb940035f0 con 0x7feba80af3d0 2024-09-16T07:34:51.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7feb9fe00640 1 -- 172.21.15.71:0/1810419132 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94003a80 con 0x7feba80af3d0 2024-09-16T07:34:51.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba8153270 con 0x7feba80af3d0 2024-09-16T07:34:51.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.720+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba809af90 con 0x7feba80af3d0 2024-09-16T07:34:51.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.721+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba8073e60 con 0x7feba80af3d0 2024-09-16T07:34:51.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.723+0000 7feb9fe00640 1 -- 172.21.15.71:0/1810419132 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7feb94003c20 con 0x7feba80af3d0 2024-09-16T07:34:51.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.723+0000 7feb9fe00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8807a260 0x7feb8807c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:51.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.724+0000 7feb9fe00640 1 -- 172.21.15.71:0/1810419132 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7feb940989a0 con 0x7feba80af3d0 2024-09-16T07:34:51.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.724+0000 7feba7e00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8807a260 0x7feb8807c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:51.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.725+0000 7feba7e00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8807a260 0x7feb8807c720 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feb98035000 tx=0x7feb98004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:51.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.728+0000 7feb9fe00640 1 -- 172.21.15.71:0/1810419132 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feb94062420 con 0x7feba80af3d0 2024-09-16T07:34:51.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.946+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feba80b2430 con 0x7feba80af3d0 2024-09-16T07:34:51.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.946+0000 7feb9fe00640 1 -- 172.21.15.71:0/1810419132 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5231 (secure 0 0 0) 0x7feb94066e50 con 0x7feba80af3d0 2024-09-16T07:34:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:51.946 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":75,"btime":"2024-09-16T07:34:51:534556+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1994777108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1994777108},{"type":"v1","addr":"172.21.15.71:6833","nonce":1994777108}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1399666780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1399666780},{"type":"v1","addr":"172.21.15.177:6836","nonce":1399666780}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2921631965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2921631965},{"type":"v1","addr":"172.21.15.71:6835","nonce":2921631965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:34:47.926434+0000","modified":"2024-09-16T07:34:50.958041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2539335973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2539335973},{"type":"v1","addr":"172.21.15.177:6837","nonce":2539335973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8382,"qdb_cluster":[8382]},"id":8}]} 2024-09-16T07:34:51.946 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 75 2024-09-16T07:34:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8807a260 msgr2=0x7feb8807c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8807a260 0x7feb8807c720 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feb98035000 tx=0x7feb98004ab0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba80af3d0 msgr2=0x7feba8160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba80af3d0 0x7feba8160280 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c400 tx=0x7feb9400c8c0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 shutdown_connections 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7feb8807a260 0x7feb8807c720 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feba8161550 0x7feba8142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba80af3d0 0x7feba8160280 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 --2- 172.21.15.71:0/1810419132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feba8072df0 0x7feba8145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 >> 172.21.15.71:0/1810419132 conn(0x7feba80b1950 msgr2=0x7feba815ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 shutdown_connections 2024-09-16T07:34:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:51.949+0000 7febaea00640 1 -- 172.21.15.71:0/1810419132 wait complete. 2024-09-16T07:34:51.964 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:34:52.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 -- 172.21.15.71:0/1536534198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf815f5a0 msgr2=0x7f3cf80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 --2- 172.21.15.71:0/1536534198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf815f5a0 0x7f3cf80b2440 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8076720 tx=0x7f3ce802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 -- 172.21.15.71:0/1536534198 shutdown_connections 2024-09-16T07:34:52.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 --2- 172.21.15.71:0/1536534198 >> v2:172.21.15.177:3300/0 conn(0x7f3cf80ad270 0x7f3cf8073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 --2- 172.21.15.71:0/1536534198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cf80ccf00 0x7f3cf80acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 --2- 172.21.15.71:0/1536534198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf815f5a0 0x7f3cf80b2440 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 -- 172.21.15.71:0/1536534198 >> 172.21.15.71:0/1536534198 conn(0x7f3cf80b1950 msgr2=0x7f3cf80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:52.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 -- 172.21.15.71:0/1536534198 shutdown_connections 2024-09-16T07:34:52.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.121+0000 7f3cfda00640 1 -- 172.21.15.71:0/1536534198 wait complete. 2024-09-16T07:34:52.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.123+0000 7f3cfda00640 1 Processor -- start 2024-09-16T07:34:52.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.123+0000 7f3cfda00640 1 -- start start 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cfda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf80ad270 0x7f3cf814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cfda00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cf80ccf00 0x7f3cf81f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cfda00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cf815f5a0 0x7f3cf80cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cfda00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf806f160 con 0x7f3cf80ad270 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cfda00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8074780 con 0x7f3cf80ccf00 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cfda00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf806f2e0 con 0x7f3cf815f5a0 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf80ad270 0x7f3cf814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf80ad270 0x7f3cf814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34544/0 (socket says 172.21.15.71:34544) 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6a00640 1 -- 172.21.15.71:0/927323035 learned_addr learned my addr 172.21.15.71:0/927323035 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:52.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6000640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cf80ccf00 0x7f3cf81f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf7400640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cf815f5a0 0x7f3cf80cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6a00640 1 -- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cf815f5a0 msgr2=0x7f3cf80cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6a00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cf815f5a0 0x7f3cf80cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6a00640 1 -- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cf80ccf00 msgr2=0x7f3cf81f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6a00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cf80ccf00 0x7f3cf81f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf6a00640 1 -- 172.21.15.71:0/927323035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf81f9390 con 0x7f3cf80ad270 2024-09-16T07:34:52.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.124+0000 7f3cf7400640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cf815f5a0 0x7f3cf80cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:52.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.125+0000 7f3cf6a00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf80ad270 0x7f3cf814e900 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80099c0 tx=0x7f3ce80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:52.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.125+0000 7f3ceea00640 1 -- 172.21.15.71:0/927323035 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce803e030 con 0x7f3cf80ad270 2024-09-16T07:34:52.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.125+0000 7f3ceea00640 1 -- 172.21.15.71:0/927323035 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ce80047a0 con 0x7f3cf80ad270 2024-09-16T07:34:52.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.125+0000 7f3ceea00640 1 -- 172.21.15.71:0/927323035 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8035040 con 0x7f3cf80ad270 2024-09-16T07:34:52.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.125+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf8156f60 con 0x7f3cf80ad270 2024-09-16T07:34:52.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.125+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf80c9500 con 0x7f3cf80ad270 2024-09-16T07:34:52.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.126+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf806f870 con 0x7f3cf80ad270 2024-09-16T07:34:52.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.127+0000 7f3ceea00640 1 -- 172.21.15.71:0/927323035 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f3ce8041030 con 0x7f3cf80ad270 2024-09-16T07:34:52.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.128+0000 7f3ceea00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cd807a230 0x7f3cd807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.128+0000 7f3ceea00640 1 -- 172.21.15.71:0/927323035 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f3ce80b9da0 con 0x7f3cf80ad270 2024-09-16T07:34:52.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.129+0000 7f3cf6000640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cd807a230 0x7f3cd807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.130+0000 7f3cf6000640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cd807a230 0x7f3cd807c6f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3cf81f5fa0 tx=0x7f3ce400b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:52.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.132+0000 7f3ceea00640 1 -- 172.21.15.71:0/927323035 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3cf81f9390 con 0x7f3cf80ad270 2024-09-16T07:34:52.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.319+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cf809aac0 con 0x7f3cf80ad270 2024-09-16T07:34:52.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.320+0000 7f3ceea00640 1 -- 172.21.15.71:0/927323035 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16843 (secure 0 0 0) 0x7f3ce8083820 con 0x7f3cf80ad270 2024-09-16T07:34:52.319 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:52.320 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":119,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:34:49.518900+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:34:45.706284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T07:34:46.892492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:34:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cd807a230 msgr2=0x7f3cd807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cd807a230 0x7f3cd807c6f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3cf81f5fa0 tx=0x7f3ce400b390 comp rx=0 tx=0).stop 2024-09-16T07:34:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf80ad270 msgr2=0x7f3cf814e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf80ad270 0x7f3cf814e900 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80099c0 tx=0x7f3ce80039e0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 shutdown_connections 2024-09-16T07:34:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3cd807a230 0x7f3cd807c6f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cf815f5a0 0x7f3cf80cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cf80ccf00 0x7f3cf81f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 --2- 172.21.15.71:0/927323035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cf80ad270 0x7f3cf814e900 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.322+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 >> 172.21.15.71:0/927323035 conn(0x7f3cf80b1950 msgr2=0x7f3cf809cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.323+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 shutdown_connections 2024-09-16T07:34:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.323+0000 7f3cfda00640 1 -- 172.21.15.71:0/927323035 wait complete. 2024-09-16T07:34:52.338 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:34:52.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:34:52.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.496+0000 7fa90d400640 1 -- 172.21.15.71:0/492707056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8ec003a80 msgr2=0x7fa8ec003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.496+0000 7fa90d400640 1 --2- 172.21.15.71:0/492707056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8ec003a80 0x7fa8ec003ea0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009b50 tx=0x7fa8ec005830 comp rx=0 tx=0).stop 2024-09-16T07:34:52.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.496+0000 7fa907200640 1 -- 172.21.15.71:0/492707056 reap_dead start 2024-09-16T07:34:52.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.496+0000 7fa90d400640 1 -- 172.21.15.71:0/492707056 shutdown_connections 2024-09-16T07:34:52.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.496+0000 7fa90d400640 1 -- 172.21.15.71:0/492707056 >> 172.21.15.71:0/492707056 conn(0x7fa9080b1950 msgr2=0x7fa9080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:52.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.496+0000 7fa90d400640 1 -- 172.21.15.71:0/492707056 shutdown_connections 2024-09-16T07:34:52.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.496+0000 7fa90d400640 1 -- 172.21.15.71:0/492707056 wait complete. 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.497+0000 7fa90d400640 1 Processor -- start 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.498+0000 7fa90d400640 1 -- start start 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.498+0000 7fa90d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8ec003a80 0x7fa9081514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa90d400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa908159900 0x7fa90814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa90d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9080b3d20 0x7fa9080b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa90d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa90806ef30 con 0x7fa9080b3d20 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa90d400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa90806fa80 con 0x7fa8ec003a80 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa90d400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa9080b2440 con 0x7fa908159900 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9080b3d20 0x7fa9080b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9080b3d20 0x7fa9080b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34566/0 (socket says 172.21.15.71:34566) 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 -- 172.21.15.71:0/2616003967 learned_addr learned my addr 172.21.15.71:0/2616003967 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa906800640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8ec003a80 0x7fa9081514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa8ffe00640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa908159900 0x7fa90814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 -- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa908159900 msgr2=0x7fa90814f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa908159900 0x7fa90814f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 -- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8ec003a80 msgr2=0x7fa9081514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8ec003a80 0x7fa9081514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 -- 172.21.15.71:0/2616003967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90815c4c0 con 0x7fa9080b3d20 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.499+0000 7fa907200640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9080b3d20 0x7fa9080b41d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa9080b5080 tx=0x7fa90000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.500+0000 7fa8fea00640 1 -- 172.21.15.71:0/2616003967 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90000cc20 con 0x7fa9080b3d20 2024-09-16T07:34:52.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.500+0000 7fa8fea00640 1 -- 172.21.15.71:0/2616003967 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa900018070 con 0x7fa9080b3d20 2024-09-16T07:34:52.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.500+0000 7fa8fea00640 1 -- 172.21.15.71:0/2616003967 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000116c0 con 0x7fa9080b3d20 2024-09-16T07:34:52.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.500+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90815c7a0 con 0x7fa9080b3d20 2024-09-16T07:34:52.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.500+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90815ca80 con 0x7fa9080b3d20 2024-09-16T07:34:52.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.501+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90815e9d0 con 0x7fa9080b3d20 2024-09-16T07:34:52.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.502+0000 7fa8fea00640 1 -- 172.21.15.71:0/2616003967 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fa900011860 con 0x7fa9080b3d20 2024-09-16T07:34:52.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.503+0000 7fa8fea00640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e407a260 0x7fa8e407c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.503+0000 7fa8fea00640 1 -- 172.21.15.71:0/2616003967 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fa900099c60 con 0x7fa9080b3d20 2024-09-16T07:34:52.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.503+0000 7fa906800640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e407a260 0x7fa8e407c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.504+0000 7fa906800640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e407a260 0x7fa8e407c720 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8035000 tx=0x7fa8f802d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:52.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.507+0000 7fa8fea00640 1 -- 172.21.15.71:0/2616003967 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa900063d30 con 0x7fa9080b3d20 2024-09-16T07:34:52.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.691+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa90809dcf0 con 0x7fa8e407a260 2024-09-16T07:34:52.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.697+0000 7fa8fea00640 1 -- 172.21.15.71:0/2616003967 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250404 (secure 0 0 0) 0x7fa90809dcf0 con 0x7fa8e407a260 2024-09-16T07:34:52.701 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:52.702 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:34:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e407a260 msgr2=0x7fa8e407c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e407a260 0x7fa8e407c720 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8035000 tx=0x7fa8f802d040 comp rx=0 tx=0).stop 2024-09-16T07:34:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9080b3d20 msgr2=0x7fa9080b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9080b3d20 0x7fa9080b41d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa9080b5080 tx=0x7fa90000ed90 comp rx=0 tx=0).stop 2024-09-16T07:34:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 shutdown_connections 2024-09-16T07:34:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8e407a260 0x7fa8e407c720 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9080b3d20 0x7fa9080b41d0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa908159900 0x7fa90814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 --2- 172.21.15.71:0/2616003967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8ec003a80 0x7fa9081514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.701+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 >> 172.21.15.71:0/2616003967 conn(0x7fa9080b1950 msgr2=0x7fa9080acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.702+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 shutdown_connections 2024-09-16T07:34:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.702+0000 7fa90d400640 1 -- 172.21.15.71:0/2616003967 wait complete. 2024-09-16T07:34:52.716 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-16T07:34:52.604187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592632,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1052772,"kb_used_data":12464,"kb_used_omap":12,"kb_used_meta":1040115,"kb_avail":753921948,"statfs":{"total":773094113280,"available":772016074752,"internally_reserved":0,"allocated":12763136,"data_stored":6223997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1065078320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2264,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001350"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.519868+0000","last_change":"2024-09-16T07:34:47.519868+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.519868+0000","last_undegraded":"2024-09-16T07:34:47.519868+0000","last_fullsized":"2024-09-16T07:34:47.519868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.542589+0000","last_change":"2024-09-16T07:34:48.541678+0000","last_active":"2024-09-16T07:34:48.542589+0000","last_peered":"2024-09-16T07:34:48.542589+0000","last_clean":"2024-09-16T07:34:48.542589+0000","last_became_active":"2024-09-16T07:34:48.541360+0000","last_became_peered":"2024-09-16T07:34:48.541360+0000","last_unstale":"2024-09-16T07:34:48.542589+0000","last_undegraded":"2024-09-16T07:34:48.542589+0000","last_fullsized":"2024-09-16T07:34:48.542589+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:42.661091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535282+0000","last_change":"2024-09-16T07:34:48.534129+0000","last_active":"2024-09-16T07:34:48.535282+0000","last_peered":"2024-09-16T07:34:48.535282+0000","last_clean":"2024-09-16T07:34:48.535282+0000","last_became_active":"2024-09-16T07:34:48.533584+0000","last_became_peered":"2024-09-16T07:34:48.533584+0000","last_unstale":"2024-09-16T07:34:48.535282+0000","last_undegraded":"2024-09-16T07:34:48.535282+0000","last_fullsized":"2024-09-16T07:34:48.535282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:57.899237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531563+0000","last_change":"2024-09-16T07:34:48.526281+0000","last_active":"2024-09-16T07:34:49.531563+0000","last_peered":"2024-09-16T07:34:49.531563+0000","last_clean":"2024-09-16T07:34:49.531563+0000","last_became_active":"2024-09-16T07:34:48.525843+0000","last_became_peered":"2024-09-16T07:34:48.525843+0000","last_unstale":"2024-09-16T07:34:49.531563+0000","last_undegraded":"2024-09-16T07:34:49.531563+0000","last_fullsized":"2024-09-16T07:34:49.531563+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:50.039555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513726+0000","last_change":"2024-09-16T07:34:47.513726+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513726+0000","last_undegraded":"2024-09-16T07:34:47.513726+0000","last_fullsized":"2024-09-16T07:34:47.513726+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.507372+0000","last_change":"2024-09-16T07:34:47.507372+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.507372+0000","last_undegraded":"2024-09-16T07:34:47.507372+0000","last_fullsized":"2024-09-16T07:34:47.507372+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534985+0000","last_change":"2024-09-16T07:34:48.526151+0000","last_active":"2024-09-16T07:34:48.534985+0000","last_peered":"2024-09-16T07:34:48.534985+0000","last_clean":"2024-09-16T07:34:48.534985+0000","last_became_active":"2024-09-16T07:34:48.525914+0000","last_became_peered":"2024-09-16T07:34:48.525914+0000","last_unstale":"2024-09-16T07:34:48.534985+0000","last_undegraded":"2024-09-16T07:34:48.534985+0000","last_fullsized":"2024-09-16T07:34:48.534985+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:31.962809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535012+0000","last_change":"2024-09-16T07:34:48.528915+0000","last_active":"2024-09-16T07:34:48.535012+0000","last_peered":"2024-09-16T07:34:48.535012+0000","last_clean":"2024-09-16T07:34:48.535012+0000","last_became_active":"2024-09-16T07:34:48.528648+0000","last_became_peered":"2024-09-16T07:34:48.528648+0000","last_unstale":"2024-09-16T07:34:48.535012+0000","last_undegraded":"2024-09-16T07:34:48.535012+0000","last_fullsized":"2024-09-16T07:34:48.535012+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:49.801672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.519440+0000","last_change":"2024-09-16T07:34:47.519440+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.519440+0000","last_undegraded":"2024-09-16T07:34:47.519440+0000","last_fullsized":"2024-09-16T07:34:47.519440+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.514995+0000","last_change":"2024-09-16T07:34:47.514995+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.514995+0000","last_undegraded":"2024-09-16T07:34:47.514995+0000","last_fullsized":"2024-09-16T07:34:47.514995+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534388+0000","last_change":"2024-09-16T07:34:48.530703+0000","last_active":"2024-09-16T07:34:48.534388+0000","last_peered":"2024-09-16T07:34:48.534388+0000","last_clean":"2024-09-16T07:34:48.534388+0000","last_became_active":"2024-09-16T07:34:48.530406+0000","last_became_peered":"2024-09-16T07:34:48.530406+0000","last_unstale":"2024-09-16T07:34:48.534388+0000","last_undegraded":"2024-09-16T07:34:48.534388+0000","last_fullsized":"2024-09-16T07:34:48.534388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:24.273074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.515387+0000","last_change":"2024-09-16T07:34:47.515387+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.515387+0000","last_undegraded":"2024-09-16T07:34:47.515387+0000","last_fullsized":"2024-09-16T07:34:47.515387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.538475+0000","last_change":"2024-09-16T07:34:48.535432+0000","last_active":"2024-09-16T07:34:48.538475+0000","last_peered":"2024-09-16T07:34:48.538475+0000","last_clean":"2024-09-16T07:34:48.538475+0000","last_became_active":"2024-09-16T07:34:48.532194+0000","last_became_peered":"2024-09-16T07:34:48.532194+0000","last_unstale":"2024-09-16T07:34:48.538475+0000","last_undegraded":"2024-09-16T07:34:48.538475+0000","last_fullsized":"2024-09-16T07:34:48.538475+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:02:06.782991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535669+0000","last_change":"2024-09-16T07:34:48.531769+0000","last_active":"2024-09-16T07:34:48.535669+0000","last_peered":"2024-09-16T07:34:48.535669+0000","last_clean":"2024-09-16T07:34:48.535669+0000","last_became_active":"2024-09-16T07:34:48.531251+0000","last_became_peered":"2024-09-16T07:34:48.531251+0000","last_unstale":"2024-09-16T07:34:48.535669+0000","last_undegraded":"2024-09-16T07:34:48.535669+0000","last_fullsized":"2024-09-16T07:34:48.535669+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:03.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513794+0000","last_change":"2024-09-16T07:34:47.513794+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513794+0000","last_undegraded":"2024-09-16T07:34:47.513794+0000","last_fullsized":"2024-09-16T07:34:47.513794+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516736+0000","last_change":"2024-09-16T07:34:47.516736+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516736+0000","last_undegraded":"2024-09-16T07:34:47.516736+0000","last_fullsized":"2024-09-16T07:34:47.516736+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.522422+0000","last_change":"2024-09-16T07:34:48.520786+0000","last_active":"2024-09-16T07:34:48.522422+0000","last_peered":"2024-09-16T07:34:48.522422+0000","last_clean":"2024-09-16T07:34:48.522422+0000","last_became_active":"2024-09-16T07:34:48.519475+0000","last_became_peered":"2024-09-16T07:34:48.519475+0000","last_unstale":"2024-09-16T07:34:48.522422+0000","last_undegraded":"2024-09-16T07:34:48.522422+0000","last_fullsized":"2024-09-16T07:34:48.522422+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:38.065118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535936+0000","last_change":"2024-09-16T07:34:48.527942+0000","last_active":"2024-09-16T07:34:48.535936+0000","last_peered":"2024-09-16T07:34:48.535936+0000","last_clean":"2024-09-16T07:34:48.535936+0000","last_became_active":"2024-09-16T07:34:48.527669+0000","last_became_peered":"2024-09-16T07:34:48.527669+0000","last_unstale":"2024-09-16T07:34:48.535936+0000","last_undegraded":"2024-09-16T07:34:48.535936+0000","last_fullsized":"2024-09-16T07:34:48.535936+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:56.714125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535262+0000","last_change":"2024-09-16T07:34:48.534101+0000","last_active":"2024-09-16T07:34:48.535262+0000","last_peered":"2024-09-16T07:34:48.535262+0000","last_clean":"2024-09-16T07:34:48.535262+0000","last_became_active":"2024-09-16T07:34:48.529574+0000","last_became_peered":"2024-09-16T07:34:48.529574+0000","last_unstale":"2024-09-16T07:34:48.535262+0000","last_undegraded":"2024-09-16T07:34:48.535262+0000","last_fullsized":"2024-09-16T07:34:48.535262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:30:49.545255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533243+0000","last_change":"2024-09-16T07:34:48.529587+0000","last_active":"2024-09-16T07:34:48.533243+0000","last_peered":"2024-09-16T07:34:48.533243+0000","last_clean":"2024-09-16T07:34:48.533243+0000","last_became_active":"2024-09-16T07:34:48.529304+0000","last_became_peered":"2024-09-16T07:34:48.529304+0000","last_unstale":"2024-09-16T07:34:48.533243+0000","last_undegraded":"2024-09-16T07:34:48.533243+0000","last_fullsized":"2024-09-16T07:34:48.533243+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:06:46.119567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534304+0000","last_change":"2024-09-16T07:34:48.533118+0000","last_active":"2024-09-16T07:34:48.534304+0000","last_peered":"2024-09-16T07:34:48.534304+0000","last_clean":"2024-09-16T07:34:48.534304+0000","last_became_active":"2024-09-16T07:34:48.532777+0000","last_became_peered":"2024-09-16T07:34:48.532777+0000","last_unstale":"2024-09-16T07:34:48.534304+0000","last_undegraded":"2024-09-16T07:34:48.534304+0000","last_fullsized":"2024-09-16T07:34:48.534304+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:02.406116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505332+0000","last_change":"2024-09-16T07:34:47.505332+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505332+0000","last_undegraded":"2024-09-16T07:34:47.505332+0000","last_fullsized":"2024-09-16T07:34:47.505332+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535054+0000","last_change":"2024-09-16T07:34:48.528984+0000","last_active":"2024-09-16T07:34:48.535054+0000","last_peered":"2024-09-16T07:34:48.535054+0000","last_clean":"2024-09-16T07:34:48.535054+0000","last_became_active":"2024-09-16T07:34:48.528481+0000","last_became_peered":"2024-09-16T07:34:48.528481+0000","last_unstale":"2024-09-16T07:34:48.535054+0000","last_undegraded":"2024-09-16T07:34:48.535054+0000","last_fullsized":"2024-09-16T07:34:48.535054+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:43.572025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530706+0000","last_change":"2024-09-16T07:34:48.533570+0000","last_active":"2024-09-16T07:34:49.530706+0000","last_peered":"2024-09-16T07:34:49.530706+0000","last_clean":"2024-09-16T07:34:49.530706+0000","last_became_active":"2024-09-16T07:34:48.533039+0000","last_became_peered":"2024-09-16T07:34:48.533039+0000","last_unstale":"2024-09-16T07:34:49.530706+0000","last_undegraded":"2024-09-16T07:34:49.530706+0000","last_fullsized":"2024-09-16T07:34:49.530706+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:00:09.699334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533344+0000","last_change":"2024-09-16T07:34:48.527254+0000","last_active":"2024-09-16T07:34:48.533344+0000","last_peered":"2024-09-16T07:34:48.533344+0000","last_clean":"2024-09-16T07:34:48.533344+0000","last_became_active":"2024-09-16T07:34:48.527013+0000","last_became_peered":"2024-09-16T07:34:48.527013+0000","last_unstale":"2024-09-16T07:34:48.533344+0000","last_undegraded":"2024-09-16T07:34:48.533344+0000","last_fullsized":"2024-09-16T07:34:48.533344+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:16.029488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505317+0000","last_change":"2024-09-16T07:34:47.505317+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505317+0000","last_undegraded":"2024-09-16T07:34:47.505317+0000","last_fullsized":"2024-09-16T07:34:47.505317+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531251+0000","last_change":"2024-09-16T07:34:48.541120+0000","last_active":"2024-09-16T07:34:49.531251+0000","last_peered":"2024-09-16T07:34:49.531251+0000","last_clean":"2024-09-16T07:34:49.531251+0000","last_became_active":"2024-09-16T07:34:48.540074+0000","last_became_peered":"2024-09-16T07:34:48.540074+0000","last_unstale":"2024-09-16T07:34:49.531251+0000","last_undegraded":"2024-09-16T07:34:49.531251+0000","last_fullsized":"2024-09-16T07:34:49.531251+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:53.929219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.507462+0000","last_change":"2024-09-16T07:34:47.507462+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.507462+0000","last_undegraded":"2024-09-16T07:34:47.507462+0000","last_fullsized":"2024-09-16T07:34:47.507462+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535813+0000","last_change":"2024-09-16T07:34:48.534695+0000","last_active":"2024-09-16T07:34:48.535813+0000","last_peered":"2024-09-16T07:34:48.535813+0000","last_clean":"2024-09-16T07:34:48.535813+0000","last_became_active":"2024-09-16T07:34:48.534435+0000","last_became_peered":"2024-09-16T07:34:48.534435+0000","last_unstale":"2024-09-16T07:34:48.535813+0000","last_undegraded":"2024-09-16T07:34:48.535813+0000","last_fullsized":"2024-09-16T07:34:48.535813+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:11.262607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534350+0000","last_change":"2024-09-16T07:34:48.525252+0000","last_active":"2024-09-16T07:34:48.534350+0000","last_peered":"2024-09-16T07:34:48.534350+0000","last_clean":"2024-09-16T07:34:48.534350+0000","last_became_active":"2024-09-16T07:34:48.524453+0000","last_became_peered":"2024-09-16T07:34:48.524453+0000","last_unstale":"2024-09-16T07:34:48.534350+0000","last_undegraded":"2024-09-16T07:34:48.534350+0000","last_fullsized":"2024-09-16T07:34:48.534350+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:18.325757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513371+0000","last_change":"2024-09-16T07:34:47.513371+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513371+0000","last_undegraded":"2024-09-16T07:34:47.513371+0000","last_fullsized":"2024-09-16T07:34:47.513371+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516259+0000","last_change":"2024-09-16T07:34:47.516259+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516259+0000","last_undegraded":"2024-09-16T07:34:47.516259+0000","last_fullsized":"2024-09-16T07:34:47.516259+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513227+0000","last_change":"2024-09-16T07:34:47.513227+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513227+0000","last_undegraded":"2024-09-16T07:34:47.513227+0000","last_fullsized":"2024-09-16T07:34:47.513227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"118'4","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.537397+0000","last_change":"2024-09-16T07:34:47.528565+0000","last_active":"2024-09-16T07:34:48.537397+0000","last_peered":"2024-09-16T07:34:48.537397+0000","last_clean":"2024-09-16T07:34:48.537397+0000","last_became_active":"2024-09-16T07:34:47.528270+0000","last_became_peered":"2024-09-16T07:34:47.528270+0000","last_unstale":"2024-09-16T07:34:48.537397+0000","last_undegraded":"2024-09-16T07:34:48.537397+0000","last_fullsized":"2024-09-16T07:34:48.537397+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:27.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535195+0000","last_change":"2024-09-16T07:34:48.529917+0000","last_active":"2024-09-16T07:34:48.535195+0000","last_peered":"2024-09-16T07:34:48.535195+0000","last_clean":"2024-09-16T07:34:48.535195+0000","last_became_active":"2024-09-16T07:34:48.529631+0000","last_became_peered":"2024-09-16T07:34:48.529631+0000","last_unstale":"2024-09-16T07:34:48.535195+0000","last_undegraded":"2024-09-16T07:34:48.535195+0000","last_fullsized":"2024-09-16T07:34:48.535195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:13.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.537388+0000","last_change":"2024-09-16T07:34:47.526481+0000","last_active":"2024-09-16T07:34:48.537388+0000","last_peered":"2024-09-16T07:34:48.537388+0000","last_clean":"2024-09-16T07:34:48.537388+0000","last_became_active":"2024-09-16T07:34:47.526215+0000","last_became_peered":"2024-09-16T07:34:47.526215+0000","last_unstale":"2024-09-16T07:34:48.537388+0000","last_undegraded":"2024-09-16T07:34:48.537388+0000","last_fullsized":"2024-09-16T07:34:48.537388+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:08.617410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.506925+0000","last_change":"2024-09-16T07:34:47.506925+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.506925+0000","last_undegraded":"2024-09-16T07:34:47.506925+0000","last_fullsized":"2024-09-16T07:34:47.506925+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531333+0000","last_change":"2024-09-16T07:34:47.522199+0000","last_active":"2024-09-16T07:34:49.531333+0000","last_peered":"2024-09-16T07:34:49.531333+0000","last_clean":"2024-09-16T07:34:49.531333+0000","last_became_active":"2024-09-16T07:34:47.521943+0000","last_became_peered":"2024-09-16T07:34:47.521943+0000","last_unstale":"2024-09-16T07:34:49.531333+0000","last_undegraded":"2024-09-16T07:34:49.531333+0000","last_fullsized":"2024-09-16T07:34:49.531333+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:08.548657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535239+0000","last_change":"2024-09-16T07:34:48.533772+0000","last_active":"2024-09-16T07:34:48.535239+0000","last_peered":"2024-09-16T07:34:48.535239+0000","last_clean":"2024-09-16T07:34:48.535239+0000","last_became_active":"2024-09-16T07:34:48.529223+0000","last_became_peered":"2024-09-16T07:34:48.529223+0000","last_unstale":"2024-09-16T07:34:48.535239+0000","last_undegraded":"2024-09-16T07:34:48.535239+0000","last_fullsized":"2024-09-16T07:34:48.535239+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:56.315624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531454+0000","last_change":"2024-09-16T07:34:47.523452+0000","last_active":"2024-09-16T07:34:49.531454+0000","last_peered":"2024-09-16T07:34:49.531454+0000","last_clean":"2024-09-16T07:34:49.531454+0000","last_became_active":"2024-09-16T07:34:47.521451+0000","last_became_peered":"2024-09-16T07:34:47.521451+0000","last_unstale":"2024-09-16T07:34:49.531454+0000","last_undegraded":"2024-09-16T07:34:49.531454+0000","last_fullsized":"2024-09-16T07:34:49.531454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.908074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531052+0000","last_change":"2024-09-16T07:34:48.538587+0000","last_active":"2024-09-16T07:34:49.531052+0000","last_peered":"2024-09-16T07:34:49.531052+0000","last_clean":"2024-09-16T07:34:49.531052+0000","last_became_active":"2024-09-16T07:34:48.538287+0000","last_became_peered":"2024-09-16T07:34:48.538287+0000","last_unstale":"2024-09-16T07:34:49.531052+0000","last_undegraded":"2024-09-16T07:34:49.531052+0000","last_fullsized":"2024-09-16T07:34:49.531052+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:22.418271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.539626+0000","last_change":"2024-09-16T07:34:47.520652+0000","last_active":"2024-09-16T07:34:48.539626+0000","last_peered":"2024-09-16T07:34:48.539626+0000","last_clean":"2024-09-16T07:34:48.539626+0000","last_became_active":"2024-09-16T07:34:47.520279+0000","last_became_peered":"2024-09-16T07:34:47.520279+0000","last_unstale":"2024-09-16T07:34:48.539626+0000","last_undegraded":"2024-09-16T07:34:48.539626+0000","last_fullsized":"2024-09-16T07:34:48.539626+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:17.809818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534281+0000","last_change":"2024-09-16T07:34:48.532853+0000","last_active":"2024-09-16T07:34:48.534281+0000","last_peered":"2024-09-16T07:34:48.534281+0000","last_clean":"2024-09-16T07:34:48.534281+0000","last_became_active":"2024-09-16T07:34:48.532447+0000","last_became_peered":"2024-09-16T07:34:48.532447+0000","last_unstale":"2024-09-16T07:34:48.534281+0000","last_undegraded":"2024-09-16T07:34:48.534281+0000","last_fullsized":"2024-09-16T07:34:48.534281+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:43.058025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.528166+0000","last_change":"2024-09-16T07:34:47.527304+0000","last_active":"2024-09-16T07:34:47.528166+0000","last_peered":"2024-09-16T07:34:47.528166+0000","last_clean":"2024-09-16T07:34:47.528166+0000","last_became_active":"2024-09-16T07:34:47.526260+0000","last_became_peered":"2024-09-16T07:34:47.526260+0000","last_unstale":"2024-09-16T07:34:47.528166+0000","last_undegraded":"2024-09-16T07:34:47.528166+0000","last_fullsized":"2024-09-16T07:34:47.528166+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:10.669755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.515707+0000","last_change":"2024-09-16T07:34:47.515707+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.515707+0000","last_undegraded":"2024-09-16T07:34:47.515707+0000","last_fullsized":"2024-09-16T07:34:47.515707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.517911+0000","last_change":"2024-09-16T07:34:47.520199+0000","last_active":"2024-09-16T07:34:48.517911+0000","last_peered":"2024-09-16T07:34:48.517911+0000","last_clean":"2024-09-16T07:34:48.517911+0000","last_became_active":"2024-09-16T07:34:47.519941+0000","last_became_peered":"2024-09-16T07:34:47.519941+0000","last_unstale":"2024-09-16T07:34:48.517911+0000","last_undegraded":"2024-09-16T07:34:48.517911+0000","last_fullsized":"2024-09-16T07:34:48.517911+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:54.626775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.538297+0000","last_change":"2024-09-16T07:34:48.534789+0000","last_active":"2024-09-16T07:34:48.538297+0000","last_peered":"2024-09-16T07:34:48.538297+0000","last_clean":"2024-09-16T07:34:48.538297+0000","last_became_active":"2024-09-16T07:34:48.531669+0000","last_became_peered":"2024-09-16T07:34:48.531669+0000","last_unstale":"2024-09-16T07:34:48.538297+0000","last_undegraded":"2024-09-16T07:34:48.538297+0000","last_fullsized":"2024-09-16T07:34:48.538297+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:27.517094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.528190+0000","last_change":"2024-09-16T07:34:47.527047+0000","last_active":"2024-09-16T07:34:47.528190+0000","last_peered":"2024-09-16T07:34:47.528190+0000","last_clean":"2024-09-16T07:34:47.528190+0000","last_became_active":"2024-09-16T07:34:47.526733+0000","last_became_peered":"2024-09-16T07:34:47.526733+0000","last_unstale":"2024-09-16T07:34:47.528190+0000","last_undegraded":"2024-09-16T07:34:47.528190+0000","last_fullsized":"2024-09-16T07:34:47.528190+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.388814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534332+0000","last_change":"2024-09-16T07:34:48.530402+0000","last_active":"2024-09-16T07:34:48.534332+0000","last_peered":"2024-09-16T07:34:48.534332+0000","last_clean":"2024-09-16T07:34:48.534332+0000","last_became_active":"2024-09-16T07:34:48.530123+0000","last_became_peered":"2024-09-16T07:34:48.530123+0000","last_unstale":"2024-09-16T07:34:48.534332+0000","last_undegraded":"2024-09-16T07:34:48.534332+0000","last_fullsized":"2024-09-16T07:34:48.534332+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:02.344047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531255+0000","last_change":"2024-09-16T07:34:47.521688+0000","last_active":"2024-09-16T07:34:49.531255+0000","last_peered":"2024-09-16T07:34:49.531255+0000","last_clean":"2024-09-16T07:34:49.531255+0000","last_became_active":"2024-09-16T07:34:47.521350+0000","last_became_peered":"2024-09-16T07:34:47.521350+0000","last_unstale":"2024-09-16T07:34:49.531255+0000","last_undegraded":"2024-09-16T07:34:49.531255+0000","last_fullsized":"2024-09-16T07:34:49.531255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:10.584170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.536928+0000","last_change":"2024-09-16T07:34:48.532035+0000","last_active":"2024-09-16T07:34:48.536928+0000","last_peered":"2024-09-16T07:34:48.536928+0000","last_clean":"2024-09-16T07:34:48.536928+0000","last_became_active":"2024-09-16T07:34:48.527917+0000","last_became_peered":"2024-09-16T07:34:48.527917+0000","last_unstale":"2024-09-16T07:34:48.536928+0000","last_undegraded":"2024-09-16T07:34:48.536928+0000","last_fullsized":"2024-09-16T07:34:48.536928+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:14.371171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530405+0000","last_change":"2024-09-16T07:34:47.529347+0000","last_active":"2024-09-16T07:34:47.530405+0000","last_peered":"2024-09-16T07:34:47.530405+0000","last_clean":"2024-09-16T07:34:47.530405+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:47.530405+0000","last_undegraded":"2024-09-16T07:34:47.530405+0000","last_fullsized":"2024-09-16T07:34:47.530405+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:11.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505542+0000","last_change":"2024-09-16T07:34:47.505542+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505542+0000","last_undegraded":"2024-09-16T07:34:47.505542+0000","last_fullsized":"2024-09-16T07:34:47.505542+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530531+0000","last_change":"2024-09-16T07:34:47.529669+0000","last_active":"2024-09-16T07:34:47.530531+0000","last_peered":"2024-09-16T07:34:47.530531+0000","last_clean":"2024-09-16T07:34:47.530531+0000","last_became_active":"2024-09-16T07:34:47.529378+0000","last_became_peered":"2024-09-16T07:34:47.529378+0000","last_unstale":"2024-09-16T07:34:47.530531+0000","last_undegraded":"2024-09-16T07:34:47.530531+0000","last_fullsized":"2024-09-16T07:34:47.530531+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:58.330923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.531202+0000","last_change":"2024-09-16T07:34:48.525029+0000","last_active":"2024-09-16T07:34:48.531202+0000","last_peered":"2024-09-16T07:34:48.531202+0000","last_clean":"2024-09-16T07:34:48.531202+0000","last_became_active":"2024-09-16T07:34:48.524129+0000","last_became_peered":"2024-09-16T07:34:48.524129+0000","last_unstale":"2024-09-16T07:34:48.531202+0000","last_undegraded":"2024-09-16T07:34:48.531202+0000","last_fullsized":"2024-09-16T07:34:48.531202+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:10.118549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"118'6","reported_seq":21,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531542+0000","last_change":"2024-09-16T07:34:47.524021+0000","last_active":"2024-09-16T07:34:49.531542+0000","last_peered":"2024-09-16T07:34:49.531542+0000","last_clean":"2024-09-16T07:34:49.531542+0000","last_became_active":"2024-09-16T07:34:47.520956+0000","last_became_peered":"2024-09-16T07:34:47.520956+0000","last_unstale":"2024-09-16T07:34:49.531542+0000","last_undegraded":"2024-09-16T07:34:49.531542+0000","last_fullsized":"2024-09-16T07:34:49.531542+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:42.661372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.514437+0000","last_change":"2024-09-16T07:34:47.514437+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.514437+0000","last_undegraded":"2024-09-16T07:34:47.514437+0000","last_fullsized":"2024-09-16T07:34:47.514437+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.529914+0000","last_change":"2024-09-16T07:34:47.528845+0000","last_active":"2024-09-16T07:34:47.529914+0000","last_peered":"2024-09-16T07:34:47.529914+0000","last_clean":"2024-09-16T07:34:47.529914+0000","last_became_active":"2024-09-16T07:34:47.528568+0000","last_became_peered":"2024-09-16T07:34:47.528568+0000","last_unstale":"2024-09-16T07:34:47.529914+0000","last_undegraded":"2024-09-16T07:34:47.529914+0000","last_fullsized":"2024-09-16T07:34:47.529914+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.497545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531719+0000","last_change":"2024-09-16T07:34:48.525677+0000","last_active":"2024-09-16T07:34:49.531719+0000","last_peered":"2024-09-16T07:34:49.531719+0000","last_clean":"2024-09-16T07:34:49.531719+0000","last_became_active":"2024-09-16T07:34:48.522263+0000","last_became_peered":"2024-09-16T07:34:48.522263+0000","last_unstale":"2024-09-16T07:34:49.531719+0000","last_undegraded":"2024-09-16T07:34:49.531719+0000","last_fullsized":"2024-09-16T07:34:49.531719+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:28.526205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.508493+0000","last_change":"2024-09-16T07:34:47.523404+0000","last_active":"2024-09-16T07:34:48.508493+0000","last_peered":"2024-09-16T07:34:48.508493+0000","last_clean":"2024-09-16T07:34:48.508493+0000","last_became_active":"2024-09-16T07:34:47.523140+0000","last_became_peered":"2024-09-16T07:34:47.523140+0000","last_unstale":"2024-09-16T07:34:48.508493+0000","last_undegraded":"2024-09-16T07:34:48.508493+0000","last_fullsized":"2024-09-16T07:34:48.508493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:28.805823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531011+0000","last_change":"2024-09-16T07:34:48.530763+0000","last_active":"2024-09-16T07:34:49.531011+0000","last_peered":"2024-09-16T07:34:49.531011+0000","last_clean":"2024-09-16T07:34:49.531011+0000","last_became_active":"2024-09-16T07:34:48.530263+0000","last_became_peered":"2024-09-16T07:34:48.530263+0000","last_unstale":"2024-09-16T07:34:49.531011+0000","last_undegraded":"2024-09-16T07:34:49.531011+0000","last_fullsized":"2024-09-16T07:34:49.531011+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:16.945235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.527329+0000","last_change":"2024-09-16T07:34:47.524692+0000","last_active":"2024-09-16T07:34:47.527329+0000","last_peered":"2024-09-16T07:34:47.527329+0000","last_clean":"2024-09-16T07:34:47.527329+0000","last_became_active":"2024-09-16T07:34:47.524434+0000","last_became_peered":"2024-09-16T07:34:47.524434+0000","last_unstale":"2024-09-16T07:34:47.527329+0000","last_undegraded":"2024-09-16T07:34:47.527329+0000","last_fullsized":"2024-09-16T07:34:47.527329+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:33.456945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.512413+0000","last_change":"2024-09-16T07:34:47.512413+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.512413+0000","last_undegraded":"2024-09-16T07:34:47.512413+0000","last_fullsized":"2024-09-16T07:34:47.512413+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.517308+0000","last_change":"2024-09-16T07:34:47.515933+0000","last_active":"2024-09-16T07:34:47.517308+0000","last_peered":"2024-09-16T07:34:47.517308+0000","last_clean":"2024-09-16T07:34:47.517308+0000","last_became_active":"2024-09-16T07:34:47.515670+0000","last_became_peered":"2024-09-16T07:34:47.515670+0000","last_unstale":"2024-09-16T07:34:47.517308+0000","last_undegraded":"2024-09-16T07:34:47.517308+0000","last_fullsized":"2024-09-16T07:34:47.517308+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:40.298789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.517983+0000","last_change":"2024-09-16T07:34:47.517983+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.517983+0000","last_undegraded":"2024-09-16T07:34:47.517983+0000","last_fullsized":"2024-09-16T07:34:47.517983+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.529910+0000","last_change":"2024-09-16T07:34:47.528922+0000","last_active":"2024-09-16T07:34:47.529910+0000","last_peered":"2024-09-16T07:34:47.529910+0000","last_clean":"2024-09-16T07:34:47.529910+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:47.529910+0000","last_undegraded":"2024-09-16T07:34:47.529910+0000","last_fullsized":"2024-09-16T07:34:47.529910+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:04:45.751140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534692+0000","last_change":"2024-09-16T07:34:48.528054+0000","last_active":"2024-09-16T07:34:48.534692+0000","last_peered":"2024-09-16T07:34:48.534692+0000","last_clean":"2024-09-16T07:34:48.534692+0000","last_became_active":"2024-09-16T07:34:48.527775+0000","last_became_peered":"2024-09-16T07:34:48.527775+0000","last_unstale":"2024-09-16T07:34:48.534692+0000","last_undegraded":"2024-09-16T07:34:48.534692+0000","last_fullsized":"2024-09-16T07:34:48.534692+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:51.806935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.543374+0000","last_change":"2024-09-16T07:34:47.527129+0000","last_active":"2024-09-16T07:34:48.543374+0000","last_peered":"2024-09-16T07:34:48.543374+0000","last_clean":"2024-09-16T07:34:48.543374+0000","last_became_active":"2024-09-16T07:34:47.526046+0000","last_became_peered":"2024-09-16T07:34:47.526046+0000","last_unstale":"2024-09-16T07:34:48.543374+0000","last_undegraded":"2024-09-16T07:34:48.543374+0000","last_fullsized":"2024-09-16T07:34:48.543374+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:40.775080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.524666+0000","last_change":"2024-09-16T07:34:48.522447+0000","last_active":"2024-09-16T07:34:48.524666+0000","last_peered":"2024-09-16T07:34:48.524666+0000","last_clean":"2024-09-16T07:34:48.524666+0000","last_became_active":"2024-09-16T07:34:48.521166+0000","last_became_peered":"2024-09-16T07:34:48.521166+0000","last_unstale":"2024-09-16T07:34:48.524666+0000","last_undegraded":"2024-09-16T07:34:48.524666+0000","last_fullsized":"2024-09-16T07:34:48.524666+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:34.127356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.516801+0000","last_change":"2024-09-16T07:34:47.522843+0000","last_active":"2024-09-16T07:34:48.516801+0000","last_peered":"2024-09-16T07:34:48.516801+0000","last_clean":"2024-09-16T07:34:48.516801+0000","last_became_active":"2024-09-16T07:34:47.522587+0000","last_became_peered":"2024-09-16T07:34:47.522587+0000","last_unstale":"2024-09-16T07:34:48.516801+0000","last_undegraded":"2024-09-16T07:34:48.516801+0000","last_fullsized":"2024-09-16T07:34:48.516801+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:26.555192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533092+0000","last_change":"2024-09-16T07:34:48.528425+0000","last_active":"2024-09-16T07:34:48.533092+0000","last_peered":"2024-09-16T07:34:48.533092+0000","last_clean":"2024-09-16T07:34:48.533092+0000","last_became_active":"2024-09-16T07:34:48.526307+0000","last_became_peered":"2024-09-16T07:34:48.526307+0000","last_unstale":"2024-09-16T07:34:48.533092+0000","last_undegraded":"2024-09-16T07:34:48.533092+0000","last_fullsized":"2024-09-16T07:34:48.533092+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:27.397201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530294+0000","last_change":"2024-09-16T07:34:47.529098+0000","last_active":"2024-09-16T07:34:47.530294+0000","last_peered":"2024-09-16T07:34:47.530294+0000","last_clean":"2024-09-16T07:34:47.530294+0000","last_became_active":"2024-09-16T07:34:47.528650+0000","last_became_peered":"2024-09-16T07:34:47.528650+0000","last_unstale":"2024-09-16T07:34:47.530294+0000","last_undegraded":"2024-09-16T07:34:47.530294+0000","last_fullsized":"2024-09-16T07:34:47.530294+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:15:47.135693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.517824+0000","last_change":"2024-09-16T07:34:47.517824+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.517824+0000","last_undegraded":"2024-09-16T07:34:47.517824+0000","last_fullsized":"2024-09-16T07:34:47.517824+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.526192+0000","last_change":"2024-09-16T07:34:47.521880+0000","last_active":"2024-09-16T07:34:47.526192+0000","last_peered":"2024-09-16T07:34:47.526192+0000","last_clean":"2024-09-16T07:34:47.526192+0000","last_became_active":"2024-09-16T07:34:47.521610+0000","last_became_peered":"2024-09-16T07:34:47.521610+0000","last_unstale":"2024-09-16T07:34:47.526192+0000","last_undegraded":"2024-09-16T07:34:47.526192+0000","last_fullsized":"2024-09-16T07:34:47.526192+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:27.814061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535109+0000","last_change":"2024-09-16T07:34:48.530713+0000","last_active":"2024-09-16T07:34:48.535109+0000","last_peered":"2024-09-16T07:34:48.535109+0000","last_clean":"2024-09-16T07:34:48.535109+0000","last_became_active":"2024-09-16T07:34:48.526504+0000","last_became_peered":"2024-09-16T07:34:48.526504+0000","last_unstale":"2024-09-16T07:34:48.535109+0000","last_undegraded":"2024-09-16T07:34:48.535109+0000","last_fullsized":"2024-09-16T07:34:48.535109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:56.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530916+0000","last_change":"2024-09-16T07:34:47.518969+0000","last_active":"2024-09-16T07:34:49.530916+0000","last_peered":"2024-09-16T07:34:49.530916+0000","last_clean":"2024-09-16T07:34:49.530916+0000","last_became_active":"2024-09-16T07:34:47.518693+0000","last_became_peered":"2024-09-16T07:34:47.518693+0000","last_unstale":"2024-09-16T07:34:49.530916+0000","last_undegraded":"2024-09-16T07:34:49.530916+0000","last_fullsized":"2024-09-16T07:34:49.530916+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:33.694870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516386+0000","last_change":"2024-09-16T07:34:47.516386+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516386+0000","last_undegraded":"2024-09-16T07:34:47.516386+0000","last_fullsized":"2024-09-16T07:34:47.516386+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530779+0000","last_change":"2024-09-16T07:34:47.517277+0000","last_active":"2024-09-16T07:34:49.530779+0000","last_peered":"2024-09-16T07:34:49.530779+0000","last_clean":"2024-09-16T07:34:49.530779+0000","last_became_active":"2024-09-16T07:34:47.516911+0000","last_became_peered":"2024-09-16T07:34:47.516911+0000","last_unstale":"2024-09-16T07:34:49.530779+0000","last_undegraded":"2024-09-16T07:34:49.530779+0000","last_fullsized":"2024-09-16T07:34:49.530779+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:31:40.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.527395+0000","last_change":"2024-09-16T07:34:48.526046+0000","last_active":"2024-09-16T07:34:48.527395+0000","last_peered":"2024-09-16T07:34:48.527395+0000","last_clean":"2024-09-16T07:34:48.527395+0000","last_became_active":"2024-09-16T07:34:48.525684+0000","last_became_peered":"2024-09-16T07:34:48.525684+0000","last_unstale":"2024-09-16T07:34:48.527395+0000","last_undegraded":"2024-09-16T07:34:48.527395+0000","last_fullsized":"2024-09-16T07:34:48.527395+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:50.191171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531230+0000","last_change":"2024-09-16T07:34:47.521736+0000","last_active":"2024-09-16T07:34:49.531230+0000","last_peered":"2024-09-16T07:34:49.531230+0000","last_clean":"2024-09-16T07:34:49.531230+0000","last_became_active":"2024-09-16T07:34:47.521422+0000","last_became_peered":"2024-09-16T07:34:47.521422+0000","last_unstale":"2024-09-16T07:34:49.531230+0000","last_undegraded":"2024-09-16T07:34:49.531230+0000","last_fullsized":"2024-09-16T07:34:49.531230+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:02.620783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505633+0000","last_change":"2024-09-16T07:34:47.505633+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505633+0000","last_undegraded":"2024-09-16T07:34:47.505633+0000","last_fullsized":"2024-09-16T07:34:47.505633+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.518493+0000","last_change":"2024-09-16T07:34:47.511494+0000","last_active":"2024-09-16T07:34:48.518493+0000","last_peered":"2024-09-16T07:34:48.518493+0000","last_clean":"2024-09-16T07:34:48.518493+0000","last_became_active":"2024-09-16T07:34:47.511196+0000","last_became_peered":"2024-09-16T07:34:47.511196+0000","last_unstale":"2024-09-16T07:34:48.518493+0000","last_undegraded":"2024-09-16T07:34:48.518493+0000","last_fullsized":"2024-09-16T07:34:48.518493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:18.504885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.512300+0000","last_change":"2024-09-16T07:34:47.512300+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.512300+0000","last_undegraded":"2024-09-16T07:34:47.512300+0000","last_fullsized":"2024-09-16T07:34:47.512300+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531451+0000","last_change":"2024-09-16T07:34:47.516251+0000","last_active":"2024-09-16T07:34:49.531451+0000","last_peered":"2024-09-16T07:34:49.531451+0000","last_clean":"2024-09-16T07:34:49.531451+0000","last_became_active":"2024-09-16T07:34:47.515980+0000","last_became_peered":"2024-09-16T07:34:47.515980+0000","last_unstale":"2024-09-16T07:34:49.531451+0000","last_undegraded":"2024-09-16T07:34:49.531451+0000","last_fullsized":"2024-09-16T07:34:49.531451+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:16.654899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.542525+0000","last_change":"2024-09-16T07:34:48.542025+0000","last_active":"2024-09-16T07:34:48.542525+0000","last_peered":"2024-09-16T07:34:48.542525+0000","last_clean":"2024-09-16T07:34:48.542525+0000","last_became_active":"2024-09-16T07:34:48.541713+0000","last_became_peered":"2024-09-16T07:34:48.541713+0000","last_unstale":"2024-09-16T07:34:48.542525+0000","last_undegraded":"2024-09-16T07:34:48.542525+0000","last_fullsized":"2024-09-16T07:34:48.542525+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:03.040043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531609+0000","last_change":"2024-09-16T07:34:47.513397+0000","last_active":"2024-09-16T07:34:49.531609+0000","last_peered":"2024-09-16T07:34:49.531609+0000","last_clean":"2024-09-16T07:34:49.531609+0000","last_became_active":"2024-09-16T07:34:47.513084+0000","last_became_peered":"2024-09-16T07:34:47.513084+0000","last_unstale":"2024-09-16T07:34:49.531609+0000","last_undegraded":"2024-09-16T07:34:49.531609+0000","last_fullsized":"2024-09-16T07:34:49.531609+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:49.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533116+0000","last_change":"2024-09-16T07:34:48.526515+0000","last_active":"2024-09-16T07:34:48.533116+0000","last_peered":"2024-09-16T07:34:48.533116+0000","last_clean":"2024-09-16T07:34:48.533116+0000","last_became_active":"2024-09-16T07:34:48.525964+0000","last_became_peered":"2024-09-16T07:34:48.525964+0000","last_unstale":"2024-09-16T07:34:48.533116+0000","last_undegraded":"2024-09-16T07:34:48.533116+0000","last_fullsized":"2024-09-16T07:34:48.533116+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:11.702095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.520393+0000","last_change":"2024-09-16T07:34:47.518568+0000","last_active":"2024-09-16T07:34:48.520393+0000","last_peered":"2024-09-16T07:34:48.520393+0000","last_clean":"2024-09-16T07:34:48.520393+0000","last_became_active":"2024-09-16T07:34:47.518279+0000","last_became_peered":"2024-09-16T07:34:47.518279+0000","last_unstale":"2024-09-16T07:34:48.520393+0000","last_undegraded":"2024-09-16T07:34:48.520393+0000","last_fullsized":"2024-09-16T07:34:48.520393+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:36.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534614+0000","last_change":"2024-09-16T07:34:48.528073+0000","last_active":"2024-09-16T07:34:48.534614+0000","last_peered":"2024-09-16T07:34:48.534614+0000","last_clean":"2024-09-16T07:34:48.534614+0000","last_became_active":"2024-09-16T07:34:48.527807+0000","last_became_peered":"2024-09-16T07:34:48.527807+0000","last_unstale":"2024-09-16T07:34:48.534614+0000","last_undegraded":"2024-09-16T07:34:48.534614+0000","last_fullsized":"2024-09-16T07:34:48.534614+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:22.775050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.514866+0000","last_change":"2024-09-16T07:34:47.527175+0000","last_active":"2024-09-16T07:34:48.514866+0000","last_peered":"2024-09-16T07:34:48.514866+0000","last_clean":"2024-09-16T07:34:48.514866+0000","last_became_active":"2024-09-16T07:34:47.526929+0000","last_became_peered":"2024-09-16T07:34:47.526929+0000","last_unstale":"2024-09-16T07:34:48.514866+0000","last_undegraded":"2024-09-16T07:34:48.514866+0000","last_fullsized":"2024-09-16T07:34:48.514866+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:56.405916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.527000+0000","last_change":"2024-09-16T07:34:48.524675+0000","last_active":"2024-09-16T07:34:48.527000+0000","last_peered":"2024-09-16T07:34:48.527000+0000","last_clean":"2024-09-16T07:34:48.527000+0000","last_became_active":"2024-09-16T07:34:48.524324+0000","last_became_peered":"2024-09-16T07:34:48.524324+0000","last_unstale":"2024-09-16T07:34:48.527000+0000","last_undegraded":"2024-09-16T07:34:48.527000+0000","last_fullsized":"2024-09-16T07:34:48.527000+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:55.577009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.527035+0000","last_change":"2024-09-16T07:34:47.525571+0000","last_active":"2024-09-16T07:34:47.527035+0000","last_peered":"2024-09-16T07:34:47.527035+0000","last_clean":"2024-09-16T07:34:47.527035+0000","last_became_active":"2024-09-16T07:34:47.525295+0000","last_became_peered":"2024-09-16T07:34:47.525295+0000","last_unstale":"2024-09-16T07:34:47.527035+0000","last_undegraded":"2024-09-16T07:34:47.527035+0000","last_fullsized":"2024-09-16T07:34:47.527035+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:53.256795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.506924+0000","last_change":"2024-09-16T07:34:47.506924+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.506924+0000","last_undegraded":"2024-09-16T07:34:47.506924+0000","last_fullsized":"2024-09-16T07:34:47.506924+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.520862+0000","last_change":"2024-09-16T07:34:47.518753+0000","last_active":"2024-09-16T07:34:47.520862+0000","last_peered":"2024-09-16T07:34:47.520862+0000","last_clean":"2024-09-16T07:34:47.520862+0000","last_became_active":"2024-09-16T07:34:47.518486+0000","last_became_peered":"2024-09-16T07:34:47.518486+0000","last_unstale":"2024-09-16T07:34:47.520862+0000","last_undegraded":"2024-09-16T07:34:47.520862+0000","last_fullsized":"2024-09-16T07:34:47.520862+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:21.437105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531155+0000","last_change":"2024-09-16T07:34:48.533301+0000","last_active":"2024-09-16T07:34:49.531155+0000","last_peered":"2024-09-16T07:34:49.531155+0000","last_clean":"2024-09-16T07:34:49.531155+0000","last_became_active":"2024-09-16T07:34:48.532697+0000","last_became_peered":"2024-09-16T07:34:48.532697+0000","last_unstale":"2024-09-16T07:34:49.531155+0000","last_undegraded":"2024-09-16T07:34:49.531155+0000","last_fullsized":"2024-09-16T07:34:49.531155+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:14:10.393373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.520135+0000","last_change":"2024-09-16T07:34:47.521842+0000","last_active":"2024-09-16T07:34:48.520135+0000","last_peered":"2024-09-16T07:34:48.520135+0000","last_clean":"2024-09-16T07:34:48.520135+0000","last_became_active":"2024-09-16T07:34:47.521578+0000","last_became_peered":"2024-09-16T07:34:47.521578+0000","last_unstale":"2024-09-16T07:34:48.520135+0000","last_undegraded":"2024-09-16T07:34:48.520135+0000","last_fullsized":"2024-09-16T07:34:48.520135+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:20.091370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":288,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.507819+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:34:48.507819+0000","last_peered":"2024-09-16T07:34:48.507819+0000","last_clean":"2024-09-16T07:34:48.507819+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:34:48.507819+0000","last_undegraded":"2024-09-16T07:34:48.507819+0000","last_fullsized":"2024-09-16T07:34:48.507819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2264,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126944,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244896,"statfs":{"total":96636764160,"available":96506773504,"internally_reserved":0,"allocated":1454080,"data_stored":635315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126932,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244908,"statfs":{"total":96636764160,"available":96506785792,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123352,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248488,"statfs":{"total":96636764160,"available":96510451712,"internally_reserved":0,"allocated":2035712,"data_stored":1218852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140820,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231020,"statfs":{"total":96636764160,"available":96492564480,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122768,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249072,"statfs":{"total":96636764160,"available":96511049728,"internally_reserved":0,"allocated":1429504,"data_stored":622153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509590,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153308,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218532,"statfs":{"total":96636764160,"available":96479776768,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131672,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240168,"statfs":{"total":96636764160,"available":96501932032,"internally_reserved":0,"allocated":2035712,"data_stored":1218852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126976,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244864,"statfs":{"total":96636764160,"available":96506740736,"internally_reserved":0,"allocated":1482752,"data_stored":643373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:34:52.721 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:34:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.914+0000 7f2cd2800640 1 -- 172.21.15.71:0/3715852639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0d0540 msgr2=0x7f2ccc06f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.914+0000 7f2cd2800640 1 --2- 172.21.15.71:0/3715852639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0d0540 0x7f2ccc06f320 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0cd570 tx=0x7f2cc402a8b0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.914+0000 7f2cd2800640 1 -- 172.21.15.71:0/3715852639 shutdown_connections 2024-09-16T07:34:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.914+0000 7f2cd2800640 1 --2- 172.21.15.71:0/3715852639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0d0540 0x7f2ccc06f320 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.914+0000 7f2cd2800640 1 --2- 172.21.15.71:0/3715852639 >> v2:172.21.15.177:3300/0 conn(0x7f2ccc15bc30 0x7f2ccc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.914+0000 7f2cd2800640 1 -- 172.21.15.71:0/3715852639 >> 172.21.15.71:0/3715852639 conn(0x7f2ccc0cfa20 msgr2=0x7f2ccc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.914+0000 7f2cd2800640 1 -- 172.21.15.71:0/3715852639 shutdown_connections 2024-09-16T07:34:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.914+0000 7f2cd2800640 1 -- 172.21.15.71:0/3715852639 wait complete. 2024-09-16T07:34:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.916+0000 7f2cd2800640 1 Processor -- start 2024-09-16T07:34:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.916+0000 7f2cd2800640 1 -- start start 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd2800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0d0540 0x7f2ccc0af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd2800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15bc30 0x7f2ccc1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0afec0 0x7f2ccc1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd2800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc074660 con 0x7f2ccc0afec0 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd2800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc06f860 con 0x7f2ccc15bc30 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd2800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2ccc0c7420 con 0x7f2ccc0d0540 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0afec0 0x7f2ccc1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0afec0 0x7f2ccc1587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34592/0 (socket says 172.21.15.71:34592) 2024-09-16T07:34:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1e00640 1 -- 172.21.15.71:0/1022238545 learned_addr learned my addr 172.21.15.71:0/1022238545 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1400640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0d0540 0x7f2ccc0af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd0a00640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15bc30 0x7f2ccc1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1e00640 1 -- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0d0540 msgr2=0x7f2ccc0af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1e00640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0d0540 0x7f2ccc0af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1e00640 1 -- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15bc30 msgr2=0x7f2ccc1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1e00640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15bc30 0x7f2ccc1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.917+0000 7f2cd1e00640 1 -- 172.21.15.71:0/1022238545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc0b0420 con 0x7f2ccc0afec0 2024-09-16T07:34:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.918+0000 7f2cd1e00640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0afec0 0x7f2ccc1587e0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402aea0 tx=0x7f2cc4031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.918+0000 7f2cc9600640 1 -- 172.21.15.71:0/1022238545 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4040070 con 0x7f2ccc0afec0 2024-09-16T07:34:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.918+0000 7f2cc9600640 1 -- 172.21.15.71:0/1022238545 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cc4004040 con 0x7f2ccc0afec0 2024-09-16T07:34:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.918+0000 7f2cc9600640 1 -- 172.21.15.71:0/1022238545 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004470 con 0x7f2ccc0afec0 2024-09-16T07:34:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.918+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc0c6af0 con 0x7f2ccc0afec0 2024-09-16T07:34:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.918+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc0c70b0 con 0x7f2ccc0afec0 2024-09-16T07:34:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.919+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc06fdf0 con 0x7f2ccc0afec0 2024-09-16T07:34:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.920+0000 7f2cc9600640 1 -- 172.21.15.71:0/1022238545 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f2cc4039040 con 0x7f2ccc0afec0 2024-09-16T07:34:52.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.921+0000 7f2cc9600640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2cac07a230 0x7f2cac07c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:52.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.921+0000 7f2cc9600640 1 -- 172.21.15.71:0/1022238545 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f2cc40bad60 con 0x7f2ccc0afec0 2024-09-16T07:34:52.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.922+0000 7f2cd1400640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2cac07a230 0x7f2cac07c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:52.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.922+0000 7f2cd1400640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2cac07a230 0x7f2cac07c6f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000bb50 tx=0x7f2cc000b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:52.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:52.925+0000 7f2cc9600640 1 -- 172.21.15.71:0/1022238545 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2cc40847e0 con 0x7f2ccc0afec0 2024-09-16T07:34:53.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.107+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ccc156a70 con 0x7f2cac07a230 2024-09-16T07:34:53.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.113+0000 7f2cc9600640 1 -- 172.21.15.71:0/1022238545 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250404 (secure 0 0 0) 0x7f2ccc156a70 con 0x7f2cac07a230 2024-09-16T07:34:53.115 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:53.116 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:34:53.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2cac07a230 msgr2=0x7f2cac07c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2cac07a230 0x7f2cac07c6f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000bb50 tx=0x7f2cc000b2e0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0afec0 msgr2=0x7f2ccc1587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0afec0 0x7f2ccc1587e0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402aea0 tx=0x7f2cc4031760 comp rx=0 tx=0).stop 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 shutdown_connections 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2cac07a230 0x7f2cac07c6f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc0afec0 0x7f2ccc1587e0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15bc30 0x7f2ccc1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 --2- 172.21.15.71:0/1022238545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0d0540 0x7f2ccc0af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 >> 172.21.15.71:0/1022238545 conn(0x7f2ccc0cfa20 msgr2=0x7f2ccc0ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 shutdown_connections 2024-09-16T07:34:53.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.117+0000 7f2cd2800640 1 -- 172.21.15.71:0/1022238545 wait complete. 2024-09-16T07:34:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-16T07:34:52.604187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592632,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1052772,"kb_used_data":12464,"kb_used_omap":12,"kb_used_meta":1040115,"kb_avail":753921948,"statfs":{"total":773094113280,"available":772016074752,"internally_reserved":0,"allocated":12763136,"data_stored":6223997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1065078320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2264,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001350"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.519868+0000","last_change":"2024-09-16T07:34:47.519868+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.519868+0000","last_undegraded":"2024-09-16T07:34:47.519868+0000","last_fullsized":"2024-09-16T07:34:47.519868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.542589+0000","last_change":"2024-09-16T07:34:48.541678+0000","last_active":"2024-09-16T07:34:48.542589+0000","last_peered":"2024-09-16T07:34:48.542589+0000","last_clean":"2024-09-16T07:34:48.542589+0000","last_became_active":"2024-09-16T07:34:48.541360+0000","last_became_peered":"2024-09-16T07:34:48.541360+0000","last_unstale":"2024-09-16T07:34:48.542589+0000","last_undegraded":"2024-09-16T07:34:48.542589+0000","last_fullsized":"2024-09-16T07:34:48.542589+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:42.661091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535282+0000","last_change":"2024-09-16T07:34:48.534129+0000","last_active":"2024-09-16T07:34:48.535282+0000","last_peered":"2024-09-16T07:34:48.535282+0000","last_clean":"2024-09-16T07:34:48.535282+0000","last_became_active":"2024-09-16T07:34:48.533584+0000","last_became_peered":"2024-09-16T07:34:48.533584+0000","last_unstale":"2024-09-16T07:34:48.535282+0000","last_undegraded":"2024-09-16T07:34:48.535282+0000","last_fullsized":"2024-09-16T07:34:48.535282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:57.899237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531563+0000","last_change":"2024-09-16T07:34:48.526281+0000","last_active":"2024-09-16T07:34:49.531563+0000","last_peered":"2024-09-16T07:34:49.531563+0000","last_clean":"2024-09-16T07:34:49.531563+0000","last_became_active":"2024-09-16T07:34:48.525843+0000","last_became_peered":"2024-09-16T07:34:48.525843+0000","last_unstale":"2024-09-16T07:34:49.531563+0000","last_undegraded":"2024-09-16T07:34:49.531563+0000","last_fullsized":"2024-09-16T07:34:49.531563+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:50.039555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513726+0000","last_change":"2024-09-16T07:34:47.513726+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513726+0000","last_undegraded":"2024-09-16T07:34:47.513726+0000","last_fullsized":"2024-09-16T07:34:47.513726+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.507372+0000","last_change":"2024-09-16T07:34:47.507372+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.507372+0000","last_undegraded":"2024-09-16T07:34:47.507372+0000","last_fullsized":"2024-09-16T07:34:47.507372+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534985+0000","last_change":"2024-09-16T07:34:48.526151+0000","last_active":"2024-09-16T07:34:48.534985+0000","last_peered":"2024-09-16T07:34:48.534985+0000","last_clean":"2024-09-16T07:34:48.534985+0000","last_became_active":"2024-09-16T07:34:48.525914+0000","last_became_peered":"2024-09-16T07:34:48.525914+0000","last_unstale":"2024-09-16T07:34:48.534985+0000","last_undegraded":"2024-09-16T07:34:48.534985+0000","last_fullsized":"2024-09-16T07:34:48.534985+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:31.962809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535012+0000","last_change":"2024-09-16T07:34:48.528915+0000","last_active":"2024-09-16T07:34:48.535012+0000","last_peered":"2024-09-16T07:34:48.535012+0000","last_clean":"2024-09-16T07:34:48.535012+0000","last_became_active":"2024-09-16T07:34:48.528648+0000","last_became_peered":"2024-09-16T07:34:48.528648+0000","last_unstale":"2024-09-16T07:34:48.535012+0000","last_undegraded":"2024-09-16T07:34:48.535012+0000","last_fullsized":"2024-09-16T07:34:48.535012+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:49.801672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.519440+0000","last_change":"2024-09-16T07:34:47.519440+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.519440+0000","last_undegraded":"2024-09-16T07:34:47.519440+0000","last_fullsized":"2024-09-16T07:34:47.519440+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.514995+0000","last_change":"2024-09-16T07:34:47.514995+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.514995+0000","last_undegraded":"2024-09-16T07:34:47.514995+0000","last_fullsized":"2024-09-16T07:34:47.514995+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534388+0000","last_change":"2024-09-16T07:34:48.530703+0000","last_active":"2024-09-16T07:34:48.534388+0000","last_peered":"2024-09-16T07:34:48.534388+0000","last_clean":"2024-09-16T07:34:48.534388+0000","last_became_active":"2024-09-16T07:34:48.530406+0000","last_became_peered":"2024-09-16T07:34:48.530406+0000","last_unstale":"2024-09-16T07:34:48.534388+0000","last_undegraded":"2024-09-16T07:34:48.534388+0000","last_fullsized":"2024-09-16T07:34:48.534388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:24.273074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.515387+0000","last_change":"2024-09-16T07:34:47.515387+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.515387+0000","last_undegraded":"2024-09-16T07:34:47.515387+0000","last_fullsized":"2024-09-16T07:34:47.515387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.538475+0000","last_change":"2024-09-16T07:34:48.535432+0000","last_active":"2024-09-16T07:34:48.538475+0000","last_peered":"2024-09-16T07:34:48.538475+0000","last_clean":"2024-09-16T07:34:48.538475+0000","last_became_active":"2024-09-16T07:34:48.532194+0000","last_became_peered":"2024-09-16T07:34:48.532194+0000","last_unstale":"2024-09-16T07:34:48.538475+0000","last_undegraded":"2024-09-16T07:34:48.538475+0000","last_fullsized":"2024-09-16T07:34:48.538475+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:02:06.782991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535669+0000","last_change":"2024-09-16T07:34:48.531769+0000","last_active":"2024-09-16T07:34:48.535669+0000","last_peered":"2024-09-16T07:34:48.535669+0000","last_clean":"2024-09-16T07:34:48.535669+0000","last_became_active":"2024-09-16T07:34:48.531251+0000","last_became_peered":"2024-09-16T07:34:48.531251+0000","last_unstale":"2024-09-16T07:34:48.535669+0000","last_undegraded":"2024-09-16T07:34:48.535669+0000","last_fullsized":"2024-09-16T07:34:48.535669+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:03.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513794+0000","last_change":"2024-09-16T07:34:47.513794+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513794+0000","last_undegraded":"2024-09-16T07:34:47.513794+0000","last_fullsized":"2024-09-16T07:34:47.513794+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516736+0000","last_change":"2024-09-16T07:34:47.516736+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516736+0000","last_undegraded":"2024-09-16T07:34:47.516736+0000","last_fullsized":"2024-09-16T07:34:47.516736+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.522422+0000","last_change":"2024-09-16T07:34:48.520786+0000","last_active":"2024-09-16T07:34:48.522422+0000","last_peered":"2024-09-16T07:34:48.522422+0000","last_clean":"2024-09-16T07:34:48.522422+0000","last_became_active":"2024-09-16T07:34:48.519475+0000","last_became_peered":"2024-09-16T07:34:48.519475+0000","last_unstale":"2024-09-16T07:34:48.522422+0000","last_undegraded":"2024-09-16T07:34:48.522422+0000","last_fullsized":"2024-09-16T07:34:48.522422+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:38.065118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535936+0000","last_change":"2024-09-16T07:34:48.527942+0000","last_active":"2024-09-16T07:34:48.535936+0000","last_peered":"2024-09-16T07:34:48.535936+0000","last_clean":"2024-09-16T07:34:48.535936+0000","last_became_active":"2024-09-16T07:34:48.527669+0000","last_became_peered":"2024-09-16T07:34:48.527669+0000","last_unstale":"2024-09-16T07:34:48.535936+0000","last_undegraded":"2024-09-16T07:34:48.535936+0000","last_fullsized":"2024-09-16T07:34:48.535936+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:56.714125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535262+0000","last_change":"2024-09-16T07:34:48.534101+0000","last_active":"2024-09-16T07:34:48.535262+0000","last_peered":"2024-09-16T07:34:48.535262+0000","last_clean":"2024-09-16T07:34:48.535262+0000","last_became_active":"2024-09-16T07:34:48.529574+0000","last_became_peered":"2024-09-16T07:34:48.529574+0000","last_unstale":"2024-09-16T07:34:48.535262+0000","last_undegraded":"2024-09-16T07:34:48.535262+0000","last_fullsized":"2024-09-16T07:34:48.535262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:30:49.545255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533243+0000","last_change":"2024-09-16T07:34:48.529587+0000","last_active":"2024-09-16T07:34:48.533243+0000","last_peered":"2024-09-16T07:34:48.533243+0000","last_clean":"2024-09-16T07:34:48.533243+0000","last_became_active":"2024-09-16T07:34:48.529304+0000","last_became_peered":"2024-09-16T07:34:48.529304+0000","last_unstale":"2024-09-16T07:34:48.533243+0000","last_undegraded":"2024-09-16T07:34:48.533243+0000","last_fullsized":"2024-09-16T07:34:48.533243+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:06:46.119567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534304+0000","last_change":"2024-09-16T07:34:48.533118+0000","last_active":"2024-09-16T07:34:48.534304+0000","last_peered":"2024-09-16T07:34:48.534304+0000","last_clean":"2024-09-16T07:34:48.534304+0000","last_became_active":"2024-09-16T07:34:48.532777+0000","last_became_peered":"2024-09-16T07:34:48.532777+0000","last_unstale":"2024-09-16T07:34:48.534304+0000","last_undegraded":"2024-09-16T07:34:48.534304+0000","last_fullsized":"2024-09-16T07:34:48.534304+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:02.406116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505332+0000","last_change":"2024-09-16T07:34:47.505332+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505332+0000","last_undegraded":"2024-09-16T07:34:47.505332+0000","last_fullsized":"2024-09-16T07:34:47.505332+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535054+0000","last_change":"2024-09-16T07:34:48.528984+0000","last_active":"2024-09-16T07:34:48.535054+0000","last_peered":"2024-09-16T07:34:48.535054+0000","last_clean":"2024-09-16T07:34:48.535054+0000","last_became_active":"2024-09-16T07:34:48.528481+0000","last_became_peered":"2024-09-16T07:34:48.528481+0000","last_unstale":"2024-09-16T07:34:48.535054+0000","last_undegraded":"2024-09-16T07:34:48.535054+0000","last_fullsized":"2024-09-16T07:34:48.535054+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:43.572025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530706+0000","last_change":"2024-09-16T07:34:48.533570+0000","last_active":"2024-09-16T07:34:49.530706+0000","last_peered":"2024-09-16T07:34:49.530706+0000","last_clean":"2024-09-16T07:34:49.530706+0000","last_became_active":"2024-09-16T07:34:48.533039+0000","last_became_peered":"2024-09-16T07:34:48.533039+0000","last_unstale":"2024-09-16T07:34:49.530706+0000","last_undegraded":"2024-09-16T07:34:49.530706+0000","last_fullsized":"2024-09-16T07:34:49.530706+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:00:09.699334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533344+0000","last_change":"2024-09-16T07:34:48.527254+0000","last_active":"2024-09-16T07:34:48.533344+0000","last_peered":"2024-09-16T07:34:48.533344+0000","last_clean":"2024-09-16T07:34:48.533344+0000","last_became_active":"2024-09-16T07:34:48.527013+0000","last_became_peered":"2024-09-16T07:34:48.527013+0000","last_unstale":"2024-09-16T07:34:48.533344+0000","last_undegraded":"2024-09-16T07:34:48.533344+0000","last_fullsized":"2024-09-16T07:34:48.533344+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:16.029488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505317+0000","last_change":"2024-09-16T07:34:47.505317+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505317+0000","last_undegraded":"2024-09-16T07:34:47.505317+0000","last_fullsized":"2024-09-16T07:34:47.505317+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531251+0000","last_change":"2024-09-16T07:34:48.541120+0000","last_active":"2024-09-16T07:34:49.531251+0000","last_peered":"2024-09-16T07:34:49.531251+0000","last_clean":"2024-09-16T07:34:49.531251+0000","last_became_active":"2024-09-16T07:34:48.540074+0000","last_became_peered":"2024-09-16T07:34:48.540074+0000","last_unstale":"2024-09-16T07:34:49.531251+0000","last_undegraded":"2024-09-16T07:34:49.531251+0000","last_fullsized":"2024-09-16T07:34:49.531251+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:53.929219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.507462+0000","last_change":"2024-09-16T07:34:47.507462+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.507462+0000","last_undegraded":"2024-09-16T07:34:47.507462+0000","last_fullsized":"2024-09-16T07:34:47.507462+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535813+0000","last_change":"2024-09-16T07:34:48.534695+0000","last_active":"2024-09-16T07:34:48.535813+0000","last_peered":"2024-09-16T07:34:48.535813+0000","last_clean":"2024-09-16T07:34:48.535813+0000","last_became_active":"2024-09-16T07:34:48.534435+0000","last_became_peered":"2024-09-16T07:34:48.534435+0000","last_unstale":"2024-09-16T07:34:48.535813+0000","last_undegraded":"2024-09-16T07:34:48.535813+0000","last_fullsized":"2024-09-16T07:34:48.535813+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:11.262607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534350+0000","last_change":"2024-09-16T07:34:48.525252+0000","last_active":"2024-09-16T07:34:48.534350+0000","last_peered":"2024-09-16T07:34:48.534350+0000","last_clean":"2024-09-16T07:34:48.534350+0000","last_became_active":"2024-09-16T07:34:48.524453+0000","last_became_peered":"2024-09-16T07:34:48.524453+0000","last_unstale":"2024-09-16T07:34:48.534350+0000","last_undegraded":"2024-09-16T07:34:48.534350+0000","last_fullsized":"2024-09-16T07:34:48.534350+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:18.325757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513371+0000","last_change":"2024-09-16T07:34:47.513371+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513371+0000","last_undegraded":"2024-09-16T07:34:47.513371+0000","last_fullsized":"2024-09-16T07:34:47.513371+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516259+0000","last_change":"2024-09-16T07:34:47.516259+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516259+0000","last_undegraded":"2024-09-16T07:34:47.516259+0000","last_fullsized":"2024-09-16T07:34:47.516259+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513227+0000","last_change":"2024-09-16T07:34:47.513227+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513227+0000","last_undegraded":"2024-09-16T07:34:47.513227+0000","last_fullsized":"2024-09-16T07:34:47.513227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"118'4","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.537397+0000","last_change":"2024-09-16T07:34:47.528565+0000","last_active":"2024-09-16T07:34:48.537397+0000","last_peered":"2024-09-16T07:34:48.537397+0000","last_clean":"2024-09-16T07:34:48.537397+0000","last_became_active":"2024-09-16T07:34:47.528270+0000","last_became_peered":"2024-09-16T07:34:47.528270+0000","last_unstale":"2024-09-16T07:34:48.537397+0000","last_undegraded":"2024-09-16T07:34:48.537397+0000","last_fullsized":"2024-09-16T07:34:48.537397+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:27.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535195+0000","last_change":"2024-09-16T07:34:48.529917+0000","last_active":"2024-09-16T07:34:48.535195+0000","last_peered":"2024-09-16T07:34:48.535195+0000","last_clean":"2024-09-16T07:34:48.535195+0000","last_became_active":"2024-09-16T07:34:48.529631+0000","last_became_peered":"2024-09-16T07:34:48.529631+0000","last_unstale":"2024-09-16T07:34:48.535195+0000","last_undegraded":"2024-09-16T07:34:48.535195+0000","last_fullsized":"2024-09-16T07:34:48.535195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:13.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.537388+0000","last_change":"2024-09-16T07:34:47.526481+0000","last_active":"2024-09-16T07:34:48.537388+0000","last_peered":"2024-09-16T07:34:48.537388+0000","last_clean":"2024-09-16T07:34:48.537388+0000","last_became_active":"2024-09-16T07:34:47.526215+0000","last_became_peered":"2024-09-16T07:34:47.526215+0000","last_unstale":"2024-09-16T07:34:48.537388+0000","last_undegraded":"2024-09-16T07:34:48.537388+0000","last_fullsized":"2024-09-16T07:34:48.537388+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:08.617410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.506925+0000","last_change":"2024-09-16T07:34:47.506925+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.506925+0000","last_undegraded":"2024-09-16T07:34:47.506925+0000","last_fullsized":"2024-09-16T07:34:47.506925+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531333+0000","last_change":"2024-09-16T07:34:47.522199+0000","last_active":"2024-09-16T07:34:49.531333+0000","last_peered":"2024-09-16T07:34:49.531333+0000","last_clean":"2024-09-16T07:34:49.531333+0000","last_became_active":"2024-09-16T07:34:47.521943+0000","last_became_peered":"2024-09-16T07:34:47.521943+0000","last_unstale":"2024-09-16T07:34:49.531333+0000","last_undegraded":"2024-09-16T07:34:49.531333+0000","last_fullsized":"2024-09-16T07:34:49.531333+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:08.548657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535239+0000","last_change":"2024-09-16T07:34:48.533772+0000","last_active":"2024-09-16T07:34:48.535239+0000","last_peered":"2024-09-16T07:34:48.535239+0000","last_clean":"2024-09-16T07:34:48.535239+0000","last_became_active":"2024-09-16T07:34:48.529223+0000","last_became_peered":"2024-09-16T07:34:48.529223+0000","last_unstale":"2024-09-16T07:34:48.535239+0000","last_undegraded":"2024-09-16T07:34:48.535239+0000","last_fullsized":"2024-09-16T07:34:48.535239+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:56.315624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531454+0000","last_change":"2024-09-16T07:34:47.523452+0000","last_active":"2024-09-16T07:34:49.531454+0000","last_peered":"2024-09-16T07:34:49.531454+0000","last_clean":"2024-09-16T07:34:49.531454+0000","last_became_active":"2024-09-16T07:34:47.521451+0000","last_became_peered":"2024-09-16T07:34:47.521451+0000","last_unstale":"2024-09-16T07:34:49.531454+0000","last_undegraded":"2024-09-16T07:34:49.531454+0000","last_fullsized":"2024-09-16T07:34:49.531454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.908074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531052+0000","last_change":"2024-09-16T07:34:48.538587+0000","last_active":"2024-09-16T07:34:49.531052+0000","last_peered":"2024-09-16T07:34:49.531052+0000","last_clean":"2024-09-16T07:34:49.531052+0000","last_became_active":"2024-09-16T07:34:48.538287+0000","last_became_peered":"2024-09-16T07:34:48.538287+0000","last_unstale":"2024-09-16T07:34:49.531052+0000","last_undegraded":"2024-09-16T07:34:49.531052+0000","last_fullsized":"2024-09-16T07:34:49.531052+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:22.418271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.539626+0000","last_change":"2024-09-16T07:34:47.520652+0000","last_active":"2024-09-16T07:34:48.539626+0000","last_peered":"2024-09-16T07:34:48.539626+0000","last_clean":"2024-09-16T07:34:48.539626+0000","last_became_active":"2024-09-16T07:34:47.520279+0000","last_became_peered":"2024-09-16T07:34:47.520279+0000","last_unstale":"2024-09-16T07:34:48.539626+0000","last_undegraded":"2024-09-16T07:34:48.539626+0000","last_fullsized":"2024-09-16T07:34:48.539626+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:17.809818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534281+0000","last_change":"2024-09-16T07:34:48.532853+0000","last_active":"2024-09-16T07:34:48.534281+0000","last_peered":"2024-09-16T07:34:48.534281+0000","last_clean":"2024-09-16T07:34:48.534281+0000","last_became_active":"2024-09-16T07:34:48.532447+0000","last_became_peered":"2024-09-16T07:34:48.532447+0000","last_unstale":"2024-09-16T07:34:48.534281+0000","last_undegraded":"2024-09-16T07:34:48.534281+0000","last_fullsized":"2024-09-16T07:34:48.534281+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:43.058025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.528166+0000","last_change":"2024-09-16T07:34:47.527304+0000","last_active":"2024-09-16T07:34:47.528166+0000","last_peered":"2024-09-16T07:34:47.528166+0000","last_clean":"2024-09-16T07:34:47.528166+0000","last_became_active":"2024-09-16T07:34:47.526260+0000","last_became_peered":"2024-09-16T07:34:47.526260+0000","last_unstale":"2024-09-16T07:34:47.528166+0000","last_undegraded":"2024-09-16T07:34:47.528166+0000","last_fullsized":"2024-09-16T07:34:47.528166+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:10.669755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.515707+0000","last_change":"2024-09-16T07:34:47.515707+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.515707+0000","last_undegraded":"2024-09-16T07:34:47.515707+0000","last_fullsized":"2024-09-16T07:34:47.515707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.517911+0000","last_change":"2024-09-16T07:34:47.520199+0000","last_active":"2024-09-16T07:34:48.517911+0000","last_peered":"2024-09-16T07:34:48.517911+0000","last_clean":"2024-09-16T07:34:48.517911+0000","last_became_active":"2024-09-16T07:34:47.519941+0000","last_became_peered":"2024-09-16T07:34:47.519941+0000","last_unstale":"2024-09-16T07:34:48.517911+0000","last_undegraded":"2024-09-16T07:34:48.517911+0000","last_fullsized":"2024-09-16T07:34:48.517911+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:54.626775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.538297+0000","last_change":"2024-09-16T07:34:48.534789+0000","last_active":"2024-09-16T07:34:48.538297+0000","last_peered":"2024-09-16T07:34:48.538297+0000","last_clean":"2024-09-16T07:34:48.538297+0000","last_became_active":"2024-09-16T07:34:48.531669+0000","last_became_peered":"2024-09-16T07:34:48.531669+0000","last_unstale":"2024-09-16T07:34:48.538297+0000","last_undegraded":"2024-09-16T07:34:48.538297+0000","last_fullsized":"2024-09-16T07:34:48.538297+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:27.517094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.528190+0000","last_change":"2024-09-16T07:34:47.527047+0000","last_active":"2024-09-16T07:34:47.528190+0000","last_peered":"2024-09-16T07:34:47.528190+0000","last_clean":"2024-09-16T07:34:47.528190+0000","last_became_active":"2024-09-16T07:34:47.526733+0000","last_became_peered":"2024-09-16T07:34:47.526733+0000","last_unstale":"2024-09-16T07:34:47.528190+0000","last_undegraded":"2024-09-16T07:34:47.528190+0000","last_fullsized":"2024-09-16T07:34:47.528190+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.388814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534332+0000","last_change":"2024-09-16T07:34:48.530402+0000","last_active":"2024-09-16T07:34:48.534332+0000","last_peered":"2024-09-16T07:34:48.534332+0000","last_clean":"2024-09-16T07:34:48.534332+0000","last_became_active":"2024-09-16T07:34:48.530123+0000","last_became_peered":"2024-09-16T07:34:48.530123+0000","last_unstale":"2024-09-16T07:34:48.534332+0000","last_undegraded":"2024-09-16T07:34:48.534332+0000","last_fullsized":"2024-09-16T07:34:48.534332+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:02.344047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531255+0000","last_change":"2024-09-16T07:34:47.521688+0000","last_active":"2024-09-16T07:34:49.531255+0000","last_peered":"2024-09-16T07:34:49.531255+0000","last_clean":"2024-09-16T07:34:49.531255+0000","last_became_active":"2024-09-16T07:34:47.521350+0000","last_became_peered":"2024-09-16T07:34:47.521350+0000","last_unstale":"2024-09-16T07:34:49.531255+0000","last_undegraded":"2024-09-16T07:34:49.531255+0000","last_fullsized":"2024-09-16T07:34:49.531255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:10.584170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.536928+0000","last_change":"2024-09-16T07:34:48.532035+0000","last_active":"2024-09-16T07:34:48.536928+0000","last_peered":"2024-09-16T07:34:48.536928+0000","last_clean":"2024-09-16T07:34:48.536928+0000","last_became_active":"2024-09-16T07:34:48.527917+0000","last_became_peered":"2024-09-16T07:34:48.527917+0000","last_unstale":"2024-09-16T07:34:48.536928+0000","last_undegraded":"2024-09-16T07:34:48.536928+0000","last_fullsized":"2024-09-16T07:34:48.536928+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:14.371171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530405+0000","last_change":"2024-09-16T07:34:47.529347+0000","last_active":"2024-09-16T07:34:47.530405+0000","last_peered":"2024-09-16T07:34:47.530405+0000","last_clean":"2024-09-16T07:34:47.530405+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:47.530405+0000","last_undegraded":"2024-09-16T07:34:47.530405+0000","last_fullsized":"2024-09-16T07:34:47.530405+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:11.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505542+0000","last_change":"2024-09-16T07:34:47.505542+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505542+0000","last_undegraded":"2024-09-16T07:34:47.505542+0000","last_fullsized":"2024-09-16T07:34:47.505542+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530531+0000","last_change":"2024-09-16T07:34:47.529669+0000","last_active":"2024-09-16T07:34:47.530531+0000","last_peered":"2024-09-16T07:34:47.530531+0000","last_clean":"2024-09-16T07:34:47.530531+0000","last_became_active":"2024-09-16T07:34:47.529378+0000","last_became_peered":"2024-09-16T07:34:47.529378+0000","last_unstale":"2024-09-16T07:34:47.530531+0000","last_undegraded":"2024-09-16T07:34:47.530531+0000","last_fullsized":"2024-09-16T07:34:47.530531+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:58.330923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.531202+0000","last_change":"2024-09-16T07:34:48.525029+0000","last_active":"2024-09-16T07:34:48.531202+0000","last_peered":"2024-09-16T07:34:48.531202+0000","last_clean":"2024-09-16T07:34:48.531202+0000","last_became_active":"2024-09-16T07:34:48.524129+0000","last_became_peered":"2024-09-16T07:34:48.524129+0000","last_unstale":"2024-09-16T07:34:48.531202+0000","last_undegraded":"2024-09-16T07:34:48.531202+0000","last_fullsized":"2024-09-16T07:34:48.531202+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:10.118549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"118'6","reported_seq":21,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531542+0000","last_change":"2024-09-16T07:34:47.524021+0000","last_active":"2024-09-16T07:34:49.531542+0000","last_peered":"2024-09-16T07:34:49.531542+0000","last_clean":"2024-09-16T07:34:49.531542+0000","last_became_active":"2024-09-16T07:34:47.520956+0000","last_became_peered":"2024-09-16T07:34:47.520956+0000","last_unstale":"2024-09-16T07:34:49.531542+0000","last_undegraded":"2024-09-16T07:34:49.531542+0000","last_fullsized":"2024-09-16T07:34:49.531542+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:42.661372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.514437+0000","last_change":"2024-09-16T07:34:47.514437+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.514437+0000","last_undegraded":"2024-09-16T07:34:47.514437+0000","last_fullsized":"2024-09-16T07:34:47.514437+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.529914+0000","last_change":"2024-09-16T07:34:47.528845+0000","last_active":"2024-09-16T07:34:47.529914+0000","last_peered":"2024-09-16T07:34:47.529914+0000","last_clean":"2024-09-16T07:34:47.529914+0000","last_became_active":"2024-09-16T07:34:47.528568+0000","last_became_peered":"2024-09-16T07:34:47.528568+0000","last_unstale":"2024-09-16T07:34:47.529914+0000","last_undegraded":"2024-09-16T07:34:47.529914+0000","last_fullsized":"2024-09-16T07:34:47.529914+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.497545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531719+0000","last_change":"2024-09-16T07:34:48.525677+0000","last_active":"2024-09-16T07:34:49.531719+0000","last_peered":"2024-09-16T07:34:49.531719+0000","last_clean":"2024-09-16T07:34:49.531719+0000","last_became_active":"2024-09-16T07:34:48.522263+0000","last_became_peered":"2024-09-16T07:34:48.522263+0000","last_unstale":"2024-09-16T07:34:49.531719+0000","last_undegraded":"2024-09-16T07:34:49.531719+0000","last_fullsized":"2024-09-16T07:34:49.531719+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:28.526205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.508493+0000","last_change":"2024-09-16T07:34:47.523404+0000","last_active":"2024-09-16T07:34:48.508493+0000","last_peered":"2024-09-16T07:34:48.508493+0000","last_clean":"2024-09-16T07:34:48.508493+0000","last_became_active":"2024-09-16T07:34:47.523140+0000","last_became_peered":"2024-09-16T07:34:47.523140+0000","last_unstale":"2024-09-16T07:34:48.508493+0000","last_undegraded":"2024-09-16T07:34:48.508493+0000","last_fullsized":"2024-09-16T07:34:48.508493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:28.805823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531011+0000","last_change":"2024-09-16T07:34:48.530763+0000","last_active":"2024-09-16T07:34:49.531011+0000","last_peered":"2024-09-16T07:34:49.531011+0000","last_clean":"2024-09-16T07:34:49.531011+0000","last_became_active":"2024-09-16T07:34:48.530263+0000","last_became_peered":"2024-09-16T07:34:48.530263+0000","last_unstale":"2024-09-16T07:34:49.531011+0000","last_undegraded":"2024-09-16T07:34:49.531011+0000","last_fullsized":"2024-09-16T07:34:49.531011+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:16.945235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.527329+0000","last_change":"2024-09-16T07:34:47.524692+0000","last_active":"2024-09-16T07:34:47.527329+0000","last_peered":"2024-09-16T07:34:47.527329+0000","last_clean":"2024-09-16T07:34:47.527329+0000","last_became_active":"2024-09-16T07:34:47.524434+0000","last_became_peered":"2024-09-16T07:34:47.524434+0000","last_unstale":"2024-09-16T07:34:47.527329+0000","last_undegraded":"2024-09-16T07:34:47.527329+0000","last_fullsized":"2024-09-16T07:34:47.527329+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:33.456945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.512413+0000","last_change":"2024-09-16T07:34:47.512413+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.512413+0000","last_undegraded":"2024-09-16T07:34:47.512413+0000","last_fullsized":"2024-09-16T07:34:47.512413+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.517308+0000","last_change":"2024-09-16T07:34:47.515933+0000","last_active":"2024-09-16T07:34:47.517308+0000","last_peered":"2024-09-16T07:34:47.517308+0000","last_clean":"2024-09-16T07:34:47.517308+0000","last_became_active":"2024-09-16T07:34:47.515670+0000","last_became_peered":"2024-09-16T07:34:47.515670+0000","last_unstale":"2024-09-16T07:34:47.517308+0000","last_undegraded":"2024-09-16T07:34:47.517308+0000","last_fullsized":"2024-09-16T07:34:47.517308+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:40.298789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.517983+0000","last_change":"2024-09-16T07:34:47.517983+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.517983+0000","last_undegraded":"2024-09-16T07:34:47.517983+0000","last_fullsized":"2024-09-16T07:34:47.517983+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.529910+0000","last_change":"2024-09-16T07:34:47.528922+0000","last_active":"2024-09-16T07:34:47.529910+0000","last_peered":"2024-09-16T07:34:47.529910+0000","last_clean":"2024-09-16T07:34:47.529910+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:47.529910+0000","last_undegraded":"2024-09-16T07:34:47.529910+0000","last_fullsized":"2024-09-16T07:34:47.529910+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:04:45.751140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534692+0000","last_change":"2024-09-16T07:34:48.528054+0000","last_active":"2024-09-16T07:34:48.534692+0000","last_peered":"2024-09-16T07:34:48.534692+0000","last_clean":"2024-09-16T07:34:48.534692+0000","last_became_active":"2024-09-16T07:34:48.527775+0000","last_became_peered":"2024-09-16T07:34:48.527775+0000","last_unstale":"2024-09-16T07:34:48.534692+0000","last_undegraded":"2024-09-16T07:34:48.534692+0000","last_fullsized":"2024-09-16T07:34:48.534692+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:51.806935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.543374+0000","last_change":"2024-09-16T07:34:47.527129+0000","last_active":"2024-09-16T07:34:48.543374+0000","last_peered":"2024-09-16T07:34:48.543374+0000","last_clean":"2024-09-16T07:34:48.543374+0000","last_became_active":"2024-09-16T07:34:47.526046+0000","last_became_peered":"2024-09-16T07:34:47.526046+0000","last_unstale":"2024-09-16T07:34:48.543374+0000","last_undegraded":"2024-09-16T07:34:48.543374+0000","last_fullsized":"2024-09-16T07:34:48.543374+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:40.775080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.524666+0000","last_change":"2024-09-16T07:34:48.522447+0000","last_active":"2024-09-16T07:34:48.524666+0000","last_peered":"2024-09-16T07:34:48.524666+0000","last_clean":"2024-09-16T07:34:48.524666+0000","last_became_active":"2024-09-16T07:34:48.521166+0000","last_became_peered":"2024-09-16T07:34:48.521166+0000","last_unstale":"2024-09-16T07:34:48.524666+0000","last_undegraded":"2024-09-16T07:34:48.524666+0000","last_fullsized":"2024-09-16T07:34:48.524666+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:34.127356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.516801+0000","last_change":"2024-09-16T07:34:47.522843+0000","last_active":"2024-09-16T07:34:48.516801+0000","last_peered":"2024-09-16T07:34:48.516801+0000","last_clean":"2024-09-16T07:34:48.516801+0000","last_became_active":"2024-09-16T07:34:47.522587+0000","last_became_peered":"2024-09-16T07:34:47.522587+0000","last_unstale":"2024-09-16T07:34:48.516801+0000","last_undegraded":"2024-09-16T07:34:48.516801+0000","last_fullsized":"2024-09-16T07:34:48.516801+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:26.555192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533092+0000","last_change":"2024-09-16T07:34:48.528425+0000","last_active":"2024-09-16T07:34:48.533092+0000","last_peered":"2024-09-16T07:34:48.533092+0000","last_clean":"2024-09-16T07:34:48.533092+0000","last_became_active":"2024-09-16T07:34:48.526307+0000","last_became_peered":"2024-09-16T07:34:48.526307+0000","last_unstale":"2024-09-16T07:34:48.533092+0000","last_undegraded":"2024-09-16T07:34:48.533092+0000","last_fullsized":"2024-09-16T07:34:48.533092+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:27.397201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530294+0000","last_change":"2024-09-16T07:34:47.529098+0000","last_active":"2024-09-16T07:34:47.530294+0000","last_peered":"2024-09-16T07:34:47.530294+0000","last_clean":"2024-09-16T07:34:47.530294+0000","last_became_active":"2024-09-16T07:34:47.528650+0000","last_became_peered":"2024-09-16T07:34:47.528650+0000","last_unstale":"2024-09-16T07:34:47.530294+0000","last_undegraded":"2024-09-16T07:34:47.530294+0000","last_fullsized":"2024-09-16T07:34:47.530294+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:15:47.135693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.517824+0000","last_change":"2024-09-16T07:34:47.517824+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.517824+0000","last_undegraded":"2024-09-16T07:34:47.517824+0000","last_fullsized":"2024-09-16T07:34:47.517824+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.526192+0000","last_change":"2024-09-16T07:34:47.521880+0000","last_active":"2024-09-16T07:34:47.526192+0000","last_peered":"2024-09-16T07:34:47.526192+0000","last_clean":"2024-09-16T07:34:47.526192+0000","last_became_active":"2024-09-16T07:34:47.521610+0000","last_became_peered":"2024-09-16T07:34:47.521610+0000","last_unstale":"2024-09-16T07:34:47.526192+0000","last_undegraded":"2024-09-16T07:34:47.526192+0000","last_fullsized":"2024-09-16T07:34:47.526192+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:27.814061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535109+0000","last_change":"2024-09-16T07:34:48.530713+0000","last_active":"2024-09-16T07:34:48.535109+0000","last_peered":"2024-09-16T07:34:48.535109+0000","last_clean":"2024-09-16T07:34:48.535109+0000","last_became_active":"2024-09-16T07:34:48.526504+0000","last_became_peered":"2024-09-16T07:34:48.526504+0000","last_unstale":"2024-09-16T07:34:48.535109+0000","last_undegraded":"2024-09-16T07:34:48.535109+0000","last_fullsized":"2024-09-16T07:34:48.535109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:56.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530916+0000","last_change":"2024-09-16T07:34:47.518969+0000","last_active":"2024-09-16T07:34:49.530916+0000","last_peered":"2024-09-16T07:34:49.530916+0000","last_clean":"2024-09-16T07:34:49.530916+0000","last_became_active":"2024-09-16T07:34:47.518693+0000","last_became_peered":"2024-09-16T07:34:47.518693+0000","last_unstale":"2024-09-16T07:34:49.530916+0000","last_undegraded":"2024-09-16T07:34:49.530916+0000","last_fullsized":"2024-09-16T07:34:49.530916+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:33.694870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516386+0000","last_change":"2024-09-16T07:34:47.516386+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516386+0000","last_undegraded":"2024-09-16T07:34:47.516386+0000","last_fullsized":"2024-09-16T07:34:47.516386+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530779+0000","last_change":"2024-09-16T07:34:47.517277+0000","last_active":"2024-09-16T07:34:49.530779+0000","last_peered":"2024-09-16T07:34:49.530779+0000","last_clean":"2024-09-16T07:34:49.530779+0000","last_became_active":"2024-09-16T07:34:47.516911+0000","last_became_peered":"2024-09-16T07:34:47.516911+0000","last_unstale":"2024-09-16T07:34:49.530779+0000","last_undegraded":"2024-09-16T07:34:49.530779+0000","last_fullsized":"2024-09-16T07:34:49.530779+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:31:40.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.527395+0000","last_change":"2024-09-16T07:34:48.526046+0000","last_active":"2024-09-16T07:34:48.527395+0000","last_peered":"2024-09-16T07:34:48.527395+0000","last_clean":"2024-09-16T07:34:48.527395+0000","last_became_active":"2024-09-16T07:34:48.525684+0000","last_became_peered":"2024-09-16T07:34:48.525684+0000","last_unstale":"2024-09-16T07:34:48.527395+0000","last_undegraded":"2024-09-16T07:34:48.527395+0000","last_fullsized":"2024-09-16T07:34:48.527395+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:50.191171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531230+0000","last_change":"2024-09-16T07:34:47.521736+0000","last_active":"2024-09-16T07:34:49.531230+0000","last_peered":"2024-09-16T07:34:49.531230+0000","last_clean":"2024-09-16T07:34:49.531230+0000","last_became_active":"2024-09-16T07:34:47.521422+0000","last_became_peered":"2024-09-16T07:34:47.521422+0000","last_unstale":"2024-09-16T07:34:49.531230+0000","last_undegraded":"2024-09-16T07:34:49.531230+0000","last_fullsized":"2024-09-16T07:34:49.531230+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:02.620783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505633+0000","last_change":"2024-09-16T07:34:47.505633+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505633+0000","last_undegraded":"2024-09-16T07:34:47.505633+0000","last_fullsized":"2024-09-16T07:34:47.505633+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.518493+0000","last_change":"2024-09-16T07:34:47.511494+0000","last_active":"2024-09-16T07:34:48.518493+0000","last_peered":"2024-09-16T07:34:48.518493+0000","last_clean":"2024-09-16T07:34:48.518493+0000","last_became_active":"2024-09-16T07:34:47.511196+0000","last_became_peered":"2024-09-16T07:34:47.511196+0000","last_unstale":"2024-09-16T07:34:48.518493+0000","last_undegraded":"2024-09-16T07:34:48.518493+0000","last_fullsized":"2024-09-16T07:34:48.518493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:18.504885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.512300+0000","last_change":"2024-09-16T07:34:47.512300+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.512300+0000","last_undegraded":"2024-09-16T07:34:47.512300+0000","last_fullsized":"2024-09-16T07:34:47.512300+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531451+0000","last_change":"2024-09-16T07:34:47.516251+0000","last_active":"2024-09-16T07:34:49.531451+0000","last_peered":"2024-09-16T07:34:49.531451+0000","last_clean":"2024-09-16T07:34:49.531451+0000","last_became_active":"2024-09-16T07:34:47.515980+0000","last_became_peered":"2024-09-16T07:34:47.515980+0000","last_unstale":"2024-09-16T07:34:49.531451+0000","last_undegraded":"2024-09-16T07:34:49.531451+0000","last_fullsized":"2024-09-16T07:34:49.531451+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:16.654899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.542525+0000","last_change":"2024-09-16T07:34:48.542025+0000","last_active":"2024-09-16T07:34:48.542525+0000","last_peered":"2024-09-16T07:34:48.542525+0000","last_clean":"2024-09-16T07:34:48.542525+0000","last_became_active":"2024-09-16T07:34:48.541713+0000","last_became_peered":"2024-09-16T07:34:48.541713+0000","last_unstale":"2024-09-16T07:34:48.542525+0000","last_undegraded":"2024-09-16T07:34:48.542525+0000","last_fullsized":"2024-09-16T07:34:48.542525+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:03.040043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531609+0000","last_change":"2024-09-16T07:34:47.513397+0000","last_active":"2024-09-16T07:34:49.531609+0000","last_peered":"2024-09-16T07:34:49.531609+0000","last_clean":"2024-09-16T07:34:49.531609+0000","last_became_active":"2024-09-16T07:34:47.513084+0000","last_became_peered":"2024-09-16T07:34:47.513084+0000","last_unstale":"2024-09-16T07:34:49.531609+0000","last_undegraded":"2024-09-16T07:34:49.531609+0000","last_fullsized":"2024-09-16T07:34:49.531609+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:49.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533116+0000","last_change":"2024-09-16T07:34:48.526515+0000","last_active":"2024-09-16T07:34:48.533116+0000","last_peered":"2024-09-16T07:34:48.533116+0000","last_clean":"2024-09-16T07:34:48.533116+0000","last_became_active":"2024-09-16T07:34:48.525964+0000","last_became_peered":"2024-09-16T07:34:48.525964+0000","last_unstale":"2024-09-16T07:34:48.533116+0000","last_undegraded":"2024-09-16T07:34:48.533116+0000","last_fullsized":"2024-09-16T07:34:48.533116+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:11.702095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.520393+0000","last_change":"2024-09-16T07:34:47.518568+0000","last_active":"2024-09-16T07:34:48.520393+0000","last_peered":"2024-09-16T07:34:48.520393+0000","last_clean":"2024-09-16T07:34:48.520393+0000","last_became_active":"2024-09-16T07:34:47.518279+0000","last_became_peered":"2024-09-16T07:34:47.518279+0000","last_unstale":"2024-09-16T07:34:48.520393+0000","last_undegraded":"2024-09-16T07:34:48.520393+0000","last_fullsized":"2024-09-16T07:34:48.520393+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:36.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534614+0000","last_change":"2024-09-16T07:34:48.528073+0000","last_active":"2024-09-16T07:34:48.534614+0000","last_peered":"2024-09-16T07:34:48.534614+0000","last_clean":"2024-09-16T07:34:48.534614+0000","last_became_active":"2024-09-16T07:34:48.527807+0000","last_became_peered":"2024-09-16T07:34:48.527807+0000","last_unstale":"2024-09-16T07:34:48.534614+0000","last_undegraded":"2024-09-16T07:34:48.534614+0000","last_fullsized":"2024-09-16T07:34:48.534614+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:22.775050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.514866+0000","last_change":"2024-09-16T07:34:47.527175+0000","last_active":"2024-09-16T07:34:48.514866+0000","last_peered":"2024-09-16T07:34:48.514866+0000","last_clean":"2024-09-16T07:34:48.514866+0000","last_became_active":"2024-09-16T07:34:47.526929+0000","last_became_peered":"2024-09-16T07:34:47.526929+0000","last_unstale":"2024-09-16T07:34:48.514866+0000","last_undegraded":"2024-09-16T07:34:48.514866+0000","last_fullsized":"2024-09-16T07:34:48.514866+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:56.405916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.527000+0000","last_change":"2024-09-16T07:34:48.524675+0000","last_active":"2024-09-16T07:34:48.527000+0000","last_peered":"2024-09-16T07:34:48.527000+0000","last_clean":"2024-09-16T07:34:48.527000+0000","last_became_active":"2024-09-16T07:34:48.524324+0000","last_became_peered":"2024-09-16T07:34:48.524324+0000","last_unstale":"2024-09-16T07:34:48.527000+0000","last_undegraded":"2024-09-16T07:34:48.527000+0000","last_fullsized":"2024-09-16T07:34:48.527000+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:55.577009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.527035+0000","last_change":"2024-09-16T07:34:47.525571+0000","last_active":"2024-09-16T07:34:47.527035+0000","last_peered":"2024-09-16T07:34:47.527035+0000","last_clean":"2024-09-16T07:34:47.527035+0000","last_became_active":"2024-09-16T07:34:47.525295+0000","last_became_peered":"2024-09-16T07:34:47.525295+0000","last_unstale":"2024-09-16T07:34:47.527035+0000","last_undegraded":"2024-09-16T07:34:47.527035+0000","last_fullsized":"2024-09-16T07:34:47.527035+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:53.256795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.506924+0000","last_change":"2024-09-16T07:34:47.506924+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.506924+0000","last_undegraded":"2024-09-16T07:34:47.506924+0000","last_fullsized":"2024-09-16T07:34:47.506924+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.520862+0000","last_change":"2024-09-16T07:34:47.518753+0000","last_active":"2024-09-16T07:34:47.520862+0000","last_peered":"2024-09-16T07:34:47.520862+0000","last_clean":"2024-09-16T07:34:47.520862+0000","last_became_active":"2024-09-16T07:34:47.518486+0000","last_became_peered":"2024-09-16T07:34:47.518486+0000","last_unstale":"2024-09-16T07:34:47.520862+0000","last_undegraded":"2024-09-16T07:34:47.520862+0000","last_fullsized":"2024-09-16T07:34:47.520862+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:21.437105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531155+0000","last_change":"2024-09-16T07:34:48.533301+0000","last_active":"2024-09-16T07:34:49.531155+0000","last_peered":"2024-09-16T07:34:49.531155+0000","last_clean":"2024-09-16T07:34:49.531155+0000","last_became_active":"2024-09-16T07:34:48.532697+0000","last_became_peered":"2024-09-16T07:34:48.532697+0000","last_unstale":"2024-09-16T07:34:49.531155+0000","last_undegraded":"2024-09-16T07:34:49.531155+0000","last_fullsized":"2024-09-16T07:34:49.531155+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:14:10.393373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.520135+0000","last_change":"2024-09-16T07:34:47.521842+0000","last_active":"2024-09-16T07:34:48.520135+0000","last_peered":"2024-09-16T07:34:48.520135+0000","last_clean":"2024-09-16T07:34:48.520135+0000","last_became_active":"2024-09-16T07:34:47.521578+0000","last_became_peered":"2024-09-16T07:34:47.521578+0000","last_unstale":"2024-09-16T07:34:48.520135+0000","last_undegraded":"2024-09-16T07:34:48.520135+0000","last_fullsized":"2024-09-16T07:34:48.520135+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:20.091370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":288,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.507819+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:34:48.507819+0000","last_peered":"2024-09-16T07:34:48.507819+0000","last_clean":"2024-09-16T07:34:48.507819+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:34:48.507819+0000","last_undegraded":"2024-09-16T07:34:48.507819+0000","last_fullsized":"2024-09-16T07:34:48.507819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2264,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126944,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244896,"statfs":{"total":96636764160,"available":96506773504,"internally_reserved":0,"allocated":1454080,"data_stored":635315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126932,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244908,"statfs":{"total":96636764160,"available":96506785792,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123352,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248488,"statfs":{"total":96636764160,"available":96510451712,"internally_reserved":0,"allocated":2035712,"data_stored":1218852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140820,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231020,"statfs":{"total":96636764160,"available":96492564480,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122768,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249072,"statfs":{"total":96636764160,"available":96511049728,"internally_reserved":0,"allocated":1429504,"data_stored":622153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509590,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153308,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218532,"statfs":{"total":96636764160,"available":96479776768,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131672,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240168,"statfs":{"total":96636764160,"available":96501932032,"internally_reserved":0,"allocated":2035712,"data_stored":1218852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126976,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244864,"statfs":{"total":96636764160,"available":96506740736,"internally_reserved":0,"allocated":1482752,"data_stored":643373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:34:53.137 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:34:53.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.331+0000 7fda1ac00640 1 -- 172.21.15.71:0/198049356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda140ccf00 msgr2=0x7fda140b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.331+0000 7fda1ac00640 1 --2- 172.21.15.71:0/198049356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda140ccf00 0x7fda140b2440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fda00009830 tx=0x7fda0002a490 comp rx=0 tx=0).stop 2024-09-16T07:34:53.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.331+0000 7fda1ac00640 1 -- 172.21.15.71:0/198049356 shutdown_connections 2024-09-16T07:34:53.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.331+0000 7fda1ac00640 1 --2- 172.21.15.71:0/198049356 >> v2:172.21.15.177:3300/0 conn(0x7fda14073fb0 0x7fda14074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.331+0000 7fda1ac00640 1 --2- 172.21.15.71:0/198049356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda140ccf00 0x7fda140b2440 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.331+0000 7fda1ac00640 1 -- 172.21.15.71:0/198049356 >> 172.21.15.71:0/198049356 conn(0x7fda140b1950 msgr2=0x7fda140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:53.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.331+0000 7fda1ac00640 1 -- 172.21.15.71:0/198049356 shutdown_connections 2024-09-16T07:34:53.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.331+0000 7fda1ac00640 1 -- 172.21.15.71:0/198049356 wait complete. 2024-09-16T07:34:53.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.333+0000 7fda1ac00640 1 Processor -- start 2024-09-16T07:34:53.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.333+0000 7fda1ac00640 1 -- start start 2024-09-16T07:34:53.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda1ac00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda14073fb0 0x7fda1414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:53.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda1ac00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda140ccf00 0x7fda1414c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda1ac00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda1415f5a0 0x7fda140b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda1ac00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fda1406f2e0 con 0x7fda140ccf00 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda1ac00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda1406f160 con 0x7fda14073fb0 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda1ac00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda1406f460 con 0x7fda1415f5a0 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda140ccf00 0x7fda1414c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda140ccf00 0x7fda1414c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34608/0 (socket says 172.21.15.71:34608) 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13400640 1 -- 172.21.15.71:0/733161944 learned_addr learned my addr 172.21.15.71:0/733161944 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13e00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda14073fb0 0x7fda1414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:53.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda18c00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda1415f5a0 0x7fda140b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13400640 1 -- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda1415f5a0 msgr2=0x7fda140b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13400640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda1415f5a0 0x7fda140b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13400640 1 -- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda14073fb0 msgr2=0x7fda1414d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13400640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda14073fb0 0x7fda1414d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.334+0000 7fda13400640 1 -- 172.21.15.71:0/733161944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1414dc50 con 0x7fda140ccf00 2024-09-16T07:34:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.335+0000 7fda13400640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda140ccf00 0x7fda1414c680 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fda00000c00 tx=0x7fda00032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.335+0000 7fda0be00640 1 -- 172.21.15.71:0/733161944 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00040070 con 0x7fda140ccf00 2024-09-16T07:34:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.335+0000 7fda0be00640 1 -- 172.21.15.71:0/733161944 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda000038e0 con 0x7fda140ccf00 2024-09-16T07:34:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.335+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda141f5010 con 0x7fda140ccf00 2024-09-16T07:34:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.335+0000 7fda0be00640 1 -- 172.21.15.71:0/733161944 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00003d30 con 0x7fda140ccf00 2024-09-16T07:34:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.335+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141f58b0 con 0x7fda140ccf00 2024-09-16T07:34:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.336+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda141f5400 con 0x7fda140ccf00 2024-09-16T07:34:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.337+0000 7fda0be00640 1 -- 172.21.15.71:0/733161944 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fda00039040 con 0x7fda140ccf00 2024-09-16T07:34:53.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.338+0000 7fda0be00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9f407a230 0x7fd9f407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:53.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.338+0000 7fda0be00640 1 -- 172.21.15.71:0/733161944 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fda000babb0 con 0x7fda140ccf00 2024-09-16T07:34:53.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.339+0000 7fda13e00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9f407a230 0x7fd9f407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:53.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.339+0000 7fda13e00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9f407a230 0x7fd9f407c6f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fda04001e20 tx=0x7fda04009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:53.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.343+0000 7fda0be00640 1 -- 172.21.15.71:0/733161944 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fda00084630 con 0x7fda140ccf00 2024-09-16T07:34:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.524+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda1406f9f0 con 0x7fd9f407a230 2024-09-16T07:34:53.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.529+0000 7fda0be00640 1 -- 172.21.15.71:0/733161944 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250404 (secure 0 0 0) 0x7fda1406f9f0 con 0x7fd9f407a230 2024-09-16T07:34:53.532 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:53.533 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:34:53.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9f407a230 msgr2=0x7fd9f407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9f407a230 0x7fd9f407c6f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fda04001e20 tx=0x7fda04009040 comp rx=0 tx=0).stop 2024-09-16T07:34:53.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda140ccf00 msgr2=0x7fda1414c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda140ccf00 0x7fda1414c680 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fda00000c00 tx=0x7fda00032c10 comp rx=0 tx=0).stop 2024-09-16T07:34:53.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 shutdown_connections 2024-09-16T07:34:53.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd9f407a230 0x7fd9f407c6f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda1415f5a0 0x7fda140b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda140ccf00 0x7fda1414c680 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 --2- 172.21.15.71:0/733161944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda14073fb0 0x7fda1414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 >> 172.21.15.71:0/733161944 conn(0x7fda140b1950 msgr2=0x7fda1415e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:53.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 shutdown_connections 2024-09-16T07:34:53.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.533+0000 7fda1ac00640 1 -- 172.21.15.71:0/733161944 wait complete. 2024-09-16T07:34:53.549 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-16T07:34:52.604187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592632,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1052772,"kb_used_data":12464,"kb_used_omap":12,"kb_used_meta":1040115,"kb_avail":753921948,"statfs":{"total":773094113280,"available":772016074752,"internally_reserved":0,"allocated":12763136,"data_stored":6223997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1065078320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2264,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001350"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.519868+0000","last_change":"2024-09-16T07:34:47.519868+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.519868+0000","last_undegraded":"2024-09-16T07:34:47.519868+0000","last_fullsized":"2024-09-16T07:34:47.519868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.542589+0000","last_change":"2024-09-16T07:34:48.541678+0000","last_active":"2024-09-16T07:34:48.542589+0000","last_peered":"2024-09-16T07:34:48.542589+0000","last_clean":"2024-09-16T07:34:48.542589+0000","last_became_active":"2024-09-16T07:34:48.541360+0000","last_became_peered":"2024-09-16T07:34:48.541360+0000","last_unstale":"2024-09-16T07:34:48.542589+0000","last_undegraded":"2024-09-16T07:34:48.542589+0000","last_fullsized":"2024-09-16T07:34:48.542589+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:42.661091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535282+0000","last_change":"2024-09-16T07:34:48.534129+0000","last_active":"2024-09-16T07:34:48.535282+0000","last_peered":"2024-09-16T07:34:48.535282+0000","last_clean":"2024-09-16T07:34:48.535282+0000","last_became_active":"2024-09-16T07:34:48.533584+0000","last_became_peered":"2024-09-16T07:34:48.533584+0000","last_unstale":"2024-09-16T07:34:48.535282+0000","last_undegraded":"2024-09-16T07:34:48.535282+0000","last_fullsized":"2024-09-16T07:34:48.535282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:57.899237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531563+0000","last_change":"2024-09-16T07:34:48.526281+0000","last_active":"2024-09-16T07:34:49.531563+0000","last_peered":"2024-09-16T07:34:49.531563+0000","last_clean":"2024-09-16T07:34:49.531563+0000","last_became_active":"2024-09-16T07:34:48.525843+0000","last_became_peered":"2024-09-16T07:34:48.525843+0000","last_unstale":"2024-09-16T07:34:49.531563+0000","last_undegraded":"2024-09-16T07:34:49.531563+0000","last_fullsized":"2024-09-16T07:34:49.531563+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:50.039555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513726+0000","last_change":"2024-09-16T07:34:47.513726+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513726+0000","last_undegraded":"2024-09-16T07:34:47.513726+0000","last_fullsized":"2024-09-16T07:34:47.513726+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.507372+0000","last_change":"2024-09-16T07:34:47.507372+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.507372+0000","last_undegraded":"2024-09-16T07:34:47.507372+0000","last_fullsized":"2024-09-16T07:34:47.507372+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534985+0000","last_change":"2024-09-16T07:34:48.526151+0000","last_active":"2024-09-16T07:34:48.534985+0000","last_peered":"2024-09-16T07:34:48.534985+0000","last_clean":"2024-09-16T07:34:48.534985+0000","last_became_active":"2024-09-16T07:34:48.525914+0000","last_became_peered":"2024-09-16T07:34:48.525914+0000","last_unstale":"2024-09-16T07:34:48.534985+0000","last_undegraded":"2024-09-16T07:34:48.534985+0000","last_fullsized":"2024-09-16T07:34:48.534985+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:31.962809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535012+0000","last_change":"2024-09-16T07:34:48.528915+0000","last_active":"2024-09-16T07:34:48.535012+0000","last_peered":"2024-09-16T07:34:48.535012+0000","last_clean":"2024-09-16T07:34:48.535012+0000","last_became_active":"2024-09-16T07:34:48.528648+0000","last_became_peered":"2024-09-16T07:34:48.528648+0000","last_unstale":"2024-09-16T07:34:48.535012+0000","last_undegraded":"2024-09-16T07:34:48.535012+0000","last_fullsized":"2024-09-16T07:34:48.535012+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:49.801672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.519440+0000","last_change":"2024-09-16T07:34:47.519440+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.519440+0000","last_undegraded":"2024-09-16T07:34:47.519440+0000","last_fullsized":"2024-09-16T07:34:47.519440+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.514995+0000","last_change":"2024-09-16T07:34:47.514995+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.514995+0000","last_undegraded":"2024-09-16T07:34:47.514995+0000","last_fullsized":"2024-09-16T07:34:47.514995+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534388+0000","last_change":"2024-09-16T07:34:48.530703+0000","last_active":"2024-09-16T07:34:48.534388+0000","last_peered":"2024-09-16T07:34:48.534388+0000","last_clean":"2024-09-16T07:34:48.534388+0000","last_became_active":"2024-09-16T07:34:48.530406+0000","last_became_peered":"2024-09-16T07:34:48.530406+0000","last_unstale":"2024-09-16T07:34:48.534388+0000","last_undegraded":"2024-09-16T07:34:48.534388+0000","last_fullsized":"2024-09-16T07:34:48.534388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:24.273074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.515387+0000","last_change":"2024-09-16T07:34:47.515387+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.515387+0000","last_undegraded":"2024-09-16T07:34:47.515387+0000","last_fullsized":"2024-09-16T07:34:47.515387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.538475+0000","last_change":"2024-09-16T07:34:48.535432+0000","last_active":"2024-09-16T07:34:48.538475+0000","last_peered":"2024-09-16T07:34:48.538475+0000","last_clean":"2024-09-16T07:34:48.538475+0000","last_became_active":"2024-09-16T07:34:48.532194+0000","last_became_peered":"2024-09-16T07:34:48.532194+0000","last_unstale":"2024-09-16T07:34:48.538475+0000","last_undegraded":"2024-09-16T07:34:48.538475+0000","last_fullsized":"2024-09-16T07:34:48.538475+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:02:06.782991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535669+0000","last_change":"2024-09-16T07:34:48.531769+0000","last_active":"2024-09-16T07:34:48.535669+0000","last_peered":"2024-09-16T07:34:48.535669+0000","last_clean":"2024-09-16T07:34:48.535669+0000","last_became_active":"2024-09-16T07:34:48.531251+0000","last_became_peered":"2024-09-16T07:34:48.531251+0000","last_unstale":"2024-09-16T07:34:48.535669+0000","last_undegraded":"2024-09-16T07:34:48.535669+0000","last_fullsized":"2024-09-16T07:34:48.535669+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:03.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513794+0000","last_change":"2024-09-16T07:34:47.513794+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513794+0000","last_undegraded":"2024-09-16T07:34:47.513794+0000","last_fullsized":"2024-09-16T07:34:47.513794+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516736+0000","last_change":"2024-09-16T07:34:47.516736+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516736+0000","last_undegraded":"2024-09-16T07:34:47.516736+0000","last_fullsized":"2024-09-16T07:34:47.516736+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.522422+0000","last_change":"2024-09-16T07:34:48.520786+0000","last_active":"2024-09-16T07:34:48.522422+0000","last_peered":"2024-09-16T07:34:48.522422+0000","last_clean":"2024-09-16T07:34:48.522422+0000","last_became_active":"2024-09-16T07:34:48.519475+0000","last_became_peered":"2024-09-16T07:34:48.519475+0000","last_unstale":"2024-09-16T07:34:48.522422+0000","last_undegraded":"2024-09-16T07:34:48.522422+0000","last_fullsized":"2024-09-16T07:34:48.522422+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:38.065118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535936+0000","last_change":"2024-09-16T07:34:48.527942+0000","last_active":"2024-09-16T07:34:48.535936+0000","last_peered":"2024-09-16T07:34:48.535936+0000","last_clean":"2024-09-16T07:34:48.535936+0000","last_became_active":"2024-09-16T07:34:48.527669+0000","last_became_peered":"2024-09-16T07:34:48.527669+0000","last_unstale":"2024-09-16T07:34:48.535936+0000","last_undegraded":"2024-09-16T07:34:48.535936+0000","last_fullsized":"2024-09-16T07:34:48.535936+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:56.714125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535262+0000","last_change":"2024-09-16T07:34:48.534101+0000","last_active":"2024-09-16T07:34:48.535262+0000","last_peered":"2024-09-16T07:34:48.535262+0000","last_clean":"2024-09-16T07:34:48.535262+0000","last_became_active":"2024-09-16T07:34:48.529574+0000","last_became_peered":"2024-09-16T07:34:48.529574+0000","last_unstale":"2024-09-16T07:34:48.535262+0000","last_undegraded":"2024-09-16T07:34:48.535262+0000","last_fullsized":"2024-09-16T07:34:48.535262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:30:49.545255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533243+0000","last_change":"2024-09-16T07:34:48.529587+0000","last_active":"2024-09-16T07:34:48.533243+0000","last_peered":"2024-09-16T07:34:48.533243+0000","last_clean":"2024-09-16T07:34:48.533243+0000","last_became_active":"2024-09-16T07:34:48.529304+0000","last_became_peered":"2024-09-16T07:34:48.529304+0000","last_unstale":"2024-09-16T07:34:48.533243+0000","last_undegraded":"2024-09-16T07:34:48.533243+0000","last_fullsized":"2024-09-16T07:34:48.533243+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:06:46.119567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534304+0000","last_change":"2024-09-16T07:34:48.533118+0000","last_active":"2024-09-16T07:34:48.534304+0000","last_peered":"2024-09-16T07:34:48.534304+0000","last_clean":"2024-09-16T07:34:48.534304+0000","last_became_active":"2024-09-16T07:34:48.532777+0000","last_became_peered":"2024-09-16T07:34:48.532777+0000","last_unstale":"2024-09-16T07:34:48.534304+0000","last_undegraded":"2024-09-16T07:34:48.534304+0000","last_fullsized":"2024-09-16T07:34:48.534304+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:02.406116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505332+0000","last_change":"2024-09-16T07:34:47.505332+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505332+0000","last_undegraded":"2024-09-16T07:34:47.505332+0000","last_fullsized":"2024-09-16T07:34:47.505332+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535054+0000","last_change":"2024-09-16T07:34:48.528984+0000","last_active":"2024-09-16T07:34:48.535054+0000","last_peered":"2024-09-16T07:34:48.535054+0000","last_clean":"2024-09-16T07:34:48.535054+0000","last_became_active":"2024-09-16T07:34:48.528481+0000","last_became_peered":"2024-09-16T07:34:48.528481+0000","last_unstale":"2024-09-16T07:34:48.535054+0000","last_undegraded":"2024-09-16T07:34:48.535054+0000","last_fullsized":"2024-09-16T07:34:48.535054+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:43.572025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530706+0000","last_change":"2024-09-16T07:34:48.533570+0000","last_active":"2024-09-16T07:34:49.530706+0000","last_peered":"2024-09-16T07:34:49.530706+0000","last_clean":"2024-09-16T07:34:49.530706+0000","last_became_active":"2024-09-16T07:34:48.533039+0000","last_became_peered":"2024-09-16T07:34:48.533039+0000","last_unstale":"2024-09-16T07:34:49.530706+0000","last_undegraded":"2024-09-16T07:34:49.530706+0000","last_fullsized":"2024-09-16T07:34:49.530706+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:00:09.699334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533344+0000","last_change":"2024-09-16T07:34:48.527254+0000","last_active":"2024-09-16T07:34:48.533344+0000","last_peered":"2024-09-16T07:34:48.533344+0000","last_clean":"2024-09-16T07:34:48.533344+0000","last_became_active":"2024-09-16T07:34:48.527013+0000","last_became_peered":"2024-09-16T07:34:48.527013+0000","last_unstale":"2024-09-16T07:34:48.533344+0000","last_undegraded":"2024-09-16T07:34:48.533344+0000","last_fullsized":"2024-09-16T07:34:48.533344+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:16.029488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505317+0000","last_change":"2024-09-16T07:34:47.505317+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505317+0000","last_undegraded":"2024-09-16T07:34:47.505317+0000","last_fullsized":"2024-09-16T07:34:47.505317+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531251+0000","last_change":"2024-09-16T07:34:48.541120+0000","last_active":"2024-09-16T07:34:49.531251+0000","last_peered":"2024-09-16T07:34:49.531251+0000","last_clean":"2024-09-16T07:34:49.531251+0000","last_became_active":"2024-09-16T07:34:48.540074+0000","last_became_peered":"2024-09-16T07:34:48.540074+0000","last_unstale":"2024-09-16T07:34:49.531251+0000","last_undegraded":"2024-09-16T07:34:49.531251+0000","last_fullsized":"2024-09-16T07:34:49.531251+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:53.929219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.507462+0000","last_change":"2024-09-16T07:34:47.507462+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.507462+0000","last_undegraded":"2024-09-16T07:34:47.507462+0000","last_fullsized":"2024-09-16T07:34:47.507462+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535813+0000","last_change":"2024-09-16T07:34:48.534695+0000","last_active":"2024-09-16T07:34:48.535813+0000","last_peered":"2024-09-16T07:34:48.535813+0000","last_clean":"2024-09-16T07:34:48.535813+0000","last_became_active":"2024-09-16T07:34:48.534435+0000","last_became_peered":"2024-09-16T07:34:48.534435+0000","last_unstale":"2024-09-16T07:34:48.535813+0000","last_undegraded":"2024-09-16T07:34:48.535813+0000","last_fullsized":"2024-09-16T07:34:48.535813+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:11.262607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534350+0000","last_change":"2024-09-16T07:34:48.525252+0000","last_active":"2024-09-16T07:34:48.534350+0000","last_peered":"2024-09-16T07:34:48.534350+0000","last_clean":"2024-09-16T07:34:48.534350+0000","last_became_active":"2024-09-16T07:34:48.524453+0000","last_became_peered":"2024-09-16T07:34:48.524453+0000","last_unstale":"2024-09-16T07:34:48.534350+0000","last_undegraded":"2024-09-16T07:34:48.534350+0000","last_fullsized":"2024-09-16T07:34:48.534350+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:18.325757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513371+0000","last_change":"2024-09-16T07:34:47.513371+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513371+0000","last_undegraded":"2024-09-16T07:34:47.513371+0000","last_fullsized":"2024-09-16T07:34:47.513371+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516259+0000","last_change":"2024-09-16T07:34:47.516259+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516259+0000","last_undegraded":"2024-09-16T07:34:47.516259+0000","last_fullsized":"2024-09-16T07:34:47.516259+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513227+0000","last_change":"2024-09-16T07:34:47.513227+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513227+0000","last_undegraded":"2024-09-16T07:34:47.513227+0000","last_fullsized":"2024-09-16T07:34:47.513227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"118'4","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.537397+0000","last_change":"2024-09-16T07:34:47.528565+0000","last_active":"2024-09-16T07:34:48.537397+0000","last_peered":"2024-09-16T07:34:48.537397+0000","last_clean":"2024-09-16T07:34:48.537397+0000","last_became_active":"2024-09-16T07:34:47.528270+0000","last_became_peered":"2024-09-16T07:34:47.528270+0000","last_unstale":"2024-09-16T07:34:48.537397+0000","last_undegraded":"2024-09-16T07:34:48.537397+0000","last_fullsized":"2024-09-16T07:34:48.537397+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:27.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535195+0000","last_change":"2024-09-16T07:34:48.529917+0000","last_active":"2024-09-16T07:34:48.535195+0000","last_peered":"2024-09-16T07:34:48.535195+0000","last_clean":"2024-09-16T07:34:48.535195+0000","last_became_active":"2024-09-16T07:34:48.529631+0000","last_became_peered":"2024-09-16T07:34:48.529631+0000","last_unstale":"2024-09-16T07:34:48.535195+0000","last_undegraded":"2024-09-16T07:34:48.535195+0000","last_fullsized":"2024-09-16T07:34:48.535195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:13.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.537388+0000","last_change":"2024-09-16T07:34:47.526481+0000","last_active":"2024-09-16T07:34:48.537388+0000","last_peered":"2024-09-16T07:34:48.537388+0000","last_clean":"2024-09-16T07:34:48.537388+0000","last_became_active":"2024-09-16T07:34:47.526215+0000","last_became_peered":"2024-09-16T07:34:47.526215+0000","last_unstale":"2024-09-16T07:34:48.537388+0000","last_undegraded":"2024-09-16T07:34:48.537388+0000","last_fullsized":"2024-09-16T07:34:48.537388+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:08.617410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.506925+0000","last_change":"2024-09-16T07:34:47.506925+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.506925+0000","last_undegraded":"2024-09-16T07:34:47.506925+0000","last_fullsized":"2024-09-16T07:34:47.506925+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531333+0000","last_change":"2024-09-16T07:34:47.522199+0000","last_active":"2024-09-16T07:34:49.531333+0000","last_peered":"2024-09-16T07:34:49.531333+0000","last_clean":"2024-09-16T07:34:49.531333+0000","last_became_active":"2024-09-16T07:34:47.521943+0000","last_became_peered":"2024-09-16T07:34:47.521943+0000","last_unstale":"2024-09-16T07:34:49.531333+0000","last_undegraded":"2024-09-16T07:34:49.531333+0000","last_fullsized":"2024-09-16T07:34:49.531333+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:08.548657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535239+0000","last_change":"2024-09-16T07:34:48.533772+0000","last_active":"2024-09-16T07:34:48.535239+0000","last_peered":"2024-09-16T07:34:48.535239+0000","last_clean":"2024-09-16T07:34:48.535239+0000","last_became_active":"2024-09-16T07:34:48.529223+0000","last_became_peered":"2024-09-16T07:34:48.529223+0000","last_unstale":"2024-09-16T07:34:48.535239+0000","last_undegraded":"2024-09-16T07:34:48.535239+0000","last_fullsized":"2024-09-16T07:34:48.535239+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:56.315624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531454+0000","last_change":"2024-09-16T07:34:47.523452+0000","last_active":"2024-09-16T07:34:49.531454+0000","last_peered":"2024-09-16T07:34:49.531454+0000","last_clean":"2024-09-16T07:34:49.531454+0000","last_became_active":"2024-09-16T07:34:47.521451+0000","last_became_peered":"2024-09-16T07:34:47.521451+0000","last_unstale":"2024-09-16T07:34:49.531454+0000","last_undegraded":"2024-09-16T07:34:49.531454+0000","last_fullsized":"2024-09-16T07:34:49.531454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.908074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531052+0000","last_change":"2024-09-16T07:34:48.538587+0000","last_active":"2024-09-16T07:34:49.531052+0000","last_peered":"2024-09-16T07:34:49.531052+0000","last_clean":"2024-09-16T07:34:49.531052+0000","last_became_active":"2024-09-16T07:34:48.538287+0000","last_became_peered":"2024-09-16T07:34:48.538287+0000","last_unstale":"2024-09-16T07:34:49.531052+0000","last_undegraded":"2024-09-16T07:34:49.531052+0000","last_fullsized":"2024-09-16T07:34:49.531052+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:22.418271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.539626+0000","last_change":"2024-09-16T07:34:47.520652+0000","last_active":"2024-09-16T07:34:48.539626+0000","last_peered":"2024-09-16T07:34:48.539626+0000","last_clean":"2024-09-16T07:34:48.539626+0000","last_became_active":"2024-09-16T07:34:47.520279+0000","last_became_peered":"2024-09-16T07:34:47.520279+0000","last_unstale":"2024-09-16T07:34:48.539626+0000","last_undegraded":"2024-09-16T07:34:48.539626+0000","last_fullsized":"2024-09-16T07:34:48.539626+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:17.809818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534281+0000","last_change":"2024-09-16T07:34:48.532853+0000","last_active":"2024-09-16T07:34:48.534281+0000","last_peered":"2024-09-16T07:34:48.534281+0000","last_clean":"2024-09-16T07:34:48.534281+0000","last_became_active":"2024-09-16T07:34:48.532447+0000","last_became_peered":"2024-09-16T07:34:48.532447+0000","last_unstale":"2024-09-16T07:34:48.534281+0000","last_undegraded":"2024-09-16T07:34:48.534281+0000","last_fullsized":"2024-09-16T07:34:48.534281+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:43.058025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.528166+0000","last_change":"2024-09-16T07:34:47.527304+0000","last_active":"2024-09-16T07:34:47.528166+0000","last_peered":"2024-09-16T07:34:47.528166+0000","last_clean":"2024-09-16T07:34:47.528166+0000","last_became_active":"2024-09-16T07:34:47.526260+0000","last_became_peered":"2024-09-16T07:34:47.526260+0000","last_unstale":"2024-09-16T07:34:47.528166+0000","last_undegraded":"2024-09-16T07:34:47.528166+0000","last_fullsized":"2024-09-16T07:34:47.528166+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:10.669755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.515707+0000","last_change":"2024-09-16T07:34:47.515707+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.515707+0000","last_undegraded":"2024-09-16T07:34:47.515707+0000","last_fullsized":"2024-09-16T07:34:47.515707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.517911+0000","last_change":"2024-09-16T07:34:47.520199+0000","last_active":"2024-09-16T07:34:48.517911+0000","last_peered":"2024-09-16T07:34:48.517911+0000","last_clean":"2024-09-16T07:34:48.517911+0000","last_became_active":"2024-09-16T07:34:47.519941+0000","last_became_peered":"2024-09-16T07:34:47.519941+0000","last_unstale":"2024-09-16T07:34:48.517911+0000","last_undegraded":"2024-09-16T07:34:48.517911+0000","last_fullsized":"2024-09-16T07:34:48.517911+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:54.626775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.538297+0000","last_change":"2024-09-16T07:34:48.534789+0000","last_active":"2024-09-16T07:34:48.538297+0000","last_peered":"2024-09-16T07:34:48.538297+0000","last_clean":"2024-09-16T07:34:48.538297+0000","last_became_active":"2024-09-16T07:34:48.531669+0000","last_became_peered":"2024-09-16T07:34:48.531669+0000","last_unstale":"2024-09-16T07:34:48.538297+0000","last_undegraded":"2024-09-16T07:34:48.538297+0000","last_fullsized":"2024-09-16T07:34:48.538297+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:27.517094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.528190+0000","last_change":"2024-09-16T07:34:47.527047+0000","last_active":"2024-09-16T07:34:47.528190+0000","last_peered":"2024-09-16T07:34:47.528190+0000","last_clean":"2024-09-16T07:34:47.528190+0000","last_became_active":"2024-09-16T07:34:47.526733+0000","last_became_peered":"2024-09-16T07:34:47.526733+0000","last_unstale":"2024-09-16T07:34:47.528190+0000","last_undegraded":"2024-09-16T07:34:47.528190+0000","last_fullsized":"2024-09-16T07:34:47.528190+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.388814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534332+0000","last_change":"2024-09-16T07:34:48.530402+0000","last_active":"2024-09-16T07:34:48.534332+0000","last_peered":"2024-09-16T07:34:48.534332+0000","last_clean":"2024-09-16T07:34:48.534332+0000","last_became_active":"2024-09-16T07:34:48.530123+0000","last_became_peered":"2024-09-16T07:34:48.530123+0000","last_unstale":"2024-09-16T07:34:48.534332+0000","last_undegraded":"2024-09-16T07:34:48.534332+0000","last_fullsized":"2024-09-16T07:34:48.534332+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:02.344047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531255+0000","last_change":"2024-09-16T07:34:47.521688+0000","last_active":"2024-09-16T07:34:49.531255+0000","last_peered":"2024-09-16T07:34:49.531255+0000","last_clean":"2024-09-16T07:34:49.531255+0000","last_became_active":"2024-09-16T07:34:47.521350+0000","last_became_peered":"2024-09-16T07:34:47.521350+0000","last_unstale":"2024-09-16T07:34:49.531255+0000","last_undegraded":"2024-09-16T07:34:49.531255+0000","last_fullsized":"2024-09-16T07:34:49.531255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:10.584170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.536928+0000","last_change":"2024-09-16T07:34:48.532035+0000","last_active":"2024-09-16T07:34:48.536928+0000","last_peered":"2024-09-16T07:34:48.536928+0000","last_clean":"2024-09-16T07:34:48.536928+0000","last_became_active":"2024-09-16T07:34:48.527917+0000","last_became_peered":"2024-09-16T07:34:48.527917+0000","last_unstale":"2024-09-16T07:34:48.536928+0000","last_undegraded":"2024-09-16T07:34:48.536928+0000","last_fullsized":"2024-09-16T07:34:48.536928+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:14.371171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530405+0000","last_change":"2024-09-16T07:34:47.529347+0000","last_active":"2024-09-16T07:34:47.530405+0000","last_peered":"2024-09-16T07:34:47.530405+0000","last_clean":"2024-09-16T07:34:47.530405+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:47.530405+0000","last_undegraded":"2024-09-16T07:34:47.530405+0000","last_fullsized":"2024-09-16T07:34:47.530405+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:11.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505542+0000","last_change":"2024-09-16T07:34:47.505542+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505542+0000","last_undegraded":"2024-09-16T07:34:47.505542+0000","last_fullsized":"2024-09-16T07:34:47.505542+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530531+0000","last_change":"2024-09-16T07:34:47.529669+0000","last_active":"2024-09-16T07:34:47.530531+0000","last_peered":"2024-09-16T07:34:47.530531+0000","last_clean":"2024-09-16T07:34:47.530531+0000","last_became_active":"2024-09-16T07:34:47.529378+0000","last_became_peered":"2024-09-16T07:34:47.529378+0000","last_unstale":"2024-09-16T07:34:47.530531+0000","last_undegraded":"2024-09-16T07:34:47.530531+0000","last_fullsized":"2024-09-16T07:34:47.530531+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:58.330923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.531202+0000","last_change":"2024-09-16T07:34:48.525029+0000","last_active":"2024-09-16T07:34:48.531202+0000","last_peered":"2024-09-16T07:34:48.531202+0000","last_clean":"2024-09-16T07:34:48.531202+0000","last_became_active":"2024-09-16T07:34:48.524129+0000","last_became_peered":"2024-09-16T07:34:48.524129+0000","last_unstale":"2024-09-16T07:34:48.531202+0000","last_undegraded":"2024-09-16T07:34:48.531202+0000","last_fullsized":"2024-09-16T07:34:48.531202+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:10.118549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"118'6","reported_seq":21,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531542+0000","last_change":"2024-09-16T07:34:47.524021+0000","last_active":"2024-09-16T07:34:49.531542+0000","last_peered":"2024-09-16T07:34:49.531542+0000","last_clean":"2024-09-16T07:34:49.531542+0000","last_became_active":"2024-09-16T07:34:47.520956+0000","last_became_peered":"2024-09-16T07:34:47.520956+0000","last_unstale":"2024-09-16T07:34:49.531542+0000","last_undegraded":"2024-09-16T07:34:49.531542+0000","last_fullsized":"2024-09-16T07:34:49.531542+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:42.661372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.514437+0000","last_change":"2024-09-16T07:34:47.514437+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.514437+0000","last_undegraded":"2024-09-16T07:34:47.514437+0000","last_fullsized":"2024-09-16T07:34:47.514437+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.529914+0000","last_change":"2024-09-16T07:34:47.528845+0000","last_active":"2024-09-16T07:34:47.529914+0000","last_peered":"2024-09-16T07:34:47.529914+0000","last_clean":"2024-09-16T07:34:47.529914+0000","last_became_active":"2024-09-16T07:34:47.528568+0000","last_became_peered":"2024-09-16T07:34:47.528568+0000","last_unstale":"2024-09-16T07:34:47.529914+0000","last_undegraded":"2024-09-16T07:34:47.529914+0000","last_fullsized":"2024-09-16T07:34:47.529914+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.497545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531719+0000","last_change":"2024-09-16T07:34:48.525677+0000","last_active":"2024-09-16T07:34:49.531719+0000","last_peered":"2024-09-16T07:34:49.531719+0000","last_clean":"2024-09-16T07:34:49.531719+0000","last_became_active":"2024-09-16T07:34:48.522263+0000","last_became_peered":"2024-09-16T07:34:48.522263+0000","last_unstale":"2024-09-16T07:34:49.531719+0000","last_undegraded":"2024-09-16T07:34:49.531719+0000","last_fullsized":"2024-09-16T07:34:49.531719+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:28.526205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.508493+0000","last_change":"2024-09-16T07:34:47.523404+0000","last_active":"2024-09-16T07:34:48.508493+0000","last_peered":"2024-09-16T07:34:48.508493+0000","last_clean":"2024-09-16T07:34:48.508493+0000","last_became_active":"2024-09-16T07:34:47.523140+0000","last_became_peered":"2024-09-16T07:34:47.523140+0000","last_unstale":"2024-09-16T07:34:48.508493+0000","last_undegraded":"2024-09-16T07:34:48.508493+0000","last_fullsized":"2024-09-16T07:34:48.508493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:28.805823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531011+0000","last_change":"2024-09-16T07:34:48.530763+0000","last_active":"2024-09-16T07:34:49.531011+0000","last_peered":"2024-09-16T07:34:49.531011+0000","last_clean":"2024-09-16T07:34:49.531011+0000","last_became_active":"2024-09-16T07:34:48.530263+0000","last_became_peered":"2024-09-16T07:34:48.530263+0000","last_unstale":"2024-09-16T07:34:49.531011+0000","last_undegraded":"2024-09-16T07:34:49.531011+0000","last_fullsized":"2024-09-16T07:34:49.531011+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:16.945235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.527329+0000","last_change":"2024-09-16T07:34:47.524692+0000","last_active":"2024-09-16T07:34:47.527329+0000","last_peered":"2024-09-16T07:34:47.527329+0000","last_clean":"2024-09-16T07:34:47.527329+0000","last_became_active":"2024-09-16T07:34:47.524434+0000","last_became_peered":"2024-09-16T07:34:47.524434+0000","last_unstale":"2024-09-16T07:34:47.527329+0000","last_undegraded":"2024-09-16T07:34:47.527329+0000","last_fullsized":"2024-09-16T07:34:47.527329+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:33.456945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.512413+0000","last_change":"2024-09-16T07:34:47.512413+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.512413+0000","last_undegraded":"2024-09-16T07:34:47.512413+0000","last_fullsized":"2024-09-16T07:34:47.512413+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.517308+0000","last_change":"2024-09-16T07:34:47.515933+0000","last_active":"2024-09-16T07:34:47.517308+0000","last_peered":"2024-09-16T07:34:47.517308+0000","last_clean":"2024-09-16T07:34:47.517308+0000","last_became_active":"2024-09-16T07:34:47.515670+0000","last_became_peered":"2024-09-16T07:34:47.515670+0000","last_unstale":"2024-09-16T07:34:47.517308+0000","last_undegraded":"2024-09-16T07:34:47.517308+0000","last_fullsized":"2024-09-16T07:34:47.517308+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:40.298789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.517983+0000","last_change":"2024-09-16T07:34:47.517983+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.517983+0000","last_undegraded":"2024-09-16T07:34:47.517983+0000","last_fullsized":"2024-09-16T07:34:47.517983+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.529910+0000","last_change":"2024-09-16T07:34:47.528922+0000","last_active":"2024-09-16T07:34:47.529910+0000","last_peered":"2024-09-16T07:34:47.529910+0000","last_clean":"2024-09-16T07:34:47.529910+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:47.529910+0000","last_undegraded":"2024-09-16T07:34:47.529910+0000","last_fullsized":"2024-09-16T07:34:47.529910+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:04:45.751140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534692+0000","last_change":"2024-09-16T07:34:48.528054+0000","last_active":"2024-09-16T07:34:48.534692+0000","last_peered":"2024-09-16T07:34:48.534692+0000","last_clean":"2024-09-16T07:34:48.534692+0000","last_became_active":"2024-09-16T07:34:48.527775+0000","last_became_peered":"2024-09-16T07:34:48.527775+0000","last_unstale":"2024-09-16T07:34:48.534692+0000","last_undegraded":"2024-09-16T07:34:48.534692+0000","last_fullsized":"2024-09-16T07:34:48.534692+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:51.806935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.543374+0000","last_change":"2024-09-16T07:34:47.527129+0000","last_active":"2024-09-16T07:34:48.543374+0000","last_peered":"2024-09-16T07:34:48.543374+0000","last_clean":"2024-09-16T07:34:48.543374+0000","last_became_active":"2024-09-16T07:34:47.526046+0000","last_became_peered":"2024-09-16T07:34:47.526046+0000","last_unstale":"2024-09-16T07:34:48.543374+0000","last_undegraded":"2024-09-16T07:34:48.543374+0000","last_fullsized":"2024-09-16T07:34:48.543374+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:40.775080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.524666+0000","last_change":"2024-09-16T07:34:48.522447+0000","last_active":"2024-09-16T07:34:48.524666+0000","last_peered":"2024-09-16T07:34:48.524666+0000","last_clean":"2024-09-16T07:34:48.524666+0000","last_became_active":"2024-09-16T07:34:48.521166+0000","last_became_peered":"2024-09-16T07:34:48.521166+0000","last_unstale":"2024-09-16T07:34:48.524666+0000","last_undegraded":"2024-09-16T07:34:48.524666+0000","last_fullsized":"2024-09-16T07:34:48.524666+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:34.127356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.516801+0000","last_change":"2024-09-16T07:34:47.522843+0000","last_active":"2024-09-16T07:34:48.516801+0000","last_peered":"2024-09-16T07:34:48.516801+0000","last_clean":"2024-09-16T07:34:48.516801+0000","last_became_active":"2024-09-16T07:34:47.522587+0000","last_became_peered":"2024-09-16T07:34:47.522587+0000","last_unstale":"2024-09-16T07:34:48.516801+0000","last_undegraded":"2024-09-16T07:34:48.516801+0000","last_fullsized":"2024-09-16T07:34:48.516801+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:26.555192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533092+0000","last_change":"2024-09-16T07:34:48.528425+0000","last_active":"2024-09-16T07:34:48.533092+0000","last_peered":"2024-09-16T07:34:48.533092+0000","last_clean":"2024-09-16T07:34:48.533092+0000","last_became_active":"2024-09-16T07:34:48.526307+0000","last_became_peered":"2024-09-16T07:34:48.526307+0000","last_unstale":"2024-09-16T07:34:48.533092+0000","last_undegraded":"2024-09-16T07:34:48.533092+0000","last_fullsized":"2024-09-16T07:34:48.533092+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:27.397201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530294+0000","last_change":"2024-09-16T07:34:47.529098+0000","last_active":"2024-09-16T07:34:47.530294+0000","last_peered":"2024-09-16T07:34:47.530294+0000","last_clean":"2024-09-16T07:34:47.530294+0000","last_became_active":"2024-09-16T07:34:47.528650+0000","last_became_peered":"2024-09-16T07:34:47.528650+0000","last_unstale":"2024-09-16T07:34:47.530294+0000","last_undegraded":"2024-09-16T07:34:47.530294+0000","last_fullsized":"2024-09-16T07:34:47.530294+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:15:47.135693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.517824+0000","last_change":"2024-09-16T07:34:47.517824+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.517824+0000","last_undegraded":"2024-09-16T07:34:47.517824+0000","last_fullsized":"2024-09-16T07:34:47.517824+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.526192+0000","last_change":"2024-09-16T07:34:47.521880+0000","last_active":"2024-09-16T07:34:47.526192+0000","last_peered":"2024-09-16T07:34:47.526192+0000","last_clean":"2024-09-16T07:34:47.526192+0000","last_became_active":"2024-09-16T07:34:47.521610+0000","last_became_peered":"2024-09-16T07:34:47.521610+0000","last_unstale":"2024-09-16T07:34:47.526192+0000","last_undegraded":"2024-09-16T07:34:47.526192+0000","last_fullsized":"2024-09-16T07:34:47.526192+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:27.814061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535109+0000","last_change":"2024-09-16T07:34:48.530713+0000","last_active":"2024-09-16T07:34:48.535109+0000","last_peered":"2024-09-16T07:34:48.535109+0000","last_clean":"2024-09-16T07:34:48.535109+0000","last_became_active":"2024-09-16T07:34:48.526504+0000","last_became_peered":"2024-09-16T07:34:48.526504+0000","last_unstale":"2024-09-16T07:34:48.535109+0000","last_undegraded":"2024-09-16T07:34:48.535109+0000","last_fullsized":"2024-09-16T07:34:48.535109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:56.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530916+0000","last_change":"2024-09-16T07:34:47.518969+0000","last_active":"2024-09-16T07:34:49.530916+0000","last_peered":"2024-09-16T07:34:49.530916+0000","last_clean":"2024-09-16T07:34:49.530916+0000","last_became_active":"2024-09-16T07:34:47.518693+0000","last_became_peered":"2024-09-16T07:34:47.518693+0000","last_unstale":"2024-09-16T07:34:49.530916+0000","last_undegraded":"2024-09-16T07:34:49.530916+0000","last_fullsized":"2024-09-16T07:34:49.530916+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:33.694870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516386+0000","last_change":"2024-09-16T07:34:47.516386+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516386+0000","last_undegraded":"2024-09-16T07:34:47.516386+0000","last_fullsized":"2024-09-16T07:34:47.516386+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530779+0000","last_change":"2024-09-16T07:34:47.517277+0000","last_active":"2024-09-16T07:34:49.530779+0000","last_peered":"2024-09-16T07:34:49.530779+0000","last_clean":"2024-09-16T07:34:49.530779+0000","last_became_active":"2024-09-16T07:34:47.516911+0000","last_became_peered":"2024-09-16T07:34:47.516911+0000","last_unstale":"2024-09-16T07:34:49.530779+0000","last_undegraded":"2024-09-16T07:34:49.530779+0000","last_fullsized":"2024-09-16T07:34:49.530779+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:31:40.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.527395+0000","last_change":"2024-09-16T07:34:48.526046+0000","last_active":"2024-09-16T07:34:48.527395+0000","last_peered":"2024-09-16T07:34:48.527395+0000","last_clean":"2024-09-16T07:34:48.527395+0000","last_became_active":"2024-09-16T07:34:48.525684+0000","last_became_peered":"2024-09-16T07:34:48.525684+0000","last_unstale":"2024-09-16T07:34:48.527395+0000","last_undegraded":"2024-09-16T07:34:48.527395+0000","last_fullsized":"2024-09-16T07:34:48.527395+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:50.191171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531230+0000","last_change":"2024-09-16T07:34:47.521736+0000","last_active":"2024-09-16T07:34:49.531230+0000","last_peered":"2024-09-16T07:34:49.531230+0000","last_clean":"2024-09-16T07:34:49.531230+0000","last_became_active":"2024-09-16T07:34:47.521422+0000","last_became_peered":"2024-09-16T07:34:47.521422+0000","last_unstale":"2024-09-16T07:34:49.531230+0000","last_undegraded":"2024-09-16T07:34:49.531230+0000","last_fullsized":"2024-09-16T07:34:49.531230+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:02.620783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505633+0000","last_change":"2024-09-16T07:34:47.505633+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505633+0000","last_undegraded":"2024-09-16T07:34:47.505633+0000","last_fullsized":"2024-09-16T07:34:47.505633+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.518493+0000","last_change":"2024-09-16T07:34:47.511494+0000","last_active":"2024-09-16T07:34:48.518493+0000","last_peered":"2024-09-16T07:34:48.518493+0000","last_clean":"2024-09-16T07:34:48.518493+0000","last_became_active":"2024-09-16T07:34:47.511196+0000","last_became_peered":"2024-09-16T07:34:47.511196+0000","last_unstale":"2024-09-16T07:34:48.518493+0000","last_undegraded":"2024-09-16T07:34:48.518493+0000","last_fullsized":"2024-09-16T07:34:48.518493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:18.504885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.512300+0000","last_change":"2024-09-16T07:34:47.512300+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.512300+0000","last_undegraded":"2024-09-16T07:34:47.512300+0000","last_fullsized":"2024-09-16T07:34:47.512300+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531451+0000","last_change":"2024-09-16T07:34:47.516251+0000","last_active":"2024-09-16T07:34:49.531451+0000","last_peered":"2024-09-16T07:34:49.531451+0000","last_clean":"2024-09-16T07:34:49.531451+0000","last_became_active":"2024-09-16T07:34:47.515980+0000","last_became_peered":"2024-09-16T07:34:47.515980+0000","last_unstale":"2024-09-16T07:34:49.531451+0000","last_undegraded":"2024-09-16T07:34:49.531451+0000","last_fullsized":"2024-09-16T07:34:49.531451+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:16.654899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.542525+0000","last_change":"2024-09-16T07:34:48.542025+0000","last_active":"2024-09-16T07:34:48.542525+0000","last_peered":"2024-09-16T07:34:48.542525+0000","last_clean":"2024-09-16T07:34:48.542525+0000","last_became_active":"2024-09-16T07:34:48.541713+0000","last_became_peered":"2024-09-16T07:34:48.541713+0000","last_unstale":"2024-09-16T07:34:48.542525+0000","last_undegraded":"2024-09-16T07:34:48.542525+0000","last_fullsized":"2024-09-16T07:34:48.542525+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:03.040043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531609+0000","last_change":"2024-09-16T07:34:47.513397+0000","last_active":"2024-09-16T07:34:49.531609+0000","last_peered":"2024-09-16T07:34:49.531609+0000","last_clean":"2024-09-16T07:34:49.531609+0000","last_became_active":"2024-09-16T07:34:47.513084+0000","last_became_peered":"2024-09-16T07:34:47.513084+0000","last_unstale":"2024-09-16T07:34:49.531609+0000","last_undegraded":"2024-09-16T07:34:49.531609+0000","last_fullsized":"2024-09-16T07:34:49.531609+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:49.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533116+0000","last_change":"2024-09-16T07:34:48.526515+0000","last_active":"2024-09-16T07:34:48.533116+0000","last_peered":"2024-09-16T07:34:48.533116+0000","last_clean":"2024-09-16T07:34:48.533116+0000","last_became_active":"2024-09-16T07:34:48.525964+0000","last_became_peered":"2024-09-16T07:34:48.525964+0000","last_unstale":"2024-09-16T07:34:48.533116+0000","last_undegraded":"2024-09-16T07:34:48.533116+0000","last_fullsized":"2024-09-16T07:34:48.533116+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:11.702095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.520393+0000","last_change":"2024-09-16T07:34:47.518568+0000","last_active":"2024-09-16T07:34:48.520393+0000","last_peered":"2024-09-16T07:34:48.520393+0000","last_clean":"2024-09-16T07:34:48.520393+0000","last_became_active":"2024-09-16T07:34:47.518279+0000","last_became_peered":"2024-09-16T07:34:47.518279+0000","last_unstale":"2024-09-16T07:34:48.520393+0000","last_undegraded":"2024-09-16T07:34:48.520393+0000","last_fullsized":"2024-09-16T07:34:48.520393+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:36.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534614+0000","last_change":"2024-09-16T07:34:48.528073+0000","last_active":"2024-09-16T07:34:48.534614+0000","last_peered":"2024-09-16T07:34:48.534614+0000","last_clean":"2024-09-16T07:34:48.534614+0000","last_became_active":"2024-09-16T07:34:48.527807+0000","last_became_peered":"2024-09-16T07:34:48.527807+0000","last_unstale":"2024-09-16T07:34:48.534614+0000","last_undegraded":"2024-09-16T07:34:48.534614+0000","last_fullsized":"2024-09-16T07:34:48.534614+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:22.775050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.514866+0000","last_change":"2024-09-16T07:34:47.527175+0000","last_active":"2024-09-16T07:34:48.514866+0000","last_peered":"2024-09-16T07:34:48.514866+0000","last_clean":"2024-09-16T07:34:48.514866+0000","last_became_active":"2024-09-16T07:34:47.526929+0000","last_became_peered":"2024-09-16T07:34:47.526929+0000","last_unstale":"2024-09-16T07:34:48.514866+0000","last_undegraded":"2024-09-16T07:34:48.514866+0000","last_fullsized":"2024-09-16T07:34:48.514866+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:56.405916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.527000+0000","last_change":"2024-09-16T07:34:48.524675+0000","last_active":"2024-09-16T07:34:48.527000+0000","last_peered":"2024-09-16T07:34:48.527000+0000","last_clean":"2024-09-16T07:34:48.527000+0000","last_became_active":"2024-09-16T07:34:48.524324+0000","last_became_peered":"2024-09-16T07:34:48.524324+0000","last_unstale":"2024-09-16T07:34:48.527000+0000","last_undegraded":"2024-09-16T07:34:48.527000+0000","last_fullsized":"2024-09-16T07:34:48.527000+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:55.577009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.527035+0000","last_change":"2024-09-16T07:34:47.525571+0000","last_active":"2024-09-16T07:34:47.527035+0000","last_peered":"2024-09-16T07:34:47.527035+0000","last_clean":"2024-09-16T07:34:47.527035+0000","last_became_active":"2024-09-16T07:34:47.525295+0000","last_became_peered":"2024-09-16T07:34:47.525295+0000","last_unstale":"2024-09-16T07:34:47.527035+0000","last_undegraded":"2024-09-16T07:34:47.527035+0000","last_fullsized":"2024-09-16T07:34:47.527035+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:53.256795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.506924+0000","last_change":"2024-09-16T07:34:47.506924+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.506924+0000","last_undegraded":"2024-09-16T07:34:47.506924+0000","last_fullsized":"2024-09-16T07:34:47.506924+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.520862+0000","last_change":"2024-09-16T07:34:47.518753+0000","last_active":"2024-09-16T07:34:47.520862+0000","last_peered":"2024-09-16T07:34:47.520862+0000","last_clean":"2024-09-16T07:34:47.520862+0000","last_became_active":"2024-09-16T07:34:47.518486+0000","last_became_peered":"2024-09-16T07:34:47.518486+0000","last_unstale":"2024-09-16T07:34:47.520862+0000","last_undegraded":"2024-09-16T07:34:47.520862+0000","last_fullsized":"2024-09-16T07:34:47.520862+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:21.437105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531155+0000","last_change":"2024-09-16T07:34:48.533301+0000","last_active":"2024-09-16T07:34:49.531155+0000","last_peered":"2024-09-16T07:34:49.531155+0000","last_clean":"2024-09-16T07:34:49.531155+0000","last_became_active":"2024-09-16T07:34:48.532697+0000","last_became_peered":"2024-09-16T07:34:48.532697+0000","last_unstale":"2024-09-16T07:34:49.531155+0000","last_undegraded":"2024-09-16T07:34:49.531155+0000","last_fullsized":"2024-09-16T07:34:49.531155+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:14:10.393373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.520135+0000","last_change":"2024-09-16T07:34:47.521842+0000","last_active":"2024-09-16T07:34:48.520135+0000","last_peered":"2024-09-16T07:34:48.520135+0000","last_clean":"2024-09-16T07:34:48.520135+0000","last_became_active":"2024-09-16T07:34:47.521578+0000","last_became_peered":"2024-09-16T07:34:47.521578+0000","last_unstale":"2024-09-16T07:34:48.520135+0000","last_undegraded":"2024-09-16T07:34:48.520135+0000","last_fullsized":"2024-09-16T07:34:48.520135+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:20.091370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":288,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.507819+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:34:48.507819+0000","last_peered":"2024-09-16T07:34:48.507819+0000","last_clean":"2024-09-16T07:34:48.507819+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:34:48.507819+0000","last_undegraded":"2024-09-16T07:34:48.507819+0000","last_fullsized":"2024-09-16T07:34:48.507819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2264,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126944,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244896,"statfs":{"total":96636764160,"available":96506773504,"internally_reserved":0,"allocated":1454080,"data_stored":635315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126932,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244908,"statfs":{"total":96636764160,"available":96506785792,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123352,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248488,"statfs":{"total":96636764160,"available":96510451712,"internally_reserved":0,"allocated":2035712,"data_stored":1218852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140820,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231020,"statfs":{"total":96636764160,"available":96492564480,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122768,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249072,"statfs":{"total":96636764160,"available":96511049728,"internally_reserved":0,"allocated":1429504,"data_stored":622153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509590,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153308,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218532,"statfs":{"total":96636764160,"available":96479776768,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131672,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240168,"statfs":{"total":96636764160,"available":96501932032,"internally_reserved":0,"allocated":2035712,"data_stored":1218852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126976,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244864,"statfs":{"total":96636764160,"available":96506740736,"internally_reserved":0,"allocated":1482752,"data_stored":643373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3f is not active+clean 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.519868+0000', 'last_change': '2024-09-16T07:34:47.519868+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.519868+0000', 'last_undegraded': '2024-09-16T07:34:47.519868+0000', 'last_fullsized': '2024-09-16T07:34:47.519868+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3b is not active+clean 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.513726+0000', 'last_change': '2024-09-16T07:34:47.513726+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.513726+0000', 'last_undegraded': '2024-09-16T07:34:47.513726+0000', 'last_fullsized': '2024-09-16T07:34:47.513726+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3a is not active+clean 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.507372+0000', 'last_change': '2024-09-16T07:34:47.507372+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.507372+0000', 'last_undegraded': '2024-09-16T07:34:47.507372+0000', 'last_fullsized': '2024-09-16T07:34:47.507372+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:PG 17.37 is not active+clean 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.519440+0000', 'last_change': '2024-09-16T07:34:47.519440+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.519440+0000', 'last_undegraded': '2024-09-16T07:34:47.519440+0000', 'last_fullsized': '2024-09-16T07:34:47.519440+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.554 INFO:tasks.ceph.ceph_manager.ceph:PG 17.36 is not active+clean 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.514995+0000', 'last_change': '2024-09-16T07:34:47.514995+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.514995+0000', 'last_undegraded': '2024-09-16T07:34:47.514995+0000', 'last_fullsized': '2024-09-16T07:34:47.514995+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:PG 17.34 is not active+clean 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.515387+0000', 'last_change': '2024-09-16T07:34:47.515387+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.515387+0000', 'last_undegraded': '2024-09-16T07:34:47.515387+0000', 'last_fullsized': '2024-09-16T07:34:47.515387+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:PG 17.31 is not active+clean 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.513794+0000', 'last_change': '2024-09-16T07:34:47.513794+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.513794+0000', 'last_undegraded': '2024-09-16T07:34:47.513794+0000', 'last_fullsized': '2024-09-16T07:34:47.513794+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:PG 17.30 is not active+clean 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.516736+0000', 'last_change': '2024-09-16T07:34:47.516736+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.516736+0000', 'last_undegraded': '2024-09-16T07:34:47.516736+0000', 'last_fullsized': '2024-09-16T07:34:47.516736+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2a is not active+clean 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.505332+0000', 'last_change': '2024-09-16T07:34:47.505332+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.505332+0000', 'last_undegraded': '2024-09-16T07:34:47.505332+0000', 'last_fullsized': '2024-09-16T07:34:47.505332+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:PG 17.26 is not active+clean 2024-09-16T07:34:53.555 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.505317+0000', 'last_change': '2024-09-16T07:34:47.505317+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.505317+0000', 'last_undegraded': '2024-09-16T07:34:47.505317+0000', 'last_fullsized': '2024-09-16T07:34:47.505317+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:PG 17.24 is not active+clean 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.507462+0000', 'last_change': '2024-09-16T07:34:47.507462+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.507462+0000', 'last_undegraded': '2024-09-16T07:34:47.507462+0000', 'last_fullsized': '2024-09-16T07:34:47.507462+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:PG 17.21 is not active+clean 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.513371+0000', 'last_change': '2024-09-16T07:34:47.513371+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.513371+0000', 'last_undegraded': '2024-09-16T07:34:47.513371+0000', 'last_fullsized': '2024-09-16T07:34:47.513371+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:PG 17.20 is not active+clean 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.516259+0000', 'last_change': '2024-09-16T07:34:47.516259+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.516259+0000', 'last_undegraded': '2024-09-16T07:34:47.516259+0000', 'last_fullsized': '2024-09-16T07:34:47.516259+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1e is not active+clean 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.513227+0000', 'last_change': '2024-09-16T07:34:47.513227+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.513227+0000', 'last_undegraded': '2024-09-16T07:34:47.513227+0000', 'last_fullsized': '2024-09-16T07:34:47.513227+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1c is not active+clean 2024-09-16T07:34:53.556 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.506925+0000', 'last_change': '2024-09-16T07:34:47.506925+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.506925+0000', 'last_undegraded': '2024-09-16T07:34:47.506925+0000', 'last_fullsized': '2024-09-16T07:34:47.506925+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:PG 17.18 is not active+clean 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.515707+0000', 'last_change': '2024-09-16T07:34:47.515707+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.515707+0000', 'last_undegraded': '2024-09-16T07:34:47.515707+0000', 'last_fullsized': '2024-09-16T07:34:47.515707+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:PG 17.14 is not active+clean 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.505542+0000', 'last_change': '2024-09-16T07:34:47.505542+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.505542+0000', 'last_undegraded': '2024-09-16T07:34:47.505542+0000', 'last_fullsized': '2024-09-16T07:34:47.505542+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:PG 17.12 is not active+clean 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.514437+0000', 'last_change': '2024-09-16T07:34:47.514437+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.514437+0000', 'last_undegraded': '2024-09-16T07:34:47.514437+0000', 'last_fullsized': '2024-09-16T07:34:47.514437+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:PG 17.e is not active+clean 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.512413+0000', 'last_change': '2024-09-16T07:34:47.512413+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.512413+0000', 'last_undegraded': '2024-09-16T07:34:47.512413+0000', 'last_fullsized': '2024-09-16T07:34:47.512413+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:PG 17.f is not active+clean 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.517983+0000', 'last_change': '2024-09-16T07:34:47.517983+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.517983+0000', 'last_undegraded': '2024-09-16T07:34:47.517983+0000', 'last_fullsized': '2024-09-16T07:34:47.517983+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.557 INFO:tasks.ceph.ceph_manager.ceph:PG 17.b is not active+clean 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.517824+0000', 'last_change': '2024-09-16T07:34:47.517824+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.517824+0000', 'last_undegraded': '2024-09-16T07:34:47.517824+0000', 'last_fullsized': '2024-09-16T07:34:47.517824+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:PG 17.9 is not active+clean 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.516386+0000', 'last_change': '2024-09-16T07:34:47.516386+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.516386+0000', 'last_undegraded': '2024-09-16T07:34:47.516386+0000', 'last_fullsized': '2024-09-16T07:34:47.516386+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:PG 17.7 is not active+clean 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.505633+0000', 'last_change': '2024-09-16T07:34:47.505633+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.505633+0000', 'last_undegraded': '2024-09-16T07:34:47.505633+0000', 'last_fullsized': '2024-09-16T07:34:47.505633+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:PG 17.4 is not active+clean 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.512300+0000', 'last_change': '2024-09-16T07:34:47.512300+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.512300+0000', 'last_undegraded': '2024-09-16T07:34:47.512300+0000', 'last_fullsized': '2024-09-16T07:34:47.512300+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1 is not active+clean 2024-09-16T07:34:53.558 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:34:47.506924+0000', 'last_change': '2024-09-16T07:34:47.506924+0000', 'last_active': '2024-09-16T07:34:47.492910+0000', 'last_peered': '2024-09-16T07:34:47.492910+0000', 'last_clean': '2024-09-16T07:34:47.492910+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:34:47.506924+0000', 'last_undegraded': '2024-09-16T07:34:47.506924+0000', 'last_fullsized': '2024-09-16T07:34:47.506924+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'last_clean_scrub_stamp': '2024-09-16T07:34:47.492910+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T07:34:53.559 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T07:34:53.724 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-09-16T07:34:53.724+0000 7f0041800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:34:53.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.748+0000 7fedf3600640 1 -- 172.21.15.71:0/910238366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd4004d40 msgr2=0x7fedd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.748+0000 7fedf3600640 1 --2- 172.21.15.71:0/910238366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd4004d40 0x7fedd40051c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fede4009f00 tx=0x7fedd4005830 comp rx=0 tx=0).stop 2024-09-16T07:34:53.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.748+0000 7fedf1600640 1 -- 172.21.15.71:0/910238366 reap_dead start 2024-09-16T07:34:53.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.748+0000 7fedf3600640 1 -- 172.21.15.71:0/910238366 shutdown_connections 2024-09-16T07:34:53.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.748+0000 7fedf3600640 1 -- 172.21.15.71:0/910238366 >> 172.21.15.71:0/910238366 conn(0x7fedec0b18e0 msgr2=0x7fedec0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:53.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.748+0000 7fedf3600640 1 -- 172.21.15.71:0/910238366 shutdown_connections 2024-09-16T07:34:53.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.748+0000 7fedf3600640 1 -- 172.21.15.71:0/910238366 wait complete. 2024-09-16T07:34:53.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.750+0000 7fedf3600640 1 Processor -- start 2024-09-16T07:34:53.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.750+0000 7fedf3600640 1 -- start start 2024-09-16T07:34:53.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf3600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd4004d40 0x7fedec1527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf3600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedec09d8d0 0x7fedec0ce400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf3600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedec1f2a00 0x7fedec154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf3600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fedec06ec70 con 0x7fedec1f2a00 2024-09-16T07:34:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf3600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fedec06eaf0 con 0x7fedd4004d40 2024-09-16T07:34:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf3600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fedec0cc190 con 0x7fedec09d8d0 2024-09-16T07:34:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedec1f2a00 0x7fedec154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf0c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd4004d40 0x7fedec1527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedec1f2a00 0x7fedec154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34632/0 (socket says 172.21.15.71:34632) 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf1600640 1 -- 172.21.15.71:0/2812995833 learned_addr learned my addr 172.21.15.71:0/2812995833 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedebe00640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedec09d8d0 0x7fedec0ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf1600640 1 -- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedec09d8d0 msgr2=0x7fedec0ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf1600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedec09d8d0 0x7fedec0ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf1600640 1 -- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd4004d40 msgr2=0x7fedec1527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf1600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd4004d40 0x7fedec1527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:53.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.751+0000 7fedf1600640 1 -- 172.21.15.71:0/2812995833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec152ce0 con 0x7fedec1f2a00 2024-09-16T07:34:53.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.752+0000 7fedf1600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedec1f2a00 0x7fedec154260 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fede400e380 tx=0x7fede4033d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:53.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.752+0000 7fede8c00640 1 -- 172.21.15.71:0/2812995833 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4041070 con 0x7fedec1f2a00 2024-09-16T07:34:53.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.752+0000 7fede8c00640 1 -- 172.21.15.71:0/2812995833 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fede4003c60 con 0x7fedec1f2a00 2024-09-16T07:34:53.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.752+0000 7fede8c00640 1 -- 172.21.15.71:0/2812995833 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4038040 con 0x7fedec1f2a00 2024-09-16T07:34:53.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.752+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec1f33a0 con 0x7fedec1f2a00 2024-09-16T07:34:53.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.752+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec0cdc00 con 0x7fedec1f2a00 2024-09-16T07:34:53.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.753+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedec072860 con 0x7fedec1f2a00 2024-09-16T07:34:53.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.754+0000 7fede8c00640 1 -- 172.21.15.71:0/2812995833 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fede4004290 con 0x7fedec1f2a00 2024-09-16T07:34:53.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.755+0000 7fede8c00640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedcc07a230 0x7fedcc07c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:53.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.755+0000 7fede8c00640 1 -- 172.21.15.71:0/2812995833 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fede40b9ae0 con 0x7fedec1f2a00 2024-09-16T07:34:53.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.756+0000 7fedf0c00640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedcc07a230 0x7fedcc07c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:53.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.756+0000 7fedf0c00640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedcc07a230 0x7fedcc07c6f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fede0001220 tx=0x7fede0001110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:53.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:53.759+0000 7fede8c00640 1 -- 172.21.15.71:0/2812995833 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fedec072860 con 0x7fedec1f2a00 2024-09-16T07:34:54.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.013+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fedec155cd0 con 0x7fedec1f2a00 2024-09-16T07:34:54.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.013+0000 7fede8c00640 1 -- 172.21.15.71:0/2812995833 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1304 (secure 0 0 0) 0x7fede4083560 con 0x7fedec1f2a00 2024-09-16T07:34:54.013 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:54.013 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":750,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1726471357,"num_in_osds":8,"osd_in_since":1726471347,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":72},{"state_name":"creating+peering","count":25}],"num_pgs":97,"num_pools":3,"num_objects":12,"data_bytes":592632,"bytes_used":1078038528,"bytes_avail":772016074752,"bytes_total":773094113280,"inactive_pgs_ratio":0.25773194432258606,"write_bytes_sec":853,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":75,"btime":"2024-09-16T07:34:51:534556+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8382}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-09-16T07:34:30.599503+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T07:34:54.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.015+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedcc07a230 msgr2=0x7fedcc07c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:54.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.015+0000 7fedf3600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedcc07a230 0x7fedcc07c6f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fede0001220 tx=0x7fede0001110 comp rx=0 tx=0).stop 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedec1f2a00 msgr2=0x7fedec154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedec1f2a00 0x7fedec154260 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fede400e380 tx=0x7fede4033d50 comp rx=0 tx=0).stop 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 shutdown_connections 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedcc07a230 0x7fedcc07c6f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedec1f2a00 0x7fedec154260 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedec09d8d0 0x7fedec0ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 --2- 172.21.15.71:0/2812995833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd4004d40 0x7fedec1527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 >> 172.21.15.71:0/2812995833 conn(0x7fedec0b18e0 msgr2=0x7fedec15c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 shutdown_connections 2024-09-16T07:34:54.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.016+0000 7fedf3600640 1 -- 172.21.15.71:0/2812995833 wait complete. 2024-09-16T07:34:54.032 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T07:34:54.032 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:34:54.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.187+0000 7fe88b200640 1 -- 172.21.15.71:0/565082323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0b1680 msgr2=0x7fe88c0b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:54.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.187+0000 7fe88b200640 1 --2- 172.21.15.71:0/565082323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0b1680 0x7fe88c0b1b30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0c52c0 tx=0x7fe88402a8b0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.188+0000 7fe88b200640 1 -- 172.21.15.71:0/565082323 shutdown_connections 2024-09-16T07:34:54.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.188+0000 7fe88b200640 1 --2- 172.21.15.71:0/565082323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0b1680 0x7fe88c0b1b30 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.188+0000 7fe88b200640 1 --2- 172.21.15.71:0/565082323 >> v2:172.21.15.177:3300/0 conn(0x7fe88c0d0670 0x7fe88c0d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.188+0000 7fe88b200640 1 -- 172.21.15.71:0/565082323 >> 172.21.15.71:0/565082323 conn(0x7fe88c0d1c40 msgr2=0x7fe88c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:54.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.188+0000 7fe88b200640 1 -- 172.21.15.71:0/565082323 shutdown_connections 2024-09-16T07:34:54.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.188+0000 7fe88b200640 1 -- 172.21.15.71:0/565082323 wait complete. 2024-09-16T07:34:54.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.189+0000 7fe88b200640 1 Processor -- start 2024-09-16T07:34:54.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.189+0000 7fe88b200640 1 -- start start 2024-09-16T07:34:54.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.190+0000 7fe88b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0b1680 0x7fe88c156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:54.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.190+0000 7fe88b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0d0670 0x7fe88c15e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:54.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.190+0000 7fe88b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c15bd30 0x7fe88c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:54.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.190+0000 7fe88b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0b21f0 con 0x7fe88c0d0670 2024-09-16T07:34:54.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.190+0000 7fe88b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0b2070 con 0x7fe88c15bd30 2024-09-16T07:34:54.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.190+0000 7fe88b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c0b2370 con 0x7fe88c0b1680 2024-09-16T07:34:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.190+0000 7fe881400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0d0670 0x7fe88c15e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.190+0000 7fe881400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0d0670 0x7fe88c15e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34636/0 (socket says 172.21.15.71:34636) 2024-09-16T07:34:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe881400640 1 -- 172.21.15.71:0/4255297934 learned_addr learned my addr 172.21.15.71:0/4255297934 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe889e00640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0b1680 0x7fe88c156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe88a800640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c15bd30 0x7fe88c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe881400640 1 -- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0b1680 msgr2=0x7fe88c156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe881400640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0b1680 0x7fe88c156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe881400640 1 -- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c15bd30 msgr2=0x7fe88c15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:54.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe881400640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c15bd30 0x7fe88c15e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe881400640 1 -- 172.21.15.71:0/4255297934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c0cba70 con 0x7fe88c0d0670 2024-09-16T07:34:54.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe881400640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0d0670 0x7fe88c15e2b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe87400d7a0 tx=0x7fe87400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:54.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.191+0000 7fe882a00640 1 -- 172.21.15.71:0/4255297934 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874017070 con 0x7fe88c0d0670 2024-09-16T07:34:54.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.192+0000 7fe882a00640 1 -- 172.21.15.71:0/4255297934 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe874009d20 con 0x7fe88c0d0670 2024-09-16T07:34:54.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.192+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c0cb790 con 0x7fe88c0d0670 2024-09-16T07:34:54.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.192+0000 7fe882a00640 1 -- 172.21.15.71:0/4255297934 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874004280 con 0x7fe88c0d0670 2024-09-16T07:34:54.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.192+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c0c7d30 con 0x7fe88c0d0670 2024-09-16T07:34:54.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.193+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c0b3010 con 0x7fe88c0d0670 2024-09-16T07:34:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.194+0000 7fe882a00640 1 -- 172.21.15.71:0/4255297934 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fe8740035f0 con 0x7fe88c0d0670 2024-09-16T07:34:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.195+0000 7fe882a00640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe86807a260 0x7fe86807c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.195+0000 7fe882a00640 1 -- 172.21.15.71:0/4255297934 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fe874099850 con 0x7fe88c0d0670 2024-09-16T07:34:54.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.195+0000 7fe889e00640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe86807a260 0x7fe86807c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:54.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.196+0000 7fe889e00640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe86807a260 0x7fe86807c720 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe87c000f60 tx=0x7fe87c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:54.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.199+0000 7fe882a00640 1 -- 172.21.15.71:0/4255297934 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe88c0b3010 con 0x7fe88c0d0670 2024-09-16T07:34:54.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.380+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe88c0b2820 con 0x7fe86807a260 2024-09-16T07:34:54.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.385+0000 7fe882a00640 1 -- 172.21.15.71:0/4255297934 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250404 (secure 0 0 0) 0x7fe88c0b2820 con 0x7fe86807a260 2024-09-16T07:34:54.388 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:54.390 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:34:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe86807a260 msgr2=0x7fe86807c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe86807a260 0x7fe86807c720 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe87c000f60 tx=0x7fe87c009040 comp rx=0 tx=0).stop 2024-09-16T07:34:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0d0670 msgr2=0x7fe88c15e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0d0670 0x7fe88c15e2b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe87400d7a0 tx=0x7fe87400dc60 comp rx=0 tx=0).stop 2024-09-16T07:34:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 shutdown_connections 2024-09-16T07:34:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe86807a260 0x7fe86807c720 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe88c15bd30 0x7fe88c15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe88c0d0670 0x7fe88c15e2b0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 --2- 172.21.15.71:0/4255297934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe88c0b1680 0x7fe88c156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 >> 172.21.15.71:0/4255297934 conn(0x7fe88c0d1c40 msgr2=0x7fe88c0ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 shutdown_connections 2024-09-16T07:34:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:54.389+0000 7fe88b200640 1 -- 172.21.15.71:0/4255297934 wait complete. 2024-09-16T07:34:54.404 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-16T07:34:52.604187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592632,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1052772,"kb_used_data":12464,"kb_used_omap":12,"kb_used_meta":1040115,"kb_avail":753921948,"statfs":{"total":773094113280,"available":772016074752,"internally_reserved":0,"allocated":12763136,"data_stored":6223997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1065078320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2264,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001350"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.519868+0000","last_change":"2024-09-16T07:34:47.519868+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.519868+0000","last_undegraded":"2024-09-16T07:34:47.519868+0000","last_fullsized":"2024-09-16T07:34:47.519868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.542589+0000","last_change":"2024-09-16T07:34:48.541678+0000","last_active":"2024-09-16T07:34:48.542589+0000","last_peered":"2024-09-16T07:34:48.542589+0000","last_clean":"2024-09-16T07:34:48.542589+0000","last_became_active":"2024-09-16T07:34:48.541360+0000","last_became_peered":"2024-09-16T07:34:48.541360+0000","last_unstale":"2024-09-16T07:34:48.542589+0000","last_undegraded":"2024-09-16T07:34:48.542589+0000","last_fullsized":"2024-09-16T07:34:48.542589+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:42.661091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535282+0000","last_change":"2024-09-16T07:34:48.534129+0000","last_active":"2024-09-16T07:34:48.535282+0000","last_peered":"2024-09-16T07:34:48.535282+0000","last_clean":"2024-09-16T07:34:48.535282+0000","last_became_active":"2024-09-16T07:34:48.533584+0000","last_became_peered":"2024-09-16T07:34:48.533584+0000","last_unstale":"2024-09-16T07:34:48.535282+0000","last_undegraded":"2024-09-16T07:34:48.535282+0000","last_fullsized":"2024-09-16T07:34:48.535282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:57.899237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531563+0000","last_change":"2024-09-16T07:34:48.526281+0000","last_active":"2024-09-16T07:34:49.531563+0000","last_peered":"2024-09-16T07:34:49.531563+0000","last_clean":"2024-09-16T07:34:49.531563+0000","last_became_active":"2024-09-16T07:34:48.525843+0000","last_became_peered":"2024-09-16T07:34:48.525843+0000","last_unstale":"2024-09-16T07:34:49.531563+0000","last_undegraded":"2024-09-16T07:34:49.531563+0000","last_fullsized":"2024-09-16T07:34:49.531563+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:50.039555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513726+0000","last_change":"2024-09-16T07:34:47.513726+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513726+0000","last_undegraded":"2024-09-16T07:34:47.513726+0000","last_fullsized":"2024-09-16T07:34:47.513726+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.507372+0000","last_change":"2024-09-16T07:34:47.507372+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.507372+0000","last_undegraded":"2024-09-16T07:34:47.507372+0000","last_fullsized":"2024-09-16T07:34:47.507372+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534985+0000","last_change":"2024-09-16T07:34:48.526151+0000","last_active":"2024-09-16T07:34:48.534985+0000","last_peered":"2024-09-16T07:34:48.534985+0000","last_clean":"2024-09-16T07:34:48.534985+0000","last_became_active":"2024-09-16T07:34:48.525914+0000","last_became_peered":"2024-09-16T07:34:48.525914+0000","last_unstale":"2024-09-16T07:34:48.534985+0000","last_undegraded":"2024-09-16T07:34:48.534985+0000","last_fullsized":"2024-09-16T07:34:48.534985+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:31.962809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535012+0000","last_change":"2024-09-16T07:34:48.528915+0000","last_active":"2024-09-16T07:34:48.535012+0000","last_peered":"2024-09-16T07:34:48.535012+0000","last_clean":"2024-09-16T07:34:48.535012+0000","last_became_active":"2024-09-16T07:34:48.528648+0000","last_became_peered":"2024-09-16T07:34:48.528648+0000","last_unstale":"2024-09-16T07:34:48.535012+0000","last_undegraded":"2024-09-16T07:34:48.535012+0000","last_fullsized":"2024-09-16T07:34:48.535012+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:49.801672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.519440+0000","last_change":"2024-09-16T07:34:47.519440+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.519440+0000","last_undegraded":"2024-09-16T07:34:47.519440+0000","last_fullsized":"2024-09-16T07:34:47.519440+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.514995+0000","last_change":"2024-09-16T07:34:47.514995+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.514995+0000","last_undegraded":"2024-09-16T07:34:47.514995+0000","last_fullsized":"2024-09-16T07:34:47.514995+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534388+0000","last_change":"2024-09-16T07:34:48.530703+0000","last_active":"2024-09-16T07:34:48.534388+0000","last_peered":"2024-09-16T07:34:48.534388+0000","last_clean":"2024-09-16T07:34:48.534388+0000","last_became_active":"2024-09-16T07:34:48.530406+0000","last_became_peered":"2024-09-16T07:34:48.530406+0000","last_unstale":"2024-09-16T07:34:48.534388+0000","last_undegraded":"2024-09-16T07:34:48.534388+0000","last_fullsized":"2024-09-16T07:34:48.534388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:24.273074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.515387+0000","last_change":"2024-09-16T07:34:47.515387+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.515387+0000","last_undegraded":"2024-09-16T07:34:47.515387+0000","last_fullsized":"2024-09-16T07:34:47.515387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.538475+0000","last_change":"2024-09-16T07:34:48.535432+0000","last_active":"2024-09-16T07:34:48.538475+0000","last_peered":"2024-09-16T07:34:48.538475+0000","last_clean":"2024-09-16T07:34:48.538475+0000","last_became_active":"2024-09-16T07:34:48.532194+0000","last_became_peered":"2024-09-16T07:34:48.532194+0000","last_unstale":"2024-09-16T07:34:48.538475+0000","last_undegraded":"2024-09-16T07:34:48.538475+0000","last_fullsized":"2024-09-16T07:34:48.538475+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:02:06.782991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535669+0000","last_change":"2024-09-16T07:34:48.531769+0000","last_active":"2024-09-16T07:34:48.535669+0000","last_peered":"2024-09-16T07:34:48.535669+0000","last_clean":"2024-09-16T07:34:48.535669+0000","last_became_active":"2024-09-16T07:34:48.531251+0000","last_became_peered":"2024-09-16T07:34:48.531251+0000","last_unstale":"2024-09-16T07:34:48.535669+0000","last_undegraded":"2024-09-16T07:34:48.535669+0000","last_fullsized":"2024-09-16T07:34:48.535669+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:03.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513794+0000","last_change":"2024-09-16T07:34:47.513794+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513794+0000","last_undegraded":"2024-09-16T07:34:47.513794+0000","last_fullsized":"2024-09-16T07:34:47.513794+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516736+0000","last_change":"2024-09-16T07:34:47.516736+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516736+0000","last_undegraded":"2024-09-16T07:34:47.516736+0000","last_fullsized":"2024-09-16T07:34:47.516736+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.522422+0000","last_change":"2024-09-16T07:34:48.520786+0000","last_active":"2024-09-16T07:34:48.522422+0000","last_peered":"2024-09-16T07:34:48.522422+0000","last_clean":"2024-09-16T07:34:48.522422+0000","last_became_active":"2024-09-16T07:34:48.519475+0000","last_became_peered":"2024-09-16T07:34:48.519475+0000","last_unstale":"2024-09-16T07:34:48.522422+0000","last_undegraded":"2024-09-16T07:34:48.522422+0000","last_fullsized":"2024-09-16T07:34:48.522422+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:38.065118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535936+0000","last_change":"2024-09-16T07:34:48.527942+0000","last_active":"2024-09-16T07:34:48.535936+0000","last_peered":"2024-09-16T07:34:48.535936+0000","last_clean":"2024-09-16T07:34:48.535936+0000","last_became_active":"2024-09-16T07:34:48.527669+0000","last_became_peered":"2024-09-16T07:34:48.527669+0000","last_unstale":"2024-09-16T07:34:48.535936+0000","last_undegraded":"2024-09-16T07:34:48.535936+0000","last_fullsized":"2024-09-16T07:34:48.535936+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:56.714125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535262+0000","last_change":"2024-09-16T07:34:48.534101+0000","last_active":"2024-09-16T07:34:48.535262+0000","last_peered":"2024-09-16T07:34:48.535262+0000","last_clean":"2024-09-16T07:34:48.535262+0000","last_became_active":"2024-09-16T07:34:48.529574+0000","last_became_peered":"2024-09-16T07:34:48.529574+0000","last_unstale":"2024-09-16T07:34:48.535262+0000","last_undegraded":"2024-09-16T07:34:48.535262+0000","last_fullsized":"2024-09-16T07:34:48.535262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:30:49.545255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533243+0000","last_change":"2024-09-16T07:34:48.529587+0000","last_active":"2024-09-16T07:34:48.533243+0000","last_peered":"2024-09-16T07:34:48.533243+0000","last_clean":"2024-09-16T07:34:48.533243+0000","last_became_active":"2024-09-16T07:34:48.529304+0000","last_became_peered":"2024-09-16T07:34:48.529304+0000","last_unstale":"2024-09-16T07:34:48.533243+0000","last_undegraded":"2024-09-16T07:34:48.533243+0000","last_fullsized":"2024-09-16T07:34:48.533243+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:06:46.119567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534304+0000","last_change":"2024-09-16T07:34:48.533118+0000","last_active":"2024-09-16T07:34:48.534304+0000","last_peered":"2024-09-16T07:34:48.534304+0000","last_clean":"2024-09-16T07:34:48.534304+0000","last_became_active":"2024-09-16T07:34:48.532777+0000","last_became_peered":"2024-09-16T07:34:48.532777+0000","last_unstale":"2024-09-16T07:34:48.534304+0000","last_undegraded":"2024-09-16T07:34:48.534304+0000","last_fullsized":"2024-09-16T07:34:48.534304+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:02.406116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505332+0000","last_change":"2024-09-16T07:34:47.505332+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505332+0000","last_undegraded":"2024-09-16T07:34:47.505332+0000","last_fullsized":"2024-09-16T07:34:47.505332+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535054+0000","last_change":"2024-09-16T07:34:48.528984+0000","last_active":"2024-09-16T07:34:48.535054+0000","last_peered":"2024-09-16T07:34:48.535054+0000","last_clean":"2024-09-16T07:34:48.535054+0000","last_became_active":"2024-09-16T07:34:48.528481+0000","last_became_peered":"2024-09-16T07:34:48.528481+0000","last_unstale":"2024-09-16T07:34:48.535054+0000","last_undegraded":"2024-09-16T07:34:48.535054+0000","last_fullsized":"2024-09-16T07:34:48.535054+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:43.572025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530706+0000","last_change":"2024-09-16T07:34:48.533570+0000","last_active":"2024-09-16T07:34:49.530706+0000","last_peered":"2024-09-16T07:34:49.530706+0000","last_clean":"2024-09-16T07:34:49.530706+0000","last_became_active":"2024-09-16T07:34:48.533039+0000","last_became_peered":"2024-09-16T07:34:48.533039+0000","last_unstale":"2024-09-16T07:34:49.530706+0000","last_undegraded":"2024-09-16T07:34:49.530706+0000","last_fullsized":"2024-09-16T07:34:49.530706+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:00:09.699334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533344+0000","last_change":"2024-09-16T07:34:48.527254+0000","last_active":"2024-09-16T07:34:48.533344+0000","last_peered":"2024-09-16T07:34:48.533344+0000","last_clean":"2024-09-16T07:34:48.533344+0000","last_became_active":"2024-09-16T07:34:48.527013+0000","last_became_peered":"2024-09-16T07:34:48.527013+0000","last_unstale":"2024-09-16T07:34:48.533344+0000","last_undegraded":"2024-09-16T07:34:48.533344+0000","last_fullsized":"2024-09-16T07:34:48.533344+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:16.029488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505317+0000","last_change":"2024-09-16T07:34:47.505317+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505317+0000","last_undegraded":"2024-09-16T07:34:47.505317+0000","last_fullsized":"2024-09-16T07:34:47.505317+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531251+0000","last_change":"2024-09-16T07:34:48.541120+0000","last_active":"2024-09-16T07:34:49.531251+0000","last_peered":"2024-09-16T07:34:49.531251+0000","last_clean":"2024-09-16T07:34:49.531251+0000","last_became_active":"2024-09-16T07:34:48.540074+0000","last_became_peered":"2024-09-16T07:34:48.540074+0000","last_unstale":"2024-09-16T07:34:49.531251+0000","last_undegraded":"2024-09-16T07:34:49.531251+0000","last_fullsized":"2024-09-16T07:34:49.531251+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:53.929219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.507462+0000","last_change":"2024-09-16T07:34:47.507462+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.507462+0000","last_undegraded":"2024-09-16T07:34:47.507462+0000","last_fullsized":"2024-09-16T07:34:47.507462+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535813+0000","last_change":"2024-09-16T07:34:48.534695+0000","last_active":"2024-09-16T07:34:48.535813+0000","last_peered":"2024-09-16T07:34:48.535813+0000","last_clean":"2024-09-16T07:34:48.535813+0000","last_became_active":"2024-09-16T07:34:48.534435+0000","last_became_peered":"2024-09-16T07:34:48.534435+0000","last_unstale":"2024-09-16T07:34:48.535813+0000","last_undegraded":"2024-09-16T07:34:48.535813+0000","last_fullsized":"2024-09-16T07:34:48.535813+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:11.262607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534350+0000","last_change":"2024-09-16T07:34:48.525252+0000","last_active":"2024-09-16T07:34:48.534350+0000","last_peered":"2024-09-16T07:34:48.534350+0000","last_clean":"2024-09-16T07:34:48.534350+0000","last_became_active":"2024-09-16T07:34:48.524453+0000","last_became_peered":"2024-09-16T07:34:48.524453+0000","last_unstale":"2024-09-16T07:34:48.534350+0000","last_undegraded":"2024-09-16T07:34:48.534350+0000","last_fullsized":"2024-09-16T07:34:48.534350+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:18.325757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513371+0000","last_change":"2024-09-16T07:34:47.513371+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513371+0000","last_undegraded":"2024-09-16T07:34:47.513371+0000","last_fullsized":"2024-09-16T07:34:47.513371+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516259+0000","last_change":"2024-09-16T07:34:47.516259+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516259+0000","last_undegraded":"2024-09-16T07:34:47.516259+0000","last_fullsized":"2024-09-16T07:34:47.516259+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.513227+0000","last_change":"2024-09-16T07:34:47.513227+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.513227+0000","last_undegraded":"2024-09-16T07:34:47.513227+0000","last_fullsized":"2024-09-16T07:34:47.513227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"118'4","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.537397+0000","last_change":"2024-09-16T07:34:47.528565+0000","last_active":"2024-09-16T07:34:48.537397+0000","last_peered":"2024-09-16T07:34:48.537397+0000","last_clean":"2024-09-16T07:34:48.537397+0000","last_became_active":"2024-09-16T07:34:47.528270+0000","last_became_peered":"2024-09-16T07:34:47.528270+0000","last_unstale":"2024-09-16T07:34:48.537397+0000","last_undegraded":"2024-09-16T07:34:48.537397+0000","last_fullsized":"2024-09-16T07:34:48.537397+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:27.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535195+0000","last_change":"2024-09-16T07:34:48.529917+0000","last_active":"2024-09-16T07:34:48.535195+0000","last_peered":"2024-09-16T07:34:48.535195+0000","last_clean":"2024-09-16T07:34:48.535195+0000","last_became_active":"2024-09-16T07:34:48.529631+0000","last_became_peered":"2024-09-16T07:34:48.529631+0000","last_unstale":"2024-09-16T07:34:48.535195+0000","last_undegraded":"2024-09-16T07:34:48.535195+0000","last_fullsized":"2024-09-16T07:34:48.535195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:13.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.537388+0000","last_change":"2024-09-16T07:34:47.526481+0000","last_active":"2024-09-16T07:34:48.537388+0000","last_peered":"2024-09-16T07:34:48.537388+0000","last_clean":"2024-09-16T07:34:48.537388+0000","last_became_active":"2024-09-16T07:34:47.526215+0000","last_became_peered":"2024-09-16T07:34:47.526215+0000","last_unstale":"2024-09-16T07:34:48.537388+0000","last_undegraded":"2024-09-16T07:34:48.537388+0000","last_fullsized":"2024-09-16T07:34:48.537388+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:08.617410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.506925+0000","last_change":"2024-09-16T07:34:47.506925+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.506925+0000","last_undegraded":"2024-09-16T07:34:47.506925+0000","last_fullsized":"2024-09-16T07:34:47.506925+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531333+0000","last_change":"2024-09-16T07:34:47.522199+0000","last_active":"2024-09-16T07:34:49.531333+0000","last_peered":"2024-09-16T07:34:49.531333+0000","last_clean":"2024-09-16T07:34:49.531333+0000","last_became_active":"2024-09-16T07:34:47.521943+0000","last_became_peered":"2024-09-16T07:34:47.521943+0000","last_unstale":"2024-09-16T07:34:49.531333+0000","last_undegraded":"2024-09-16T07:34:49.531333+0000","last_fullsized":"2024-09-16T07:34:49.531333+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:08.548657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535239+0000","last_change":"2024-09-16T07:34:48.533772+0000","last_active":"2024-09-16T07:34:48.535239+0000","last_peered":"2024-09-16T07:34:48.535239+0000","last_clean":"2024-09-16T07:34:48.535239+0000","last_became_active":"2024-09-16T07:34:48.529223+0000","last_became_peered":"2024-09-16T07:34:48.529223+0000","last_unstale":"2024-09-16T07:34:48.535239+0000","last_undegraded":"2024-09-16T07:34:48.535239+0000","last_fullsized":"2024-09-16T07:34:48.535239+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:56.315624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531454+0000","last_change":"2024-09-16T07:34:47.523452+0000","last_active":"2024-09-16T07:34:49.531454+0000","last_peered":"2024-09-16T07:34:49.531454+0000","last_clean":"2024-09-16T07:34:49.531454+0000","last_became_active":"2024-09-16T07:34:47.521451+0000","last_became_peered":"2024-09-16T07:34:47.521451+0000","last_unstale":"2024-09-16T07:34:49.531454+0000","last_undegraded":"2024-09-16T07:34:49.531454+0000","last_fullsized":"2024-09-16T07:34:49.531454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.908074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531052+0000","last_change":"2024-09-16T07:34:48.538587+0000","last_active":"2024-09-16T07:34:49.531052+0000","last_peered":"2024-09-16T07:34:49.531052+0000","last_clean":"2024-09-16T07:34:49.531052+0000","last_became_active":"2024-09-16T07:34:48.538287+0000","last_became_peered":"2024-09-16T07:34:48.538287+0000","last_unstale":"2024-09-16T07:34:49.531052+0000","last_undegraded":"2024-09-16T07:34:49.531052+0000","last_fullsized":"2024-09-16T07:34:49.531052+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:22.418271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.539626+0000","last_change":"2024-09-16T07:34:47.520652+0000","last_active":"2024-09-16T07:34:48.539626+0000","last_peered":"2024-09-16T07:34:48.539626+0000","last_clean":"2024-09-16T07:34:48.539626+0000","last_became_active":"2024-09-16T07:34:47.520279+0000","last_became_peered":"2024-09-16T07:34:47.520279+0000","last_unstale":"2024-09-16T07:34:48.539626+0000","last_undegraded":"2024-09-16T07:34:48.539626+0000","last_fullsized":"2024-09-16T07:34:48.539626+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:17.809818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534281+0000","last_change":"2024-09-16T07:34:48.532853+0000","last_active":"2024-09-16T07:34:48.534281+0000","last_peered":"2024-09-16T07:34:48.534281+0000","last_clean":"2024-09-16T07:34:48.534281+0000","last_became_active":"2024-09-16T07:34:48.532447+0000","last_became_peered":"2024-09-16T07:34:48.532447+0000","last_unstale":"2024-09-16T07:34:48.534281+0000","last_undegraded":"2024-09-16T07:34:48.534281+0000","last_fullsized":"2024-09-16T07:34:48.534281+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:43.058025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.528166+0000","last_change":"2024-09-16T07:34:47.527304+0000","last_active":"2024-09-16T07:34:47.528166+0000","last_peered":"2024-09-16T07:34:47.528166+0000","last_clean":"2024-09-16T07:34:47.528166+0000","last_became_active":"2024-09-16T07:34:47.526260+0000","last_became_peered":"2024-09-16T07:34:47.526260+0000","last_unstale":"2024-09-16T07:34:47.528166+0000","last_undegraded":"2024-09-16T07:34:47.528166+0000","last_fullsized":"2024-09-16T07:34:47.528166+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:10.669755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.515707+0000","last_change":"2024-09-16T07:34:47.515707+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.515707+0000","last_undegraded":"2024-09-16T07:34:47.515707+0000","last_fullsized":"2024-09-16T07:34:47.515707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.517911+0000","last_change":"2024-09-16T07:34:47.520199+0000","last_active":"2024-09-16T07:34:48.517911+0000","last_peered":"2024-09-16T07:34:48.517911+0000","last_clean":"2024-09-16T07:34:48.517911+0000","last_became_active":"2024-09-16T07:34:47.519941+0000","last_became_peered":"2024-09-16T07:34:47.519941+0000","last_unstale":"2024-09-16T07:34:48.517911+0000","last_undegraded":"2024-09-16T07:34:48.517911+0000","last_fullsized":"2024-09-16T07:34:48.517911+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:54.626775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.538297+0000","last_change":"2024-09-16T07:34:48.534789+0000","last_active":"2024-09-16T07:34:48.538297+0000","last_peered":"2024-09-16T07:34:48.538297+0000","last_clean":"2024-09-16T07:34:48.538297+0000","last_became_active":"2024-09-16T07:34:48.531669+0000","last_became_peered":"2024-09-16T07:34:48.531669+0000","last_unstale":"2024-09-16T07:34:48.538297+0000","last_undegraded":"2024-09-16T07:34:48.538297+0000","last_fullsized":"2024-09-16T07:34:48.538297+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:27.517094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.528190+0000","last_change":"2024-09-16T07:34:47.527047+0000","last_active":"2024-09-16T07:34:47.528190+0000","last_peered":"2024-09-16T07:34:47.528190+0000","last_clean":"2024-09-16T07:34:47.528190+0000","last_became_active":"2024-09-16T07:34:47.526733+0000","last_became_peered":"2024-09-16T07:34:47.526733+0000","last_unstale":"2024-09-16T07:34:47.528190+0000","last_undegraded":"2024-09-16T07:34:47.528190+0000","last_fullsized":"2024-09-16T07:34:47.528190+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.388814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534332+0000","last_change":"2024-09-16T07:34:48.530402+0000","last_active":"2024-09-16T07:34:48.534332+0000","last_peered":"2024-09-16T07:34:48.534332+0000","last_clean":"2024-09-16T07:34:48.534332+0000","last_became_active":"2024-09-16T07:34:48.530123+0000","last_became_peered":"2024-09-16T07:34:48.530123+0000","last_unstale":"2024-09-16T07:34:48.534332+0000","last_undegraded":"2024-09-16T07:34:48.534332+0000","last_fullsized":"2024-09-16T07:34:48.534332+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:02.344047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531255+0000","last_change":"2024-09-16T07:34:47.521688+0000","last_active":"2024-09-16T07:34:49.531255+0000","last_peered":"2024-09-16T07:34:49.531255+0000","last_clean":"2024-09-16T07:34:49.531255+0000","last_became_active":"2024-09-16T07:34:47.521350+0000","last_became_peered":"2024-09-16T07:34:47.521350+0000","last_unstale":"2024-09-16T07:34:49.531255+0000","last_undegraded":"2024-09-16T07:34:49.531255+0000","last_fullsized":"2024-09-16T07:34:49.531255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:10.584170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.536928+0000","last_change":"2024-09-16T07:34:48.532035+0000","last_active":"2024-09-16T07:34:48.536928+0000","last_peered":"2024-09-16T07:34:48.536928+0000","last_clean":"2024-09-16T07:34:48.536928+0000","last_became_active":"2024-09-16T07:34:48.527917+0000","last_became_peered":"2024-09-16T07:34:48.527917+0000","last_unstale":"2024-09-16T07:34:48.536928+0000","last_undegraded":"2024-09-16T07:34:48.536928+0000","last_fullsized":"2024-09-16T07:34:48.536928+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:14.371171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530405+0000","last_change":"2024-09-16T07:34:47.529347+0000","last_active":"2024-09-16T07:34:47.530405+0000","last_peered":"2024-09-16T07:34:47.530405+0000","last_clean":"2024-09-16T07:34:47.530405+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:47.530405+0000","last_undegraded":"2024-09-16T07:34:47.530405+0000","last_fullsized":"2024-09-16T07:34:47.530405+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:11.365590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505542+0000","last_change":"2024-09-16T07:34:47.505542+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505542+0000","last_undegraded":"2024-09-16T07:34:47.505542+0000","last_fullsized":"2024-09-16T07:34:47.505542+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530531+0000","last_change":"2024-09-16T07:34:47.529669+0000","last_active":"2024-09-16T07:34:47.530531+0000","last_peered":"2024-09-16T07:34:47.530531+0000","last_clean":"2024-09-16T07:34:47.530531+0000","last_became_active":"2024-09-16T07:34:47.529378+0000","last_became_peered":"2024-09-16T07:34:47.529378+0000","last_unstale":"2024-09-16T07:34:47.530531+0000","last_undegraded":"2024-09-16T07:34:47.530531+0000","last_fullsized":"2024-09-16T07:34:47.530531+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:58.330923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.531202+0000","last_change":"2024-09-16T07:34:48.525029+0000","last_active":"2024-09-16T07:34:48.531202+0000","last_peered":"2024-09-16T07:34:48.531202+0000","last_clean":"2024-09-16T07:34:48.531202+0000","last_became_active":"2024-09-16T07:34:48.524129+0000","last_became_peered":"2024-09-16T07:34:48.524129+0000","last_unstale":"2024-09-16T07:34:48.531202+0000","last_undegraded":"2024-09-16T07:34:48.531202+0000","last_fullsized":"2024-09-16T07:34:48.531202+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:10.118549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"118'6","reported_seq":21,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531542+0000","last_change":"2024-09-16T07:34:47.524021+0000","last_active":"2024-09-16T07:34:49.531542+0000","last_peered":"2024-09-16T07:34:49.531542+0000","last_clean":"2024-09-16T07:34:49.531542+0000","last_became_active":"2024-09-16T07:34:47.520956+0000","last_became_peered":"2024-09-16T07:34:47.520956+0000","last_unstale":"2024-09-16T07:34:49.531542+0000","last_undegraded":"2024-09-16T07:34:49.531542+0000","last_fullsized":"2024-09-16T07:34:49.531542+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:42.661372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.514437+0000","last_change":"2024-09-16T07:34:47.514437+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.514437+0000","last_undegraded":"2024-09-16T07:34:47.514437+0000","last_fullsized":"2024-09-16T07:34:47.514437+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.529914+0000","last_change":"2024-09-16T07:34:47.528845+0000","last_active":"2024-09-16T07:34:47.529914+0000","last_peered":"2024-09-16T07:34:47.529914+0000","last_clean":"2024-09-16T07:34:47.529914+0000","last_became_active":"2024-09-16T07:34:47.528568+0000","last_became_peered":"2024-09-16T07:34:47.528568+0000","last_unstale":"2024-09-16T07:34:47.529914+0000","last_undegraded":"2024-09-16T07:34:47.529914+0000","last_fullsized":"2024-09-16T07:34:47.529914+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.497545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531719+0000","last_change":"2024-09-16T07:34:48.525677+0000","last_active":"2024-09-16T07:34:49.531719+0000","last_peered":"2024-09-16T07:34:49.531719+0000","last_clean":"2024-09-16T07:34:49.531719+0000","last_became_active":"2024-09-16T07:34:48.522263+0000","last_became_peered":"2024-09-16T07:34:48.522263+0000","last_unstale":"2024-09-16T07:34:49.531719+0000","last_undegraded":"2024-09-16T07:34:49.531719+0000","last_fullsized":"2024-09-16T07:34:49.531719+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:28.526205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.508493+0000","last_change":"2024-09-16T07:34:47.523404+0000","last_active":"2024-09-16T07:34:48.508493+0000","last_peered":"2024-09-16T07:34:48.508493+0000","last_clean":"2024-09-16T07:34:48.508493+0000","last_became_active":"2024-09-16T07:34:47.523140+0000","last_became_peered":"2024-09-16T07:34:47.523140+0000","last_unstale":"2024-09-16T07:34:48.508493+0000","last_undegraded":"2024-09-16T07:34:48.508493+0000","last_fullsized":"2024-09-16T07:34:48.508493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:28.805823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531011+0000","last_change":"2024-09-16T07:34:48.530763+0000","last_active":"2024-09-16T07:34:49.531011+0000","last_peered":"2024-09-16T07:34:49.531011+0000","last_clean":"2024-09-16T07:34:49.531011+0000","last_became_active":"2024-09-16T07:34:48.530263+0000","last_became_peered":"2024-09-16T07:34:48.530263+0000","last_unstale":"2024-09-16T07:34:49.531011+0000","last_undegraded":"2024-09-16T07:34:49.531011+0000","last_fullsized":"2024-09-16T07:34:49.531011+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:16.945235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.527329+0000","last_change":"2024-09-16T07:34:47.524692+0000","last_active":"2024-09-16T07:34:47.527329+0000","last_peered":"2024-09-16T07:34:47.527329+0000","last_clean":"2024-09-16T07:34:47.527329+0000","last_became_active":"2024-09-16T07:34:47.524434+0000","last_became_peered":"2024-09-16T07:34:47.524434+0000","last_unstale":"2024-09-16T07:34:47.527329+0000","last_undegraded":"2024-09-16T07:34:47.527329+0000","last_fullsized":"2024-09-16T07:34:47.527329+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:33.456945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.512413+0000","last_change":"2024-09-16T07:34:47.512413+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.512413+0000","last_undegraded":"2024-09-16T07:34:47.512413+0000","last_fullsized":"2024-09-16T07:34:47.512413+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.517308+0000","last_change":"2024-09-16T07:34:47.515933+0000","last_active":"2024-09-16T07:34:47.517308+0000","last_peered":"2024-09-16T07:34:47.517308+0000","last_clean":"2024-09-16T07:34:47.517308+0000","last_became_active":"2024-09-16T07:34:47.515670+0000","last_became_peered":"2024-09-16T07:34:47.515670+0000","last_unstale":"2024-09-16T07:34:47.517308+0000","last_undegraded":"2024-09-16T07:34:47.517308+0000","last_fullsized":"2024-09-16T07:34:47.517308+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:40.298789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.517983+0000","last_change":"2024-09-16T07:34:47.517983+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.517983+0000","last_undegraded":"2024-09-16T07:34:47.517983+0000","last_fullsized":"2024-09-16T07:34:47.517983+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.529910+0000","last_change":"2024-09-16T07:34:47.528922+0000","last_active":"2024-09-16T07:34:47.529910+0000","last_peered":"2024-09-16T07:34:47.529910+0000","last_clean":"2024-09-16T07:34:47.529910+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:47.529910+0000","last_undegraded":"2024-09-16T07:34:47.529910+0000","last_fullsized":"2024-09-16T07:34:47.529910+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:04:45.751140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534692+0000","last_change":"2024-09-16T07:34:48.528054+0000","last_active":"2024-09-16T07:34:48.534692+0000","last_peered":"2024-09-16T07:34:48.534692+0000","last_clean":"2024-09-16T07:34:48.534692+0000","last_became_active":"2024-09-16T07:34:48.527775+0000","last_became_peered":"2024-09-16T07:34:48.527775+0000","last_unstale":"2024-09-16T07:34:48.534692+0000","last_undegraded":"2024-09-16T07:34:48.534692+0000","last_fullsized":"2024-09-16T07:34:48.534692+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:04:51.806935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"118'1","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.543374+0000","last_change":"2024-09-16T07:34:47.527129+0000","last_active":"2024-09-16T07:34:48.543374+0000","last_peered":"2024-09-16T07:34:48.543374+0000","last_clean":"2024-09-16T07:34:48.543374+0000","last_became_active":"2024-09-16T07:34:47.526046+0000","last_became_peered":"2024-09-16T07:34:47.526046+0000","last_unstale":"2024-09-16T07:34:48.543374+0000","last_undegraded":"2024-09-16T07:34:48.543374+0000","last_fullsized":"2024-09-16T07:34:48.543374+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:40.775080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.524666+0000","last_change":"2024-09-16T07:34:48.522447+0000","last_active":"2024-09-16T07:34:48.524666+0000","last_peered":"2024-09-16T07:34:48.524666+0000","last_clean":"2024-09-16T07:34:48.524666+0000","last_became_active":"2024-09-16T07:34:48.521166+0000","last_became_peered":"2024-09-16T07:34:48.521166+0000","last_unstale":"2024-09-16T07:34:48.524666+0000","last_undegraded":"2024-09-16T07:34:48.524666+0000","last_fullsized":"2024-09-16T07:34:48.524666+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:34.127356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.516801+0000","last_change":"2024-09-16T07:34:47.522843+0000","last_active":"2024-09-16T07:34:48.516801+0000","last_peered":"2024-09-16T07:34:48.516801+0000","last_clean":"2024-09-16T07:34:48.516801+0000","last_became_active":"2024-09-16T07:34:47.522587+0000","last_became_peered":"2024-09-16T07:34:47.522587+0000","last_unstale":"2024-09-16T07:34:48.516801+0000","last_undegraded":"2024-09-16T07:34:48.516801+0000","last_fullsized":"2024-09-16T07:34:48.516801+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:26.555192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533092+0000","last_change":"2024-09-16T07:34:48.528425+0000","last_active":"2024-09-16T07:34:48.533092+0000","last_peered":"2024-09-16T07:34:48.533092+0000","last_clean":"2024-09-16T07:34:48.533092+0000","last_became_active":"2024-09-16T07:34:48.526307+0000","last_became_peered":"2024-09-16T07:34:48.526307+0000","last_unstale":"2024-09-16T07:34:48.533092+0000","last_undegraded":"2024-09-16T07:34:48.533092+0000","last_fullsized":"2024-09-16T07:34:48.533092+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:34:27.397201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.530294+0000","last_change":"2024-09-16T07:34:47.529098+0000","last_active":"2024-09-16T07:34:47.530294+0000","last_peered":"2024-09-16T07:34:47.530294+0000","last_clean":"2024-09-16T07:34:47.530294+0000","last_became_active":"2024-09-16T07:34:47.528650+0000","last_became_peered":"2024-09-16T07:34:47.528650+0000","last_unstale":"2024-09-16T07:34:47.530294+0000","last_undegraded":"2024-09-16T07:34:47.530294+0000","last_fullsized":"2024-09-16T07:34:47.530294+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:15:47.135693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.517824+0000","last_change":"2024-09-16T07:34:47.517824+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.517824+0000","last_undegraded":"2024-09-16T07:34:47.517824+0000","last_fullsized":"2024-09-16T07:34:47.517824+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.526192+0000","last_change":"2024-09-16T07:34:47.521880+0000","last_active":"2024-09-16T07:34:47.526192+0000","last_peered":"2024-09-16T07:34:47.526192+0000","last_clean":"2024-09-16T07:34:47.526192+0000","last_became_active":"2024-09-16T07:34:47.521610+0000","last_became_peered":"2024-09-16T07:34:47.521610+0000","last_unstale":"2024-09-16T07:34:47.526192+0000","last_undegraded":"2024-09-16T07:34:47.526192+0000","last_fullsized":"2024-09-16T07:34:47.526192+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:27.814061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.535109+0000","last_change":"2024-09-16T07:34:48.530713+0000","last_active":"2024-09-16T07:34:48.535109+0000","last_peered":"2024-09-16T07:34:48.535109+0000","last_clean":"2024-09-16T07:34:48.535109+0000","last_became_active":"2024-09-16T07:34:48.526504+0000","last_became_peered":"2024-09-16T07:34:48.526504+0000","last_unstale":"2024-09-16T07:34:48.535109+0000","last_undegraded":"2024-09-16T07:34:48.535109+0000","last_fullsized":"2024-09-16T07:34:48.535109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:56.537892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530916+0000","last_change":"2024-09-16T07:34:47.518969+0000","last_active":"2024-09-16T07:34:49.530916+0000","last_peered":"2024-09-16T07:34:49.530916+0000","last_clean":"2024-09-16T07:34:49.530916+0000","last_became_active":"2024-09-16T07:34:47.518693+0000","last_became_peered":"2024-09-16T07:34:47.518693+0000","last_unstale":"2024-09-16T07:34:49.530916+0000","last_undegraded":"2024-09-16T07:34:49.530916+0000","last_fullsized":"2024-09-16T07:34:49.530916+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:33.694870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.516386+0000","last_change":"2024-09-16T07:34:47.516386+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.516386+0000","last_undegraded":"2024-09-16T07:34:47.516386+0000","last_fullsized":"2024-09-16T07:34:47.516386+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530779+0000","last_change":"2024-09-16T07:34:47.517277+0000","last_active":"2024-09-16T07:34:49.530779+0000","last_peered":"2024-09-16T07:34:49.530779+0000","last_clean":"2024-09-16T07:34:49.530779+0000","last_became_active":"2024-09-16T07:34:47.516911+0000","last_became_peered":"2024-09-16T07:34:47.516911+0000","last_unstale":"2024-09-16T07:34:49.530779+0000","last_undegraded":"2024-09-16T07:34:49.530779+0000","last_fullsized":"2024-09-16T07:34:49.530779+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:31:40.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.527395+0000","last_change":"2024-09-16T07:34:48.526046+0000","last_active":"2024-09-16T07:34:48.527395+0000","last_peered":"2024-09-16T07:34:48.527395+0000","last_clean":"2024-09-16T07:34:48.527395+0000","last_became_active":"2024-09-16T07:34:48.525684+0000","last_became_peered":"2024-09-16T07:34:48.525684+0000","last_unstale":"2024-09-16T07:34:48.527395+0000","last_undegraded":"2024-09-16T07:34:48.527395+0000","last_fullsized":"2024-09-16T07:34:48.527395+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:50.191171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531230+0000","last_change":"2024-09-16T07:34:47.521736+0000","last_active":"2024-09-16T07:34:49.531230+0000","last_peered":"2024-09-16T07:34:49.531230+0000","last_clean":"2024-09-16T07:34:49.531230+0000","last_became_active":"2024-09-16T07:34:47.521422+0000","last_became_peered":"2024-09-16T07:34:47.521422+0000","last_unstale":"2024-09-16T07:34:49.531230+0000","last_undegraded":"2024-09-16T07:34:49.531230+0000","last_fullsized":"2024-09-16T07:34:49.531230+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:02.620783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.505633+0000","last_change":"2024-09-16T07:34:47.505633+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.505633+0000","last_undegraded":"2024-09-16T07:34:47.505633+0000","last_fullsized":"2024-09-16T07:34:47.505633+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.518493+0000","last_change":"2024-09-16T07:34:47.511494+0000","last_active":"2024-09-16T07:34:48.518493+0000","last_peered":"2024-09-16T07:34:48.518493+0000","last_clean":"2024-09-16T07:34:48.518493+0000","last_became_active":"2024-09-16T07:34:47.511196+0000","last_became_peered":"2024-09-16T07:34:47.511196+0000","last_unstale":"2024-09-16T07:34:48.518493+0000","last_undegraded":"2024-09-16T07:34:48.518493+0000","last_fullsized":"2024-09-16T07:34:48.518493+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:18.504885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.512300+0000","last_change":"2024-09-16T07:34:47.512300+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.512300+0000","last_undegraded":"2024-09-16T07:34:47.512300+0000","last_fullsized":"2024-09-16T07:34:47.512300+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531451+0000","last_change":"2024-09-16T07:34:47.516251+0000","last_active":"2024-09-16T07:34:49.531451+0000","last_peered":"2024-09-16T07:34:49.531451+0000","last_clean":"2024-09-16T07:34:49.531451+0000","last_became_active":"2024-09-16T07:34:47.515980+0000","last_became_peered":"2024-09-16T07:34:47.515980+0000","last_unstale":"2024-09-16T07:34:49.531451+0000","last_undegraded":"2024-09-16T07:34:49.531451+0000","last_fullsized":"2024-09-16T07:34:49.531451+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:16.654899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.542525+0000","last_change":"2024-09-16T07:34:48.542025+0000","last_active":"2024-09-16T07:34:48.542525+0000","last_peered":"2024-09-16T07:34:48.542525+0000","last_clean":"2024-09-16T07:34:48.542525+0000","last_became_active":"2024-09-16T07:34:48.541713+0000","last_became_peered":"2024-09-16T07:34:48.541713+0000","last_unstale":"2024-09-16T07:34:48.542525+0000","last_undegraded":"2024-09-16T07:34:48.542525+0000","last_fullsized":"2024-09-16T07:34:48.542525+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:03.040043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531609+0000","last_change":"2024-09-16T07:34:47.513397+0000","last_active":"2024-09-16T07:34:49.531609+0000","last_peered":"2024-09-16T07:34:49.531609+0000","last_clean":"2024-09-16T07:34:49.531609+0000","last_became_active":"2024-09-16T07:34:47.513084+0000","last_became_peered":"2024-09-16T07:34:47.513084+0000","last_unstale":"2024-09-16T07:34:49.531609+0000","last_undegraded":"2024-09-16T07:34:49.531609+0000","last_fullsized":"2024-09-16T07:34:49.531609+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:49.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.533116+0000","last_change":"2024-09-16T07:34:48.526515+0000","last_active":"2024-09-16T07:34:48.533116+0000","last_peered":"2024-09-16T07:34:48.533116+0000","last_clean":"2024-09-16T07:34:48.533116+0000","last_became_active":"2024-09-16T07:34:48.525964+0000","last_became_peered":"2024-09-16T07:34:48.525964+0000","last_unstale":"2024-09-16T07:34:48.533116+0000","last_undegraded":"2024-09-16T07:34:48.533116+0000","last_fullsized":"2024-09-16T07:34:48.533116+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:11.702095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.520393+0000","last_change":"2024-09-16T07:34:47.518568+0000","last_active":"2024-09-16T07:34:48.520393+0000","last_peered":"2024-09-16T07:34:48.520393+0000","last_clean":"2024-09-16T07:34:48.520393+0000","last_became_active":"2024-09-16T07:34:47.518279+0000","last_became_peered":"2024-09-16T07:34:47.518279+0000","last_unstale":"2024-09-16T07:34:48.520393+0000","last_undegraded":"2024-09-16T07:34:48.520393+0000","last_fullsized":"2024-09-16T07:34:48.520393+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:36.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.534614+0000","last_change":"2024-09-16T07:34:48.528073+0000","last_active":"2024-09-16T07:34:48.534614+0000","last_peered":"2024-09-16T07:34:48.534614+0000","last_clean":"2024-09-16T07:34:48.534614+0000","last_became_active":"2024-09-16T07:34:48.527807+0000","last_became_peered":"2024-09-16T07:34:48.527807+0000","last_unstale":"2024-09-16T07:34:48.534614+0000","last_undegraded":"2024-09-16T07:34:48.534614+0000","last_fullsized":"2024-09-16T07:34:48.534614+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:22.775050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.514866+0000","last_change":"2024-09-16T07:34:47.527175+0000","last_active":"2024-09-16T07:34:48.514866+0000","last_peered":"2024-09-16T07:34:48.514866+0000","last_clean":"2024-09-16T07:34:48.514866+0000","last_became_active":"2024-09-16T07:34:47.526929+0000","last_became_peered":"2024-09-16T07:34:47.526929+0000","last_unstale":"2024-09-16T07:34:48.514866+0000","last_undegraded":"2024-09-16T07:34:48.514866+0000","last_fullsized":"2024-09-16T07:34:48.514866+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:56.405916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.527000+0000","last_change":"2024-09-16T07:34:48.524675+0000","last_active":"2024-09-16T07:34:48.527000+0000","last_peered":"2024-09-16T07:34:48.527000+0000","last_clean":"2024-09-16T07:34:48.527000+0000","last_became_active":"2024-09-16T07:34:48.524324+0000","last_became_peered":"2024-09-16T07:34:48.524324+0000","last_unstale":"2024-09-16T07:34:48.527000+0000","last_undegraded":"2024-09-16T07:34:48.527000+0000","last_fullsized":"2024-09-16T07:34:48.527000+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:55.577009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.527035+0000","last_change":"2024-09-16T07:34:47.525571+0000","last_active":"2024-09-16T07:34:47.527035+0000","last_peered":"2024-09-16T07:34:47.527035+0000","last_clean":"2024-09-16T07:34:47.527035+0000","last_became_active":"2024-09-16T07:34:47.525295+0000","last_became_peered":"2024-09-16T07:34:47.525295+0000","last_unstale":"2024-09-16T07:34:47.527035+0000","last_undegraded":"2024-09-16T07:34:47.527035+0000","last_fullsized":"2024-09-16T07:34:47.527035+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:53.256795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-16T07:34:47.506924+0000","last_change":"2024-09-16T07:34:47.506924+0000","last_active":"2024-09-16T07:34:47.492910+0000","last_peered":"2024-09-16T07:34:47.492910+0000","last_clean":"2024-09-16T07:34:47.492910+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:34:47.506924+0000","last_undegraded":"2024-09-16T07:34:47.506924+0000","last_fullsized":"2024-09-16T07:34:47.506924+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-16T07:34:47.520862+0000","last_change":"2024-09-16T07:34:47.518753+0000","last_active":"2024-09-16T07:34:47.520862+0000","last_peered":"2024-09-16T07:34:47.520862+0000","last_clean":"2024-09-16T07:34:47.520862+0000","last_became_active":"2024-09-16T07:34:47.518486+0000","last_became_peered":"2024-09-16T07:34:47.518486+0000","last_unstale":"2024-09-16T07:34:47.520862+0000","last_undegraded":"2024-09-16T07:34:47.520862+0000","last_fullsized":"2024-09-16T07:34:47.520862+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:21.437105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531155+0000","last_change":"2024-09-16T07:34:48.533301+0000","last_active":"2024-09-16T07:34:49.531155+0000","last_peered":"2024-09-16T07:34:49.531155+0000","last_clean":"2024-09-16T07:34:49.531155+0000","last_became_active":"2024-09-16T07:34:48.532697+0000","last_became_peered":"2024-09-16T07:34:48.532697+0000","last_unstale":"2024-09-16T07:34:49.531155+0000","last_undegraded":"2024-09-16T07:34:49.531155+0000","last_fullsized":"2024-09-16T07:34:49.531155+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:14:10.393373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.520135+0000","last_change":"2024-09-16T07:34:47.521842+0000","last_active":"2024-09-16T07:34:48.520135+0000","last_peered":"2024-09-16T07:34:48.520135+0000","last_clean":"2024-09-16T07:34:48.520135+0000","last_became_active":"2024-09-16T07:34:47.521578+0000","last_became_peered":"2024-09-16T07:34:47.521578+0000","last_unstale":"2024-09-16T07:34:48.520135+0000","last_undegraded":"2024-09-16T07:34:48.520135+0000","last_fullsized":"2024-09-16T07:34:48.520135+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:20.091370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":288,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T07:34:48.507819+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:34:48.507819+0000","last_peered":"2024-09-16T07:34:48.507819+0000","last_clean":"2024-09-16T07:34:48.507819+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:34:48.507819+0000","last_undegraded":"2024-09-16T07:34:48.507819+0000","last_fullsized":"2024-09-16T07:34:48.507819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2264,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126944,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244896,"statfs":{"total":96636764160,"available":96506773504,"internally_reserved":0,"allocated":1454080,"data_stored":635315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126932,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244908,"statfs":{"total":96636764160,"available":96506785792,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123352,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248488,"statfs":{"total":96636764160,"available":96510451712,"internally_reserved":0,"allocated":2035712,"data_stored":1218852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140820,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231020,"statfs":{"total":96636764160,"available":96492564480,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509588,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122768,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249072,"statfs":{"total":96636764160,"available":96511049728,"internally_reserved":0,"allocated":1429504,"data_stored":622153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509590,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153308,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218532,"statfs":{"total":96636764160,"available":96479776768,"internally_reserved":0,"allocated":1441792,"data_stored":628484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131672,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240168,"statfs":{"total":96636764160,"available":96501932032,"internally_reserved":0,"allocated":2035712,"data_stored":1218852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126976,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244864,"statfs":{"total":96636764160,"available":96506740736,"internally_reserved":0,"allocated":1482752,"data_stored":643373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:34:57.411 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:34:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.579+0000 7f962fa00640 1 -- 172.21.15.71:0/3684044986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96280ccf00 msgr2=0x7f96280acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.579+0000 7f962fa00640 1 --2- 172.21.15.71:0/3684044986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96280ccf00 0x7f96280acd30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9614005fd0 tx=0x7f961402a540 comp rx=0 tx=0).stop 2024-09-16T07:34:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.579+0000 7f962fa00640 1 -- 172.21.15.71:0/3684044986 shutdown_connections 2024-09-16T07:34:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.579+0000 7f962fa00640 1 --2- 172.21.15.71:0/3684044986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96280ccf00 0x7f96280acd30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.579+0000 7f962fa00640 1 --2- 172.21.15.71:0/3684044986 >> v2:172.21.15.177:3300/0 conn(0x7f962815f5a0 0x7f96280b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.579+0000 7f962fa00640 1 -- 172.21.15.71:0/3684044986 >> 172.21.15.71:0/3684044986 conn(0x7f96280b1950 msgr2=0x7f96280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.579+0000 7f962fa00640 1 -- 172.21.15.71:0/3684044986 shutdown_connections 2024-09-16T07:34:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.579+0000 7f962fa00640 1 -- 172.21.15.71:0/3684044986 wait complete. 2024-09-16T07:34:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.581+0000 7f962fa00640 1 Processor -- start 2024-09-16T07:34:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.581+0000 7f962fa00640 1 -- start start 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962fa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96280ad270 0x7f96280b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962fa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96280ccf00 0x7f962815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962fa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962815f5a0 0x7f962815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962fa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f962806f2e0 con 0x7f96280ad270 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962fa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f962806f160 con 0x7f96280ccf00 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962fa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f962806f460 con 0x7f962815f5a0 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96280ad270 0x7f96280b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96280ad270 0x7f96280b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33598/0 (socket says 172.21.15.71:33598) 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962d000640 1 -- 172.21.15.71:0/2351649044 learned_addr learned my addr 172.21.15.71:0/2351649044 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.582+0000 7f962da00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962815f5a0 0x7f962815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:57.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f9627e00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96280ccf00 0x7f962815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f962d000640 1 -- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962815f5a0 msgr2=0x7f962815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f962d000640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962815f5a0 0x7f962815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f962d000640 1 -- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96280ccf00 msgr2=0x7f962815b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f962d000640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96280ccf00 0x7f962815b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f962d000640 1 -- 172.21.15.71:0/2351649044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96281f83a0 con 0x7f96280ad270 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f962d000640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96280ad270 0x7f96280b46e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f961800ece0 tx=0x7f961800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f9624c00640 1 -- 172.21.15.71:0/2351649044 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618003b70 con 0x7f96280ad270 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f9624c00640 1 -- 172.21.15.71:0/2351649044 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9618003d10 con 0x7f96280ad270 2024-09-16T07:34:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f9624c00640 1 -- 172.21.15.71:0/2351649044 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618004280 con 0x7f96280ad270 2024-09-16T07:34:57.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.583+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9628157ae0 con 0x7f96280ad270 2024-09-16T07:34:57.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.584+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281f8920 con 0x7f96280ad270 2024-09-16T07:34:57.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.585+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962806f9f0 con 0x7f96280ad270 2024-09-16T07:34:57.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.586+0000 7f9624c00640 1 -- 172.21.15.71:0/2351649044 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f9618004420 con 0x7f96280ad270 2024-09-16T07:34:57.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.587+0000 7f9624c00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f960807a230 0x7f960807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:57.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.587+0000 7f9624c00640 1 -- 172.21.15.71:0/2351649044 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f961809a480 con 0x7f96280ad270 2024-09-16T07:34:57.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.587+0000 7f9627e00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f960807a230 0x7f960807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:57.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.588+0000 7f9627e00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f960807a230 0x7f960807c6f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9614033000 tx=0x7f9614004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:57.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.591+0000 7f9624c00640 1 -- 172.21.15.71:0/2351649044 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9618063f00 con 0x7f96280ad270 2024-09-16T07:34:57.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.771+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f962809d8a0 con 0x7f960807a230 2024-09-16T07:34:57.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.776+0000 7f9624c00640 1 -- 172.21.15.71:0/2351649044 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252459 (secure 0 0 0) 0x7f962809d8a0 con 0x7f960807a230 2024-09-16T07:34:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:57.779 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f960807a230 msgr2=0x7f960807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f960807a230 0x7f960807c6f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9614033000 tx=0x7f9614004730 comp rx=0 tx=0).stop 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96280ad270 msgr2=0x7f96280b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96280ad270 0x7f96280b46e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f961800ece0 tx=0x7f961800c6a0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 shutdown_connections 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f960807a230 0x7f960807c6f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f962815f5a0 0x7f962815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96280ccf00 0x7f962815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 --2- 172.21.15.71:0/2351649044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96280ad270 0x7f96280b46e0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:57.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 >> 172.21.15.71:0/2351649044 conn(0x7f96280b1950 msgr2=0x7f962815e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:57.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 shutdown_connections 2024-09-16T07:34:57.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:57.780+0000 7f962fa00640 1 -- 172.21.15.71:0/2351649044 wait complete. 2024-09-16T07:34:57.797 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-16T07:34:56.605514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1053016,"kb_used_data":12680,"kb_used_omap":12,"kb_used_meta":1040115,"kb_avail":753921704,"statfs":{"total":773094113280,"available":772015824896,"internally_reserved":0,"allocated":12984320,"data_stored":6318924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1065078320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.099702"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554081+0000","last_change":"2024-09-16T07:34:48.527176+0000","last_active":"2024-09-16T07:34:49.554081+0000","last_peered":"2024-09-16T07:34:49.554081+0000","last_clean":"2024-09-16T07:34:49.554081+0000","last_became_active":"2024-09-16T07:34:48.526305+0000","last_became_peered":"2024-09-16T07:34:48.526305+0000","last_unstale":"2024-09-16T07:34:49.554081+0000","last_undegraded":"2024-09-16T07:34:49.554081+0000","last_fullsized":"2024-09-16T07:34:49.554081+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:37.444763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553141+0000","last_change":"2024-09-16T07:34:48.541678+0000","last_active":"2024-09-16T07:34:49.553141+0000","last_peered":"2024-09-16T07:34:49.553141+0000","last_clean":"2024-09-16T07:34:49.553141+0000","last_became_active":"2024-09-16T07:34:48.541360+0000","last_became_peered":"2024-09-16T07:34:48.541360+0000","last_unstale":"2024-09-16T07:34:49.553141+0000","last_undegraded":"2024-09-16T07:34:49.553141+0000","last_fullsized":"2024-09-16T07:34:49.553141+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:54.031210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527829+0000","last_change":"2024-09-16T07:34:48.534129+0000","last_active":"2024-09-16T07:34:49.527829+0000","last_peered":"2024-09-16T07:34:49.527829+0000","last_clean":"2024-09-16T07:34:49.527829+0000","last_became_active":"2024-09-16T07:34:48.533584+0000","last_became_peered":"2024-09-16T07:34:48.533584+0000","last_unstale":"2024-09-16T07:34:49.527829+0000","last_undegraded":"2024-09-16T07:34:49.527829+0000","last_fullsized":"2024-09-16T07:34:49.527829+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:56.381392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531563+0000","last_change":"2024-09-16T07:34:48.526281+0000","last_active":"2024-09-16T07:34:49.531563+0000","last_peered":"2024-09-16T07:34:49.531563+0000","last_clean":"2024-09-16T07:34:49.531563+0000","last_became_active":"2024-09-16T07:34:48.525843+0000","last_became_peered":"2024-09-16T07:34:48.525843+0000","last_unstale":"2024-09-16T07:34:49.531563+0000","last_undegraded":"2024-09-16T07:34:49.531563+0000","last_fullsized":"2024-09-16T07:34:49.531563+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:50.039555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552802+0000","last_change":"2024-09-16T07:34:48.538786+0000","last_active":"2024-09-16T07:34:49.552802+0000","last_peered":"2024-09-16T07:34:49.552802+0000","last_clean":"2024-09-16T07:34:49.552802+0000","last_became_active":"2024-09-16T07:34:48.538497+0000","last_became_peered":"2024-09-16T07:34:48.538497+0000","last_unstale":"2024-09-16T07:34:49.552802+0000","last_undegraded":"2024-09-16T07:34:49.552802+0000","last_fullsized":"2024-09-16T07:34:49.552802+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:55.768025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533075+0000","last_change":"2024-09-16T07:34:48.537870+0000","last_active":"2024-09-16T07:34:49.533075+0000","last_peered":"2024-09-16T07:34:49.533075+0000","last_clean":"2024-09-16T07:34:49.533075+0000","last_became_active":"2024-09-16T07:34:48.537608+0000","last_became_peered":"2024-09-16T07:34:48.537608+0000","last_unstale":"2024-09-16T07:34:49.533075+0000","last_undegraded":"2024-09-16T07:34:49.533075+0000","last_fullsized":"2024-09-16T07:34:49.533075+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:06.655363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553551+0000","last_change":"2024-09-16T07:34:48.526151+0000","last_active":"2024-09-16T07:34:49.553551+0000","last_peered":"2024-09-16T07:34:49.553551+0000","last_clean":"2024-09-16T07:34:49.553551+0000","last_became_active":"2024-09-16T07:34:48.525914+0000","last_became_peered":"2024-09-16T07:34:48.525914+0000","last_unstale":"2024-09-16T07:34:49.553551+0000","last_undegraded":"2024-09-16T07:34:49.553551+0000","last_fullsized":"2024-09-16T07:34:49.553551+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:09.375911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552969+0000","last_change":"2024-09-16T07:34:48.528915+0000","last_active":"2024-09-16T07:34:49.552969+0000","last_peered":"2024-09-16T07:34:49.552969+0000","last_clean":"2024-09-16T07:34:49.552969+0000","last_became_active":"2024-09-16T07:34:48.528648+0000","last_became_peered":"2024-09-16T07:34:48.528648+0000","last_unstale":"2024-09-16T07:34:49.552969+0000","last_undegraded":"2024-09-16T07:34:49.552969+0000","last_fullsized":"2024-09-16T07:34:49.552969+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:02:07.450037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554195+0000","last_change":"2024-09-16T07:34:48.530431+0000","last_active":"2024-09-16T07:34:49.554195+0000","last_peered":"2024-09-16T07:34:49.554195+0000","last_clean":"2024-09-16T07:34:49.554195+0000","last_became_active":"2024-09-16T07:34:48.529993+0000","last_became_peered":"2024-09-16T07:34:48.529993+0000","last_unstale":"2024-09-16T07:34:49.554195+0000","last_undegraded":"2024-09-16T07:34:49.554195+0000","last_fullsized":"2024-09-16T07:34:49.554195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:51.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552769+0000","last_change":"2024-09-16T07:34:48.541592+0000","last_active":"2024-09-16T07:34:49.552769+0000","last_peered":"2024-09-16T07:34:49.552769+0000","last_clean":"2024-09-16T07:34:49.552769+0000","last_became_active":"2024-09-16T07:34:48.540848+0000","last_became_peered":"2024-09-16T07:34:48.540848+0000","last_unstale":"2024-09-16T07:34:49.552769+0000","last_undegraded":"2024-09-16T07:34:49.552769+0000","last_fullsized":"2024-09-16T07:34:49.552769+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:38.996763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554041+0000","last_change":"2024-09-16T07:34:48.530703+0000","last_active":"2024-09-16T07:34:49.554041+0000","last_peered":"2024-09-16T07:34:49.554041+0000","last_clean":"2024-09-16T07:34:49.554041+0000","last_became_active":"2024-09-16T07:34:48.530406+0000","last_became_peered":"2024-09-16T07:34:48.530406+0000","last_unstale":"2024-09-16T07:34:49.554041+0000","last_undegraded":"2024-09-16T07:34:49.554041+0000","last_fullsized":"2024-09-16T07:34:49.554041+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:18.571722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552658+0000","last_change":"2024-09-16T07:34:48.547086+0000","last_active":"2024-09-16T07:34:49.552658+0000","last_peered":"2024-09-16T07:34:49.552658+0000","last_clean":"2024-09-16T07:34:49.552658+0000","last_became_active":"2024-09-16T07:34:48.546695+0000","last_became_peered":"2024-09-16T07:34:48.546695+0000","last_unstale":"2024-09-16T07:34:49.552658+0000","last_undegraded":"2024-09-16T07:34:49.552658+0000","last_fullsized":"2024-09-16T07:34:49.552658+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:22:56.680697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552691+0000","last_change":"2024-09-16T07:34:48.535432+0000","last_active":"2024-09-16T07:34:49.552691+0000","last_peered":"2024-09-16T07:34:49.552691+0000","last_clean":"2024-09-16T07:34:49.552691+0000","last_became_active":"2024-09-16T07:34:48.532194+0000","last_became_peered":"2024-09-16T07:34:48.532194+0000","last_unstale":"2024-09-16T07:34:49.552691+0000","last_undegraded":"2024-09-16T07:34:49.552691+0000","last_fullsized":"2024-09-16T07:34:49.552691+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:36.742379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552524+0000","last_change":"2024-09-16T07:34:48.531769+0000","last_active":"2024-09-16T07:34:49.552524+0000","last_peered":"2024-09-16T07:34:49.552524+0000","last_clean":"2024-09-16T07:34:49.552524+0000","last_became_active":"2024-09-16T07:34:48.531251+0000","last_became_peered":"2024-09-16T07:34:48.531251+0000","last_unstale":"2024-09-16T07:34:49.552524+0000","last_undegraded":"2024-09-16T07:34:49.552524+0000","last_fullsized":"2024-09-16T07:34:49.552524+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:28.817214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553616+0000","last_change":"2024-09-16T07:34:48.529706+0000","last_active":"2024-09-16T07:34:49.553616+0000","last_peered":"2024-09-16T07:34:49.553616+0000","last_clean":"2024-09-16T07:34:49.553616+0000","last_became_active":"2024-09-16T07:34:48.529416+0000","last_became_peered":"2024-09-16T07:34:48.529416+0000","last_unstale":"2024-09-16T07:34:49.553616+0000","last_undegraded":"2024-09-16T07:34:49.553616+0000","last_fullsized":"2024-09-16T07:34:49.553616+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:57.398462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552319+0000","last_change":"2024-09-16T07:34:48.531422+0000","last_active":"2024-09-16T07:34:49.552319+0000","last_peered":"2024-09-16T07:34:49.552319+0000","last_clean":"2024-09-16T07:34:49.552319+0000","last_became_active":"2024-09-16T07:34:48.531139+0000","last_became_peered":"2024-09-16T07:34:48.531139+0000","last_unstale":"2024-09-16T07:34:49.552319+0000","last_undegraded":"2024-09-16T07:34:49.552319+0000","last_fullsized":"2024-09-16T07:34:49.552319+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:41.469054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527890+0000","last_change":"2024-09-16T07:34:48.520786+0000","last_active":"2024-09-16T07:34:49.527890+0000","last_peered":"2024-09-16T07:34:49.527890+0000","last_clean":"2024-09-16T07:34:49.527890+0000","last_became_active":"2024-09-16T07:34:48.519475+0000","last_became_peered":"2024-09-16T07:34:48.519475+0000","last_unstale":"2024-09-16T07:34:49.527890+0000","last_undegraded":"2024-09-16T07:34:49.527890+0000","last_fullsized":"2024-09-16T07:34:49.527890+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:22:56.680697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554051+0000","last_change":"2024-09-16T07:34:48.527942+0000","last_active":"2024-09-16T07:34:49.554051+0000","last_peered":"2024-09-16T07:34:49.554051+0000","last_clean":"2024-09-16T07:34:49.554051+0000","last_became_active":"2024-09-16T07:34:48.527669+0000","last_became_peered":"2024-09-16T07:34:48.527669+0000","last_unstale":"2024-09-16T07:34:49.554051+0000","last_undegraded":"2024-09-16T07:34:49.554051+0000","last_fullsized":"2024-09-16T07:34:49.554051+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:20.404773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527850+0000","last_change":"2024-09-16T07:34:48.534101+0000","last_active":"2024-09-16T07:34:49.527850+0000","last_peered":"2024-09-16T07:34:49.527850+0000","last_clean":"2024-09-16T07:34:49.527850+0000","last_became_active":"2024-09-16T07:34:48.529574+0000","last_became_peered":"2024-09-16T07:34:48.529574+0000","last_unstale":"2024-09-16T07:34:49.527850+0000","last_undegraded":"2024-09-16T07:34:49.527850+0000","last_fullsized":"2024-09-16T07:34:49.527850+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:03.152562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527456+0000","last_change":"2024-09-16T07:34:48.529587+0000","last_active":"2024-09-16T07:34:49.527456+0000","last_peered":"2024-09-16T07:34:49.527456+0000","last_clean":"2024-09-16T07:34:49.527456+0000","last_became_active":"2024-09-16T07:34:48.529304+0000","last_became_peered":"2024-09-16T07:34:48.529304+0000","last_unstale":"2024-09-16T07:34:49.527456+0000","last_undegraded":"2024-09-16T07:34:49.527456+0000","last_fullsized":"2024-09-16T07:34:49.527456+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:35.101409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554078+0000","last_change":"2024-09-16T07:34:48.533118+0000","last_active":"2024-09-16T07:34:49.554078+0000","last_peered":"2024-09-16T07:34:49.554078+0000","last_clean":"2024-09-16T07:34:49.554078+0000","last_became_active":"2024-09-16T07:34:48.532777+0000","last_became_peered":"2024-09-16T07:34:48.532777+0000","last_unstale":"2024-09-16T07:34:49.554078+0000","last_undegraded":"2024-09-16T07:34:49.554078+0000","last_fullsized":"2024-09-16T07:34:49.554078+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:56.774434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533126+0000","last_change":"2024-09-16T07:34:48.531641+0000","last_active":"2024-09-16T07:34:49.533126+0000","last_peered":"2024-09-16T07:34:49.533126+0000","last_clean":"2024-09-16T07:34:49.533126+0000","last_became_active":"2024-09-16T07:34:48.531171+0000","last_became_peered":"2024-09-16T07:34:48.531171+0000","last_unstale":"2024-09-16T07:34:49.533126+0000","last_undegraded":"2024-09-16T07:34:49.533126+0000","last_fullsized":"2024-09-16T07:34:49.533126+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:02.001458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552312+0000","last_change":"2024-09-16T07:34:48.528984+0000","last_active":"2024-09-16T07:34:49.552312+0000","last_peered":"2024-09-16T07:34:49.552312+0000","last_clean":"2024-09-16T07:34:49.552312+0000","last_became_active":"2024-09-16T07:34:48.528481+0000","last_became_peered":"2024-09-16T07:34:48.528481+0000","last_unstale":"2024-09-16T07:34:49.552312+0000","last_undegraded":"2024-09-16T07:34:49.552312+0000","last_fullsized":"2024-09-16T07:34:49.552312+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:51.151910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530706+0000","last_change":"2024-09-16T07:34:48.533570+0000","last_active":"2024-09-16T07:34:49.530706+0000","last_peered":"2024-09-16T07:34:49.530706+0000","last_clean":"2024-09-16T07:34:49.530706+0000","last_became_active":"2024-09-16T07:34:48.533039+0000","last_became_peered":"2024-09-16T07:34:48.533039+0000","last_unstale":"2024-09-16T07:34:49.530706+0000","last_undegraded":"2024-09-16T07:34:49.530706+0000","last_fullsized":"2024-09-16T07:34:49.530706+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:00:09.699334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554281+0000","last_change":"2024-09-16T07:34:48.527254+0000","last_active":"2024-09-16T07:34:49.554281+0000","last_peered":"2024-09-16T07:34:49.554281+0000","last_clean":"2024-09-16T07:34:49.554281+0000","last_became_active":"2024-09-16T07:34:48.527013+0000","last_became_peered":"2024-09-16T07:34:48.527013+0000","last_unstale":"2024-09-16T07:34:49.554281+0000","last_undegraded":"2024-09-16T07:34:49.554281+0000","last_fullsized":"2024-09-16T07:34:49.554281+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:05:23.255801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533313+0000","last_change":"2024-09-16T07:34:48.515667+0000","last_active":"2024-09-16T07:34:49.533313+0000","last_peered":"2024-09-16T07:34:49.533313+0000","last_clean":"2024-09-16T07:34:49.533313+0000","last_became_active":"2024-09-16T07:34:48.515389+0000","last_became_peered":"2024-09-16T07:34:48.515389+0000","last_unstale":"2024-09-16T07:34:49.533313+0000","last_undegraded":"2024-09-16T07:34:49.533313+0000","last_fullsized":"2024-09-16T07:34:49.533313+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:30.045069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531251+0000","last_change":"2024-09-16T07:34:48.541120+0000","last_active":"2024-09-16T07:34:49.531251+0000","last_peered":"2024-09-16T07:34:49.531251+0000","last_clean":"2024-09-16T07:34:49.531251+0000","last_became_active":"2024-09-16T07:34:48.540074+0000","last_became_peered":"2024-09-16T07:34:48.540074+0000","last_unstale":"2024-09-16T07:34:49.531251+0000","last_undegraded":"2024-09-16T07:34:49.531251+0000","last_fullsized":"2024-09-16T07:34:49.531251+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:53.929219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533273+0000","last_change":"2024-09-16T07:34:48.535196+0000","last_active":"2024-09-16T07:34:49.533273+0000","last_peered":"2024-09-16T07:34:49.533273+0000","last_clean":"2024-09-16T07:34:49.533273+0000","last_became_active":"2024-09-16T07:34:48.534863+0000","last_became_peered":"2024-09-16T07:34:48.534863+0000","last_unstale":"2024-09-16T07:34:49.533273+0000","last_undegraded":"2024-09-16T07:34:49.533273+0000","last_fullsized":"2024-09-16T07:34:49.533273+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:55.097740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552947+0000","last_change":"2024-09-16T07:34:48.534695+0000","last_active":"2024-09-16T07:34:49.552947+0000","last_peered":"2024-09-16T07:34:49.552947+0000","last_clean":"2024-09-16T07:34:49.552947+0000","last_became_active":"2024-09-16T07:34:48.534435+0000","last_became_peered":"2024-09-16T07:34:48.534435+0000","last_unstale":"2024-09-16T07:34:49.552947+0000","last_undegraded":"2024-09-16T07:34:49.552947+0000","last_fullsized":"2024-09-16T07:34:49.552947+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:34.460098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553679+0000","last_change":"2024-09-16T07:34:48.525252+0000","last_active":"2024-09-16T07:34:49.553679+0000","last_peered":"2024-09-16T07:34:49.553679+0000","last_clean":"2024-09-16T07:34:49.553679+0000","last_became_active":"2024-09-16T07:34:48.524453+0000","last_became_peered":"2024-09-16T07:34:48.524453+0000","last_unstale":"2024-09-16T07:34:49.553679+0000","last_undegraded":"2024-09-16T07:34:49.553679+0000","last_fullsized":"2024-09-16T07:34:49.553679+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:02.267512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553634+0000","last_change":"2024-09-16T07:34:48.520747+0000","last_active":"2024-09-16T07:34:49.553634+0000","last_peered":"2024-09-16T07:34:49.553634+0000","last_clean":"2024-09-16T07:34:49.553634+0000","last_became_active":"2024-09-16T07:34:48.520474+0000","last_became_peered":"2024-09-16T07:34:48.520474+0000","last_unstale":"2024-09-16T07:34:49.553634+0000","last_undegraded":"2024-09-16T07:34:49.553634+0000","last_fullsized":"2024-09-16T07:34:49.553634+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:46:56.079763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552318+0000","last_change":"2024-09-16T07:34:48.527776+0000","last_active":"2024-09-16T07:34:49.552318+0000","last_peered":"2024-09-16T07:34:49.552318+0000","last_clean":"2024-09-16T07:34:49.552318+0000","last_became_active":"2024-09-16T07:34:48.527420+0000","last_became_peered":"2024-09-16T07:34:48.527420+0000","last_unstale":"2024-09-16T07:34:49.552318+0000","last_undegraded":"2024-09-16T07:34:49.552318+0000","last_fullsized":"2024-09-16T07:34:49.552318+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:43.572025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554249+0000","last_change":"2024-09-16T07:34:48.532763+0000","last_active":"2024-09-16T07:34:49.554249+0000","last_peered":"2024-09-16T07:34:49.554249+0000","last_clean":"2024-09-16T07:34:49.554249+0000","last_became_active":"2024-09-16T07:34:48.532289+0000","last_became_peered":"2024-09-16T07:34:48.532289+0000","last_unstale":"2024-09-16T07:34:49.554249+0000","last_undegraded":"2024-09-16T07:34:49.554249+0000","last_fullsized":"2024-09-16T07:34:49.554249+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:14.274828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"118'4","reported_seq":19,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553695+0000","last_change":"2024-09-16T07:34:47.528565+0000","last_active":"2024-09-16T07:34:49.553695+0000","last_peered":"2024-09-16T07:34:49.553695+0000","last_clean":"2024-09-16T07:34:49.553695+0000","last_became_active":"2024-09-16T07:34:47.528270+0000","last_became_peered":"2024-09-16T07:34:47.528270+0000","last_unstale":"2024-09-16T07:34:49.553695+0000","last_undegraded":"2024-09-16T07:34:49.553695+0000","last_fullsized":"2024-09-16T07:34:49.553695+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:27.293106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553686+0000","last_change":"2024-09-16T07:34:48.529917+0000","last_active":"2024-09-16T07:34:49.553686+0000","last_peered":"2024-09-16T07:34:49.553686+0000","last_clean":"2024-09-16T07:34:49.553686+0000","last_became_active":"2024-09-16T07:34:48.529631+0000","last_became_peered":"2024-09-16T07:34:48.529631+0000","last_unstale":"2024-09-16T07:34:49.553686+0000","last_undegraded":"2024-09-16T07:34:49.553686+0000","last_fullsized":"2024-09-16T07:34:49.553686+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:21:01.979669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553277+0000","last_change":"2024-09-16T07:34:47.526481+0000","last_active":"2024-09-16T07:34:49.553277+0000","last_peered":"2024-09-16T07:34:49.553277+0000","last_clean":"2024-09-16T07:34:49.553277+0000","last_became_active":"2024-09-16T07:34:47.526215+0000","last_became_peered":"2024-09-16T07:34:47.526215+0000","last_unstale":"2024-09-16T07:34:49.553277+0000","last_undegraded":"2024-09-16T07:34:49.553277+0000","last_fullsized":"2024-09-16T07:34:49.553277+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:53:08.617410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533538+0000","last_change":"2024-09-16T07:34:48.526954+0000","last_active":"2024-09-16T07:34:49.533538+0000","last_peered":"2024-09-16T07:34:49.533538+0000","last_clean":"2024-09-16T07:34:49.533538+0000","last_became_active":"2024-09-16T07:34:48.526645+0000","last_became_peered":"2024-09-16T07:34:48.526645+0000","last_unstale":"2024-09-16T07:34:49.533538+0000","last_undegraded":"2024-09-16T07:34:49.533538+0000","last_fullsized":"2024-09-16T07:34:49.533538+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:38.150105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531333+0000","last_change":"2024-09-16T07:34:47.522199+0000","last_active":"2024-09-16T07:34:49.531333+0000","last_peered":"2024-09-16T07:34:49.531333+0000","last_clean":"2024-09-16T07:34:49.531333+0000","last_became_active":"2024-09-16T07:34:47.521943+0000","last_became_peered":"2024-09-16T07:34:47.521943+0000","last_unstale":"2024-09-16T07:34:49.531333+0000","last_undegraded":"2024-09-16T07:34:49.531333+0000","last_fullsized":"2024-09-16T07:34:49.531333+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:08.548657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527670+0000","last_change":"2024-09-16T07:34:48.533772+0000","last_active":"2024-09-16T07:34:49.527670+0000","last_peered":"2024-09-16T07:34:49.527670+0000","last_clean":"2024-09-16T07:34:49.527670+0000","last_became_active":"2024-09-16T07:34:48.529223+0000","last_became_peered":"2024-09-16T07:34:48.529223+0000","last_unstale":"2024-09-16T07:34:49.527670+0000","last_undegraded":"2024-09-16T07:34:49.527670+0000","last_fullsized":"2024-09-16T07:34:49.527670+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:15.508360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531454+0000","last_change":"2024-09-16T07:34:47.523452+0000","last_active":"2024-09-16T07:34:49.531454+0000","last_peered":"2024-09-16T07:34:49.531454+0000","last_clean":"2024-09-16T07:34:49.531454+0000","last_became_active":"2024-09-16T07:34:47.521451+0000","last_became_peered":"2024-09-16T07:34:47.521451+0000","last_unstale":"2024-09-16T07:34:49.531454+0000","last_undegraded":"2024-09-16T07:34:49.531454+0000","last_fullsized":"2024-09-16T07:34:49.531454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.908074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531052+0000","last_change":"2024-09-16T07:34:48.538587+0000","last_active":"2024-09-16T07:34:49.531052+0000","last_peered":"2024-09-16T07:34:49.531052+0000","last_clean":"2024-09-16T07:34:49.531052+0000","last_became_active":"2024-09-16T07:34:48.538287+0000","last_became_peered":"2024-09-16T07:34:48.538287+0000","last_unstale":"2024-09-16T07:34:49.531052+0000","last_undegraded":"2024-09-16T07:34:49.531052+0000","last_fullsized":"2024-09-16T07:34:49.531052+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:22.418271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527420+0000","last_change":"2024-09-16T07:34:47.520652+0000","last_active":"2024-09-16T07:34:49.527420+0000","last_peered":"2024-09-16T07:34:49.527420+0000","last_clean":"2024-09-16T07:34:49.527420+0000","last_became_active":"2024-09-16T07:34:47.520279+0000","last_became_peered":"2024-09-16T07:34:47.520279+0000","last_unstale":"2024-09-16T07:34:49.527420+0000","last_undegraded":"2024-09-16T07:34:49.527420+0000","last_fullsized":"2024-09-16T07:34:49.527420+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:17.809818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553996+0000","last_change":"2024-09-16T07:34:48.532853+0000","last_active":"2024-09-16T07:34:49.553996+0000","last_peered":"2024-09-16T07:34:49.553996+0000","last_clean":"2024-09-16T07:34:49.553996+0000","last_became_active":"2024-09-16T07:34:48.532447+0000","last_became_peered":"2024-09-16T07:34:48.532447+0000","last_unstale":"2024-09-16T07:34:49.553996+0000","last_undegraded":"2024-09-16T07:34:49.553996+0000","last_fullsized":"2024-09-16T07:34:49.553996+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:34:48.511658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553896+0000","last_change":"2024-09-16T07:34:47.527304+0000","last_active":"2024-09-16T07:34:49.553896+0000","last_peered":"2024-09-16T07:34:49.553896+0000","last_clean":"2024-09-16T07:34:49.553896+0000","last_became_active":"2024-09-16T07:34:47.526260+0000","last_became_peered":"2024-09-16T07:34:47.526260+0000","last_unstale":"2024-09-16T07:34:49.553896+0000","last_undegraded":"2024-09-16T07:34:49.553896+0000","last_fullsized":"2024-09-16T07:34:49.553896+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:38:41.657937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552588+0000","last_change":"2024-09-16T07:34:48.542723+0000","last_active":"2024-09-16T07:34:49.552588+0000","last_peered":"2024-09-16T07:34:49.552588+0000","last_clean":"2024-09-16T07:34:49.552588+0000","last_became_active":"2024-09-16T07:34:48.542469+0000","last_became_peered":"2024-09-16T07:34:48.542469+0000","last_unstale":"2024-09-16T07:34:49.552588+0000","last_undegraded":"2024-09-16T07:34:49.552588+0000","last_fullsized":"2024-09-16T07:34:49.552588+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:02:11.262607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553867+0000","last_change":"2024-09-16T07:34:47.520199+0000","last_active":"2024-09-16T07:34:49.553867+0000","last_peered":"2024-09-16T07:34:49.553867+0000","last_clean":"2024-09-16T07:34:49.553867+0000","last_became_active":"2024-09-16T07:34:47.519941+0000","last_became_peered":"2024-09-16T07:34:47.519941+0000","last_unstale":"2024-09-16T07:34:49.553867+0000","last_undegraded":"2024-09-16T07:34:49.553867+0000","last_fullsized":"2024-09-16T07:34:49.553867+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:46:54.626775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553731+0000","last_change":"2024-09-16T07:34:48.534789+0000","last_active":"2024-09-16T07:34:49.553731+0000","last_peered":"2024-09-16T07:34:49.553731+0000","last_clean":"2024-09-16T07:34:49.553731+0000","last_became_active":"2024-09-16T07:34:48.531669+0000","last_became_peered":"2024-09-16T07:34:48.531669+0000","last_unstale":"2024-09-16T07:34:49.553731+0000","last_undegraded":"2024-09-16T07:34:49.553731+0000","last_fullsized":"2024-09-16T07:34:49.553731+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:03.265559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552860+0000","last_change":"2024-09-16T07:34:47.527047+0000","last_active":"2024-09-16T07:34:49.552860+0000","last_peered":"2024-09-16T07:34:49.552860+0000","last_clean":"2024-09-16T07:34:49.552860+0000","last_became_active":"2024-09-16T07:34:47.526733+0000","last_became_peered":"2024-09-16T07:34:47.526733+0000","last_unstale":"2024-09-16T07:34:49.552860+0000","last_undegraded":"2024-09-16T07:34:49.552860+0000","last_fullsized":"2024-09-16T07:34:49.552860+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:40.017911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553196+0000","last_change":"2024-09-16T07:34:48.530402+0000","last_active":"2024-09-16T07:34:49.553196+0000","last_peered":"2024-09-16T07:34:49.553196+0000","last_clean":"2024-09-16T07:34:49.553196+0000","last_became_active":"2024-09-16T07:34:48.530123+0000","last_became_peered":"2024-09-16T07:34:48.530123+0000","last_unstale":"2024-09-16T07:34:49.553196+0000","last_undegraded":"2024-09-16T07:34:49.553196+0000","last_fullsized":"2024-09-16T07:34:49.553196+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:35.523939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531255+0000","last_change":"2024-09-16T07:34:47.521688+0000","last_active":"2024-09-16T07:34:49.531255+0000","last_peered":"2024-09-16T07:34:49.531255+0000","last_clean":"2024-09-16T07:34:49.531255+0000","last_became_active":"2024-09-16T07:34:47.521350+0000","last_became_peered":"2024-09-16T07:34:47.521350+0000","last_unstale":"2024-09-16T07:34:49.531255+0000","last_undegraded":"2024-09-16T07:34:49.531255+0000","last_fullsized":"2024-09-16T07:34:49.531255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:10.584170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552631+0000","last_change":"2024-09-16T07:34:48.532035+0000","last_active":"2024-09-16T07:34:49.552631+0000","last_peered":"2024-09-16T07:34:49.552631+0000","last_clean":"2024-09-16T07:34:49.552631+0000","last_became_active":"2024-09-16T07:34:48.527917+0000","last_became_peered":"2024-09-16T07:34:48.527917+0000","last_unstale":"2024-09-16T07:34:49.552631+0000","last_undegraded":"2024-09-16T07:34:49.552631+0000","last_fullsized":"2024-09-16T07:34:49.552631+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:10.676540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552758+0000","last_change":"2024-09-16T07:34:47.529347+0000","last_active":"2024-09-16T07:34:49.552758+0000","last_peered":"2024-09-16T07:34:49.552758+0000","last_clean":"2024-09-16T07:34:49.552758+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:49.552758+0000","last_undegraded":"2024-09-16T07:34:49.552758+0000","last_fullsized":"2024-09-16T07:34:49.552758+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:10.392438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533585+0000","last_change":"2024-09-16T07:34:48.526788+0000","last_active":"2024-09-16T07:34:49.533585+0000","last_peered":"2024-09-16T07:34:49.533585+0000","last_clean":"2024-09-16T07:34:49.533585+0000","last_became_active":"2024-09-16T07:34:48.526279+0000","last_became_peered":"2024-09-16T07:34:48.526279+0000","last_unstale":"2024-09-16T07:34:49.533585+0000","last_undegraded":"2024-09-16T07:34:49.533585+0000","last_fullsized":"2024-09-16T07:34:49.533585+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:27:43.669315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"118'3","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552653+0000","last_change":"2024-09-16T07:34:47.529669+0000","last_active":"2024-09-16T07:34:49.552653+0000","last_peered":"2024-09-16T07:34:49.552653+0000","last_clean":"2024-09-16T07:34:49.552653+0000","last_became_active":"2024-09-16T07:34:47.529378+0000","last_became_peered":"2024-09-16T07:34:47.529378+0000","last_unstale":"2024-09-16T07:34:49.552653+0000","last_undegraded":"2024-09-16T07:34:49.552653+0000","last_fullsized":"2024-09-16T07:34:49.552653+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:58:21.671847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554073+0000","last_change":"2024-09-16T07:34:48.525029+0000","last_active":"2024-09-16T07:34:49.554073+0000","last_peered":"2024-09-16T07:34:49.554073+0000","last_clean":"2024-09-16T07:34:49.554073+0000","last_became_active":"2024-09-16T07:34:48.524129+0000","last_became_peered":"2024-09-16T07:34:48.524129+0000","last_unstale":"2024-09-16T07:34:49.554073+0000","last_undegraded":"2024-09-16T07:34:49.554073+0000","last_fullsized":"2024-09-16T07:34:49.554073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:43.451158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"118'6","reported_seq":21,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531542+0000","last_change":"2024-09-16T07:34:47.524021+0000","last_active":"2024-09-16T07:34:49.531542+0000","last_peered":"2024-09-16T07:34:49.531542+0000","last_clean":"2024-09-16T07:34:49.531542+0000","last_became_active":"2024-09-16T07:34:47.520956+0000","last_became_peered":"2024-09-16T07:34:47.520956+0000","last_unstale":"2024-09-16T07:34:49.531542+0000","last_undegraded":"2024-09-16T07:34:49.531542+0000","last_fullsized":"2024-09-16T07:34:49.531542+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:42.661372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552349+0000","last_change":"2024-09-16T07:34:48.534721+0000","last_active":"2024-09-16T07:34:49.552349+0000","last_peered":"2024-09-16T07:34:49.552349+0000","last_clean":"2024-09-16T07:34:49.552349+0000","last_became_active":"2024-09-16T07:34:48.534456+0000","last_became_peered":"2024-09-16T07:34:48.534456+0000","last_unstale":"2024-09-16T07:34:49.552349+0000","last_undegraded":"2024-09-16T07:34:49.552349+0000","last_fullsized":"2024-09-16T07:34:49.552349+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:56.381392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552719+0000","last_change":"2024-09-16T07:34:47.528845+0000","last_active":"2024-09-16T07:34:49.552719+0000","last_peered":"2024-09-16T07:34:49.552719+0000","last_clean":"2024-09-16T07:34:49.552719+0000","last_became_active":"2024-09-16T07:34:47.528568+0000","last_became_peered":"2024-09-16T07:34:47.528568+0000","last_unstale":"2024-09-16T07:34:49.552719+0000","last_undegraded":"2024-09-16T07:34:49.552719+0000","last_fullsized":"2024-09-16T07:34:49.552719+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:18.426091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531719+0000","last_change":"2024-09-16T07:34:48.525677+0000","last_active":"2024-09-16T07:34:49.531719+0000","last_peered":"2024-09-16T07:34:49.531719+0000","last_clean":"2024-09-16T07:34:49.531719+0000","last_became_active":"2024-09-16T07:34:48.522263+0000","last_became_peered":"2024-09-16T07:34:48.522263+0000","last_unstale":"2024-09-16T07:34:49.531719+0000","last_undegraded":"2024-09-16T07:34:49.531719+0000","last_fullsized":"2024-09-16T07:34:49.531719+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:28.526205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527593+0000","last_change":"2024-09-16T07:34:47.523404+0000","last_active":"2024-09-16T07:34:49.527593+0000","last_peered":"2024-09-16T07:34:49.527593+0000","last_clean":"2024-09-16T07:34:49.527593+0000","last_became_active":"2024-09-16T07:34:47.523140+0000","last_became_peered":"2024-09-16T07:34:47.523140+0000","last_unstale":"2024-09-16T07:34:49.527593+0000","last_undegraded":"2024-09-16T07:34:49.527593+0000","last_fullsized":"2024-09-16T07:34:49.527593+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:28.805823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531011+0000","last_change":"2024-09-16T07:34:48.530763+0000","last_active":"2024-09-16T07:34:49.531011+0000","last_peered":"2024-09-16T07:34:49.531011+0000","last_clean":"2024-09-16T07:34:49.531011+0000","last_became_active":"2024-09-16T07:34:48.530263+0000","last_became_peered":"2024-09-16T07:34:48.530263+0000","last_unstale":"2024-09-16T07:34:49.531011+0000","last_undegraded":"2024-09-16T07:34:49.531011+0000","last_fullsized":"2024-09-16T07:34:49.531011+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:16.945235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"118'8","reported_seq":23,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553052+0000","last_change":"2024-09-16T07:34:47.524692+0000","last_active":"2024-09-16T07:34:49.553052+0000","last_peered":"2024-09-16T07:34:49.553052+0000","last_clean":"2024-09-16T07:34:49.553052+0000","last_became_active":"2024-09-16T07:34:47.524434+0000","last_became_peered":"2024-09-16T07:34:47.524434+0000","last_unstale":"2024-09-16T07:34:49.553052+0000","last_undegraded":"2024-09-16T07:34:49.553052+0000","last_fullsized":"2024-09-16T07:34:49.553052+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:13.368017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554252+0000","last_change":"2024-09-16T07:34:48.525861+0000","last_active":"2024-09-16T07:34:49.554252+0000","last_peered":"2024-09-16T07:34:49.554252+0000","last_clean":"2024-09-16T07:34:49.554252+0000","last_became_active":"2024-09-16T07:34:48.525154+0000","last_became_peered":"2024-09-16T07:34:48.525154+0000","last_unstale":"2024-09-16T07:34:49.554252+0000","last_undegraded":"2024-09-16T07:34:49.554252+0000","last_fullsized":"2024-09-16T07:34:49.554252+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:17:29.690436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533331+0000","last_change":"2024-09-16T07:34:47.515933+0000","last_active":"2024-09-16T07:34:49.533331+0000","last_peered":"2024-09-16T07:34:49.533331+0000","last_clean":"2024-09-16T07:34:49.533331+0000","last_became_active":"2024-09-16T07:34:47.515670+0000","last_became_peered":"2024-09-16T07:34:47.515670+0000","last_unstale":"2024-09-16T07:34:49.533331+0000","last_undegraded":"2024-09-16T07:34:49.533331+0000","last_fullsized":"2024-09-16T07:34:49.533331+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:50:26.235372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554316+0000","last_change":"2024-09-16T07:34:48.530421+0000","last_active":"2024-09-16T07:34:49.554316+0000","last_peered":"2024-09-16T07:34:49.554316+0000","last_clean":"2024-09-16T07:34:49.554316+0000","last_became_active":"2024-09-16T07:34:48.529631+0000","last_became_peered":"2024-09-16T07:34:48.529631+0000","last_unstale":"2024-09-16T07:34:49.554316+0000","last_undegraded":"2024-09-16T07:34:49.554316+0000","last_fullsized":"2024-09-16T07:34:49.554316+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:20.813107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552967+0000","last_change":"2024-09-16T07:34:47.528922+0000","last_active":"2024-09-16T07:34:49.552967+0000","last_peered":"2024-09-16T07:34:49.552967+0000","last_clean":"2024-09-16T07:34:49.552967+0000","last_became_active":"2024-09-16T07:34:47.528617+0000","last_became_peered":"2024-09-16T07:34:47.528617+0000","last_unstale":"2024-09-16T07:34:49.552967+0000","last_undegraded":"2024-09-16T07:34:49.552967+0000","last_fullsized":"2024-09-16T07:34:49.552967+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:56:40.422966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554028+0000","last_change":"2024-09-16T07:34:48.528054+0000","last_active":"2024-09-16T07:34:49.554028+0000","last_peered":"2024-09-16T07:34:49.554028+0000","last_clean":"2024-09-16T07:34:49.554028+0000","last_became_active":"2024-09-16T07:34:48.527775+0000","last_became_peered":"2024-09-16T07:34:48.527775+0000","last_unstale":"2024-09-16T07:34:49.554028+0000","last_undegraded":"2024-09-16T07:34:49.554028+0000","last_fullsized":"2024-09-16T07:34:49.554028+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:19.344525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"118'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553383+0000","last_change":"2024-09-16T07:34:47.527129+0000","last_active":"2024-09-16T07:34:49.553383+0000","last_peered":"2024-09-16T07:34:49.553383+0000","last_clean":"2024-09-16T07:34:49.553383+0000","last_became_active":"2024-09-16T07:34:47.526046+0000","last_became_peered":"2024-09-16T07:34:47.526046+0000","last_unstale":"2024-09-16T07:34:49.553383+0000","last_undegraded":"2024-09-16T07:34:49.553383+0000","last_fullsized":"2024-09-16T07:34:49.553383+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:35:40.775080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553239+0000","last_change":"2024-09-16T07:34:48.522447+0000","last_active":"2024-09-16T07:34:49.553239+0000","last_peered":"2024-09-16T07:34:49.553239+0000","last_clean":"2024-09-16T07:34:49.553239+0000","last_became_active":"2024-09-16T07:34:48.521166+0000","last_became_peered":"2024-09-16T07:34:48.521166+0000","last_unstale":"2024-09-16T07:34:49.553239+0000","last_undegraded":"2024-09-16T07:34:49.553239+0000","last_fullsized":"2024-09-16T07:34:49.553239+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:11.672909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553105+0000","last_change":"2024-09-16T07:34:47.522843+0000","last_active":"2024-09-16T07:34:49.553105+0000","last_peered":"2024-09-16T07:34:49.553105+0000","last_clean":"2024-09-16T07:34:49.553105+0000","last_became_active":"2024-09-16T07:34:47.522587+0000","last_became_peered":"2024-09-16T07:34:47.522587+0000","last_unstale":"2024-09-16T07:34:49.553105+0000","last_undegraded":"2024-09-16T07:34:49.553105+0000","last_fullsized":"2024-09-16T07:34:49.553105+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:26.555192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527422+0000","last_change":"2024-09-16T07:34:48.528425+0000","last_active":"2024-09-16T07:34:49.527422+0000","last_peered":"2024-09-16T07:34:49.527422+0000","last_clean":"2024-09-16T07:34:49.527422+0000","last_became_active":"2024-09-16T07:34:48.526307+0000","last_became_peered":"2024-09-16T07:34:48.526307+0000","last_unstale":"2024-09-16T07:34:49.527422+0000","last_undegraded":"2024-09-16T07:34:49.527422+0000","last_fullsized":"2024-09-16T07:34:49.527422+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:31.730858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552950+0000","last_change":"2024-09-16T07:34:47.529098+0000","last_active":"2024-09-16T07:34:49.552950+0000","last_peered":"2024-09-16T07:34:49.552950+0000","last_clean":"2024-09-16T07:34:49.552950+0000","last_became_active":"2024-09-16T07:34:47.528650+0000","last_became_peered":"2024-09-16T07:34:47.528650+0000","last_unstale":"2024-09-16T07:34:49.552950+0000","last_undegraded":"2024-09-16T07:34:49.552950+0000","last_fullsized":"2024-09-16T07:34:49.552950+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:29.504051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553879+0000","last_change":"2024-09-16T07:34:48.525987+0000","last_active":"2024-09-16T07:34:49.553879+0000","last_peered":"2024-09-16T07:34:49.553879+0000","last_clean":"2024-09-16T07:34:49.553879+0000","last_became_active":"2024-09-16T07:34:48.523672+0000","last_became_peered":"2024-09-16T07:34:48.523672+0000","last_unstale":"2024-09-16T07:34:49.553879+0000","last_undegraded":"2024-09-16T07:34:49.553879+0000","last_fullsized":"2024-09-16T07:34:49.553879+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:19.407333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553766+0000","last_change":"2024-09-16T07:34:47.521880+0000","last_active":"2024-09-16T07:34:49.553766+0000","last_peered":"2024-09-16T07:34:49.553766+0000","last_clean":"2024-09-16T07:34:49.553766+0000","last_became_active":"2024-09-16T07:34:47.521610+0000","last_became_peered":"2024-09-16T07:34:47.521610+0000","last_unstale":"2024-09-16T07:34:49.553766+0000","last_undegraded":"2024-09-16T07:34:49.553766+0000","last_fullsized":"2024-09-16T07:34:49.553766+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:54:26.850724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552033+0000","last_change":"2024-09-16T07:34:48.530713+0000","last_active":"2024-09-16T07:34:49.552033+0000","last_peered":"2024-09-16T07:34:49.552033+0000","last_clean":"2024-09-16T07:34:49.552033+0000","last_became_active":"2024-09-16T07:34:48.526504+0000","last_became_peered":"2024-09-16T07:34:48.526504+0000","last_unstale":"2024-09-16T07:34:49.552033+0000","last_undegraded":"2024-09-16T07:34:49.552033+0000","last_fullsized":"2024-09-16T07:34:49.552033+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:59:38.996763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530916+0000","last_change":"2024-09-16T07:34:47.518969+0000","last_active":"2024-09-16T07:34:49.530916+0000","last_peered":"2024-09-16T07:34:49.530916+0000","last_clean":"2024-09-16T07:34:49.530916+0000","last_became_active":"2024-09-16T07:34:47.518693+0000","last_became_peered":"2024-09-16T07:34:47.518693+0000","last_unstale":"2024-09-16T07:34:49.530916+0000","last_undegraded":"2024-09-16T07:34:49.530916+0000","last_fullsized":"2024-09-16T07:34:49.530916+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:33.694870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552317+0000","last_change":"2024-09-16T07:34:48.541705+0000","last_active":"2024-09-16T07:34:49.552317+0000","last_peered":"2024-09-16T07:34:49.552317+0000","last_clean":"2024-09-16T07:34:49.552317+0000","last_became_active":"2024-09-16T07:34:48.541457+0000","last_became_peered":"2024-09-16T07:34:48.541457+0000","last_unstale":"2024-09-16T07:34:49.552317+0000","last_undegraded":"2024-09-16T07:34:49.552317+0000","last_fullsized":"2024-09-16T07:34:49.552317+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:13.367118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.530779+0000","last_change":"2024-09-16T07:34:47.517277+0000","last_active":"2024-09-16T07:34:49.530779+0000","last_peered":"2024-09-16T07:34:49.530779+0000","last_clean":"2024-09-16T07:34:49.530779+0000","last_became_active":"2024-09-16T07:34:47.516911+0000","last_became_peered":"2024-09-16T07:34:47.516911+0000","last_unstale":"2024-09-16T07:34:49.530779+0000","last_undegraded":"2024-09-16T07:34:49.530779+0000","last_fullsized":"2024-09-16T07:34:49.530779+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:31:40.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553315+0000","last_change":"2024-09-16T07:34:48.526046+0000","last_active":"2024-09-16T07:34:49.553315+0000","last_peered":"2024-09-16T07:34:49.553315+0000","last_clean":"2024-09-16T07:34:49.553315+0000","last_became_active":"2024-09-16T07:34:48.525684+0000","last_became_peered":"2024-09-16T07:34:48.525684+0000","last_unstale":"2024-09-16T07:34:49.553315+0000","last_undegraded":"2024-09-16T07:34:49.553315+0000","last_fullsized":"2024-09-16T07:34:49.553315+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:38:46.581976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"118'2","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531230+0000","last_change":"2024-09-16T07:34:47.521736+0000","last_active":"2024-09-16T07:34:49.531230+0000","last_peered":"2024-09-16T07:34:49.531230+0000","last_clean":"2024-09-16T07:34:49.531230+0000","last_became_active":"2024-09-16T07:34:47.521422+0000","last_became_peered":"2024-09-16T07:34:47.521422+0000","last_unstale":"2024-09-16T07:34:49.531230+0000","last_undegraded":"2024-09-16T07:34:49.531230+0000","last_fullsized":"2024-09-16T07:34:49.531230+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:02.620783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533267+0000","last_change":"2024-09-16T07:34:48.530060+0000","last_active":"2024-09-16T07:34:49.533267+0000","last_peered":"2024-09-16T07:34:49.533267+0000","last_clean":"2024-09-16T07:34:49.533267+0000","last_became_active":"2024-09-16T07:34:48.529780+0000","last_became_peered":"2024-09-16T07:34:48.529780+0000","last_unstale":"2024-09-16T07:34:49.533267+0000","last_undegraded":"2024-09-16T07:34:49.533267+0000","last_fullsized":"2024-09-16T07:34:49.533267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:56:07.703091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"119'1","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554239+0000","last_change":"2024-09-16T07:34:47.511494+0000","last_active":"2024-09-16T07:34:49.554239+0000","last_peered":"2024-09-16T07:34:49.554239+0000","last_clean":"2024-09-16T07:34:49.554239+0000","last_became_active":"2024-09-16T07:34:47.511196+0000","last_became_peered":"2024-09-16T07:34:47.511196+0000","last_unstale":"2024-09-16T07:34:49.554239+0000","last_undegraded":"2024-09-16T07:34:49.554239+0000","last_fullsized":"2024-09-16T07:34:49.554239+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:18.504885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553999+0000","last_change":"2024-09-16T07:34:48.530843+0000","last_active":"2024-09-16T07:34:49.553999+0000","last_peered":"2024-09-16T07:34:49.553999+0000","last_clean":"2024-09-16T07:34:49.553999+0000","last_became_active":"2024-09-16T07:34:48.530078+0000","last_became_peered":"2024-09-16T07:34:48.530078+0000","last_unstale":"2024-09-16T07:34:49.553999+0000","last_undegraded":"2024-09-16T07:34:49.553999+0000","last_fullsized":"2024-09-16T07:34:49.553999+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:11.338665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531451+0000","last_change":"2024-09-16T07:34:47.516251+0000","last_active":"2024-09-16T07:34:49.531451+0000","last_peered":"2024-09-16T07:34:49.531451+0000","last_clean":"2024-09-16T07:34:49.531451+0000","last_became_active":"2024-09-16T07:34:47.515980+0000","last_became_peered":"2024-09-16T07:34:47.515980+0000","last_unstale":"2024-09-16T07:34:49.531451+0000","last_undegraded":"2024-09-16T07:34:49.531451+0000","last_fullsized":"2024-09-16T07:34:49.531451+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:16.654899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553282+0000","last_change":"2024-09-16T07:34:48.542025+0000","last_active":"2024-09-16T07:34:49.553282+0000","last_peered":"2024-09-16T07:34:49.553282+0000","last_clean":"2024-09-16T07:34:49.553282+0000","last_became_active":"2024-09-16T07:34:48.541713+0000","last_became_peered":"2024-09-16T07:34:48.541713+0000","last_unstale":"2024-09-16T07:34:49.553282+0000","last_undegraded":"2024-09-16T07:34:49.553282+0000","last_fullsized":"2024-09-16T07:34:49.553282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:49:31.500851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531609+0000","last_change":"2024-09-16T07:34:47.513397+0000","last_active":"2024-09-16T07:34:49.531609+0000","last_peered":"2024-09-16T07:34:49.531609+0000","last_clean":"2024-09-16T07:34:49.531609+0000","last_became_active":"2024-09-16T07:34:47.513084+0000","last_became_peered":"2024-09-16T07:34:47.513084+0000","last_unstale":"2024-09-16T07:34:49.531609+0000","last_undegraded":"2024-09-16T07:34:49.531609+0000","last_fullsized":"2024-09-16T07:34:49.531609+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:49.301770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527582+0000","last_change":"2024-09-16T07:34:48.526515+0000","last_active":"2024-09-16T07:34:49.527582+0000","last_peered":"2024-09-16T07:34:49.527582+0000","last_clean":"2024-09-16T07:34:49.527582+0000","last_became_active":"2024-09-16T07:34:48.525964+0000","last_became_peered":"2024-09-16T07:34:48.525964+0000","last_unstale":"2024-09-16T07:34:49.527582+0000","last_undegraded":"2024-09-16T07:34:49.527582+0000","last_fullsized":"2024-09-16T07:34:49.527582+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:41.469054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.554153+0000","last_change":"2024-09-16T07:34:47.518568+0000","last_active":"2024-09-16T07:34:49.554153+0000","last_peered":"2024-09-16T07:34:49.554153+0000","last_clean":"2024-09-16T07:34:49.554153+0000","last_became_active":"2024-09-16T07:34:47.518279+0000","last_became_peered":"2024-09-16T07:34:47.518279+0000","last_unstale":"2024-09-16T07:34:49.554153+0000","last_undegraded":"2024-09-16T07:34:49.554153+0000","last_fullsized":"2024-09-16T07:34:49.554153+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:36.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552985+0000","last_change":"2024-09-16T07:34:48.528073+0000","last_active":"2024-09-16T07:34:49.552985+0000","last_peered":"2024-09-16T07:34:49.552985+0000","last_clean":"2024-09-16T07:34:49.552985+0000","last_became_active":"2024-09-16T07:34:48.527807+0000","last_became_peered":"2024-09-16T07:34:48.527807+0000","last_unstale":"2024-09-16T07:34:49.552985+0000","last_undegraded":"2024-09-16T07:34:49.552985+0000","last_fullsized":"2024-09-16T07:34:49.552985+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:26:54.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552711+0000","last_change":"2024-09-16T07:34:47.527175+0000","last_active":"2024-09-16T07:34:49.552711+0000","last_peered":"2024-09-16T07:34:49.552711+0000","last_clean":"2024-09-16T07:34:49.552711+0000","last_became_active":"2024-09-16T07:34:47.526929+0000","last_became_peered":"2024-09-16T07:34:47.526929+0000","last_unstale":"2024-09-16T07:34:49.552711+0000","last_undegraded":"2024-09-16T07:34:49.552711+0000","last_fullsized":"2024-09-16T07:34:49.552711+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:03:56.405916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552082+0000","last_change":"2024-09-16T07:34:48.524675+0000","last_active":"2024-09-16T07:34:49.552082+0000","last_peered":"2024-09-16T07:34:49.552082+0000","last_clean":"2024-09-16T07:34:49.552082+0000","last_became_active":"2024-09-16T07:34:48.524324+0000","last_became_peered":"2024-09-16T07:34:48.524324+0000","last_unstale":"2024-09-16T07:34:49.552082+0000","last_undegraded":"2024-09-16T07:34:49.552082+0000","last_fullsized":"2024-09-16T07:34:49.552082+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:55.768025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.553842+0000","last_change":"2024-09-16T07:34:47.525571+0000","last_active":"2024-09-16T07:34:49.553842+0000","last_peered":"2024-09-16T07:34:49.553842+0000","last_clean":"2024-09-16T07:34:49.553842+0000","last_became_active":"2024-09-16T07:34:47.525295+0000","last_became_peered":"2024-09-16T07:34:47.525295+0000","last_unstale":"2024-09-16T07:34:49.553842+0000","last_undegraded":"2024-09-16T07:34:49.553842+0000","last_fullsized":"2024-09-16T07:34:49.553842+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:02:05.779838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.533513+0000","last_change":"2024-09-16T07:34:48.528449+0000","last_active":"2024-09-16T07:34:49.533513+0000","last_peered":"2024-09-16T07:34:49.533513+0000","last_clean":"2024-09-16T07:34:49.533513+0000","last_became_active":"2024-09-16T07:34:48.528185+0000","last_became_peered":"2024-09-16T07:34:48.528185+0000","last_unstale":"2024-09-16T07:34:49.533513+0000","last_undegraded":"2024-09-16T07:34:49.533513+0000","last_fullsized":"2024-09-16T07:34:49.533513+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:28:45.460596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.532882+0000","last_change":"2024-09-16T07:34:47.518753+0000","last_active":"2024-09-16T07:34:49.532882+0000","last_peered":"2024-09-16T07:34:49.532882+0000","last_clean":"2024-09-16T07:34:49.532882+0000","last_became_active":"2024-09-16T07:34:47.518486+0000","last_became_peered":"2024-09-16T07:34:47.518486+0000","last_unstale":"2024-09-16T07:34:49.532882+0000","last_undegraded":"2024-09-16T07:34:49.532882+0000","last_fullsized":"2024-09-16T07:34:49.532882+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:36.903769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.531155+0000","last_change":"2024-09-16T07:34:48.533301+0000","last_active":"2024-09-16T07:34:49.531155+0000","last_peered":"2024-09-16T07:34:49.531155+0000","last_clean":"2024-09-16T07:34:49.531155+0000","last_became_active":"2024-09-16T07:34:48.532697+0000","last_became_peered":"2024-09-16T07:34:48.532697+0000","last_unstale":"2024-09-16T07:34:49.531155+0000","last_undegraded":"2024-09-16T07:34:49.531155+0000","last_fullsized":"2024-09-16T07:34:49.531155+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:47.492910+0000","last_clean_scrub_stamp":"2024-09-16T07:34:47.492910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:14:10.393373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.552313+0000","last_change":"2024-09-16T07:34:47.521842+0000","last_active":"2024-09-16T07:34:49.552313+0000","last_peered":"2024-09-16T07:34:49.552313+0000","last_clean":"2024-09-16T07:34:49.552313+0000","last_became_active":"2024-09-16T07:34:47.521578+0000","last_became_peered":"2024-09-16T07:34:47.521578+0000","last_unstale":"2024-09-16T07:34:49.552313+0000","last_undegraded":"2024-09-16T07:34:49.552313+0000","last_fullsized":"2024-09-16T07:34:49.552313+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:34:46.489756+0000","last_clean_scrub_stamp":"2024-09-16T07:34:46.489756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:20.091370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":290,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-16T07:34:49.527353+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:34:49.527353+0000","last_peered":"2024-09-16T07:34:49.527353+0000","last_clean":"2024-09-16T07:34:49.527353+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:34:49.527353+0000","last_undegraded":"2024-09-16T07:34:49.527353+0000","last_fullsized":"2024-09-16T07:34:49.527353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126960,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244880,"statfs":{"total":96636764160,"available":96506757120,"internally_reserved":0,"allocated":1466368,"data_stored":641671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126976,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244864,"statfs":{"total":96636764160,"available":96506740736,"internally_reserved":0,"allocated":1478656,"data_stored":642309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123384,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248456,"statfs":{"total":96636764160,"available":96510418944,"internally_reserved":0,"allocated":2060288,"data_stored":1232039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140860,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230980,"statfs":{"total":96636764160,"available":96492523520,"internally_reserved":0,"allocated":1474560,"data_stored":642323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122820,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249020,"statfs":{"total":96636764160,"available":96510996480,"internally_reserved":0,"allocated":1486848,"data_stored":643407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153328,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218512,"statfs":{"total":96636764160,"available":96479756288,"internally_reserved":0,"allocated":1466368,"data_stored":641671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131712,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240128,"statfs":{"total":96636764160,"available":96501891072,"internally_reserved":0,"allocated":2068480,"data_stored":1232131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126976,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244864,"statfs":{"total":96636764160,"available":96506740736,"internally_reserved":0,"allocated":1482752,"data_stored":643373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:34:57.801 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:34:57.801 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:34:58.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.002+0000 7f128fe00640 1 -- 172.21.15.71:0/1990047915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 msgr2=0x7f1274004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.002+0000 7f128fe00640 1 --2- 172.21.15.71:0/1990047915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 0x7f1274004c80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1290075440 tx=0x7f1274005830 comp rx=0 tx=0).stop 2024-09-16T07:34:58.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.002+0000 7f128f400640 1 -- 172.21.15.71:0/1990047915 reap_dead start 2024-09-16T07:34:58.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.002+0000 7f128fe00640 1 -- 172.21.15.71:0/1990047915 shutdown_connections 2024-09-16T07:34:58.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.002+0000 7f128fe00640 1 -- 172.21.15.71:0/1990047915 >> 172.21.15.71:0/1990047915 conn(0x7f12900cc490 msgr2=0x7f12900cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:58.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.002+0000 7f128fe00640 1 -- 172.21.15.71:0/1990047915 shutdown_connections 2024-09-16T07:34:58.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.002+0000 7f128fe00640 1 -- 172.21.15.71:0/1990047915 wait complete. 2024-09-16T07:34:58.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.004+0000 7f128fe00640 1 Processor -- start 2024-09-16T07:34:58.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.004+0000 7f128fe00640 1 -- start start 2024-09-16T07:34:58.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.004+0000 7f128fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 0x7f12900b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:58.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1290147420 0x7f1290072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:58.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1290072db0 0x7f12900b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:58.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12900ca4f0 con 0x7f1274004800 2024-09-16T07:34:58.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f12900ca370 con 0x7f1290072db0 2024-09-16T07:34:58.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f12900c6430 con 0x7f1290147420 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 0x7f12900b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 0x7f12900b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33620/0 (socket says 172.21.15.71:33620) 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 -- 172.21.15.71:0/899053880 learned_addr learned my addr 172.21.15.71:0/899053880 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128e000640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1290147420 0x7f1290072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 -- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1290147420 msgr2=0x7f1290072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128f400640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1290072db0 0x7f12900b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1290147420 0x7f1290072870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 -- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1290072db0 msgr2=0x7f12900b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1290072db0 0x7f12900b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 -- 172.21.15.71:0/899053880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12900af630 con 0x7f1274004800 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128f400640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1290072db0 0x7f12900b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128e000640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1290147420 0x7f1290072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.005+0000 7f128ea00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 0x7f12900b00d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f128000f9b0 tx=0x7f128000fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.006+0000 7f1287400640 1 -- 172.21.15.71:0/899053880 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128000bc50 con 0x7f1274004800 2024-09-16T07:34:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.006+0000 7f1287400640 1 -- 172.21.15.71:0/899053880 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1280004030 con 0x7f1274004800 2024-09-16T07:34:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.006+0000 7f1287400640 1 -- 172.21.15.71:0/899053880 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280004420 con 0x7f1274004800 2024-09-16T07:34:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.006+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12900b1d40 con 0x7f1274004800 2024-09-16T07:34:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.006+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12900b2020 con 0x7f1274004800 2024-09-16T07:34:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.007+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129015fa50 con 0x7f1274004800 2024-09-16T07:34:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.008+0000 7f1287400640 1 -- 172.21.15.71:0/899053880 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f12800035f0 con 0x7f1274004800 2024-09-16T07:34:58.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.009+0000 7f1287400640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f127007a230 0x7f127007c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:58.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.009+0000 7f1287400640 1 -- 172.21.15.71:0/899053880 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f128009a030 con 0x7f1274004800 2024-09-16T07:34:58.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.009+0000 7f128e000640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f127007a230 0x7f127007c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:58.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.010+0000 7f128e000640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f127007a230 0x7f127007c6f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f127c033000 tx=0x7f12901566d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:58.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.013+0000 7f1287400640 1 -- 172.21.15.71:0/899053880 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1280063b50 con 0x7f1274004800 2024-09-16T07:34:58.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.267+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f12900ce690 con 0x7f1274004800 2024-09-16T07:34:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.270+0000 7f1287400640 1 -- 172.21.15.71:0/899053880 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f1280068580 con 0x7f1274004800 2024-09-16T07:34:58.270 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:34:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:34:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:34:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:34:58.270 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:34:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f127007a230 msgr2=0x7f127007c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f127007a230 0x7f127007c6f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f127c033000 tx=0x7f12901566d0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 msgr2=0x7f12900b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 0x7f12900b00d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f128000f9b0 tx=0x7f128000fe70 comp rx=0 tx=0).stop 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 shutdown_connections 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f127007a230 0x7f127007c6f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1290072db0 0x7f12900b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1290147420 0x7f1290072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 --2- 172.21.15.71:0/899053880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1274004800 0x7f12900b00d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 >> 172.21.15.71:0/899053880 conn(0x7f12900cc490 msgr2=0x7f12900739b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:58.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 shutdown_connections 2024-09-16T07:34:58.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.273+0000 7f128fe00640 1 -- 172.21.15.71:0/899053880 wait complete. 2024-09-16T07:34:58.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:34:58.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.448+0000 7f8f94000640 1 -- 172.21.15.71:0/1316602156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f8c073fb0 msgr2=0x7f8f740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.448+0000 7f8f94000640 1 --2- 172.21.15.71:0/1316602156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f8c073fb0 0x7f8f740051c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005fd0 tx=0x7f8f74005830 comp rx=0 tx=0).stop 2024-09-16T07:34:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.448+0000 7f8f92000640 1 -- 172.21.15.71:0/1316602156 reap_dead start 2024-09-16T07:34:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.448+0000 7f8f94000640 1 -- 172.21.15.71:0/1316602156 shutdown_connections 2024-09-16T07:34:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.448+0000 7f8f94000640 1 -- 172.21.15.71:0/1316602156 >> 172.21.15.71:0/1316602156 conn(0x7f8f8c0b1950 msgr2=0x7f8f8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.448+0000 7f8f94000640 1 -- 172.21.15.71:0/1316602156 shutdown_connections 2024-09-16T07:34:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.448+0000 7f8f94000640 1 -- 172.21.15.71:0/1316602156 wait complete. 2024-09-16T07:34:58.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.449+0000 7f8f94000640 1 Processor -- start 2024-09-16T07:34:58.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.450+0000 7f8f94000640 1 -- start start 2024-09-16T07:34:58.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.450+0000 7f8f94000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f8c073fb0 0x7f8f8c14a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:58.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f94000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f8c15b840 0x7f8f8c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f94000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c0c7b50 0x7f8f8c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f94000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c0745c0 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f94000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c074440 con 0x7f8f8c15b840 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f94000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8f8c074740 con 0x7f8f8c073fb0 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f92000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c0c7b50 0x7f8f8c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f92000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c0c7b50 0x7f8f8c0c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33636/0 (socket says 172.21.15.71:33636) 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f92000640 1 -- 172.21.15.71:0/1521665874 learned_addr learned my addr 172.21.15.71:0/1521665874 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f91600640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f8c073fb0 0x7f8f8c14a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f90c00640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f8c15b840 0x7f8f8c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f92000640 1 -- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f8c073fb0 msgr2=0x7f8f8c14a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f92000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f8c073fb0 0x7f8f8c14a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f92000640 1 -- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f8c15b840 msgr2=0x7f8f8c15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f92000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f8c15b840 0x7f8f8c15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f92000640 1 -- 172.21.15.71:0/1521665874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c159b60 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f91600640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f8c073fb0 0x7f8f8c14a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:58.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.451+0000 7f8f90c00640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f8c15b840 0x7f8f8c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:58.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.452+0000 7f8f92000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c0c7b50 0x7f8f8c0c7fd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8f8402aec0 tx=0x7f8f84033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:58.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.452+0000 7f8f89600640 1 -- 172.21.15.71:0/1521665874 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84042070 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.452+0000 7f8f89600640 1 -- 172.21.15.71:0/1521665874 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f84003970 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.452+0000 7f8f89600640 1 -- 172.21.15.71:0/1521665874 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84038040 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.452+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c0c8510 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.452+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c15c420 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.453+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c15f100 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.454+0000 7f8f89600640 1 -- 172.21.15.71:0/1521665874 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f8f84004030 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.456+0000 7f8f89600640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f6c07a230 0x7f8f6c07c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:58.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.456+0000 7f8f89600640 1 -- 172.21.15.71:0/1521665874 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f8f840b9880 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.456+0000 7f8f91600640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f6c07a230 0x7f8f6c07c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:58.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.457+0000 7f8f91600640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f6c07a230 0x7f8f6c07c6f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009cd0 tx=0x7f8f80005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:58.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.460+0000 7f8f89600640 1 -- 172.21.15.71:0/1521665874 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8f8c15f100 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.682+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f8c0b2440 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.683+0000 7f8f89600640 1 -- 172.21.15.71:0/1521665874 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5231 (secure 0 0 0) 0x7f8f8c159b60 con 0x7f8f8c0c7b50 2024-09-16T07:34:58.682 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:34:58.683 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":75,"btime":"2024-09-16T07:34:51:534556+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1994777108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1994777108},{"type":"v1","addr":"172.21.15.71:6833","nonce":1994777108}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1399666780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1399666780},{"type":"v1","addr":"172.21.15.177:6836","nonce":1399666780}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2921631965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2921631965},{"type":"v1","addr":"172.21.15.71:6835","nonce":2921631965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:34:47.926434+0000","modified":"2024-09-16T07:34:50.958041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2539335973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2539335973},{"type":"v1","addr":"172.21.15.177:6837","nonce":2539335973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8382,"qdb_cluster":[8382]},"id":8}]} 2024-09-16T07:34:58.683 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 75 2024-09-16T07:34:58.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f6c07a230 msgr2=0x7f8f6c07c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f6c07a230 0x7f8f6c07c6f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009cd0 tx=0x7f8f80005f90 comp rx=0 tx=0).stop 2024-09-16T07:34:58.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c0c7b50 msgr2=0x7f8f8c0c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:58.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c0c7b50 0x7f8f8c0c7fd0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f8f8402aec0 tx=0x7f8f84033c10 comp rx=0 tx=0).stop 2024-09-16T07:34:58.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 shutdown_connections 2024-09-16T07:34:58.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f6c07a230 0x7f8f6c07c6f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f8c0c7b50 0x7f8f8c0c7fd0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f8c15b840 0x7f8f8c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.685+0000 7f8f94000640 1 --2- 172.21.15.71:0/1521665874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f8c073fb0 0x7f8f8c14a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:58.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.686+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 >> 172.21.15.71:0/1521665874 conn(0x7f8f8c0b1950 msgr2=0x7f8f8c0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:58.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.686+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 shutdown_connections 2024-09-16T07:34:58.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:58.686+0000 7f8f94000640 1 -- 172.21.15.71:0/1521665874 wait complete. 2024-09-16T07:34:59.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:34:47.926434+0000', 'modified': '2024-09-16T07:34:50.958041+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8382}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8382': {'gid': 8382, 'name': 'd', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6837/2539335973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6835', 'nonce': 2539335973}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 2539335973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8382, 'qdb_cluster': [8382]} 2024-09-16T07:34:59.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:34:59.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-16T07:34:59.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.265+0000 7f6238800640 1 -- 172.21.15.71:0/1398922008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62300ccf00 msgr2=0x7f62300b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.265+0000 7f6238800640 1 --2- 172.21.15.71:0/1398922008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62300ccf00 0x7f62300b2440 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f623006dac0 tx=0x7f622002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.265+0000 7f6238800640 1 -- 172.21.15.71:0/1398922008 shutdown_connections 2024-09-16T07:34:59.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.265+0000 7f6238800640 1 --2- 172.21.15.71:0/1398922008 >> v2:172.21.15.177:3300/0 conn(0x7f6230073fb0 0x7f6230074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.265+0000 7f6238800640 1 --2- 172.21.15.71:0/1398922008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62300ccf00 0x7f62300b2440 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.265+0000 7f6238800640 1 -- 172.21.15.71:0/1398922008 >> 172.21.15.71:0/1398922008 conn(0x7f62300b1950 msgr2=0x7f62300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:59.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.265+0000 7f6238800640 1 -- 172.21.15.71:0/1398922008 shutdown_connections 2024-09-16T07:34:59.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.265+0000 7f6238800640 1 -- 172.21.15.71:0/1398922008 wait complete. 2024-09-16T07:34:59.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.267+0000 7f6238800640 1 Processor -- start 2024-09-16T07:34:59.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.267+0000 7f6238800640 1 -- start start 2024-09-16T07:34:59.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6238800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6230073fb0 0x7f62300b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6238800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300ccf00 0x7f62300c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6238800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f623015f5a0 0x7f62301f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6238800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f623006f2e0 con 0x7f6230073fb0 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6238800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f623006f160 con 0x7f62300ccf00 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6238800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f623006f460 con 0x7f623015f5a0 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6230073fb0 0x7f62300b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6230073fb0 0x7f62300b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33648/0 (socket says 172.21.15.71:33648) 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235e00640 1 -- 172.21.15.71:0/1771582803 learned_addr learned my addr 172.21.15.71:0/1771582803 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235400640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300ccf00 0x7f62300c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6236800640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f623015f5a0 0x7f62301f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235e00640 1 -- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f623015f5a0 msgr2=0x7f62301f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235e00640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f623015f5a0 0x7f62301f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235e00640 1 -- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300ccf00 msgr2=0x7f62300c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235e00640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300ccf00 0x7f62300c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.268+0000 7f6235e00640 1 -- 172.21.15.71:0/1771582803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62300c9510 con 0x7f6230073fb0 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.269+0000 7f6235400640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300ccf00 0x7f62300c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.269+0000 7f6235e00640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6230073fb0 0x7f62300b3e00 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f622400b3f0 tx=0x7f622400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.269+0000 7f622a000640 1 -- 172.21.15.71:0/1771582803 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224015020 con 0x7f6230073fb0 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.269+0000 7f622a000640 1 -- 172.21.15.71:0/1771582803 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6224011920 con 0x7f6230073fb0 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.269+0000 7f6238800640 1 -- 172.21.15.71:0/1771582803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62300c9a90 con 0x7f6230073fb0 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.269+0000 7f622a000640 1 -- 172.21.15.71:0/1771582803 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224011d70 con 0x7f6230073fb0 2024-09-16T07:34:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.269+0000 7f6238800640 1 -- 172.21.15.71:0/1771582803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62300ca050 con 0x7f6230073fb0 2024-09-16T07:34:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.271+0000 7f622a000640 1 -- 172.21.15.71:0/1771582803 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f6224018020 con 0x7f6230073fb0 2024-09-16T07:34:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.272+0000 7f622a000640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f621407a260 0x7f621407c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.273+0000 7f623b843740 1 Processor -- start 2024-09-16T07:34:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.273+0000 7f622a000640 1 -- 172.21.15.71:0/1771582803 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f6224099480 con 0x7f6230073fb0 2024-09-16T07:34:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.273+0000 7f623b843740 1 -- start start 2024-09-16T07:34:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.273+0000 7f6235400640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f621407a260 0x7f621407c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.273+0000 7f623b843740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797c68b0 0x55f4797c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.273+0000 7f623b843740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f479781a20 0x55f4797c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f623b843740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f623b843740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f479704430 con 0x55f479781fa0 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f623b843740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4796f45d0 con 0x55f4797c68b0 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f623b843740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f4796e3a00 con 0x55f479781a20 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6235400640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f621407a260 0x7f621407c720 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6220033000 tx=0x7f6220004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6236800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6236800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33664/0 (socket says 172.21.15.71:33664) 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6236800640 1 -- 172.21.15.71:0/1118297594 learned_addr learned my addr 172.21.15.71:0/1118297594 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6235400640 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f479781a20 0x55f4797c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6235e00640 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797c68b0 0x55f4797c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6236800640 1 -- 172.21.15.71:0/1118297594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f479781a20 msgr2=0x55f4797c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6236800640 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f479781a20 0x55f4797c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6236800640 1 -- 172.21.15.71:0/1118297594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797c68b0 msgr2=0x55f4797c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6236800640 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797c68b0 0x55f4797c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.274+0000 7f6236800640 1 -- 172.21.15.71:0/1118297594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4796ac520 con 0x55f479781fa0 2024-09-16T07:34:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f6236800640 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797c58d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f622c009ec0 tx=0x7f622c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e685ab45094b8f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:59.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f621e600640 1 -- 172.21.15.71:0/1118297594 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c016070 con 0x55f479781fa0 2024-09-16T07:34:59.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f621e600640 1 -- 172.21.15.71:0/1118297594 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f622c01b750 con 0x55f479781fa0 2024-09-16T07:34:59.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f621e600640 1 -- 172.21.15.71:0/1118297594 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c01bb00 con 0x55f479781fa0 2024-09-16T07:34:59.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f623b843740 1 -- 172.21.15.71:0/1118297594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 msgr2=0x55f4797c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f623b843740 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797c58d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f622c009ec0 tx=0x7f622c010740 comp rx=0 tx=0).stop 2024-09-16T07:34:59.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f623b843740 1 -- 172.21.15.71:0/1118297594 shutdown_connections 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f623b843740 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797c58d0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f623b843740 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f479781a20 0x55f4797c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f623b843740 1 --2- 172.21.15.71:0/1118297594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797c68b0 0x55f4797c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.275+0000 7f623b843740 1 -- 172.21.15.71:0/1118297594 >> 172.21.15.71:0/1118297594 conn(0x55f47977cff0 msgr2=0x55f479780a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.276+0000 7f623b843740 1 -- 172.21.15.71:0/1118297594 shutdown_connections 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.276+0000 7f623b843740 1 -- 172.21.15.71:0/1118297594 wait complete. 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.276+0000 7f623b843740 1 -- start start 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.276+0000 7f623b843740 1 Processor -- start 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.277+0000 7f623b843740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 0x55f4797a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.277+0000 7f623b843740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 0x55f4797aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.277+0000 7f623b843740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797d8af0 0x55f4797d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.277+0000 7f623b843740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f47968a330 con 0x55f4797d8af0 2024-09-16T07:34:59.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.277+0000 7f623b843740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4796e7810 con 0x55f479781fa0 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.277+0000 7f623b843740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f479454620 con 0x55f4797c68b0 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.277+0000 7f623b843740 10 client.0 _ll_register_callbacks cb 0x55f47968f420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797d8af0 0x55f4797d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797d8af0 0x55f4797d8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33680/0 (socket says 172.21.15.71:33680) 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 -- 172.21.15.71:0/3200224427 learned_addr learned my addr 172.21.15.71:0/3200224427 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 -- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 msgr2=0x55f4797aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6235400640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 0x55f4797aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6235e00640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 0x55f4797a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 0x55f4797aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 -- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 msgr2=0x55f4797a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 0x55f4797a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 -- 172.21.15.71:0/3200224427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f479445850 con 0x55f4797d8af0 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6235400640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 0x55f4797aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:59.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6235e00640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 0x55f4797a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6236800640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797d8af0 0x55f4797d8f70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f622c010c40 tx=0x7f622c01b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c883d8edee21dcf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.278+0000 7f6212a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f6212a00640 1 -- 172.21.15.71:0/3200224427 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c016070 con 0x55f4797d8af0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 1 -- 172.21.15.71:0/3200224427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f622c01b750 con 0x55f4797d8af0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f6212a00640 1 -- 172.21.15.71:0/3200224427 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f622c012450 con 0x55f4797d8af0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f6212a00640 1 -- 172.21.15.71:0/3200224427 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f622c02b050 con 0x55f4797d8af0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 10 client.8523 fetch_fsmap learned FSMap version 75 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 1 -- 172.21.15.71:0/3200224427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55f4794bc6b0 con 0x55f4797d8af0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f6212a00640 1 -- 172.21.15.71:0/3200224427 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f622c01b750 con 0x55f4797d8af0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f6212a00640 1 -- 172.21.15.71:0/3200224427 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f622c02e020 con 0x55f4797d8af0 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 10 client.8523 fetch_fsmap finished waiting for FSMap version 75 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 10 client.8523 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:34:59.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 20 client.8523 populate_metadata read hostname 'smithi071' 2024-09-16T07:34:59.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] conn(0x55f4797a82b0 0x55f4797a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 4 client.8523 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:34:59.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.279+0000 7f623b843740 1 -- 172.21.15.71:0/3200224427 --> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4796f6520 con 0x55f4797a82b0 2024-09-16T07:34:59.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.280+0000 7f6235e00640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] conn(0x55f4797a82b0 0x55f4797a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.280+0000 7f6212a00640 10 client.8523 ms_handle_connect on v2:172.21.15.177:6835/2539335973 2024-09-16T07:34:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.280+0000 7f6235e00640 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] conn(0x55f4797a82b0 0x55f4797a8690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbb14639386e74f server_cookie=1f5782845715e99a in_seq=0 out_seq=0 2024-09-16T07:34:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.281+0000 7f6212a00640 1 -- 172.21.15.71:0/3200224427 <== mds.0 v2:172.21.15.177:6835/2539335973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4796f6520 con 0x55f4797a82b0 2024-09-16T07:34:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.281+0000 7f6212a00640 10 client.8523 handle_command_reply: tid=0 2024-09-16T07:34:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 client.8523 shutdown 2024-09-16T07:34:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 -- 172.21.15.71:0/3200224427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797d8af0 msgr2=0x55f4797d8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797d8af0 0x55f4797d8f70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f622c010c40 tx=0x7f622c01b460 comp rx=0 tx=0).stop 2024-09-16T07:34:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 -- 172.21.15.71:0/3200224427 shutdown_connections 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] conn(0x55f4797a82b0 0x55f4797a8690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797d8af0 0x55f4797d8f70 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 0x55f4797aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 --2- 172.21.15.71:0/3200224427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 0x55f4797a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f6212a00640 0 client.8523 ms_handle_reset on v2:172.21.15.177:6835/2539335973 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 -- 172.21.15.71:0/3200224427 >> 172.21.15.71:0/3200224427 conn(0x55f479781a20 msgr2=0x55f4797c5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 -- 172.21.15.71:0/3200224427 shutdown_connections 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 1 -- 172.21.15.71:0/3200224427 wait complete. 2024-09-16T07:34:59.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.282+0000 7f623b843740 20 client.8523 trim_cache size 0 max 16384 2024-09-16T07:34:59.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.301+0000 7f623b843740 1 Processor -- start 2024-09-16T07:34:59.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.301+0000 7f623b843740 1 -- start start 2024-09-16T07:34:59.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f623b843740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f623b843740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 0x55f4797a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f623b843740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797d8af0 0x55f4797a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f623b843740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62301f8110 con 0x55f479781fa0 2024-09-16T07:34:59.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f623b843740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f623014e270 con 0x55f4797d8af0 2024-09-16T07:34:59.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f623b843740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f623014e8c0 con 0x55f4797c68b0 2024-09-16T07:34:59.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f6235e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f6235e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33686/0 (socket says 172.21.15.71:33686) 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f6235e00640 1 -- 172.21.15.71:0/2084161522 learned_addr learned my addr 172.21.15.71:0/2084161522 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f6235e00640 1 -- 172.21.15.71:0/2084161522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 msgr2=0x55f4797a78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f6235e00640 1 --2- 172.21.15.71:0/2084161522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 0x55f4797a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f6235e00640 1 -- 172.21.15.71:0/2084161522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797d8af0 msgr2=0x55f4797a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f6235e00640 1 --2- 172.21.15.71:0/2084161522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797d8af0 0x55f4797a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.302+0000 7f6235e00640 1 -- 172.21.15.71:0/2084161522 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f47977a6e0 con 0x55f479781fa0 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f6235e00640 1 --2- 172.21.15.71:0/2084161522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797a73a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f6224062d60 tx=0x7f6224011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5914c56f610dd83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f621dc00640 1 -- 172.21.15.71:0/2084161522 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224066040 con 0x55f479781fa0 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f621dc00640 1 -- 172.21.15.71:0/2084161522 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f622405f040 con 0x55f479781fa0 2024-09-16T07:34:59.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f621dc00640 1 -- 172.21.15.71:0/2084161522 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622406ae00 con 0x55f479781fa0 2024-09-16T07:34:59.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 -- 172.21.15.71:0/2084161522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 msgr2=0x55f4797a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 --2- 172.21.15.71:0/2084161522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797a73a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f6224062d60 tx=0x7f6224011f10 comp rx=0 tx=0).stop 2024-09-16T07:34:59.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 -- 172.21.15.71:0/2084161522 shutdown_connections 2024-09-16T07:34:59.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 --2- 172.21.15.71:0/2084161522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4797d8af0 0x55f4797a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 --2- 172.21.15.71:0/2084161522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797c68b0 0x55f4797a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 --2- 172.21.15.71:0/2084161522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f479781fa0 0x55f4797a73a0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 -- 172.21.15.71:0/2084161522 >> 172.21.15.71:0/2084161522 conn(0x55f479781a20 msgr2=0x55f47977bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 -- 172.21.15.71:0/2084161522 shutdown_connections 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.303+0000 7f623b843740 1 -- 172.21.15.71:0/2084161522 wait complete. 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.304+0000 7f623b843740 1 -- start start 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.304+0000 7f623b843740 1 Processor -- start 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f623b843740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 0x55f4797a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f623b843740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797c68b0 0x55f4797e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f623b843740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797d8af0 0x55f4798488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f623b843740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f4797c1300 con 0x55f4797c68b0 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f623b843740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4797a96f0 con 0x55f479781fa0 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f623b843740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f4797c0aa0 con 0x55f4797d8af0 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f623b843740 10 client.0 _ll_register_callbacks cb 0x7f622c009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:34:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f6235400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797c68b0 0x55f4797e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f6235400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797c68b0 0x55f4797e94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33688/0 (socket says 172.21.15.71:33688) 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.305+0000 7f6235400640 1 -- 172.21.15.71:0/1714508495 learned_addr learned my addr 172.21.15.71:0/1714508495 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6235400640 1 -- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797d8af0 msgr2=0x55f4798488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6235400640 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797d8af0 0x55f4798488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6235400640 1 -- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 msgr2=0x55f4797a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6235400640 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 0x55f4797a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6235400640 1 -- 172.21.15.71:0/1714508495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4797805c0 con 0x55f4797c68b0 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6235400640 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797c68b0 0x55f4797e94c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f62200099c0 tx=0x7f62200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f6c84bcfd73a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6212a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:34:59.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6212a00640 1 -- 172.21.15.71:0/1714508495 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220035050 con 0x55f4797c68b0 2024-09-16T07:34:59.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6212a00640 1 -- 172.21.15.71:0/1714508495 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f6220049900 con 0x55f4797c68b0 2024-09-16T07:34:59.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f623b843740 1 -- 172.21.15.71:0/1714508495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f622405f040 con 0x55f4797c68b0 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f6212a00640 1 -- 172.21.15.71:0/1714508495 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f6220059050 con 0x55f4797c68b0 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f623b843740 10 client.8529 fetch_fsmap learned FSMap version 75 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.306+0000 7f623b843740 1 -- 172.21.15.71:0/1714508495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55f479780950 con 0x55f4797c68b0 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f6212a00640 1 -- 172.21.15.71:0/1714508495 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f622405f040 con 0x55f4797c68b0 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f6212a00640 1 -- 172.21.15.71:0/1714508495 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f622005c020 con 0x55f4797c68b0 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f623b843740 10 client.8529 fetch_fsmap finished waiting for FSMap version 75 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f623b843740 10 client.8529 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f623b843740 20 client.8529 populate_metadata read hostname 'smithi071' 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f623b843740 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] conn(0x55f4797bde10 0x55f4797be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f623b843740 4 client.8529 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f623b843740 1 -- 172.21.15.71:0/1714508495 --> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] -- command(tid 0: {"prefix": "status"}) -- 0x55f4796f1fd0 con 0x55f4797bde10 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.307+0000 7f6235e00640 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] conn(0x55f4797bde10 0x55f4797be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f6212a00640 10 client.8529 ms_handle_connect on v2:172.21.15.177:6835/2539335973 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f6235e00640 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] conn(0x55f4797bde10 0x55f4797be1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe35f6a518d3d33c server_cookie=21a09a0f6742db20 in_seq=0 out_seq=0 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f6212a00640 1 -- 172.21.15.71:0/1714508495 <== mds.0 v2:172.21.15.177:6835/2539335973 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f4796f1fd0 con 0x55f4797bde10 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f6212a00640 10 client.8529 handle_command_reply: tid=0 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 client.8529 shutdown 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 -- 172.21.15.71:0/1714508495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797c68b0 msgr2=0x55f4797e94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797c68b0 0x55f4797e94c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f62200099c0 tx=0x7f62200046c0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 -- 172.21.15.71:0/1714508495 shutdown_connections 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:6835/2539335973,v1:172.21.15.177:6837/2539335973] conn(0x55f4797bde10 0x55f4797be1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4797d8af0 0x55f4798488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4797c68b0 0x55f4797e94c0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 --2- 172.21.15.71:0/1714508495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f479781fa0 0x55f4797a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f6212a00640 0 client.8529 ms_handle_reset on v2:172.21.15.177:6835/2539335973 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.308+0000 7f623b843740 1 -- 172.21.15.71:0/1714508495 >> 172.21.15.71:0/1714508495 conn(0x55f479781a20 msgr2=0x55f47977b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:59.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f623b843740 1 -- 172.21.15.71:0/1714508495 shutdown_connections 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f623b843740 1 -- 172.21.15.71:0/1714508495 wait complete. 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f623b843740 20 client.8529 trim_cache size 0 max 16384 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 8382, 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:34:59.312 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 10.804343395, 2024-09-16T07:34:59.313 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 75, 2024-09-16T07:34:59.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 119, 2024-09-16T07:34:59.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 119, 2024-09-16T07:34:59.313 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 15.874700607999999 2024-09-16T07:34:59.313 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 -- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f621407a260 msgr2=0x7f621407c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f621407a260 0x7f621407c720 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6220033000 tx=0x7f6220004730 comp rx=0 tx=0).stop 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 -- 172.21.15.71:0/1771582803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6230073fb0 msgr2=0x7f62300b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6230073fb0 0x7f62300b3e00 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f622400b3f0 tx=0x7f622400b8b0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 -- 172.21.15.71:0/1771582803 shutdown_connections 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f621407a260 0x7f621407c720 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f623015f5a0 0x7f62301f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62300ccf00 0x7f62300c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 --2- 172.21.15.71:0/1771582803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6230073fb0 0x7f62300b3e00 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 -- 172.21.15.71:0/1771582803 >> 172.21.15.71:0/1771582803 conn(0x7f62300b1950 msgr2=0x7f623009caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 -- 172.21.15.71:0/1771582803 shutdown_connections 2024-09-16T07:34:59.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.309+0000 7f6238800640 1 -- 172.21.15.71:0/1771582803 wait complete. 2024-09-16T07:34:59.325 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:34:59.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.482+0000 7fde28a00640 1 -- 172.21.15.71:0/3995417488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 msgr2=0x7fde24074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.482+0000 7fde28a00640 1 --2- 172.21.15.71:0/3995417488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 0x7fde24074440 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005fd0 tx=0x7fde1c02a970 comp rx=0 tx=0).stop 2024-09-16T07:34:59.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.482+0000 7fde28a00640 1 -- 172.21.15.71:0/3995417488 shutdown_connections 2024-09-16T07:34:59.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.482+0000 7fde28a00640 1 --2- 172.21.15.71:0/3995417488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde240ccf00 0x7fde240b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.482+0000 7fde28a00640 1 --2- 172.21.15.71:0/3995417488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 0x7fde24074440 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.482+0000 7fde28a00640 1 -- 172.21.15.71:0/3995417488 >> 172.21.15.71:0/3995417488 conn(0x7fde240b1950 msgr2=0x7fde240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:59.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.482+0000 7fde28a00640 1 -- 172.21.15.71:0/3995417488 shutdown_connections 2024-09-16T07:34:59.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.482+0000 7fde28a00640 1 -- 172.21.15.71:0/3995417488 wait complete. 2024-09-16T07:34:59.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.483+0000 7fde28a00640 1 Processor -- start 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.484+0000 7fde28a00640 1 -- start start 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.484+0000 7fde28a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 0x7fde24151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.484+0000 7fde28a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde240ccf00 0x7fde24156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.484+0000 7fde28a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2415f5a0 0x7fde24158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.484+0000 7fde28a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde2406f2e0 con 0x7fde24073fb0 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.484+0000 7fde28a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde2406f160 con 0x7fde2415f5a0 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.484+0000 7fde28a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fde2406f460 con 0x7fde240ccf00 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 0x7fde24151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 0x7fde24151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33700/0 (socket says 172.21.15.71:33700) 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 -- 172.21.15.71:0/647442732 learned_addr learned my addr 172.21.15.71:0/647442732 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:34:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde22800640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2415f5a0 0x7fde24158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21400640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde240ccf00 0x7fde24156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 -- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde240ccf00 msgr2=0x7fde24156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde240ccf00 0x7fde24156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 -- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2415f5a0 msgr2=0x7fde24158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2415f5a0 0x7fde24158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 -- 172.21.15.71:0/647442732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde241571b0 con 0x7fde24073fb0 2024-09-16T07:34:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.485+0000 7fde21e00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 0x7fde24151810 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fde1400b3f0 tx=0x7fde1400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.486+0000 7fde1a000640 1 -- 172.21.15.71:0/647442732 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14015020 con 0x7fde24073fb0 2024-09-16T07:34:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.486+0000 7fde1a000640 1 -- 172.21.15.71:0/647442732 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde14011920 con 0x7fde24073fb0 2024-09-16T07:34:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.486+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde241f39e0 con 0x7fde24073fb0 2024-09-16T07:34:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.486+0000 7fde1a000640 1 -- 172.21.15.71:0/647442732 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14011d70 con 0x7fde24073fb0 2024-09-16T07:34:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.486+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241f3fa0 con 0x7fde24073fb0 2024-09-16T07:34:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.487+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde241574f0 con 0x7fde24073fb0 2024-09-16T07:34:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.488+0000 7fde1a000640 1 -- 172.21.15.71:0/647442732 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fde14018020 con 0x7fde24073fb0 2024-09-16T07:34:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.489+0000 7fde1a000640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde0407a230 0x7fde0407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:34:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.489+0000 7fde1a000640 1 -- 172.21.15.71:0/647442732 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fde14099380 con 0x7fde24073fb0 2024-09-16T07:34:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.489+0000 7fde21400640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde0407a230 0x7fde0407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:34:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.490+0000 7fde21400640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde0407a230 0x7fde0407c6f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fde10005fd0 tx=0x7fde10005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:34:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.493+0000 7fde1a000640 1 -- 172.21.15.71:0/647442732 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fde140a2070 con 0x7fde24073fb0 2024-09-16T07:34:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.717+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fde2409ddf0 con 0x7fde24073fb0 2024-09-16T07:34:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.717+0000 7fde1a000640 1 -- 172.21.15.71:0/647442732 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7fde1401f030 con 0x7fde24073fb0 2024-09-16T07:34:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:34:59.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde0407a230 msgr2=0x7fde0407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde0407a230 0x7fde0407c6f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fde10005fd0 tx=0x7fde10005850 comp rx=0 tx=0).stop 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 msgr2=0x7fde24151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 0x7fde24151810 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fde1400b3f0 tx=0x7fde1400b8b0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 shutdown_connections 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fde0407a230 0x7fde0407c6f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2415f5a0 0x7fde24158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde240ccf00 0x7fde24156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 --2- 172.21.15.71:0/647442732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde24073fb0 0x7fde24151810 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 >> 172.21.15.71:0/647442732 conn(0x7fde240b1950 msgr2=0x7fde2409d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 shutdown_connections 2024-09-16T07:34:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:34:59.720+0000 7fde28a00640 1 -- 172.21.15.71:0/647442732 wait complete. 2024-09-16T07:34:59.735 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:34:59.735 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:34:59.735 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:34:59.735 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:34:59.735 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:34:59.736 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:34:59.736 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:34:59.736 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:34:59.736 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:34:59.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:59.736 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:34:59.771 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:34:59.772 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 41977sec preferred_lft 41977sec 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:34:59.773 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:34:59.774 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:34:59.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:59.775 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:34:59.866 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:34:59.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:34:59.869 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:34:59.905 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:34:59.905 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:34:59.905 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:34:59.905 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:34:59.906 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:34:59.906 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:34:59.906 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:34:59.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:34:59.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:34:59.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:34:59.906 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:35:00.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:35:00.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:35:00.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:00.101 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:35:00.136 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:35:00.136 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:35:00.137 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:35:00.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:00.137 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:35:00.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:35:00.232 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:35:00.267 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:35:00.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:35:00.268 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:35:00.343 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:35:00.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:00.343 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:35:00.472 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:35:00.472 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:35:00.500 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:35:00.500 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:35:00.500 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:35:00.501 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:35:00.501 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:35:00.502 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:35:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:35:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:35:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:35:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:35:00.505 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:35:00.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:00.505 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:35:00.561 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:35:00.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:35:00.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:35:00.662 INFO:teuthology.orchestra.run.smithi071.stdout:client7726 2024-09-16T07:35:00.662 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:00.662 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:00.698 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:35:00.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.897+0000 7efcc0200640 1 -- 172.21.15.71:0/2582204664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb806f220 msgr2=0x7efcb806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:00.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.897+0000 7efcc0200640 1 --2- 172.21.15.71:0/2582204664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb806f220 0x7efcb806f6b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efcb8070850 tx=0x7efcb002a5f0 comp rx=0 tx=0).stop 2024-09-16T07:35:00.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.898+0000 7efcc0200640 1 -- 172.21.15.71:0/2582204664 shutdown_connections 2024-09-16T07:35:00.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.898+0000 7efcc0200640 1 --2- 172.21.15.71:0/2582204664 >> v2:172.21.15.177:3300/0 conn(0x7efcb815cff0 0x7efcb806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:00.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.898+0000 7efcc0200640 1 --2- 172.21.15.71:0/2582204664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb806f220 0x7efcb806f6b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:00.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.898+0000 7efcc0200640 1 -- 172.21.15.71:0/2582204664 >> 172.21.15.71:0/2582204664 conn(0x7efcb80b18e0 msgr2=0x7efcb80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:00.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.898+0000 7efcc0200640 1 -- 172.21.15.71:0/2582204664 shutdown_connections 2024-09-16T07:35:00.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.898+0000 7efcc0200640 1 -- 172.21.15.71:0/2582204664 wait complete. 2024-09-16T07:35:00.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.899+0000 7efcc0200640 1 Processor -- start 2024-09-16T07:35:00.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.899+0000 7efcc0200640 1 -- start start 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.900+0000 7efcc0200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb806f220 0x7efcb80aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.900+0000 7efcc0200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8159aa0 0x7efcb8152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.900+0000 7efcc0200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb815cff0 0x7efcb814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.900+0000 7efcc0200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efcb80cc520 con 0x7efcb8159aa0 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.900+0000 7efcc0200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efcb80cc3a0 con 0x7efcb815cff0 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.900+0000 7efcc0200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efcb80cc6a0 con 0x7efcb806f220 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8159aa0 0x7efcb8152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8159aa0 0x7efcb8152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33728/0 (socket says 172.21.15.71:33728) 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 -- 172.21.15.71:0/1725672709 learned_addr learned my addr 172.21.15.71:0/1725672709 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbd800640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb806f220 0x7efcb80aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbe200640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb815cff0 0x7efcb814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 -- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb806f220 msgr2=0x7efcb80aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:00.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb806f220 0x7efcb80aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:00.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 -- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb815cff0 msgr2=0x7efcb814fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:00.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb815cff0 0x7efcb814fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:00.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 -- 172.21.15.71:0/1725672709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb80cfea0 con 0x7efcb8159aa0 2024-09-16T07:35:00.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbe200640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb815cff0 0x7efcb814fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:00.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.901+0000 7efcbce00640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8159aa0 0x7efcb8152250 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7efca400d6d0 tx=0x7efca400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:00.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.902+0000 7efcb5600640 1 -- 172.21.15.71:0/1725672709 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4017070 con 0x7efcb8159aa0 2024-09-16T07:35:00.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.902+0000 7efcb5600640 1 -- 172.21.15.71:0/1725672709 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efca4005330 con 0x7efcb8159aa0 2024-09-16T07:35:00.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.902+0000 7efcb5600640 1 -- 172.21.15.71:0/1725672709 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4009820 con 0x7efcb8159aa0 2024-09-16T07:35:00.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.902+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb80d09e0 con 0x7efcb8159aa0 2024-09-16T07:35:00.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.902+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb80d0420 con 0x7efcb8159aa0 2024-09-16T07:35:00.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.903+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb80ccb50 con 0x7efcb8159aa0 2024-09-16T07:35:00.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.904+0000 7efcb5600640 1 -- 172.21.15.71:0/1725672709 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7efca4009bd0 con 0x7efcb8159aa0 2024-09-16T07:35:00.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.905+0000 7efcb5600640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc9807a230 0x7efc9807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:00.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.905+0000 7efcb5600640 1 -- 172.21.15.71:0/1725672709 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7efca4099ac0 con 0x7efcb8159aa0 2024-09-16T07:35:00.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.905+0000 7efcbd800640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc9807a230 0x7efc9807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:00.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.906+0000 7efcbd800640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc9807a230 0x7efc9807c6f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efcac001ec0 tx=0x7efcac009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:00.909+0000 7efcb5600640 1 -- 172.21.15.71:0/1725672709 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efca4063540 con 0x7efcb8159aa0 2024-09-16T07:35:01.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.132+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7efcb80b1e20 con 0x7efcb8159aa0 2024-09-16T07:35:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.334+0000 7efcb5600640 1 -- 172.21.15.71:0/1725672709 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v76) ==== 136+0+0 (secure 0 0 0) 0x7efca4067f70 con 0x7efcb8159aa0 2024-09-16T07:35:01.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.337+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc9807a230 msgr2=0x7efc9807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.337+0000 7efcc0200640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc9807a230 0x7efc9807c6f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efcac001ec0 tx=0x7efcac009040 comp rx=0 tx=0).stop 2024-09-16T07:35:01.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.337+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8159aa0 msgr2=0x7efcb8152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.337+0000 7efcc0200640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8159aa0 0x7efcb8152250 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7efca400d6d0 tx=0x7efca400db90 comp rx=0 tx=0).stop 2024-09-16T07:35:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.338+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 shutdown_connections 2024-09-16T07:35:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.338+0000 7efcc0200640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efc9807a230 0x7efc9807c6f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.338+0000 7efcc0200640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb815cff0 0x7efcb814fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.338+0000 7efcc0200640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcb8159aa0 0x7efcb8152250 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.338+0000 7efcc0200640 1 --2- 172.21.15.71:0/1725672709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb806f220 0x7efcb80aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.338+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 >> 172.21.15.71:0/1725672709 conn(0x7efcb80b18e0 msgr2=0x7efcb815e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.338+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 shutdown_connections 2024-09-16T07:35:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.338+0000 7efcc0200640 1 -- 172.21.15.71:0/1725672709 wait complete. 2024-09-16T07:35:01.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:35:01.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.512+0000 7f7ebae00640 1 -- 172.21.15.71:0/3185985171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e9c004800 msgr2=0x7f7e9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.512+0000 7f7ebae00640 1 --2- 172.21.15.71:0/3185985171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e9c004800 0x7f7e9c004c80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c2e0 tx=0x7f7e9c005830 comp rx=0 tx=0).stop 2024-09-16T07:35:01.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.512+0000 7f7ebae00640 1 -- 172.21.15.71:0/3185985171 shutdown_connections 2024-09-16T07:35:01.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.512+0000 7f7ebae00640 1 --2- 172.21.15.71:0/3185985171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e9c004800 0x7f7e9c004c80 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.512+0000 7f7ebae00640 1 -- 172.21.15.71:0/3185985171 >> 172.21.15.71:0/3185985171 conn(0x7f7eb40cfa20 msgr2=0x7f7eb40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:01.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.512+0000 7f7ebae00640 1 -- 172.21.15.71:0/3185985171 shutdown_connections 2024-09-16T07:35:01.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.512+0000 7f7ebae00640 1 -- 172.21.15.71:0/3185985171 wait complete. 2024-09-16T07:35:01.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.513+0000 7f7ebae00640 1 Processor -- start 2024-09-16T07:35:01.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.513+0000 7f7ebae00640 1 -- start start 2024-09-16T07:35:01.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.514+0000 7f7ebae00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e9c004800 0x7f7eb41f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:01.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.514+0000 7f7ebae00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb40cd5f0 0x7f7eb414c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:01.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.514+0000 7f7ebae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb4157b40 0x7f7eb4157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:01.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.514+0000 7f7ebae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40b2c80 con 0x7f7eb4157b40 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.514+0000 7f7ebae00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40b2b00 con 0x7f7eb40cd5f0 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.514+0000 7f7ebae00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb4073d40 con 0x7f7e9c004800 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb4157b40 0x7f7eb4157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eb9a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e9c004800 0x7f7eb41f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb4157b40 0x7f7eb4157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33740/0 (socket says 172.21.15.71:33740) 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 -- 172.21.15.71:0/1421924468 learned_addr learned my addr 172.21.15.71:0/1421924468 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eb9000640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb40cd5f0 0x7f7eb414c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 -- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e9c004800 msgr2=0x7f7eb41f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e9c004800 0x7f7eb41f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 -- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb40cd5f0 msgr2=0x7f7eb414c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb40cd5f0 0x7f7eb414c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 -- 172.21.15.71:0/1421924468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb41f5700 con 0x7f7eb4157b40 2024-09-16T07:35:01.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.515+0000 7f7eba400640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb4157b40 0x7f7eb4157ff0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4070640 tx=0x7f7eac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:01.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.516+0000 7f7eb1600640 1 -- 172.21.15.71:0/1421924468 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac003d80 con 0x7f7eb4157b40 2024-09-16T07:35:01.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.516+0000 7f7eb1600640 1 -- 172.21.15.71:0/1421924468 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eac004b20 con 0x7f7eb4157b40 2024-09-16T07:35:01.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.516+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb40cdb20 con 0x7f7eb4157b40 2024-09-16T07:35:01.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.516+0000 7f7eb1600640 1 -- 172.21.15.71:0/1421924468 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac004050 con 0x7f7eb4157b40 2024-09-16T07:35:01.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.516+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb40ce0e0 con 0x7f7eb4157b40 2024-09-16T07:35:01.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.517+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb40742a0 con 0x7f7eb4157b40 2024-09-16T07:35:01.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.518+0000 7f7eb1600640 1 -- 172.21.15.71:0/1421924468 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f7eac0041f0 con 0x7f7eb4157b40 2024-09-16T07:35:01.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.519+0000 7f7eb1600640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e9407a230 0x7f7e9407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:01.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.519+0000 7f7eb1600640 1 -- 172.21.15.71:0/1421924468 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f7eac09ab50 con 0x7f7eb4157b40 2024-09-16T07:35:01.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.519+0000 7f7eb9a00640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e9407a230 0x7f7e9407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:01.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.520+0000 7f7eb9a00640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e9407a230 0x7f7e9407c6f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006fd0 tx=0x7f7ea8006d10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:01.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.523+0000 7f7eb1600640 1 -- 172.21.15.71:0/1421924468 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7eac01f060 con 0x7f7eb4157b40 2024-09-16T07:35:01.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.729+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7eb40b4de0 con 0x7f7e9407a230 2024-09-16T07:35:01.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.731+0000 7f7eb1600640 1 -- 172.21.15.71:0/1421924468 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7eb40b4de0 con 0x7f7e9407a230 2024-09-16T07:35:01.730 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:35:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:35:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:01.731 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:35:01.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e9407a230 msgr2=0x7f7e9407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e9407a230 0x7f7e9407c6f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006fd0 tx=0x7f7ea8006d10 comp rx=0 tx=0).stop 2024-09-16T07:35:01.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb4157b40 msgr2=0x7f7eb4157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb4157b40 0x7f7eb4157ff0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4070640 tx=0x7f7eac00f790 comp rx=0 tx=0).stop 2024-09-16T07:35:01.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 shutdown_connections 2024-09-16T07:35:01.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e9407a230 0x7f7e9407c6f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eb4157b40 0x7f7eb4157ff0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb40cd5f0 0x7f7eb414c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 --2- 172.21.15.71:0/1421924468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e9c004800 0x7f7eb41f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 >> 172.21.15.71:0/1421924468 conn(0x7f7eb40cfa20 msgr2=0x7f7eb41515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 shutdown_connections 2024-09-16T07:35:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.733+0000 7f7ebae00640 1 -- 172.21.15.71:0/1421924468 wait complete. 2024-09-16T07:35:01.749 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:35:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.908+0000 7f4905400640 1 -- 172.21.15.71:0/149932566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4900145750 msgr2=0x7f4900073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.908+0000 7f4905400640 1 --2- 172.21.15.71:0/149932566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4900145750 0x7f4900073440 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48f0005fd0 tx=0x7f48f002a820 comp rx=0 tx=0).stop 2024-09-16T07:35:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.908+0000 7f4905400640 1 -- 172.21.15.71:0/149932566 shutdown_connections 2024-09-16T07:35:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.908+0000 7f4905400640 1 --2- 172.21.15.71:0/149932566 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49000cccd0 0x7f49000cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.908+0000 7f4905400640 1 --2- 172.21.15.71:0/149932566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4900145750 0x7f4900073440 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.908+0000 7f4905400640 1 -- 172.21.15.71:0/149932566 >> 172.21.15.71:0/149932566 conn(0x7f49000b1840 msgr2=0x7f49000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.908+0000 7f4905400640 1 -- 172.21.15.71:0/149932566 shutdown_connections 2024-09-16T07:35:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.908+0000 7f4905400640 1 -- 172.21.15.71:0/149932566 wait complete. 2024-09-16T07:35:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.910+0000 7f4905400640 1 Processor -- start 2024-09-16T07:35:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.910+0000 7f4905400640 1 -- start start 2024-09-16T07:35:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f4905400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490006efa0 0x7f49000aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f4905400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49000cccd0 0x7f490015e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f4905400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4900145750 0x7f49000c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f4905400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f49000ce480 con 0x7f490006efa0 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f4905400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49000ce300 con 0x7f49000cccd0 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f4905400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49000ce600 con 0x7f4900145750 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fe800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490006efa0 0x7f49000aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fe800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490006efa0 0x7f49000aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33752/0 (socket says 172.21.15.71:33752) 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fe800640 1 -- 172.21.15.71:0/2328629835 learned_addr learned my addr 172.21.15.71:0/2328629835 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fde00640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49000cccd0 0x7f490015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48ff200640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4900145750 0x7f49000c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fe800640 1 -- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4900145750 msgr2=0x7f49000c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fe800640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4900145750 0x7f49000c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fe800640 1 -- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49000cccd0 msgr2=0x7f490015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fe800640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49000cccd0 0x7f490015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.911+0000 7f48fe800640 1 -- 172.21.15.71:0/2328629835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49000b2c40 con 0x7f490006efa0 2024-09-16T07:35:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.912+0000 7f48fe800640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490006efa0 0x7f49000aef30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f48f0035040 tx=0x7f48f0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.912+0000 7f48f6a00640 1 -- 172.21.15.71:0/2328629835 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0037040 con 0x7f490006efa0 2024-09-16T07:35:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.912+0000 7f48f6a00640 1 -- 172.21.15.71:0/2328629835 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f0042070 con 0x7f490006efa0 2024-09-16T07:35:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.912+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4900141810 con 0x7f490006efa0 2024-09-16T07:35:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.912+0000 7f48f6a00640 1 -- 172.21.15.71:0/2328629835 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f00042a0 con 0x7f490006efa0 2024-09-16T07:35:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.912+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4900141dd0 con 0x7f490006efa0 2024-09-16T07:35:01.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.913+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49000cdbb0 con 0x7f490006efa0 2024-09-16T07:35:01.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.914+0000 7f48f6a00640 1 -- 172.21.15.71:0/2328629835 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f48f0004440 con 0x7f490006efa0 2024-09-16T07:35:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.915+0000 7f48f6a00640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48e007a230 0x7f48e007c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.915+0000 7f48f6a00640 1 -- 172.21.15.71:0/2328629835 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f48f00ba450 con 0x7f490006efa0 2024-09-16T07:35:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.915+0000 7f48fde00640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48e007a230 0x7f48e007c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.916+0000 7f48fde00640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48e007a230 0x7f48e007c6f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f49000cb780 tx=0x7f48ec009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:01.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:01.919+0000 7f48f6a00640 1 -- 172.21.15.71:0/2328629835 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f48f00845c0 con 0x7f490006efa0 2024-09-16T07:35:02.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.102+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4900075230 con 0x7f490006efa0 2024-09-16T07:35:02.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.103+0000 7f48f6a00640 1 -- 172.21.15.71:0/2328629835 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f48f0083ec0 con 0x7f490006efa0 2024-09-16T07:35:02.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48e007a230 msgr2=0x7f48e007c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48e007a230 0x7f48e007c6f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f49000cb780 tx=0x7f48ec009040 comp rx=0 tx=0).stop 2024-09-16T07:35:02.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490006efa0 msgr2=0x7f49000aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490006efa0 0x7f49000aef30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f48f0035040 tx=0x7f48f0032c90 comp rx=0 tx=0).stop 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 shutdown_connections 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f48e007a230 0x7f48e007c6f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4900145750 0x7f49000c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49000cccd0 0x7f490015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 --2- 172.21.15.71:0/2328629835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490006efa0 0x7f49000aef30 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 >> 172.21.15.71:0/2328629835 conn(0x7f49000b1840 msgr2=0x7f49000a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 shutdown_connections 2024-09-16T07:35:02.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.105+0000 7f4905400640 1 -- 172.21.15.71:0/2328629835 wait complete. 2024-09-16T07:35:02.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:35:02.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 -- 172.21.15.71:0/2720942425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70840ccb00 msgr2=0x7f708415f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 --2- 172.21.15.71:0/2720942425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70840ccb00 0x7f708415f9f0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f70840d0cd0 tx=0x7f707c02a980 comp rx=0 tx=0).stop 2024-09-16T07:35:02.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 -- 172.21.15.71:0/2720942425 shutdown_connections 2024-09-16T07:35:02.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 --2- 172.21.15.71:0/2720942425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70840ccb00 0x7f708415f9f0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 --2- 172.21.15.71:0/2720942425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f708406efa0 0x7f70840cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 --2- 172.21.15.71:0/2720942425 >> v2:172.21.15.177:3300/0 conn(0x7f7084145750 0x7f7084073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 -- 172.21.15.71:0/2720942425 >> 172.21.15.71:0/2720942425 conn(0x7f70840b1840 msgr2=0x7f70840b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:02.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 -- 172.21.15.71:0/2720942425 shutdown_connections 2024-09-16T07:35:02.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.281+0000 7f7089c00640 1 -- 172.21.15.71:0/2720942425 wait complete. 2024-09-16T07:35:02.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.283+0000 7f7089c00640 1 Processor -- start 2024-09-16T07:35:02.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.283+0000 7f7089c00640 1 -- start start 2024-09-16T07:35:02.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7089c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708406efa0 0x7f70841f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:02.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7089c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840ccb00 0x7f70840c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:02.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7089c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084145750 0x7f70840c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:02.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7089c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70841600b0 con 0x7f708406efa0 2024-09-16T07:35:02.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7089c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f708415ff30 con 0x7f70840ccb00 2024-09-16T07:35:02.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7089c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7084160230 con 0x7f7084145750 2024-09-16T07:35:02.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7082a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708406efa0 0x7f70841f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:02.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7082a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708406efa0 0x7f70841f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33774/0 (socket says 172.21.15.71:33774) 2024-09-16T07:35:02.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7082a00640 1 -- 172.21.15.71:0/2114636505 learned_addr learned my addr 172.21.15.71:0/2114636505 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:02.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7082000640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840ccb00 0x7f70840c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:02.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.284+0000 7f7083400640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084145750 0x7f70840c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:02.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f7082a00640 1 -- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084145750 msgr2=0x7f70840c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f7082a00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084145750 0x7f70840c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f7082a00640 1 -- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840ccb00 msgr2=0x7f70840c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f7082a00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840ccb00 0x7f70840c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f7082a00640 1 -- 172.21.15.71:0/2114636505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70841f5ac0 con 0x7f708406efa0 2024-09-16T07:35:02.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f7082a00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708406efa0 0x7f70841f5580 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f707400d740 tx=0x7f707400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:02.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f707aa00640 1 -- 172.21.15.71:0/2114636505 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074017070 con 0x7f708406efa0 2024-09-16T07:35:02.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f707aa00640 1 -- 172.21.15.71:0/2114636505 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7074009d20 con 0x7f708406efa0 2024-09-16T07:35:02.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70840ab630 con 0x7f708406efa0 2024-09-16T07:35:02.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f707aa00640 1 -- 172.21.15.71:0/2114636505 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70740106c0 con 0x7f708406efa0 2024-09-16T07:35:02.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.285+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70840abbf0 con 0x7f708406efa0 2024-09-16T07:35:02.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.286+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7084160710 con 0x7f708406efa0 2024-09-16T07:35:02.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.287+0000 7f707aa00640 1 -- 172.21.15.71:0/2114636505 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f707401a070 con 0x7f708406efa0 2024-09-16T07:35:02.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.288+0000 7f707aa00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f706407a230 0x7f706407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:02.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.288+0000 7f707aa00640 1 -- 172.21.15.71:0/2114636505 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f7074099960 con 0x7f708406efa0 2024-09-16T07:35:02.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.289+0000 7f7082000640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f706407a230 0x7f706407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:02.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.289+0000 7f7082000640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f706407a230 0x7f706407c6f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f708406ac10 tx=0x7f7070005e50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:02.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.292+0000 7f707aa00640 1 -- 172.21.15.71:0/2114636505 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7074063a30 con 0x7f708406efa0 2024-09-16T07:35:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.480+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f7084073430 con 0x7f708406efa0 2024-09-16T07:35:02.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.480+0000 7f707aa00640 1 -- 172.21.15.71:0/2114636505 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v48) ==== 116+0+2 (secure 0 0 0) 0x7f7074063330 con 0x7f708406efa0 2024-09-16T07:35:02.480 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:35:02.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.482+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f706407a230 msgr2=0x7f706407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.482+0000 7f7089c00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f706407a230 0x7f706407c6f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f708406ac10 tx=0x7f7070005e50 comp rx=0 tx=0).stop 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.482+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708406efa0 msgr2=0x7f70841f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.482+0000 7f7089c00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708406efa0 0x7f70841f5580 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f707400d740 tx=0x7f707400dc00 comp rx=0 tx=0).stop 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.483+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 shutdown_connections 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.483+0000 7f7089c00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f706407a230 0x7f706407c6f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.483+0000 7f7089c00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7084145750 0x7f70840c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.483+0000 7f7089c00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70840ccb00 0x7f70840c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.483+0000 7f7089c00640 1 --2- 172.21.15.71:0/2114636505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708406efa0 0x7f70841f5580 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.483+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 >> 172.21.15.71:0/2114636505 conn(0x7f70840b1840 msgr2=0x7f70840b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.483+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 shutdown_connections 2024-09-16T07:35:02.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.483+0000 7f7089c00640 1 -- 172.21.15.71:0/2114636505 wait complete. 2024-09-16T07:35:02.498 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:35:02.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.659+0000 7fd0dda00640 1 -- 172.21.15.71:0/766341062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 msgr2=0x7fd0d8073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.659+0000 7fd0dda00640 1 --2- 172.21.15.71:0/766341062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 0x7fd0d8073d40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009830 tx=0x7fd0c402a480 comp rx=0 tx=0).stop 2024-09-16T07:35:02.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.659+0000 7fd0dda00640 1 -- 172.21.15.71:0/766341062 shutdown_connections 2024-09-16T07:35:02.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.659+0000 7fd0dda00640 1 --2- 172.21.15.71:0/766341062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 0x7fd0d8073d40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.659+0000 7fd0dda00640 1 -- 172.21.15.71:0/766341062 >> 172.21.15.71:0/766341062 conn(0x7fd0d80b1510 msgr2=0x7fd0d80b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:02.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.659+0000 7fd0dda00640 1 -- 172.21.15.71:0/766341062 shutdown_connections 2024-09-16T07:35:02.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.659+0000 7fd0dda00640 1 -- 172.21.15.71:0/766341062 wait complete. 2024-09-16T07:35:02.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.660+0000 7fd0dda00640 1 Processor -- start 2024-09-16T07:35:02.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.661+0000 7fd0dda00640 1 -- start start 2024-09-16T07:35:02.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.661+0000 7fd0dda00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d8074280 0x7fd0d81f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:02.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.661+0000 7fd0dda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 0x7fd0d80c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:02.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.661+0000 7fd0dda00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d815bb90 0x7fd0d80c2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0dda00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d806f470 con 0x7fd0d80b2810 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0dda00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d815bf70 con 0x7fd0d8074280 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0dda00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d806f5f0 con 0x7fd0d815bb90 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 0x7fd0d80c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 0x7fd0d80c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33792/0 (socket says 172.21.15.71:33792) 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7400640 1 -- 172.21.15.71:0/1042215531 learned_addr learned my addr 172.21.15.71:0/1042215531 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7e00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d8074280 0x7fd0d81f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0dd000640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d815bb90 0x7fd0d80c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7400640 1 -- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d815bb90 msgr2=0x7fd0d80c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7400640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d815bb90 0x7fd0d80c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7400640 1 -- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d8074280 msgr2=0x7fd0d81f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7400640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d8074280 0x7fd0d81f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0d7400640 1 -- 172.21.15.71:0/1042215531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d80c41d0 con 0x7fd0d80b2810 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.662+0000 7fd0dd000640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d815bb90 0x7fd0d80c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.663+0000 7fd0d7400640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 0x7fd0d80c3c90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009950 tx=0x7fd0c4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.663+0000 7fd0cfe00640 1 -- 172.21.15.71:0/1042215531 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4040070 con 0x7fd0d80b2810 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.663+0000 7fd0cfe00640 1 -- 172.21.15.71:0/1042215531 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0c4003af0 con 0x7fd0d80b2810 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.663+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d815c220 con 0x7fd0d80b2810 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.663+0000 7fd0cfe00640 1 -- 172.21.15.71:0/1042215531 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4045460 con 0x7fd0d80b2810 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.663+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d815c7e0 con 0x7fd0d80b2810 2024-09-16T07:35:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.664+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d8073d40 con 0x7fd0d80b2810 2024-09-16T07:35:02.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.665+0000 7fd0cfe00640 1 -- 172.21.15.71:0/1042215531 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fd0c4039040 con 0x7fd0d80b2810 2024-09-16T07:35:02.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.666+0000 7fd0cfe00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0b807a230 0x7fd0b807c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:02.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.666+0000 7fd0cfe00640 1 -- 172.21.15.71:0/1042215531 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fd0c40badb0 con 0x7fd0d80b2810 2024-09-16T07:35:02.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.666+0000 7fd0d7e00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0b807a230 0x7fd0b807c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:02.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.667+0000 7fd0d7e00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0b807a230 0x7fd0b807c6f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8002200 tx=0x7fd0c80019f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:02.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.670+0000 7fd0cfe00640 1 -- 172.21.15.71:0/1042215531 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd0c4084e80 con 0x7fd0d80b2810 2024-09-16T07:35:02.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.856+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fd0d8069d20 con 0x7fd0d80b2810 2024-09-16T07:35:02.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.860+0000 7fd0cfe00640 1 -- 172.21.15.71:0/1042215531 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7fd0c4084780 con 0x7fd0d80b2810 2024-09-16T07:35:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0b807a230 msgr2=0x7fd0b807c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0b807a230 0x7fd0b807c6f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8002200 tx=0x7fd0c80019f0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 msgr2=0x7fd0d80c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 0x7fd0d80c3c90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009950 tx=0x7fd0c4032d50 comp rx=0 tx=0).stop 2024-09-16T07:35:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 shutdown_connections 2024-09-16T07:35:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd0b807a230 0x7fd0b807c6f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d815bb90 0x7fd0d80c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0d80b2810 0x7fd0d80c3c90 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 --2- 172.21.15.71:0/1042215531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d8074280 0x7fd0d81f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:02.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 >> 172.21.15.71:0/1042215531 conn(0x7fd0d80b1510 msgr2=0x7fd0d80b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:02.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 shutdown_connections 2024-09-16T07:35:02.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:02.862+0000 7fd0dda00640 1 -- 172.21.15.71:0/1042215531 wait complete. 2024-09-16T07:35:02.878 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T07:35:02.879 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:02.879+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:02.879 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:02.879+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:02.879 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:02.879+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:02.879 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:02.879+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:02.879 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:02.879+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:03.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.040+0000 7fd79b800640 1 -- 172.21.15.71:0/3955109789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd794159aa0 msgr2=0x7fd7940741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.040+0000 7fd79b800640 1 --2- 172.21.15.71:0/3955109789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd794159aa0 0x7fd7940741a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd784005fd0 tx=0x7fd78402a8b0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.040+0000 7fd79b800640 1 -- 172.21.15.71:0/3955109789 shutdown_connections 2024-09-16T07:35:03.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.040+0000 7fd79b800640 1 --2- 172.21.15.71:0/3955109789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd794159aa0 0x7fd7940741a0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.040+0000 7fd79b800640 1 -- 172.21.15.71:0/3955109789 >> 172.21.15.71:0/3955109789 conn(0x7fd7940b18e0 msgr2=0x7fd7940b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.040+0000 7fd79b800640 1 -- 172.21.15.71:0/3955109789 shutdown_connections 2024-09-16T07:35:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.040+0000 7fd79b800640 1 -- 172.21.15.71:0/3955109789 wait complete. 2024-09-16T07:35:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.042+0000 7fd79b800640 1 Processor -- start 2024-09-16T07:35:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.042+0000 7fd79b800640 1 -- start start 2024-09-16T07:35:03.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd79b800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79406f220 0x7fd7940cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd79b800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd794159aa0 0x7fd7940b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd79b800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd79415cff0 0x7fd7940b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd79b800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd79406f6b0 con 0x7fd79406f220 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd79b800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7940ccc50 con 0x7fd79415cff0 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd79b800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd79406f830 con 0x7fd794159aa0 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd798e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79406f220 0x7fd7940cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd798e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79406f220 0x7fd7940cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33808/0 (socket says 172.21.15.71:33808) 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd798e00640 1 -- 172.21.15.71:0/2897075276 learned_addr learned my addr 172.21.15.71:0/2897075276 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd793e00640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd794159aa0 0x7fd7940b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd799800640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd79415cff0 0x7fd7940b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.043+0000 7fd798e00640 1 -- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd794159aa0 msgr2=0x7fd7940b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd798e00640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd794159aa0 0x7fd7940b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd798e00640 1 -- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd79415cff0 msgr2=0x7fd7940b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd798e00640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd79415cff0 0x7fd7940b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd798e00640 1 -- 172.21.15.71:0/2897075276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79409aaa0 con 0x7fd79406f220 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd798e00640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79406f220 0x7fd7940cf530 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd784006010 tx=0x7fd784003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd790c00640 1 -- 172.21.15.71:0/2897075276 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784040070 con 0x7fd79406f220 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd790c00640 1 -- 172.21.15.71:0/2897075276 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd784004540 con 0x7fd79406f220 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7940cfa70 con 0x7fd79406f220 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd790c00640 1 -- 172.21.15.71:0/2897075276 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784003e10 con 0x7fd79406f220 2024-09-16T07:35:03.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.044+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79409ad10 con 0x7fd79406f220 2024-09-16T07:35:03.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.045+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7940741a0 con 0x7fd79406f220 2024-09-16T07:35:03.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.047+0000 7fd790c00640 1 -- 172.21.15.71:0/2897075276 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7fd784039040 con 0x7fd79406f220 2024-09-16T07:35:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.048+0000 7fd790c00640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd77407a230 0x7fd77407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.048+0000 7fd790c00640 1 -- 172.21.15.71:0/2897075276 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fd7840bb020 con 0x7fd79406f220 2024-09-16T07:35:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.048+0000 7fd793e00640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd77407a230 0x7fd77407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.049+0000 7fd793e00640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd77407a230 0x7fd77407c6f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd79415ba70 tx=0x7fd780009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.052+0000 7fd790c00640 1 -- 172.21.15.71:0/2897075276 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd7940cfa70 con 0x7fd79406f220 2024-09-16T07:35:03.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.261+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fd79415e520 con 0x7fd77407a230 2024-09-16T07:35:03.271 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:03.271+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:03.271 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:03.271+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:03.271 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:03.271+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:03.271 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:03.271+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:03.271 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:03.271+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:03.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.309+0000 7fd790c00640 1 -- 172.21.15.71:0/2897075276 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd79415e520 con 0x7fd77407a230 2024-09-16T07:35:03.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd77407a230 msgr2=0x7fd77407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd77407a230 0x7fd77407c6f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd79415ba70 tx=0x7fd780009040 comp rx=0 tx=0).stop 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79406f220 msgr2=0x7fd7940cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79406f220 0x7fd7940cf530 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd784006010 tx=0x7fd784003c20 comp rx=0 tx=0).stop 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 shutdown_connections 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd77407a230 0x7fd77407c6f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd79415cff0 0x7fd7940b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd794159aa0 0x7fd7940b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 --2- 172.21.15.71:0/2897075276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79406f220 0x7fd7940cf530 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 >> 172.21.15.71:0/2897075276 conn(0x7fd7940b18e0 msgr2=0x7fd7940aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 shutdown_connections 2024-09-16T07:35:03.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.312+0000 7fd79b800640 1 -- 172.21.15.71:0/2897075276 wait complete. 2024-09-16T07:35:03.327 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:35:03.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.490+0000 7f945e400640 1 -- 172.21.15.71:0/390318531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f945806efa0 msgr2=0x7f94580cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.490+0000 7f945e400640 1 --2- 172.21.15.71:0/390318531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f945806efa0 0x7f94580cdbb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9450005fd0 tx=0x7f945002a540 comp rx=0 tx=0).stop 2024-09-16T07:35:03.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.490+0000 7f945e400640 1 -- 172.21.15.71:0/390318531 shutdown_connections 2024-09-16T07:35:03.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.490+0000 7f945e400640 1 --2- 172.21.15.71:0/390318531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f945806efa0 0x7f94580cdbb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.490+0000 7f945e400640 1 --2- 172.21.15.71:0/390318531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94580cccd0 0x7f9458073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.490+0000 7f945e400640 1 -- 172.21.15.71:0/390318531 >> 172.21.15.71:0/390318531 conn(0x7f94580b1840 msgr2=0x7f94580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:03.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.490+0000 7f945e400640 1 -- 172.21.15.71:0/390318531 shutdown_connections 2024-09-16T07:35:03.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.490+0000 7f945e400640 1 -- 172.21.15.71:0/390318531 wait complete. 2024-09-16T07:35:03.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.491+0000 7f945e400640 1 Processor -- start 2024-09-16T07:35:03.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.492+0000 7f945e400640 1 -- start start 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.492+0000 7f945e400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945806efa0 0x7f94581f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.492+0000 7f945e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94580cccd0 0x7f945813eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.492+0000 7f945e400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9458145750 0x7f94581545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f945e400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f94580ce480 con 0x7f94580cccd0 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f945e400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f94580ce300 con 0x7f9458145750 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f945e400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f94580ce600 con 0x7f945806efa0 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9456a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94580cccd0 0x7f945813eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9456a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94580cccd0 0x7f945813eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33818/0 (socket says 172.21.15.71:33818) 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9456a00640 1 -- 172.21.15.71:0/1854859849 learned_addr learned my addr 172.21.15.71:0/1854859849 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9457400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945806efa0 0x7f94581f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9457e00640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9458145750 0x7f94581545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9456a00640 1 -- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945806efa0 msgr2=0x7f94581f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9456a00640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945806efa0 0x7f94581f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9456a00640 1 -- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9458145750 msgr2=0x7f94581545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9456a00640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9458145750 0x7f94581545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.493+0000 7f9456a00640 1 -- 172.21.15.71:0/1854859849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94581550a0 con 0x7f94580cccd0 2024-09-16T07:35:03.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.494+0000 7f9456a00640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94580cccd0 0x7f945813eb00 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f944400e9f0 tx=0x7f944400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.494+0000 7f944f400640 1 -- 172.21.15.71:0/1854859849 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944400cd80 con 0x7f94580cccd0 2024-09-16T07:35:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.494+0000 7f944f400640 1 -- 172.21.15.71:0/1854859849 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9444010430 con 0x7f94580cccd0 2024-09-16T07:35:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.494+0000 7f944f400640 1 -- 172.21.15.71:0/1854859849 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444010840 con 0x7f94580cccd0 2024-09-16T07:35:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.494+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9458155380 con 0x7f94580cccd0 2024-09-16T07:35:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.494+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94581f4a20 con 0x7f94580cccd0 2024-09-16T07:35:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.495+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94580cdbb0 con 0x7f94580cccd0 2024-09-16T07:35:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.496+0000 7f944f400640 1 -- 172.21.15.71:0/1854859849 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f94440109e0 con 0x7f94580cccd0 2024-09-16T07:35:03.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.497+0000 7f944f400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f943807a260 0x7f943807c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.497+0000 7f944f400640 1 -- 172.21.15.71:0/1854859849 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f944409a210 con 0x7f94580cccd0 2024-09-16T07:35:03.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.497+0000 7f9457400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f943807a260 0x7f943807c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.498+0000 7f9457400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f943807a260 0x7f943807c720 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9448001e60 tx=0x7f9448009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:03.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.501+0000 7f944f400640 1 -- 172.21.15.71:0/1854859849 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9444063d70 con 0x7f94580cccd0 2024-09-16T07:35:03.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.716+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f94580b1120 con 0x7f943807a260 2024-09-16T07:35:03.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.723+0000 7f944f400640 1 -- 172.21.15.71:0/1854859849 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f94580b1120 con 0x7f943807a260 2024-09-16T07:35:03.722 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f943807a260 msgr2=0x7f943807c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f943807a260 0x7f943807c720 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9448001e60 tx=0x7f9448009040 comp rx=0 tx=0).stop 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94580cccd0 msgr2=0x7f945813eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94580cccd0 0x7f945813eb00 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f944400e9f0 tx=0x7f944400eeb0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 shutdown_connections 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f943807a260 0x7f943807c720 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9458145750 0x7f94581545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94580cccd0 0x7f945813eb00 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 --2- 172.21.15.71:0/1854859849 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945806efa0 0x7f94581f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 >> 172.21.15.71:0/1854859849 conn(0x7f94580b1840 msgr2=0x7f94580cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:03.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 shutdown_connections 2024-09-16T07:35:03.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.725+0000 7f945e400640 1 -- 172.21.15.71:0/1854859849 wait complete. 2024-09-16T07:35:03.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:35:03.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.902+0000 7f01ac200640 1 -- 172.21.15.71:0/532699276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 msgr2=0x7f018c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.902+0000 7f01ac200640 1 --2- 172.21.15.71:0/532699276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 0x7f018c004c80 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f0190006fa0 tx=0x7f018c005830 comp rx=0 tx=0).stop 2024-09-16T07:35:03.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.902+0000 7f01ac200640 1 -- 172.21.15.71:0/532699276 shutdown_connections 2024-09-16T07:35:03.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.902+0000 7f01ac200640 1 --2- 172.21.15.71:0/532699276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 0x7f018c004c80 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.902+0000 7f01ac200640 1 -- 172.21.15.71:0/532699276 >> 172.21.15.71:0/532699276 conn(0x7f01a40b1840 msgr2=0x7f01a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:03.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.902+0000 7f01ac200640 1 -- 172.21.15.71:0/532699276 shutdown_connections 2024-09-16T07:35:03.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.902+0000 7f01ac200640 1 -- 172.21.15.71:0/532699276 wait complete. 2024-09-16T07:35:03.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.904+0000 7f01ac200640 1 Processor -- start 2024-09-16T07:35:03.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.904+0000 7f01ac200640 1 -- start start 2024-09-16T07:35:03.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01ac200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 0x7f01a4141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01ac200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a40cb400 0x7f01a40a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01ac200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a40cb860 0x7f01a40a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01ac200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01a40737d0 con 0x7f018c004800 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01ac200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01a4073650 con 0x7f01a40cb860 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01ac200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01a40cc5c0 con 0x7f01a40cb400 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01a9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 0x7f01a4141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01a9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 0x7f01a4141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33842/0 (socket says 172.21.15.71:33842) 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01a9800640 1 -- 172.21.15.71:0/2785317053 learned_addr learned my addr 172.21.15.71:0/2785317053 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01a8e00640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a40cb400 0x7f01a40a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.905+0000 7f01aa200640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a40cb860 0x7f01a40a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a9800640 1 -- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a40cb400 msgr2=0x7f01a40a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a9800640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a40cb400 0x7f01a40a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a9800640 1 -- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a40cb860 msgr2=0x7f01a40a9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a9800640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a40cb860 0x7f01a40a9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a9800640 1 -- 172.21.15.71:0/2785317053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a4153150 con 0x7f018c004800 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a9800640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 0x7f01a4141960 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0198001890 tx=0x7f0198001c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a1600640 1 -- 172.21.15.71:0/2785317053 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198015020 con 0x7f018c004800 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a1600640 1 -- 172.21.15.71:0/2785317053 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0198010980 con 0x7f018c004800 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a4155380 con 0x7f018c004800 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01a1600640 1 -- 172.21.15.71:0/2785317053 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198010db0 con 0x7f018c004800 2024-09-16T07:35:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.906+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a4155900 con 0x7f018c004800 2024-09-16T07:35:03.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.907+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a415f9f0 con 0x7f018c004800 2024-09-16T07:35:03.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.908+0000 7f01a1600640 1 -- 172.21.15.71:0/2785317053 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 41) ==== 103633+0+0 (secure 0 0 0) 0x7f0198018020 con 0x7f018c004800 2024-09-16T07:35:03.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.909+0000 7f01a1600640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f018407a230 0x7f018407c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:03.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.909+0000 7f01a1600640 1 -- 172.21.15.71:0/2785317053 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f019809a350 con 0x7f018c004800 2024-09-16T07:35:03.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.910+0000 7f01a8e00640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f018407a230 0x7f018407c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:03.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.910+0000 7f01a8e00640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f018407a230 0x7f018407c6f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0190033000 tx=0x7f01a406f410 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:03.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:03.913+0000 7f01a1600640 1 -- 172.21.15.71:0/2785317053 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f019801f020 con 0x7f018c004800 2024-09-16T07:35:04.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.124+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f01a415f730 con 0x7f018407a230 2024-09-16T07:35:04.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.130+0000 7f01a1600640 1 -- 172.21.15.71:0/2785317053 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f01a415f730 con 0x7f018407a230 2024-09-16T07:35:04.129 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:04.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f018407a230 msgr2=0x7f018407c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:04.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f018407a230 0x7f018407c6f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0190033000 tx=0x7f01a406f410 comp rx=0 tx=0).stop 2024-09-16T07:35:04.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 msgr2=0x7f01a4141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:04.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 0x7f01a4141960 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0198001890 tx=0x7f0198001c90 comp rx=0 tx=0).stop 2024-09-16T07:35:04.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 shutdown_connections 2024-09-16T07:35:04.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f018407a230 0x7f018407c6f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a40cb860 0x7f01a40a9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a40cb400 0x7f01a40a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 --2- 172.21.15.71:0/2785317053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c004800 0x7f01a4141960 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 >> 172.21.15.71:0/2785317053 conn(0x7f01a40b1840 msgr2=0x7f01a40b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.132+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 shutdown_connections 2024-09-16T07:35:04.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:04.133+0000 7f01ac200640 1 -- 172.21.15.71:0/2785317053 wait complete. 2024-09-16T07:35:04.149 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:04.149 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:04.149 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:04.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:04.185 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:35:11.755 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:35:11.755 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:35:11.755 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.48768 s, 143 MB/s 2024-09-16T07:35:11.757 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:11.757 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:11.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:11.793 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:35:16.555 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:35:16.559 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:35:19.201 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:35:19.201 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:35:19.201 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.33096 s, 146 MB/s 2024-09-16T07:35:19.202 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:19.202 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:19.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:19.238 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:35:26.769 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:35:26.769 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:35:26.769 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.4539 s, 144 MB/s 2024-09-16T07:35:26.771 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T07:35:26.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.935+0000 7efdefe00640 1 -- 172.21.15.71:0/1871908661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 msgr2=0x7efdf00d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.935+0000 7efdefe00640 1 --2- 172.21.15.71:0/1871908661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 0x7efdf00d0650 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efde0009830 tx=0x7efde002a510 comp rx=0 tx=0).stop 2024-09-16T07:35:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.935+0000 7efdefe00640 1 -- 172.21.15.71:0/1871908661 shutdown_connections 2024-09-16T07:35:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.935+0000 7efdefe00640 1 --2- 172.21.15.71:0/1871908661 >> v2:172.21.15.177:3300/0 conn(0x7efdf00ce550 0x7efdf00ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.935+0000 7efdefe00640 1 --2- 172.21.15.71:0/1871908661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 0x7efdf00d0650 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.935+0000 7efdefe00640 1 -- 172.21.15.71:0/1871908661 >> 172.21.15.71:0/1871908661 conn(0x7efdf00d1cd0 msgr2=0x7efdf00d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.935+0000 7efdefe00640 1 -- 172.21.15.71:0/1871908661 shutdown_connections 2024-09-16T07:35:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.935+0000 7efdefe00640 1 -- 172.21.15.71:0/1871908661 wait complete. 2024-09-16T07:35:26.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.936+0000 7efdefe00640 1 Processor -- start 2024-09-16T07:35:26.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.936+0000 7efdefe00640 1 -- start start 2024-09-16T07:35:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.937+0000 7efdefe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf0070960 0x7efdf00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.937+0000 7efdefe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdf00ce550 0x7efdf01f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.937+0000 7efdefe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 0x7efdf00c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.937+0000 7efdefe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efdf00cd690 con 0x7efdf0070960 2024-09-16T07:35:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.937+0000 7efdefe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efdf00cd510 con 0x7efdf00ce550 2024-09-16T07:35:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.937+0000 7efdefe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efdf0071520 con 0x7efdf015f2b0 2024-09-16T07:35:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf0070960 0x7efdf00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf0070960 0x7efdf00c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50412/0 (socket says 172.21.15.71:50412) 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 -- 172.21.15.71:0/318779314 learned_addr learned my addr 172.21.15.71:0/318779314 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efde7e00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdf00ce550 0x7efdf01f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdef400640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 0x7efdf00c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 -- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 msgr2=0x7efdf00c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 0x7efdf00c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 -- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdf00ce550 msgr2=0x7efdf01f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdf00ce550 0x7efdf01f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 -- 172.21.15.71:0/318779314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf00ca4c0 con 0x7efdf0070960 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdef400640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 0x7efdf00c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.938+0000 7efdeea00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf0070960 0x7efdf00c9590 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7efde0009950 tx=0x7efde0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.939+0000 7efde6a00640 1 -- 172.21.15.71:0/318779314 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0005c90 con 0x7efdf0070960 2024-09-16T07:35:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.939+0000 7efde6a00640 1 -- 172.21.15.71:0/318779314 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efde0031e50 con 0x7efdf0070960 2024-09-16T07:35:26.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.939+0000 7efdefe00640 1 -- 172.21.15.71:0/318779314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf00c9c20 con 0x7efdf0070960 2024-09-16T07:35:26.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.939+0000 7efde6a00640 1 -- 172.21.15.71:0/318779314 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0005260 con 0x7efdf0070960 2024-09-16T07:35:26.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.939+0000 7efdefe00640 1 -- 172.21.15.71:0/318779314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf00ca1e0 con 0x7efdf0070960 2024-09-16T07:35:26.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.940+0000 7efddfe00640 1 -- 172.21.15.71:0/318779314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf01f4960 con 0x7efdf0070960 2024-09-16T07:35:26.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.941+0000 7efde6a00640 1 -- 172.21.15.71:0/318779314 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 42) ==== 103735+0+0 (secure 0 0 0) 0x7efde0005400 con 0x7efdf0070960 2024-09-16T07:35:26.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.942+0000 7efde6a00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdcc07a350 0x7efdcc07c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:26.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.942+0000 7efde6a00640 1 -- 172.21.15.71:0/318779314 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7efde00ba7e0 con 0x7efdf0070960 2024-09-16T07:35:26.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.942+0000 7efde7e00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdcc07a350 0x7efdcc07c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:26.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.943+0000 7efde7e00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdcc07a350 0x7efdcc07c810 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efdf009fd10 tx=0x7efdd8005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:26.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:26.946+0000 7efde6a00640 1 -- 172.21.15.71:0/318779314 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efde003f030 con 0x7efdf0070960 2024-09-16T07:35:27.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.163+0000 7efddfe00640 1 -- 172.21.15.71:0/318779314 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7efdf0071ab0 con 0x7efdcc07a350 2024-09-16T07:35:27.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.175+0000 7efde6a00640 1 -- 172.21.15.71:0/318779314 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efdf0071ab0 con 0x7efdcc07a350 2024-09-16T07:35:27.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 -- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdcc07a350 msgr2=0x7efdcc07c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:27.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdcc07a350 0x7efdcc07c810 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efdf009fd10 tx=0x7efdd8005f90 comp rx=0 tx=0).stop 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 -- 172.21.15.71:0/318779314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf0070960 msgr2=0x7efdf00c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf0070960 0x7efdf00c9590 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7efde0009950 tx=0x7efde0033c20 comp rx=0 tx=0).stop 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 -- 172.21.15.71:0/318779314 shutdown_connections 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdcc07a350 0x7efdcc07c810 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf015f2b0 0x7efdf00c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdf00ce550 0x7efdf01f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 --2- 172.21.15.71:0/318779314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf0070960 0x7efdf00c9590 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 -- 172.21.15.71:0/318779314 >> 172.21.15.71:0/318779314 conn(0x7efdf00d1cd0 msgr2=0x7efdf00b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 -- 172.21.15.71:0/318779314 shutdown_connections 2024-09-16T07:35:27.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.178+0000 7efddfe00640 1 -- 172.21.15.71:0/318779314 wait complete. 2024-09-16T07:35:27.193 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:35:27.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.353+0000 7f742f200640 1 -- 172.21.15.71:0/3362661624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7428161290 msgr2=0x7f74280ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:27.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.353+0000 7f742f200640 1 --2- 172.21.15.71:0/3362661624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7428161290 0x7f74280ce830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f741c009860 tx=0x7f741c02a5a0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.353+0000 7f742f200640 1 -- 172.21.15.71:0/3362661624 shutdown_connections 2024-09-16T07:35:27.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.353+0000 7f742f200640 1 --2- 172.21.15.71:0/3362661624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7428161290 0x7f74280ce830 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.353+0000 7f742f200640 1 -- 172.21.15.71:0/3362661624 >> 172.21.15.71:0/3362661624 conn(0x7f74280d3d20 msgr2=0x7f74280d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:27.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.353+0000 7f742f200640 1 -- 172.21.15.71:0/3362661624 shutdown_connections 2024-09-16T07:35:27.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.353+0000 7f742f200640 1 -- 172.21.15.71:0/3362661624 wait complete. 2024-09-16T07:35:27.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.355+0000 7f742f200640 1 Processor -- start 2024-09-16T07:35:27.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.355+0000 7f742f200640 1 -- start start 2024-09-16T07:35:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742f200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74280d2000 0x7f74280cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7428161290 0x7f74280b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7428161670 0x7f7428154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74280b1580 con 0x7f7428161670 2024-09-16T07:35:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f74280b1400 con 0x7f7428161290 2024-09-16T07:35:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742f200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f74280b1700 con 0x7f74280d2000 2024-09-16T07:35:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7428161670 0x7f7428154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7428161670 0x7f7428154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50418/0 (socket says 172.21.15.71:50418) 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742e800640 1 -- 172.21.15.71:0/1697695567 learned_addr learned my addr 172.21.15.71:0/1697695567 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742de00640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74280d2000 0x7f74280cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.356+0000 7f742d400640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7428161290 0x7f74280b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f742e800640 1 -- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74280d2000 msgr2=0x7f74280cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f742e800640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74280d2000 0x7f74280cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f742e800640 1 -- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7428161290 msgr2=0x7f74280b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f742e800640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7428161290 0x7f74280b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f742e800640 1 -- 172.21.15.71:0/1697695567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74280c8cd0 con 0x7f7428161670 2024-09-16T07:35:27.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f742e800640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7428161670 0x7f7428154da0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f742400cbd0 tx=0x7f742400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:27.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f7422000640 1 -- 172.21.15.71:0/1697695567 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424016020 con 0x7f7428161670 2024-09-16T07:35:27.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f7422000640 1 -- 172.21.15.71:0/1697695567 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f742400ae50 con 0x7f7428161670 2024-09-16T07:35:27.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f7422000640 1 -- 172.21.15.71:0/1697695567 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424012680 con 0x7f7428161670 2024-09-16T07:35:27.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74280c7de0 con 0x7f7428161670 2024-09-16T07:35:27.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.357+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74280c83a0 con 0x7f7428161670 2024-09-16T07:35:27.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.358+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74280c87a0 con 0x7f7428161670 2024-09-16T07:35:27.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.360+0000 7f7422000640 1 -- 172.21.15.71:0/1697695567 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 42) ==== 103735+0+0 (secure 0 0 0) 0x7f7424019020 con 0x7f7428161670 2024-09-16T07:35:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.360+0000 7f7422000640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f740c07a350 0x7f740c07c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.361+0000 7f7422000640 1 -- 172.21.15.71:0/1697695567 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f7424099fc0 con 0x7f7428161670 2024-09-16T07:35:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.361+0000 7f742de00640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f740c07a350 0x7f740c07c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.361+0000 7f742de00640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f740c07a350 0x7f740c07c810 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f741c034000 tx=0x7f741c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:27.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.364+0000 7f7422000640 1 -- 172.21.15.71:0/1697695567 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f74280c87a0 con 0x7f7428161670 2024-09-16T07:35:27.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.581+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f74280b1c20 con 0x7f740c07a350 2024-09-16T07:35:27.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.587+0000 7f7422000640 1 -- 172.21.15.71:0/1697695567 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f74280b1c20 con 0x7f740c07a350 2024-09-16T07:35:27.586 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:27.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f740c07a350 msgr2=0x7f740c07c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:27.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f740c07a350 0x7f740c07c810 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f741c034000 tx=0x7f741c004730 comp rx=0 tx=0).stop 2024-09-16T07:35:27.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7428161670 msgr2=0x7f7428154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:27.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7428161670 0x7f7428154da0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f742400cbd0 tx=0x7f742400a430 comp rx=0 tx=0).stop 2024-09-16T07:35:27.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 shutdown_connections 2024-09-16T07:35:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f740c07a350 0x7f740c07c810 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7428161670 0x7f7428154da0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7428161290 0x7f74280b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 --2- 172.21.15.71:0/1697695567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74280d2000 0x7f74280cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 >> 172.21.15.71:0/1697695567 conn(0x7f74280d3d20 msgr2=0x7f74280b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 shutdown_connections 2024-09-16T07:35:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:27.589+0000 7f742f200640 1 -- 172.21.15.71:0/1697695567 wait complete. 2024-09-16T07:35:27.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:27.604 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:27.604 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:35:27.641 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:27.641 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T07:35:27.641 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:27.642 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T07:35:29.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:29.643 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:29.644 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:35:29.685 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:29.685 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T07:35:29.685 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:29.686 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T07:35:29.687 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone33330 2024-09-16T07:35:29.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.898+0000 7f93e7e00640 1 -- 172.21.15.71:0/2957257291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 msgr2=0x7f93c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:29.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.898+0000 7f93e7e00640 1 --2- 172.21.15.71:0/2957257291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 0x7f93c80051c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f93d80016c0 tx=0x7f93c8005830 comp rx=0 tx=0).stop 2024-09-16T07:35:29.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.898+0000 7f93e5e00640 1 -- 172.21.15.71:0/2957257291 reap_dead start 2024-09-16T07:35:29.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.898+0000 7f93e7e00640 1 -- 172.21.15.71:0/2957257291 shutdown_connections 2024-09-16T07:35:29.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.898+0000 7f93e7e00640 1 -- 172.21.15.71:0/2957257291 >> 172.21.15.71:0/2957257291 conn(0x7f93e00b1840 msgr2=0x7f93e00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:29.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.898+0000 7f93e7e00640 1 -- 172.21.15.71:0/2957257291 shutdown_connections 2024-09-16T07:35:29.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.898+0000 7f93e7e00640 1 -- 172.21.15.71:0/2957257291 wait complete. 2024-09-16T07:35:29.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.900+0000 7f93e7e00640 1 Processor -- start 2024-09-16T07:35:29.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.900+0000 7f93e7e00640 1 -- start start 2024-09-16T07:35:29.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 0x7f93e013e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:29.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93e0144030 0x7f93e013b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:29.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93e0144490 0x7f93e013b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:29.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f93e00ce2c0 con 0x7f93c8004d40 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e7e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f93e00736a0 con 0x7f93e0144490 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e7e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f93e00ce440 con 0x7f93e0144030 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 0x7f93e013e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 0x7f93e013e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50436/0 (socket says 172.21.15.71:50436) 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5400640 1 -- 172.21.15.71:0/2437015039 learned_addr learned my addr 172.21.15.71:0/2437015039 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5e00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93e0144490 0x7f93e013b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e4a00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93e0144030 0x7f93e013b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5400640 1 -- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93e0144030 msgr2=0x7f93e013b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5400640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93e0144030 0x7f93e013b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5400640 1 -- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93e0144490 msgr2=0x7f93e013b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5400640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93e0144490 0x7f93e013b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.901+0000 7f93e5400640 1 -- 172.21.15.71:0/2437015039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e0157e50 con 0x7f93c8004d40 2024-09-16T07:35:29.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.902+0000 7f93e5400640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 0x7f93e013e680 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f93d4011d60 tx=0x7f93d400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:29.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.902+0000 7f93dd600640 1 -- 172.21.15.71:0/2437015039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4003d80 con 0x7f93c8004d40 2024-09-16T07:35:29.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.902+0000 7f93dd600640 1 -- 172.21.15.71:0/2437015039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93d4011e50 con 0x7f93c8004d40 2024-09-16T07:35:29.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.902+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e013dba0 con 0x7f93c8004d40 2024-09-16T07:35:29.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.902+0000 7f93dd600640 1 -- 172.21.15.71:0/2437015039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4004d80 con 0x7f93c8004d40 2024-09-16T07:35:29.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.902+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e0157100 con 0x7f93c8004d40 2024-09-16T07:35:29.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.903+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e00ce9a0 con 0x7f93c8004d40 2024-09-16T07:35:29.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.904+0000 7f93dd600640 1 -- 172.21.15.71:0/2437015039 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 42) ==== 103735+0+0 (secure 0 0 0) 0x7f93d4004050 con 0x7f93c8004d40 2024-09-16T07:35:29.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.905+0000 7f93dd600640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f93c007a350 0x7f93c007c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:29.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.905+0000 7f93dd600640 1 -- 172.21.15.71:0/2437015039 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f93d409a9b0 con 0x7f93c8004d40 2024-09-16T07:35:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.906+0000 7f93e4a00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f93c007a350 0x7f93c007c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.906+0000 7f93e4a00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f93c007a350 0x7f93c007c810 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f93cc000c00 tx=0x7f93cc0067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:29.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:29.909+0000 7f93dd600640 1 -- 172.21.15.71:0/2437015039 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f93d4064a80 con 0x7f93c8004d40 2024-09-16T07:35:30.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.126+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone33330", "target": ["mon-mgr", ""]}) -- 0x7f93e0158130 con 0x7f93c007a350 2024-09-16T07:35:30.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.209+0000 7f93dd600640 1 -- 172.21.15.71:0/2437015039 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f93e0158130 con 0x7f93c007a350 2024-09-16T07:35:30.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f93c007a350 msgr2=0x7f93c007c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:30.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f93c007a350 0x7f93c007c810 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f93cc000c00 tx=0x7f93cc0067a0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 msgr2=0x7f93e013e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:30.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 0x7f93e013e680 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f93d4011d60 tx=0x7f93d400f790 comp rx=0 tx=0).stop 2024-09-16T07:35:30.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 shutdown_connections 2024-09-16T07:35:30.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f93c007a350 0x7f93c007c810 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93e0144490 0x7f93e013b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93e0144030 0x7f93e013b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 --2- 172.21.15.71:0/2437015039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93c8004d40 0x7f93e013e680 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 >> 172.21.15.71:0/2437015039 conn(0x7f93e00b1840 msgr2=0x7f93e00af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:30.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 shutdown_connections 2024-09-16T07:35:30.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.211+0000 7f93e7e00640 1 -- 172.21.15.71:0/2437015039 wait complete. 2024-09-16T07:35:30.212 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.212+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.212 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.212+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.212 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.212+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.212 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.212+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.213 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.212+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.227 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone60941 2024-09-16T07:35:30.228 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.229+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.229 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.229+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.229 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.229+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.229 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.229+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.229 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:30.229+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.390+0000 7fb323c00640 1 -- 172.21.15.71:0/1365302657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c145750 msgr2=0x7fb31c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.390+0000 7fb323c00640 1 --2- 172.21.15.71:0/1365302657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c145750 0x7fb31c073440 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0cfb70 tx=0x7fb30c02a560 comp rx=0 tx=0).stop 2024-09-16T07:35:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.390+0000 7fb323c00640 1 -- 172.21.15.71:0/1365302657 shutdown_connections 2024-09-16T07:35:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.390+0000 7fb323c00640 1 --2- 172.21.15.71:0/1365302657 >> v2:172.21.15.177:3300/0 conn(0x7fb31c0cccd0 0x7fb31c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.390+0000 7fb323c00640 1 --2- 172.21.15.71:0/1365302657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c145750 0x7fb31c073440 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.390+0000 7fb323c00640 1 -- 172.21.15.71:0/1365302657 >> 172.21.15.71:0/1365302657 conn(0x7fb31c0b1840 msgr2=0x7fb31c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:30.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.390+0000 7fb323c00640 1 -- 172.21.15.71:0/1365302657 shutdown_connections 2024-09-16T07:35:30.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.390+0000 7fb323c00640 1 -- 172.21.15.71:0/1365302657 wait complete. 2024-09-16T07:35:30.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.392+0000 7fb323c00640 1 Processor -- start 2024-09-16T07:35:30.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.392+0000 7fb323c00640 1 -- start start 2024-09-16T07:35:30.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb323c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c06efa0 0x7fb31c13bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:30.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb323c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb31c0cccd0 0x7fb31c1f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:30.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb323c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb31c145750 0x7fb31c1f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:30.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb323c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0ce480 con 0x7fb31c06efa0 2024-09-16T07:35:30.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb323c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0ce300 con 0x7fb31c145750 2024-09-16T07:35:30.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb323c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c0ce600 con 0x7fb31c0cccd0 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb321200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c06efa0 0x7fb31c13bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb321200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c06efa0 0x7fb31c13bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50460/0 (socket says 172.21.15.71:50460) 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.393+0000 7fb321200640 1 -- 172.21.15.71:0/2653800257 learned_addr learned my addr 172.21.15.71:0/2653800257 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb321c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb31c145750 0x7fb31c1f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb31be00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb31c0cccd0 0x7fb31c1f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb321200640 1 -- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb31c0cccd0 msgr2=0x7fb31c1f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb321200640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb31c0cccd0 0x7fb31c1f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb321200640 1 -- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb31c145750 msgr2=0x7fb31c1f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb321200640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb31c145750 0x7fb31c1f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb321200640 1 -- 172.21.15.71:0/2653800257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c0b4310 con 0x7fb31c06efa0 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb321c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb31c145750 0x7fb31c1f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb321200640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c06efa0 0x7fb31c13bcd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fb30c000c00 tx=0x7fb30c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:30.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb318c00640 1 -- 172.21.15.71:0/2653800257 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c03e030 con 0x7fb31c06efa0 2024-09-16T07:35:30.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb318c00640 1 -- 172.21.15.71:0/2653800257 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb30c003d10 con 0x7fb31c06efa0 2024-09-16T07:35:30.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c1f5b00 con 0x7fb31c06efa0 2024-09-16T07:35:30.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.394+0000 7fb318c00640 1 -- 172.21.15.71:0/2653800257 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c035040 con 0x7fb31c06efa0 2024-09-16T07:35:30.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.395+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c13c210 con 0x7fb31c06efa0 2024-09-16T07:35:30.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.395+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c0cdbb0 con 0x7fb31c06efa0 2024-09-16T07:35:30.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.397+0000 7fb318c00640 1 -- 172.21.15.71:0/2653800257 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 42) ==== 103735+0+0 (secure 0 0 0) 0x7fb30c041030 con 0x7fb31c06efa0 2024-09-16T07:35:30.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.398+0000 7fb318c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2fc07a350 0x7fb2fc07c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:30.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.398+0000 7fb318c00640 1 -- 172.21.15.71:0/2653800257 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fb30c0b9e80 con 0x7fb31c06efa0 2024-09-16T07:35:30.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.399+0000 7fb31be00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2fc07a350 0x7fb2fc07c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:30.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.399+0000 7fb31be00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2fc07a350 0x7fb2fc07c810 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0af330 tx=0x7fb308009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:30.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.402+0000 7fb318c00640 1 -- 172.21.15.71:0/2653800257 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb30c083f50 con 0x7fb31c06efa0 2024-09-16T07:35:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:30.619+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone60941", "target": ["mon-mgr", ""]}) -- 0x7fb31c144ee0 con 0x7fb2fc07a350 2024-09-16T07:35:31.213 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:31.213+0000 7fe71f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:31.213 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:31.213+0000 7fe71f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:31.213 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:31.213+0000 7fe71f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:31.213 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:31.213+0000 7fe71f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:31.214 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:31.213+0000 7fe71f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.296+0000 7fb318c00640 1 -- 172.21.15.71:0/2653800257 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb31c144ee0 con 0x7fb2fc07a350 2024-09-16T07:35:31.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2fc07a350 msgr2=0x7fb2fc07c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2fc07a350 0x7fb2fc07c810 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0af330 tx=0x7fb308009040 comp rx=0 tx=0).stop 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c06efa0 msgr2=0x7fb31c13bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c06efa0 0x7fb31c13bcd0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fb30c000c00 tx=0x7fb30c003820 comp rx=0 tx=0).stop 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 shutdown_connections 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb2fc07a350 0x7fb2fc07c810 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb31c145750 0x7fb31c1f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb31c0cccd0 0x7fb31c1f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 --2- 172.21.15.71:0/2653800257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb31c06efa0 0x7fb31c13bcd0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 >> 172.21.15.71:0/2653800257 conn(0x7fb31c0b1840 msgr2=0x7fb31c0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 shutdown_connections 2024-09-16T07:35:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.299+0000 7fb323c00640 1 -- 172.21.15.71:0/2653800257 wait complete. 2024-09-16T07:35:31.317 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone87142 2024-09-16T07:35:31.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.538+0000 7f48a1c00640 1 -- 172.21.15.71:0/3455467986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f489c0c5730 msgr2=0x7f489c0c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:31.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.538+0000 7f48a1c00640 1 --2- 172.21.15.71:0/3455467986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f489c0c5730 0x7f489c0c6430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4888009860 tx=0x7f488802a510 comp rx=0 tx=0).stop 2024-09-16T07:35:31.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.538+0000 7f48a1c00640 1 -- 172.21.15.71:0/3455467986 shutdown_connections 2024-09-16T07:35:31.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.538+0000 7f48a1c00640 1 --2- 172.21.15.71:0/3455467986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f489c0c5730 0x7f489c0c6430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:31.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.538+0000 7f48a1c00640 1 -- 172.21.15.71:0/3455467986 >> 172.21.15.71:0/3455467986 conn(0x7f489c0cc490 msgr2=0x7f489c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:31.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.538+0000 7f48a1c00640 1 -- 172.21.15.71:0/3455467986 shutdown_connections 2024-09-16T07:35:31.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.538+0000 7f48a1c00640 1 -- 172.21.15.71:0/3455467986 wait complete. 2024-09-16T07:35:31.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.541+0000 7f48a1c00640 1 Processor -- start 2024-09-16T07:35:31.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.541+0000 7f48a1c00640 1 -- start start 2024-09-16T07:35:31.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.542+0000 7f48a1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f489c06e660 0x7f489c0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.542+0000 7f48a1c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f489c0c5730 0x7f489c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.542+0000 7f48a1c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f489c15f670 0x7f489c14d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.542+0000 7f48a1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f489c0ca4f0 con 0x7f489c06e660 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.542+0000 7f48a1c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f489c0ca370 con 0x7f489c0c5730 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.542+0000 7f48a1c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f489c06f220 con 0x7f489c15f670 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f489c06e660 0x7f489c0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f489c06e660 0x7f489c0ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50472/0 (socket says 172.21.15.71:50472) 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489be00640 1 -- 172.21.15.71:0/3450824351 learned_addr learned my addr 172.21.15.71:0/3450824351 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489b400640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f489c0c5730 0x7f489c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f48a1200640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f489c15f670 0x7f489c14d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489be00640 1 -- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f489c15f670 msgr2=0x7f489c14d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489be00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f489c15f670 0x7f489c14d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:31.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489be00640 1 -- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f489c0c5730 msgr2=0x7f489c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:31.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489be00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f489c0c5730 0x7f489c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:31.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.543+0000 7f489be00640 1 -- 172.21.15.71:0/3450824351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c0b4680 con 0x7f489c06e660 2024-09-16T07:35:31.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.544+0000 7f489be00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f489c06e660 0x7f489c0ceea0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c7a0 tx=0x7f488c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:31.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.544+0000 7f4893e00640 1 -- 172.21.15.71:0/3450824351 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c017070 con 0x7f489c06e660 2024-09-16T07:35:31.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.544+0000 7f4893e00640 1 -- 172.21.15.71:0/3450824351 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f488c005070 con 0x7f489c06e660 2024-09-16T07:35:31.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.544+0000 7f4893e00640 1 -- 172.21.15.71:0/3450824351 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0054a0 con 0x7f489c06e660 2024-09-16T07:35:31.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.544+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c0b4960 con 0x7f489c06e660 2024-09-16T07:35:31.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.545+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c14db60 con 0x7f489c06e660 2024-09-16T07:35:31.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.546+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c15eda0 con 0x7f489c06e660 2024-09-16T07:35:31.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.549+0000 7f4893e00640 1 -- 172.21.15.71:0/3450824351 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 42) ==== 103735+0+0 (secure 0 0 0) 0x7f488c01a070 con 0x7f489c06e660 2024-09-16T07:35:31.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.550+0000 7f4893e00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f487c07a350 0x7f487c07c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:31.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.550+0000 7f4893e00640 1 -- 172.21.15.71:0/3450824351 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f488c098d30 con 0x7f489c06e660 2024-09-16T07:35:31.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.555+0000 7f4893e00640 1 -- 172.21.15.71:0/3450824351 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f488c062850 con 0x7f489c06e660 2024-09-16T07:35:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.564+0000 7f489b400640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f487c07a350 0x7f487c07c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.568+0000 7f489b400640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f487c07a350 0x7f487c07c810 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4888034000 tx=0x7f4888004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.716+0000 7f4893e00640 1 -- 172.21.15.71:0/3450824351 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 43) ==== 103837+0+0 (secure 0 0 0) 0x7f488c05eac0 con 0x7f489c06e660 2024-09-16T07:35:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:31.904+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone87142", "target": ["mon-mgr", ""]}) -- 0x7f489c0cf5a0 con 0x7f487c07a350 2024-09-16T07:35:32.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.313+0000 7f4893e00640 1 -- 172.21.15.71:0/3450824351 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4888004340 con 0x7f487c07a350 2024-09-16T07:35:32.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.319+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f487c07a350 msgr2=0x7f487c07c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:32.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.319+0000 7f48a1c00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f487c07a350 0x7f487c07c810 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4888034000 tx=0x7f4888004730 comp rx=0 tx=0).stop 2024-09-16T07:35:32.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.320+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f489c06e660 msgr2=0x7f489c0ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:32.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.320+0000 7f48a1c00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f489c06e660 0x7f489c0ceea0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c7a0 tx=0x7f488c00cc60 comp rx=0 tx=0).stop 2024-09-16T07:35:32.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.320+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 shutdown_connections 2024-09-16T07:35:32.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.320+0000 7f48a1c00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f487c07a350 0x7f487c07c810 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:32.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.320+0000 7f48a1c00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f489c15f670 0x7f489c14d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:32.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.320+0000 7f48a1c00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f489c0c5730 0x7f489c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:32.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.320+0000 7f48a1c00640 1 --2- 172.21.15.71:0/3450824351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f489c06e660 0x7f489c0ceea0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:32.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.320+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 >> 172.21.15.71:0/3450824351 conn(0x7f489c0cc490 msgr2=0x7f489c15e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:32.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.321+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 shutdown_connections 2024-09-16T07:35:32.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:32.321+0000 7f48a1c00640 1 -- 172.21.15.71:0/3450824351 wait complete. 2024-09-16T07:35:32.330 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.330+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.330 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.330+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.330 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.330+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.330 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.330+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.330 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.330+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.507 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.507+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.507 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.507+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.507 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.507+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.508 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.507+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:32.508 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:32.507+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:33.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone33330 2024-09-16T07:35:33.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.508+0000 7f9091800640 1 -- 172.21.15.71:0/848410157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0cccd0 msgr2=0x7f908c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:33.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.508+0000 7f9091800640 1 --2- 172.21.15.71:0/848410157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0cccd0 0x7f908c073440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f908c0cfba0 tx=0x7f907802a560 comp rx=0 tx=0).stop 2024-09-16T07:35:33.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.508+0000 7f9091800640 1 -- 172.21.15.71:0/848410157 shutdown_connections 2024-09-16T07:35:33.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.508+0000 7f9091800640 1 --2- 172.21.15.71:0/848410157 >> v2:172.21.15.177:3300/0 conn(0x7f908c06efa0 0x7f908c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:33.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.508+0000 7f9091800640 1 --2- 172.21.15.71:0/848410157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0cccd0 0x7f908c073440 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:33.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.508+0000 7f9091800640 1 -- 172.21.15.71:0/848410157 >> 172.21.15.71:0/848410157 conn(0x7f908c0b1840 msgr2=0x7f908c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:33.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.508+0000 7f9091800640 1 -- 172.21.15.71:0/848410157 shutdown_connections 2024-09-16T07:35:33.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.508+0000 7f9091800640 1 -- 172.21.15.71:0/848410157 wait complete. 2024-09-16T07:35:33.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.510+0000 7f9091800640 1 Processor -- start 2024-09-16T07:35:33.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.510+0000 7f9091800640 1 -- start start 2024-09-16T07:35:33.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f9091800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c06efa0 0x7f908c13d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f9091800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0cccd0 0x7f908c0ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f9091800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c145750 0x7f908c1f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f9091800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0ce480 con 0x7f908c145750 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f9091800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0ce300 con 0x7f908c06efa0 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f9091800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f908c0ce600 con 0x7f908c0cccd0 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f908b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c145750 0x7f908c1f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f908a000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0cccd0 0x7f908c0ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f908b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c145750 0x7f908c1f2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50492/0 (socket says 172.21.15.71:50492) 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f908aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c06efa0 0x7f908c13d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:33.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.511+0000 7f908b400640 1 -- 172.21.15.71:0/3172843754 learned_addr learned my addr 172.21.15.71:0/3172843754 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f908b400640 1 -- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0cccd0 msgr2=0x7f908c0ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f908b400640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0cccd0 0x7f908c0ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f908b400640 1 -- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c06efa0 msgr2=0x7f908c13d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f908b400640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c06efa0 0x7f908c13d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f908b400640 1 -- 172.21.15.71:0/3172843754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c1f3350 con 0x7f908c145750 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f908b400640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c145750 0x7f908c1f2900 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f908400eaa0 tx=0x7f908400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f9082a00640 1 -- 172.21.15.71:0/3172843754 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908400cdb0 con 0x7f908c145750 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f9082a00640 1 -- 172.21.15.71:0/3172843754 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9084004c10 con 0x7f908c145750 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f9082a00640 1 -- 172.21.15.71:0/3172843754 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084005020 con 0x7f908c145750 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c1f3630 con 0x7f908c145750 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.512+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c13def0 con 0x7f908c145750 2024-09-16T07:35:33.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.513+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c1f3870 con 0x7f908c145750 2024-09-16T07:35:33.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.514+0000 7f9082a00640 1 -- 172.21.15.71:0/3172843754 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 43) ==== 103837+0+0 (secure 0 0 0) 0x7f90840051c0 con 0x7f908c145750 2024-09-16T07:35:33.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.515+0000 7f9082a00640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07a4a0 0x7f906c07c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:33.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.515+0000 7f9082a00640 1 -- 172.21.15.71:0/3172843754 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5852+0+0 (secure 0 0 0) 0x7f908409a1c0 con 0x7f908c145750 2024-09-16T07:35:33.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.516+0000 7f908aa00640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07a4a0 0x7f906c07c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:33.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.516+0000 7f908aa00640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07a4a0 0x7f906c07c960 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f907c001720 tx=0x7f907c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:33.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.520+0000 7f9082a00640 1 -- 172.21.15.71:0/3172843754 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f908c1f3870 con 0x7f908c145750 2024-09-16T07:35:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.739+0000 7f9082a00640 1 -- 172.21.15.71:0/3172843754 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7f908405feb0 con 0x7f908c145750 2024-09-16T07:35:33.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:33.759+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone33330", "target": ["mon-mgr", ""]}) -- 0x7f908c0cdbb0 con 0x7f906c07a4a0 2024-09-16T07:35:35.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.252+0000 7f9082a00640 1 -- 172.21.15.71:0/3172843754 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f908c0cdbb0 con 0x7f906c07a4a0 2024-09-16T07:35:35.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.254+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07a4a0 msgr2=0x7f906c07c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:35.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.254+0000 7f9091800640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07a4a0 0x7f906c07c960 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f907c001720 tx=0x7f907c009040 comp rx=0 tx=0).stop 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.254+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c145750 msgr2=0x7f908c1f2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.254+0000 7f9091800640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c145750 0x7f908c1f2900 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f908400eaa0 tx=0x7f908400ef60 comp rx=0 tx=0).stop 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.255+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 shutdown_connections 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.255+0000 7f9091800640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07a4a0 0x7f906c07c960 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.255+0000 7f9091800640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c145750 0x7f908c1f2900 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.255+0000 7f9091800640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0cccd0 0x7f908c0ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.255+0000 7f9091800640 1 --2- 172.21.15.71:0/3172843754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c06efa0 0x7f908c13d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.255+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 >> 172.21.15.71:0/3172843754 conn(0x7f908c0b1840 msgr2=0x7f908c0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.255+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 shutdown_connections 2024-09-16T07:35:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.255+0000 7f9091800640 1 -- 172.21.15.71:0/3172843754 wait complete. 2024-09-16T07:35:35.278 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone60941 2024-09-16T07:35:35.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.533+0000 7f2ee3800640 1 -- 172.21.15.71:0/3983491514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec0003aa0 msgr2=0x7f2ec0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:35.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.533+0000 7f2ee3800640 1 --2- 172.21.15.71:0/3983491514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ec0003aa0 0x7f2ec0003ec0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009f90 tx=0x7f2ec0005830 comp rx=0 tx=0).stop 2024-09-16T07:35:35.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.533+0000 7f2ee1800640 1 -- 172.21.15.71:0/3983491514 reap_dead start 2024-09-16T07:35:35.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.533+0000 7f2ee3800640 1 -- 172.21.15.71:0/3983491514 shutdown_connections 2024-09-16T07:35:35.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.533+0000 7f2ee3800640 1 -- 172.21.15.71:0/3983491514 >> 172.21.15.71:0/3983491514 conn(0x7f2edc0b1840 msgr2=0x7f2edc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:35.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.533+0000 7f2ee3800640 1 -- 172.21.15.71:0/3983491514 shutdown_connections 2024-09-16T07:35:35.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.533+0000 7f2ee3800640 1 -- 172.21.15.71:0/3983491514 wait complete. 2024-09-16T07:35:35.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.535+0000 7f2ee3800640 1 Processor -- start 2024-09-16T07:35:35.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.535+0000 7f2ee3800640 1 -- start start 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee3800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0003aa0 0x7f2edc0737d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee3800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc073d10 0x7f2edc0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee3800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2edc0cd070 0x7f2edc1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee3800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc0cf900 con 0x7f2edc0cd070 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee3800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc0cf780 con 0x7f2ec0003aa0 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee3800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc15eb00 con 0x7f2edc073d10 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2edc0cd070 0x7f2edc1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2edc0cd070 0x7f2edc1f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33228/0 (socket says 172.21.15.71:33228) 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee1800640 1 -- 172.21.15.71:0/1878467150 learned_addr learned my addr 172.21.15.71:0/1878467150 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2edbe00640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc073d10 0x7f2edc0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:35.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee0e00640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0003aa0 0x7f2edc0737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee1800640 1 -- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc073d10 msgr2=0x7f2edc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee1800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc073d10 0x7f2edc0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee1800640 1 -- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0003aa0 msgr2=0x7f2edc0737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee1800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0003aa0 0x7f2edc0737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.537+0000 7f2ee1800640 1 -- 172.21.15.71:0/1878467150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc1f8180 con 0x7f2edc0cd070 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.538+0000 7f2ee1800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2edc0cd070 0x7f2edc1f4c60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0046d0 tx=0x7f2ecc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.538+0000 7f2ed8c00640 1 -- 172.21.15.71:0/1878467150 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc017070 con 0x7f2edc0cd070 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.538+0000 7f2ed8c00640 1 -- 172.21.15.71:0/1878467150 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ecc005c90 con 0x7f2edc0cd070 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.538+0000 7f2ed8c00640 1 -- 172.21.15.71:0/1878467150 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc007330 con 0x7f2edc0cd070 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.538+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc1f8460 con 0x7f2edc0cd070 2024-09-16T07:35:35.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.538+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc1f8a20 con 0x7f2edc0cd070 2024-09-16T07:35:35.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.539+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc15ef90 con 0x7f2edc0cd070 2024-09-16T07:35:35.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.541+0000 7f2ed8c00640 1 -- 172.21.15.71:0/1878467150 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7f2ecc0074d0 con 0x7f2edc0cd070 2024-09-16T07:35:35.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.542+0000 7f2ed8c00640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a620 0x7f2ebc07cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:35.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.542+0000 7f2ed8c00640 1 -- 172.21.15.71:0/1878467150 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7f2ecc098920 con 0x7f2edc0cd070 2024-09-16T07:35:35.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.547+0000 7f2ee0e00640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a620 0x7f2ebc07cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:35.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.550+0000 7f2ed8c00640 1 -- 172.21.15.71:0/1878467150 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ecc062440 con 0x7f2edc0cd070 2024-09-16T07:35:35.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.550+0000 7f2ee0e00640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a620 0x7f2ebc07cae0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4034000 tx=0x7f2ed4004bc0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:35.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:35.868+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone60941", "target": ["mon-mgr", ""]}) -- 0x7f2edc157070 con 0x7f2ebc07a620 2024-09-16T07:35:36.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.410+0000 7f2ed8c00640 1 -- 172.21.15.71:0/1878467150 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2edc157070 con 0x7f2ebc07a620 2024-09-16T07:35:36.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a620 msgr2=0x7f2ebc07cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:36.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a620 0x7f2ebc07cae0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4034000 tx=0x7f2ed4004bc0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2edc0cd070 msgr2=0x7f2edc1f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2edc0cd070 0x7f2edc1f4c60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0046d0 tx=0x7f2ecc00c8e0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 shutdown_connections 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a620 0x7f2ebc07cae0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2edc0cd070 0x7f2edc1f4c60 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc073d10 0x7f2edc0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 --2- 172.21.15.71:0/1878467150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0003aa0 0x7f2edc0737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 >> 172.21.15.71:0/1878467150 conn(0x7f2edc0b1840 msgr2=0x7f2edc140de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 shutdown_connections 2024-09-16T07:35:36.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.414+0000 7f2ee3800640 1 -- 172.21.15.71:0/1878467150 wait complete. 2024-09-16T07:35:36.450 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone87142 2024-09-16T07:35:36.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.692+0000 7fb09c000640 1 -- 172.21.15.71:0/2121110436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0940ccb00 msgr2=0x7fb09415f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:36.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.692+0000 7fb09c000640 1 --2- 172.21.15.71:0/2121110436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0940ccb00 0x7fb09415f9f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb0940d0cd0 tx=0x7fb08c02a980 comp rx=0 tx=0).stop 2024-09-16T07:35:36.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.693+0000 7fb09c000640 1 -- 172.21.15.71:0/2121110436 shutdown_connections 2024-09-16T07:35:36.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.693+0000 7fb09c000640 1 --2- 172.21.15.71:0/2121110436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0940ccb00 0x7fb09415f9f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.693+0000 7fb09c000640 1 --2- 172.21.15.71:0/2121110436 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb09406efa0 0x7fb0940cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.693+0000 7fb09c000640 1 --2- 172.21.15.71:0/2121110436 >> v2:172.21.15.177:3300/0 conn(0x7fb094145750 0x7fb094073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.693+0000 7fb09c000640 1 -- 172.21.15.71:0/2121110436 >> 172.21.15.71:0/2121110436 conn(0x7fb0940b1840 msgr2=0x7fb0940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:36.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.693+0000 7fb09c000640 1 -- 172.21.15.71:0/2121110436 shutdown_connections 2024-09-16T07:35:36.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.693+0000 7fb09c000640 1 -- 172.21.15.71:0/2121110436 wait complete. 2024-09-16T07:35:36.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.694+0000 7fb09c000640 1 Processor -- start 2024-09-16T07:35:36.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.694+0000 7fb09c000640 1 -- start start 2024-09-16T07:35:36.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.695+0000 7fb09c000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb09406efa0 0x7fb0940c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.695+0000 7fb09c000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0940ccb00 0x7fb0940ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.695+0000 7fb09c000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb094145750 0x7fb0941411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.695+0000 7fb09c000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941600b0 con 0x7fb094145750 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.695+0000 7fb09c000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb09415ff30 con 0x7fb09406efa0 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.695+0000 7fb09c000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb094160230 con 0x7fb0940ccb00 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb094145750 0x7fb0941411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb099600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb09406efa0 0x7fb0940c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb094145750 0x7fb0941411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33242/0 (socket says 172.21.15.71:33242) 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 -- 172.21.15.71:0/1775008835 learned_addr learned my addr 172.21.15.71:0/1775008835 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb098c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0940ccb00 0x7fb0940ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:36.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 -- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0940ccb00 msgr2=0x7fb0940ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:36.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0940ccb00 0x7fb0940ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 -- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb09406efa0 msgr2=0x7fb0940c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:36.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb09406efa0 0x7fb0940c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:36.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 -- 172.21.15.71:0/1775008835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0940c4d00 con 0x7fb094145750 2024-09-16T07:35:36.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.696+0000 7fb09a000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb094145750 0x7fb0941411c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009d60 tx=0x7fb08c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:36.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.697+0000 7fb091600640 1 -- 172.21.15.71:0/1775008835 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c034050 con 0x7fb094145750 2024-09-16T07:35:36.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.697+0000 7fb091600640 1 -- 172.21.15.71:0/1775008835 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb08c004450 con 0x7fb094145750 2024-09-16T07:35:36.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.697+0000 7fb091600640 1 -- 172.21.15.71:0/1775008835 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c043440 con 0x7fb094145750 2024-09-16T07:35:36.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.697+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0940c4fe0 con 0x7fb094145750 2024-09-16T07:35:36.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.697+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0940c55a0 con 0x7fb094145750 2024-09-16T07:35:36.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.698+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0940c57a0 con 0x7fb094145750 2024-09-16T07:35:36.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.699+0000 7fb091600640 1 -- 172.21.15.71:0/1775008835 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7fb08c037020 con 0x7fb094145750 2024-09-16T07:35:36.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.700+0000 7fb091600640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb07407a620 0x7fb07407cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.700+0000 7fb091600640 1 -- 172.21.15.71:0/1775008835 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7fb08c0baa40 con 0x7fb094145750 2024-09-16T07:35:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.700+0000 7fb099600640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb07407a620 0x7fb07407cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.701+0000 7fb099600640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb07407a620 0x7fb07407cae0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb0940ae1d0 tx=0x7fb088005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:36.704+0000 7fb091600640 1 -- 172.21.15.71:0/1775008835 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb0940c57a0 con 0x7fb094145750 2024-09-16T07:35:37.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.033+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone87142", "target": ["mon-mgr", ""]}) -- 0x7fb0940cc620 con 0x7fb07407a620 2024-09-16T07:35:37.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.197+0000 7fb091600640 1 -- 172.21.15.71:0/1775008835 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb0940cc620 con 0x7fb07407a620 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb07407a620 msgr2=0x7fb07407cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb07407a620 0x7fb07407cae0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb0940ae1d0 tx=0x7fb088005f90 comp rx=0 tx=0).stop 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb094145750 msgr2=0x7fb0941411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb094145750 0x7fb0941411c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009d60 tx=0x7fb08c003f10 comp rx=0 tx=0).stop 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 shutdown_connections 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb07407a620 0x7fb07407cae0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb094145750 0x7fb0941411c0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0940ccb00 0x7fb0940ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 --2- 172.21.15.71:0/1775008835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb09406efa0 0x7fb0940c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:37.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 >> 172.21.15.71:0/1775008835 conn(0x7fb0940b1840 msgr2=0x7fb0940af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:37.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 shutdown_connections 2024-09-16T07:35:37.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.199+0000 7fb09c000640 1 -- 172.21.15.71:0/1775008835 wait complete. 2024-09-16T07:35:37.215 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33330 2024-09-16T07:35:37.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.377+0000 7fc998e00640 1 -- 172.21.15.71:0/483316670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9940cec60 msgr2=0x7fc9940cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:37.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.377+0000 7fc998e00640 1 --2- 172.21.15.71:0/483316670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9940cec60 0x7fc9940cf070 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009830 tx=0x7fc98c02a5b0 comp rx=0 tx=0).stop 2024-09-16T07:35:37.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.377+0000 7fc998e00640 1 -- 172.21.15.71:0/483316670 shutdown_connections 2024-09-16T07:35:37.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.377+0000 7fc998e00640 1 --2- 172.21.15.71:0/483316670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9940cec60 0x7fc9940cf070 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:37.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.377+0000 7fc998e00640 1 -- 172.21.15.71:0/483316670 >> 172.21.15.71:0/483316670 conn(0x7fc9940b1840 msgr2=0x7fc9940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:37.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.377+0000 7fc998e00640 1 -- 172.21.15.71:0/483316670 shutdown_connections 2024-09-16T07:35:37.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.377+0000 7fc998e00640 1 -- 172.21.15.71:0/483316670 wait complete. 2024-09-16T07:35:37.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.379+0000 7fc998e00640 1 Processor -- start 2024-09-16T07:35:37.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.379+0000 7fc998e00640 1 -- start start 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc998e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9940af190 0x7fc994160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc998e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9940cec60 0x7fc994072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc998e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc994161760 0x7fc9941510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc998e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941623a0 con 0x7fc9940af190 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc998e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc994070530 con 0x7fc9940cec60 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc998e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc994162520 con 0x7fc994161760 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc991e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9940af190 0x7fc994160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc991e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9940af190 0x7fc994160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33252/0 (socket says 172.21.15.71:33252) 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc991e00640 1 -- 172.21.15.71:0/2962569536 learned_addr learned my addr 172.21.15.71:0/2962569536 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:37.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc992800640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc994161760 0x7fc9941510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:37.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.380+0000 7fc991400640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9940cec60 0x7fc994072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:37.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc991e00640 1 -- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc994161760 msgr2=0x7fc9941510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:37.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc991e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc994161760 0x7fc9941510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:37.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc991e00640 1 -- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9940cec60 msgr2=0x7fc994072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:37.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc991e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9940cec60 0x7fc994072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:37.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc991e00640 1 -- 172.21.15.71:0/2962569536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9941f4680 con 0x7fc9940af190 2024-09-16T07:35:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc991e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9940af190 0x7fc994160910 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc98400caf0 tx=0x7fc98400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc98a000640 1 -- 172.21.15.71:0/2962569536 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984016020 con 0x7fc9940af190 2024-09-16T07:35:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc98a000640 1 -- 172.21.15.71:0/2962569536 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc98400ad90 con 0x7fc9940af190 2024-09-16T07:35:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9941f4960 con 0x7fc9940af190 2024-09-16T07:35:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc98a000640 1 -- 172.21.15.71:0/2962569536 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984012680 con 0x7fc9940af190 2024-09-16T07:35:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.381+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941f4f20 con 0x7fc9940af190 2024-09-16T07:35:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.382+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9941f5360 con 0x7fc9940af190 2024-09-16T07:35:37.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.383+0000 7fc98a000640 1 -- 172.21.15.71:0/2962569536 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7fc984019020 con 0x7fc9940af190 2024-09-16T07:35:37.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.384+0000 7fc98a000640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc97407a620 0x7fc97407cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:37.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.384+0000 7fc98a000640 1 -- 172.21.15.71:0/2962569536 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7fc984099c10 con 0x7fc9940af190 2024-09-16T07:35:37.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.385+0000 7fc991400640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc97407a620 0x7fc97407cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.386+0000 7fc991400640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc97407a620 0x7fc97407cae0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc980009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:37.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.388+0000 7fc98a000640 1 -- 172.21.15.71:0/2962569536 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc984020030 con 0x7fc9940af190 2024-09-16T07:35:37.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:37.608+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33330", "target": ["mon-mgr", ""]}) -- 0x7fc994151ec0 con 0x7fc97407a620 2024-09-16T07:35:38.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.456+0000 7fc98a000640 1 -- 172.21.15.71:0/2962569536 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fc994151ec0 con 0x7fc97407a620 2024-09-16T07:35:38.455 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:35:38.455 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:38.456 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:35:38.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.458+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc97407a620 msgr2=0x7fc97407cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:38.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.458+0000 7fc998e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc97407a620 0x7fc97407cae0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc980009040 comp rx=0 tx=0).stop 2024-09-16T07:35:38.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.458+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9940af190 msgr2=0x7fc994160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:38.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.458+0000 7fc998e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9940af190 0x7fc994160910 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fc98400caf0 tx=0x7fc98400a060 comp rx=0 tx=0).stop 2024-09-16T07:35:38.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.458+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 shutdown_connections 2024-09-16T07:35:38.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.458+0000 7fc998e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc97407a620 0x7fc97407cae0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.459+0000 7fc998e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc994161760 0x7fc9941510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.459+0000 7fc998e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9940cec60 0x7fc994072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.459+0000 7fc998e00640 1 --2- 172.21.15.71:0/2962569536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9940af190 0x7fc994160910 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.459+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 >> 172.21.15.71:0/2962569536 conn(0x7fc9940b1840 msgr2=0x7fc9940cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:38.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.459+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 shutdown_connections 2024-09-16T07:35:38.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.459+0000 7fc998e00640 1 -- 172.21.15.71:0/2962569536 wait complete. 2024-09-16T07:35:38.477 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:35:38.478 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone60941 2024-09-16T07:35:38.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 -- 172.21.15.71:0/3841212737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb018072560 msgr2=0x7fb018073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 --2- 172.21.15.71:0/3841212737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb018072560 0x7fb018073ef0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb0180724f0 tx=0x7fb00402a560 comp rx=0 tx=0).stop 2024-09-16T07:35:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 -- 172.21.15.71:0/3841212737 shutdown_connections 2024-09-16T07:35:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 --2- 172.21.15.71:0/3841212737 >> v2:172.21.15.177:3300/0 conn(0x7fb0180cb4f0 0x7fb0180cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 --2- 172.21.15.71:0/3841212737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb018072560 0x7fb018073ef0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 --2- 172.21.15.71:0/3841212737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb01815bec0 0x7fb01806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 -- 172.21.15.71:0/3841212737 >> 172.21.15.71:0/3841212737 conn(0x7fb0180cf9f0 msgr2=0x7fb0180cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 -- 172.21.15.71:0/3841212737 shutdown_connections 2024-09-16T07:35:38.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.644+0000 7fb017200640 1 -- 172.21.15.71:0/3841212737 wait complete. 2024-09-16T07:35:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.645+0000 7fb017200640 1 Processor -- start 2024-09-16T07:35:38.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.645+0000 7fb017200640 1 -- start start 2024-09-16T07:35:38.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.646+0000 7fb017200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb018072560 0x7fb01814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.646+0000 7fb017200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0180cb4f0 0x7fb0180ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.646+0000 7fb017200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb01815bec0 0x7fb0180ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.646+0000 7fb017200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0180ae850 con 0x7fb0180cb4f0 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.646+0000 7fb017200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0180ae6d0 con 0x7fb01815bec0 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.646+0000 7fb017200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0180ae9d0 con 0x7fb018072560 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0180cb4f0 0x7fb0180ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0180cb4f0 0x7fb0180ac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33270/0 (socket says 172.21.15.71:33270) 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 -- 172.21.15.71:0/1585120267 learned_addr learned my addr 172.21.15.71:0/1585120267 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015e00640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb018072560 0x7fb01814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb016800640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb01815bec0 0x7fb0180ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 -- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb018072560 msgr2=0x7fb01814b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:38.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb018072560 0x7fb01814b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 -- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb01815bec0 msgr2=0x7fb0180ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:38.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb01815bec0 0x7fb0180ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:38.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 -- 172.21.15.71:0/1585120267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01815e130 con 0x7fb0180cb4f0 2024-09-16T07:35:38.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015e00640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb018072560 0x7fb01814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:38.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb016800640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb01815bec0 0x7fb0180ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:38.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.647+0000 7fb015400640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0180cb4f0 0x7fb0180ac240 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb004009850 tx=0x7fb004003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.648+0000 7fb00e000640 1 -- 172.21.15.71:0/1585120267 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00403f070 con 0x7fb0180cb4f0 2024-09-16T07:35:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.648+0000 7fb00e000640 1 -- 172.21.15.71:0/1585120267 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0040047a0 con 0x7fb0180cb4f0 2024-09-16T07:35:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.648+0000 7fb00e000640 1 -- 172.21.15.71:0/1585120267 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004035030 con 0x7fb0180cb4f0 2024-09-16T07:35:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.648+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01815e410 con 0x7fb0180cb4f0 2024-09-16T07:35:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.648+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01815e9d0 con 0x7fb0180cb4f0 2024-09-16T07:35:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.649+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0180aee80 con 0x7fb0180cb4f0 2024-09-16T07:35:38.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.650+0000 7fb00e000640 1 -- 172.21.15.71:0/1585120267 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7fb004042070 con 0x7fb0180cb4f0 2024-09-16T07:35:38.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.651+0000 7fb00e000640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faff807a620 0x7faff807cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:38.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.652+0000 7fb00e000640 1 -- 172.21.15.71:0/1585120267 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7fb0040ba730 con 0x7fb0180cb4f0 2024-09-16T07:35:38.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.652+0000 7fb015e00640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faff807a620 0x7faff807cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:38.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.653+0000 7fb015e00640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faff807a620 0x7faff807cae0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb008005fd0 tx=0x7fb00800f360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.656+0000 7fb00e000640 1 -- 172.21.15.71:0/1585120267 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb0040848a0 con 0x7fb0180cb4f0 2024-09-16T07:35:38.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:38.879+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone60941", "target": ["mon-mgr", ""]}) -- 0x7fb01815ed40 con 0x7faff807a620 2024-09-16T07:35:43.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.449+0000 7fb00e000640 1 -- 172.21.15.71:0/1585120267 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fb01815ed40 con 0x7faff807a620 2024-09-16T07:35:43.450 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:35:43.450 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:35:43.450 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:35:43.450 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:35:43.450 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:35:43.450 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:35:43.450 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:35:43.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:43.451 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:35:43.451 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:35:43.451 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:35:43.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:43.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:43.451 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faff807a620 msgr2=0x7faff807cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faff807a620 0x7faff807cae0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb008005fd0 tx=0x7fb00800f360 comp rx=0 tx=0).stop 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0180cb4f0 msgr2=0x7fb0180ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0180cb4f0 0x7fb0180ac240 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb004009850 tx=0x7fb004003910 comp rx=0 tx=0).stop 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 shutdown_connections 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faff807a620 0x7faff807cae0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb01815bec0 0x7fb0180ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0180cb4f0 0x7fb0180ac240 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 --2- 172.21.15.71:0/1585120267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb018072560 0x7fb01814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 >> 172.21.15.71:0/1585120267 conn(0x7fb0180cf9f0 msgr2=0x7fb0180b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:43.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 shutdown_connections 2024-09-16T07:35:43.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.452+0000 7fb017200640 1 -- 172.21.15.71:0/1585120267 wait complete. 2024-09-16T07:35:43.472 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:35:43.472 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87142 2024-09-16T07:35:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.639+0000 7f1aa5600640 1 -- 172.21.15.71:0/2053815882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a84004800 msgr2=0x7f1a84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.639+0000 7f1aa5600640 1 --2- 172.21.15.71:0/2053815882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a84004800 0x7f1a84004c80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1a880016c0 tx=0x7f1a84005830 comp rx=0 tx=0).stop 2024-09-16T07:35:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.639+0000 7f1aa5600640 1 -- 172.21.15.71:0/2053815882 shutdown_connections 2024-09-16T07:35:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.639+0000 7f1aa5600640 1 --2- 172.21.15.71:0/2053815882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a84004800 0x7f1a84004c80 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.639+0000 7f1aa5600640 1 -- 172.21.15.71:0/2053815882 >> 172.21.15.71:0/2053815882 conn(0x7f1aa00cf9f0 msgr2=0x7f1aa00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.639+0000 7f1aa5600640 1 -- 172.21.15.71:0/2053815882 shutdown_connections 2024-09-16T07:35:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.639+0000 7f1aa5600640 1 -- 172.21.15.71:0/2053815882 wait complete. 2024-09-16T07:35:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.640+0000 7f1aa5600640 1 Processor -- start 2024-09-16T07:35:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.640+0000 7f1aa5600640 1 -- start start 2024-09-16T07:35:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.641+0000 7f1aa5600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a84004800 0x7f1aa01f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.641+0000 7f1aa5600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa0158f20 0x7f1aa00c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.641+0000 7f1aa5600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa00a9470 0x7f1aa00ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.641+0000 7f1aa5600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0074070 con 0x7f1aa0158f20 2024-09-16T07:35:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.641+0000 7f1aa5600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0073ef0 con 0x7f1aa00a9470 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.641+0000 7f1aa5600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa00741f0 con 0x7f1a84004800 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa0158f20 0x7f1aa00c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa0158f20 0x7f1aa00c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33284/0 (socket says 172.21.15.71:33284) 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 -- 172.21.15.71:0/2010533891 learned_addr learned my addr 172.21.15.71:0/2010533891 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9fe00640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a84004800 0x7f1aa01f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1aa4c00640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa00a9470 0x7f1aa00ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 -- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a84004800 msgr2=0x7f1aa01f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a84004800 0x7f1aa01f3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 -- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa00a9470 msgr2=0x7f1aa00ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa00a9470 0x7f1aa00ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 -- 172.21.15.71:0/2010533891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa00ca3d0 con 0x7f1aa0158f20 2024-09-16T07:35:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1aa4c00640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa00a9470 0x7f1aa00ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:35:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.642+0000 7f1a9f400640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa0158f20 0x7f1aa00c9b80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1a880017e0 tx=0x7f1a880063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.643+0000 7f1a97e00640 1 -- 172.21.15.71:0/2010533891 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88042070 con 0x7f1aa0158f20 2024-09-16T07:35:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.643+0000 7f1a97e00640 1 -- 172.21.15.71:0/2010533891 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a88039070 con 0x7f1aa0158f20 2024-09-16T07:35:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.643+0000 7f1a97e00640 1 -- 172.21.15.71:0/2010533891 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88003d70 con 0x7f1aa0158f20 2024-09-16T07:35:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.643+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa00ca6b0 con 0x7f1aa0158f20 2024-09-16T07:35:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.643+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa00ca0c0 con 0x7f1aa0158f20 2024-09-16T07:35:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.644+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa00cb660 con 0x7f1aa0158f20 2024-09-16T07:35:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.645+0000 7f1a97e00640 1 -- 172.21.15.71:0/2010533891 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7f1a8803b070 con 0x7f1aa0158f20 2024-09-16T07:35:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.646+0000 7f1a97e00640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a7c07a650 0x7f1a7c07cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.646+0000 7f1a97e00640 1 -- 172.21.15.71:0/2010533891 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7f1a880b8d50 con 0x7f1aa0158f20 2024-09-16T07:35:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.647+0000 7f1a9fe00640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a7c07a650 0x7f1a7c07cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.647+0000 7f1a9fe00640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a7c07a650 0x7f1a7c07cb10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0070960 tx=0x7f1a900067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:43.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.650+0000 7f1a97e00640 1 -- 172.21.15.71:0/2010533891 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1a880c3020 con 0x7f1aa0158f20 2024-09-16T07:35:43.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.869+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87142", "target": ["mon-mgr", ""]}) -- 0x7f1aa00b2b40 con 0x7f1a7c07a650 2024-09-16T07:35:43.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.880+0000 7f1a97e00640 1 -- 172.21.15.71:0/2010533891 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f1aa00b2b40 con 0x7f1a7c07a650 2024-09-16T07:35:43.880 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:35:43.880 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:35:43.880 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:35:43.880 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:35:43.880 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:35:43.880 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:35:43.881 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:35:43.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:43.881 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:35:43.881 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:35:43.881 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:35:43.881 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:43.882 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:43.882 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:35:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a7c07a650 msgr2=0x7f1a7c07cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a7c07a650 0x7f1a7c07cb10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0070960 tx=0x7f1a900067a0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa0158f20 msgr2=0x7f1aa00c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa0158f20 0x7f1aa00c9b80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1a880017e0 tx=0x7f1a880063a0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 shutdown_connections 2024-09-16T07:35:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a7c07a650 0x7f1a7c07cb10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1aa00a9470 0x7f1aa00ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa0158f20 0x7f1aa00c9b80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 --2- 172.21.15.71:0/2010533891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a84004800 0x7f1aa01f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:43.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 >> 172.21.15.71:0/2010533891 conn(0x7f1aa00cf9f0 msgr2=0x7f1aa015ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:43.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 shutdown_connections 2024-09-16T07:35:43.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:43.882+0000 7f1aa5600640 1 -- 172.21.15.71:0/2010533891 wait complete. 2024-09-16T07:35:43.898 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:35:43.898 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:35:44.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.053+0000 7f0583200640 1 -- 172.21.15.71:0/2542587136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c15cff0 msgr2=0x7f057c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.053+0000 7f0583200640 1 --2- 172.21.15.71:0/2542587136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c15cff0 0x7f057c06e6f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f057c0b2680 tx=0x7f056802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.053+0000 7f0583200640 1 -- 172.21.15.71:0/2542587136 shutdown_connections 2024-09-16T07:35:44.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.053+0000 7f0583200640 1 --2- 172.21.15.71:0/2542587136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c15cff0 0x7f057c06e6f0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.053+0000 7f0583200640 1 --2- 172.21.15.71:0/2542587136 >> v2:172.21.15.177:3300/0 conn(0x7f057c159aa0 0x7f057c0741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.053+0000 7f0583200640 1 -- 172.21.15.71:0/2542587136 >> 172.21.15.71:0/2542587136 conn(0x7f057c0b18e0 msgr2=0x7f057c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:44.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.054+0000 7f0583200640 1 -- 172.21.15.71:0/2542587136 shutdown_connections 2024-09-16T07:35:44.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.054+0000 7f0583200640 1 -- 172.21.15.71:0/2542587136 wait complete. 2024-09-16T07:35:44.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.055+0000 7f0583200640 1 Processor -- start 2024-09-16T07:35:44.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.055+0000 7f0583200640 1 -- start start 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f0583200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057c06f220 0x7f057c14b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f0583200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c159aa0 0x7f057c09d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f0583200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057c15cff0 0x7f057c09d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f0583200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f057c0ccc30 con 0x7f057c159aa0 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f0583200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f057c0ccab0 con 0x7f057c06f220 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f0583200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f057c0ccdb0 con 0x7f057c15cff0 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f057b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c159aa0 0x7f057c09d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f057b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c159aa0 0x7f057c09d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33300/0 (socket says 172.21.15.71:33300) 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f057b400640 1 -- 172.21.15.71:0/3950104620 learned_addr learned my addr 172.21.15.71:0/3950104620 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f057be00640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057c06f220 0x7f057c14b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.056+0000 7f0581200640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057c15cff0 0x7f057c09d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f057b400640 1 -- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057c15cff0 msgr2=0x7f057c09d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f057b400640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057c15cff0 0x7f057c09d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f057b400640 1 -- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057c06f220 msgr2=0x7f057c14b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f057b400640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057c06f220 0x7f057c14b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f057b400640 1 -- 172.21.15.71:0/3950104620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c154e30 con 0x7f057c159aa0 2024-09-16T07:35:44.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f057be00640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057c06f220 0x7f057c14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:44.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f0581200640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057c15cff0 0x7f057c09d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:44.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f057b400640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c159aa0 0x7f057c09d120 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0568034040 tx=0x7f0568003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:44.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f0573e00640 1 -- 172.21.15.71:0/3950104620 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056803e030 con 0x7f057c159aa0 2024-09-16T07:35:44.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f0573e00640 1 -- 172.21.15.71:0/3950104620 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0568003d10 con 0x7f057c159aa0 2024-09-16T07:35:44.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f0573e00640 1 -- 172.21.15.71:0/3950104620 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568035040 con 0x7f057c159aa0 2024-09-16T07:35:44.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057c158020 con 0x7f057c159aa0 2024-09-16T07:35:44.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.057+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c1585e0 con 0x7f057c159aa0 2024-09-16T07:35:44.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.058+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057c069360 con 0x7f057c159aa0 2024-09-16T07:35:44.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.060+0000 7f0573e00640 1 -- 172.21.15.71:0/3950104620 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7f0568041030 con 0x7f057c159aa0 2024-09-16T07:35:44.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.060+0000 7f0573e00640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c07a620 0x7f055c07cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.061+0000 7f0573e00640 1 -- 172.21.15.71:0/3950104620 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7f05680b9ef0 con 0x7f057c159aa0 2024-09-16T07:35:44.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.061+0000 7f057be00640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c07a620 0x7f055c07cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.062+0000 7f057be00640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c07a620 0x7f055c07cae0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f056c005fd0 tx=0x7f056c009500 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:44.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.064+0000 7f0573e00640 1 -- 172.21.15.71:0/3950104620 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f057c154e30 con 0x7f057c159aa0 2024-09-16T07:35:44.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.316+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f057c070980 con 0x7f057c159aa0 2024-09-16T07:35:44.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.316+0000 7f0573e00640 1 -- 172.21.15.71:0/3950104620 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2611 (secure 0 0 0) 0x7f0568084060 con 0x7f057c159aa0 2024-09-16T07:35:44.317 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:44.317 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:35:44.317 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:35:44.317 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:35:44.317 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:35:44.317 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:35:44.318 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:35:44.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:44.318 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:35:44.318 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:35:44.318 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:35:44.318 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:35:44.319 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:35:44.319 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:35:44.319 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:35:44.319 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:35:44.319 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:35:44.319 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:35:44.319 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:35:44.320 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 801, 2024-09-16T07:35:44.320 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:35:44.320 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:35:44.320 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:35:44.320 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:35:44.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:44.320 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:35:44.321 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 121, 2024-09-16T07:35:44.321 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:35:44.321 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:35:44.321 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:35:44.321 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:35:44.321 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 1586, 2024-09-16T07:35:44.322 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 6510830348, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 14104633344, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 758989479936, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_objects_per_sec": 2, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_bytes_per_sec": 11488622, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_keys_per_sec": 0, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects_recovered": 29, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "num_bytes_recovered": 113246400, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "num_keys_recovered": 0, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 337853141, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 331507358, 2024-09-16T07:35:44.323 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 83, 2024-09-16T07:35:44.324 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 94 2024-09-16T07:35:44.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:44.324 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:35:44.324 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 76, 2024-09-16T07:35:44.324 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:35:01:329397+0000", 2024-09-16T07:35:44.324 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 8, 2024-09-16T07:35:44.324 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:35:44.325 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:35:44.325 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:35:44.325 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:35:44.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:44.325 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 8, 2024-09-16T07:35:44.325 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:35:44.326 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:35:44.326 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:35:44.326 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 8382 2024-09-16T07:35:44.326 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:44.326 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:35:44.326 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:35:44.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:44.327 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:35:44.327 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:35:44.327 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:35:44.327 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:35:44.327 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:35:44.327 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:35:44.327 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:35:44.327 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 19, 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:34:30.599503+0000", 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "services": { 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "daemons": { 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "summary": "", 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "a": { 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:35:44.328 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": {} 2024-09-16T07:35:44.329 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c07a620 msgr2=0x7f055c07cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c07a620 0x7f055c07cae0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f056c005fd0 tx=0x7f056c009500 comp rx=0 tx=0).stop 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c159aa0 msgr2=0x7f057c09d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c159aa0 0x7f057c09d120 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f0568034040 tx=0x7f0568003820 comp rx=0 tx=0).stop 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 shutdown_connections 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f055c07a620 0x7f055c07cae0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057c15cff0 0x7f057c09d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f057c159aa0 0x7f057c09d120 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.318+0000 7f0583200640 1 --2- 172.21.15.71:0/3950104620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057c06f220 0x7f057c14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.319+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 >> 172.21.15.71:0/3950104620 conn(0x7f057c0b18e0 msgr2=0x7f057c0b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.319+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 shutdown_connections 2024-09-16T07:35:44.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.319+0000 7f0583200640 1 -- 172.21.15.71:0/3950104620 wait complete. 2024-09-16T07:35:44.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:44.335 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone33330/479939c2-5096-4e1c-9486-9b76a3a091ea 2024-09-16T07:35:44.335 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:35:44.373 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T07:35:44.373 INFO:teuthology.orchestra.run.smithi071.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone33330/479939c2-5096-4e1c-9486-9b76a3a091ea': No such file or directory 2024-09-16T07:35:44.374 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:35:44.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.575+0000 7fc22e800640 1 -- 172.21.15.71:0/3815715247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2280d0540 msgr2=0x7fc22806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.575+0000 7fc22e800640 1 --2- 172.21.15.71:0/3815715247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2280d0540 0x7fc22806f320 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc220005fd0 tx=0x7fc22002a8b0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.575+0000 7fc22e800640 1 -- 172.21.15.71:0/3815715247 shutdown_connections 2024-09-16T07:35:44.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.575+0000 7fc22e800640 1 --2- 172.21.15.71:0/3815715247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2280d0540 0x7fc22806f320 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.575+0000 7fc22e800640 1 --2- 172.21.15.71:0/3815715247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22815bc30 0x7fc2280b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.575+0000 7fc22e800640 1 -- 172.21.15.71:0/3815715247 >> 172.21.15.71:0/3815715247 conn(0x7fc2280cfa20 msgr2=0x7fc2280cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:44.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.576+0000 7fc22e800640 1 -- 172.21.15.71:0/3815715247 shutdown_connections 2024-09-16T07:35:44.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.576+0000 7fc22e800640 1 -- 172.21.15.71:0/3815715247 wait complete. 2024-09-16T07:35:44.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.577+0000 7fc22e800640 1 Processor -- start 2024-09-16T07:35:44.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.577+0000 7fc22e800640 1 -- start start 2024-09-16T07:35:44.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22e800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2280d0540 0x7fc2280cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc22815bc30 0x7fc2280c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2280c8080 0x7fc228150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22e800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc228074660 con 0x7fc2280c8080 2024-09-16T07:35:44.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22e800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815c200 con 0x7fc22815bc30 2024-09-16T07:35:44.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22e800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc2281f7b80 con 0x7fc2280d0540 2024-09-16T07:35:44.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2280c8080 0x7fc228150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2280c8080 0x7fc228150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33308/0 (socket says 172.21.15.71:33308) 2024-09-16T07:35:44.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.578+0000 7fc22de00640 1 -- 172.21.15.71:0/697284800 learned_addr learned my addr 172.21.15.71:0/697284800 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:44.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22d400640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2280d0540 0x7fc2280cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22ca00640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc22815bc30 0x7fc2280c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22de00640 1 -- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2280d0540 msgr2=0x7fc2280cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22de00640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2280d0540 0x7fc2280cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22de00640 1 -- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc22815bc30 msgr2=0x7fc2280c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22de00640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc22815bc30 0x7fc2280c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22de00640 1 -- 172.21.15.71:0/697284800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2280c8800 con 0x7fc2280c8080 2024-09-16T07:35:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22d400640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2280d0540 0x7fc2280cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:35:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.579+0000 7fc22de00640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2280c8080 0x7fc228150b30 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc220009b30 tx=0x7fc2200316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:44.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.580+0000 7fc225600640 1 -- 172.21.15.71:0/697284800 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220040070 con 0x7fc2280c8080 2024-09-16T07:35:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.580+0000 7fc225600640 1 -- 172.21.15.71:0/697284800 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc220004040 con 0x7fc2280c8080 2024-09-16T07:35:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.580+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281f7dd0 con 0x7fc2280c8080 2024-09-16T07:35:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.580+0000 7fc225600640 1 -- 172.21.15.71:0/697284800 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220004490 con 0x7fc2280c8080 2024-09-16T07:35:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.580+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281f8390 con 0x7fc2280c8080 2024-09-16T07:35:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.581+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22806fc40 con 0x7fc2280c8080 2024-09-16T07:35:44.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.582+0000 7fc225600640 1 -- 172.21.15.71:0/697284800 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7fc220039040 con 0x7fc2280c8080 2024-09-16T07:35:44.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.583+0000 7fc225600640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc20807a620 0x7fc20807cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.583+0000 7fc225600640 1 -- 172.21.15.71:0/697284800 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7fc2200bae10 con 0x7fc2280c8080 2024-09-16T07:35:44.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.583+0000 7fc22d400640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc20807a620 0x7fc20807cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.584+0000 7fc22d400640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc20807a620 0x7fc20807cae0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005fd0 tx=0x7fc21c009260 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:44.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.587+0000 7fc225600640 1 -- 172.21.15.71:0/697284800 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc220084f00 con 0x7fc2280c8080 2024-09-16T07:35:44.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.793+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc22806ebe0 con 0x7fc20807a620 2024-09-16T07:35:44.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.796+0000 7fc225600640 1 -- 172.21.15.71:0/697284800 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc22806ebe0 con 0x7fc20807a620 2024-09-16T07:35:44.796 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:35:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:35:44.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:44.796 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:35:44.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc20807a620 msgr2=0x7fc20807cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc20807a620 0x7fc20807cae0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005fd0 tx=0x7fc21c009260 comp rx=0 tx=0).stop 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2280c8080 msgr2=0x7fc228150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2280c8080 0x7fc228150b30 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc220009b30 tx=0x7fc2200316e0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 shutdown_connections 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc20807a620 0x7fc20807cae0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2280c8080 0x7fc228150b30 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc22815bc30 0x7fc2280c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 --2- 172.21.15.71:0/697284800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2280d0540 0x7fc2280cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 >> 172.21.15.71:0/697284800 conn(0x7fc2280cfa20 msgr2=0x7fc2280b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 shutdown_connections 2024-09-16T07:35:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.798+0000 7fc22e800640 1 -- 172.21.15.71:0/697284800 wait complete. 2024-09-16T07:35:44.815 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:35:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.972+0000 7ffa27400640 1 -- 172.21.15.71:0/2268292683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 msgr2=0x7ffa20073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.972+0000 7ffa27400640 1 --2- 172.21.15.71:0/2268292683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 0x7ffa20073440 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ffa200cfba0 tx=0x7ffa0c02a560 comp rx=0 tx=0).stop 2024-09-16T07:35:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.972+0000 7ffa27400640 1 -- 172.21.15.71:0/2268292683 shutdown_connections 2024-09-16T07:35:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.972+0000 7ffa27400640 1 --2- 172.21.15.71:0/2268292683 >> v2:172.21.15.177:3300/0 conn(0x7ffa2006efa0 0x7ffa200cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.972+0000 7ffa27400640 1 --2- 172.21.15.71:0/2268292683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 0x7ffa20073440 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.972+0000 7ffa27400640 1 -- 172.21.15.71:0/2268292683 >> 172.21.15.71:0/2268292683 conn(0x7ffa200b1840 msgr2=0x7ffa200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.972+0000 7ffa27400640 1 -- 172.21.15.71:0/2268292683 shutdown_connections 2024-09-16T07:35:44.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.972+0000 7ffa27400640 1 -- 172.21.15.71:0/2268292683 wait complete. 2024-09-16T07:35:44.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.974+0000 7ffa27400640 1 Processor -- start 2024-09-16T07:35:44.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.974+0000 7ffa27400640 1 -- start start 2024-09-16T07:35:44.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa27400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa2006efa0 0x7ffa201f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa27400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 0x7ffa20145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa27400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa20145750 0x7ffa200cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa27400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffa200ce480 con 0x7ffa200cccd0 2024-09-16T07:35:44.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa27400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffa200ce300 con 0x7ffa20145750 2024-09-16T07:35:44.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa27400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffa200ce600 con 0x7ffa2006efa0 2024-09-16T07:35:44.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa1fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 0x7ffa20145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa1fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 0x7ffa20145ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33128/0 (socket says 172.21.15.71:33128) 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa1fe00640 1 -- 172.21.15.71:0/78329316 learned_addr learned my addr 172.21.15.71:0/78329316 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa24a00640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa2006efa0 0x7ffa201f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa25400640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa20145750 0x7ffa200cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa1fe00640 1 -- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa2006efa0 msgr2=0x7ffa201f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa1fe00640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa2006efa0 0x7ffa201f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa1fe00640 1 -- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa20145750 msgr2=0x7ffa200cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa1fe00640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa20145750 0x7ffa200cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.975+0000 7ffa1fe00640 1 -- 172.21.15.71:0/78329316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa20146220 con 0x7ffa200cccd0 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.976+0000 7ffa1fe00640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 0x7ffa20145ce0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c02ab50 tx=0x7ffa0c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.976+0000 7ffa1cc00640 1 -- 172.21.15.71:0/78329316 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c03f070 con 0x7ffa200cccd0 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.976+0000 7ffa1cc00640 1 -- 172.21.15.71:0/78329316 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa0c004890 con 0x7ffa200cccd0 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.976+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa201464f0 con 0x7ffa200cccd0 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.976+0000 7ffa1cc00640 1 -- 172.21.15.71:0/78329316 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c035030 con 0x7ffa200cccd0 2024-09-16T07:35:44.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.976+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201f4ac0 con 0x7ffa200cccd0 2024-09-16T07:35:44.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.977+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa200cdbb0 con 0x7ffa200cccd0 2024-09-16T07:35:44.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.978+0000 7ffa1cc00640 1 -- 172.21.15.71:0/78329316 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7ffa0c042070 con 0x7ffa200cccd0 2024-09-16T07:35:44.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.979+0000 7ffa1cc00640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffa0007a620 0x7ffa0007cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:44.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.979+0000 7ffa1cc00640 1 -- 172.21.15.71:0/78329316 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7ffa0c0ba760 con 0x7ffa200cccd0 2024-09-16T07:35:44.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.980+0000 7ffa24a00640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffa0007a620 0x7ffa0007cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:44.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.981+0000 7ffa24a00640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffa0007a620 0x7ffa0007cae0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007fd0 tx=0x7ffa10009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:44.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:44.984+0000 7ffa1cc00640 1 -- 172.21.15.71:0/78329316 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffa0c084830 con 0x7ffa200cccd0 2024-09-16T07:35:45.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.193+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa20144de0 con 0x7ffa0007a620 2024-09-16T07:35:45.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.201+0000 7ffa1cc00640 1 -- 172.21.15.71:0/78329316 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+185 (secure 0 0 0) 0x7ffa20144de0 con 0x7ffa0007a620 2024-09-16T07:35:45.201 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:45.201 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:35:45.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:45.201 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sv1" 2024-09-16T07:35:45.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:45.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone33330" 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone60941" 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone87142" 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:45.202 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:35:45.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffa0007a620 msgr2=0x7ffa0007cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffa0007a620 0x7ffa0007cae0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007fd0 tx=0x7ffa10009040 comp rx=0 tx=0).stop 2024-09-16T07:35:45.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 msgr2=0x7ffa20145ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 0x7ffa20145ce0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c02ab50 tx=0x7ffa0c0039e0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 shutdown_connections 2024-09-16T07:35:45.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffa0007a620 0x7ffa0007cae0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa20145750 0x7ffa200cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa200cccd0 0x7ffa20145ce0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 --2- 172.21.15.71:0/78329316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa2006efa0 0x7ffa201f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 >> 172.21.15.71:0/78329316 conn(0x7ffa200b1840 msgr2=0x7ffa200aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:45.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 shutdown_connections 2024-09-16T07:35:45.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.203+0000 7ffa27400640 1 -- 172.21.15.71:0/78329316 wait complete. 2024-09-16T07:35:45.220 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T07:35:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.381+0000 7ff092a00640 1 -- 172.21.15.71:0/4169835323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08c15b9b0 msgr2=0x7ff08c15ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.381+0000 7ff092a00640 1 --2- 172.21.15.71:0/4169835323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08c15b9b0 0x7ff08c15ee00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff080009830 tx=0x7ff08002a5b0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.381+0000 7ff092a00640 1 -- 172.21.15.71:0/4169835323 shutdown_connections 2024-09-16T07:35:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.381+0000 7ff092a00640 1 --2- 172.21.15.71:0/4169835323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08c0ce7f0 0x7ff08c0ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.381+0000 7ff092a00640 1 --2- 172.21.15.71:0/4169835323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08c15b9b0 0x7ff08c15ee00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.381+0000 7ff092a00640 1 -- 172.21.15.71:0/4169835323 >> 172.21.15.71:0/4169835323 conn(0x7ff08c0cfa70 msgr2=0x7ff08c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.381+0000 7ff092a00640 1 -- 172.21.15.71:0/4169835323 shutdown_connections 2024-09-16T07:35:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.381+0000 7ff092a00640 1 -- 172.21.15.71:0/4169835323 wait complete. 2024-09-16T07:35:45.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.383+0000 7ff092a00640 1 Processor -- start 2024-09-16T07:35:45.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.383+0000 7ff092a00640 1 -- start start 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff092a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08c0ce7f0 0x7ff08c151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff092a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff08c0cf180 0x7ff08c1f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff092a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08c15b9b0 0x7ff08c06e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff092a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c0caa40 con 0x7ff08c0cf180 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff092a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c0ca8c0 con 0x7ff08c0ce7f0 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff092a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c0cabc0 con 0x7ff08c15b9b0 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff090c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff08c0cf180 0x7ff08c1f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff090c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff08c0cf180 0x7ff08c1f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33138/0 (socket says 172.21.15.71:33138) 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff091600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08c0ce7f0 0x7ff08c151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff090c00640 1 -- 172.21.15.71:0/1531637220 learned_addr learned my addr 172.21.15.71:0/1531637220 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.384+0000 7ff092000640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08c15b9b0 0x7ff08c06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff090c00640 1 -- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08c15b9b0 msgr2=0x7ff08c06e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff090c00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08c15b9b0 0x7ff08c06e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff090c00640 1 -- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08c0ce7f0 msgr2=0x7ff08c151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff090c00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08c0ce7f0 0x7ff08c151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff090c00640 1 -- 172.21.15.71:0/1531637220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c1f45d0 con 0x7ff08c0cf180 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff090c00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff08c0cf180 0x7ff08c1f4ad0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff07800e9f0 tx=0x7ff07800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff089600640 1 -- 172.21.15.71:0/1531637220 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07800cd80 con 0x7ff08c0cf180 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff089600640 1 -- 172.21.15.71:0/1531637220 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff078010430 con 0x7ff08c0cf180 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff089600640 1 -- 172.21.15.71:0/1531637220 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078010840 con 0x7ff08c0cf180 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c15a2f0 con 0x7ff08c0cf180 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.385+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1f87a0 con 0x7ff08c0cf180 2024-09-16T07:35:45.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.386+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c0ca380 con 0x7ff08c0cf180 2024-09-16T07:35:45.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.388+0000 7ff089600640 1 -- 172.21.15.71:0/1531637220 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7ff0780109e0 con 0x7ff08c0cf180 2024-09-16T07:35:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.389+0000 7ff089600640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff06c07a650 0x7ff06c07cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.389+0000 7ff089600640 1 -- 172.21.15.71:0/1531637220 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7ff07809a370 con 0x7ff08c0cf180 2024-09-16T07:35:45.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.389+0000 7ff091600640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff06c07a650 0x7ff06c07cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:45.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.390+0000 7ff091600640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff06c07a650 0x7ff06c07cb10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff080034000 tx=0x7ff080004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:45.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.393+0000 7ff089600640 1 -- 172.21.15.71:0/1531637220 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff078063f50 con 0x7ff08c0cf180 2024-09-16T07:35:45.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.614+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff08c0b59f0 con 0x7ff06c07a650 2024-09-16T07:35:45.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.629+0000 7ff089600640 1 -- 172.21.15.71:0/1531637220 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff08c0b59f0 con 0x7ff06c07a650 2024-09-16T07:35:45.628 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:45.629 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:35:45.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:35:45.629 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1" 2024-09-16T07:35:45.629 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:35:45.629 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:35:45.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff06c07a650 msgr2=0x7ff06c07cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff06c07a650 0x7ff06c07cb10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff080034000 tx=0x7ff080004730 comp rx=0 tx=0).stop 2024-09-16T07:35:45.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff08c0cf180 msgr2=0x7ff08c1f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff08c0cf180 0x7ff08c1f4ad0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff07800e9f0 tx=0x7ff07800eeb0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 shutdown_connections 2024-09-16T07:35:45.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff06c07a650 0x7ff06c07cb10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08c15b9b0 0x7ff08c06e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff08c0cf180 0x7ff08c1f4ad0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 --2- 172.21.15.71:0/1531637220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08c0ce7f0 0x7ff08c151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.630+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 >> 172.21.15.71:0/1531637220 conn(0x7ff08c0cfa70 msgr2=0x7ff08c0ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:45.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.631+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 shutdown_connections 2024-09-16T07:35:45.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.631+0000 7ff092a00640 1 -- 172.21.15.71:0/1531637220 wait complete. 2024-09-16T07:35:45.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T07:35:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.809+0000 7f6103600640 1 -- 172.21.15.71:0/807351978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc145750 msgr2=0x7f60fc073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.809+0000 7f6103600640 1 --2- 172.21.15.71:0/807351978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc145750 0x7f60fc073440 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0cfb70 tx=0x7f60f002a4d0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.810+0000 7f6103600640 1 -- 172.21.15.71:0/807351978 shutdown_connections 2024-09-16T07:35:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.810+0000 7f6103600640 1 --2- 172.21.15.71:0/807351978 >> v2:172.21.15.177:3300/0 conn(0x7f60fc0cccd0 0x7f60fc0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.810+0000 7f6103600640 1 --2- 172.21.15.71:0/807351978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc145750 0x7f60fc073440 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.810+0000 7f6103600640 1 -- 172.21.15.71:0/807351978 >> 172.21.15.71:0/807351978 conn(0x7f60fc0b1840 msgr2=0x7f60fc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.810+0000 7f6103600640 1 -- 172.21.15.71:0/807351978 shutdown_connections 2024-09-16T07:35:45.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.810+0000 7f6103600640 1 -- 172.21.15.71:0/807351978 wait complete. 2024-09-16T07:35:45.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.811+0000 7f6103600640 1 Processor -- start 2024-09-16T07:35:45.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.811+0000 7f6103600640 1 -- start start 2024-09-16T07:35:45.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.812+0000 7f6103600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc06efa0 0x7f60fc13d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:45.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.812+0000 7f6103600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60fc0cccd0 0x7f60fc140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:45.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.812+0000 7f6103600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60fc145750 0x7f60fc141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:45.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.812+0000 7f6103600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc0ce480 con 0x7f60fc06efa0 2024-09-16T07:35:45.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.812+0000 7f6103600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc0ce300 con 0x7f60fc145750 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.812+0000 7f6103600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc0ce600 con 0x7f60fc0cccd0 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc06efa0 0x7f60fc13d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc06efa0 0x7f60fc13d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33150/0 (socket says 172.21.15.71:33150) 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 -- 172.21.15.71:0/167226297 learned_addr learned my addr 172.21.15.71:0/167226297 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 -- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60fc0cccd0 msgr2=0x7f60fc140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60fc0cccd0 0x7f60fc140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 -- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60fc145750 msgr2=0x7f60fc141000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60fc145750 0x7f60fc141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:45.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 -- 172.21.15.71:0/167226297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc140080 con 0x7f60fc06efa0 2024-09-16T07:35:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.813+0000 7f6100c00640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc06efa0 0x7f60fc13d760 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f60f0034040 tx=0x7f60f0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.814+0000 7f60f8c00640 1 -- 172.21.15.71:0/167226297 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f003e030 con 0x7f60fc06efa0 2024-09-16T07:35:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.814+0000 7f60f8c00640 1 -- 172.21.15.71:0/167226297 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60f0003e00 con 0x7f60fc06efa0 2024-09-16T07:35:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.814+0000 7f60f8c00640 1 -- 172.21.15.71:0/167226297 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0035040 con 0x7f60fc06efa0 2024-09-16T07:35:45.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.814+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc140360 con 0x7f60fc06efa0 2024-09-16T07:35:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.814+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc0c4e30 con 0x7f60fc06efa0 2024-09-16T07:35:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.815+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc0cdbb0 con 0x7f60fc06efa0 2024-09-16T07:35:45.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.816+0000 7f60f8c00640 1 -- 172.21.15.71:0/167226297 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7f60f0041030 con 0x7f60fc06efa0 2024-09-16T07:35:45.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.817+0000 7f60f8c00640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60dc07a620 0x7f60dc07cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:45.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.817+0000 7f60f8c00640 1 -- 172.21.15.71:0/167226297 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7f60f00b9fc0 con 0x7f60fc06efa0 2024-09-16T07:35:45.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.817+0000 7f60fbe00640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60dc07a620 0x7f60dc07cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:45.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.818+0000 7f60fbe00640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60dc07a620 0x7f60dc07cae0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f60fc1f81a0 tx=0x7f60e80079b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:45.821+0000 7f60f8c00640 1 -- 172.21.15.71:0/167226297 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f60f0084090 con 0x7f60fc06efa0 2024-09-16T07:35:46.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.038+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60fc06e690 con 0x7f60dc07a620 2024-09-16T07:35:46.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.071+0000 7f60f8c00640 1 -- 172.21.15.71:0/167226297 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f60fc06e690 con 0x7f60dc07a620 2024-09-16T07:35:46.071 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:46.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60dc07a620 msgr2=0x7f60dc07cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60dc07a620 0x7f60dc07cae0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f60fc1f81a0 tx=0x7f60e80079b0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc06efa0 msgr2=0x7f60fc13d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc06efa0 0x7f60fc13d760 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f60f0034040 tx=0x7f60f0003910 comp rx=0 tx=0).stop 2024-09-16T07:35:46.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 shutdown_connections 2024-09-16T07:35:46.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60dc07a620 0x7f60dc07cae0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60fc145750 0x7f60fc141000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60fc0cccd0 0x7f60fc140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 --2- 172.21.15.71:0/167226297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc06efa0 0x7f60fc13d760 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 >> 172.21.15.71:0/167226297 conn(0x7f60fc0b1840 msgr2=0x7f60fc0b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:46.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 shutdown_connections 2024-09-16T07:35:46.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.073+0000 7f6103600640 1 -- 172.21.15.71:0/167226297 wait complete. 2024-09-16T07:35:46.090 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T07:35:46.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.250+0000 7f7e14800640 1 -- 172.21.15.71:0/3591600198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c0cccd0 msgr2=0x7f7e0c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.250+0000 7f7e14800640 1 --2- 172.21.15.71:0/3591600198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c0cccd0 0x7f7e0c073440 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005fd0 tx=0x7f7dfc02a540 comp rx=0 tx=0).stop 2024-09-16T07:35:46.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.250+0000 7f7e14800640 1 -- 172.21.15.71:0/3591600198 shutdown_connections 2024-09-16T07:35:46.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.250+0000 7f7e14800640 1 --2- 172.21.15.71:0/3591600198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e0c06efa0 0x7f7e0c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.250+0000 7f7e14800640 1 --2- 172.21.15.71:0/3591600198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c0cccd0 0x7f7e0c073440 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.250+0000 7f7e14800640 1 -- 172.21.15.71:0/3591600198 >> 172.21.15.71:0/3591600198 conn(0x7f7e0c0b1840 msgr2=0x7f7e0c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:46.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.250+0000 7f7e14800640 1 -- 172.21.15.71:0/3591600198 shutdown_connections 2024-09-16T07:35:46.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.250+0000 7f7e14800640 1 -- 172.21.15.71:0/3591600198 wait complete. 2024-09-16T07:35:46.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e14800640 1 Processor -- start 2024-09-16T07:35:46.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e14800640 1 -- start start 2024-09-16T07:35:46.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e14800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e0c06efa0 0x7f7e0c1f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:46.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e14800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e0c0cccd0 0x7f7e0c1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:46.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e14800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c145750 0x7f7e0c15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:46.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e14800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c0ce480 con 0x7f7e0c145750 2024-09-16T07:35:46.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e14800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c0ce300 con 0x7f7e0c06efa0 2024-09-16T07:35:46.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e14800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0c0ce600 con 0x7f7e0c0cccd0 2024-09-16T07:35:46.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e12800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c145750 0x7f7e0c15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:46.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e12800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c145750 0x7f7e0c15eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33162/0 (socket says 172.21.15.71:33162) 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e12800640 1 -- 172.21.15.71:0/3829207828 learned_addr learned my addr 172.21.15.71:0/3829207828 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e11400640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e0c0cccd0 0x7f7e0c1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e11e00640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e0c06efa0 0x7f7e0c1f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e12800640 1 -- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e0c0cccd0 msgr2=0x7f7e0c1f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e12800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e0c0cccd0 0x7f7e0c1f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e12800640 1 -- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e0c06efa0 msgr2=0x7f7e0c1f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.253+0000 7f7e12800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e0c06efa0 0x7f7e0c1f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.254+0000 7f7e12800640 1 -- 172.21.15.71:0/3829207828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c15f020 con 0x7f7e0c145750 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.254+0000 7f7e11e00640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e0c06efa0 0x7f7e0c1f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:35:46.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.254+0000 7f7e12800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c145750 0x7f7e0c15eae0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800eaa0 tx=0x7f7e0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:46.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.254+0000 7f7e06000640 1 -- 172.21.15.71:0/3829207828 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0800cdb0 con 0x7f7e0c145750 2024-09-16T07:35:46.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.254+0000 7f7e06000640 1 -- 172.21.15.71:0/3829207828 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e08004c10 con 0x7f7e0c145750 2024-09-16T07:35:46.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.254+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c1f6110 con 0x7f7e0c145750 2024-09-16T07:35:46.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.254+0000 7f7e06000640 1 -- 172.21.15.71:0/3829207828 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08005020 con 0x7f7e0c145750 2024-09-16T07:35:46.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.254+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c1f66d0 con 0x7f7e0c145750 2024-09-16T07:35:46.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.255+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e0c0aa6f0 con 0x7f7e0c145750 2024-09-16T07:35:46.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.257+0000 7f7e06000640 1 -- 172.21.15.71:0/3829207828 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7f7e080051c0 con 0x7f7e0c145750 2024-09-16T07:35:46.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.257+0000 7f7e06000640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7df007a620 0x7f7df007cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:46.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.259+0000 7f7e06000640 1 -- 172.21.15.71:0/3829207828 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7f7e0809a270 con 0x7f7e0c145750 2024-09-16T07:35:46.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.259+0000 7f7e11e00640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7df007a620 0x7f7df007cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:46.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.260+0000 7f7e11e00640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7df007a620 0x7f7df007cae0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7e00000f60 tx=0x7f7e00009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:46.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.262+0000 7f7e06000640 1 -- 172.21.15.71:0/3829207828 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7e08016030 con 0x7f7e0c145750 2024-09-16T07:35:46.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.473+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f7e0c0cdbb0 con 0x7f7df007a620 2024-09-16T07:35:46.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.497+0000 7f7e06000640 1 -- 172.21.15.71:0/3829207828 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e0c0cdbb0 con 0x7f7df007a620 2024-09-16T07:35:46.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.498+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7df007a620 msgr2=0x7f7df007cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.498+0000 7f7e14800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7df007a620 0x7f7df007cae0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7e00000f60 tx=0x7f7e00009040 comp rx=0 tx=0).stop 2024-09-16T07:35:46.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.498+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c145750 msgr2=0x7f7e0c15eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.498+0000 7f7e14800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c145750 0x7f7e0c15eae0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800eaa0 tx=0x7f7e0800ef60 comp rx=0 tx=0).stop 2024-09-16T07:35:46.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.499+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 shutdown_connections 2024-09-16T07:35:46.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.499+0000 7f7e14800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7df007a620 0x7f7df007cae0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.499+0000 7f7e14800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e0c145750 0x7f7e0c15eae0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.499+0000 7f7e14800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e0c0cccd0 0x7f7e0c1f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.499+0000 7f7e14800640 1 --2- 172.21.15.71:0/3829207828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e0c06efa0 0x7f7e0c1f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.499+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 >> 172.21.15.71:0/3829207828 conn(0x7f7e0c0b1840 msgr2=0x7f7e0c0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:46.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.499+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 shutdown_connections 2024-09-16T07:35:46.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.499+0000 7f7e14800640 1 -- 172.21.15.71:0/3829207828 wait complete. 2024-09-16T07:35:46.512 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:46.512+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:46.512 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:46.512+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:46.512 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:46.512+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:46.512 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:46.512+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:46.512 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:35:46.512+0000 7fe76ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:35:46.515 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone33330 2024-09-16T07:35:46.653 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:35:46.659 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:35:46.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.680+0000 7fae32800640 1 -- 172.21.15.71:0/4224338247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c0b12a0 msgr2=0x7fae2c0aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.680+0000 7fae32800640 1 --2- 172.21.15.71:0/4224338247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c0b12a0 0x7fae2c0aea10 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0737f0 tx=0x7fae2402aa30 comp rx=0 tx=0).stop 2024-09-16T07:35:46.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.680+0000 7fae32800640 1 -- 172.21.15.71:0/4224338247 shutdown_connections 2024-09-16T07:35:46.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.680+0000 7fae32800640 1 --2- 172.21.15.71:0/4224338247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c0b12a0 0x7fae2c0aea10 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.680+0000 7fae32800640 1 --2- 172.21.15.71:0/4224338247 >> v2:172.21.15.177:3300/0 conn(0x7fae2c15f440 0x7fae2c070a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.680+0000 7fae32800640 1 -- 172.21.15.71:0/4224338247 >> 172.21.15.71:0/4224338247 conn(0x7fae2c0d1d40 msgr2=0x7fae2c0d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:46.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.680+0000 7fae32800640 1 -- 172.21.15.71:0/4224338247 shutdown_connections 2024-09-16T07:35:46.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.680+0000 7fae32800640 1 -- 172.21.15.71:0/4224338247 wait complete. 2024-09-16T07:35:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.681+0000 7fae32800640 1 Processor -- start 2024-09-16T07:35:46.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.681+0000 7fae32800640 1 -- start start 2024-09-16T07:35:46.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.682+0000 7fae32800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae2c0b12a0 0x7fae2c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:46.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.682+0000 7fae32800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae2c0d10a0 0x7fae2c1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:46.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.682+0000 7fae32800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c15f440 0x7fae2c0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:46.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.682+0000 7fae32800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c0d0e50 con 0x7fae2c15f440 2024-09-16T07:35:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.682+0000 7fae32800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c0d0cd0 con 0x7fae2c0b12a0 2024-09-16T07:35:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.682+0000 7fae32800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fae2c0aef50 con 0x7fae2c0d10a0 2024-09-16T07:35:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c15f440 0x7fae2c0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:46.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c15f440 0x7fae2c0cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33166/0 (socket says 172.21.15.71:33166) 2024-09-16T07:35:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31e00640 1 -- 172.21.15.71:0/2400828725 learned_addr learned my addr 172.21.15.71:0/2400828725 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31400640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae2c0b12a0 0x7fae2c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae30a00640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae2c0d10a0 0x7fae2c1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31e00640 1 -- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae2c0d10a0 msgr2=0x7fae2c1592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31e00640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae2c0d10a0 0x7fae2c1592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31e00640 1 -- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae2c0b12a0 msgr2=0x7fae2c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31e00640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae2c0b12a0 0x7fae2c0b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.683+0000 7fae31e00640 1 -- 172.21.15.71:0/2400828725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c0c98e0 con 0x7fae2c15f440 2024-09-16T07:35:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.684+0000 7fae31e00640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c15f440 0x7fae2c0cb400 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fae24005fd0 tx=0x7fae24003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.684+0000 7fae29600640 1 -- 172.21.15.71:0/2400828725 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24046070 con 0x7fae2c15f440 2024-09-16T07:35:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.684+0000 7fae29600640 1 -- 172.21.15.71:0/2400828725 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae2402f690 con 0x7fae2c15f440 2024-09-16T07:35:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.684+0000 7fae29600640 1 -- 172.21.15.71:0/2400828725 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24033050 con 0x7fae2c15f440 2024-09-16T07:35:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.684+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2c0c9bc0 con 0x7fae2c15f440 2024-09-16T07:35:46.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.684+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2c159950 con 0x7fae2c15f440 2024-09-16T07:35:46.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.685+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae2c0aea10 con 0x7fae2c15f440 2024-09-16T07:35:46.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.686+0000 7fae29600640 1 -- 172.21.15.71:0/2400828725 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7fae24049070 con 0x7fae2c15f440 2024-09-16T07:35:46.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.688+0000 7fae29600640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae0c07a620 0x7fae0c07cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:46.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.688+0000 7fae29600640 1 -- 172.21.15.71:0/2400828725 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7fae240ba750 con 0x7fae2c15f440 2024-09-16T07:35:46.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.688+0000 7fae31400640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae0c07a620 0x7fae0c07cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:46.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.689+0000 7fae31400640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae0c07a620 0x7fae0c07cae0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fae20009b30 tx=0x7fae20005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:46.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.692+0000 7fae29600640 1 -- 172.21.15.71:0/2400828725 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fae240841d0 con 0x7fae2c15f440 2024-09-16T07:35:46.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:46.911+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone33330", "target": ["mon-mgr", ""]}) -- 0x7fae2c0749f0 con 0x7fae0c07a620 2024-09-16T07:35:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.195+0000 7fae29600640 1 -- 172.21.15.71:0/2400828725 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fae2c0749f0 con 0x7fae0c07a620 2024-09-16T07:35:47.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae0c07a620 msgr2=0x7fae0c07cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae0c07a620 0x7fae0c07cae0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fae20009b30 tx=0x7fae20005f90 comp rx=0 tx=0).stop 2024-09-16T07:35:47.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c15f440 msgr2=0x7fae2c0cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c15f440 0x7fae2c0cb400 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fae24005fd0 tx=0x7fae24003f10 comp rx=0 tx=0).stop 2024-09-16T07:35:47.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 shutdown_connections 2024-09-16T07:35:47.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae0c07a620 0x7fae0c07cae0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae2c15f440 0x7fae2c0cb400 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae2c0d10a0 0x7fae2c1592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 --2- 172.21.15.71:0/2400828725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae2c0b12a0 0x7fae2c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 >> 172.21.15.71:0/2400828725 conn(0x7fae2c0d1d40 msgr2=0x7fae2c070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 shutdown_connections 2024-09-16T07:35:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.198+0000 7fae32800640 1 -- 172.21.15.71:0/2400828725 wait complete. 2024-09-16T07:35:47.214 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone60941 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 -- 172.21.15.71:0/3433733886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9184070380 msgr2=0x7f91840739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 --2- 172.21.15.71:0/3433733886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9184070380 0x7f91840739d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f918415f760 tx=0x7f917c02aa30 comp rx=0 tx=0).stop 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 -- 172.21.15.71:0/3433733886 shutdown_connections 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 --2- 172.21.15.71:0/3433733886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9184070380 0x7f91840739d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 --2- 172.21.15.71:0/3433733886 >> v2:172.21.15.177:3300/0 conn(0x7f918406e590 0x7f918406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 --2- 172.21.15.71:0/3433733886 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f918415bda0 0x7f9184073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 -- 172.21.15.71:0/3433733886 >> 172.21.15.71:0/3433733886 conn(0x7f91840cfa70 msgr2=0x7f91840cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 -- 172.21.15.71:0/3433733886 shutdown_connections 2024-09-16T07:35:47.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.375+0000 7f9183200640 1 -- 172.21.15.71:0/3433733886 wait complete. 2024-09-16T07:35:47.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.376+0000 7f9183200640 1 Processor -- start 2024-09-16T07:35:47.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.376+0000 7f9183200640 1 -- start start 2024-09-16T07:35:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.377+0000 7f9183200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f918406e590 0x7f9184151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.377+0000 7f9183200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9184070380 0x7f9184155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.377+0000 7f9183200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f918415bda0 0x7f918415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.377+0000 7f9183200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9184074090 con 0x7f918415bda0 2024-09-16T07:35:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.377+0000 7f9183200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9184073f10 con 0x7f9184070380 2024-09-16T07:35:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.377+0000 7f9183200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9184074210 con 0x7f918406e590 2024-09-16T07:35:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f918415bda0 0x7f918415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f918415bda0 0x7f918415d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33182/0 (socket says 172.21.15.71:33182) 2024-09-16T07:35:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 -- 172.21.15.71:0/420412073 learned_addr learned my addr 172.21.15.71:0/420412073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9181e00640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f918406e590 0x7f9184151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9181400640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9184070380 0x7f9184155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 -- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f918406e590 msgr2=0x7f9184151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f918406e590 0x7f9184151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 -- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9184070380 msgr2=0x7f9184155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9184070380 0x7f9184155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 -- 172.21.15.71:0/420412073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91841f6120 con 0x7f918415bda0 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9181400640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9184070380 0x7f9184155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.378+0000 7f9182800640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f918415bda0 0x7f918415d820 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f917c009d60 tx=0x7f917c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.379+0000 7f917a000640 1 -- 172.21.15.71:0/420412073 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c047070 con 0x7f918415bda0 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.379+0000 7f917a000640 1 -- 172.21.15.71:0/420412073 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f917c004250 con 0x7f918415bda0 2024-09-16T07:35:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.379+0000 7f917a000640 1 -- 172.21.15.71:0/420412073 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c033050 con 0x7f918415bda0 2024-09-16T07:35:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.379+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91841f6400 con 0x7f918415bda0 2024-09-16T07:35:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.379+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91841f69c0 con 0x7f918415bda0 2024-09-16T07:35:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.380+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9184074740 con 0x7f918415bda0 2024-09-16T07:35:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.381+0000 7f917a000640 1 -- 172.21.15.71:0/420412073 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 44) ==== 103990+0+0 (secure 0 0 0) 0x7f917c04a070 con 0x7f918415bda0 2024-09-16T07:35:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.382+0000 7f917a000640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f916407a620 0x7f916407cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.382+0000 7f917a000640 1 -- 172.21.15.71:0/420412073 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7f917c0bac40 con 0x7f918415bda0 2024-09-16T07:35:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.382+0000 7f9181e00640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f916407a620 0x7f916407cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.383+0000 7f9181e00640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f916407a620 0x7f916407cae0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9174004630 tx=0x7f917400b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.386+0000 7f917a000640 1 -- 172.21.15.71:0/420412073 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f917c0846c0 con 0x7f918415bda0 2024-09-16T07:35:47.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.537+0000 7f917a000640 1 -- 172.21.15.71:0/420412073 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f917c036070 con 0x7f918415bda0 2024-09-16T07:35:47.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.601+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone60941", "target": ["mon-mgr", ""]}) -- 0x7f91840b4310 con 0x7f916407a620 2024-09-16T07:35:47.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.618+0000 7f917a000640 1 -- 172.21.15.71:0/420412073 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f91840b4310 con 0x7f916407a620 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f916407a620 msgr2=0x7f916407cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f916407a620 0x7f916407cae0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9174004630 tx=0x7f917400b2e0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f918415bda0 msgr2=0x7f918415d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f918415bda0 0x7f918415d820 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f917c009d60 tx=0x7f917c003f10 comp rx=0 tx=0).stop 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 shutdown_connections 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f916407a620 0x7f916407cae0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f918415bda0 0x7f918415d820 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9184070380 0x7f9184155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 --2- 172.21.15.71:0/420412073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f918406e590 0x7f9184151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 >> 172.21.15.71:0/420412073 conn(0x7f91840cfa70 msgr2=0x7f91840b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 shutdown_connections 2024-09-16T07:35:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.621+0000 7f9183200640 1 -- 172.21.15.71:0/420412073 wait complete. 2024-09-16T07:35:47.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone87142 2024-09-16T07:35:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.798+0000 7f3442400640 1 -- 172.21.15.71:0/2670742038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4770 msgr2=0x7f343c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.798+0000 7f3442400640 1 --2- 172.21.15.71:0/2670742038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4770 0x7f343c06e6f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3428009830 tx=0x7f342802a510 comp rx=0 tx=0).stop 2024-09-16T07:35:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.798+0000 7f3442400640 1 -- 172.21.15.71:0/2670742038 shutdown_connections 2024-09-16T07:35:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.798+0000 7f3442400640 1 --2- 172.21.15.71:0/2670742038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4770 0x7f343c06e6f0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.798+0000 7f3442400640 1 --2- 172.21.15.71:0/2670742038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f343c15dfc0 0x7f343c06f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.798+0000 7f3442400640 1 -- 172.21.15.71:0/2670742038 >> 172.21.15.71:0/2670742038 conn(0x7f343c0b18e0 msgr2=0x7f343c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.798+0000 7f3442400640 1 -- 172.21.15.71:0/2670742038 shutdown_connections 2024-09-16T07:35:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.798+0000 7f3442400640 1 -- 172.21.15.71:0/2670742038 wait complete. 2024-09-16T07:35:47.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.799+0000 7f3442400640 1 Processor -- start 2024-09-16T07:35:47.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.799+0000 7f3442400640 1 -- start start 2024-09-16T07:35:47.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.800+0000 7f3442400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c06ec30 0x7f343c09cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.800+0000 7f3442400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f343c0b4770 0x7f343c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.800+0000 7f3442400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f343c15dfc0 0x7f343c1f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.800+0000 7f3442400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1622c0 con 0x7f343c06ec30 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.800+0000 7f3442400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f343c0ae8f0 con 0x7f343c0b4770 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.800+0000 7f3442400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f343c162440 con 0x7f343c15dfc0 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.800+0000 7f343b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c06ec30 0x7f343c09cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c06ec30 0x7f343c09cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33204/0 (socket says 172.21.15.71:33204) 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343b400640 1 -- 172.21.15.71:0/2921162305 learned_addr learned my addr 172.21.15.71:0/2921162305 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343b400640 1 -- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f343c15dfc0 msgr2=0x7f343c1f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343aa00640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f343c0b4770 0x7f343c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:47.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343be00640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f343c15dfc0 0x7f343c1f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343b400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f343c15dfc0 0x7f343c1f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343b400640 1 -- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f343c0b4770 msgr2=0x7f343c156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343b400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f343c0b4770 0x7f343c156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343b400640 1 -- 172.21.15.71:0/2921162305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c157020 con 0x7f343c06ec30 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343aa00640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f343c0b4770 0x7f343c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343be00640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f343c15dfc0 0x7f343c1f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f343b400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c06ec30 0x7f343c09cfe0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f342c00eb30 tx=0x7f342c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.801+0000 7f3433400640 1 -- 172.21.15.71:0/2921162305 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c010430 con 0x7f343c06ec30 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.802+0000 7f3433400640 1 -- 172.21.15.71:0/2921162305 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f342c0105d0 con 0x7f343c06ec30 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.802+0000 7f3433400640 1 -- 172.21.15.71:0/2921162305 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c010a20 con 0x7f343c06ec30 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.802+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c155040 con 0x7f343c06ec30 2024-09-16T07:35:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.802+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c1f5470 con 0x7f343c06ec30 2024-09-16T07:35:47.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.803+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343c069360 con 0x7f343c06ec30 2024-09-16T07:35:47.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.804+0000 7f3433400640 1 -- 172.21.15.71:0/2921162305 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f342c010bc0 con 0x7f343c06ec30 2024-09-16T07:35:47.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.805+0000 7f3433400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f341c07a6b0 0x7f341c07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:47.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.805+0000 7f3433400640 1 -- 172.21.15.71:0/2921162305 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7f342c09a790 con 0x7f343c06ec30 2024-09-16T07:35:47.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.805+0000 7f343aa00640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f341c07a6b0 0x7f341c07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:47.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.806+0000 7f343aa00640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f341c07a6b0 0x7f341c07cb70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3428034000 tx=0x7f3428004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:47.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:47.809+0000 7f3433400640 1 -- 172.21.15.71:0/2921162305 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f342c064860 con 0x7f343c06ec30 2024-09-16T07:35:48.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.018+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone87142", "target": ["mon-mgr", ""]}) -- 0x7f343c0735e0 con 0x7f341c07a6b0 2024-09-16T07:35:48.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.036+0000 7f3433400640 1 -- 172.21.15.71:0/2921162305 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f343c0735e0 con 0x7f341c07a6b0 2024-09-16T07:35:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.038+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f341c07a6b0 msgr2=0x7f341c07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.038+0000 7f3442400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f341c07a6b0 0x7f341c07cb70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3428034000 tx=0x7f3428004730 comp rx=0 tx=0).stop 2024-09-16T07:35:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.038+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c06ec30 msgr2=0x7f343c09cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.038+0000 7f3442400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c06ec30 0x7f343c09cfe0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f342c00eb30 tx=0x7f342c00c2d0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.039+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 shutdown_connections 2024-09-16T07:35:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.039+0000 7f3442400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f341c07a6b0 0x7f341c07cb70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.039+0000 7f3442400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f343c15dfc0 0x7f343c1f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.039+0000 7f3442400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f343c0b4770 0x7f343c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.039+0000 7f3442400640 1 --2- 172.21.15.71:0/2921162305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c06ec30 0x7f343c09cfe0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.039+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 >> 172.21.15.71:0/2921162305 conn(0x7f343c0b18e0 msgr2=0x7f343c073b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.039+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 shutdown_connections 2024-09-16T07:35:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.039+0000 7f3442400640 1 -- 172.21.15.71:0/2921162305 wait complete. 2024-09-16T07:35:48.055 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:35:48.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:35:48.139 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:35:48.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:35:48.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:35:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.310+0000 7fc116e00640 1 -- 172.21.15.71:0/146293855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0f8004800 msgr2=0x7fc0f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.310+0000 7fc116e00640 1 --2- 172.21.15.71:0/146293855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0f8004800 0x7fc0f8004c80 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009b20 tx=0x7fc0f8005830 comp rx=0 tx=0).stop 2024-09-16T07:35:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.311+0000 7fc116e00640 1 -- 172.21.15.71:0/146293855 shutdown_connections 2024-09-16T07:35:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.311+0000 7fc116e00640 1 --2- 172.21.15.71:0/146293855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0f8004800 0x7fc0f8004c80 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.311+0000 7fc116e00640 1 -- 172.21.15.71:0/146293855 >> 172.21.15.71:0/146293855 conn(0x7fc1100d0290 msgr2=0x7fc1100d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.311+0000 7fc116e00640 1 -- 172.21.15.71:0/146293855 shutdown_connections 2024-09-16T07:35:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.311+0000 7fc116e00640 1 -- 172.21.15.71:0/146293855 wait complete. 2024-09-16T07:35:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.312+0000 7fc116e00640 1 Processor -- start 2024-09-16T07:35:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.312+0000 7fc116e00640 1 -- start start 2024-09-16T07:35:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.313+0000 7fc116e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f8004800 0x7fc110070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.313+0000 7fc116e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc11014e680 0x7fc1101512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.313+0000 7fc116e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc110151830 0x7fc1101f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.313+0000 7fc116e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc11015e6f0 con 0x7fc11014e680 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.313+0000 7fc116e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc11015e570 con 0x7fc110151830 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.313+0000 7fc116e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1100ce340 con 0x7fc0f8004800 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc11014e680 0x7fc1101512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc11014e680 0x7fc1101512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33230/0 (socket says 172.21.15.71:33230) 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 -- 172.21.15.71:0/4147258712 learned_addr learned my addr 172.21.15.71:0/4147258712 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc116400640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc110151830 0x7fc1101f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115a00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f8004800 0x7fc110070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 -- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f8004800 msgr2=0x7fc110070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f8004800 0x7fc110070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 -- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc110151830 msgr2=0x7fc1101f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc110151830 0x7fc1101f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 -- 172.21.15.71:0/4147258712 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11014dbc0 con 0x7fc11014e680 2024-09-16T07:35:48.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.314+0000 7fc115000640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc11014e680 0x7fc1101512f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009a70 tx=0x7fc0fc033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:48.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.315+0000 7fc10d600640 1 -- 172.21.15.71:0/4147258712 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc038040 con 0x7fc11014e680 2024-09-16T07:35:48.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.315+0000 7fc10d600640 1 -- 172.21.15.71:0/4147258712 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0fc033950 con 0x7fc11014e680 2024-09-16T07:35:48.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.315+0000 7fc10d600640 1 -- 172.21.15.71:0/4147258712 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc033da0 con 0x7fc11014e680 2024-09-16T07:35:48.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.315+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1101f5020 con 0x7fc11014e680 2024-09-16T07:35:48.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.315+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11014e160 con 0x7fc11014e680 2024-09-16T07:35:48.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.316+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11014d9f0 con 0x7fc11014e680 2024-09-16T07:35:48.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.317+0000 7fc10d600640 1 -- 172.21.15.71:0/4147258712 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fc0fc043020 con 0x7fc11014e680 2024-09-16T07:35:48.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.318+0000 7fc10d600640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0f007a6b0 0x7fc0f007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:48.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.318+0000 7fc10d600640 1 -- 172.21.15.71:0/4147258712 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5852+0+0 (secure 0 0 0) 0x7fc0fc0b98f0 con 0x7fc11014e680 2024-09-16T07:35:48.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.318+0000 7fc115a00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0f007a6b0 0x7fc0f007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:48.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.319+0000 7fc115a00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0f007a6b0 0x7fc0f007cb70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc1101596a0 tx=0x7fc104006b70 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:48.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.322+0000 7fc10d600640 1 -- 172.21.15.71:0/4147258712 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc0fc04a020 con 0x7fc11014e680 2024-09-16T07:35:48.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.509+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fc1100cbaa0 con 0x7fc11014e680 2024-09-16T07:35:48.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.509+0000 7fc10d600640 1 -- 172.21.15.71:0/4147258712 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7fc1100cbaa0 con 0x7fc11014e680 2024-09-16T07:35:48.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0f007a6b0 msgr2=0x7fc0f007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:48.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0f007a6b0 0x7fc0f007cb70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc1101596a0 tx=0x7fc104006b70 comp rx=0 tx=0).stop 2024-09-16T07:35:48.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc11014e680 msgr2=0x7fc1101512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc11014e680 0x7fc1101512f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009a70 tx=0x7fc0fc033670 comp rx=0 tx=0).stop 2024-09-16T07:35:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 shutdown_connections 2024-09-16T07:35:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc0f007a6b0 0x7fc0f007cb70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc110151830 0x7fc1101f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc11014e680 0x7fc1101512f0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 --2- 172.21.15.71:0/4147258712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f8004800 0x7fc110070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:48.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 >> 172.21.15.71:0/4147258712 conn(0x7fc1100d0290 msgr2=0x7fc1100b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:48.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.511+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 shutdown_connections 2024-09-16T07:35:48.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:48.512+0000 7fc116e00640 1 -- 172.21.15.71:0/4147258712 wait complete. 2024-09-16T07:35:48.527 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:35:48.527 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:35:48.625 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:35:48.751 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:35:48.751 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:35:48.752 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:48.752 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:48.812 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:48.812 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:48.874 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:35:49.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.060+0000 7fa2baa00640 1 -- 172.21.15.71:0/3511930743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2b415f5a0 msgr2=0x7fa2b40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:49.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.060+0000 7fa2baa00640 1 --2- 172.21.15.71:0/3511930743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2b415f5a0 0x7fa2b40b2440 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009830 tx=0x7fa2a402a530 comp rx=0 tx=0).stop 2024-09-16T07:35:49.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.060+0000 7fa2baa00640 1 -- 172.21.15.71:0/3511930743 shutdown_connections 2024-09-16T07:35:49.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.061+0000 7fa2baa00640 1 --2- 172.21.15.71:0/3511930743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2b415f5a0 0x7fa2b40b2440 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.061+0000 7fa2baa00640 1 -- 172.21.15.71:0/3511930743 >> 172.21.15.71:0/3511930743 conn(0x7fa2b40b1950 msgr2=0x7fa2b40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:49.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.061+0000 7fa2baa00640 1 -- 172.21.15.71:0/3511930743 shutdown_connections 2024-09-16T07:35:49.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.061+0000 7fa2baa00640 1 -- 172.21.15.71:0/3511930743 wait complete. 2024-09-16T07:35:49.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.062+0000 7fa2baa00640 1 Processor -- start 2024-09-16T07:35:49.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.062+0000 7fa2baa00640 1 -- start start 2024-09-16T07:35:49.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.063+0000 7fa2baa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2b4073fb0 0x7fa2b40cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.063+0000 7fa2baa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2b40ccf00 0x7fa2b4143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.063+0000 7fa2baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b415f5a0 0x7fa2b41f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.063+0000 7fa2baa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b406f9f0 con 0x7fa2b415f5a0 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.063+0000 7fa2baa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b406f870 con 0x7fa2b40ccf00 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.063+0000 7fa2baa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b4074420 con 0x7fa2b4073fb0 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.063+0000 7fa2b8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b415f5a0 0x7fa2b41f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b415f5a0 0x7fa2b41f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33244/0 (socket says 172.21.15.71:33244) 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b8a00640 1 -- 172.21.15.71:0/4006007007 learned_addr learned my addr 172.21.15.71:0/4006007007 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2abe00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2b40ccf00 0x7fa2b4143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b3e00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2b4073fb0 0x7fa2b40cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b8a00640 1 -- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2b4073fb0 msgr2=0x7fa2b40cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b8a00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2b4073fb0 0x7fa2b40cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b8a00640 1 -- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2b40ccf00 msgr2=0x7fa2b4143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b8a00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2b40ccf00 0x7fa2b4143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b8a00640 1 -- 172.21.15.71:0/4006007007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b41f8db0 con 0x7fa2b415f5a0 2024-09-16T07:35:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b8a00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b415f5a0 0x7fa2b41f4c80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00c780 tx=0x7fa2ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b0c00640 1 -- 172.21.15.71:0/4006007007 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac003b70 con 0x7fa2b415f5a0 2024-09-16T07:35:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2b0c00640 1 -- 172.21.15.71:0/4006007007 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2ac003d10 con 0x7fa2b415f5a0 2024-09-16T07:35:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.064+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b41f9090 con 0x7fa2b415f5a0 2024-09-16T07:35:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.065+0000 7fa2b0c00640 1 -- 172.21.15.71:0/4006007007 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac00aab0 con 0x7fa2b415f5a0 2024-09-16T07:35:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.065+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b409b5c0 con 0x7fa2b415f5a0 2024-09-16T07:35:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.066+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b40b2440 con 0x7fa2b415f5a0 2024-09-16T07:35:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.067+0000 7fa2b0c00640 1 -- 172.21.15.71:0/4006007007 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fa2ac00ac50 con 0x7fa2b415f5a0 2024-09-16T07:35:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.068+0000 7fa2b0c00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa29007a6b0 0x7fa29007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.068+0000 7fa2b0c00640 1 -- 172.21.15.71:0/4006007007 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5908+0+0 (secure 0 0 0) 0x7fa2ac099730 con 0x7fa2b415f5a0 2024-09-16T07:35:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.068+0000 7fa2b3e00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa29007a6b0 0x7fa29007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.069+0000 7fa2b3e00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa29007a6b0 0x7fa29007cb70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4034000 tx=0x7fa2a4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.072+0000 7fa2b0c00640 1 -- 172.21.15.71:0/4006007007 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa2ac063130 con 0x7fa2b415f5a0 2024-09-16T07:35:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.271+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2b409b790 con 0x7fa2b415f5a0 2024-09-16T07:35:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.272+0000 7fa2b0c00640 1 -- 172.21.15.71:0/4006007007 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7fa2ac067b60 con 0x7fa2b415f5a0 2024-09-16T07:35:49.271 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa29007a6b0 msgr2=0x7fa29007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa29007a6b0 0x7fa29007cb70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4034000 tx=0x7fa2a4004730 comp rx=0 tx=0).stop 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b415f5a0 msgr2=0x7fa2b41f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b415f5a0 0x7fa2b41f4c80 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00c780 tx=0x7fa2ac00cc40 comp rx=0 tx=0).stop 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 shutdown_connections 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa29007a6b0 0x7fa29007cb70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2b415f5a0 0x7fa2b41f4c80 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2b40ccf00 0x7fa2b4143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 --2- 172.21.15.71:0/4006007007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2b4073fb0 0x7fa2b40cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 >> 172.21.15.71:0/4006007007 conn(0x7fa2b40b1950 msgr2=0x7fa2b40acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 shutdown_connections 2024-09-16T07:35:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.274+0000 7fa2baa00640 1 -- 172.21.15.71:0/4006007007 wait complete. 2024-09-16T07:35:49.290 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:35:49.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:49.290 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:35:49.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:35:49.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:35:49.369 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:35:49.422 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:35:49.422 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:35:49.422 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:49.458 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:35:49.458 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:35:49.458 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:49.458 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:49.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:35:49.494 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:35:49.494 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:35:49.494 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:49.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:35:49.534 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:35:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.759+0000 7f0ebf200640 1 -- 172.21.15.71:0/806135116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb80b4020 msgr2=0x7f0eb80b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.759+0000 7f0ebf200640 1 --2- 172.21.15.71:0/806135116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb80b4020 0x7f0eb80b1770 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80d4ea0 tx=0x7f0eb402a7f0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.759+0000 7f0ebf200640 1 -- 172.21.15.71:0/806135116 shutdown_connections 2024-09-16T07:35:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.759+0000 7f0ebf200640 1 --2- 172.21.15.71:0/806135116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb80b4020 0x7f0eb80b1770 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.759+0000 7f0ebf200640 1 --2- 172.21.15.71:0/806135116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb8161440 0x7f0eb80b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.759+0000 7f0ebf200640 1 -- 172.21.15.71:0/806135116 >> 172.21.15.71:0/806135116 conn(0x7f0eb80d3df0 msgr2=0x7f0eb80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.759+0000 7f0ebf200640 1 -- 172.21.15.71:0/806135116 shutdown_connections 2024-09-16T07:35:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.759+0000 7f0ebf200640 1 -- 172.21.15.71:0/806135116 wait complete. 2024-09-16T07:35:49.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.762+0000 7f0ebf200640 1 Processor -- start 2024-09-16T07:35:49.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.762+0000 7f0ebf200640 1 -- start start 2024-09-16T07:35:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.764+0000 7f0ebf200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb80b34b0 0x7f0eb8154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.764+0000 7f0ebf200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb80b4020 0x7f0eb806e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.764+0000 7f0ebf200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb8161440 0x7f0eb806e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.764+0000 7f0ebf200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb80b2040 con 0x7f0eb80b4020 2024-09-16T07:35:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.764+0000 7f0ebf200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb80b1ec0 con 0x7f0eb8161440 2024-09-16T07:35:49.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.764+0000 7f0ebf200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb80b21c0 con 0x7f0eb80b34b0 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.765+0000 7f0ebd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb80b4020 0x7f0eb806e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.765+0000 7f0ebde00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb80b34b0 0x7f0eb8154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.765+0000 7f0ebe800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb8161440 0x7f0eb806e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.765+0000 7f0ebd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb80b4020 0x7f0eb806e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33246/0 (socket says 172.21.15.71:33246) 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.766+0000 7f0ebd400640 1 -- 172.21.15.71:0/3701898143 learned_addr learned my addr 172.21.15.71:0/3701898143 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.766+0000 7f0ebd400640 1 -- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb80b34b0 msgr2=0x7f0eb8154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.766+0000 7f0ebd400640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb80b34b0 0x7f0eb8154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.766+0000 7f0ebd400640 1 -- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb8161440 msgr2=0x7f0eb806e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.766+0000 7f0ebd400640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb8161440 0x7f0eb806e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:49.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.766+0000 7f0ebd400640 1 -- 172.21.15.71:0/3701898143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb8160b00 con 0x7f0eb80b4020 2024-09-16T07:35:49.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.767+0000 7f0ebd400640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb80b4020 0x7f0eb806e0e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800c460 tx=0x7f0ea800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.768+0000 7f0eb2000640 1 -- 172.21.15.71:0/3701898143 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8016020 con 0x7f0eb80b4020 2024-09-16T07:35:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.768+0000 7f0eb2000640 1 -- 172.21.15.71:0/3701898143 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ea8012920 con 0x7f0eb80b4020 2024-09-16T07:35:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.768+0000 7f0eb2000640 1 -- 172.21.15.71:0/3701898143 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8012d50 con 0x7f0eb80b4020 2024-09-16T07:35:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.768+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb8155e80 con 0x7f0eb80b4020 2024-09-16T07:35:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.768+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb8154fb0 con 0x7f0eb80b4020 2024-09-16T07:35:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.770+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb80b1770 con 0x7f0eb80b4020 2024-09-16T07:35:49.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.773+0000 7f0eb2000640 1 -- 172.21.15.71:0/3701898143 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f0ea8019020 con 0x7f0eb80b4020 2024-09-16T07:35:49.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.774+0000 7f0eb2000640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e9c07a6e0 0x7f0e9c07cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:49.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.775+0000 7f0eb2000640 1 -- 172.21.15.71:0/3701898143 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5908+0+0 (secure 0 0 0) 0x7f0ea8099550 con 0x7f0eb80b4020 2024-09-16T07:35:49.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.776+0000 7f0ebde00640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e9c07a6e0 0x7f0e9c07cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:49.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.778+0000 7f0ebde00640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e9c07a6e0 0x7f0e9c07cba0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009ac0 tx=0x7f0eac005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:49.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:49.782+0000 7f0eb2000640 1 -- 172.21.15.71:0/3701898143 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0ea8020030 con 0x7f0eb80b4020 2024-09-16T07:35:50.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.007+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0eb806ed90 con 0x7f0eb80b4020 2024-09-16T07:35:50.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.008+0000 7f0eb2000640 1 -- 172.21.15.71:0/3701898143 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5231 (secure 0 0 0) 0x7f0ea80636e0 con 0x7f0eb80b4020 2024-09-16T07:35:50.317 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:50.317 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-16T07:35:01:329397+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1994777108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1994777108},{"type":"v1","addr":"172.21.15.71:6833","nonce":1994777108}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1399666780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1399666780},{"type":"v1","addr":"172.21.15.177:6836","nonce":1399666780}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2921631965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2921631965},{"type":"v1","addr":"172.21.15.71:6835","nonce":2921631965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:34:47.926434+0000","modified":"2024-09-16T07:34:50.958041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2539335973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2539335973},{"type":"v1","addr":"172.21.15.177:6837","nonce":2539335973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8382,"qdb_cluster":[8382]},"id":8}]} 2024-09-16T07:35:50.318 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e9c07a6e0 msgr2=0x7f0e9c07cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e9c07a6e0 0x7f0e9c07cba0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009ac0 tx=0x7f0eac005f90 comp rx=0 tx=0).stop 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb80b4020 msgr2=0x7f0eb806e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb80b4020 0x7f0eb806e0e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800c460 tx=0x7f0ea800c920 comp rx=0 tx=0).stop 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 shutdown_connections 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0e9c07a6e0 0x7f0e9c07cba0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0eb8161440 0x7f0eb806e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0eb80b4020 0x7f0eb806e0e0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 --2- 172.21.15.71:0/3701898143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0eb80b34b0 0x7f0eb8154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.010+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 >> 172.21.15.71:0/3701898143 conn(0x7f0eb80d3df0 msgr2=0x7f0eb8158710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.011+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 shutdown_connections 2024-09-16T07:35:50.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.011+0000 7f0ebf200640 1 -- 172.21.15.71:0/3701898143 wait complete. 2024-09-16T07:35:50.320 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:35:50.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.477+0000 7fa55f000640 1 -- 172.21.15.71:0/752670308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558073fb0 msgr2=0x7fa558074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.477+0000 7fa55f000640 1 --2- 172.21.15.71:0/752670308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558073fb0 0x7fa558074420 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa558075130 tx=0x7fa55002a980 comp rx=0 tx=0).stop 2024-09-16T07:35:50.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.477+0000 7fa55f000640 1 -- 172.21.15.71:0/752670308 shutdown_connections 2024-09-16T07:35:50.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.477+0000 7fa55f000640 1 --2- 172.21.15.71:0/752670308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558073fb0 0x7fa558074420 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.477+0000 7fa55f000640 1 --2- 172.21.15.71:0/752670308 >> v2:172.21.15.177:3300/0 conn(0x7fa55815f5a0 0x7fa5580b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.477+0000 7fa55f000640 1 -- 172.21.15.71:0/752670308 >> 172.21.15.71:0/752670308 conn(0x7fa5580b1950 msgr2=0x7fa5580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.477+0000 7fa55f000640 1 -- 172.21.15.71:0/752670308 shutdown_connections 2024-09-16T07:35:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.478+0000 7fa55f000640 1 -- 172.21.15.71:0/752670308 wait complete. 2024-09-16T07:35:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.479+0000 7fa55f000640 1 Processor -- start 2024-09-16T07:35:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.479+0000 7fa55f000640 1 -- start start 2024-09-16T07:35:50.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa55f000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa558073fb0 0x7fa5581f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:50.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa55f000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5580ccf00 0x7fa558148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa55f000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa55815f5a0 0x7fa55815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa55f000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa55806f2e0 con 0x7fa5580ccf00 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa55f000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa55806f160 con 0x7fa558073fb0 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa55f000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa55806f460 con 0x7fa55815f5a0 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa557400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5580ccf00 0x7fa558148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa557e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa558073fb0 0x7fa5581f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa557400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5580ccf00 0x7fa558148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33258/0 (socket says 172.21.15.71:33258) 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.480+0000 7fa557400640 1 -- 172.21.15.71:0/1028050048 learned_addr learned my addr 172.21.15.71:0/1028050048 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa55d000640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa55815f5a0 0x7fa55815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa557400640 1 -- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa55815f5a0 msgr2=0x7fa55815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa557400640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa55815f5a0 0x7fa55815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa557400640 1 -- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa558073fb0 msgr2=0x7fa5581f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa557400640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa558073fb0 0x7fa5581f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa557400640 1 -- 172.21.15.71:0/1028050048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa558152f70 con 0x7fa5580ccf00 2024-09-16T07:35:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa55d000640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa55815f5a0 0x7fa55815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa557400640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5580ccf00 0x7fa558148680 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c430 tx=0x7fa54400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa54fe00640 1 -- 172.21.15.71:0/1028050048 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544011790 con 0x7fa5580ccf00 2024-09-16T07:35:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.481+0000 7fa54fe00640 1 -- 172.21.15.71:0/1028050048 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa544017070 con 0x7fa5580ccf00 2024-09-16T07:35:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.482+0000 7fa54fe00640 1 -- 172.21.15.71:0/1028050048 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544011d00 con 0x7fa5580ccf00 2024-09-16T07:35:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.482+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa558153250 con 0x7fa5580ccf00 2024-09-16T07:35:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.482+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55809abf0 con 0x7fa5580ccf00 2024-09-16T07:35:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.483+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55806f9f0 con 0x7fa5580ccf00 2024-09-16T07:35:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.484+0000 7fa54fe00640 1 -- 172.21.15.71:0/1028050048 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fa54401c460 con 0x7fa5580ccf00 2024-09-16T07:35:50.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.485+0000 7fa54fe00640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa53807a6b0 0x7fa53807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:50.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.485+0000 7fa54fe00640 1 -- 172.21.15.71:0/1028050048 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5908+0+0 (secure 0 0 0) 0x7fa5440997a0 con 0x7fa5580ccf00 2024-09-16T07:35:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.485+0000 7fa557e00640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa53807a6b0 0x7fa53807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.486+0000 7fa557e00640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa53807a6b0 0x7fa53807cb70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa548004630 tx=0x7fa54800b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:50.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.489+0000 7fa54fe00640 1 -- 172.21.15.71:0/1028050048 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa544063870 con 0x7fa5580ccf00 2024-09-16T07:35:50.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.777+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5580ace90 con 0x7fa5580ccf00 2024-09-16T07:35:50.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.778+0000 7fa54fe00640 1 -- 172.21.15.71:0/1028050048 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5231 (secure 0 0 0) 0x7fa544063170 con 0x7fa5580ccf00 2024-09-16T07:35:50.778 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:50.778 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-16T07:35:01:329397+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1994777108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1994777108},{"type":"v1","addr":"172.21.15.71:6833","nonce":1994777108}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1399666780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1399666780},{"type":"v1","addr":"172.21.15.177:6836","nonce":1399666780}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2921631965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2921631965},{"type":"v1","addr":"172.21.15.71:6835","nonce":2921631965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:34:47.926434+0000","modified":"2024-09-16T07:34:50.958041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2539335973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2539335973},{"type":"v1","addr":"172.21.15.177:6837","nonce":2539335973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8382,"qdb_cluster":[8382]},"id":8}]} 2024-09-16T07:35:50.778 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-16T07:35:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa53807a6b0 msgr2=0x7fa53807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa53807a6b0 0x7fa53807cb70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa548004630 tx=0x7fa54800b2e0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5580ccf00 msgr2=0x7fa558148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5580ccf00 0x7fa558148680 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c430 tx=0x7fa54400c8f0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 shutdown_connections 2024-09-16T07:35:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa53807a6b0 0x7fa53807cb70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa55815f5a0 0x7fa55815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5580ccf00 0x7fa558148680 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 --2- 172.21.15.71:0/1028050048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa558073fb0 0x7fa5581f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.780+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 >> 172.21.15.71:0/1028050048 conn(0x7fa5580b1950 msgr2=0x7fa5580b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.781+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 shutdown_connections 2024-09-16T07:35:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.781+0000 7fa55f000640 1 -- 172.21.15.71:0/1028050048 wait complete. 2024-09-16T07:35:50.795 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:35:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.952+0000 7f1287400640 1 -- 172.21.15.71:0/932412400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f128015f5a0 msgr2=0x7f12800b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.952+0000 7f1287400640 1 --2- 172.21.15.71:0/932412400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f128015f5a0 0x7f12800b2440 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f1270005fd0 tx=0x7f127002a540 comp rx=0 tx=0).stop 2024-09-16T07:35:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.952+0000 7f1287400640 1 -- 172.21.15.71:0/932412400 shutdown_connections 2024-09-16T07:35:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.952+0000 7f1287400640 1 --2- 172.21.15.71:0/932412400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1280073fb0 0x7f1280074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.952+0000 7f1287400640 1 --2- 172.21.15.71:0/932412400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f128015f5a0 0x7f12800b2440 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.952+0000 7f1287400640 1 -- 172.21.15.71:0/932412400 >> 172.21.15.71:0/932412400 conn(0x7f12800b1950 msgr2=0x7f12800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.952+0000 7f1287400640 1 -- 172.21.15.71:0/932412400 shutdown_connections 2024-09-16T07:35:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.952+0000 7f1287400640 1 -- 172.21.15.71:0/932412400 wait complete. 2024-09-16T07:35:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.954+0000 7f1287400640 1 Processor -- start 2024-09-16T07:35:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.954+0000 7f1287400640 1 -- start start 2024-09-16T07:35:50.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1287400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280073fb0 0x7f12800b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:50.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1287400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12800ccf00 0x7f12800cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:50.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1287400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f128015f5a0 0x7f12801f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:50.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1287400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f128006f2e0 con 0x7f1280073fb0 2024-09-16T07:35:50.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1287400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f128006f160 con 0x7f128015f5a0 2024-09-16T07:35:50.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1287400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f128006f460 con 0x7f12800ccf00 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1284a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280073fb0 0x7f12800b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1284a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280073fb0 0x7f12800b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33288/0 (socket says 172.21.15.71:33288) 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1284a00640 1 -- 172.21.15.71:0/2882287472 learned_addr learned my addr 172.21.15.71:0/2882287472 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f127fe00640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12800ccf00 0x7f12800cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.955+0000 7f1285400640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f128015f5a0 0x7f12801f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f1284a00640 1 -- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12800ccf00 msgr2=0x7f12800cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f1284a00640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12800ccf00 0x7f12800cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f1284a00640 1 -- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f128015f5a0 msgr2=0x7f12801f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f1284a00640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f128015f5a0 0x7f12801f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f1284a00640 1 -- 172.21.15.71:0/2882287472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12801f56b0 con 0x7f1280073fb0 2024-09-16T07:35:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f1284a00640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280073fb0 0x7f12800b39d0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1270034040 tx=0x7f1270003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f127cc00640 1 -- 172.21.15.71:0/2882287472 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127003e030 con 0x7f1280073fb0 2024-09-16T07:35:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f127cc00640 1 -- 172.21.15.71:0/2882287472 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1270003e00 con 0x7f1280073fb0 2024-09-16T07:35:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f127cc00640 1 -- 172.21.15.71:0/2882287472 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270035040 con 0x7f1280073fb0 2024-09-16T07:35:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12800b2ef0 con 0x7f1280073fb0 2024-09-16T07:35:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.956+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12800b34b0 con 0x7f1280073fb0 2024-09-16T07:35:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.957+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128006f980 con 0x7f1280073fb0 2024-09-16T07:35:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.959+0000 7f127cc00640 1 -- 172.21.15.71:0/2882287472 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f1270041030 con 0x7f1280073fb0 2024-09-16T07:35:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.959+0000 7f127cc00640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f126007a6b0 0x7f126007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.960+0000 7f127cc00640 1 -- 172.21.15.71:0/2882287472 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5908+0+0 (secure 0 0 0) 0x7f12700b9f30 con 0x7f1280073fb0 2024-09-16T07:35:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.960+0000 7f127fe00640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f126007a6b0 0x7f126007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.960+0000 7f127fe00640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f126007a6b0 0x7f126007cb70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f12801550a0 tx=0x7f126c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:50.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:50.964+0000 7f127cc00640 1 -- 172.21.15.71:0/2882287472 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f12700839b0 con 0x7f1280073fb0 2024-09-16T07:35:51.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.152+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12800b3680 con 0x7f1280073fb0 2024-09-16T07:35:51.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.153+0000 7f127cc00640 1 -- 172.21.15.71:0/2882287472 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+17086 (secure 0 0 0) 0x7f12700883e0 con 0x7f1280073fb0 2024-09-16T07:35:51.153 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:51.153 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":122,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:35:48.532744+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:34:45.706284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T07:34:46.892492+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":122,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.8","mappings":[{"from":4,"to":3}]},{"pgid":"17.b","mappings":[{"from":6,"to":1}]},{"pgid":"17.c","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:35:51.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f126007a6b0 msgr2=0x7f126007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f126007a6b0 0x7f126007cb70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f12801550a0 tx=0x7f126c009040 comp rx=0 tx=0).stop 2024-09-16T07:35:51.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280073fb0 msgr2=0x7f12800b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280073fb0 0x7f12800b39d0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1270034040 tx=0x7f1270003910 comp rx=0 tx=0).stop 2024-09-16T07:35:51.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 shutdown_connections 2024-09-16T07:35:51.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f126007a6b0 0x7f126007cb70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f128015f5a0 0x7f12801f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12800ccf00 0x7f12800cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 --2- 172.21.15.71:0/2882287472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280073fb0 0x7f12800b39d0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 >> 172.21.15.71:0/2882287472 conn(0x7f12800b1950 msgr2=0x7f12800b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:51.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 shutdown_connections 2024-09-16T07:35:51.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.155+0000 7f1287400640 1 -- 172.21.15.71:0/2882287472 wait complete. 2024-09-16T07:35:51.172 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:35:51.172 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:35:51.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.329+0000 7f2ba6000640 1 -- 172.21.15.71:0/50363664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 msgr2=0x7f2ba00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.329+0000 7f2ba6000640 1 --2- 172.21.15.71:0/50363664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 0x7f2ba00b2440 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2b90005fd0 tx=0x7f2b9002aaa0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.329+0000 7f2ba6000640 1 -- 172.21.15.71:0/50363664 shutdown_connections 2024-09-16T07:35:51.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.329+0000 7f2ba6000640 1 --2- 172.21.15.71:0/50363664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 0x7f2ba00b2440 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.329+0000 7f2ba6000640 1 -- 172.21.15.71:0/50363664 >> 172.21.15.71:0/50363664 conn(0x7f2ba00b1950 msgr2=0x7f2ba00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:51.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.329+0000 7f2ba6000640 1 -- 172.21.15.71:0/50363664 shutdown_connections 2024-09-16T07:35:51.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.329+0000 7f2ba6000640 1 -- 172.21.15.71:0/50363664 wait complete. 2024-09-16T07:35:51.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.331+0000 7f2ba6000640 1 Processor -- start 2024-09-16T07:35:51.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.331+0000 7f2ba6000640 1 -- start start 2024-09-16T07:35:51.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2ba6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0073fb0 0x7f2ba01f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:51.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2ba6000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba00ccf00 0x7f2ba00adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:51.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2ba6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 0x7f2ba00ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2ba6000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba006ef30 con 0x7f2ba015f5a0 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2ba6000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba006fa80 con 0x7f2ba0073fb0 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2ba6000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba0074420 con 0x7f2ba00ccf00 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 0x7f2ba00ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 0x7f2ba00ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33296/0 (socket says 172.21.15.71:33296) 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9fe00640 1 -- 172.21.15.71:0/1638947711 learned_addr learned my addr 172.21.15.71:0/1638947711 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9ea00640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba00ccf00 0x7f2ba00adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9f400640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0073fb0 0x7f2ba01f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9fe00640 1 -- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba00ccf00 msgr2=0x7f2ba00adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9fe00640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba00ccf00 0x7f2ba00adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9fe00640 1 -- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0073fb0 msgr2=0x7f2ba01f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.332+0000 7f2b9fe00640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0073fb0 0x7f2ba01f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.333+0000 7f2b9fe00640 1 -- 172.21.15.71:0/1638947711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba01f5290 con 0x7f2ba015f5a0 2024-09-16T07:35:51.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.333+0000 7f2b9ea00640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba00ccf00 0x7f2ba00adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:35:51.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.333+0000 7f2b9fe00640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 0x7f2ba00ae4c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c400 tx=0x7f2b9800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:51.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.333+0000 7f2b97400640 1 -- 172.21.15.71:0/1638947711 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98017070 con 0x7f2ba015f5a0 2024-09-16T07:35:51.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.333+0000 7f2b97400640 1 -- 172.21.15.71:0/1638947711 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b980053c0 con 0x7f2ba015f5a0 2024-09-16T07:35:51.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.333+0000 7f2b97400640 1 -- 172.21.15.71:0/1638947711 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98003850 con 0x7f2ba015f5a0 2024-09-16T07:35:51.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.333+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba00b2ed0 con 0x7f2ba015f5a0 2024-09-16T07:35:51.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.333+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01f8930 con 0x7f2ba015f5a0 2024-09-16T07:35:51.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.334+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba00b2440 con 0x7f2ba015f5a0 2024-09-16T07:35:51.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.335+0000 7f2b97400640 1 -- 172.21.15.71:0/1638947711 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f2b9801a070 con 0x7f2ba015f5a0 2024-09-16T07:35:51.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.336+0000 7f2b97400640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b8007a6b0 0x7f2b8007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:51.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.337+0000 7f2b97400640 1 -- 172.21.15.71:0/1638947711 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5908+0+0 (secure 0 0 0) 0x7f2b98098c90 con 0x7f2ba015f5a0 2024-09-16T07:35:51.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.337+0000 7f2b9f400640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b8007a6b0 0x7f2b8007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:51.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.338+0000 7f2b9f400640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b8007a6b0 0x7f2b8007cb70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2b90035000 tx=0x7f2b90038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:51.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.340+0000 7f2b97400640 1 -- 172.21.15.71:0/1638947711 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2b98062710 con 0x7f2ba015f5a0 2024-09-16T07:35:51.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.562+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ba0069ac0 con 0x7f2ba015f5a0 2024-09-16T07:35:51.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.563+0000 7f2b97400640 1 -- 172.21.15.71:0/1638947711 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5231 (secure 0 0 0) 0x7f2b98067140 con 0x7f2ba015f5a0 2024-09-16T07:35:51.562 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:51.563 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-16T07:35:01:329397+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1994777108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1994777108},{"type":"v1","addr":"172.21.15.71:6833","nonce":1994777108}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1399666780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1399666780},{"type":"v1","addr":"172.21.15.177:6836","nonce":1399666780}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2921631965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2921631965},{"type":"v1","addr":"172.21.15.71:6835","nonce":2921631965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:34:47.926434+0000","modified":"2024-09-16T07:34:50.958041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2539335973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2539335973},{"type":"v1","addr":"172.21.15.177:6837","nonce":2539335973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8382,"qdb_cluster":[8382]},"id":8}]} 2024-09-16T07:35:51.563 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-16T07:35:51.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b8007a6b0 msgr2=0x7f2b8007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b8007a6b0 0x7f2b8007cb70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2b90035000 tx=0x7f2b90038040 comp rx=0 tx=0).stop 2024-09-16T07:35:51.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 msgr2=0x7f2ba00ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 0x7f2ba00ae4c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c400 tx=0x7f2b9800c8c0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 shutdown_connections 2024-09-16T07:35:51.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b8007a6b0 0x7f2b8007cb70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ba015f5a0 0x7f2ba00ae4c0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba00ccf00 0x7f2ba00adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 --2- 172.21.15.71:0/1638947711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0073fb0 0x7f2ba01f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 >> 172.21.15.71:0/1638947711 conn(0x7f2ba00b1950 msgr2=0x7f2ba00ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:51.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 shutdown_connections 2024-09-16T07:35:51.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.565+0000 7f2ba6000640 1 -- 172.21.15.71:0/1638947711 wait complete. 2024-09-16T07:35:51.581 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:35:51.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 -- 172.21.15.71:0/571659898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4080ad270 msgr2=0x7fd408073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 --2- 172.21.15.71:0/571659898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4080ad270 0x7fd408073fb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd408075130 tx=0x7fd40002a980 comp rx=0 tx=0).stop 2024-09-16T07:35:51.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 -- 172.21.15.71:0/571659898 shutdown_connections 2024-09-16T07:35:51.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 --2- 172.21.15.71:0/571659898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4080ad270 0x7fd408073fb0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 --2- 172.21.15.71:0/571659898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4080ccf00 0x7fd4080acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 --2- 172.21.15.71:0/571659898 >> v2:172.21.15.177:3300/0 conn(0x7fd40815f5a0 0x7fd4080b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 -- 172.21.15.71:0/571659898 >> 172.21.15.71:0/571659898 conn(0x7fd4080b1950 msgr2=0x7fd4080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:51.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 -- 172.21.15.71:0/571659898 shutdown_connections 2024-09-16T07:35:51.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.737+0000 7fd40e600640 1 -- 172.21.15.71:0/571659898 wait complete. 2024-09-16T07:35:51.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.739+0000 7fd40e600640 1 Processor -- start 2024-09-16T07:35:51.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.739+0000 7fd40e600640 1 -- start start 2024-09-16T07:35:51.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd40e600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4080ad270 0x7fd40812a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:51.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd40e600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4080ccf00 0x7fd4081f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:51.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd40e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40815f5a0 0x7fd4080cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:51.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd40e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd40806f160 con 0x7fd40815f5a0 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd40e600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd408074780 con 0x7fd4080ad270 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd40e600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd40806f2e0 con 0x7fd4080ccf00 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd407e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40815f5a0 0x7fd4080cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd407e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40815f5a0 0x7fd4080cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33314/0 (socket says 172.21.15.71:33314) 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd407e00640 1 -- 172.21.15.71:0/1318172574 learned_addr learned my addr 172.21.15.71:0/1318172574 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd407400640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4080ad270 0x7fd40812a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.740+0000 7fd406a00640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4080ccf00 0x7fd4081f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd407e00640 1 -- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4080ccf00 msgr2=0x7fd4081f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd407e00640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4080ccf00 0x7fd4081f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd407e00640 1 -- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4080ad270 msgr2=0x7fd40812a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd407e00640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4080ad270 0x7fd40812a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd407e00640 1 -- 172.21.15.71:0/1318172574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4081f4d70 con 0x7fd40815f5a0 2024-09-16T07:35:51.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd407400640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4080ad270 0x7fd40812a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:35:51.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd406a00640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4080ccf00 0x7fd4081f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:35:51.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd407e00640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40815f5a0 0x7fd4080cfdc0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fd400009d60 tx=0x7fd400003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:51.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd3ff400640 1 -- 172.21.15.71:0/1318172574 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400034050 con 0x7fd40815f5a0 2024-09-16T07:35:51.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd3ff400640 1 -- 172.21.15.71:0/1318172574 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd400004450 con 0x7fd40815f5a0 2024-09-16T07:35:51.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd3ff400640 1 -- 172.21.15.71:0/1318172574 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400043440 con 0x7fd40815f5a0 2024-09-16T07:35:51.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.741+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4081f4fe0 con 0x7fd40815f5a0 2024-09-16T07:35:51.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.742+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4081f55a0 con 0x7fd40815f5a0 2024-09-16T07:35:51.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.742+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4080cb0e0 con 0x7fd40815f5a0 2024-09-16T07:35:51.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.744+0000 7fd3ff400640 1 -- 172.21.15.71:0/1318172574 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fd400037020 con 0x7fd40815f5a0 2024-09-16T07:35:51.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.744+0000 7fd3ff400640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3e807a6e0 0x7fd3e807cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:51.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.745+0000 7fd3ff400640 1 -- 172.21.15.71:0/1318172574 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5908+0+0 (secure 0 0 0) 0x7fd4000bab20 con 0x7fd40815f5a0 2024-09-16T07:35:51.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.745+0000 7fd407400640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3e807a6e0 0x7fd3e807cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:51.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.746+0000 7fd407400640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3e807a6e0 0x7fd3e807cba0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009f20 tx=0x7fd3f8005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:51.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.749+0000 7fd3ff400640 1 -- 172.21.15.71:0/1318172574 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd400084520 con 0x7fd40815f5a0 2024-09-16T07:35:51.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.970+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd40806f810 con 0x7fd40815f5a0 2024-09-16T07:35:51.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.970+0000 7fd3ff400640 1 -- 172.21.15.71:0/1318172574 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5231 (secure 0 0 0) 0x7fd400088f50 con 0x7fd40815f5a0 2024-09-16T07:35:51.970 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:51.970 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-16T07:35:01:329397+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1994777108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1994777108},{"type":"v1","addr":"172.21.15.71:6833","nonce":1994777108}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1399666780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1399666780},{"type":"v1","addr":"172.21.15.177:6836","nonce":1399666780}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2921631965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2921631965},{"type":"v1","addr":"172.21.15.71:6835","nonce":2921631965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:34:47.926434+0000","modified":"2024-09-16T07:34:50.958041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2539335973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2539335973},{"type":"v1","addr":"172.21.15.177:6837","nonce":2539335973}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8382,"qdb_cluster":[8382]},"id":8}]} 2024-09-16T07:35:51.970 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3e807a6e0 msgr2=0x7fd3e807cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3e807a6e0 0x7fd3e807cba0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009f20 tx=0x7fd3f8005f90 comp rx=0 tx=0).stop 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40815f5a0 msgr2=0x7fd4080cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40815f5a0 0x7fd4080cfdc0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fd400009d60 tx=0x7fd400003f10 comp rx=0 tx=0).stop 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 shutdown_connections 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd3e807a6e0 0x7fd3e807cba0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40815f5a0 0x7fd4080cfdc0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4080ccf00 0x7fd4081f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 --2- 172.21.15.71:0/1318172574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4080ad270 0x7fd40812a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:51.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 >> 172.21.15.71:0/1318172574 conn(0x7fd4080b1950 msgr2=0x7fd4080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:51.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 shutdown_connections 2024-09-16T07:35:51.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:51.973+0000 7fd40e600640 1 -- 172.21.15.71:0/1318172574 wait complete. 2024-09-16T07:35:51.989 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:35:52.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.146+0000 7f206aa00640 1 -- 172.21.15.71:0/2285158264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20640ccf00 msgr2=0x7f20640b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.146+0000 7f206aa00640 1 --2- 172.21.15.71:0/2285158264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20640ccf00 0x7f20640b2440 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2050005fd0 tx=0x7f205002a5f0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.146+0000 7f206aa00640 1 -- 172.21.15.71:0/2285158264 shutdown_connections 2024-09-16T07:35:52.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.146+0000 7f206aa00640 1 --2- 172.21.15.71:0/2285158264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20640ccf00 0x7f20640b2440 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.146+0000 7f206aa00640 1 -- 172.21.15.71:0/2285158264 >> 172.21.15.71:0/2285158264 conn(0x7f20640b1950 msgr2=0x7f20640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:52.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.146+0000 7f206aa00640 1 -- 172.21.15.71:0/2285158264 shutdown_connections 2024-09-16T07:35:52.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.146+0000 7f206aa00640 1 -- 172.21.15.71:0/2285158264 wait complete. 2024-09-16T07:35:52.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.148+0000 7f206aa00640 1 Processor -- start 2024-09-16T07:35:52.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.148+0000 7f206aa00640 1 -- start start 2024-09-16T07:35:52.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f206aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2064073fb0 0x7f20641f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f206aa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20640ccf00 0x7f2064156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f206aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f206415f5a0 0x7f20641565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f206aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f206406f9f0 con 0x7f206415f5a0 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f206aa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f206406f870 con 0x7f2064073fb0 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f206aa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2064074420 con 0x7f20640ccf00 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2068a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f206415f5a0 0x7f20641565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2068a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f206415f5a0 0x7f20641565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33332/0 (socket says 172.21.15.71:33332) 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2068a00640 1 -- 172.21.15.71:0/101201347 learned_addr learned my addr 172.21.15.71:0/101201347 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2063e00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2064073fb0 0x7f20641f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2063400640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20640ccf00 0x7f2064156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2068a00640 1 -- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20640ccf00 msgr2=0x7f2064156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2068a00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20640ccf00 0x7f2064156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2068a00640 1 -- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2064073fb0 msgr2=0x7f20641f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2068a00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2064073fb0 0x7f20641f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.149+0000 7f2068a00640 1 -- 172.21.15.71:0/101201347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20641f3520 con 0x7f206415f5a0 2024-09-16T07:35:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.150+0000 7f2068a00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f206415f5a0 0x7f20641565d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f205c00c7a0 tx=0x7f205c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:52.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.150+0000 7f205be00640 1 -- 172.21.15.71:0/101201347 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c003da0 con 0x7f206415f5a0 2024-09-16T07:35:52.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.150+0000 7f205be00640 1 -- 172.21.15.71:0/101201347 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f205c005070 con 0x7f206415f5a0 2024-09-16T07:35:52.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.150+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20640cfdd0 con 0x7f206415f5a0 2024-09-16T07:35:52.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.150+0000 7f205be00640 1 -- 172.21.15.71:0/101201347 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c0054c0 con 0x7f206415f5a0 2024-09-16T07:35:52.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.150+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20640d0390 con 0x7f206415f5a0 2024-09-16T07:35:52.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.151+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20640b2440 con 0x7f206415f5a0 2024-09-16T07:35:52.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.152+0000 7f205be00640 1 -- 172.21.15.71:0/101201347 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f205c005660 con 0x7f206415f5a0 2024-09-16T07:35:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.154+0000 7f205be00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f204407a6b0 0x7f204407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.154+0000 7f205be00640 1 -- 172.21.15.71:0/101201347 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5908+0+0 (secure 0 0 0) 0x7f205c099850 con 0x7f206415f5a0 2024-09-16T07:35:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.154+0000 7f2063e00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f204407a6b0 0x7f204407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.155+0000 7f2063e00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f204407a6b0 0x7f204407cb70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2054000f60 tx=0x7f2054009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:52.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.158+0000 7f205be00640 1 -- 172.21.15.71:0/101201347 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f205c063920 con 0x7f206415f5a0 2024-09-16T07:35:52.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.344+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20640b1520 con 0x7f206415f5a0 2024-09-16T07:35:52.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.345+0000 7f205be00640 1 -- 172.21.15.71:0/101201347 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+17086 (secure 0 0 0) 0x7f205c09ae20 con 0x7f206415f5a0 2024-09-16T07:35:52.345 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:52.345 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":122,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:35:48.532744+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:34:45.706284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T07:34:46.892492+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":122,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.8","mappings":[{"from":4,"to":3}]},{"pgid":"17.b","mappings":[{"from":6,"to":1}]},{"pgid":"17.c","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:35:52.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.347+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f204407a6b0 msgr2=0x7f204407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.347+0000 7f206aa00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f204407a6b0 0x7f204407cb70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2054000f60 tx=0x7f2054009040 comp rx=0 tx=0).stop 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.347+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f206415f5a0 msgr2=0x7f20641565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.347+0000 7f206aa00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f206415f5a0 0x7f20641565d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f205c00c7a0 tx=0x7f205c00cc60 comp rx=0 tx=0).stop 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.348+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 shutdown_connections 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.348+0000 7f206aa00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f204407a6b0 0x7f204407cb70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.348+0000 7f206aa00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f206415f5a0 0x7f20641565d0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.348+0000 7f206aa00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20640ccf00 0x7f2064156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.348+0000 7f206aa00640 1 --2- 172.21.15.71:0/101201347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2064073fb0 0x7f20641f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.348+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 >> 172.21.15.71:0/101201347 conn(0x7f20640b1950 msgr2=0x7f2064075b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.348+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 shutdown_connections 2024-09-16T07:35:52.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.348+0000 7f206aa00640 1 -- 172.21.15.71:0/101201347 wait complete. 2024-09-16T07:35:52.364 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:35:52.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.522+0000 7f855b000640 1 -- 172.21.15.71:0/403174554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8554161550 msgr2=0x7f855406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.522+0000 7f855b000640 1 --2- 172.21.15.71:0/403174554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8554161550 0x7f855406ef50 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8544005fd0 tx=0x7f854402a970 comp rx=0 tx=0).stop 2024-09-16T07:35:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.522+0000 7f855b000640 1 -- 172.21.15.71:0/403174554 shutdown_connections 2024-09-16T07:35:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.522+0000 7f855b000640 1 --2- 172.21.15.71:0/403174554 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85540af3d0 0x7f855406f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.522+0000 7f855b000640 1 --2- 172.21.15.71:0/403174554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8554161550 0x7f855406ef50 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.522+0000 7f855b000640 1 -- 172.21.15.71:0/403174554 >> 172.21.15.71:0/403174554 conn(0x7f85540b1950 msgr2=0x7f85540b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.523+0000 7f855b000640 1 -- 172.21.15.71:0/403174554 shutdown_connections 2024-09-16T07:35:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.523+0000 7f855b000640 1 -- 172.21.15.71:0/403174554 wait complete. 2024-09-16T07:35:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.524+0000 7f855b000640 1 Processor -- start 2024-09-16T07:35:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.525+0000 7f855b000640 1 -- start start 2024-09-16T07:35:52.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.525+0000 7f855b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855406f9d0 0x7f85541590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.525+0000 7f855b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540af3d0 0x7f85540adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.525+0000 7f855b000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8554161550 0x7f8554156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.525+0000 7f855b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f85541623a0 con 0x7f855406f9d0 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.525+0000 7f855b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8554074010 con 0x7f85540af3d0 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f855b000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8554162520 con 0x7f8554161550 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855406f9d0 0x7f85541590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855406f9d0 0x7f85541590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33356/0 (socket says 172.21.15.71:33356) 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553e00640 1 -- 172.21.15.71:0/472351065 learned_addr learned my addr 172.21.15.71:0/472351065 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553400640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540af3d0 0x7f85540adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8559000640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8554161550 0x7f8554156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553e00640 1 -- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8554161550 msgr2=0x7f8554156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553e00640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8554161550 0x7f8554156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553e00640 1 -- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540af3d0 msgr2=0x7f85540adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553e00640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540af3d0 0x7f85540adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.526+0000 7f8553e00640 1 -- 172.21.15.71:0/472351065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85541599d0 con 0x7f855406f9d0 2024-09-16T07:35:52.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.527+0000 7f8553e00640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855406f9d0 0x7f85541590c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f8544006010 tx=0x7f8544003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:52.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.527+0000 7f854be00640 1 -- 172.21.15.71:0/472351065 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544034040 con 0x7f855406f9d0 2024-09-16T07:35:52.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.527+0000 7f854be00640 1 -- 172.21.15.71:0/472351065 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85440045c0 con 0x7f855406f9d0 2024-09-16T07:35:52.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.527+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85541fca90 con 0x7f855406f9d0 2024-09-16T07:35:52.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.527+0000 7f854be00640 1 -- 172.21.15.71:0/472351065 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544045480 con 0x7f855406f9d0 2024-09-16T07:35:52.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.528+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85541fd050 con 0x7f855406f9d0 2024-09-16T07:35:52.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.529+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85540760b0 con 0x7f855406f9d0 2024-09-16T07:35:52.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.530+0000 7f854be00640 1 -- 172.21.15.71:0/472351065 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f8544037020 con 0x7f855406f9d0 2024-09-16T07:35:52.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.532+0000 7f854be00640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f853407a6b0 0x7f853407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.532+0000 7f854be00640 1 -- 172.21.15.71:0/472351065 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5908+0+0 (secure 0 0 0) 0x7f85440bafc0 con 0x7f855406f9d0 2024-09-16T07:35:52.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.532+0000 7f8553400640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f853407a6b0 0x7f853407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.533+0000 7f8553400640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f853407a6b0 0x7f853407cb70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8540005fd0 tx=0x7f854000b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:52.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.536+0000 7f854be00640 1 -- 172.21.15.71:0/472351065 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8544084ae0 con 0x7f855406f9d0 2024-09-16T07:35:52.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.759+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f85540b1730 con 0x7f855406f9d0 2024-09-16T07:35:52.759 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:35:52.760+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:35:52.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.762+0000 7f854be00640 1 -- 172.21.15.71:0/472351065 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7f8544089510 con 0x7f855406f9d0 2024-09-16T07:35:52.762 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:35:52.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f853407a6b0 msgr2=0x7f853407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f853407a6b0 0x7f853407cb70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8540005fd0 tx=0x7f854000b390 comp rx=0 tx=0).stop 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855406f9d0 msgr2=0x7f85541590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855406f9d0 0x7f85541590c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f8544006010 tx=0x7f8544003f10 comp rx=0 tx=0).stop 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 shutdown_connections 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f853407a6b0 0x7f853407cb70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8554161550 0x7f8554156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540af3d0 0x7f85540adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 --2- 172.21.15.71:0/472351065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855406f9d0 0x7f85541590c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 >> 172.21.15.71:0/472351065 conn(0x7f85540b1950 msgr2=0x7f855415ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:52.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 shutdown_connections 2024-09-16T07:35:52.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.765+0000 7f855b000640 1 -- 172.21.15.71:0/472351065 wait complete. 2024-09-16T07:35:52.781 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:35:52.834 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:35:52.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.941+0000 7ff30e400640 1 -- 172.21.15.71:0/4291823162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2ec003a00 msgr2=0x7ff2ec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.941+0000 7ff30e400640 1 --2- 172.21.15.71:0/4291823162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2ec003a00 0x7ff2ec003e20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800c290 tx=0x7ff2ec005830 comp rx=0 tx=0).stop 2024-09-16T07:35:52.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.941+0000 7ff307e00640 1 -- 172.21.15.71:0/4291823162 reap_dead start 2024-09-16T07:35:52.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.941+0000 7ff30e400640 1 -- 172.21.15.71:0/4291823162 shutdown_connections 2024-09-16T07:35:52.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.941+0000 7ff30e400640 1 -- 172.21.15.71:0/4291823162 >> 172.21.15.71:0/4291823162 conn(0x7ff3080b1840 msgr2=0x7ff3080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:52.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.941+0000 7ff30e400640 1 -- 172.21.15.71:0/4291823162 shutdown_connections 2024-09-16T07:35:52.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.941+0000 7ff30e400640 1 -- 172.21.15.71:0/4291823162 wait complete. 2024-09-16T07:35:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.943+0000 7ff30e400640 1 Processor -- start 2024-09-16T07:35:52.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.943+0000 7ff30e400640 1 -- start start 2024-09-16T07:35:52.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff30e400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2ec003a00 0x7ff3080cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff30e400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30812d8f0 0x7ff308156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff30e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff308151250 0x7ff3080ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff30e400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3080737d0 con 0x7ff308151250 2024-09-16T07:35:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff30e400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff308073650 con 0x7ff30812d8f0 2024-09-16T07:35:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff30e400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3080ce0f0 con 0x7ff2ec003a00 2024-09-16T07:35:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff308151250 0x7ff3080ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff308151250 0x7ff3080ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33374/0 (socket says 172.21.15.71:33374) 2024-09-16T07:35:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307e00640 1 -- 172.21.15.71:0/2897453354 learned_addr learned my addr 172.21.15.71:0/2897453354 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff306a00640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30812d8f0 0x7ff308156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2ec003a00 0x7ff3080cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307e00640 1 -- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2ec003a00 msgr2=0x7ff3080cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307e00640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2ec003a00 0x7ff3080cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307e00640 1 -- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30812d8f0 msgr2=0x7ff308156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307e00640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30812d8f0 0x7ff308156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.944+0000 7ff307e00640 1 -- 172.21.15.71:0/2897453354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3080ccb00 con 0x7ff308151250 2024-09-16T07:35:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.945+0000 7ff307e00640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff308151250 0x7ff3080ce600 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff3080cfba0 tx=0x7ff30000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.945+0000 7ff2ff400640 1 -- 172.21.15.71:0/2897453354 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4009860 con 0x7ff308151250 2024-09-16T07:35:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.945+0000 7ff2ff400640 1 -- 172.21.15.71:0/2897453354 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff300003d80 con 0x7ff308151250 2024-09-16T07:35:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.945+0000 7ff2ff400640 1 -- 172.21.15.71:0/2897453354 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300004d80 con 0x7ff308151250 2024-09-16T07:35:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.945+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff308153410 con 0x7ff308151250 2024-09-16T07:35:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.945+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff308153990 con 0x7ff308151250 2024-09-16T07:35:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.946+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3080cdbb0 con 0x7ff308151250 2024-09-16T07:35:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.947+0000 7ff2ff400640 1 -- 172.21.15.71:0/2897453354 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7ff300004030 con 0x7ff308151250 2024-09-16T07:35:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.948+0000 7ff2ff400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2e807a6b0 0x7ff2e807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.948+0000 7ff2ff400640 1 -- 172.21.15.71:0/2897453354 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5966+0+0 (secure 0 0 0) 0x7ff30009a460 con 0x7ff308151250 2024-09-16T07:35:52.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.949+0000 7ff307400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2e807a6b0 0x7ff2e807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:52.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.949+0000 7ff307400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2e807a6b0 0x7ff2e807cb70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8033000 tx=0x7ff2f8038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:52.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:52.952+0000 7ff2ff400640 1 -- 172.21.15.71:0/2897453354 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff300063ee0 con 0x7ff308151250 2024-09-16T07:35:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.173+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff3080cf740 con 0x7ff308151250 2024-09-16T07:35:53.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.764+0000 7ff2ff400640 1 -- 172.21.15.71:0/2897453354 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7ff300068910 con 0x7ff308151250 2024-09-16T07:35:53.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2e807a6b0 msgr2=0x7ff2e807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2e807a6b0 0x7ff2e807cb70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8033000 tx=0x7ff2f8038040 comp rx=0 tx=0).stop 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff308151250 msgr2=0x7ff3080ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff308151250 0x7ff3080ce600 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff3080cfba0 tx=0x7ff30000f790 comp rx=0 tx=0).stop 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 shutdown_connections 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2e807a6b0 0x7ff2e807cb70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff308151250 0x7ff3080ce600 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30812d8f0 0x7ff308156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 --2- 172.21.15.71:0/2897453354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2ec003a00 0x7ff3080cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 >> 172.21.15.71:0/2897453354 conn(0x7ff3080b1840 msgr2=0x7ff3080a9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 shutdown_connections 2024-09-16T07:35:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.767+0000 7ff30e400640 1 -- 172.21.15.71:0/2897453354 wait complete. 2024-09-16T07:35:53.784 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:35:53.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.946+0000 7f730ac00640 1 -- 172.21.15.71:0/2393072965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304070a50 msgr2=0x7f7304160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:53.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.946+0000 7f730ac00640 1 --2- 172.21.15.71:0/2393072965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304070a50 0x7f7304160280 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005fd0 tx=0x7f72ec02a540 comp rx=0 tx=0).stop 2024-09-16T07:35:53.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.946+0000 7f730ac00640 1 -- 172.21.15.71:0/2393072965 shutdown_connections 2024-09-16T07:35:53.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.946+0000 7f730ac00640 1 --2- 172.21.15.71:0/2393072965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73040cce30 0x7f730415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:53.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.946+0000 7f730ac00640 1 --2- 172.21.15.71:0/2393072965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304070a50 0x7f7304160280 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:53.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.946+0000 7f730ac00640 1 -- 172.21.15.71:0/2393072965 >> 172.21.15.71:0/2393072965 conn(0x7f730406cb70 msgr2=0x7f730406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:53.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.946+0000 7f730ac00640 1 -- 172.21.15.71:0/2393072965 shutdown_connections 2024-09-16T07:35:53.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.946+0000 7f730ac00640 1 -- 172.21.15.71:0/2393072965 wait complete. 2024-09-16T07:35:53.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.947+0000 7f730ac00640 1 Processor -- start 2024-09-16T07:35:53.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.948+0000 7f730ac00640 1 -- start start 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.948+0000 7f730ac00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7304070a50 0x7f7304153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.948+0000 7f730ac00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73040cce30 0x7f7304153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f730ac00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304156460 0x7f730415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f730ac00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f730415cba0 con 0x7f7304156460 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f730ac00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f730415ca20 con 0x7f7304070a50 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f730ac00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f730415cd20 con 0x7f73040cce30 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7308c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304156460 0x7f730415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7308c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304156460 0x7f730415d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33386/0 (socket says 172.21.15.71:33386) 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7308c00640 1 -- 172.21.15.71:0/904023276 learned_addr learned my addr 172.21.15.71:0/904023276 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:53.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7303e00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7304070a50 0x7f7304153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:53.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f72fbe00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73040cce30 0x7f7304153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:53.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7308c00640 1 -- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73040cce30 msgr2=0x7f7304153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:53.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7308c00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73040cce30 0x7f7304153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:53.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7308c00640 1 -- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7304070a50 msgr2=0x7f7304153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:53.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7308c00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7304070a50 0x7f7304153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:53.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.949+0000 7f7308c00640 1 -- 172.21.15.71:0/904023276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7304150920 con 0x7f7304156460 2024-09-16T07:35:53.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.950+0000 7f7308c00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304156460 0x7f730415d450 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00eaa0 tx=0x7f72fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:53.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.950+0000 7f7300c00640 1 -- 172.21.15.71:0/904023276 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00cdb0 con 0x7f7304156460 2024-09-16T07:35:53.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.950+0000 7f7300c00640 1 -- 172.21.15.71:0/904023276 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72fc004c10 con 0x7f7304156460 2024-09-16T07:35:53.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.950+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7304150c00 con 0x7f7304156460 2024-09-16T07:35:53.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.950+0000 7f7300c00640 1 -- 172.21.15.71:0/904023276 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc005020 con 0x7f7304156460 2024-09-16T07:35:53.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.950+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73041511c0 con 0x7f7304156460 2024-09-16T07:35:53.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.951+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730415d240 con 0x7f7304156460 2024-09-16T07:35:53.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.952+0000 7f7300c00640 1 -- 172.21.15.71:0/904023276 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f72fc0051c0 con 0x7f7304156460 2024-09-16T07:35:53.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.953+0000 7f7300c00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f72e007a6b0 0x7f72e007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:53.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.953+0000 7f7300c00640 1 -- 172.21.15.71:0/904023276 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5966+0+0 (secure 0 0 0) 0x7f72fc09a290 con 0x7f7304156460 2024-09-16T07:35:53.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.954+0000 7f7303e00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f72e007a6b0 0x7f72e007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:53.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.954+0000 7f7303e00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f72e007a6b0 0x7f72e007cb70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f73040b0070 tx=0x7f72f4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:53.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:53.957+0000 7f7300c00640 1 -- 172.21.15.71:0/904023276 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f72fc016030 con 0x7f7304156460 2024-09-16T07:35:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.141+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f73040d1100 con 0x7f7304156460 2024-09-16T07:35:54.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.768+0000 7f7300c00640 1 -- 172.21.15.71:0/904023276 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) ==== 181+0+0 (secure 0 0 0) 0x7f72fc063d10 con 0x7f7304156460 2024-09-16T07:35:54.767 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:35:54.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.770+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f72e007a6b0 msgr2=0x7f72e007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:54.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.770+0000 7f730ac00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f72e007a6b0 0x7f72e007cb70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f73040b0070 tx=0x7f72f4009040 comp rx=0 tx=0).stop 2024-09-16T07:35:54.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.770+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304156460 msgr2=0x7f730415d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:54.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.770+0000 7f730ac00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304156460 0x7f730415d450 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00eaa0 tx=0x7f72fc00ef60 comp rx=0 tx=0).stop 2024-09-16T07:35:54.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.771+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 shutdown_connections 2024-09-16T07:35:54.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.771+0000 7f730ac00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f72e007a6b0 0x7f72e007cb70 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:54.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.771+0000 7f730ac00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7304156460 0x7f730415d450 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:54.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.771+0000 7f730ac00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73040cce30 0x7f7304153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:54.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.771+0000 7f730ac00640 1 --2- 172.21.15.71:0/904023276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7304070a50 0x7f7304153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:54.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.771+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 >> 172.21.15.71:0/904023276 conn(0x7f730406cb70 msgr2=0x7f730415a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:54.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.771+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 shutdown_connections 2024-09-16T07:35:54.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.771+0000 7f730ac00640 1 -- 172.21.15.71:0/904023276 wait complete. 2024-09-16T07:35:54.790 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:35:54.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.952+0000 7f2ee5400640 1 -- 172.21.15.71:0/4077739920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00d0500 msgr2=0x7f2ee00d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:54.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.952+0000 7f2ee5400640 1 --2- 172.21.15.71:0/4077739920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00d0500 0x7f2ee00d0990 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f2ee006fa90 tx=0x7f2ed802aa30 comp rx=0 tx=0).stop 2024-09-16T07:35:54.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.952+0000 7f2ee5400640 1 -- 172.21.15.71:0/4077739920 shutdown_connections 2024-09-16T07:35:54.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.952+0000 7f2ee5400640 1 --2- 172.21.15.71:0/4077739920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00d0500 0x7f2ee00d0990 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:54.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.952+0000 7f2ee5400640 1 --2- 172.21.15.71:0/4077739920 >> v2:172.21.15.177:3300/0 conn(0x7f2ee015dfb0 0x7f2ee00d4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:54.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.952+0000 7f2ee5400640 1 -- 172.21.15.71:0/4077739920 >> 172.21.15.71:0/4077739920 conn(0x7f2ee009d180 msgr2=0x7f2ee00d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:54.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.952+0000 7f2ee5400640 1 -- 172.21.15.71:0/4077739920 shutdown_connections 2024-09-16T07:35:54.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.952+0000 7f2ee5400640 1 -- 172.21.15.71:0/4077739920 wait complete. 2024-09-16T07:35:54.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.953+0000 7f2ee5400640 1 Processor -- start 2024-09-16T07:35:54.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.954+0000 7f2ee5400640 1 -- start start 2024-09-16T07:35:54.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.954+0000 7f2ee5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00cf470 0x7f2ee00ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:54.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.954+0000 7f2ee5400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee00d0500 0x7f2ee0149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:54.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.954+0000 7f2ee5400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee015dfb0 0x7f2ee0157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:54.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.954+0000 7f2ee5400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee00b6e90 con 0x7f2ee00cf470 2024-09-16T07:35:54.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.954+0000 7f2ee5400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee00b6d10 con 0x7f2ee015dfb0 2024-09-16T07:35:54.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.954+0000 7f2ee5400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee00b7010 con 0x7f2ee00d0500 2024-09-16T07:35:54.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00cf470 0x7f2ee00ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:54.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00cf470 0x7f2ee00ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35640/0 (socket says 172.21.15.71:35640) 2024-09-16T07:35:54.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 -- 172.21.15.71:0/1648173248 learned_addr learned my addr 172.21.15.71:0/1648173248 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:54.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2ee4a00640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee015dfb0 0x7f2ee0157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:54.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 -- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee00d0500 msgr2=0x7f2ee0149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:54.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee00d0500 0x7f2ee0149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:54.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 -- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee015dfb0 msgr2=0x7f2ee0157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:54.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee015dfb0 0x7f2ee0157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:54.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 -- 172.21.15.71:0/1648173248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee00cde10 con 0x7f2ee00cf470 2024-09-16T07:35:54.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2ee4a00640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee015dfb0 0x7f2ee0157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:35:54.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.955+0000 7f2edfe00640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00cf470 0x7f2ee00ccdd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000b410 tx=0x7f2ed000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:54.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.956+0000 7f2edcc00640 1 -- 172.21.15.71:0/1648173248 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0015020 con 0x7f2ee00cf470 2024-09-16T07:35:54.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.956+0000 7f2edcc00640 1 -- 172.21.15.71:0/1648173248 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ed0011940 con 0x7f2ee00cf470 2024-09-16T07:35:54.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.956+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee00cd570 con 0x7f2ee00cf470 2024-09-16T07:35:54.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.956+0000 7f2edcc00640 1 -- 172.21.15.71:0/1648173248 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0011d90 con 0x7f2ee00cf470 2024-09-16T07:35:54.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.956+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee00cdb30 con 0x7f2ee00cf470 2024-09-16T07:35:54.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.957+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee00b7530 con 0x7f2ee00cf470 2024-09-16T07:35:54.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.958+0000 7f2edcc00640 1 -- 172.21.15.71:0/1648173248 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f2ed0018020 con 0x7f2ee00cf470 2024-09-16T07:35:54.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.959+0000 7f2edcc00640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a6b0 0x7f2ebc07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:54.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.959+0000 7f2edcc00640 1 -- 172.21.15.71:0/1648173248 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5499+0+0 (secure 0 0 0) 0x7f2ed0099950 con 0x7f2ee00cf470 2024-09-16T07:35:54.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.959+0000 7f2ed7e00640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a6b0 0x7f2ebc07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:54.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.960+0000 7f2ed7e00640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a6b0 0x7f2ebc07cb70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2ee015b070 tx=0x7f2ec8007990 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:54.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:54.963+0000 7f2edcc00640 1 -- 172.21.15.71:0/1648173248 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ed001f020 con 0x7f2ee00cf470 2024-09-16T07:35:55.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.149+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2ee008df30 con 0x7f2ee00cf470 2024-09-16T07:35:55.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.771+0000 7f2edcc00640 1 -- 172.21.15.71:0/1648173248 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) ==== 169+0+0 (secure 0 0 0) 0x7f2ed00639a0 con 0x7f2ee00cf470 2024-09-16T07:35:55.771 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:35:55.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a6b0 msgr2=0x7f2ebc07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:55.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a6b0 0x7f2ebc07cb70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2ee015b070 tx=0x7f2ec8007990 comp rx=0 tx=0).stop 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00cf470 msgr2=0x7f2ee00ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00cf470 0x7f2ee00ccdd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000b410 tx=0x7f2ed000b8d0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 shutdown_connections 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ebc07a6b0 0x7f2ebc07cb70 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee015dfb0 0x7f2ee0157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee00d0500 0x7f2ee0149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 --2- 172.21.15.71:0/1648173248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ee00cf470 0x7f2ee00ccdd0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 >> 172.21.15.71:0/1648173248 conn(0x7f2ee009d180 msgr2=0x7f2ee00b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 shutdown_connections 2024-09-16T07:35:55.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.774+0000 7f2ee5400640 1 -- 172.21.15.71:0/1648173248 wait complete. 2024-09-16T07:35:55.794 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:35:55.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.964+0000 7f9b67e00640 1 -- 172.21.15.71:0/3251237280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b600ccf00 msgr2=0x7f9b600b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:55.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.964+0000 7f9b67e00640 1 --2- 172.21.15.71:0/3251237280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b600ccf00 0x7f9b600b2440 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f9b6006dac0 tx=0x7f9b4c02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.964+0000 7f9b67e00640 1 -- 172.21.15.71:0/3251237280 shutdown_connections 2024-09-16T07:35:55.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.964+0000 7f9b67e00640 1 --2- 172.21.15.71:0/3251237280 >> v2:172.21.15.177:3300/0 conn(0x7f9b60073fb0 0x7f9b60074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.964+0000 7f9b67e00640 1 --2- 172.21.15.71:0/3251237280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b600ccf00 0x7f9b600b2440 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.964+0000 7f9b67e00640 1 -- 172.21.15.71:0/3251237280 >> 172.21.15.71:0/3251237280 conn(0x7f9b600b1950 msgr2=0x7f9b600b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:55.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.964+0000 7f9b67e00640 1 -- 172.21.15.71:0/3251237280 shutdown_connections 2024-09-16T07:35:55.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.965+0000 7f9b67e00640 1 -- 172.21.15.71:0/3251237280 wait complete. 2024-09-16T07:35:55.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.966+0000 7f9b67e00640 1 Processor -- start 2024-09-16T07:35:55.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.966+0000 7f9b67e00640 1 -- start start 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.967+0000 7f9b67e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b60073fb0 0x7f9b6014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.967+0000 7f9b67e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b600ccf00 0x7f9b600ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.967+0000 7f9b67e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b6015f5a0 0x7f9b6009cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.967+0000 7f9b67e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6006f2e0 con 0x7f9b60073fb0 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.967+0000 7f9b67e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6006f160 con 0x7f9b6015f5a0 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.967+0000 7f9b67e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6006f460 con 0x7f9b600ccf00 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.967+0000 7f9b65400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b60073fb0 0x7f9b6014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b60073fb0 0x7f9b6014e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35670/0 (socket says 172.21.15.71:35670) 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65400640 1 -- 172.21.15.71:0/424484945 learned_addr learned my addr 172.21.15.71:0/424484945 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:55.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65e00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b6015f5a0 0x7f9b6009cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b64a00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b600ccf00 0x7f9b600ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65400640 1 -- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b600ccf00 msgr2=0x7f9b600ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65400640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b600ccf00 0x7f9b600ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65400640 1 -- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b6015f5a0 msgr2=0x7f9b6009cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65400640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b6015f5a0 0x7f9b6009cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65400640 1 -- 172.21.15.71:0/424484945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b600c9c30 con 0x7f9b60073fb0 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b65400640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b60073fb0 0x7f9b6014e750 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400ba80 tx=0x7f9b5400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b5d600640 1 -- 172.21.15.71:0/424484945 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400cc00 con 0x7f9b60073fb0 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b5d600640 1 -- 172.21.15.71:0/424484945 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b54003970 con 0x7f9b60073fb0 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.968+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b600c9f10 con 0x7f9b60073fb0 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.969+0000 7f9b5d600640 1 -- 172.21.15.71:0/424484945 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54003e00 con 0x7f9b60073fb0 2024-09-16T07:35:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.969+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b600ca4d0 con 0x7f9b60073fb0 2024-09-16T07:35:55.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.970+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6006f9f0 con 0x7f9b60073fb0 2024-09-16T07:35:55.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.971+0000 7f9b5d600640 1 -- 172.21.15.71:0/424484945 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f9b54004000 con 0x7f9b60073fb0 2024-09-16T07:35:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.972+0000 7f9b5d600640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b4007a6e0 0x7f9b4007cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.972+0000 7f9b5d600640 1 -- 172.21.15.71:0/424484945 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7f9b54099780 con 0x7f9b60073fb0 2024-09-16T07:35:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.972+0000 7f9b64a00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b4007a6e0 0x7f9b4007cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.973+0000 7f9b64a00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b4007a6e0 0x7f9b4007cba0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c033000 tx=0x7f9b4c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:55.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:55.976+0000 7f9b5d600640 1 -- 172.21.15.71:0/424484945 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9b540638f0 con 0x7f9b60073fb0 2024-09-16T07:35:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.162+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f9b600b15b0 con 0x7f9b60073fb0 2024-09-16T07:35:56.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.166+0000 7f9b5d600640 1 -- 172.21.15.71:0/424484945 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7f9b540631f0 con 0x7f9b60073fb0 2024-09-16T07:35:56.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b4007a6e0 msgr2=0x7f9b4007cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b4007a6e0 0x7f9b4007cba0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c033000 tx=0x7f9b4c004730 comp rx=0 tx=0).stop 2024-09-16T07:35:56.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b60073fb0 msgr2=0x7f9b6014e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b60073fb0 0x7f9b6014e750 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400ba80 tx=0x7f9b5400bf40 comp rx=0 tx=0).stop 2024-09-16T07:35:56.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 shutdown_connections 2024-09-16T07:35:56.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9b4007a6e0 0x7f9b4007cba0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b6015f5a0 0x7f9b6009cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b600ccf00 0x7f9b600ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 --2- 172.21.15.71:0/424484945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b60073fb0 0x7f9b6014e750 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.168+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 >> 172.21.15.71:0/424484945 conn(0x7f9b600b1950 msgr2=0x7f9b6015cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.169+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 shutdown_connections 2024-09-16T07:35:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.169+0000 7f9b67e00640 1 -- 172.21.15.71:0/424484945 wait complete. 2024-09-16T07:35:56.184 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-16T07:35:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.343+0000 7fedde800640 1 -- 172.21.15.71:0/1286368668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedc0004080 msgr2=0x7fedc0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.343+0000 7fedde800640 1 --2- 172.21.15.71:0/1286368668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedc0004080 0x7fedc0004800 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fedc400baf0 tx=0x7fedc0005830 comp rx=0 tx=0).stop 2024-09-16T07:35:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.343+0000 7feddde00640 1 -- 172.21.15.71:0/1286368668 reap_dead start 2024-09-16T07:35:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.343+0000 7fedde800640 1 -- 172.21.15.71:0/1286368668 shutdown_connections 2024-09-16T07:35:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.343+0000 7fedde800640 1 -- 172.21.15.71:0/1286368668 >> 172.21.15.71:0/1286368668 conn(0x7fedd80cfa20 msgr2=0x7fedd80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.343+0000 7fedde800640 1 -- 172.21.15.71:0/1286368668 shutdown_connections 2024-09-16T07:35:56.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.343+0000 7fedde800640 1 -- 172.21.15.71:0/1286368668 wait complete. 2024-09-16T07:35:56.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.345+0000 7fedde800640 1 Processor -- start 2024-09-16T07:35:56.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.345+0000 7fedde800640 1 -- start start 2024-09-16T07:35:56.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7fedde800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc0004080 0x7fedd81f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:56.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7fedde800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd80c6af0 0x7fedd80b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:56.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7fedde800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd80c80f0 0x7fedd80c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7fedde800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fedd80b2c80 con 0x7fedd80c80f0 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7fedde800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fedd80b2b00 con 0x7fedc0004080 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7fedde800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fedd81f6c60 con 0x7fedd80c6af0 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd80c80f0 0x7fedd80c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd80c80f0 0x7fedd80c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35688/0 (socket says 172.21.15.71:35688) 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddde00640 1 -- 172.21.15.71:0/670237799 learned_addr learned my addr 172.21.15.71:0/670237799 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddd400640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc0004080 0x7fedd81f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddca00640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd80c6af0 0x7fedd80b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddde00640 1 -- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd80c6af0 msgr2=0x7fedd80b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddde00640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd80c6af0 0x7fedd80b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddde00640 1 -- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc0004080 msgr2=0x7fedd81f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddde00640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc0004080 0x7fedd81f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.346+0000 7feddde00640 1 -- 172.21.15.71:0/670237799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd80c7070 con 0x7fedd80c80f0 2024-09-16T07:35:56.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.347+0000 7feddde00640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd80c80f0 0x7fedd80c56f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fedd0000fb0 tx=0x7fedd00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:56.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.347+0000 7fedd5600640 1 -- 172.21.15.71:0/670237799 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0015070 con 0x7fedd80c80f0 2024-09-16T07:35:56.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.347+0000 7fedd5600640 1 -- 172.21.15.71:0/670237799 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedd0011880 con 0x7fedd80c80f0 2024-09-16T07:35:56.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.347+0000 7fedd5600640 1 -- 172.21.15.71:0/670237799 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0011cd0 con 0x7fedd80c80f0 2024-09-16T07:35:56.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.347+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd80c7350 con 0x7fedd80c80f0 2024-09-16T07:35:56.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.347+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd80c7910 con 0x7fedd80c80f0 2024-09-16T07:35:56.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.348+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd8074570 con 0x7fedd80c80f0 2024-09-16T07:35:56.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.349+0000 7fedd5600640 1 -- 172.21.15.71:0/670237799 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fedd0018070 con 0x7fedd80c80f0 2024-09-16T07:35:56.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.350+0000 7fedd5600640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb807a6b0 0x7fedb807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:56.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.350+0000 7fedd5600640 1 -- 172.21.15.71:0/670237799 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7fedd009a4e0 con 0x7fedd80c80f0 2024-09-16T07:35:56.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.351+0000 7feddd400640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb807a6b0 0x7fedb807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:56.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.352+0000 7feddd400640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb807a6b0 0x7fedb807cb70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fedcc005fd0 tx=0x7fedcc009530 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:56.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.355+0000 7fedd5600640 1 -- 172.21.15.71:0/670237799 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fedd0063f60 con 0x7fedd80c80f0 2024-09-16T07:35:56.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.607+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7fedd806ad20 con 0x7fedd80c80f0 2024-09-16T07:35:56.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.784+0000 7fedd5600640 1 -- 172.21.15.71:0/670237799 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v615) ==== 185+0+0 (secure 0 0 0) 0x7fedd0068990 con 0x7fedd80c80f0 2024-09-16T07:35:56.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb807a6b0 msgr2=0x7fedb807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb807a6b0 0x7fedb807cb70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fedcc005fd0 tx=0x7fedcc009530 comp rx=0 tx=0).stop 2024-09-16T07:35:56.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd80c80f0 msgr2=0x7fedd80c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd80c80f0 0x7fedd80c56f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fedd0000fb0 tx=0x7fedd00039c0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 shutdown_connections 2024-09-16T07:35:56.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fedb807a6b0 0x7fedb807cb70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd80c80f0 0x7fedd80c56f0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd80c6af0 0x7fedd80b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 --2- 172.21.15.71:0/670237799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc0004080 0x7fedd81f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 >> 172.21.15.71:0/670237799 conn(0x7fedd80cfa20 msgr2=0x7fedd80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:56.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 shutdown_connections 2024-09-16T07:35:56.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.786+0000 7fedde800640 1 -- 172.21.15.71:0/670237799 wait complete. 2024-09-16T07:35:56.803 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-09-16T07:35:56.804 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar that is printed for 1 ongoing clone job is 2024-09-16T07:35:56.804 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:35:56.804 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:35:56.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.962+0000 7ff043a00640 1 -- 172.21.15.71:0/1737438553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c0ccf00 msgr2=0x7ff03c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.962+0000 7ff043a00640 1 --2- 172.21.15.71:0/1737438553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c0ccf00 0x7ff03c0b2440 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7ff028009830 tx=0x7ff02802a530 comp rx=0 tx=0).stop 2024-09-16T07:35:56.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.962+0000 7ff043a00640 1 -- 172.21.15.71:0/1737438553 shutdown_connections 2024-09-16T07:35:56.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.962+0000 7ff043a00640 1 --2- 172.21.15.71:0/1737438553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c0ccf00 0x7ff03c0b2440 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.962+0000 7ff043a00640 1 -- 172.21.15.71:0/1737438553 >> 172.21.15.71:0/1737438553 conn(0x7ff03c0b1950 msgr2=0x7ff03c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:56.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.962+0000 7ff043a00640 1 -- 172.21.15.71:0/1737438553 shutdown_connections 2024-09-16T07:35:56.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.962+0000 7ff043a00640 1 -- 172.21.15.71:0/1737438553 wait complete. 2024-09-16T07:35:56.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.963+0000 7ff043a00640 1 Processor -- start 2024-09-16T07:35:56.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.964+0000 7ff043a00640 1 -- start start 2024-09-16T07:35:56.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.964+0000 7ff043a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff03c073fb0 0x7ff03c1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:56.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.964+0000 7ff043a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff03c0ccf00 0x7ff03c15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:56.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.964+0000 7ff043a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c15f5a0 0x7ff03c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:56.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.964+0000 7ff043a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c06f9f0 con 0x7ff03c15f5a0 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.964+0000 7ff043a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c06f870 con 0x7ff03c0ccf00 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.964+0000 7ff043a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c074420 con 0x7ff03c073fb0 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c15f5a0 0x7ff03c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c15f5a0 0x7ff03c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35716/0 (socket says 172.21.15.71:35716) 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041a00640 1 -- 172.21.15.71:0/1741111305 learned_addr learned my addr 172.21.15.71:0/1741111305 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041000640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff03c073fb0 0x7ff03c1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff03be00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff03c0ccf00 0x7ff03c15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041a00640 1 -- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff03c073fb0 msgr2=0x7ff03c1520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff03c073fb0 0x7ff03c1520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041a00640 1 -- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff03c0ccf00 msgr2=0x7ff03c15c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff03c0ccf00 0x7ff03c15c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041a00640 1 -- 172.21.15.71:0/1741111305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c15c960 con 0x7ff03c15f5a0 2024-09-16T07:35:56.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff041000640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff03c073fb0 0x7ff03c1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:56.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.965+0000 7ff03be00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff03c0ccf00 0x7ff03c15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:56.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.966+0000 7ff041a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c15f5a0 0x7ff03c0cff20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ff03400c7a0 tx=0x7ff03400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:56.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.966+0000 7ff038c00640 1 -- 172.21.15.71:0/1741111305 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034003b70 con 0x7ff03c15f5a0 2024-09-16T07:35:56.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.966+0000 7ff038c00640 1 -- 172.21.15.71:0/1741111305 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff034003d10 con 0x7ff03c15f5a0 2024-09-16T07:35:56.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.966+0000 7ff038c00640 1 -- 172.21.15.71:0/1741111305 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03400aab0 con 0x7ff03c15f5a0 2024-09-16T07:35:56.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.966+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c1526c0 con 0x7ff03c15f5a0 2024-09-16T07:35:56.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.966+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c152c80 con 0x7ff03c15f5a0 2024-09-16T07:35:56.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.967+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03c0b2440 con 0x7ff03c15f5a0 2024-09-16T07:35:56.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.968+0000 7ff038c00640 1 -- 172.21.15.71:0/1741111305 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7ff03400ac50 con 0x7ff03c15f5a0 2024-09-16T07:35:56.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.969+0000 7ff038c00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff01c07a6b0 0x7ff01c07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:56.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.969+0000 7ff038c00640 1 -- 172.21.15.71:0/1741111305 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7ff0340997f0 con 0x7ff03c15f5a0 2024-09-16T07:35:56.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.969+0000 7ff041000640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff01c07a6b0 0x7ff01c07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:56.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.970+0000 7ff041000640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff01c07a6b0 0x7ff01c07cb70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff0300046b0 tx=0x7ff030009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:56.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:56.973+0000 7ff038c00640 1 -- 172.21.15.71:0/1741111305 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff0340638c0 con 0x7ff03c15f5a0 2024-09-16T07:35:57.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.159+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff03c0b20f0 con 0x7ff03c15f5a0 2024-09-16T07:35:57.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.159+0000 7ff038c00640 1 -- 172.21.15.71:0/1741111305 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+13189 (secure 0 0 0) 0x7ff0340631c0 con 0x7ff03c15f5a0 2024-09-16T07:35:57.159 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:35:57.159 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":125,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:35:55.768954+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6837/2539335973":"2024-09-17T07:35:52.761213+0000","172.21.15.177:6835/2539335973":"2024-09-17T07:35:52.761213+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff01c07a6b0 msgr2=0x7ff01c07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff01c07a6b0 0x7ff01c07cb70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff0300046b0 tx=0x7ff030009040 comp rx=0 tx=0).stop 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c15f5a0 msgr2=0x7ff03c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c15f5a0 0x7ff03c0cff20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ff03400c7a0 tx=0x7ff03400cc60 comp rx=0 tx=0).stop 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 shutdown_connections 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff01c07a6b0 0x7ff01c07cb70 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03c15f5a0 0x7ff03c0cff20 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff03c0ccf00 0x7ff03c15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 --2- 172.21.15.71:0/1741111305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff03c073fb0 0x7ff03c1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 >> 172.21.15.71:0/1741111305 conn(0x7ff03c0b1950 msgr2=0x7ff03c15dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:57.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 shutdown_connections 2024-09-16T07:35:57.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.162+0000 7ff043a00640 1 -- 172.21.15.71:0/1741111305 wait complete. 2024-09-16T07:35:57.178 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:35:57.179 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:35:57.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.339+0000 7fb89d600640 1 -- 172.21.15.71:0/1599973588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb898145750 msgr2=0x7fb898073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.339+0000 7fb89d600640 1 --2- 172.21.15.71:0/1599973588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb898145750 0x7fb898073440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb888009830 tx=0x7fb88802a480 comp rx=0 tx=0).stop 2024-09-16T07:35:57.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.339+0000 7fb89d600640 1 -- 172.21.15.71:0/1599973588 shutdown_connections 2024-09-16T07:35:57.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.339+0000 7fb89d600640 1 --2- 172.21.15.71:0/1599973588 >> v2:172.21.15.177:3300/0 conn(0x7fb89806efa0 0x7fb8980cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.339+0000 7fb89d600640 1 --2- 172.21.15.71:0/1599973588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb898145750 0x7fb898073440 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.339+0000 7fb89d600640 1 -- 172.21.15.71:0/1599973588 >> 172.21.15.71:0/1599973588 conn(0x7fb8980b1840 msgr2=0x7fb8980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:57.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.339+0000 7fb89d600640 1 -- 172.21.15.71:0/1599973588 shutdown_connections 2024-09-16T07:35:57.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.339+0000 7fb89d600640 1 -- 172.21.15.71:0/1599973588 wait complete. 2024-09-16T07:35:57.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.341+0000 7fb89d600640 1 Processor -- start 2024-09-16T07:35:57.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.341+0000 7fb89d600640 1 -- start start 2024-09-16T07:35:57.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb89d600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb89806efa0 0x7fb8980caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:57.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb89d600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8980ccb00 0x7fb898135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:57.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb89d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb898145750 0x7fb898136410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:57.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb89d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb8980b26b0 con 0x7fb898145750 2024-09-16T07:35:57.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb89d600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8980cedd0 con 0x7fb89806efa0 2024-09-16T07:35:57.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb89d600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb898160100 con 0x7fb8980ccb00 2024-09-16T07:35:57.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb897400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb898145750 0x7fb898136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:57.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb897400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb898145750 0x7fb898136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35730/0 (socket says 172.21.15.71:35730) 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb897400640 1 -- 172.21.15.71:0/4146042357 learned_addr learned my addr 172.21.15.71:0/4146042357 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.342+0000 7fb896000640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8980ccb00 0x7fb898135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb896a00640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb89806efa0 0x7fb8980caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb897400640 1 -- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8980ccb00 msgr2=0x7fb898135ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb897400640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8980ccb00 0x7fb898135ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb897400640 1 -- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb89806efa0 msgr2=0x7fb8980caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb897400640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb89806efa0 0x7fb8980caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb897400640 1 -- 172.21.15.71:0/4146042357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8980c6010 con 0x7fb898145750 2024-09-16T07:35:57.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb896a00640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb89806efa0 0x7fb8980caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:35:57.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb897400640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb898145750 0x7fb898136410 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb89000cbc0 tx=0x7fb89000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:57.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb88ea00640 1 -- 172.21.15.71:0/4146042357 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890016020 con 0x7fb898145750 2024-09-16T07:35:57.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb88ea00640 1 -- 172.21.15.71:0/4146042357 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb89000add0 con 0x7fb898145750 2024-09-16T07:35:57.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb88ea00640 1 -- 172.21.15.71:0/4146042357 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890012680 con 0x7fb898145750 2024-09-16T07:35:57.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8980c62f0 con 0x7fb898145750 2024-09-16T07:35:57.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.343+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8980c68b0 con 0x7fb898145750 2024-09-16T07:35:57.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.344+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8980c6c20 con 0x7fb898145750 2024-09-16T07:35:57.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.345+0000 7fb88ea00640 1 -- 172.21.15.71:0/4146042357 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fb890019020 con 0x7fb898145750 2024-09-16T07:35:57.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.346+0000 7fb88ea00640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb87807a6b0 0x7fb87807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:57.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.346+0000 7fb88ea00640 1 -- 172.21.15.71:0/4146042357 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7fb890099c70 con 0x7fb898145750 2024-09-16T07:35:57.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.347+0000 7fb896a00640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb87807a6b0 0x7fb87807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:57.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.348+0000 7fb896a00640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb87807a6b0 0x7fb87807cb70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb888034000 tx=0x7fb888004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:57.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.350+0000 7fb88ea00640 1 -- 172.21.15.71:0/4146042357 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb890020030 con 0x7fb898145750 2024-09-16T07:35:57.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.533+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8981605b0 con 0x7fb898145750 2024-09-16T07:35:57.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.534+0000 7fb88ea00640 1 -- 172.21.15.71:0/4146042357 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) ==== 93+0+10 (secure 0 0 0) 0x7fb8900636f0 con 0x7fb898145750 2024-09-16T07:35:57.533 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:35:57.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb87807a6b0 msgr2=0x7fb87807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb87807a6b0 0x7fb87807cb70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb888034000 tx=0x7fb888004730 comp rx=0 tx=0).stop 2024-09-16T07:35:57.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb898145750 msgr2=0x7fb898136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb898145750 0x7fb898136410 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb89000cbc0 tx=0x7fb89000a430 comp rx=0 tx=0).stop 2024-09-16T07:35:57.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 shutdown_connections 2024-09-16T07:35:57.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb87807a6b0 0x7fb87807cb70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb898145750 0x7fb898136410 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8980ccb00 0x7fb898135ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 --2- 172.21.15.71:0/4146042357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb89806efa0 0x7fb8980caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 >> 172.21.15.71:0/4146042357 conn(0x7fb8980b1840 msgr2=0x7fb8980a98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:57.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 shutdown_connections 2024-09-16T07:35:57.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.536+0000 7fb89d600640 1 -- 172.21.15.71:0/4146042357 wait complete. 2024-09-16T07:35:57.552 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-16T07:35:57.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.709+0000 7fe0b3c00640 1 -- 172.21.15.71:0/3934326706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ccf00 msgr2=0x7fe0ac0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.710+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/3934326706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ccf00 0x7fe0ac0acd30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac075130 tx=0x7fe09802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.710+0000 7fe0b3c00640 1 -- 172.21.15.71:0/3934326706 shutdown_connections 2024-09-16T07:35:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.710+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/3934326706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ac0ad270 0x7fe0ac073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.710+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/3934326706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ccf00 0x7fe0ac0acd30 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.710+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/3934326706 >> v2:172.21.15.177:3300/0 conn(0x7fe0ac15f5a0 0x7fe0ac0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.710+0000 7fe0b3c00640 1 -- 172.21.15.71:0/3934326706 >> 172.21.15.71:0/3934326706 conn(0x7fe0ac0b1950 msgr2=0x7fe0ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.710+0000 7fe0b3c00640 1 -- 172.21.15.71:0/3934326706 shutdown_connections 2024-09-16T07:35:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.710+0000 7fe0b3c00640 1 -- 172.21.15.71:0/3934326706 wait complete. 2024-09-16T07:35:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.711+0000 7fe0b3c00640 1 Processor -- start 2024-09-16T07:35:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.711+0000 7fe0b3c00640 1 -- start start 2024-09-16T07:35:57.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.712+0000 7fe0b3c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ad270 0x7fe0ac1f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:57.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.712+0000 7fe0b3c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ac0ccf00 0x7fe0ac1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:57.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.712+0000 7fe0b3c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ac15f5a0 0x7fe0ac14b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:57.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.712+0000 7fe0b3c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac074880 con 0x7fe0ac0ad270 2024-09-16T07:35:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.712+0000 7fe0b3c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac074700 con 0x7fe0ac15f5a0 2024-09-16T07:35:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.712+0000 7fe0b3c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ac06f160 con 0x7fe0ac0ccf00 2024-09-16T07:35:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ad270 0x7fe0ac1f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ad270 0x7fe0ac1f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35758/0 (socket says 172.21.15.71:35758) 2024-09-16T07:35:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 -- 172.21.15.71:0/517610426 learned_addr learned my addr 172.21.15.71:0/517610426 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0abe00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ac0ccf00 0x7fe0ac1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1c00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ac15f5a0 0x7fe0ac14b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 -- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ac0ccf00 msgr2=0x7fe0ac1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ac0ccf00 0x7fe0ac1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 -- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ac15f5a0 msgr2=0x7fe0ac14b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ac15f5a0 0x7fe0ac14b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 -- 172.21.15.71:0/517610426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac1f8990 con 0x7fe0ac0ad270 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0abe00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ac0ccf00 0x7fe0ac1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0b1200640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ad270 0x7fe0ac1f7d40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c970 tx=0x7fe0a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.713+0000 7fe0a8c00640 1 -- 172.21.15.71:0/517610426 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0007bb0 con 0x7fe0ac0ad270 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.714+0000 7fe0a8c00640 1 -- 172.21.15.71:0/517610426 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0a0007e40 con 0x7fe0ac0ad270 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.714+0000 7fe0a8c00640 1 -- 172.21.15.71:0/517610426 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a000f690 con 0x7fe0ac0ad270 2024-09-16T07:35:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.714+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac156480 con 0x7fe0ac0ad270 2024-09-16T07:35:57.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.714+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac156a40 con 0x7fe0ac0ad270 2024-09-16T07:35:57.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.715+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ac073fb0 con 0x7fe0ac0ad270 2024-09-16T07:35:57.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.716+0000 7fe0a8c00640 1 -- 172.21.15.71:0/517610426 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fe0a000f830 con 0x7fe0ac0ad270 2024-09-16T07:35:57.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.717+0000 7fe0a8c00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe08c07a6e0 0x7fe08c07cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:57.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.718+0000 7fe0a8c00640 1 -- 172.21.15.71:0/517610426 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7fe0a009a0c0 con 0x7fe0ac0ad270 2024-09-16T07:35:57.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.718+0000 7fe0abe00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe08c07a6e0 0x7fe08c07cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.719+0000 7fe0abe00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe08c07a6e0 0x7fe08c07cba0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe098033000 tx=0x7fe098004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:57.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.722+0000 7fe0a8c00640 1 -- 172.21.15.71:0/517610426 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe0a0014030 con 0x7fe0ac0ad270 2024-09-16T07:35:57.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:57.976+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7fe0ac0adad0 con 0x7fe0ac0ad270 2024-09-16T07:35:58.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.791+0000 7fe0a8c00640 1 -- 172.21.15.71:0/517610426 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v617) ==== 189+0+0 (secure 0 0 0) 0x7fe0a0064230 con 0x7fe0ac0ad270 2024-09-16T07:35:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe08c07a6e0 msgr2=0x7fe08c07cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe08c07a6e0 0x7fe08c07cba0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe098033000 tx=0x7fe098004730 comp rx=0 tx=0).stop 2024-09-16T07:35:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ad270 msgr2=0x7fe0ac1f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ad270 0x7fe0ac1f7d40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c970 tx=0x7fe0a000ce30 comp rx=0 tx=0).stop 2024-09-16T07:35:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 shutdown_connections 2024-09-16T07:35:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe08c07a6e0 0x7fe08c07cba0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ac15f5a0 0x7fe0ac14b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ac0ccf00 0x7fe0ac1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 --2- 172.21.15.71:0/517610426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ac0ad270 0x7fe0ac1f7d40 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 >> 172.21.15.71:0/517610426 conn(0x7fe0ac0b1950 msgr2=0x7fe0ac06f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 shutdown_connections 2024-09-16T07:35:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.794+0000 7fe0b3c00640 1 -- 172.21.15.71:0/517610426 wait complete. 2024-09-16T07:35:58.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:35:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.968+0000 7ff87fc00640 1 -- 172.21.15.71:0/3993243642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 msgr2=0x7ff878074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.968+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3993243642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 0x7ff878074420 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff878075130 tx=0x7ff87002a820 comp rx=0 tx=0).stop 2024-09-16T07:35:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.968+0000 7ff87fc00640 1 -- 172.21.15.71:0/3993243642 shutdown_connections 2024-09-16T07:35:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.968+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3993243642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 0x7ff878074420 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.968+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3993243642 >> v2:172.21.15.177:3300/0 conn(0x7ff87815f5a0 0x7ff8780b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.968+0000 7ff87fc00640 1 -- 172.21.15.71:0/3993243642 >> 172.21.15.71:0/3993243642 conn(0x7ff8780b1950 msgr2=0x7ff8780b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.968+0000 7ff87fc00640 1 -- 172.21.15.71:0/3993243642 shutdown_connections 2024-09-16T07:35:58.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.968+0000 7ff87fc00640 1 -- 172.21.15.71:0/3993243642 wait complete. 2024-09-16T07:35:58.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.970+0000 7ff87fc00640 1 Processor -- start 2024-09-16T07:35:58.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.970+0000 7ff87fc00640 1 -- start start 2024-09-16T07:35:58.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87fc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 0x7ff87815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:58.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87fc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8780ccf00 0x7ff878152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:58.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87fc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87815f5a0 0x7ff878152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87fc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff87806f2e0 con 0x7ff878073fb0 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87fc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff87806f160 con 0x7ff87815f5a0 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87fc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff87806f460 con 0x7ff8780ccf00 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 0x7ff87815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 0x7ff87815d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35778/0 (socket says 172.21.15.71:35778) 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87d200640 1 -- 172.21.15.71:0/3017256132 learned_addr learned my addr 172.21.15.71:0/3017256132 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87dc00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87815f5a0 0x7ff878152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff877e00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8780ccf00 0x7ff878152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87d200640 1 -- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8780ccf00 msgr2=0x7ff878152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87d200640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8780ccf00 0x7ff878152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87d200640 1 -- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87815f5a0 msgr2=0x7ff878152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87d200640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87815f5a0 0x7ff878152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:58.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.971+0000 7ff87d200640 1 -- 172.21.15.71:0/3017256132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87815e040 con 0x7ff878073fb0 2024-09-16T07:35:58.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.972+0000 7ff87d200640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 0x7ff87815d5a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff86800d740 tx=0x7ff86800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:58.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.972+0000 7ff874c00640 1 -- 172.21.15.71:0/3017256132 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868017070 con 0x7ff878073fb0 2024-09-16T07:35:58.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.972+0000 7ff874c00640 1 -- 172.21.15.71:0/3017256132 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff868009d20 con 0x7ff878073fb0 2024-09-16T07:35:58.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.972+0000 7ff874c00640 1 -- 172.21.15.71:0/3017256132 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8680106c0 con 0x7ff878073fb0 2024-09-16T07:35:58.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.972+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87815e320 con 0x7ff878073fb0 2024-09-16T07:35:58.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.972+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781e8720 con 0x7ff878073fb0 2024-09-16T07:35:58.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.973+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8780760b0 con 0x7ff878073fb0 2024-09-16T07:35:58.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.974+0000 7ff874c00640 1 -- 172.21.15.71:0/3017256132 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7ff86801a070 con 0x7ff878073fb0 2024-09-16T07:35:58.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.975+0000 7ff874c00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff85807a6e0 0x7ff85807cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:58.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.975+0000 7ff874c00640 1 -- 172.21.15.71:0/3017256132 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7ff868099a10 con 0x7ff878073fb0 2024-09-16T07:35:58.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.975+0000 7ff877e00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff85807a6e0 0x7ff85807cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:58.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.976+0000 7ff877e00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff85807a6e0 0x7ff85807cba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff8780cc1d0 tx=0x7ff864009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:58.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:58.979+0000 7ff874c00640 1 -- 172.21.15.71:0/3017256132 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff8780760b0 con 0x7ff878073fb0 2024-09-16T07:35:59.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.164+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff8781eab50 con 0x7ff878073fb0 2024-09-16T07:35:59.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.164+0000 7ff874c00640 1 -- 172.21.15.71:0/3017256132 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v50) ==== 59+0+1998 (secure 0 0 0) 0x7ff868063b80 con 0x7ff878073fb0 2024-09-16T07:35:59.164 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:35:59.164 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:35:59.164 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:35:59.164 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:35:59.164 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:35:59.164 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:35:59.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:35:59.166 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:35:59.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff85807a6e0 msgr2=0x7ff85807cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:59.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff85807a6e0 0x7ff85807cba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff8780cc1d0 tx=0x7ff864009040 comp rx=0 tx=0).stop 2024-09-16T07:35:59.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 msgr2=0x7ff87815d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 0x7ff87815d5a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff86800d740 tx=0x7ff86800dc00 comp rx=0 tx=0).stop 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 shutdown_connections 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff85807a6e0 0x7ff85807cba0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87815f5a0 0x7ff878152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff8780ccf00 0x7ff878152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 --2- 172.21.15.71:0/3017256132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878073fb0 0x7ff87815d5a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 >> 172.21.15.71:0/3017256132 conn(0x7ff8780b1950 msgr2=0x7ff8780ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 shutdown_connections 2024-09-16T07:35:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.167+0000 7ff87fc00640 1 -- 172.21.15.71:0/3017256132 wait complete. 2024-09-16T07:35:59.182 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:35:59.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.343+0000 7f1682e00640 1 -- 172.21.15.71:0/2789954426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f167c0b3450 msgr2=0x7f167c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:59.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.343+0000 7f1682e00640 1 --2- 172.21.15.71:0/2789954426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f167c0b3450 0x7f167c0b3890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1668009830 tx=0x7f166802a400 comp rx=0 tx=0).stop 2024-09-16T07:35:59.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.344+0000 7f1682e00640 1 -- 172.21.15.71:0/2789954426 shutdown_connections 2024-09-16T07:35:59.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.344+0000 7f1682e00640 1 --2- 172.21.15.71:0/2789954426 >> v2:172.21.15.177:3300/0 conn(0x7f167c0cf4b0 0x7f167c0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.344+0000 7f1682e00640 1 --2- 172.21.15.71:0/2789954426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f167c0b3450 0x7f167c0b3890 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.344+0000 7f1682e00640 1 -- 172.21.15.71:0/2789954426 >> 172.21.15.71:0/2789954426 conn(0x7f167c0d3d50 msgr2=0x7f167c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:59.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.344+0000 7f1682e00640 1 -- 172.21.15.71:0/2789954426 shutdown_connections 2024-09-16T07:35:59.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.344+0000 7f1682e00640 1 -- 172.21.15.71:0/2789954426 wait complete. 2024-09-16T07:35:59.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.345+0000 7f1682e00640 1 Processor -- start 2024-09-16T07:35:59.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.345+0000 7f1682e00640 1 -- start start 2024-09-16T07:35:59.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.346+0000 7f1682e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f167c0b3450 0x7f167c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:59.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.346+0000 7f1682e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f167c0cf4b0 0x7f167c0cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:59.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.346+0000 7f1682e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f167c15df60 0x7f167c1521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:59.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.346+0000 7f1682e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f167c0d24b0 con 0x7f167c15df60 2024-09-16T07:35:59.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.346+0000 7f1682e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f167c0d2330 con 0x7f167c0cf4b0 2024-09-16T07:35:59.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.346+0000 7f1682e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f167c0d2630 con 0x7f167c0b3450 2024-09-16T07:35:59.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f167c15df60 0x7f167c1521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:59.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f167c15df60 0x7f167c1521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35784/0 (socket says 172.21.15.71:35784) 2024-09-16T07:35:59.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 -- 172.21.15.71:0/3603373913 learned_addr learned my addr 172.21.15.71:0/3603373913 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:35:59.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1681000640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f167c0cf4b0 0x7f167c0cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:59.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1681a00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f167c0b3450 0x7f167c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 -- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f167c0b3450 msgr2=0x7f167c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f167c0b3450 0x7f167c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 -- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f167c0cf4b0 msgr2=0x7f167c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f167c0cf4b0 0x7f167c0cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 -- 172.21.15.71:0/3603373913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c1f5880 con 0x7f167c15df60 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.347+0000 7f1682400640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f167c15df60 0x7f167c1521d0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f167400eb40 tx=0x7f167400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.348+0000 7f1679600640 1 -- 172.21.15.71:0/3603373913 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674004b20 con 0x7f167c15df60 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.348+0000 7f1679600640 1 -- 172.21.15.71:0/3603373913 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1674004ce0 con 0x7f167c15df60 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.348+0000 7f1679600640 1 -- 172.21.15.71:0/3603373913 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674005280 con 0x7f167c15df60 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.348+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c0cae20 con 0x7f167c15df60 2024-09-16T07:35:59.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.348+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c1f5d80 con 0x7f167c15df60 2024-09-16T07:35:59.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.349+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f167c0d2b50 con 0x7f167c15df60 2024-09-16T07:35:59.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.350+0000 7f1679600640 1 -- 172.21.15.71:0/3603373913 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f1674005420 con 0x7f167c15df60 2024-09-16T07:35:59.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.351+0000 7f1679600640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f165c07a6b0 0x7f165c07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:35:59.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.351+0000 7f1679600640 1 -- 172.21.15.71:0/3603373913 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7f167409a590 con 0x7f167c15df60 2024-09-16T07:35:59.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.352+0000 7f1681a00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f165c07a6b0 0x7f165c07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:35:59.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.352+0000 7f1681a00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f165c07a6b0 0x7f165c07cb70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f167c0b6830 tx=0x7f1670007e90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:35:59.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.355+0000 7f1679600640 1 -- 172.21.15.71:0/3603373913 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1674064010 con 0x7f167c15df60 2024-09-16T07:35:59.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.541+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f167c06ef10 con 0x7f167c15df60 2024-09-16T07:35:59.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.545+0000 7f1679600640 1 -- 172.21.15.71:0/3603373913 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f1674068a40 con 0x7f167c15df60 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f165c07a6b0 msgr2=0x7f165c07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f165c07a6b0 0x7f165c07cb70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f167c0b6830 tx=0x7f1670007e90 comp rx=0 tx=0).stop 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f167c15df60 msgr2=0x7f167c1521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f167c15df60 0x7f167c1521d0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f167400eb40 tx=0x7f167400c2d0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 shutdown_connections 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f165c07a6b0 0x7f165c07cb70 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f167c15df60 0x7f167c1521d0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f167c0cf4b0 0x7f167c0cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 --2- 172.21.15.71:0/3603373913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f167c0b3450 0x7f167c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 >> 172.21.15.71:0/3603373913 conn(0x7f167c0d3d50 msgr2=0x7f167c0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:35:59.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 shutdown_connections 2024-09-16T07:35:59.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:35:59.548+0000 7f1682e00640 1 -- 172.21.15.71:0/3603373913 wait complete. 2024-09-16T07:35:59.563 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:35:59.563 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:35:59.647 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:35:59.756 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:35:59.756 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:59.756 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:59.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:35:59.793 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:35:59.793 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:59.830 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:35:59.830 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:35:59.830 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:59.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:35:59.868 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:35:59.868 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:35:59.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:35:59.907 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:35:59.908 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:36:00.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.065+0000 7f3828e00640 1 -- 172.21.15.71:0/2321213112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f382415f5a0 msgr2=0x7f38240b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.065+0000 7f3828e00640 1 --2- 172.21.15.71:0/2321213112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f382415f5a0 0x7f38240b2440 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3814009830 tx=0x7f381402a480 comp rx=0 tx=0).stop 2024-09-16T07:36:00.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.065+0000 7f3828e00640 1 -- 172.21.15.71:0/2321213112 shutdown_connections 2024-09-16T07:36:00.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.065+0000 7f3828e00640 1 --2- 172.21.15.71:0/2321213112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f382415f5a0 0x7f38240b2440 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.065+0000 7f3828e00640 1 -- 172.21.15.71:0/2321213112 >> 172.21.15.71:0/2321213112 conn(0x7f38240b1950 msgr2=0x7f38240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:00.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.065+0000 7f3828e00640 1 -- 172.21.15.71:0/2321213112 shutdown_connections 2024-09-16T07:36:00.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.065+0000 7f3828e00640 1 -- 172.21.15.71:0/2321213112 wait complete. 2024-09-16T07:36:00.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.067+0000 7f3828e00640 1 Processor -- start 2024-09-16T07:36:00.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.067+0000 7f3828e00640 1 -- start start 2024-09-16T07:36:00.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3828e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3824073fb0 0x7f38241e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3828e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38240ccf00 0x7f3824147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3828e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f382415f5a0 0x7f38241482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3828e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f382406f9f0 con 0x7f3824073fb0 2024-09-16T07:36:00.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3828e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f382406f870 con 0x7f382415f5a0 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3828e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3824074420 con 0x7f38240ccf00 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3824073fb0 0x7f38241e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3824073fb0 0x7f38241e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35804/0 (socket says 172.21.15.71:35804) 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821e00640 1 -- 172.21.15.71:0/1994026016 learned_addr learned my addr 172.21.15.71:0/1994026016 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821400640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38240ccf00 0x7f3824147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3822800640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f382415f5a0 0x7f38241482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821e00640 1 -- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38240ccf00 msgr2=0x7f3824147d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38240ccf00 0x7f3824147d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821e00640 1 -- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f382415f5a0 msgr2=0x7f38241482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f382415f5a0 0x7f38241482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.068+0000 7f3821e00640 1 -- 172.21.15.71:0/1994026016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38241e48d0 con 0x7f3824073fb0 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.069+0000 7f3821e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3824073fb0 0x7f38241e88a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3814000c00 tx=0x7f3814033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.069+0000 7f381a000640 1 -- 172.21.15.71:0/1994026016 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814031d60 con 0x7f3824073fb0 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.069+0000 7f381a000640 1 -- 172.21.15.71:0/1994026016 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3814033070 con 0x7f3824073fb0 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.069+0000 7f381a000640 1 -- 172.21.15.71:0/1994026016 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38140334c0 con 0x7f3824073fb0 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.069+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38241e4bb0 con 0x7f3824073fb0 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.069+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38240ce2f0 con 0x7f3824073fb0 2024-09-16T07:36:00.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.070+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38240b2440 con 0x7f3824073fb0 2024-09-16T07:36:00.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.072+0000 7f381a000640 1 -- 172.21.15.71:0/1994026016 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f3814033660 con 0x7f3824073fb0 2024-09-16T07:36:00.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.072+0000 7f381a000640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f380407a6b0 0x7f380407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.073+0000 7f381a000640 1 -- 172.21.15.71:0/1994026016 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7f38140bad40 con 0x7f3824073fb0 2024-09-16T07:36:00.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.073+0000 7f3821400640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f380407a6b0 0x7f380407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.074+0000 7f3821400640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f380407a6b0 0x7f380407cb70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3810007fd0 tx=0x7f3810009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:00.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.076+0000 7f381a000640 1 -- 172.21.15.71:0/1994026016 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f381403f030 con 0x7f3824073fb0 2024-09-16T07:36:00.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.296+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38241e4d80 con 0x7f3824073fb0 2024-09-16T07:36:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.297+0000 7f381a000640 1 -- 172.21.15.71:0/1994026016 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+3917 (secure 0 0 0) 0x7f3814084eb0 con 0x7f3824073fb0 2024-09-16T07:36:00.297 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:00.297 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":78,"btime":"2024-09-16T07:35:53:763384+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6911,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1994777108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1994777108},{"type":"v1","addr":"172.21.15.71:6833","nonce":1994777108}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6917,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1399666780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1399666780},{"type":"v1","addr":"172.21.15.177:6836","nonce":1399666780}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3844714720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":3844714720},{"type":"v1","addr":"172.21.15.177:6837","nonce":3844714720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2921631965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2921631965},{"type":"v1","addr":"172.21.15.71:6835","nonce":2921631965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[]} 2024-09-16T07:36:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 78 2024-09-16T07:36:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f380407a6b0 msgr2=0x7f380407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f380407a6b0 0x7f380407cb70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3810007fd0 tx=0x7f3810009040 comp rx=0 tx=0).stop 2024-09-16T07:36:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3824073fb0 msgr2=0x7f38241e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3824073fb0 0x7f38241e88a0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3814000c00 tx=0x7f3814033d60 comp rx=0 tx=0).stop 2024-09-16T07:36:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 shutdown_connections 2024-09-16T07:36:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f380407a6b0 0x7f380407cb70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f382415f5a0 0x7f38241482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38240ccf00 0x7f3824147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 --2- 172.21.15.71:0/1994026016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3824073fb0 0x7f38241e88a0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 >> 172.21.15.71:0/1994026016 conn(0x7f38240b1950 msgr2=0x7f38240b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 shutdown_connections 2024-09-16T07:36:00.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.299+0000 7f3828e00640 1 -- 172.21.15.71:0/1994026016 wait complete. 2024-09-16T07:36:00.316 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:36:00.316 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:36:00.316 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:36:00.316 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:36:00.317 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:36:00.317 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:36:00.317 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:36:00.317 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:36:00.317 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:36:00.317 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:36:00.318 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:36:00.318 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:36:00.318 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:36:00.318 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:36:00.318 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:36:00.318 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:36:00.331 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:36:00.331 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:36:00.332 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:36:00.332 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:36:00.336 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:36:00.336 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:36:00.337 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:36:00.338 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:36:00.375 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:36:00.376 DEBUG:teuthology.parallel:result is None 2024-09-16T07:36:00.378 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:36:00.378 DEBUG:teuthology.parallel:result is None 2024-09-16T07:36:00.381 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:36:00.381 DEBUG:teuthology.parallel:result is None 2024-09-16T07:36:00.385 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:36:00.385 DEBUG:teuthology.parallel:result is None 2024-09-16T07:36:00.385 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:36:00.504 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:36:00.506 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:36:00.519 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:36:00.519 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:36:00.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.554+0000 7f7985200640 1 -- 172.21.15.71:0/587710914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b4d10 msgr2=0x7f7980162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.554+0000 7f7985200640 1 --2- 172.21.15.71:0/587710914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b4d10 0x7f7980162730 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f79800b3230 tx=0x7f797802aac0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.554+0000 7f7985200640 1 -- 172.21.15.71:0/587710914 shutdown_connections 2024-09-16T07:36:00.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.554+0000 7f7985200640 1 --2- 172.21.15.71:0/587710914 >> v2:172.21.15.177:3300/0 conn(0x7f79800b28d0 0x7f79800d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.554+0000 7f7985200640 1 --2- 172.21.15.71:0/587710914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b4d10 0x7f7980162730 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.554+0000 7f7985200640 1 -- 172.21.15.71:0/587710914 >> 172.21.15.71:0/587710914 conn(0x7f79800d3de0 msgr2=0x7f79800d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:00.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.554+0000 7f7985200640 1 -- 172.21.15.71:0/587710914 shutdown_connections 2024-09-16T07:36:00.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.555+0000 7f7985200640 1 -- 172.21.15.71:0/587710914 wait complete. 2024-09-16T07:36:00.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.556+0000 7f7985200640 1 Processor -- start 2024-09-16T07:36:00.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.556+0000 7f7985200640 1 -- start start 2024-09-16T07:36:00.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f7985200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b28d0 0x7f798014c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f7985200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79800b4d10 0x7f798014c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f7985200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79800d1ec0 0x7f79801f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f7985200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79800b3900 con 0x7f79800b28d0 2024-09-16T07:36:00.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f7985200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7980162eb0 con 0x7f79800d1ec0 2024-09-16T07:36:00.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f7985200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f79800b3a80 con 0x7f79800b4d10 2024-09-16T07:36:00.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f797f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b28d0 0x7f798014c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f797f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b28d0 0x7f798014c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35820/0 (socket says 172.21.15.71:35820) 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.557+0000 7f797f400640 1 -- 172.21.15.71:0/2092624561 learned_addr learned my addr 172.21.15.71:0/2092624561 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f797ea00640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79800b4d10 0x7f798014c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f797fe00640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79800d1ec0 0x7f79801f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f797f400640 1 -- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79800b4d10 msgr2=0x7f798014c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f797f400640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79800b4d10 0x7f798014c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f797f400640 1 -- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79800d1ec0 msgr2=0x7f79801f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f797f400640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79800d1ec0 0x7f79801f3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f797f400640 1 -- 172.21.15.71:0/2092624561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798015a770 con 0x7f79800b28d0 2024-09-16T07:36:00.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f797f400640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b28d0 0x7f798014c190 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f797000c3a0 tx=0x7f797000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:00.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f7977400640 1 -- 172.21.15.71:0/2092624561 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970017070 con 0x7f79800b28d0 2024-09-16T07:36:00.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f7977400640 1 -- 172.21.15.71:0/2092624561 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79700053c0 con 0x7f79800b28d0 2024-09-16T07:36:00.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f7977400640 1 -- 172.21.15.71:0/2092624561 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970003850 con 0x7f79800b28d0 2024-09-16T07:36:00.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.558+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798015aa50 con 0x7f79800b28d0 2024-09-16T07:36:00.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.559+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798015d120 con 0x7f79800b28d0 2024-09-16T07:36:00.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.560+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79800d22a0 con 0x7f79800b28d0 2024-09-16T07:36:00.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.561+0000 7f7977400640 1 -- 172.21.15.71:0/2092624561 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f797001a070 con 0x7f79800b28d0 2024-09-16T07:36:00.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.562+0000 7f7977400640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f796007a6b0 0x7f796007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.562+0000 7f7977400640 1 -- 172.21.15.71:0/2092624561 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5027+0+0 (secure 0 0 0) 0x7f7970098c60 con 0x7f79800b28d0 2024-09-16T07:36:00.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.562+0000 7f797ea00640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f796007a6b0 0x7f796007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.563+0000 7f797ea00640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f796007a6b0 0x7f796007cb70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f79800b7750 tx=0x7f796c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:00.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.566+0000 7f7977400640 1 -- 172.21.15.71:0/2092624561 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7970062dd0 con 0x7f79800b28d0 2024-09-16T07:36:00.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.762+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f79800cf8e0 con 0x7f79800b28d0 2024-09-16T07:36:00.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.805+0000 7f7977400640 1 -- 172.21.15.71:0/2092624561 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v127) ==== 97+0+0 (secure 0 0 0) 0x7f79700626d0 con 0x7f79800b28d0 2024-09-16T07:36:00.804 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:36:00.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f796007a6b0 msgr2=0x7f796007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f796007a6b0 0x7f796007cb70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f79800b7750 tx=0x7f796c005f90 comp rx=0 tx=0).stop 2024-09-16T07:36:00.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b28d0 msgr2=0x7f798014c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b28d0 0x7f798014c190 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f797000c3a0 tx=0x7f797000c860 comp rx=0 tx=0).stop 2024-09-16T07:36:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 shutdown_connections 2024-09-16T07:36:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f796007a6b0 0x7f796007cb70 secure :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f79800b7750 tx=0x7f796c005f90 comp rx=0 tx=0).stop 2024-09-16T07:36:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79800d1ec0 0x7f79801f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79800b4d10 0x7f798014c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 --2- 172.21.15.71:0/2092624561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79800b28d0 0x7f798014c190 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.807+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 >> 172.21.15.71:0/2092624561 conn(0x7f79800d3de0 msgr2=0x7f79800d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.808+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 shutdown_connections 2024-09-16T07:36:00.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.808+0000 7f7985200640 1 -- 172.21.15.71:0/2092624561 wait complete. 2024-09-16T07:36:00.826 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:36:00.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.983+0000 7f374e200640 1 -- 172.21.15.71:0/3576041517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 msgr2=0x7f37480b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.983+0000 7f374e200640 1 --2- 172.21.15.71:0/3576041517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 0x7f37480b2440 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3738005fd0 tx=0x7f373802a540 comp rx=0 tx=0).stop 2024-09-16T07:36:00.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.983+0000 7f374e200640 1 -- 172.21.15.71:0/3576041517 shutdown_connections 2024-09-16T07:36:00.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.983+0000 7f374e200640 1 --2- 172.21.15.71:0/3576041517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37480ccf00 0x7f37480acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.983+0000 7f374e200640 1 --2- 172.21.15.71:0/3576041517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 0x7f37480b2440 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.983+0000 7f374e200640 1 -- 172.21.15.71:0/3576041517 >> 172.21.15.71:0/3576041517 conn(0x7f37480b1950 msgr2=0x7f37480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.983+0000 7f374e200640 1 -- 172.21.15.71:0/3576041517 shutdown_connections 2024-09-16T07:36:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.983+0000 7f374e200640 1 -- 172.21.15.71:0/3576041517 wait complete. 2024-09-16T07:36:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.985+0000 7f374e200640 1 Processor -- start 2024-09-16T07:36:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.985+0000 7f374e200640 1 -- start start 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f374e200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37480ad270 0x7f374814c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f374e200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37480ccf00 0x7f3748150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f374e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 0x7f374814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f374e200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f374806f2e0 con 0x7f374815f5a0 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f374e200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f374806f160 con 0x7f37480ccf00 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f374e200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f374806f460 con 0x7f37480ad270 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f3747e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 0x7f374814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f3747e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 0x7f374814da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35840/0 (socket says 172.21.15.71:35840) 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f3747e00640 1 -- 172.21.15.71:0/2891292822 learned_addr learned my addr 172.21.15.71:0/2891292822 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f3747400640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37480ad270 0x7f374814c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.986+0000 7f3746a00640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37480ccf00 0x7f3748150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f3747e00640 1 -- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37480ad270 msgr2=0x7f374814c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f3747e00640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37480ad270 0x7f374814c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f3747e00640 1 -- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37480ccf00 msgr2=0x7f3748150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f3747e00640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37480ccf00 0x7f3748150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f3747e00640 1 -- 172.21.15.71:0/2891292822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3748153770 con 0x7f374815f5a0 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f3747e00640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 0x7f374814da00 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f374000c7a0 tx=0x7f374000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f373f400640 1 -- 172.21.15.71:0/2891292822 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740003b70 con 0x7f374815f5a0 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f373f400640 1 -- 172.21.15.71:0/2891292822 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3740003d10 con 0x7f374815f5a0 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f373f400640 1 -- 172.21.15.71:0/2891292822 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374000aab0 con 0x7f374815f5a0 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3748153a50 con 0x7f374815f5a0 2024-09-16T07:36:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.987+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3748150570 con 0x7f374815f5a0 2024-09-16T07:36:00.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.989+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374806f9f0 con 0x7f374815f5a0 2024-09-16T07:36:00.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.990+0000 7f373f400640 1 -- 172.21.15.71:0/2891292822 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f374000ac50 con 0x7f374815f5a0 2024-09-16T07:36:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.990+0000 7f373f400640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f372807a6b0 0x7f372807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.991+0000 7f373f400640 1 -- 172.21.15.71:0/2891292822 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4941+0+0 (secure 0 0 0) 0x7f37400996a0 con 0x7f374815f5a0 2024-09-16T07:36:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.991+0000 7f3747400640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f372807a6b0 0x7f372807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:00.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.992+0000 7f3747400640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f372807a6b0 0x7f372807cb70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3738033000 tx=0x7f3738004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:00.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:00.995+0000 7f373f400640 1 -- 172.21.15.71:0/2891292822 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3740063120 con 0x7f374815f5a0 2024-09-16T07:36:01.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.247+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f37480b3440 con 0x7f374815f5a0 2024-09-16T07:36:01.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.248+0000 7f373f400640 1 -- 172.21.15.71:0/2891292822 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f3740067b50 con 0x7f374815f5a0 2024-09-16T07:36:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:01.247 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:36:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:36:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:36:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:36:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:36:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:36:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:36:01.261 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:36:01.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f372807a6b0 msgr2=0x7f372807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f372807a6b0 0x7f372807cb70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3738033000 tx=0x7f3738004730 comp rx=0 tx=0).stop 2024-09-16T07:36:01.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 msgr2=0x7f374814da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 0x7f374814da00 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f374000c7a0 tx=0x7f374000cc60 comp rx=0 tx=0).stop 2024-09-16T07:36:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 shutdown_connections 2024-09-16T07:36:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f372807a6b0 0x7f372807cb70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f374815f5a0 0x7f374814da00 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37480ccf00 0x7f3748150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 --2- 172.21.15.71:0/2891292822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37480ad270 0x7f374814c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 >> 172.21.15.71:0/2891292822 conn(0x7f37480b1950 msgr2=0x7f37480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 shutdown_connections 2024-09-16T07:36:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.250+0000 7f374e200640 1 -- 172.21.15.71:0/2891292822 wait complete. 2024-09-16T07:36:01.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:36:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.424+0000 7f6eb6200640 1 -- 172.21.15.71:0/2558884383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e940039e0 msgr2=0x7f6e94003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.424+0000 7f6eb6200640 1 --2- 172.21.15.71:0/2558884383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e940039e0 0x7f6e94003dc0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009b80 tx=0x7f6e94005830 comp rx=0 tx=0).stop 2024-09-16T07:36:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.424+0000 7f6eafe00640 1 -- 172.21.15.71:0/2558884383 reap_dead start 2024-09-16T07:36:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.424+0000 7f6eb6200640 1 -- 172.21.15.71:0/2558884383 shutdown_connections 2024-09-16T07:36:01.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.424+0000 7f6eb6200640 1 -- 172.21.15.71:0/2558884383 >> 172.21.15.71:0/2558884383 conn(0x7f6eb00b1950 msgr2=0x7f6eb00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:01.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.424+0000 7f6eb6200640 1 -- 172.21.15.71:0/2558884383 shutdown_connections 2024-09-16T07:36:01.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.424+0000 7f6eb6200640 1 -- 172.21.15.71:0/2558884383 wait complete. 2024-09-16T07:36:01.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.426+0000 7f6eb6200640 1 Processor -- start 2024-09-16T07:36:01.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.426+0000 7f6eb6200640 1 -- start start 2024-09-16T07:36:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eb6200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e940039e0 0x7f6eb01558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eb6200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6eb015b900 0x7f6eb0158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eb6200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eb01590a0 0x7f6eb00aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eb6200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb006ef30 con 0x7f6eb01590a0 2024-09-16T07:36:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eb6200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb006fa80 con 0x7f6eb015b900 2024-09-16T07:36:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eb6200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb00b2440 con 0x7f6e940039e0 2024-09-16T07:36:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eb01590a0 0x7f6eb00aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:01.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eb01590a0 0x7f6eb00aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35862/0 (socket says 172.21.15.71:35862) 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eafe00640 1 -- 172.21.15.71:0/735457265 learned_addr learned my addr 172.21.15.71:0/735457265 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eaf400640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e940039e0 0x7f6eb01558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.427+0000 7f6eaea00640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6eb015b900 0x7f6eb0158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6eafe00640 1 -- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e940039e0 msgr2=0x7f6eb01558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6eafe00640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e940039e0 0x7f6eb01558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6eafe00640 1 -- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6eb015b900 msgr2=0x7f6eb0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6eafe00640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6eb015b900 0x7f6eb0158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6eafe00640 1 -- 172.21.15.71:0/735457265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb00ccd00 con 0x7f6eb01590a0 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6eafe00640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eb01590a0 0x7f6eb00aded0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8000f80 tx=0x7f6ea800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6ea7400640 1 -- 172.21.15.71:0/735457265 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800c9d0 con 0x7f6eb01590a0 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6ea7400640 1 -- 172.21.15.71:0/735457265 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ea8017070 con 0x7f6eb01590a0 2024-09-16T07:36:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6ea7400640 1 -- 172.21.15.71:0/735457265 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8011460 con 0x7f6eb01590a0 2024-09-16T07:36:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.428+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb01f4730 con 0x7f6eb01590a0 2024-09-16T07:36:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.429+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb01f4cc0 con 0x7f6eb01590a0 2024-09-16T07:36:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.430+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb01f5660 con 0x7f6eb01590a0 2024-09-16T07:36:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.431+0000 7f6ea7400640 1 -- 172.21.15.71:0/735457265 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f6ea8011600 con 0x7f6eb01590a0 2024-09-16T07:36:01.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.432+0000 7f6ea7400640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e9007a6b0 0x7f6e9007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:01.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.432+0000 7f6ea7400640 1 -- 172.21.15.71:0/735457265 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4941+0+0 (secure 0 0 0) 0x7f6ea8099be0 con 0x7f6eb01590a0 2024-09-16T07:36:01.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.433+0000 7f6eaf400640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e9007a6b0 0x7f6e9007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:01.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.433+0000 7f6eaf400640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e9007a6b0 0x7f6e9007cb70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0035000 tx=0x7f6ea00046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:01.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.437+0000 7f6ea7400640 1 -- 172.21.15.71:0/735457265 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6eb01f5660 con 0x7f6eb01590a0 2024-09-16T07:36:01.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.621+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6eb015ec10 con 0x7f6eb01590a0 2024-09-16T07:36:01.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.622+0000 7f6ea7400640 1 -- 172.21.15.71:0/735457265 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v129) ==== 84+0+65 (secure 0 0 0) 0x7f6eb00ccd00 con 0x7f6eb01590a0 2024-09-16T07:36:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:01.622 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:36:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:36:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:36:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:01.622 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:36:01.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e9007a6b0 msgr2=0x7f6e9007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e9007a6b0 0x7f6e9007cb70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0035000 tx=0x7f6ea00046c0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eb01590a0 msgr2=0x7f6eb00aded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eb01590a0 0x7f6eb00aded0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8000f80 tx=0x7f6ea800eb00 comp rx=0 tx=0).stop 2024-09-16T07:36:01.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 shutdown_connections 2024-09-16T07:36:01.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6e9007a6b0 0x7f6e9007cb70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6eb01590a0 0x7f6eb00aded0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6eb015b900 0x7f6eb0158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 --2- 172.21.15.71:0/735457265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e940039e0 0x7f6eb01558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 >> 172.21.15.71:0/735457265 conn(0x7f6eb00b1950 msgr2=0x7f6eb015cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:01.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 shutdown_connections 2024-09-16T07:36:01.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.624+0000 7f6eb6200640 1 -- 172.21.15.71:0/735457265 wait complete. 2024-09-16T07:36:01.639 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:36:01.640 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:36:01.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.800+0000 7f7527000640 1 -- 172.21.15.71:0/1838762904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f752015b700 msgr2=0x7f75200ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.800+0000 7f7527000640 1 --2- 172.21.15.71:0/1838762904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f752015b700 0x7f75200ccca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f750c005fd0 tx=0x7f750c02a5c0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.801+0000 7f7527000640 1 -- 172.21.15.71:0/1838762904 shutdown_connections 2024-09-16T07:36:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.801+0000 7f7527000640 1 --2- 172.21.15.71:0/1838762904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f752015b700 0x7f75200ccca0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.801+0000 7f7527000640 1 --2- 172.21.15.71:0/1838762904 >> v2:172.21.15.177:3300/0 conn(0x7f7520158900 0x7f752015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.801+0000 7f7527000640 1 -- 172.21.15.71:0/1838762904 >> 172.21.15.71:0/1838762904 conn(0x7f75200b18e0 msgr2=0x7f75200b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.801+0000 7f7527000640 1 -- 172.21.15.71:0/1838762904 shutdown_connections 2024-09-16T07:36:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.801+0000 7f7527000640 1 -- 172.21.15.71:0/1838762904 wait complete. 2024-09-16T07:36:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.802+0000 7f7527000640 1 Processor -- start 2024-09-16T07:36:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.803+0000 7f7527000640 1 -- start start 2024-09-16T07:36:01.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.803+0000 7f7527000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75200cd1e0 0x7f75200cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:01.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.803+0000 7f7527000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7520158900 0x7f7520155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:01.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.803+0000 7f7527000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f752015b700 0x7f75200c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:01.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.803+0000 7f7527000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75200ced60 con 0x7f75200cd1e0 2024-09-16T07:36:01.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.803+0000 7f7527000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f75200cebe0 con 0x7f752015b700 2024-09-16T07:36:01.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.803+0000 7f7527000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f75200ceee0 con 0x7f7520158900 2024-09-16T07:36:01.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75200cd1e0 0x7f75200cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:01.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75200cd1e0 0x7f75200cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35872/0 (socket says 172.21.15.71:35872) 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751fe00640 1 -- 172.21.15.71:0/2685168437 learned_addr learned my addr 172.21.15.71:0/2685168437 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751f400640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7520158900 0x7f7520155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f7525000640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f752015b700 0x7f75200c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751fe00640 1 -- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7520158900 msgr2=0x7f7520155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751fe00640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7520158900 0x7f7520155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751fe00640 1 -- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f752015b700 msgr2=0x7f75200c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751fe00640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f752015b700 0x7f75200c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.804+0000 7f751fe00640 1 -- 172.21.15.71:0/2685168437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75200b0440 con 0x7f75200cd1e0 2024-09-16T07:36:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.805+0000 7f751fe00640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75200cd1e0 0x7f75200cb8b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f751000e9f0 tx=0x7f751000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.806+0000 7f7517e00640 1 -- 172.21.15.71:0/2685168437 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751000cd80 con 0x7f75200cd1e0 2024-09-16T07:36:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.806+0000 7f7517e00640 1 -- 172.21.15.71:0/2685168437 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7510010430 con 0x7f75200cd1e0 2024-09-16T07:36:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.807+0000 7f7517e00640 1 -- 172.21.15.71:0/2685168437 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510010840 con 0x7f75200cd1e0 2024-09-16T07:36:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.807+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75200b0720 con 0x7f75200cd1e0 2024-09-16T07:36:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.807+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75200b0ce0 con 0x7f75200cd1e0 2024-09-16T07:36:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.808+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7520069360 con 0x7f75200cd1e0 2024-09-16T07:36:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.809+0000 7f7517e00640 1 -- 172.21.15.71:0/2685168437 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f75100109e0 con 0x7f75200cd1e0 2024-09-16T07:36:01.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.810+0000 7f7517e00640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f750007a6b0 0x7f750007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:01.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.811+0000 7f7517e00640 1 -- 172.21.15.71:0/2685168437 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4941+0+0 (secure 0 0 0) 0x7f751009a3c0 con 0x7f75200cd1e0 2024-09-16T07:36:01.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.811+0000 7f751f400640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f750007a6b0 0x7f750007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:01.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.812+0000 7f751f400640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f750007a6b0 0x7f750007cb70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f750c033000 tx=0x7f750c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:01.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:01.816+0000 7f7517e00640 1 -- 172.21.15.71:0/2685168437 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7510063fa0 con 0x7f75200cd1e0 2024-09-16T07:36:02.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.001+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f75200b1120 con 0x7f75200cd1e0 2024-09-16T07:36:02.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.817+0000 7f7517e00640 1 -- 172.21.15.71:0/2685168437 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v130) ==== 138+0+0 (secure 0 0 0) 0x7f75100689d0 con 0x7f75200cd1e0 2024-09-16T07:36:02.817 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:36:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f750007a6b0 msgr2=0x7f750007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f750007a6b0 0x7f750007cb70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f750c033000 tx=0x7f750c004730 comp rx=0 tx=0).stop 2024-09-16T07:36:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75200cd1e0 msgr2=0x7f75200cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75200cd1e0 0x7f75200cb8b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f751000e9f0 tx=0x7f751000eeb0 comp rx=0 tx=0).stop 2024-09-16T07:36:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 shutdown_connections 2024-09-16T07:36:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f750007a6b0 0x7f750007cb70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f752015b700 0x7f75200c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7520158900 0x7f7520155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 --2- 172.21.15.71:0/2685168437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75200cd1e0 0x7f75200cb8b0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.820+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 >> 172.21.15.71:0/2685168437 conn(0x7f75200b18e0 msgr2=0x7f75201577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.821+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 shutdown_connections 2024-09-16T07:36:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:02.821+0000 7f7527000640 1 -- 172.21.15.71:0/2685168437 wait complete. 2024-09-16T07:36:02.842 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:36:03.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.010+0000 7f2654600640 1 -- 172.21.15.71:0/3718114068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2634004d40 msgr2=0x7f26340051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:03.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.010+0000 7f2654600640 1 --2- 172.21.15.71:0/3718114068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2634004d40 0x7f26340051a0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f2644006f90 tx=0x7f2634005830 comp rx=0 tx=0).stop 2024-09-16T07:36:03.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.010+0000 7f2652600640 1 -- 172.21.15.71:0/3718114068 reap_dead start 2024-09-16T07:36:03.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.010+0000 7f2654600640 1 -- 172.21.15.71:0/3718114068 shutdown_connections 2024-09-16T07:36:03.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.010+0000 7f2654600640 1 -- 172.21.15.71:0/3718114068 >> 172.21.15.71:0/3718114068 conn(0x7f264c0b1840 msgr2=0x7f264c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:03.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.010+0000 7f2654600640 1 -- 172.21.15.71:0/3718114068 shutdown_connections 2024-09-16T07:36:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.010+0000 7f2654600640 1 -- 172.21.15.71:0/3718114068 wait complete. 2024-09-16T07:36:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.012+0000 7f2654600640 1 Processor -- start 2024-09-16T07:36:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.012+0000 7f2654600640 1 -- start start 2024-09-16T07:36:03.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2654600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c0ac4a0 0x7f264c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:03.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2654600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f264c1f4c70 0x7f264c13f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:03.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2654600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c1f50f0 0x7f264c0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:03.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2654600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0ce2c0 con 0x7f264c0ac4a0 2024-09-16T07:36:03.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2654600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0736a0 con 0x7f264c1f50f0 2024-09-16T07:36:03.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2654600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0ce440 con 0x7f264c1f4c70 2024-09-16T07:36:03.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2651c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c0ac4a0 0x7f264c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:03.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2651200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f264c1f4c70 0x7f264c13f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:03.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2651c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c0ac4a0 0x7f264c0ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35876/0 (socket says 172.21.15.71:35876) 2024-09-16T07:36:03.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2651c00640 1 -- 172.21.15.71:0/2666663199 learned_addr learned my addr 172.21.15.71:0/2666663199 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:03.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.013+0000 7f2652600640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c1f50f0 0x7f264c0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2651c00640 1 -- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f264c1f4c70 msgr2=0x7f264c13f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2651c00640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f264c1f4c70 0x7f264c13f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2651c00640 1 -- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c1f50f0 msgr2=0x7f264c0b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2651c00640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c1f50f0 0x7f264c0b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2651c00640 1 -- 172.21.15.71:0/2666663199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c13ecc0 con 0x7f264c0ac4a0 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2651c00640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c0ac4a0 0x7f264c0ae230 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2640011d60 tx=0x7f264000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2649600640 1 -- 172.21.15.71:0/2666663199 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640003d80 con 0x7f264c0ac4a0 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2649600640 1 -- 172.21.15.71:0/2666663199 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2640011e50 con 0x7f264c0ac4a0 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2649600640 1 -- 172.21.15.71:0/2666663199 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640004d40 con 0x7f264c0ac4a0 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c1f5590 con 0x7f264c0ac4a0 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.014+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c1f5b50 con 0x7f264c0ac4a0 2024-09-16T07:36:03.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.015+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c0ce9a0 con 0x7f264c0ac4a0 2024-09-16T07:36:03.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.017+0000 7f2649600640 1 -- 172.21.15.71:0/2666663199 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f2640004050 con 0x7f264c0ac4a0 2024-09-16T07:36:03.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.018+0000 7f2649600640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f262c07a6b0 0x7f262c07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:03.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.018+0000 7f2649600640 1 -- 172.21.15.71:0/2666663199 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5311+0+0 (secure 0 0 0) 0x7f264009ad50 con 0x7f264c0ac4a0 2024-09-16T07:36:03.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.018+0000 7f2651200640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f262c07a6b0 0x7f262c07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:03.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.019+0000 7f2651200640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f262c07a6b0 0x7f262c07cb70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f264c1f9380 tx=0x7f2638006b70 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:03.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.021+0000 7f2649600640 1 -- 172.21.15.71:0/2666663199 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f264009afd0 con 0x7f264c0ac4a0 2024-09-16T07:36:03.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.209+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f264c0b03d0 con 0x7f264c0ac4a0 2024-09-16T07:36:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.828+0000 7f2649600640 1 -- 172.21.15.71:0/2666663199 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v131) ==== 170+0+0 (secure 0 0 0) 0x7f2640064850 con 0x7f264c0ac4a0 2024-09-16T07:36:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:36:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f262c07a6b0 msgr2=0x7f262c07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f262c07a6b0 0x7f262c07cb70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f264c1f9380 tx=0x7f2638006b70 comp rx=0 tx=0).stop 2024-09-16T07:36:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c0ac4a0 msgr2=0x7f264c0ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c0ac4a0 0x7f264c0ae230 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2640011d60 tx=0x7f264000f790 comp rx=0 tx=0).stop 2024-09-16T07:36:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 shutdown_connections 2024-09-16T07:36:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f262c07a6b0 0x7f262c07cb70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:03.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c1f50f0 0x7f264c0b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:03.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f264c1f4c70 0x7f264c13f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:03.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 --2- 172.21.15.71:0/2666663199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f264c0ac4a0 0x7f264c0ae230 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:03.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 >> 172.21.15.71:0/2666663199 conn(0x7f264c0b1840 msgr2=0x7f264c1437b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:03.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 shutdown_connections 2024-09-16T07:36:03.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:03.831+0000 7f2654600640 1 -- 172.21.15.71:0/2666663199 wait complete. 2024-09-16T07:36:03.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:36:04.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.021+0000 7f6513200640 1 -- 172.21.15.71:0/849048701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f00052e0 msgr2=0x7f64f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:04.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.021+0000 7f6513200640 1 --2- 172.21.15.71:0/849048701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f00052e0 0x7f64f000b9c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6504009ed0 tx=0x7f64f0005830 comp rx=0 tx=0).stop 2024-09-16T07:36:04.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.021+0000 7f6513200640 1 -- 172.21.15.71:0/849048701 shutdown_connections 2024-09-16T07:36:04.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.021+0000 7f6513200640 1 --2- 172.21.15.71:0/849048701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f00052e0 0x7f64f000b9c0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:04.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.021+0000 7f6513200640 1 -- 172.21.15.71:0/849048701 >> 172.21.15.71:0/849048701 conn(0x7f650c0b1840 msgr2=0x7f650c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:04.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.021+0000 7f6513200640 1 -- 172.21.15.71:0/849048701 shutdown_connections 2024-09-16T07:36:04.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.021+0000 7f6513200640 1 -- 172.21.15.71:0/849048701 wait complete. 2024-09-16T07:36:04.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.022+0000 7f6513200640 1 Processor -- start 2024-09-16T07:36:04.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.022+0000 7f6513200640 1 -- start start 2024-09-16T07:36:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.023+0000 7f6513200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f00052e0 0x7f650c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.023+0000 7f6513200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650c1444e0 0x7f650c144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.023+0000 7f6513200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c0b3c90 0x7f650c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.023+0000 7f6513200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f650c0737d0 con 0x7f650c0b3c90 2024-09-16T07:36:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.023+0000 7f6513200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f650c073650 con 0x7f64f00052e0 2024-09-16T07:36:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.023+0000 7f6513200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f650c0cc5c0 con 0x7f650c1444e0 2024-09-16T07:36:04.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f6511200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c0b3c90 0x7f650c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:04.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f6511200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c0b3c90 0x7f650c0b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35898/0 (socket says 172.21.15.71:35898) 2024-09-16T07:36:04.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f6511200640 1 -- 172.21.15.71:0/3125684453 learned_addr learned my addr 172.21.15.71:0/3125684453 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:04.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f650be00640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f00052e0 0x7f650c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:04.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f650b400640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650c1444e0 0x7f650c144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:04.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f6511200640 1 -- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650c1444e0 msgr2=0x7f650c144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:04.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f6511200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650c1444e0 0x7f650c144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:04.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f6511200640 1 -- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f00052e0 msgr2=0x7f650c15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:04.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f6511200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f00052e0 0x7f650c15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.024+0000 7f6511200640 1 -- 172.21.15.71:0/3125684453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c1f59e0 con 0x7f650c0b3c90 2024-09-16T07:36:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.025+0000 7f6511200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c0b3c90 0x7f650c0b4140 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6504009ed0 tx=0x7f65040042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.025+0000 7f6503e00640 1 -- 172.21.15.71:0/3125684453 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504009200 con 0x7f650c0b3c90 2024-09-16T07:36:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.025+0000 7f6503e00640 1 -- 172.21.15.71:0/3125684453 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65040047c0 con 0x7f650c0b3c90 2024-09-16T07:36:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.025+0000 7f6503e00640 1 -- 172.21.15.71:0/3125684453 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650403c6e0 con 0x7f650c0b3c90 2024-09-16T07:36:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.025+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c1f4aa0 con 0x7f650c0b3c90 2024-09-16T07:36:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.025+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c1f5040 con 0x7f650c0b3c90 2024-09-16T07:36:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.026+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c15f9f0 con 0x7f650c0b3c90 2024-09-16T07:36:04.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.028+0000 7f6503e00640 1 -- 172.21.15.71:0/3125684453 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f6504044020 con 0x7f650c0b3c90 2024-09-16T07:36:04.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.029+0000 7f6503e00640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64ec07a6b0 0x7f64ec07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:04.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.029+0000 7f6503e00640 1 -- 172.21.15.71:0/3125684453 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5693+0+0 (secure 0 0 0) 0x7f65040bb130 con 0x7f650c0b3c90 2024-09-16T07:36:04.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.029+0000 7f650be00640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64ec07a6b0 0x7f64ec07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:04.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.030+0000 7f650be00640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64ec07a6b0 0x7f64ec07cb70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f64fc000c00 tx=0x7f64fc006b70 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:04.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.032+0000 7f6503e00640 1 -- 172.21.15.71:0/3125684453 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f650404b020 con 0x7f650c0b3c90 2024-09-16T07:36:04.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.251+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f650c0cf180 con 0x7f650c0b3c90 2024-09-16T07:36:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.840+0000 7f6503e00640 1 -- 172.21.15.71:0/3125684453 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v81) ==== 174+0+0 (secure 0 0 0) 0x7f6504085280 con 0x7f650c0b3c90 2024-09-16T07:36:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 18 and data pool 19 2024-09-16T07:36:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64ec07a6b0 msgr2=0x7f64ec07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64ec07a6b0 0x7f64ec07cb70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f64fc000c00 tx=0x7f64fc006b70 comp rx=0 tx=0).stop 2024-09-16T07:36:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c0b3c90 msgr2=0x7f650c0b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c0b3c90 0x7f650c0b4140 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6504009ed0 tx=0x7f65040042d0 comp rx=0 tx=0).stop 2024-09-16T07:36:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 shutdown_connections 2024-09-16T07:36:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64ec07a6b0 0x7f64ec07cb70 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c0b3c90 0x7f650c0b4140 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650c1444e0 0x7f650c144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 --2- 172.21.15.71:0/3125684453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f00052e0 0x7f650c15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 >> 172.21.15.71:0/3125684453 conn(0x7f650c0b1840 msgr2=0x7f650c0cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:04.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 shutdown_connections 2024-09-16T07:36:04.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:04.846+0000 7f6513200640 1 -- 172.21.15.71:0/3125684453 wait complete. 2024-09-16T07:36:04.876 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:36:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.033+0000 7f3ee6000640 1 -- 172.21.15.71:0/806076813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee00ad270 msgr2=0x7f3ee0073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.033+0000 7f3ee6000640 1 --2- 172.21.15.71:0/806076813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee00ad270 0x7f3ee0073fb0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00723f0 tx=0x7f3ed802a980 comp rx=0 tx=0).stop 2024-09-16T07:36:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.034+0000 7f3ee6000640 1 -- 172.21.15.71:0/806076813 shutdown_connections 2024-09-16T07:36:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.034+0000 7f3ee6000640 1 --2- 172.21.15.71:0/806076813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee00ad270 0x7f3ee0073fb0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.034+0000 7f3ee6000640 1 --2- 172.21.15.71:0/806076813 >> v2:172.21.15.177:3300/0 conn(0x7f3ee00ccf00 0x7f3ee00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.034+0000 7f3ee6000640 1 --2- 172.21.15.71:0/806076813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ee015f5a0 0x7f3ee00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.034+0000 7f3ee6000640 1 -- 172.21.15.71:0/806076813 >> 172.21.15.71:0/806076813 conn(0x7f3ee00b1950 msgr2=0x7f3ee00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.034+0000 7f3ee6000640 1 -- 172.21.15.71:0/806076813 shutdown_connections 2024-09-16T07:36:05.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.034+0000 7f3ee6000640 1 -- 172.21.15.71:0/806076813 wait complete. 2024-09-16T07:36:05.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.035+0000 7f3ee6000640 1 Processor -- start 2024-09-16T07:36:05.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.035+0000 7f3ee6000640 1 -- start start 2024-09-16T07:36:05.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.036+0000 7f3ee6000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ee00ad270 0x7f3ee0153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.036+0000 7f3ee6000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ee00ccf00 0x7f3ee01547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.036+0000 7f3ee6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee015f5a0 0x7f3ee0154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.036+0000 7f3ee6000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee006f160 con 0x7f3ee015f5a0 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.036+0000 7f3ee6000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee0074780 con 0x7f3ee00ccf00 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.036+0000 7f3ee6000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee006f2e0 con 0x7f3ee00ad270 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.036+0000 7f3edfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee015f5a0 0x7f3ee0154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee015f5a0 0x7f3ee0154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40344/0 (socket says 172.21.15.71:40344) 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edfe00640 1 -- 172.21.15.71:0/3579276962 learned_addr learned my addr 172.21.15.71:0/3579276962 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edea00640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ee00ccf00 0x7f3ee01547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edf400640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ee00ad270 0x7f3ee0153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edfe00640 1 -- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ee00ad270 msgr2=0x7f3ee0153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edfe00640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ee00ad270 0x7f3ee0153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edfe00640 1 -- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ee00ccf00 msgr2=0x7f3ee01547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edfe00640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ee00ccf00 0x7f3ee01547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edfe00640 1 -- 172.21.15.71:0/3579276962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee01f2670 con 0x7f3ee015f5a0 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3edfe00640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee015f5a0 0x7f3ee0154d10 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009d60 tx=0x7f3ed8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3ed7400640 1 -- 172.21.15.71:0/3579276962 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8034050 con 0x7f3ee015f5a0 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.037+0000 7f3ed7400640 1 -- 172.21.15.71:0/3579276962 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ed8004450 con 0x7f3ee015f5a0 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.038+0000 7f3ed7400640 1 -- 172.21.15.71:0/3579276962 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8043440 con 0x7f3ee015f5a0 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.038+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee01f6950 con 0x7f3ee015f5a0 2024-09-16T07:36:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.038+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee01f2bd0 con 0x7f3ee015f5a0 2024-09-16T07:36:05.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.039+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee006f870 con 0x7f3ee015f5a0 2024-09-16T07:36:05.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.040+0000 7f3ed7400640 1 -- 172.21.15.71:0/3579276962 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f3ed8037020 con 0x7f3ee015f5a0 2024-09-16T07:36:05.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.041+0000 7f3ed7400640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ec007a6b0 0x7f3ec007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.042+0000 7f3ed7400640 1 -- 172.21.15.71:0/3579276962 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f3ed80baee0 con 0x7f3ee015f5a0 2024-09-16T07:36:05.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.042+0000 7f3edf400640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ec007a6b0 0x7f3ec007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.043+0000 7f3edf400640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ec007a6b0 0x7f3ec007cb70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0005fd0 tx=0x7f3ed0005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:05.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.046+0000 7f3ed7400640 1 -- 172.21.15.71:0/3579276962 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3ee01f2670 con 0x7f3ee015f5a0 2024-09-16T07:36:05.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.230+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ee00afc00 con 0x7f3ee015f5a0 2024-09-16T07:36:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.231+0000 7f3ed7400640 1 -- 172.21.15.71:0/3579276962 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16845 (secure 0 0 0) 0x7f3ed8084f30 con 0x7f3ee015f5a0 2024-09-16T07:36:05.231 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:05.231 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":132,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:36:04.253055+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:36:02.011721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T07:36:03.220374+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:36:05.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.233+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ec007a6b0 msgr2=0x7f3ec007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.233+0000 7f3ee6000640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ec007a6b0 0x7f3ec007cb70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0005fd0 tx=0x7f3ed0005850 comp rx=0 tx=0).stop 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee015f5a0 msgr2=0x7f3ee0154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee015f5a0 0x7f3ee0154d10 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009d60 tx=0x7f3ed8003f10 comp rx=0 tx=0).stop 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 shutdown_connections 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3ec007a6b0 0x7f3ec007cb70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee015f5a0 0x7f3ee0154d10 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ee00ccf00 0x7f3ee01547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 --2- 172.21.15.71:0/3579276962 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ee00ad270 0x7f3ee0153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 >> 172.21.15.71:0/3579276962 conn(0x7f3ee00b1950 msgr2=0x7f3ee009b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 shutdown_connections 2024-09-16T07:36:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.234+0000 7f3ee6000640 1 -- 172.21.15.71:0/3579276962 wait complete. 2024-09-16T07:36:05.249 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:36:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 -- 172.21.15.71:0/697979876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc80cc200 msgr2=0x7f7fc8074290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 --2- 172.21.15.71:0/697979876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc80cc200 0x7f7fc8074290 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8075330 tx=0x7f7fc002aa10 comp rx=0 tx=0).stop 2024-09-16T07:36:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 -- 172.21.15.71:0/697979876 shutdown_connections 2024-09-16T07:36:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 --2- 172.21.15.71:0/697979876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc80cc200 0x7f7fc8074290 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 --2- 172.21.15.71:0/697979876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc80cc750 0x7f7fc80b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 --2- 172.21.15.71:0/697979876 >> v2:172.21.15.177:3300/0 conn(0x7f7fc815e0f0 0x7f7fc80b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 -- 172.21.15.71:0/697979876 >> 172.21.15.71:0/697979876 conn(0x7f7fc80b1e90 msgr2=0x7f7fc80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 -- 172.21.15.71:0/697979876 shutdown_connections 2024-09-16T07:36:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.407+0000 7f7fcda00640 1 -- 172.21.15.71:0/697979876 wait complete. 2024-09-16T07:36:05.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.409+0000 7f7fcda00640 1 Processor -- start 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.409+0000 7f7fcda00640 1 -- start start 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcda00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc80cc200 0x7f7fc80c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcda00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc80cc750 0x7f7fc81f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc815e0f0 0x7f7fc806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcda00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8074950 con 0x7f7fc815e0f0 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcda00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc80747d0 con 0x7f7fc80cc200 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcda00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc8074ad0 con 0x7f7fc80cc750 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcd000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc815e0f0 0x7f7fc806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcd000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc815e0f0 0x7f7fc806f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40362/0 (socket says 172.21.15.71:40362) 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcd000640 1 -- 172.21.15.71:0/3247327774 learned_addr learned my addr 172.21.15.71:0/3247327774 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fc7e00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc80cc200 0x7f7fc80c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fc7400640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc80cc750 0x7f7fc81f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcd000640 1 -- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc80cc750 msgr2=0x7f7fc81f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcd000640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc80cc750 0x7f7fc81f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcd000640 1 -- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc80cc200 msgr2=0x7f7fc80c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcd000640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc80cc200 0x7f7fc80c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fcd000640 1 -- 172.21.15.71:0/3247327774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc80c4850 con 0x7f7fc815e0f0 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fc7e00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc80cc200 0x7f7fc80c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.410+0000 7f7fc7400640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc80cc750 0x7f7fc81f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.411+0000 7f7fcd000640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc815e0f0 0x7f7fc806f120 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009d60 tx=0x7f7fc0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.411+0000 7f7fbfe00640 1 -- 172.21.15.71:0/3247327774 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0047070 con 0x7f7fc815e0f0 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.411+0000 7f7fbfe00640 1 -- 172.21.15.71:0/3247327774 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fc0004250 con 0x7f7fc815e0f0 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.411+0000 7f7fbfe00640 1 -- 172.21.15.71:0/3247327774 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0033050 con 0x7f7fc815e0f0 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.411+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc80c4b30 con 0x7f7fc815e0f0 2024-09-16T07:36:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.411+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc81f49b0 con 0x7f7fc815e0f0 2024-09-16T07:36:05.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.412+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc8074fc0 con 0x7f7fc815e0f0 2024-09-16T07:36:05.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.413+0000 7f7fbfe00640 1 -- 172.21.15.71:0/3247327774 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f7fc004a070 con 0x7f7fc815e0f0 2024-09-16T07:36:05.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.414+0000 7f7fbfe00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7fa807a6b0 0x7f7fa807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.414+0000 7f7fbfe00640 1 -- 172.21.15.71:0/3247327774 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f7fc00baca0 con 0x7f7fc815e0f0 2024-09-16T07:36:05.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.415+0000 7f7fc7e00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7fa807a6b0 0x7f7fa807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.416+0000 7f7fc7e00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7fa807a6b0 0x7f7fa807cb70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800ab40 tx=0x7f7fb800a520 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:05.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.418+0000 7f7fbfe00640 1 -- 172.21.15.71:0/3247327774 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7fc00c5020 con 0x7f7fc815e0f0 2024-09-16T07:36:05.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.603+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fc80d04c0 con 0x7f7fc815e0f0 2024-09-16T07:36:05.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.605+0000 7f7fbfe00640 1 -- 172.21.15.71:0/3247327774 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16845 (secure 0 0 0) 0x7f7fc0084d70 con 0x7f7fc815e0f0 2024-09-16T07:36:05.604 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:05.604 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":132,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:36:04.253055+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:36:02.011721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T07:36:03.220374+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7fa807a6b0 msgr2=0x7f7fa807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7fa807a6b0 0x7f7fa807cb70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800ab40 tx=0x7f7fb800a520 comp rx=0 tx=0).stop 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc815e0f0 msgr2=0x7f7fc806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc815e0f0 0x7f7fc806f120 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009d60 tx=0x7f7fc0003f10 comp rx=0 tx=0).stop 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 shutdown_connections 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7fa807a6b0 0x7f7fa807cb70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc815e0f0 0x7f7fc806f120 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc80cc750 0x7f7fc81f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 --2- 172.21.15.71:0/3247327774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc80cc200 0x7f7fc80c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 >> 172.21.15.71:0/3247327774 conn(0x7f7fc80b1e90 msgr2=0x7f7fc80d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:05.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 shutdown_connections 2024-09-16T07:36:05.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.607+0000 7f7fcda00640 1 -- 172.21.15.71:0/3247327774 wait complete. 2024-09-16T07:36:05.622 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:36:05.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.782+0000 7fd7f9400640 1 -- 172.21.15.71:0/2668907831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41618c0 msgr2=0x7fd7f40b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.782+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2668907831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41618c0 0x7fd7f40b39e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd7f408e020 tx=0x7fd7dc02a560 comp rx=0 tx=0).stop 2024-09-16T07:36:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.783+0000 7fd7f9400640 1 -- 172.21.15.71:0/2668907831 shutdown_connections 2024-09-16T07:36:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.783+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2668907831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7f40b3f20 0x7fd7f40cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.783+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2668907831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41618c0 0x7fd7f40b39e0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.783+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2668907831 >> v2:172.21.15.177:3300/0 conn(0x7fd7f41614e0 0x7fd7f40d4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.783+0000 7fd7f9400640 1 -- 172.21.15.71:0/2668907831 >> 172.21.15.71:0/2668907831 conn(0x7fd7f40d40a0 msgr2=0x7fd7f40d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.783+0000 7fd7f9400640 1 -- 172.21.15.71:0/2668907831 shutdown_connections 2024-09-16T07:36:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.783+0000 7fd7f9400640 1 -- 172.21.15.71:0/2668907831 wait complete. 2024-09-16T07:36:05.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.784+0000 7fd7f9400640 1 Processor -- start 2024-09-16T07:36:05.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.784+0000 7fd7f9400640 1 -- start start 2024-09-16T07:36:05.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7f9400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7f40b3f20 0x7fd7f40cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7f9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41614e0 0x7fd7f40c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7f9400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7f41618c0 0x7fd7f4150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7f9400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4162130 con 0x7fd7f41614e0 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7f9400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4161fb0 con 0x7fd7f40b3f20 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7f9400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f41622b0 con 0x7fd7f41618c0 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7e9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41614e0 0x7fd7f40c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7e9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41614e0 0x7fd7f40c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40388/0 (socket says 172.21.15.71:40388) 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.785+0000 7fd7e9e00640 1 -- 172.21.15.71:0/2774375541 learned_addr learned my addr 172.21.15.71:0/2774375541 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.786+0000 7fd7f3200640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7f41618c0 0x7fd7f4150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.786+0000 7fd7f2800640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7f40b3f20 0x7fd7f40cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.786+0000 7fd7e9e00640 1 -- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7f41618c0 msgr2=0x7fd7f4150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.786+0000 7fd7e9e00640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7f41618c0 0x7fd7f4150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.786+0000 7fd7e9e00640 1 -- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7f40b3f20 msgr2=0x7fd7f40cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.786+0000 7fd7e9e00640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7f40b3f20 0x7fd7f40cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:05.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.786+0000 7fd7e9e00640 1 -- 172.21.15.71:0/2774375541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f40b8b80 con 0x7fd7f41614e0 2024-09-16T07:36:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.786+0000 7fd7e9e00640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41614e0 0x7fd7f40c7870 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc034040 tx=0x7fd7dc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.787+0000 7fd7eb400640 1 -- 172.21.15.71:0/2774375541 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc03e030 con 0x7fd7f41614e0 2024-09-16T07:36:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.787+0000 7fd7eb400640 1 -- 172.21.15.71:0/2774375541 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7dc0047a0 con 0x7fd7f41614e0 2024-09-16T07:36:05.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.787+0000 7fd7eb400640 1 -- 172.21.15.71:0/2774375541 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc035040 con 0x7fd7f41614e0 2024-09-16T07:36:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.787+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f40b8e60 con 0x7fd7f41614e0 2024-09-16T07:36:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.787+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f40b9420 con 0x7fd7f41614e0 2024-09-16T07:36:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.788+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f40d4bc0 con 0x7fd7f41614e0 2024-09-16T07:36:05.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.789+0000 7fd7eb400640 1 -- 172.21.15.71:0/2774375541 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fd7dc041030 con 0x7fd7f41614e0 2024-09-16T07:36:05.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.790+0000 7fd7eb400640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd7d007a6b0 0x7fd7d007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:05.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.790+0000 7fd7eb400640 1 -- 172.21.15.71:0/2774375541 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7fd7dc0ba040 con 0x7fd7f41614e0 2024-09-16T07:36:05.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.790+0000 7fd7f2800640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd7d007a6b0 0x7fd7d007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:05.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.791+0000 7fd7f2800640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd7d007a6b0 0x7fd7d007cb70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400a400 tx=0x7fd7e400d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:05.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:05.794+0000 7fd7eb400640 1 -- 172.21.15.71:0/2774375541 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd7dc083ac0 con 0x7fd7f41614e0 2024-09-16T07:36:06.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.023+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd7f40ba020 con 0x7fd7f41614e0 2024-09-16T07:36:06.821 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-09-16T07:36:06.821+0000 7f8418e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:36:06.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.853+0000 7fd7eb400640 1 -- 172.21.15.71:0/2774375541 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) ==== 118+0+0 (secure 0 0 0) 0x7fd7dc0884f0 con 0x7fd7f41614e0 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd7d007a6b0 msgr2=0x7fd7d007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd7d007a6b0 0x7fd7d007cb70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400a400 tx=0x7fd7e400d040 comp rx=0 tx=0).stop 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41614e0 msgr2=0x7fd7f40c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41614e0 0x7fd7f40c7870 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc034040 tx=0x7fd7dc0039e0 comp rx=0 tx=0).stop 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 shutdown_connections 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd7d007a6b0 0x7fd7d007cb70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7f41618c0 0x7fd7f4150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f41614e0 0x7fd7f40c7870 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 --2- 172.21.15.71:0/2774375541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7f40b3f20 0x7fd7f40cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 >> 172.21.15.71:0/2774375541 conn(0x7fd7f40d40a0 msgr2=0x7fd7f40b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 shutdown_connections 2024-09-16T07:36:06.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:06.856+0000 7fd7f9400640 1 -- 172.21.15.71:0/2774375541 wait complete. 2024-09-16T07:36:06.875 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:36:06.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:36:07.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.040+0000 7f15da600640 1 -- 172.21.15.71:0/3345460998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 msgr2=0x7f15d406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.040+0000 7f15da600640 1 --2- 172.21.15.71:0/3345460998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 0x7f15d406e860 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f15d40cd540 tx=0x7f15cc02aac0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.040+0000 7f15da600640 1 -- 172.21.15.71:0/3345460998 shutdown_connections 2024-09-16T07:36:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.040+0000 7f15da600640 1 --2- 172.21.15.71:0/3345460998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 0x7f15d406e860 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.040+0000 7f15da600640 1 --2- 172.21.15.71:0/3345460998 >> v2:172.21.15.177:3300/0 conn(0x7f15d40b4bd0 0x7f15d40b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.040+0000 7f15da600640 1 -- 172.21.15.71:0/3345460998 >> 172.21.15.71:0/3345460998 conn(0x7f15d40b1840 msgr2=0x7f15d40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.040+0000 7f15da600640 1 -- 172.21.15.71:0/3345460998 shutdown_connections 2024-09-16T07:36:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.040+0000 7f15da600640 1 -- 172.21.15.71:0/3345460998 wait complete. 2024-09-16T07:36:07.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.041+0000 7f15da600640 1 Processor -- start 2024-09-16T07:36:07.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.042+0000 7f15da600640 1 -- start start 2024-09-16T07:36:07.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.042+0000 7f15da600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 0x7f15d41f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:07.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.042+0000 7f15da600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d40b4bd0 0x7f15d4153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:07.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.042+0000 7f15da600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15d41437c0 0x7f15d40ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:07.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.042+0000 7f15da600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f15d406ef20 con 0x7f15d40b4400 2024-09-16T07:36:07.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.042+0000 7f15da600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15d406eda0 con 0x7f15d40b4bd0 2024-09-16T07:36:07.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.042+0000 7f15da600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f15d406f0a0 con 0x7f15d41437c0 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 0x7f15d41f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 0x7f15d41f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40408/0 (socket says 172.21.15.71:40408) 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 -- 172.21.15.71:0/4285239539 learned_addr learned my addr 172.21.15.71:0/4285239539 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15cbe00640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d40b4bd0 0x7f15d4153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3e00640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15d41437c0 0x7f15d40ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 -- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15d41437c0 msgr2=0x7f15d40ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15d41437c0 0x7f15d40ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 -- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d40b4bd0 msgr2=0x7f15d4153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d40b4bd0 0x7f15d4153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 -- 172.21.15.71:0/4285239539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d41f6500 con 0x7f15d40b4400 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15cbe00640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d40b4bd0 0x7f15d4153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.043+0000 7f15d3400640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 0x7f15d41f5c20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c3f0 tx=0x7f15c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:07.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.044+0000 7f15cb400640 1 -- 172.21.15.71:0/4285239539 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4017070 con 0x7f15d40b4400 2024-09-16T07:36:07.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.044+0000 7f15cb400640 1 -- 172.21.15.71:0/4285239539 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15c40035f0 con 0x7f15d40b4400 2024-09-16T07:36:07.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.044+0000 7f15cb400640 1 -- 172.21.15.71:0/4285239539 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4003a40 con 0x7f15d40b4400 2024-09-16T07:36:07.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.044+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41f67e0 con 0x7f15d40b4400 2024-09-16T07:36:07.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.044+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d4140d90 con 0x7f15d40b4400 2024-09-16T07:36:07.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.045+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d40d0620 con 0x7f15d40b4400 2024-09-16T07:36:07.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.046+0000 7f15cb400640 1 -- 172.21.15.71:0/4285239539 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f15c401a070 con 0x7f15d40b4400 2024-09-16T07:36:07.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.047+0000 7f15cb400640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f15b007a6e0 0x7f15b007cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:07.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.047+0000 7f15cb400640 1 -- 172.21.15.71:0/4285239539 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f15c4098c80 con 0x7f15d40b4400 2024-09-16T07:36:07.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.047+0000 7f15cbe00640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f15b007a6e0 0x7f15b007cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:07.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.048+0000 7f15cbe00640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f15b007a6e0 0x7f15b007cba0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f15d40ccf80 tx=0x7f15bc005e50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:07.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.051+0000 7f15cb400640 1 -- 172.21.15.71:0/4285239539 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f15d40d0620 con 0x7f15d40b4400 2024-09-16T07:36:07.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.273+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f15d41411c0 con 0x7f15d40b4400 2024-09-16T07:36:07.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.860+0000 7f15cb400640 1 -- 172.21.15.71:0/4285239539 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) ==== 122+0+0 (secure 0 0 0) 0x7f15c4062d50 con 0x7f15d40b4400 2024-09-16T07:36:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f15b007a6e0 msgr2=0x7f15b007cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f15b007a6e0 0x7f15b007cba0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f15d40ccf80 tx=0x7f15bc005e50 comp rx=0 tx=0).stop 2024-09-16T07:36:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 msgr2=0x7f15d41f5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 0x7f15d41f5c20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c3f0 tx=0x7f15c400c8b0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 shutdown_connections 2024-09-16T07:36:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f15b007a6e0 0x7f15b007cba0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15d41437c0 0x7f15d40ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d40b4bd0 0x7f15d4153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 --2- 172.21.15.71:0/4285239539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15d40b4400 0x7f15d41f5c20 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 >> 172.21.15.71:0/4285239539 conn(0x7f15d40b1840 msgr2=0x7f15d415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.863+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 shutdown_connections 2024-09-16T07:36:07.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:07.864+0000 7f15da600640 1 -- 172.21.15.71:0/4285239539 wait complete. 2024-09-16T07:36:07.881 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:36:07.881 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:36:08.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.043+0000 7fe656600640 1 -- 172.21.15.71:0/1849909113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe634003a30 msgr2=0x7fe634003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.043+0000 7fe656600640 1 --2- 172.21.15.71:0/1849909113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe634003a30 0x7fe634003e50 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe6400016c0 tx=0x7fe634005830 comp rx=0 tx=0).stop 2024-09-16T07:36:08.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.043+0000 7fe64fe00640 1 -- 172.21.15.71:0/1849909113 reap_dead start 2024-09-16T07:36:08.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.043+0000 7fe656600640 1 -- 172.21.15.71:0/1849909113 shutdown_connections 2024-09-16T07:36:08.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.043+0000 7fe656600640 1 -- 172.21.15.71:0/1849909113 >> 172.21.15.71:0/1849909113 conn(0x7fe6500b1950 msgr2=0x7fe6500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:08.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.043+0000 7fe656600640 1 -- 172.21.15.71:0/1849909113 shutdown_connections 2024-09-16T07:36:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.043+0000 7fe656600640 1 -- 172.21.15.71:0/1849909113 wait complete. 2024-09-16T07:36:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.045+0000 7fe656600640 1 Processor -- start 2024-09-16T07:36:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.045+0000 7fe656600640 1 -- start start 2024-09-16T07:36:08.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe656600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe634003a30 0x7fe6501576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe656600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe650151df0 0x7fe65015f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe656600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe650151030 0x7fe6501514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe656600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe650166f80 con 0x7fe650151df0 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe656600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe650166e00 con 0x7fe650151030 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe656600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe650167100 con 0x7fe634003a30 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe647e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe650151df0 0x7fe65015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe647e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe650151df0 0x7fe65015f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40436/0 (socket says 172.21.15.71:40436) 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe647e00640 1 -- 172.21.15.71:0/3921866059 learned_addr learned my addr 172.21.15.71:0/3921866059 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe64f400640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe634003a30 0x7fe6501576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe64fe00640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe650151030 0x7fe6501514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe647e00640 1 -- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe634003a30 msgr2=0x7fe6501576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe647e00640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe634003a30 0x7fe6501576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe647e00640 1 -- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe650151030 msgr2=0x7fe6501514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe647e00640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe650151030 0x7fe6501514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe647e00640 1 -- 172.21.15.71:0/3921866059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6500b0b30 con 0x7fe650151df0 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.046+0000 7fe64fe00640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe650151030 0x7fe6501514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.047+0000 7fe647e00640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe650151df0 0x7fe65015f600 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe6380066e0 tx=0x7fe6380039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.047+0000 7fe647400640 1 -- 172.21.15.71:0/3921866059 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638015070 con 0x7fe650151df0 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.047+0000 7fe647400640 1 -- 172.21.15.71:0/3921866059 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe638011900 con 0x7fe650151df0 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.047+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6500b0e10 con 0x7fe650151df0 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.047+0000 7fe647400640 1 -- 172.21.15.71:0/3921866059 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638011d30 con 0x7fe650151df0 2024-09-16T07:36:08.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.047+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6501f8d80 con 0x7fe650151df0 2024-09-16T07:36:08.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.048+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65006ef50 con 0x7fe650151df0 2024-09-16T07:36:08.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.049+0000 7fe647400640 1 -- 172.21.15.71:0/3921866059 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fe638018070 con 0x7fe650151df0 2024-09-16T07:36:08.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.050+0000 7fe647400640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe62c07a6b0 0x7fe62c07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.050+0000 7fe647400640 1 -- 172.21.15.71:0/3921866059 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7fe63809a6f0 con 0x7fe650151df0 2024-09-16T07:36:08.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.051+0000 7fe64f400640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe62c07a6b0 0x7fe62c07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.051+0000 7fe64f400640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe62c07a6b0 0x7fe62c07cb70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe640032000 tx=0x7fe640001190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:08.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.054+0000 7fe647400640 1 -- 172.21.15.71:0/3921866059 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe638064190 con 0x7fe650151df0 2024-09-16T07:36:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.275+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6500ada50 con 0x7fe650151df0 2024-09-16T07:36:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.275+0000 7fe647400640 1 -- 172.21.15.71:0/3921866059 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5231 (secure 0 0 0) 0x7fe638068bc0 con 0x7fe650151df0 2024-09-16T07:36:08.276 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:08.276 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":85,"btime":"2024-09-16T07:36:07:854566+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4275610322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":4275610322},{"type":"v1","addr":"172.21.15.71:6835","nonce":4275610322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/2559248159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2559248159},{"type":"v1","addr":"172.21.15.177:6836","nonce":2559248159}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/1515242906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1515242906},{"type":"v1","addr":"172.21.15.71:6834","nonce":1515242906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:36:04.253023+0000","modified":"2024-09-16T07:36:07.274434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2933333093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2933333093},{"type":"v1","addr":"172.21.15.177:6837","nonce":2933333093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":9}]} 2024-09-16T07:36:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 85 2024-09-16T07:36:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe62c07a6b0 msgr2=0x7fe62c07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe62c07a6b0 0x7fe62c07cb70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe640032000 tx=0x7fe640001190 comp rx=0 tx=0).stop 2024-09-16T07:36:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe650151df0 msgr2=0x7fe65015f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe650151df0 0x7fe65015f600 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe6380066e0 tx=0x7fe6380039c0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 shutdown_connections 2024-09-16T07:36:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe62c07a6b0 0x7fe62c07cb70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe650151030 0x7fe6501514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe650151df0 0x7fe65015f600 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 --2- 172.21.15.71:0/3921866059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe634003a30 0x7fe6501576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 >> 172.21.15.71:0/3921866059 conn(0x7fe6500b1950 msgr2=0x7fe6500b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 shutdown_connections 2024-09-16T07:36:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.278+0000 7fe656600640 1 -- 172.21.15.71:0/3921866059 wait complete. 2024-09-16T07:36:08.293 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:36:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.449+0000 7f22ace00640 1 -- 172.21.15.71:0/3961217198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a80b4020 msgr2=0x7f22a80b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.449+0000 7f22ace00640 1 --2- 172.21.15.71:0/3961217198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a80b4020 0x7f22a80b1770 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f22a80d4ea0 tx=0x7f22a002a490 comp rx=0 tx=0).stop 2024-09-16T07:36:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.450+0000 7f22ace00640 1 -- 172.21.15.71:0/3961217198 shutdown_connections 2024-09-16T07:36:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.450+0000 7f22ace00640 1 --2- 172.21.15.71:0/3961217198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a80b4020 0x7f22a80b1770 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.450+0000 7f22ace00640 1 --2- 172.21.15.71:0/3961217198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22a8161440 0x7f22a80b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.450+0000 7f22ace00640 1 -- 172.21.15.71:0/3961217198 >> 172.21.15.71:0/3961217198 conn(0x7f22a80d3df0 msgr2=0x7f22a80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.450+0000 7f22ace00640 1 -- 172.21.15.71:0/3961217198 shutdown_connections 2024-09-16T07:36:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.450+0000 7f22ace00640 1 -- 172.21.15.71:0/3961217198 wait complete. 2024-09-16T07:36:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.451+0000 7f22ace00640 1 Processor -- start 2024-09-16T07:36:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.451+0000 7f22ace00640 1 -- start start 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22ace00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22a80b34b0 0x7f22a81f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22ace00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22a80b4020 0x7f22a8156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22ace00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a8161440 0x7f22a80b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22ace00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80b2040 con 0x7f22a8161440 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22ace00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80b1ec0 con 0x7f22a80b4020 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22ace00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f22a80b21c0 con 0x7f22a80b34b0 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a8161440 0x7f22a80b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a8161440 0x7f22a80b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40446/0 (socket says 172.21.15.71:40446) 2024-09-16T07:36:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.452+0000 7f22a7e00640 1 -- 172.21.15.71:0/1815089467 learned_addr learned my addr 172.21.15.71:0/1815089467 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:08.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f22a7400640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22a80b34b0 0x7f22a81f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f22a6a00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22a80b4020 0x7f22a8156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f22a7e00640 1 -- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22a80b34b0 msgr2=0x7f22a81f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f22a7e00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22a80b34b0 0x7f22a81f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f22a7e00640 1 -- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22a80b4020 msgr2=0x7f22a8156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f22a7e00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22a80b4020 0x7f22a8156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f22a7e00640 1 -- 172.21.15.71:0/1815089467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a80b6f70 con 0x7f22a8161440 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f22a7e00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a8161440 0x7f22a80b8010 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009230 tx=0x7f22a0003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f229f400640 1 -- 172.21.15.71:0/1815089467 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a003f070 con 0x7f22a8161440 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.453+0000 7f229f400640 1 -- 172.21.15.71:0/1815089467 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22a0004890 con 0x7f22a8161440 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.454+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a80b7250 con 0x7f22a8161440 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.454+0000 7f229f400640 1 -- 172.21.15.71:0/1815089467 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0035030 con 0x7f22a8161440 2024-09-16T07:36:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.454+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a80b7810 con 0x7f22a8161440 2024-09-16T07:36:08.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.455+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a80b11d0 con 0x7f22a8161440 2024-09-16T07:36:08.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.456+0000 7f229f400640 1 -- 172.21.15.71:0/1815089467 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f22a0042070 con 0x7f22a8161440 2024-09-16T07:36:08.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.457+0000 7f229f400640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f228807a6e0 0x7f228807cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.457+0000 7f229f400640 1 -- 172.21.15.71:0/1815089467 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f22a00ba7d0 con 0x7f22a8161440 2024-09-16T07:36:08.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.457+0000 7f22a7400640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f228807a6e0 0x7f228807cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.458+0000 7f22a7400640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f228807a6e0 0x7f228807cba0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f229800ae50 tx=0x7f229800a540 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:08.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.461+0000 7f229f400640 1 -- 172.21.15.71:0/1815089467 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f22a80b11d0 con 0x7f22a8161440 2024-09-16T07:36:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.647+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22a80b2f60 con 0x7f22a8161440 2024-09-16T07:36:08.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.648+0000 7f229f400640 1 -- 172.21.15.71:0/1815089467 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16832 (secure 0 0 0) 0x7f22a80b2f60 con 0x7f22a8161440 2024-09-16T07:36:08.648 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:08.648 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":133,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:36:05.841435+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:36:02.011721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T07:36:03.220374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:36:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f228807a6e0 msgr2=0x7f228807cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f228807a6e0 0x7f228807cba0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f229800ae50 tx=0x7f229800a540 comp rx=0 tx=0).stop 2024-09-16T07:36:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a8161440 msgr2=0x7f22a80b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a8161440 0x7f22a80b8010 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009230 tx=0x7f22a0003960 comp rx=0 tx=0).stop 2024-09-16T07:36:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 shutdown_connections 2024-09-16T07:36:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f228807a6e0 0x7f228807cba0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a8161440 0x7f22a80b8010 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22a80b4020 0x7f22a8156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 --2- 172.21.15.71:0/1815089467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22a80b34b0 0x7f22a81f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 >> 172.21.15.71:0/1815089467 conn(0x7f22a80d3df0 msgr2=0x7f22a8160650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 shutdown_connections 2024-09-16T07:36:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.651+0000 7f22ace00640 1 -- 172.21.15.71:0/1815089467 wait complete. 2024-09-16T07:36:08.668 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:36:08.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:36:08.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.825+0000 7f1341400640 1 -- 172.21.15.71:0/1031030624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 msgr2=0x7f133c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.825+0000 7f1341400640 1 --2- 172.21.15.71:0/1031030624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 0x7f133c06f160 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1334005fd0 tx=0x7f133402a820 comp rx=0 tx=0).stop 2024-09-16T07:36:08.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.825+0000 7f1341400640 1 -- 172.21.15.71:0/1031030624 shutdown_connections 2024-09-16T07:36:08.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.825+0000 7f1341400640 1 --2- 172.21.15.71:0/1031030624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 0x7f133c06f160 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.825+0000 7f1341400640 1 -- 172.21.15.71:0/1031030624 >> 172.21.15.71:0/1031030624 conn(0x7f133c0b1950 msgr2=0x7f133c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:08.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.825+0000 7f1341400640 1 -- 172.21.15.71:0/1031030624 shutdown_connections 2024-09-16T07:36:08.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.825+0000 7f1341400640 1 -- 172.21.15.71:0/1031030624 wait complete. 2024-09-16T07:36:08.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.826+0000 7f1341400640 1 Processor -- start 2024-09-16T07:36:08.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.827+0000 7f1341400640 1 -- start start 2024-09-16T07:36:08.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.827+0000 7f1341400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 0x7f133c1f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.827+0000 7f1341400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133c0ccf00 0x7f133c148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.827+0000 7f1341400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133c15f5a0 0x7f133c0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.827+0000 7f1341400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f133c074840 con 0x7f133c0b2440 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.827+0000 7f1341400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0746c0 con 0x7f133c15f5a0 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.827+0000 7f1341400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f133c06f6a0 con 0x7f133c0ccf00 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 0x7f133c1f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 0x7f133c1f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40458/0 (socket says 172.21.15.71:40458) 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 -- 172.21.15.71:0/1098186819 learned_addr learned my addr 172.21.15.71:0/1098186819 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133b200640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133c15f5a0 0x7f133c0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f1331e00640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133c0ccf00 0x7f133c148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 -- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133c0ccf00 msgr2=0x7f133c148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133c0ccf00 0x7f133c148f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 -- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133c15f5a0 msgr2=0x7f133c0cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133c15f5a0 0x7f133c0cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:08.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 -- 172.21.15.71:0/1098186819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c1f7210 con 0x7f133c0b2440 2024-09-16T07:36:08.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.828+0000 7f133a800640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 0x7f133c1f62b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f132c00c3a0 tx=0x7f132c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:08.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.829+0000 7f1333400640 1 -- 172.21.15.71:0/1098186819 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c011790 con 0x7f133c0b2440 2024-09-16T07:36:08.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.829+0000 7f1333400640 1 -- 172.21.15.71:0/1098186819 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f132c017070 con 0x7f133c0b2440 2024-09-16T07:36:08.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.829+0000 7f1333400640 1 -- 172.21.15.71:0/1098186819 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c011cc0 con 0x7f133c0b2440 2024-09-16T07:36:08.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.829+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1f6970 con 0x7f133c0b2440 2024-09-16T07:36:08.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.829+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1f6f30 con 0x7f133c0b2440 2024-09-16T07:36:08.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.830+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c06f160 con 0x7f133c0b2440 2024-09-16T07:36:08.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.831+0000 7f1333400640 1 -- 172.21.15.71:0/1098186819 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f132c01c460 con 0x7f133c0b2440 2024-09-16T07:36:08.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.832+0000 7f1333400640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f131807a6b0 0x7f131807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:08.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.832+0000 7f1333400640 1 -- 172.21.15.71:0/1098186819 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f132c099730 con 0x7f133c0b2440 2024-09-16T07:36:08.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.833+0000 7f1331e00640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f131807a6b0 0x7f131807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:08.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.833+0000 7f1331e00640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f131807a6b0 0x7f131807cb70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1324004ea0 tx=0x7f1324009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:08.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:08.836+0000 7f1333400640 1 -- 172.21.15.71:0/1098186819 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f132c063800 con 0x7f133c0b2440 2024-09-16T07:36:09.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.017+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f133c09b440 con 0x7f131807a6b0 2024-09-16T07:36:09.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.022+0000 7f1333400640 1 -- 172.21.15.71:0/1098186819 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248831 (secure 0 0 0) 0x7f133c09b440 con 0x7f131807a6b0 2024-09-16T07:36:09.025 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:09.030 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:36:09.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f131807a6b0 msgr2=0x7f131807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f131807a6b0 0x7f131807cb70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1324004ea0 tx=0x7f1324009040 comp rx=0 tx=0).stop 2024-09-16T07:36:09.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 msgr2=0x7f133c1f62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 0x7f133c1f62b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f132c00c3a0 tx=0x7f132c00c860 comp rx=0 tx=0).stop 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 shutdown_connections 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f131807a6b0 0x7f131807cb70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133c15f5a0 0x7f133c0cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133c0ccf00 0x7f133c148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 --2- 172.21.15.71:0/1098186819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c0b2440 0x7f133c1f62b0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 >> 172.21.15.71:0/1098186819 conn(0x7f133c0b1950 msgr2=0x7f133c0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 shutdown_connections 2024-09-16T07:36:09.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.026+0000 7f1341400640 1 -- 172.21.15.71:0/1098186819 wait complete. 2024-09-16T07:36:09.042 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-09-16T07:36:08.626004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076396,"kb_used_data":13884,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898324,"statfs":{"total":773094113280,"available":771991883776,"internally_reserved":0,"allocated":14217216,"data_stored":6894520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1087819312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805957"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.853420+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.853157+0000","last_became_peered":"2024-09-16T07:36:04.853157+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:29.795783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063845+0000","last_change":"2024-09-16T07:36:04.855589+0000","last_active":"2024-09-16T07:36:06.063845+0000","last_peered":"2024-09-16T07:36:06.063845+0000","last_clean":"2024-09-16T07:36:06.063845+0000","last_became_active":"2024-09-16T07:36:04.855323+0000","last_became_peered":"2024-09-16T07:36:04.855323+0000","last_unstale":"2024-09-16T07:36:06.063845+0000","last_undegraded":"2024-09-16T07:36:06.063845+0000","last_fullsized":"2024-09-16T07:36:06.063845+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:53.457746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847311+0000","last_change":"2024-09-16T07:36:03.847311+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847311+0000","last_undegraded":"2024-09-16T07:36:03.847311+0000","last_fullsized":"2024-09-16T07:36:03.847311+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855470+0000","last_change":"2024-09-16T07:36:04.859510+0000","last_active":"2024-09-16T07:36:05.855470+0000","last_peered":"2024-09-16T07:36:05.855470+0000","last_clean":"2024-09-16T07:36:05.855470+0000","last_became_active":"2024-09-16T07:36:04.859175+0000","last_became_peered":"2024-09-16T07:36:04.859175+0000","last_unstale":"2024-09-16T07:36:05.855470+0000","last_undegraded":"2024-09-16T07:36:05.855470+0000","last_fullsized":"2024-09-16T07:36:05.855470+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:42:35.105087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.843162+0000","last_change":"2024-09-16T07:36:03.843162+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.843162+0000","last_undegraded":"2024-09-16T07:36:03.843162+0000","last_fullsized":"2024-09-16T07:36:03.843162+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.841267+0000","last_change":"2024-09-16T07:36:03.841267+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.841267+0000","last_undegraded":"2024-09-16T07:36:03.841267+0000","last_fullsized":"2024-09-16T07:36:03.841267+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842354+0000","last_change":"2024-09-16T07:36:03.842354+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842354+0000","last_undegraded":"2024-09-16T07:36:03.842354+0000","last_fullsized":"2024-09-16T07:36:03.842354+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847905+0000","last_change":"2024-09-16T07:36:03.847905+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847905+0000","last_undegraded":"2024-09-16T07:36:03.847905+0000","last_fullsized":"2024-09-16T07:36:03.847905+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.851248+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.850709+0000","last_became_peered":"2024-09-16T07:36:04.850709+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:07.411898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855892+0000","last_change":"2024-09-16T07:36:04.854097+0000","last_active":"2024-09-16T07:36:05.855892+0000","last_peered":"2024-09-16T07:36:05.855892+0000","last_clean":"2024-09-16T07:36:05.855892+0000","last_became_active":"2024-09-16T07:36:04.853827+0000","last_became_peered":"2024-09-16T07:36:04.853827+0000","last_unstale":"2024-09-16T07:36:05.855892+0000","last_undegraded":"2024-09-16T07:36:05.855892+0000","last_fullsized":"2024-09-16T07:36:05.855892+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:21.875425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855153+0000","last_change":"2024-09-16T07:36:04.848716+0000","last_active":"2024-09-16T07:36:05.855153+0000","last_peered":"2024-09-16T07:36:05.855153+0000","last_clean":"2024-09-16T07:36:05.855153+0000","last_became_active":"2024-09-16T07:36:04.848457+0000","last_became_peered":"2024-09-16T07:36:04.848457+0000","last_unstale":"2024-09-16T07:36:05.855153+0000","last_undegraded":"2024-09-16T07:36:05.855153+0000","last_fullsized":"2024-09-16T07:36:05.855153+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:21.475235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856556+0000","last_change":"2024-09-16T07:36:04.859096+0000","last_active":"2024-09-16T07:36:05.856556+0000","last_peered":"2024-09-16T07:36:05.856556+0000","last_clean":"2024-09-16T07:36:05.856556+0000","last_became_active":"2024-09-16T07:36:04.858812+0000","last_became_peered":"2024-09-16T07:36:04.858812+0000","last_unstale":"2024-09-16T07:36:05.856556+0000","last_undegraded":"2024-09-16T07:36:05.856556+0000","last_fullsized":"2024-09-16T07:36:05.856556+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:59.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855973+0000","last_change":"2024-09-16T07:36:04.862176+0000","last_active":"2024-09-16T07:36:05.855973+0000","last_peered":"2024-09-16T07:36:05.855973+0000","last_clean":"2024-09-16T07:36:05.855973+0000","last_became_active":"2024-09-16T07:36:04.860550+0000","last_became_peered":"2024-09-16T07:36:04.860550+0000","last_unstale":"2024-09-16T07:36:05.855973+0000","last_undegraded":"2024-09-16T07:36:05.855973+0000","last_fullsized":"2024-09-16T07:36:05.855973+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:10.040307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854370+0000","last_change":"2024-09-16T07:36:04.855363+0000","last_active":"2024-09-16T07:36:05.854370+0000","last_peered":"2024-09-16T07:36:05.854370+0000","last_clean":"2024-09-16T07:36:05.854370+0000","last_became_active":"2024-09-16T07:36:04.855013+0000","last_became_peered":"2024-09-16T07:36:04.855013+0000","last_unstale":"2024-09-16T07:36:05.854370+0000","last_undegraded":"2024-09-16T07:36:05.854370+0000","last_fullsized":"2024-09-16T07:36:05.854370+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:13.040194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.846503+0000","last_change":"2024-09-16T07:36:03.846503+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.846503+0000","last_undegraded":"2024-09-16T07:36:03.846503+0000","last_fullsized":"2024-09-16T07:36:03.846503+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855025+0000","last_change":"2024-09-16T07:36:04.862436+0000","last_active":"2024-09-16T07:36:05.855025+0000","last_peered":"2024-09-16T07:36:05.855025+0000","last_clean":"2024-09-16T07:36:05.855025+0000","last_became_active":"2024-09-16T07:36:04.862125+0000","last_became_peered":"2024-09-16T07:36:04.862125+0000","last_unstale":"2024-09-16T07:36:05.855025+0000","last_undegraded":"2024-09-16T07:36:05.855025+0000","last_fullsized":"2024-09-16T07:36:05.855025+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:38.787658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063678+0000","last_change":"2024-09-16T07:36:04.857741+0000","last_active":"2024-09-16T07:36:06.063678+0000","last_peered":"2024-09-16T07:36:06.063678+0000","last_clean":"2024-09-16T07:36:06.063678+0000","last_became_active":"2024-09-16T07:36:04.854444+0000","last_became_peered":"2024-09-16T07:36:04.854444+0000","last_unstale":"2024-09-16T07:36:06.063678+0000","last_undegraded":"2024-09-16T07:36:06.063678+0000","last_fullsized":"2024-09-16T07:36:06.063678+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:15.100689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856475+0000","last_change":"2024-09-16T07:36:04.869720+0000","last_active":"2024-09-16T07:36:05.856475+0000","last_peered":"2024-09-16T07:36:05.856475+0000","last_clean":"2024-09-16T07:36:05.856475+0000","last_became_active":"2024-09-16T07:36:04.868956+0000","last_became_peered":"2024-09-16T07:36:04.868956+0000","last_unstale":"2024-09-16T07:36:05.856475+0000","last_undegraded":"2024-09-16T07:36:05.856475+0000","last_fullsized":"2024-09-16T07:36:05.856475+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:29.852710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063836+0000","last_change":"2024-09-16T07:36:04.845962+0000","last_active":"2024-09-16T07:36:06.063836+0000","last_peered":"2024-09-16T07:36:06.063836+0000","last_clean":"2024-09-16T07:36:06.063836+0000","last_became_active":"2024-09-16T07:36:04.845664+0000","last_became_peered":"2024-09-16T07:36:04.845664+0000","last_unstale":"2024-09-16T07:36:06.063836+0000","last_undegraded":"2024-09-16T07:36:06.063836+0000","last_fullsized":"2024-09-16T07:36:06.063836+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:32.167063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837882+0000","last_change":"2024-09-16T07:36:03.837882+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837882+0000","last_undegraded":"2024-09-16T07:36:03.837882+0000","last_fullsized":"2024-09-16T07:36:03.837882+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854174+0000","last_change":"2024-09-16T07:36:04.865540+0000","last_active":"2024-09-16T07:36:05.854174+0000","last_peered":"2024-09-16T07:36:05.854174+0000","last_clean":"2024-09-16T07:36:05.854174+0000","last_became_active":"2024-09-16T07:36:04.865232+0000","last_became_peered":"2024-09-16T07:36:04.865232+0000","last_unstale":"2024-09-16T07:36:05.854174+0000","last_undegraded":"2024-09-16T07:36:05.854174+0000","last_fullsized":"2024-09-16T07:36:05.854174+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:24.118557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855774+0000","last_change":"2024-09-16T07:36:04.857205+0000","last_active":"2024-09-16T07:36:05.855774+0000","last_peered":"2024-09-16T07:36:05.855774+0000","last_clean":"2024-09-16T07:36:05.855774+0000","last_became_active":"2024-09-16T07:36:04.856823+0000","last_became_peered":"2024-09-16T07:36:04.856823+0000","last_unstale":"2024-09-16T07:36:05.855774+0000","last_undegraded":"2024-09-16T07:36:05.855774+0000","last_fullsized":"2024-09-16T07:36:05.855774+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:16.355699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064085+0000","last_change":"2024-09-16T07:36:04.850897+0000","last_active":"2024-09-16T07:36:06.064085+0000","last_peered":"2024-09-16T07:36:06.064085+0000","last_clean":"2024-09-16T07:36:06.064085+0000","last_became_active":"2024-09-16T07:36:04.850635+0000","last_became_peered":"2024-09-16T07:36:04.850635+0000","last_unstale":"2024-09-16T07:36:06.064085+0000","last_undegraded":"2024-09-16T07:36:06.064085+0000","last_fullsized":"2024-09-16T07:36:06.064085+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:38.194427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857379+0000","last_change":"2024-09-16T07:36:04.855205+0000","last_active":"2024-09-16T07:36:05.857379+0000","last_peered":"2024-09-16T07:36:05.857379+0000","last_clean":"2024-09-16T07:36:05.857379+0000","last_became_active":"2024-09-16T07:36:04.854356+0000","last_became_peered":"2024-09-16T07:36:04.854356+0000","last_unstale":"2024-09-16T07:36:05.857379+0000","last_undegraded":"2024-09-16T07:36:05.857379+0000","last_fullsized":"2024-09-16T07:36:05.857379+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:23.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837934+0000","last_change":"2024-09-16T07:36:03.837934+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837934+0000","last_undegraded":"2024-09-16T07:36:03.837934+0000","last_fullsized":"2024-09-16T07:36:03.837934+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063430+0000","last_change":"2024-09-16T07:36:04.849621+0000","last_active":"2024-09-16T07:36:06.063430+0000","last_peered":"2024-09-16T07:36:06.063430+0000","last_clean":"2024-09-16T07:36:06.063430+0000","last_became_active":"2024-09-16T07:36:04.849325+0000","last_became_peered":"2024-09-16T07:36:04.849325+0000","last_unstale":"2024-09-16T07:36:06.063430+0000","last_undegraded":"2024-09-16T07:36:06.063430+0000","last_fullsized":"2024-09-16T07:36:06.063430+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:20.048326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063880+0000","last_change":"2024-09-16T07:36:04.840820+0000","last_active":"2024-09-16T07:36:06.063880+0000","last_peered":"2024-09-16T07:36:06.063880+0000","last_clean":"2024-09-16T07:36:06.063880+0000","last_became_active":"2024-09-16T07:36:04.839397+0000","last_became_peered":"2024-09-16T07:36:04.839397+0000","last_unstale":"2024-09-16T07:36:06.063880+0000","last_undegraded":"2024-09-16T07:36:06.063880+0000","last_fullsized":"2024-09-16T07:36:06.063880+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:01.349093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837391+0000","last_change":"2024-09-16T07:36:03.837391+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837391+0000","last_undegraded":"2024-09-16T07:36:03.837391+0000","last_fullsized":"2024-09-16T07:36:03.837391+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842895+0000","last_change":"2024-09-16T07:36:03.842895+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842895+0000","last_undegraded":"2024-09-16T07:36:03.842895+0000","last_fullsized":"2024-09-16T07:36:03.842895+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857460+0000","last_change":"2024-09-16T07:36:04.859595+0000","last_active":"2024-09-16T07:36:05.857460+0000","last_peered":"2024-09-16T07:36:05.857460+0000","last_clean":"2024-09-16T07:36:05.857460+0000","last_became_active":"2024-09-16T07:36:04.856512+0000","last_became_peered":"2024-09-16T07:36:04.856512+0000","last_unstale":"2024-09-16T07:36:05.857460+0000","last_undegraded":"2024-09-16T07:36:05.857460+0000","last_fullsized":"2024-09-16T07:36:05.857460+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:25.701980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858752+0000","last_change":"2024-09-16T07:36:03.855477+0000","last_active":"2024-09-16T07:36:03.858752+0000","last_peered":"2024-09-16T07:36:03.858752+0000","last_clean":"2024-09-16T07:36:03.858752+0000","last_became_active":"2024-09-16T07:36:03.852255+0000","last_became_peered":"2024-09-16T07:36:03.852255+0000","last_unstale":"2024-09-16T07:36:03.858752+0000","last_undegraded":"2024-09-16T07:36:03.858752+0000","last_fullsized":"2024-09-16T07:36:03.858752+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:58.216679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847493+0000","last_change":"2024-09-16T07:36:03.847493+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847493+0000","last_undegraded":"2024-09-16T07:36:03.847493+0000","last_fullsized":"2024-09-16T07:36:03.847493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.855023+0000","last_change":"2024-09-16T07:36:03.853595+0000","last_active":"2024-09-16T07:36:03.855023+0000","last_peered":"2024-09-16T07:36:03.855023+0000","last_clean":"2024-09-16T07:36:03.855023+0000","last_became_active":"2024-09-16T07:36:03.853315+0000","last_became_peered":"2024-09-16T07:36:03.853315+0000","last_unstale":"2024-09-16T07:36:03.855023+0000","last_undegraded":"2024-09-16T07:36:03.855023+0000","last_fullsized":"2024-09-16T07:36:03.855023+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:09.161606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.845779+0000","last_change":"2024-09-16T07:36:03.845779+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.845779+0000","last_undegraded":"2024-09-16T07:36:03.845779+0000","last_fullsized":"2024-09-16T07:36:03.845779+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856585+0000","last_change":"2024-09-16T07:36:03.849019+0000","last_active":"2024-09-16T07:36:05.856585+0000","last_peered":"2024-09-16T07:36:05.856585+0000","last_clean":"2024-09-16T07:36:05.856585+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.856585+0000","last_undegraded":"2024-09-16T07:36:05.856585+0000","last_fullsized":"2024-09-16T07:36:05.856585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:53.651818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063690+0000","last_change":"2024-09-16T07:36:03.853178+0000","last_active":"2024-09-16T07:36:06.063690+0000","last_peered":"2024-09-16T07:36:06.063690+0000","last_clean":"2024-09-16T07:36:06.063690+0000","last_became_active":"2024-09-16T07:36:03.852914+0000","last_became_peered":"2024-09-16T07:36:03.852914+0000","last_unstale":"2024-09-16T07:36:06.063690+0000","last_undegraded":"2024-09-16T07:36:06.063690+0000","last_fullsized":"2024-09-16T07:36:06.063690+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.868825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855929+0000","last_change":"2024-09-16T07:36:04.860375+0000","last_active":"2024-09-16T07:36:05.855929+0000","last_peered":"2024-09-16T07:36:05.855929+0000","last_clean":"2024-09-16T07:36:05.855929+0000","last_became_active":"2024-09-16T07:36:04.860110+0000","last_became_peered":"2024-09-16T07:36:04.860110+0000","last_unstale":"2024-09-16T07:36:05.855929+0000","last_undegraded":"2024-09-16T07:36:05.855929+0000","last_fullsized":"2024-09-16T07:36:05.855929+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:44.383231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063342+0000","last_change":"2024-09-16T07:36:03.848993+0000","last_active":"2024-09-16T07:36:06.063342+0000","last_peered":"2024-09-16T07:36:06.063342+0000","last_clean":"2024-09-16T07:36:06.063342+0000","last_became_active":"2024-09-16T07:36:03.848700+0000","last_became_peered":"2024-09-16T07:36:03.848700+0000","last_unstale":"2024-09-16T07:36:06.063342+0000","last_undegraded":"2024-09-16T07:36:06.063342+0000","last_fullsized":"2024-09-16T07:36:06.063342+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:54.220868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856372+0000","last_change":"2024-09-16T07:36:03.855637+0000","last_active":"2024-09-16T07:36:05.856372+0000","last_peered":"2024-09-16T07:36:05.856372+0000","last_clean":"2024-09-16T07:36:05.856372+0000","last_became_active":"2024-09-16T07:36:03.855357+0000","last_became_peered":"2024-09-16T07:36:03.855357+0000","last_unstale":"2024-09-16T07:36:05.856372+0000","last_undegraded":"2024-09-16T07:36:05.856372+0000","last_fullsized":"2024-09-16T07:36:05.856372+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.392485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858894+0000","last_change":"2024-09-16T07:36:03.858168+0000","last_active":"2024-09-16T07:36:03.858894+0000","last_peered":"2024-09-16T07:36:03.858894+0000","last_clean":"2024-09-16T07:36:03.858894+0000","last_became_active":"2024-09-16T07:36:03.857697+0000","last_became_peered":"2024-09-16T07:36:03.857697+0000","last_unstale":"2024-09-16T07:36:03.858894+0000","last_undegraded":"2024-09-16T07:36:03.858894+0000","last_fullsized":"2024-09-16T07:36:03.858894+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:36.388550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063343+0000","last_change":"2024-09-16T07:36:04.853058+0000","last_active":"2024-09-16T07:36:06.063343+0000","last_peered":"2024-09-16T07:36:06.063343+0000","last_clean":"2024-09-16T07:36:06.063343+0000","last_became_active":"2024-09-16T07:36:04.852535+0000","last_became_peered":"2024-09-16T07:36:04.852535+0000","last_unstale":"2024-09-16T07:36:06.063343+0000","last_undegraded":"2024-09-16T07:36:06.063343+0000","last_fullsized":"2024-09-16T07:36:06.063343+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:36.784633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063356+0000","last_change":"2024-09-16T07:36:03.846732+0000","last_active":"2024-09-16T07:36:06.063356+0000","last_peered":"2024-09-16T07:36:06.063356+0000","last_clean":"2024-09-16T07:36:06.063356+0000","last_became_active":"2024-09-16T07:36:03.845789+0000","last_became_peered":"2024-09-16T07:36:03.845789+0000","last_unstale":"2024-09-16T07:36:06.063356+0000","last_undegraded":"2024-09-16T07:36:06.063356+0000","last_fullsized":"2024-09-16T07:36:06.063356+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:53.374127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064066+0000","last_change":"2024-09-16T07:36:04.852160+0000","last_active":"2024-09-16T07:36:06.064066+0000","last_peered":"2024-09-16T07:36:06.064066+0000","last_clean":"2024-09-16T07:36:06.064066+0000","last_became_active":"2024-09-16T07:36:04.851903+0000","last_became_peered":"2024-09-16T07:36:04.851903+0000","last_unstale":"2024-09-16T07:36:06.064066+0000","last_undegraded":"2024-09-16T07:36:06.064066+0000","last_fullsized":"2024-09-16T07:36:06.064066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:32.387978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.832128+0000","last_change":"2024-09-16T07:36:02.832128+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.832128+0000","last_undegraded":"2024-09-16T07:36:02.832128+0000","last_fullsized":"2024-09-16T07:36:02.832128+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842513+0000","last_change":"2024-09-16T07:36:03.842513+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842513+0000","last_undegraded":"2024-09-16T07:36:03.842513+0000","last_fullsized":"2024-09-16T07:36:03.842513+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855092+0000","last_change":"2024-09-16T07:36:03.852706+0000","last_active":"2024-09-16T07:36:05.855092+0000","last_peered":"2024-09-16T07:36:05.855092+0000","last_clean":"2024-09-16T07:36:05.855092+0000","last_became_active":"2024-09-16T07:36:03.852295+0000","last_became_peered":"2024-09-16T07:36:03.852295+0000","last_unstale":"2024-09-16T07:36:05.855092+0000","last_undegraded":"2024-09-16T07:36:05.855092+0000","last_fullsized":"2024-09-16T07:36:05.855092+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:51.417749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.859083+0000","last_change":"2024-09-16T07:36:03.857233+0000","last_active":"2024-09-16T07:36:03.859083+0000","last_peered":"2024-09-16T07:36:03.859083+0000","last_clean":"2024-09-16T07:36:03.859083+0000","last_became_active":"2024-09-16T07:36:03.856817+0000","last_became_peered":"2024-09-16T07:36:03.856817+0000","last_unstale":"2024-09-16T07:36:03.859083+0000","last_undegraded":"2024-09-16T07:36:03.859083+0000","last_fullsized":"2024-09-16T07:36:03.859083+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:45.111727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837921+0000","last_change":"2024-09-16T07:36:03.837921+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837921+0000","last_undegraded":"2024-09-16T07:36:03.837921+0000","last_fullsized":"2024-09-16T07:36:03.837921+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.854803+0000","last_change":"2024-09-16T07:36:03.851395+0000","last_active":"2024-09-16T07:36:03.854803+0000","last_peered":"2024-09-16T07:36:03.854803+0000","last_clean":"2024-09-16T07:36:03.854803+0000","last_became_active":"2024-09-16T07:36:03.851127+0000","last_became_peered":"2024-09-16T07:36:03.851127+0000","last_unstale":"2024-09-16T07:36:03.854803+0000","last_undegraded":"2024-09-16T07:36:03.854803+0000","last_fullsized":"2024-09-16T07:36:03.854803+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:50.717571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856941+0000","last_change":"2024-09-16T07:36:04.863490+0000","last_active":"2024-09-16T07:36:05.856941+0000","last_peered":"2024-09-16T07:36:05.856941+0000","last_clean":"2024-09-16T07:36:05.856941+0000","last_became_active":"2024-09-16T07:36:04.862482+0000","last_became_peered":"2024-09-16T07:36:04.862482+0000","last_unstale":"2024-09-16T07:36:05.856941+0000","last_undegraded":"2024-09-16T07:36:05.856941+0000","last_fullsized":"2024-09-16T07:36:05.856941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:18.732185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856621+0000","last_change":"2024-09-16T07:36:03.852858+0000","last_active":"2024-09-16T07:36:05.856621+0000","last_peered":"2024-09-16T07:36:05.856621+0000","last_clean":"2024-09-16T07:36:05.856621+0000","last_became_active":"2024-09-16T07:36:03.852596+0000","last_became_peered":"2024-09-16T07:36:03.852596+0000","last_unstale":"2024-09-16T07:36:05.856621+0000","last_undegraded":"2024-09-16T07:36:05.856621+0000","last_fullsized":"2024-09-16T07:36:05.856621+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:29.307382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854662+0000","last_change":"2024-09-16T07:36:04.855631+0000","last_active":"2024-09-16T07:36:05.854662+0000","last_peered":"2024-09-16T07:36:05.854662+0000","last_clean":"2024-09-16T07:36:05.854662+0000","last_became_active":"2024-09-16T07:36:04.855341+0000","last_became_peered":"2024-09-16T07:36:04.855341+0000","last_unstale":"2024-09-16T07:36:05.854662+0000","last_undegraded":"2024-09-16T07:36:05.854662+0000","last_fullsized":"2024-09-16T07:36:05.854662+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:06.127741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854531+0000","last_change":"2024-09-16T07:36:03.854082+0000","last_active":"2024-09-16T07:36:05.854531+0000","last_peered":"2024-09-16T07:36:05.854531+0000","last_clean":"2024-09-16T07:36:05.854531+0000","last_became_active":"2024-09-16T07:36:03.853794+0000","last_became_peered":"2024-09-16T07:36:03.853794+0000","last_unstale":"2024-09-16T07:36:05.854531+0000","last_undegraded":"2024-09-16T07:36:05.854531+0000","last_fullsized":"2024-09-16T07:36:05.854531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:10.982614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.830540+0000","last_change":"2024-09-16T07:36:02.830540+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.830540+0000","last_undegraded":"2024-09-16T07:36:02.830540+0000","last_fullsized":"2024-09-16T07:36:02.830540+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.843486+0000","last_change":"2024-09-16T07:36:03.843486+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.843486+0000","last_undegraded":"2024-09-16T07:36:03.843486+0000","last_fullsized":"2024-09-16T07:36:03.843486+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.828755+0000","last_change":"2024-09-16T07:36:02.828755+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.828755+0000","last_undegraded":"2024-09-16T07:36:02.828755+0000","last_fullsized":"2024-09-16T07:36:02.828755+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842116+0000","last_change":"2024-09-16T07:36:03.842116+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842116+0000","last_undegraded":"2024-09-16T07:36:03.842116+0000","last_fullsized":"2024-09-16T07:36:03.842116+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856205+0000","last_change":"2024-09-16T07:36:03.850133+0000","last_active":"2024-09-16T07:36:05.856205+0000","last_peered":"2024-09-16T07:36:05.856205+0000","last_clean":"2024-09-16T07:36:05.856205+0000","last_became_active":"2024-09-16T07:36:03.849070+0000","last_became_peered":"2024-09-16T07:36:03.849070+0000","last_unstale":"2024-09-16T07:36:05.856205+0000","last_undegraded":"2024-09-16T07:36:05.856205+0000","last_fullsized":"2024-09-16T07:36:05.856205+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:35.166163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063646+0000","last_change":"2024-09-16T07:36:04.848685+0000","last_active":"2024-09-16T07:36:06.063646+0000","last_peered":"2024-09-16T07:36:06.063646+0000","last_clean":"2024-09-16T07:36:06.063646+0000","last_became_active":"2024-09-16T07:36:04.844479+0000","last_became_peered":"2024-09-16T07:36:04.844479+0000","last_unstale":"2024-09-16T07:36:06.063646+0000","last_undegraded":"2024-09-16T07:36:06.063646+0000","last_fullsized":"2024-09-16T07:36:06.063646+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:36.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.857074+0000","last_change":"2024-09-16T07:36:03.855122+0000","last_active":"2024-09-16T07:36:03.857074+0000","last_peered":"2024-09-16T07:36:03.857074+0000","last_clean":"2024-09-16T07:36:03.857074+0000","last_became_active":"2024-09-16T07:36:03.854786+0000","last_became_peered":"2024-09-16T07:36:03.854786+0000","last_unstale":"2024-09-16T07:36:03.857074+0000","last_undegraded":"2024-09-16T07:36:03.857074+0000","last_fullsized":"2024-09-16T07:36:03.857074+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:02.212316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063917+0000","last_change":"2024-09-16T07:36:04.853245+0000","last_active":"2024-09-16T07:36:06.063917+0000","last_peered":"2024-09-16T07:36:06.063917+0000","last_clean":"2024-09-16T07:36:06.063917+0000","last_became_active":"2024-09-16T07:36:04.852895+0000","last_became_peered":"2024-09-16T07:36:04.852895+0000","last_unstale":"2024-09-16T07:36:06.063917+0000","last_undegraded":"2024-09-16T07:36:06.063917+0000","last_fullsized":"2024-09-16T07:36:06.063917+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.199780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858884+0000","last_change":"2024-09-16T07:36:03.856130+0000","last_active":"2024-09-16T07:36:03.858884+0000","last_peered":"2024-09-16T07:36:03.858884+0000","last_clean":"2024-09-16T07:36:03.858884+0000","last_became_active":"2024-09-16T07:36:03.852405+0000","last_became_peered":"2024-09-16T07:36:03.852405+0000","last_unstale":"2024-09-16T07:36:03.858884+0000","last_undegraded":"2024-09-16T07:36:03.858884+0000","last_fullsized":"2024-09-16T07:36:03.858884+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:51.417749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064061+0000","last_change":"2024-09-16T07:36:04.847598+0000","last_active":"2024-09-16T07:36:06.064061+0000","last_peered":"2024-09-16T07:36:06.064061+0000","last_clean":"2024-09-16T07:36:06.064061+0000","last_became_active":"2024-09-16T07:36:04.847063+0000","last_became_peered":"2024-09-16T07:36:04.847063+0000","last_unstale":"2024-09-16T07:36:06.064061+0000","last_undegraded":"2024-09-16T07:36:06.064061+0000","last_fullsized":"2024-09-16T07:36:06.064061+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:50.838203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858292+0000","last_change":"2024-09-16T07:36:03.856847+0000","last_active":"2024-09-16T07:36:03.858292+0000","last_peered":"2024-09-16T07:36:03.858292+0000","last_clean":"2024-09-16T07:36:03.858292+0000","last_became_active":"2024-09-16T07:36:03.856321+0000","last_became_peered":"2024-09-16T07:36:03.856321+0000","last_unstale":"2024-09-16T07:36:03.858292+0000","last_undegraded":"2024-09-16T07:36:03.858292+0000","last_fullsized":"2024-09-16T07:36:03.858292+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:57.138229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837398+0000","last_change":"2024-09-16T07:36:03.837398+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837398+0000","last_undegraded":"2024-09-16T07:36:03.837398+0000","last_fullsized":"2024-09-16T07:36:03.837398+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856080+0000","last_change":"2024-09-16T07:36:03.855412+0000","last_active":"2024-09-16T07:36:05.856080+0000","last_peered":"2024-09-16T07:36:05.856080+0000","last_clean":"2024-09-16T07:36:05.856080+0000","last_became_active":"2024-09-16T07:36:03.854933+0000","last_became_peered":"2024-09-16T07:36:03.854933+0000","last_unstale":"2024-09-16T07:36:05.856080+0000","last_undegraded":"2024-09-16T07:36:05.856080+0000","last_fullsized":"2024-09-16T07:36:05.856080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:58.216679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854581+0000","last_change":"2024-09-16T07:36:04.850083+0000","last_active":"2024-09-16T07:36:05.854581+0000","last_peered":"2024-09-16T07:36:05.854581+0000","last_clean":"2024-09-16T07:36:05.854581+0000","last_became_active":"2024-09-16T07:36:04.849731+0000","last_became_peered":"2024-09-16T07:36:04.849731+0000","last_unstale":"2024-09-16T07:36:05.854581+0000","last_undegraded":"2024-09-16T07:36:05.854581+0000","last_fullsized":"2024-09-16T07:36:05.854581+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:08.133005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855297+0000","last_change":"2024-09-16T07:36:03.848269+0000","last_active":"2024-09-16T07:36:05.855297+0000","last_peered":"2024-09-16T07:36:05.855297+0000","last_clean":"2024-09-16T07:36:05.855297+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.855297+0000","last_undegraded":"2024-09-16T07:36:05.855297+0000","last_fullsized":"2024-09-16T07:36:05.855297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:06.554515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856499+0000","last_change":"2024-09-16T07:36:03.847015+0000","last_active":"2024-09-16T07:36:05.856499+0000","last_peered":"2024-09-16T07:36:05.856499+0000","last_clean":"2024-09-16T07:36:05.856499+0000","last_became_active":"2024-09-16T07:36:03.846719+0000","last_became_peered":"2024-09-16T07:36:03.846719+0000","last_unstale":"2024-09-16T07:36:05.856499+0000","last_undegraded":"2024-09-16T07:36:05.856499+0000","last_fullsized":"2024-09-16T07:36:05.856499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:46.177351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.838152+0000","last_change":"2024-09-16T07:36:03.838152+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.838152+0000","last_undegraded":"2024-09-16T07:36:03.838152+0000","last_fullsized":"2024-09-16T07:36:03.838152+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856245+0000","last_change":"2024-09-16T07:36:03.854810+0000","last_active":"2024-09-16T07:36:05.856245+0000","last_peered":"2024-09-16T07:36:05.856245+0000","last_clean":"2024-09-16T07:36:05.856245+0000","last_became_active":"2024-09-16T07:36:03.854533+0000","last_became_peered":"2024-09-16T07:36:03.854533+0000","last_unstale":"2024-09-16T07:36:05.856245+0000","last_undegraded":"2024-09-16T07:36:05.856245+0000","last_fullsized":"2024-09-16T07:36:05.856245+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:23.851837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063875+0000","last_change":"2024-09-16T07:36:04.855496+0000","last_active":"2024-09-16T07:36:06.063875+0000","last_peered":"2024-09-16T07:36:06.063875+0000","last_clean":"2024-09-16T07:36:06.063875+0000","last_became_active":"2024-09-16T07:36:04.855167+0000","last_became_peered":"2024-09-16T07:36:04.855167+0000","last_unstale":"2024-09-16T07:36:06.063875+0000","last_undegraded":"2024-09-16T07:36:06.063875+0000","last_fullsized":"2024-09-16T07:36:06.063875+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:24.513277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.859106+0000","last_change":"2024-09-16T07:36:03.857828+0000","last_active":"2024-09-16T07:36:03.859106+0000","last_peered":"2024-09-16T07:36:03.859106+0000","last_clean":"2024-09-16T07:36:03.859106+0000","last_became_active":"2024-09-16T07:36:03.853109+0000","last_became_peered":"2024-09-16T07:36:03.853109+0000","last_unstale":"2024-09-16T07:36:03.859106+0000","last_undegraded":"2024-09-16T07:36:03.859106+0000","last_fullsized":"2024-09-16T07:36:03.859106+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:04.946096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857004+0000","last_change":"2024-09-16T07:36:04.852909+0000","last_active":"2024-09-16T07:36:05.857004+0000","last_peered":"2024-09-16T07:36:05.857004+0000","last_clean":"2024-09-16T07:36:05.857004+0000","last_became_active":"2024-09-16T07:36:04.852496+0000","last_became_peered":"2024-09-16T07:36:04.852496+0000","last_unstale":"2024-09-16T07:36:05.857004+0000","last_undegraded":"2024-09-16T07:36:05.857004+0000","last_fullsized":"2024-09-16T07:36:05.857004+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:43.843163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854718+0000","last_change":"2024-09-16T07:36:03.853605+0000","last_active":"2024-09-16T07:36:05.854718+0000","last_peered":"2024-09-16T07:36:05.854718+0000","last_clean":"2024-09-16T07:36:05.854718+0000","last_became_active":"2024-09-16T07:36:03.853349+0000","last_became_peered":"2024-09-16T07:36:03.853349+0000","last_unstale":"2024-09-16T07:36:05.854718+0000","last_undegraded":"2024-09-16T07:36:05.854718+0000","last_fullsized":"2024-09-16T07:36:05.854718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:29.592986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.846573+0000","last_change":"2024-09-16T07:36:03.846573+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.846573+0000","last_undegraded":"2024-09-16T07:36:03.846573+0000","last_fullsized":"2024-09-16T07:36:03.846573+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857065+0000","last_change":"2024-09-16T07:36:03.851730+0000","last_active":"2024-09-16T07:36:05.857065+0000","last_peered":"2024-09-16T07:36:05.857065+0000","last_clean":"2024-09-16T07:36:05.857065+0000","last_became_active":"2024-09-16T07:36:03.851457+0000","last_became_peered":"2024-09-16T07:36:03.851457+0000","last_unstale":"2024-09-16T07:36:05.857065+0000","last_undegraded":"2024-09-16T07:36:05.857065+0000","last_fullsized":"2024-09-16T07:36:05.857065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:37.222531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063468+0000","last_change":"2024-09-16T07:36:03.857845+0000","last_active":"2024-09-16T07:36:06.063468+0000","last_peered":"2024-09-16T07:36:06.063468+0000","last_clean":"2024-09-16T07:36:06.063468+0000","last_became_active":"2024-09-16T07:36:03.857366+0000","last_became_peered":"2024-09-16T07:36:03.857366+0000","last_unstale":"2024-09-16T07:36:06.063468+0000","last_undegraded":"2024-09-16T07:36:06.063468+0000","last_fullsized":"2024-09-16T07:36:06.063468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:36.413668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.830304+0000","last_change":"2024-09-16T07:36:02.830304+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.830304+0000","last_undegraded":"2024-09-16T07:36:02.830304+0000","last_fullsized":"2024-09-16T07:36:02.830304+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855281+0000","last_change":"2024-09-16T07:36:04.863145+0000","last_active":"2024-09-16T07:36:05.855281+0000","last_peered":"2024-09-16T07:36:05.855281+0000","last_clean":"2024-09-16T07:36:05.855281+0000","last_became_active":"2024-09-16T07:36:04.862893+0000","last_became_peered":"2024-09-16T07:36:04.862893+0000","last_unstale":"2024-09-16T07:36:05.855281+0000","last_undegraded":"2024-09-16T07:36:05.855281+0000","last_fullsized":"2024-09-16T07:36:05.855281+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:26.614434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854639+0000","last_change":"2024-09-16T07:36:03.851573+0000","last_active":"2024-09-16T07:36:05.854639+0000","last_peered":"2024-09-16T07:36:05.854639+0000","last_clean":"2024-09-16T07:36:05.854639+0000","last_became_active":"2024-09-16T07:36:03.851286+0000","last_became_peered":"2024-09-16T07:36:03.851286+0000","last_unstale":"2024-09-16T07:36:05.854639+0000","last_undegraded":"2024-09-16T07:36:05.854639+0000","last_fullsized":"2024-09-16T07:36:05.854639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.290737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856129+0000","last_change":"2024-09-16T07:36:04.858119+0000","last_active":"2024-09-16T07:36:05.856129+0000","last_peered":"2024-09-16T07:36:05.856129+0000","last_clean":"2024-09-16T07:36:05.856129+0000","last_became_active":"2024-09-16T07:36:04.855972+0000","last_became_peered":"2024-09-16T07:36:04.855972+0000","last_unstale":"2024-09-16T07:36:05.856129+0000","last_undegraded":"2024-09-16T07:36:05.856129+0000","last_fullsized":"2024-09-16T07:36:05.856129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:40.599143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854647+0000","last_change":"2024-09-16T07:36:03.854300+0000","last_active":"2024-09-16T07:36:05.854647+0000","last_peered":"2024-09-16T07:36:05.854647+0000","last_clean":"2024-09-16T07:36:05.854647+0000","last_became_active":"2024-09-16T07:36:03.853971+0000","last_became_peered":"2024-09-16T07:36:03.853971+0000","last_unstale":"2024-09-16T07:36:05.854647+0000","last_undegraded":"2024-09-16T07:36:05.854647+0000","last_fullsized":"2024-09-16T07:36:05.854647+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:25.287389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.844781+0000","last_change":"2024-09-16T07:36:03.844781+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.844781+0000","last_undegraded":"2024-09-16T07:36:03.844781+0000","last_fullsized":"2024-09-16T07:36:03.844781+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.828990+0000","last_change":"2024-09-16T07:36:02.828990+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.828990+0000","last_undegraded":"2024-09-16T07:36:02.828990+0000","last_fullsized":"2024-09-16T07:36:02.828990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":310,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.839687+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:36:03.839687+0000","last_peered":"2024-09-16T07:36:03.839687+0000","last_clean":"2024-09-16T07:36:03.839687+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:36:03.839687+0000","last_undegraded":"2024-09-16T07:36:03.839687+0000","last_fullsized":"2024-09-16T07:36:03.839687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476902,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131260,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240580,"statfs":{"total":96636764160,"available":96502353920,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131256,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240584,"statfs":{"total":96636764160,"available":96502358016,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127676,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244164,"statfs":{"total":96636764160,"available":96506023936,"internally_reserved":0,"allocated":2203648,"data_stored":1295491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145144,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226696,"statfs":{"total":96636764160,"available":96488136704,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127140,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244700,"statfs":{"total":96636764160,"available":96506572800,"internally_reserved":0,"allocated":1650688,"data_stored":724759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509606,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159076,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212764,"statfs":{"total":96636764160,"available":96473870336,"internally_reserved":0,"allocated":1646592,"data_stored":724725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161216966},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509606,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119404,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252436,"statfs":{"total":96636764160,"available":96514494464,"internally_reserved":0,"allocated":2252800,"data_stored":1315835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542308,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135440,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236400,"statfs":{"total":96636764160,"available":96498073600,"internally_reserved":0,"allocated":1634304,"data_stored":718341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":137034181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:36:09.047 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:36:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.241+0000 7f9742800640 1 -- 172.21.15.71:0/906424585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9720003a50 msgr2=0x7f9720003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.241+0000 7f9742800640 1 --2- 172.21.15.71:0/906424585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9720003a50 0x7f9720003e70 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f972c00f460 tx=0x7f9720005830 comp rx=0 tx=0).stop 2024-09-16T07:36:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.241+0000 7f9742800640 1 -- 172.21.15.71:0/906424585 shutdown_connections 2024-09-16T07:36:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.241+0000 7f9742800640 1 --2- 172.21.15.71:0/906424585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9720003a50 0x7f9720003e70 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.241+0000 7f9742800640 1 -- 172.21.15.71:0/906424585 >> 172.21.15.71:0/906424585 conn(0x7f973c0b1950 msgr2=0x7f973c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.241+0000 7f9742800640 1 -- 172.21.15.71:0/906424585 shutdown_connections 2024-09-16T07:36:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.241+0000 7f9742800640 1 -- 172.21.15.71:0/906424585 wait complete. 2024-09-16T07:36:09.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.243+0000 7f9742800640 1 Processor -- start 2024-09-16T07:36:09.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.243+0000 7f9742800640 1 -- start start 2024-09-16T07:36:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f9742800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9720003a50 0x7f973c1e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f9742800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f973c0cc830 0x7f973c15bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f9742800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f973c0ccc90 0x7f973c157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f9742800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f973c15dd20 con 0x7f973c0ccc90 2024-09-16T07:36:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f9742800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f973c0746a0 con 0x7f973c0cc830 2024-09-16T07:36:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f9742800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f973c074820 con 0x7f9720003a50 2024-09-16T07:36:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f973be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f973c0ccc90 0x7f973c157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f973be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f973c0ccc90 0x7f973c157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40466/0 (socket says 172.21.15.71:40466) 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f973be00640 1 -- 172.21.15.71:0/178699939 learned_addr learned my addr 172.21.15.71:0/178699939 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f973aa00640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f973c0cc830 0x7f973c15bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f973b400640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9720003a50 0x7f973c1e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f973be00640 1 -- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9720003a50 msgr2=0x7f973c1e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.244+0000 7f973be00640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9720003a50 0x7f973c1e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f973be00640 1 -- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f973c0cc830 msgr2=0x7f973c15bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f973be00640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f973c0cc830 0x7f973c15bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f973be00640 1 -- 172.21.15.71:0/178699939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c15b5f0 con 0x7f973c0ccc90 2024-09-16T07:36:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f973be00640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f973c0ccc90 0x7f973c157260 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f973c0af890 tx=0x7f9734001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f9733400640 1 -- 172.21.15.71:0/178699939 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734011660 con 0x7f973c0ccc90 2024-09-16T07:36:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f9733400640 1 -- 172.21.15.71:0/178699939 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9734017070 con 0x7f973c0ccc90 2024-09-16T07:36:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f9733400640 1 -- 172.21.15.71:0/178699939 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734011bb0 con 0x7f973c0ccc90 2024-09-16T07:36:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c15b8d0 con 0x7f973c0ccc90 2024-09-16T07:36:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.245+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c15c4e0 con 0x7f973c0ccc90 2024-09-16T07:36:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.246+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973c15c7a0 con 0x7f973c0ccc90 2024-09-16T07:36:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.247+0000 7f9733400640 1 -- 172.21.15.71:0/178699939 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f973401c460 con 0x7f973c0ccc90 2024-09-16T07:36:09.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.248+0000 7f9733400640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f971c07a6e0 0x7f971c07cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:09.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.248+0000 7f9733400640 1 -- 172.21.15.71:0/178699939 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f9734099cb0 con 0x7f973c0ccc90 2024-09-16T07:36:09.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.249+0000 7f973b400640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f971c07a6e0 0x7f971c07cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:09.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.250+0000 7f973b400640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f971c07a6e0 0x7f971c07cba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f972c036000 tx=0x7f972c004a40 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:09.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.253+0000 7f9733400640 1 -- 172.21.15.71:0/178699939 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9734063730 con 0x7f973c0ccc90 2024-09-16T07:36:09.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.435+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f973c15b3a0 con 0x7f971c07a6e0 2024-09-16T07:36:09.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.440+0000 7f9733400640 1 -- 172.21.15.71:0/178699939 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248831 (secure 0 0 0) 0x7f973c15b3a0 con 0x7f971c07a6e0 2024-09-16T07:36:09.442 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:09.443 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:36:09.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f971c07a6e0 msgr2=0x7f971c07cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f971c07a6e0 0x7f971c07cba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f972c036000 tx=0x7f972c004a40 comp rx=0 tx=0).stop 2024-09-16T07:36:09.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f973c0ccc90 msgr2=0x7f973c157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f973c0ccc90 0x7f973c157260 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f973c0af890 tx=0x7f9734001630 comp rx=0 tx=0).stop 2024-09-16T07:36:09.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 shutdown_connections 2024-09-16T07:36:09.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f971c07a6e0 0x7f971c07cba0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f973c0ccc90 0x7f973c157260 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f973c0cc830 0x7f973c15bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.444+0000 7f9742800640 1 --2- 172.21.15.71:0/178699939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9720003a50 0x7f973c1e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.445+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 >> 172.21.15.71:0/178699939 conn(0x7f973c0b1950 msgr2=0x7f973c0afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:09.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.445+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 shutdown_connections 2024-09-16T07:36:09.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.445+0000 7f9742800640 1 -- 172.21.15.71:0/178699939 wait complete. 2024-09-16T07:36:09.460 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-09-16T07:36:08.626004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076396,"kb_used_data":13884,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898324,"statfs":{"total":773094113280,"available":771991883776,"internally_reserved":0,"allocated":14217216,"data_stored":6894520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1087819312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805957"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.853420+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.853157+0000","last_became_peered":"2024-09-16T07:36:04.853157+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:29.795783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063845+0000","last_change":"2024-09-16T07:36:04.855589+0000","last_active":"2024-09-16T07:36:06.063845+0000","last_peered":"2024-09-16T07:36:06.063845+0000","last_clean":"2024-09-16T07:36:06.063845+0000","last_became_active":"2024-09-16T07:36:04.855323+0000","last_became_peered":"2024-09-16T07:36:04.855323+0000","last_unstale":"2024-09-16T07:36:06.063845+0000","last_undegraded":"2024-09-16T07:36:06.063845+0000","last_fullsized":"2024-09-16T07:36:06.063845+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:53.457746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847311+0000","last_change":"2024-09-16T07:36:03.847311+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847311+0000","last_undegraded":"2024-09-16T07:36:03.847311+0000","last_fullsized":"2024-09-16T07:36:03.847311+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855470+0000","last_change":"2024-09-16T07:36:04.859510+0000","last_active":"2024-09-16T07:36:05.855470+0000","last_peered":"2024-09-16T07:36:05.855470+0000","last_clean":"2024-09-16T07:36:05.855470+0000","last_became_active":"2024-09-16T07:36:04.859175+0000","last_became_peered":"2024-09-16T07:36:04.859175+0000","last_unstale":"2024-09-16T07:36:05.855470+0000","last_undegraded":"2024-09-16T07:36:05.855470+0000","last_fullsized":"2024-09-16T07:36:05.855470+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:42:35.105087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.843162+0000","last_change":"2024-09-16T07:36:03.843162+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.843162+0000","last_undegraded":"2024-09-16T07:36:03.843162+0000","last_fullsized":"2024-09-16T07:36:03.843162+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.841267+0000","last_change":"2024-09-16T07:36:03.841267+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.841267+0000","last_undegraded":"2024-09-16T07:36:03.841267+0000","last_fullsized":"2024-09-16T07:36:03.841267+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842354+0000","last_change":"2024-09-16T07:36:03.842354+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842354+0000","last_undegraded":"2024-09-16T07:36:03.842354+0000","last_fullsized":"2024-09-16T07:36:03.842354+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847905+0000","last_change":"2024-09-16T07:36:03.847905+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847905+0000","last_undegraded":"2024-09-16T07:36:03.847905+0000","last_fullsized":"2024-09-16T07:36:03.847905+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.851248+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.850709+0000","last_became_peered":"2024-09-16T07:36:04.850709+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:07.411898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855892+0000","last_change":"2024-09-16T07:36:04.854097+0000","last_active":"2024-09-16T07:36:05.855892+0000","last_peered":"2024-09-16T07:36:05.855892+0000","last_clean":"2024-09-16T07:36:05.855892+0000","last_became_active":"2024-09-16T07:36:04.853827+0000","last_became_peered":"2024-09-16T07:36:04.853827+0000","last_unstale":"2024-09-16T07:36:05.855892+0000","last_undegraded":"2024-09-16T07:36:05.855892+0000","last_fullsized":"2024-09-16T07:36:05.855892+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:21.875425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855153+0000","last_change":"2024-09-16T07:36:04.848716+0000","last_active":"2024-09-16T07:36:05.855153+0000","last_peered":"2024-09-16T07:36:05.855153+0000","last_clean":"2024-09-16T07:36:05.855153+0000","last_became_active":"2024-09-16T07:36:04.848457+0000","last_became_peered":"2024-09-16T07:36:04.848457+0000","last_unstale":"2024-09-16T07:36:05.855153+0000","last_undegraded":"2024-09-16T07:36:05.855153+0000","last_fullsized":"2024-09-16T07:36:05.855153+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:21.475235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856556+0000","last_change":"2024-09-16T07:36:04.859096+0000","last_active":"2024-09-16T07:36:05.856556+0000","last_peered":"2024-09-16T07:36:05.856556+0000","last_clean":"2024-09-16T07:36:05.856556+0000","last_became_active":"2024-09-16T07:36:04.858812+0000","last_became_peered":"2024-09-16T07:36:04.858812+0000","last_unstale":"2024-09-16T07:36:05.856556+0000","last_undegraded":"2024-09-16T07:36:05.856556+0000","last_fullsized":"2024-09-16T07:36:05.856556+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:59.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855973+0000","last_change":"2024-09-16T07:36:04.862176+0000","last_active":"2024-09-16T07:36:05.855973+0000","last_peered":"2024-09-16T07:36:05.855973+0000","last_clean":"2024-09-16T07:36:05.855973+0000","last_became_active":"2024-09-16T07:36:04.860550+0000","last_became_peered":"2024-09-16T07:36:04.860550+0000","last_unstale":"2024-09-16T07:36:05.855973+0000","last_undegraded":"2024-09-16T07:36:05.855973+0000","last_fullsized":"2024-09-16T07:36:05.855973+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:10.040307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854370+0000","last_change":"2024-09-16T07:36:04.855363+0000","last_active":"2024-09-16T07:36:05.854370+0000","last_peered":"2024-09-16T07:36:05.854370+0000","last_clean":"2024-09-16T07:36:05.854370+0000","last_became_active":"2024-09-16T07:36:04.855013+0000","last_became_peered":"2024-09-16T07:36:04.855013+0000","last_unstale":"2024-09-16T07:36:05.854370+0000","last_undegraded":"2024-09-16T07:36:05.854370+0000","last_fullsized":"2024-09-16T07:36:05.854370+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:13.040194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.846503+0000","last_change":"2024-09-16T07:36:03.846503+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.846503+0000","last_undegraded":"2024-09-16T07:36:03.846503+0000","last_fullsized":"2024-09-16T07:36:03.846503+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855025+0000","last_change":"2024-09-16T07:36:04.862436+0000","last_active":"2024-09-16T07:36:05.855025+0000","last_peered":"2024-09-16T07:36:05.855025+0000","last_clean":"2024-09-16T07:36:05.855025+0000","last_became_active":"2024-09-16T07:36:04.862125+0000","last_became_peered":"2024-09-16T07:36:04.862125+0000","last_unstale":"2024-09-16T07:36:05.855025+0000","last_undegraded":"2024-09-16T07:36:05.855025+0000","last_fullsized":"2024-09-16T07:36:05.855025+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:38.787658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063678+0000","last_change":"2024-09-16T07:36:04.857741+0000","last_active":"2024-09-16T07:36:06.063678+0000","last_peered":"2024-09-16T07:36:06.063678+0000","last_clean":"2024-09-16T07:36:06.063678+0000","last_became_active":"2024-09-16T07:36:04.854444+0000","last_became_peered":"2024-09-16T07:36:04.854444+0000","last_unstale":"2024-09-16T07:36:06.063678+0000","last_undegraded":"2024-09-16T07:36:06.063678+0000","last_fullsized":"2024-09-16T07:36:06.063678+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:15.100689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856475+0000","last_change":"2024-09-16T07:36:04.869720+0000","last_active":"2024-09-16T07:36:05.856475+0000","last_peered":"2024-09-16T07:36:05.856475+0000","last_clean":"2024-09-16T07:36:05.856475+0000","last_became_active":"2024-09-16T07:36:04.868956+0000","last_became_peered":"2024-09-16T07:36:04.868956+0000","last_unstale":"2024-09-16T07:36:05.856475+0000","last_undegraded":"2024-09-16T07:36:05.856475+0000","last_fullsized":"2024-09-16T07:36:05.856475+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:29.852710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063836+0000","last_change":"2024-09-16T07:36:04.845962+0000","last_active":"2024-09-16T07:36:06.063836+0000","last_peered":"2024-09-16T07:36:06.063836+0000","last_clean":"2024-09-16T07:36:06.063836+0000","last_became_active":"2024-09-16T07:36:04.845664+0000","last_became_peered":"2024-09-16T07:36:04.845664+0000","last_unstale":"2024-09-16T07:36:06.063836+0000","last_undegraded":"2024-09-16T07:36:06.063836+0000","last_fullsized":"2024-09-16T07:36:06.063836+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:32.167063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837882+0000","last_change":"2024-09-16T07:36:03.837882+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837882+0000","last_undegraded":"2024-09-16T07:36:03.837882+0000","last_fullsized":"2024-09-16T07:36:03.837882+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854174+0000","last_change":"2024-09-16T07:36:04.865540+0000","last_active":"2024-09-16T07:36:05.854174+0000","last_peered":"2024-09-16T07:36:05.854174+0000","last_clean":"2024-09-16T07:36:05.854174+0000","last_became_active":"2024-09-16T07:36:04.865232+0000","last_became_peered":"2024-09-16T07:36:04.865232+0000","last_unstale":"2024-09-16T07:36:05.854174+0000","last_undegraded":"2024-09-16T07:36:05.854174+0000","last_fullsized":"2024-09-16T07:36:05.854174+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:24.118557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855774+0000","last_change":"2024-09-16T07:36:04.857205+0000","last_active":"2024-09-16T07:36:05.855774+0000","last_peered":"2024-09-16T07:36:05.855774+0000","last_clean":"2024-09-16T07:36:05.855774+0000","last_became_active":"2024-09-16T07:36:04.856823+0000","last_became_peered":"2024-09-16T07:36:04.856823+0000","last_unstale":"2024-09-16T07:36:05.855774+0000","last_undegraded":"2024-09-16T07:36:05.855774+0000","last_fullsized":"2024-09-16T07:36:05.855774+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:16.355699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064085+0000","last_change":"2024-09-16T07:36:04.850897+0000","last_active":"2024-09-16T07:36:06.064085+0000","last_peered":"2024-09-16T07:36:06.064085+0000","last_clean":"2024-09-16T07:36:06.064085+0000","last_became_active":"2024-09-16T07:36:04.850635+0000","last_became_peered":"2024-09-16T07:36:04.850635+0000","last_unstale":"2024-09-16T07:36:06.064085+0000","last_undegraded":"2024-09-16T07:36:06.064085+0000","last_fullsized":"2024-09-16T07:36:06.064085+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:38.194427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857379+0000","last_change":"2024-09-16T07:36:04.855205+0000","last_active":"2024-09-16T07:36:05.857379+0000","last_peered":"2024-09-16T07:36:05.857379+0000","last_clean":"2024-09-16T07:36:05.857379+0000","last_became_active":"2024-09-16T07:36:04.854356+0000","last_became_peered":"2024-09-16T07:36:04.854356+0000","last_unstale":"2024-09-16T07:36:05.857379+0000","last_undegraded":"2024-09-16T07:36:05.857379+0000","last_fullsized":"2024-09-16T07:36:05.857379+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:23.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837934+0000","last_change":"2024-09-16T07:36:03.837934+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837934+0000","last_undegraded":"2024-09-16T07:36:03.837934+0000","last_fullsized":"2024-09-16T07:36:03.837934+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063430+0000","last_change":"2024-09-16T07:36:04.849621+0000","last_active":"2024-09-16T07:36:06.063430+0000","last_peered":"2024-09-16T07:36:06.063430+0000","last_clean":"2024-09-16T07:36:06.063430+0000","last_became_active":"2024-09-16T07:36:04.849325+0000","last_became_peered":"2024-09-16T07:36:04.849325+0000","last_unstale":"2024-09-16T07:36:06.063430+0000","last_undegraded":"2024-09-16T07:36:06.063430+0000","last_fullsized":"2024-09-16T07:36:06.063430+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:20.048326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063880+0000","last_change":"2024-09-16T07:36:04.840820+0000","last_active":"2024-09-16T07:36:06.063880+0000","last_peered":"2024-09-16T07:36:06.063880+0000","last_clean":"2024-09-16T07:36:06.063880+0000","last_became_active":"2024-09-16T07:36:04.839397+0000","last_became_peered":"2024-09-16T07:36:04.839397+0000","last_unstale":"2024-09-16T07:36:06.063880+0000","last_undegraded":"2024-09-16T07:36:06.063880+0000","last_fullsized":"2024-09-16T07:36:06.063880+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:01.349093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837391+0000","last_change":"2024-09-16T07:36:03.837391+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837391+0000","last_undegraded":"2024-09-16T07:36:03.837391+0000","last_fullsized":"2024-09-16T07:36:03.837391+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842895+0000","last_change":"2024-09-16T07:36:03.842895+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842895+0000","last_undegraded":"2024-09-16T07:36:03.842895+0000","last_fullsized":"2024-09-16T07:36:03.842895+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857460+0000","last_change":"2024-09-16T07:36:04.859595+0000","last_active":"2024-09-16T07:36:05.857460+0000","last_peered":"2024-09-16T07:36:05.857460+0000","last_clean":"2024-09-16T07:36:05.857460+0000","last_became_active":"2024-09-16T07:36:04.856512+0000","last_became_peered":"2024-09-16T07:36:04.856512+0000","last_unstale":"2024-09-16T07:36:05.857460+0000","last_undegraded":"2024-09-16T07:36:05.857460+0000","last_fullsized":"2024-09-16T07:36:05.857460+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:25.701980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858752+0000","last_change":"2024-09-16T07:36:03.855477+0000","last_active":"2024-09-16T07:36:03.858752+0000","last_peered":"2024-09-16T07:36:03.858752+0000","last_clean":"2024-09-16T07:36:03.858752+0000","last_became_active":"2024-09-16T07:36:03.852255+0000","last_became_peered":"2024-09-16T07:36:03.852255+0000","last_unstale":"2024-09-16T07:36:03.858752+0000","last_undegraded":"2024-09-16T07:36:03.858752+0000","last_fullsized":"2024-09-16T07:36:03.858752+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:58.216679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847493+0000","last_change":"2024-09-16T07:36:03.847493+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847493+0000","last_undegraded":"2024-09-16T07:36:03.847493+0000","last_fullsized":"2024-09-16T07:36:03.847493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.855023+0000","last_change":"2024-09-16T07:36:03.853595+0000","last_active":"2024-09-16T07:36:03.855023+0000","last_peered":"2024-09-16T07:36:03.855023+0000","last_clean":"2024-09-16T07:36:03.855023+0000","last_became_active":"2024-09-16T07:36:03.853315+0000","last_became_peered":"2024-09-16T07:36:03.853315+0000","last_unstale":"2024-09-16T07:36:03.855023+0000","last_undegraded":"2024-09-16T07:36:03.855023+0000","last_fullsized":"2024-09-16T07:36:03.855023+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:09.161606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.845779+0000","last_change":"2024-09-16T07:36:03.845779+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.845779+0000","last_undegraded":"2024-09-16T07:36:03.845779+0000","last_fullsized":"2024-09-16T07:36:03.845779+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856585+0000","last_change":"2024-09-16T07:36:03.849019+0000","last_active":"2024-09-16T07:36:05.856585+0000","last_peered":"2024-09-16T07:36:05.856585+0000","last_clean":"2024-09-16T07:36:05.856585+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.856585+0000","last_undegraded":"2024-09-16T07:36:05.856585+0000","last_fullsized":"2024-09-16T07:36:05.856585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:53.651818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063690+0000","last_change":"2024-09-16T07:36:03.853178+0000","last_active":"2024-09-16T07:36:06.063690+0000","last_peered":"2024-09-16T07:36:06.063690+0000","last_clean":"2024-09-16T07:36:06.063690+0000","last_became_active":"2024-09-16T07:36:03.852914+0000","last_became_peered":"2024-09-16T07:36:03.852914+0000","last_unstale":"2024-09-16T07:36:06.063690+0000","last_undegraded":"2024-09-16T07:36:06.063690+0000","last_fullsized":"2024-09-16T07:36:06.063690+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.868825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855929+0000","last_change":"2024-09-16T07:36:04.860375+0000","last_active":"2024-09-16T07:36:05.855929+0000","last_peered":"2024-09-16T07:36:05.855929+0000","last_clean":"2024-09-16T07:36:05.855929+0000","last_became_active":"2024-09-16T07:36:04.860110+0000","last_became_peered":"2024-09-16T07:36:04.860110+0000","last_unstale":"2024-09-16T07:36:05.855929+0000","last_undegraded":"2024-09-16T07:36:05.855929+0000","last_fullsized":"2024-09-16T07:36:05.855929+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:44.383231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063342+0000","last_change":"2024-09-16T07:36:03.848993+0000","last_active":"2024-09-16T07:36:06.063342+0000","last_peered":"2024-09-16T07:36:06.063342+0000","last_clean":"2024-09-16T07:36:06.063342+0000","last_became_active":"2024-09-16T07:36:03.848700+0000","last_became_peered":"2024-09-16T07:36:03.848700+0000","last_unstale":"2024-09-16T07:36:06.063342+0000","last_undegraded":"2024-09-16T07:36:06.063342+0000","last_fullsized":"2024-09-16T07:36:06.063342+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:54.220868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856372+0000","last_change":"2024-09-16T07:36:03.855637+0000","last_active":"2024-09-16T07:36:05.856372+0000","last_peered":"2024-09-16T07:36:05.856372+0000","last_clean":"2024-09-16T07:36:05.856372+0000","last_became_active":"2024-09-16T07:36:03.855357+0000","last_became_peered":"2024-09-16T07:36:03.855357+0000","last_unstale":"2024-09-16T07:36:05.856372+0000","last_undegraded":"2024-09-16T07:36:05.856372+0000","last_fullsized":"2024-09-16T07:36:05.856372+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.392485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858894+0000","last_change":"2024-09-16T07:36:03.858168+0000","last_active":"2024-09-16T07:36:03.858894+0000","last_peered":"2024-09-16T07:36:03.858894+0000","last_clean":"2024-09-16T07:36:03.858894+0000","last_became_active":"2024-09-16T07:36:03.857697+0000","last_became_peered":"2024-09-16T07:36:03.857697+0000","last_unstale":"2024-09-16T07:36:03.858894+0000","last_undegraded":"2024-09-16T07:36:03.858894+0000","last_fullsized":"2024-09-16T07:36:03.858894+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:36.388550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063343+0000","last_change":"2024-09-16T07:36:04.853058+0000","last_active":"2024-09-16T07:36:06.063343+0000","last_peered":"2024-09-16T07:36:06.063343+0000","last_clean":"2024-09-16T07:36:06.063343+0000","last_became_active":"2024-09-16T07:36:04.852535+0000","last_became_peered":"2024-09-16T07:36:04.852535+0000","last_unstale":"2024-09-16T07:36:06.063343+0000","last_undegraded":"2024-09-16T07:36:06.063343+0000","last_fullsized":"2024-09-16T07:36:06.063343+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:36.784633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063356+0000","last_change":"2024-09-16T07:36:03.846732+0000","last_active":"2024-09-16T07:36:06.063356+0000","last_peered":"2024-09-16T07:36:06.063356+0000","last_clean":"2024-09-16T07:36:06.063356+0000","last_became_active":"2024-09-16T07:36:03.845789+0000","last_became_peered":"2024-09-16T07:36:03.845789+0000","last_unstale":"2024-09-16T07:36:06.063356+0000","last_undegraded":"2024-09-16T07:36:06.063356+0000","last_fullsized":"2024-09-16T07:36:06.063356+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:53.374127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064066+0000","last_change":"2024-09-16T07:36:04.852160+0000","last_active":"2024-09-16T07:36:06.064066+0000","last_peered":"2024-09-16T07:36:06.064066+0000","last_clean":"2024-09-16T07:36:06.064066+0000","last_became_active":"2024-09-16T07:36:04.851903+0000","last_became_peered":"2024-09-16T07:36:04.851903+0000","last_unstale":"2024-09-16T07:36:06.064066+0000","last_undegraded":"2024-09-16T07:36:06.064066+0000","last_fullsized":"2024-09-16T07:36:06.064066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:32.387978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.832128+0000","last_change":"2024-09-16T07:36:02.832128+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.832128+0000","last_undegraded":"2024-09-16T07:36:02.832128+0000","last_fullsized":"2024-09-16T07:36:02.832128+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842513+0000","last_change":"2024-09-16T07:36:03.842513+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842513+0000","last_undegraded":"2024-09-16T07:36:03.842513+0000","last_fullsized":"2024-09-16T07:36:03.842513+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855092+0000","last_change":"2024-09-16T07:36:03.852706+0000","last_active":"2024-09-16T07:36:05.855092+0000","last_peered":"2024-09-16T07:36:05.855092+0000","last_clean":"2024-09-16T07:36:05.855092+0000","last_became_active":"2024-09-16T07:36:03.852295+0000","last_became_peered":"2024-09-16T07:36:03.852295+0000","last_unstale":"2024-09-16T07:36:05.855092+0000","last_undegraded":"2024-09-16T07:36:05.855092+0000","last_fullsized":"2024-09-16T07:36:05.855092+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:51.417749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.859083+0000","last_change":"2024-09-16T07:36:03.857233+0000","last_active":"2024-09-16T07:36:03.859083+0000","last_peered":"2024-09-16T07:36:03.859083+0000","last_clean":"2024-09-16T07:36:03.859083+0000","last_became_active":"2024-09-16T07:36:03.856817+0000","last_became_peered":"2024-09-16T07:36:03.856817+0000","last_unstale":"2024-09-16T07:36:03.859083+0000","last_undegraded":"2024-09-16T07:36:03.859083+0000","last_fullsized":"2024-09-16T07:36:03.859083+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:45.111727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837921+0000","last_change":"2024-09-16T07:36:03.837921+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837921+0000","last_undegraded":"2024-09-16T07:36:03.837921+0000","last_fullsized":"2024-09-16T07:36:03.837921+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.854803+0000","last_change":"2024-09-16T07:36:03.851395+0000","last_active":"2024-09-16T07:36:03.854803+0000","last_peered":"2024-09-16T07:36:03.854803+0000","last_clean":"2024-09-16T07:36:03.854803+0000","last_became_active":"2024-09-16T07:36:03.851127+0000","last_became_peered":"2024-09-16T07:36:03.851127+0000","last_unstale":"2024-09-16T07:36:03.854803+0000","last_undegraded":"2024-09-16T07:36:03.854803+0000","last_fullsized":"2024-09-16T07:36:03.854803+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:50.717571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856941+0000","last_change":"2024-09-16T07:36:04.863490+0000","last_active":"2024-09-16T07:36:05.856941+0000","last_peered":"2024-09-16T07:36:05.856941+0000","last_clean":"2024-09-16T07:36:05.856941+0000","last_became_active":"2024-09-16T07:36:04.862482+0000","last_became_peered":"2024-09-16T07:36:04.862482+0000","last_unstale":"2024-09-16T07:36:05.856941+0000","last_undegraded":"2024-09-16T07:36:05.856941+0000","last_fullsized":"2024-09-16T07:36:05.856941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:18.732185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856621+0000","last_change":"2024-09-16T07:36:03.852858+0000","last_active":"2024-09-16T07:36:05.856621+0000","last_peered":"2024-09-16T07:36:05.856621+0000","last_clean":"2024-09-16T07:36:05.856621+0000","last_became_active":"2024-09-16T07:36:03.852596+0000","last_became_peered":"2024-09-16T07:36:03.852596+0000","last_unstale":"2024-09-16T07:36:05.856621+0000","last_undegraded":"2024-09-16T07:36:05.856621+0000","last_fullsized":"2024-09-16T07:36:05.856621+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:29.307382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854662+0000","last_change":"2024-09-16T07:36:04.855631+0000","last_active":"2024-09-16T07:36:05.854662+0000","last_peered":"2024-09-16T07:36:05.854662+0000","last_clean":"2024-09-16T07:36:05.854662+0000","last_became_active":"2024-09-16T07:36:04.855341+0000","last_became_peered":"2024-09-16T07:36:04.855341+0000","last_unstale":"2024-09-16T07:36:05.854662+0000","last_undegraded":"2024-09-16T07:36:05.854662+0000","last_fullsized":"2024-09-16T07:36:05.854662+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:06.127741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854531+0000","last_change":"2024-09-16T07:36:03.854082+0000","last_active":"2024-09-16T07:36:05.854531+0000","last_peered":"2024-09-16T07:36:05.854531+0000","last_clean":"2024-09-16T07:36:05.854531+0000","last_became_active":"2024-09-16T07:36:03.853794+0000","last_became_peered":"2024-09-16T07:36:03.853794+0000","last_unstale":"2024-09-16T07:36:05.854531+0000","last_undegraded":"2024-09-16T07:36:05.854531+0000","last_fullsized":"2024-09-16T07:36:05.854531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:10.982614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.830540+0000","last_change":"2024-09-16T07:36:02.830540+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.830540+0000","last_undegraded":"2024-09-16T07:36:02.830540+0000","last_fullsized":"2024-09-16T07:36:02.830540+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.843486+0000","last_change":"2024-09-16T07:36:03.843486+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.843486+0000","last_undegraded":"2024-09-16T07:36:03.843486+0000","last_fullsized":"2024-09-16T07:36:03.843486+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.828755+0000","last_change":"2024-09-16T07:36:02.828755+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.828755+0000","last_undegraded":"2024-09-16T07:36:02.828755+0000","last_fullsized":"2024-09-16T07:36:02.828755+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842116+0000","last_change":"2024-09-16T07:36:03.842116+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842116+0000","last_undegraded":"2024-09-16T07:36:03.842116+0000","last_fullsized":"2024-09-16T07:36:03.842116+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856205+0000","last_change":"2024-09-16T07:36:03.850133+0000","last_active":"2024-09-16T07:36:05.856205+0000","last_peered":"2024-09-16T07:36:05.856205+0000","last_clean":"2024-09-16T07:36:05.856205+0000","last_became_active":"2024-09-16T07:36:03.849070+0000","last_became_peered":"2024-09-16T07:36:03.849070+0000","last_unstale":"2024-09-16T07:36:05.856205+0000","last_undegraded":"2024-09-16T07:36:05.856205+0000","last_fullsized":"2024-09-16T07:36:05.856205+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:35.166163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063646+0000","last_change":"2024-09-16T07:36:04.848685+0000","last_active":"2024-09-16T07:36:06.063646+0000","last_peered":"2024-09-16T07:36:06.063646+0000","last_clean":"2024-09-16T07:36:06.063646+0000","last_became_active":"2024-09-16T07:36:04.844479+0000","last_became_peered":"2024-09-16T07:36:04.844479+0000","last_unstale":"2024-09-16T07:36:06.063646+0000","last_undegraded":"2024-09-16T07:36:06.063646+0000","last_fullsized":"2024-09-16T07:36:06.063646+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:36.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.857074+0000","last_change":"2024-09-16T07:36:03.855122+0000","last_active":"2024-09-16T07:36:03.857074+0000","last_peered":"2024-09-16T07:36:03.857074+0000","last_clean":"2024-09-16T07:36:03.857074+0000","last_became_active":"2024-09-16T07:36:03.854786+0000","last_became_peered":"2024-09-16T07:36:03.854786+0000","last_unstale":"2024-09-16T07:36:03.857074+0000","last_undegraded":"2024-09-16T07:36:03.857074+0000","last_fullsized":"2024-09-16T07:36:03.857074+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:02.212316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063917+0000","last_change":"2024-09-16T07:36:04.853245+0000","last_active":"2024-09-16T07:36:06.063917+0000","last_peered":"2024-09-16T07:36:06.063917+0000","last_clean":"2024-09-16T07:36:06.063917+0000","last_became_active":"2024-09-16T07:36:04.852895+0000","last_became_peered":"2024-09-16T07:36:04.852895+0000","last_unstale":"2024-09-16T07:36:06.063917+0000","last_undegraded":"2024-09-16T07:36:06.063917+0000","last_fullsized":"2024-09-16T07:36:06.063917+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.199780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858884+0000","last_change":"2024-09-16T07:36:03.856130+0000","last_active":"2024-09-16T07:36:03.858884+0000","last_peered":"2024-09-16T07:36:03.858884+0000","last_clean":"2024-09-16T07:36:03.858884+0000","last_became_active":"2024-09-16T07:36:03.852405+0000","last_became_peered":"2024-09-16T07:36:03.852405+0000","last_unstale":"2024-09-16T07:36:03.858884+0000","last_undegraded":"2024-09-16T07:36:03.858884+0000","last_fullsized":"2024-09-16T07:36:03.858884+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:51.417749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064061+0000","last_change":"2024-09-16T07:36:04.847598+0000","last_active":"2024-09-16T07:36:06.064061+0000","last_peered":"2024-09-16T07:36:06.064061+0000","last_clean":"2024-09-16T07:36:06.064061+0000","last_became_active":"2024-09-16T07:36:04.847063+0000","last_became_peered":"2024-09-16T07:36:04.847063+0000","last_unstale":"2024-09-16T07:36:06.064061+0000","last_undegraded":"2024-09-16T07:36:06.064061+0000","last_fullsized":"2024-09-16T07:36:06.064061+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:50.838203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858292+0000","last_change":"2024-09-16T07:36:03.856847+0000","last_active":"2024-09-16T07:36:03.858292+0000","last_peered":"2024-09-16T07:36:03.858292+0000","last_clean":"2024-09-16T07:36:03.858292+0000","last_became_active":"2024-09-16T07:36:03.856321+0000","last_became_peered":"2024-09-16T07:36:03.856321+0000","last_unstale":"2024-09-16T07:36:03.858292+0000","last_undegraded":"2024-09-16T07:36:03.858292+0000","last_fullsized":"2024-09-16T07:36:03.858292+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:57.138229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837398+0000","last_change":"2024-09-16T07:36:03.837398+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837398+0000","last_undegraded":"2024-09-16T07:36:03.837398+0000","last_fullsized":"2024-09-16T07:36:03.837398+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856080+0000","last_change":"2024-09-16T07:36:03.855412+0000","last_active":"2024-09-16T07:36:05.856080+0000","last_peered":"2024-09-16T07:36:05.856080+0000","last_clean":"2024-09-16T07:36:05.856080+0000","last_became_active":"2024-09-16T07:36:03.854933+0000","last_became_peered":"2024-09-16T07:36:03.854933+0000","last_unstale":"2024-09-16T07:36:05.856080+0000","last_undegraded":"2024-09-16T07:36:05.856080+0000","last_fullsized":"2024-09-16T07:36:05.856080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:58.216679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854581+0000","last_change":"2024-09-16T07:36:04.850083+0000","last_active":"2024-09-16T07:36:05.854581+0000","last_peered":"2024-09-16T07:36:05.854581+0000","last_clean":"2024-09-16T07:36:05.854581+0000","last_became_active":"2024-09-16T07:36:04.849731+0000","last_became_peered":"2024-09-16T07:36:04.849731+0000","last_unstale":"2024-09-16T07:36:05.854581+0000","last_undegraded":"2024-09-16T07:36:05.854581+0000","last_fullsized":"2024-09-16T07:36:05.854581+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:08.133005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855297+0000","last_change":"2024-09-16T07:36:03.848269+0000","last_active":"2024-09-16T07:36:05.855297+0000","last_peered":"2024-09-16T07:36:05.855297+0000","last_clean":"2024-09-16T07:36:05.855297+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.855297+0000","last_undegraded":"2024-09-16T07:36:05.855297+0000","last_fullsized":"2024-09-16T07:36:05.855297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:06.554515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856499+0000","last_change":"2024-09-16T07:36:03.847015+0000","last_active":"2024-09-16T07:36:05.856499+0000","last_peered":"2024-09-16T07:36:05.856499+0000","last_clean":"2024-09-16T07:36:05.856499+0000","last_became_active":"2024-09-16T07:36:03.846719+0000","last_became_peered":"2024-09-16T07:36:03.846719+0000","last_unstale":"2024-09-16T07:36:05.856499+0000","last_undegraded":"2024-09-16T07:36:05.856499+0000","last_fullsized":"2024-09-16T07:36:05.856499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:46.177351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.838152+0000","last_change":"2024-09-16T07:36:03.838152+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.838152+0000","last_undegraded":"2024-09-16T07:36:03.838152+0000","last_fullsized":"2024-09-16T07:36:03.838152+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856245+0000","last_change":"2024-09-16T07:36:03.854810+0000","last_active":"2024-09-16T07:36:05.856245+0000","last_peered":"2024-09-16T07:36:05.856245+0000","last_clean":"2024-09-16T07:36:05.856245+0000","last_became_active":"2024-09-16T07:36:03.854533+0000","last_became_peered":"2024-09-16T07:36:03.854533+0000","last_unstale":"2024-09-16T07:36:05.856245+0000","last_undegraded":"2024-09-16T07:36:05.856245+0000","last_fullsized":"2024-09-16T07:36:05.856245+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:23.851837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063875+0000","last_change":"2024-09-16T07:36:04.855496+0000","last_active":"2024-09-16T07:36:06.063875+0000","last_peered":"2024-09-16T07:36:06.063875+0000","last_clean":"2024-09-16T07:36:06.063875+0000","last_became_active":"2024-09-16T07:36:04.855167+0000","last_became_peered":"2024-09-16T07:36:04.855167+0000","last_unstale":"2024-09-16T07:36:06.063875+0000","last_undegraded":"2024-09-16T07:36:06.063875+0000","last_fullsized":"2024-09-16T07:36:06.063875+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:24.513277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.859106+0000","last_change":"2024-09-16T07:36:03.857828+0000","last_active":"2024-09-16T07:36:03.859106+0000","last_peered":"2024-09-16T07:36:03.859106+0000","last_clean":"2024-09-16T07:36:03.859106+0000","last_became_active":"2024-09-16T07:36:03.853109+0000","last_became_peered":"2024-09-16T07:36:03.853109+0000","last_unstale":"2024-09-16T07:36:03.859106+0000","last_undegraded":"2024-09-16T07:36:03.859106+0000","last_fullsized":"2024-09-16T07:36:03.859106+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:04.946096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857004+0000","last_change":"2024-09-16T07:36:04.852909+0000","last_active":"2024-09-16T07:36:05.857004+0000","last_peered":"2024-09-16T07:36:05.857004+0000","last_clean":"2024-09-16T07:36:05.857004+0000","last_became_active":"2024-09-16T07:36:04.852496+0000","last_became_peered":"2024-09-16T07:36:04.852496+0000","last_unstale":"2024-09-16T07:36:05.857004+0000","last_undegraded":"2024-09-16T07:36:05.857004+0000","last_fullsized":"2024-09-16T07:36:05.857004+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:43.843163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854718+0000","last_change":"2024-09-16T07:36:03.853605+0000","last_active":"2024-09-16T07:36:05.854718+0000","last_peered":"2024-09-16T07:36:05.854718+0000","last_clean":"2024-09-16T07:36:05.854718+0000","last_became_active":"2024-09-16T07:36:03.853349+0000","last_became_peered":"2024-09-16T07:36:03.853349+0000","last_unstale":"2024-09-16T07:36:05.854718+0000","last_undegraded":"2024-09-16T07:36:05.854718+0000","last_fullsized":"2024-09-16T07:36:05.854718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:29.592986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.846573+0000","last_change":"2024-09-16T07:36:03.846573+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.846573+0000","last_undegraded":"2024-09-16T07:36:03.846573+0000","last_fullsized":"2024-09-16T07:36:03.846573+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857065+0000","last_change":"2024-09-16T07:36:03.851730+0000","last_active":"2024-09-16T07:36:05.857065+0000","last_peered":"2024-09-16T07:36:05.857065+0000","last_clean":"2024-09-16T07:36:05.857065+0000","last_became_active":"2024-09-16T07:36:03.851457+0000","last_became_peered":"2024-09-16T07:36:03.851457+0000","last_unstale":"2024-09-16T07:36:05.857065+0000","last_undegraded":"2024-09-16T07:36:05.857065+0000","last_fullsized":"2024-09-16T07:36:05.857065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:37.222531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063468+0000","last_change":"2024-09-16T07:36:03.857845+0000","last_active":"2024-09-16T07:36:06.063468+0000","last_peered":"2024-09-16T07:36:06.063468+0000","last_clean":"2024-09-16T07:36:06.063468+0000","last_became_active":"2024-09-16T07:36:03.857366+0000","last_became_peered":"2024-09-16T07:36:03.857366+0000","last_unstale":"2024-09-16T07:36:06.063468+0000","last_undegraded":"2024-09-16T07:36:06.063468+0000","last_fullsized":"2024-09-16T07:36:06.063468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:36.413668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.830304+0000","last_change":"2024-09-16T07:36:02.830304+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.830304+0000","last_undegraded":"2024-09-16T07:36:02.830304+0000","last_fullsized":"2024-09-16T07:36:02.830304+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855281+0000","last_change":"2024-09-16T07:36:04.863145+0000","last_active":"2024-09-16T07:36:05.855281+0000","last_peered":"2024-09-16T07:36:05.855281+0000","last_clean":"2024-09-16T07:36:05.855281+0000","last_became_active":"2024-09-16T07:36:04.862893+0000","last_became_peered":"2024-09-16T07:36:04.862893+0000","last_unstale":"2024-09-16T07:36:05.855281+0000","last_undegraded":"2024-09-16T07:36:05.855281+0000","last_fullsized":"2024-09-16T07:36:05.855281+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:26.614434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854639+0000","last_change":"2024-09-16T07:36:03.851573+0000","last_active":"2024-09-16T07:36:05.854639+0000","last_peered":"2024-09-16T07:36:05.854639+0000","last_clean":"2024-09-16T07:36:05.854639+0000","last_became_active":"2024-09-16T07:36:03.851286+0000","last_became_peered":"2024-09-16T07:36:03.851286+0000","last_unstale":"2024-09-16T07:36:05.854639+0000","last_undegraded":"2024-09-16T07:36:05.854639+0000","last_fullsized":"2024-09-16T07:36:05.854639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.290737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856129+0000","last_change":"2024-09-16T07:36:04.858119+0000","last_active":"2024-09-16T07:36:05.856129+0000","last_peered":"2024-09-16T07:36:05.856129+0000","last_clean":"2024-09-16T07:36:05.856129+0000","last_became_active":"2024-09-16T07:36:04.855972+0000","last_became_peered":"2024-09-16T07:36:04.855972+0000","last_unstale":"2024-09-16T07:36:05.856129+0000","last_undegraded":"2024-09-16T07:36:05.856129+0000","last_fullsized":"2024-09-16T07:36:05.856129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:40.599143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854647+0000","last_change":"2024-09-16T07:36:03.854300+0000","last_active":"2024-09-16T07:36:05.854647+0000","last_peered":"2024-09-16T07:36:05.854647+0000","last_clean":"2024-09-16T07:36:05.854647+0000","last_became_active":"2024-09-16T07:36:03.853971+0000","last_became_peered":"2024-09-16T07:36:03.853971+0000","last_unstale":"2024-09-16T07:36:05.854647+0000","last_undegraded":"2024-09-16T07:36:05.854647+0000","last_fullsized":"2024-09-16T07:36:05.854647+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:25.287389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.844781+0000","last_change":"2024-09-16T07:36:03.844781+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.844781+0000","last_undegraded":"2024-09-16T07:36:03.844781+0000","last_fullsized":"2024-09-16T07:36:03.844781+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.828990+0000","last_change":"2024-09-16T07:36:02.828990+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.828990+0000","last_undegraded":"2024-09-16T07:36:02.828990+0000","last_fullsized":"2024-09-16T07:36:02.828990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":310,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.839687+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:36:03.839687+0000","last_peered":"2024-09-16T07:36:03.839687+0000","last_clean":"2024-09-16T07:36:03.839687+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:36:03.839687+0000","last_undegraded":"2024-09-16T07:36:03.839687+0000","last_fullsized":"2024-09-16T07:36:03.839687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476902,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131260,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240580,"statfs":{"total":96636764160,"available":96502353920,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131256,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240584,"statfs":{"total":96636764160,"available":96502358016,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127676,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244164,"statfs":{"total":96636764160,"available":96506023936,"internally_reserved":0,"allocated":2203648,"data_stored":1295491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145144,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226696,"statfs":{"total":96636764160,"available":96488136704,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127140,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244700,"statfs":{"total":96636764160,"available":96506572800,"internally_reserved":0,"allocated":1650688,"data_stored":724759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509606,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159076,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212764,"statfs":{"total":96636764160,"available":96473870336,"internally_reserved":0,"allocated":1646592,"data_stored":724725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161216966},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509606,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119404,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252436,"statfs":{"total":96636764160,"available":96514494464,"internally_reserved":0,"allocated":2252800,"data_stored":1315835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542308,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135440,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236400,"statfs":{"total":96636764160,"available":96498073600,"internally_reserved":0,"allocated":1634304,"data_stored":718341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":137034181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:36:09.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:36:09.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.658+0000 7f7227200640 1 -- 172.21.15.71:0/3631913755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72280ad880 msgr2=0x7f72280adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.658+0000 7f7227200640 1 --2- 172.21.15.71:0/3631913755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72280ad880 0x7f72280adcd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7214005fd0 tx=0x7f721402a5d0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.658+0000 7f7227200640 1 -- 172.21.15.71:0/3631913755 shutdown_connections 2024-09-16T07:36:09.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.658+0000 7f7227200640 1 --2- 172.21.15.71:0/3631913755 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72280cd5f0 0x7f72280cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.658+0000 7f7227200640 1 --2- 172.21.15.71:0/3631913755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72280ad880 0x7f72280adcd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.658+0000 7f7227200640 1 -- 172.21.15.71:0/3631913755 >> 172.21.15.71:0/3631913755 conn(0x7f72280cfa20 msgr2=0x7f72280cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:09.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.658+0000 7f7227200640 1 -- 172.21.15.71:0/3631913755 shutdown_connections 2024-09-16T07:36:09.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.658+0000 7f7227200640 1 -- 172.21.15.71:0/3631913755 wait complete. 2024-09-16T07:36:09.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.660+0000 7f7227200640 1 Processor -- start 2024-09-16T07:36:09.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.660+0000 7f7227200640 1 -- start start 2024-09-16T07:36:09.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7227200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722806fb90 0x7f72281f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:09.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7227200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72280ad880 0x7f72280b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:09.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7227200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72280cd5f0 0x7f72280b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7227200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f722815edf0 con 0x7f72280cd5f0 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7227200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f722815ec70 con 0x7f722806fb90 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7227200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f722815ef70 con 0x7f72280ad880 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7226800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72280cd5f0 0x7f72280b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7226800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72280cd5f0 0x7f72280b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40468/0 (socket says 172.21.15.71:40468) 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7226800640 1 -- 172.21.15.71:0/4265987858 learned_addr learned my addr 172.21.15.71:0/4265987858 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7225400640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72280ad880 0x7f72280b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7225e00640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722806fb90 0x7f72281f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7226800640 1 -- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72280ad880 msgr2=0x7f72280b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7226800640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72280ad880 0x7f72280b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7226800640 1 -- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722806fb90 msgr2=0x7f72281f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7226800640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722806fb90 0x7f72281f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.661+0000 7f7226800640 1 -- 172.21.15.71:0/4265987858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72280c2b90 con 0x7f72280cd5f0 2024-09-16T07:36:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.662+0000 7f7226800640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72280cd5f0 0x7f72280b2e30 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f722000ea80 tx=0x7f722000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.662+0000 7f721e000640 1 -- 172.21.15.71:0/4265987858 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220004b20 con 0x7f72280cd5f0 2024-09-16T07:36:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.662+0000 7f721e000640 1 -- 172.21.15.71:0/4265987858 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7220004ce0 con 0x7f72280cd5f0 2024-09-16T07:36:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.662+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72280c26b0 con 0x7f72280cd5f0 2024-09-16T07:36:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.662+0000 7f721e000640 1 -- 172.21.15.71:0/4265987858 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220005280 con 0x7f72280cd5f0 2024-09-16T07:36:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.662+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72280706d0 con 0x7f72280cd5f0 2024-09-16T07:36:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.663+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72280c68a0 con 0x7f72280cd5f0 2024-09-16T07:36:09.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.664+0000 7f721e000640 1 -- 172.21.15.71:0/4265987858 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f7220005420 con 0x7f72280cd5f0 2024-09-16T07:36:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.666+0000 7f721e000640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f720807a6b0 0x7f720807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.666+0000 7f721e000640 1 -- 172.21.15.71:0/4265987858 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f722009a560 con 0x7f72280cd5f0 2024-09-16T07:36:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.666+0000 7f7225e00640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f720807a6b0 0x7f720807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.667+0000 7f7225e00640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f720807a6b0 0x7f720807cb70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f72280708d0 tx=0x7f7218009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:09.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.670+0000 7f721e000640 1 -- 172.21.15.71:0/4265987858 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f72280c68a0 con 0x7f72280cd5f0 2024-09-16T07:36:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.852+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72280adcd0 con 0x7f720807a6b0 2024-09-16T07:36:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.857+0000 7f721e000640 1 -- 172.21.15.71:0/4265987858 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248831 (secure 0 0 0) 0x7f72280adcd0 con 0x7f720807a6b0 2024-09-16T07:36:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:09.860 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f720807a6b0 msgr2=0x7f720807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f720807a6b0 0x7f720807cb70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f72280708d0 tx=0x7f7218009040 comp rx=0 tx=0).stop 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72280cd5f0 msgr2=0x7f72280b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72280cd5f0 0x7f72280b2e30 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f722000ea80 tx=0x7f722000ef40 comp rx=0 tx=0).stop 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 shutdown_connections 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f720807a6b0 0x7f720807cb70 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72280cd5f0 0x7f72280b2e30 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72280ad880 0x7f72280b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 --2- 172.21.15.71:0/4265987858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722806fb90 0x7f72281f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 >> 172.21.15.71:0/4265987858 conn(0x7f72280cfa20 msgr2=0x7f72280725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:09.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 shutdown_connections 2024-09-16T07:36:09.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:09.861+0000 7f7227200640 1 -- 172.21.15.71:0/4265987858 wait complete. 2024-09-16T07:36:09.876 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-09-16T07:36:08.626004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076396,"kb_used_data":13884,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898324,"statfs":{"total":773094113280,"available":771991883776,"internally_reserved":0,"allocated":14217216,"data_stored":6894520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1087819312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.805957"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.853420+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.853157+0000","last_became_peered":"2024-09-16T07:36:04.853157+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:29.795783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063845+0000","last_change":"2024-09-16T07:36:04.855589+0000","last_active":"2024-09-16T07:36:06.063845+0000","last_peered":"2024-09-16T07:36:06.063845+0000","last_clean":"2024-09-16T07:36:06.063845+0000","last_became_active":"2024-09-16T07:36:04.855323+0000","last_became_peered":"2024-09-16T07:36:04.855323+0000","last_unstale":"2024-09-16T07:36:06.063845+0000","last_undegraded":"2024-09-16T07:36:06.063845+0000","last_fullsized":"2024-09-16T07:36:06.063845+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:53.457746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847311+0000","last_change":"2024-09-16T07:36:03.847311+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847311+0000","last_undegraded":"2024-09-16T07:36:03.847311+0000","last_fullsized":"2024-09-16T07:36:03.847311+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855470+0000","last_change":"2024-09-16T07:36:04.859510+0000","last_active":"2024-09-16T07:36:05.855470+0000","last_peered":"2024-09-16T07:36:05.855470+0000","last_clean":"2024-09-16T07:36:05.855470+0000","last_became_active":"2024-09-16T07:36:04.859175+0000","last_became_peered":"2024-09-16T07:36:04.859175+0000","last_unstale":"2024-09-16T07:36:05.855470+0000","last_undegraded":"2024-09-16T07:36:05.855470+0000","last_fullsized":"2024-09-16T07:36:05.855470+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:42:35.105087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.843162+0000","last_change":"2024-09-16T07:36:03.843162+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.843162+0000","last_undegraded":"2024-09-16T07:36:03.843162+0000","last_fullsized":"2024-09-16T07:36:03.843162+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.841267+0000","last_change":"2024-09-16T07:36:03.841267+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.841267+0000","last_undegraded":"2024-09-16T07:36:03.841267+0000","last_fullsized":"2024-09-16T07:36:03.841267+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842354+0000","last_change":"2024-09-16T07:36:03.842354+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842354+0000","last_undegraded":"2024-09-16T07:36:03.842354+0000","last_fullsized":"2024-09-16T07:36:03.842354+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847905+0000","last_change":"2024-09-16T07:36:03.847905+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847905+0000","last_undegraded":"2024-09-16T07:36:03.847905+0000","last_fullsized":"2024-09-16T07:36:03.847905+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.851248+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.850709+0000","last_became_peered":"2024-09-16T07:36:04.850709+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:07.411898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855892+0000","last_change":"2024-09-16T07:36:04.854097+0000","last_active":"2024-09-16T07:36:05.855892+0000","last_peered":"2024-09-16T07:36:05.855892+0000","last_clean":"2024-09-16T07:36:05.855892+0000","last_became_active":"2024-09-16T07:36:04.853827+0000","last_became_peered":"2024-09-16T07:36:04.853827+0000","last_unstale":"2024-09-16T07:36:05.855892+0000","last_undegraded":"2024-09-16T07:36:05.855892+0000","last_fullsized":"2024-09-16T07:36:05.855892+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:21.875425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855153+0000","last_change":"2024-09-16T07:36:04.848716+0000","last_active":"2024-09-16T07:36:05.855153+0000","last_peered":"2024-09-16T07:36:05.855153+0000","last_clean":"2024-09-16T07:36:05.855153+0000","last_became_active":"2024-09-16T07:36:04.848457+0000","last_became_peered":"2024-09-16T07:36:04.848457+0000","last_unstale":"2024-09-16T07:36:05.855153+0000","last_undegraded":"2024-09-16T07:36:05.855153+0000","last_fullsized":"2024-09-16T07:36:05.855153+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:21.475235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856556+0000","last_change":"2024-09-16T07:36:04.859096+0000","last_active":"2024-09-16T07:36:05.856556+0000","last_peered":"2024-09-16T07:36:05.856556+0000","last_clean":"2024-09-16T07:36:05.856556+0000","last_became_active":"2024-09-16T07:36:04.858812+0000","last_became_peered":"2024-09-16T07:36:04.858812+0000","last_unstale":"2024-09-16T07:36:05.856556+0000","last_undegraded":"2024-09-16T07:36:05.856556+0000","last_fullsized":"2024-09-16T07:36:05.856556+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:59.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855973+0000","last_change":"2024-09-16T07:36:04.862176+0000","last_active":"2024-09-16T07:36:05.855973+0000","last_peered":"2024-09-16T07:36:05.855973+0000","last_clean":"2024-09-16T07:36:05.855973+0000","last_became_active":"2024-09-16T07:36:04.860550+0000","last_became_peered":"2024-09-16T07:36:04.860550+0000","last_unstale":"2024-09-16T07:36:05.855973+0000","last_undegraded":"2024-09-16T07:36:05.855973+0000","last_fullsized":"2024-09-16T07:36:05.855973+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:10.040307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854370+0000","last_change":"2024-09-16T07:36:04.855363+0000","last_active":"2024-09-16T07:36:05.854370+0000","last_peered":"2024-09-16T07:36:05.854370+0000","last_clean":"2024-09-16T07:36:05.854370+0000","last_became_active":"2024-09-16T07:36:04.855013+0000","last_became_peered":"2024-09-16T07:36:04.855013+0000","last_unstale":"2024-09-16T07:36:05.854370+0000","last_undegraded":"2024-09-16T07:36:05.854370+0000","last_fullsized":"2024-09-16T07:36:05.854370+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:13.040194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.846503+0000","last_change":"2024-09-16T07:36:03.846503+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.846503+0000","last_undegraded":"2024-09-16T07:36:03.846503+0000","last_fullsized":"2024-09-16T07:36:03.846503+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855025+0000","last_change":"2024-09-16T07:36:04.862436+0000","last_active":"2024-09-16T07:36:05.855025+0000","last_peered":"2024-09-16T07:36:05.855025+0000","last_clean":"2024-09-16T07:36:05.855025+0000","last_became_active":"2024-09-16T07:36:04.862125+0000","last_became_peered":"2024-09-16T07:36:04.862125+0000","last_unstale":"2024-09-16T07:36:05.855025+0000","last_undegraded":"2024-09-16T07:36:05.855025+0000","last_fullsized":"2024-09-16T07:36:05.855025+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:38.787658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063678+0000","last_change":"2024-09-16T07:36:04.857741+0000","last_active":"2024-09-16T07:36:06.063678+0000","last_peered":"2024-09-16T07:36:06.063678+0000","last_clean":"2024-09-16T07:36:06.063678+0000","last_became_active":"2024-09-16T07:36:04.854444+0000","last_became_peered":"2024-09-16T07:36:04.854444+0000","last_unstale":"2024-09-16T07:36:06.063678+0000","last_undegraded":"2024-09-16T07:36:06.063678+0000","last_fullsized":"2024-09-16T07:36:06.063678+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:15.100689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856475+0000","last_change":"2024-09-16T07:36:04.869720+0000","last_active":"2024-09-16T07:36:05.856475+0000","last_peered":"2024-09-16T07:36:05.856475+0000","last_clean":"2024-09-16T07:36:05.856475+0000","last_became_active":"2024-09-16T07:36:04.868956+0000","last_became_peered":"2024-09-16T07:36:04.868956+0000","last_unstale":"2024-09-16T07:36:05.856475+0000","last_undegraded":"2024-09-16T07:36:05.856475+0000","last_fullsized":"2024-09-16T07:36:05.856475+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:29.852710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063836+0000","last_change":"2024-09-16T07:36:04.845962+0000","last_active":"2024-09-16T07:36:06.063836+0000","last_peered":"2024-09-16T07:36:06.063836+0000","last_clean":"2024-09-16T07:36:06.063836+0000","last_became_active":"2024-09-16T07:36:04.845664+0000","last_became_peered":"2024-09-16T07:36:04.845664+0000","last_unstale":"2024-09-16T07:36:06.063836+0000","last_undegraded":"2024-09-16T07:36:06.063836+0000","last_fullsized":"2024-09-16T07:36:06.063836+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:32.167063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837882+0000","last_change":"2024-09-16T07:36:03.837882+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837882+0000","last_undegraded":"2024-09-16T07:36:03.837882+0000","last_fullsized":"2024-09-16T07:36:03.837882+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854174+0000","last_change":"2024-09-16T07:36:04.865540+0000","last_active":"2024-09-16T07:36:05.854174+0000","last_peered":"2024-09-16T07:36:05.854174+0000","last_clean":"2024-09-16T07:36:05.854174+0000","last_became_active":"2024-09-16T07:36:04.865232+0000","last_became_peered":"2024-09-16T07:36:04.865232+0000","last_unstale":"2024-09-16T07:36:05.854174+0000","last_undegraded":"2024-09-16T07:36:05.854174+0000","last_fullsized":"2024-09-16T07:36:05.854174+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:24.118557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855774+0000","last_change":"2024-09-16T07:36:04.857205+0000","last_active":"2024-09-16T07:36:05.855774+0000","last_peered":"2024-09-16T07:36:05.855774+0000","last_clean":"2024-09-16T07:36:05.855774+0000","last_became_active":"2024-09-16T07:36:04.856823+0000","last_became_peered":"2024-09-16T07:36:04.856823+0000","last_unstale":"2024-09-16T07:36:05.855774+0000","last_undegraded":"2024-09-16T07:36:05.855774+0000","last_fullsized":"2024-09-16T07:36:05.855774+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:16.355699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064085+0000","last_change":"2024-09-16T07:36:04.850897+0000","last_active":"2024-09-16T07:36:06.064085+0000","last_peered":"2024-09-16T07:36:06.064085+0000","last_clean":"2024-09-16T07:36:06.064085+0000","last_became_active":"2024-09-16T07:36:04.850635+0000","last_became_peered":"2024-09-16T07:36:04.850635+0000","last_unstale":"2024-09-16T07:36:06.064085+0000","last_undegraded":"2024-09-16T07:36:06.064085+0000","last_fullsized":"2024-09-16T07:36:06.064085+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:38.194427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857379+0000","last_change":"2024-09-16T07:36:04.855205+0000","last_active":"2024-09-16T07:36:05.857379+0000","last_peered":"2024-09-16T07:36:05.857379+0000","last_clean":"2024-09-16T07:36:05.857379+0000","last_became_active":"2024-09-16T07:36:04.854356+0000","last_became_peered":"2024-09-16T07:36:04.854356+0000","last_unstale":"2024-09-16T07:36:05.857379+0000","last_undegraded":"2024-09-16T07:36:05.857379+0000","last_fullsized":"2024-09-16T07:36:05.857379+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:23.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837934+0000","last_change":"2024-09-16T07:36:03.837934+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837934+0000","last_undegraded":"2024-09-16T07:36:03.837934+0000","last_fullsized":"2024-09-16T07:36:03.837934+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063430+0000","last_change":"2024-09-16T07:36:04.849621+0000","last_active":"2024-09-16T07:36:06.063430+0000","last_peered":"2024-09-16T07:36:06.063430+0000","last_clean":"2024-09-16T07:36:06.063430+0000","last_became_active":"2024-09-16T07:36:04.849325+0000","last_became_peered":"2024-09-16T07:36:04.849325+0000","last_unstale":"2024-09-16T07:36:06.063430+0000","last_undegraded":"2024-09-16T07:36:06.063430+0000","last_fullsized":"2024-09-16T07:36:06.063430+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:20.048326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063880+0000","last_change":"2024-09-16T07:36:04.840820+0000","last_active":"2024-09-16T07:36:06.063880+0000","last_peered":"2024-09-16T07:36:06.063880+0000","last_clean":"2024-09-16T07:36:06.063880+0000","last_became_active":"2024-09-16T07:36:04.839397+0000","last_became_peered":"2024-09-16T07:36:04.839397+0000","last_unstale":"2024-09-16T07:36:06.063880+0000","last_undegraded":"2024-09-16T07:36:06.063880+0000","last_fullsized":"2024-09-16T07:36:06.063880+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:01.349093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837391+0000","last_change":"2024-09-16T07:36:03.837391+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837391+0000","last_undegraded":"2024-09-16T07:36:03.837391+0000","last_fullsized":"2024-09-16T07:36:03.837391+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842895+0000","last_change":"2024-09-16T07:36:03.842895+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842895+0000","last_undegraded":"2024-09-16T07:36:03.842895+0000","last_fullsized":"2024-09-16T07:36:03.842895+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857460+0000","last_change":"2024-09-16T07:36:04.859595+0000","last_active":"2024-09-16T07:36:05.857460+0000","last_peered":"2024-09-16T07:36:05.857460+0000","last_clean":"2024-09-16T07:36:05.857460+0000","last_became_active":"2024-09-16T07:36:04.856512+0000","last_became_peered":"2024-09-16T07:36:04.856512+0000","last_unstale":"2024-09-16T07:36:05.857460+0000","last_undegraded":"2024-09-16T07:36:05.857460+0000","last_fullsized":"2024-09-16T07:36:05.857460+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:25.701980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858752+0000","last_change":"2024-09-16T07:36:03.855477+0000","last_active":"2024-09-16T07:36:03.858752+0000","last_peered":"2024-09-16T07:36:03.858752+0000","last_clean":"2024-09-16T07:36:03.858752+0000","last_became_active":"2024-09-16T07:36:03.852255+0000","last_became_peered":"2024-09-16T07:36:03.852255+0000","last_unstale":"2024-09-16T07:36:03.858752+0000","last_undegraded":"2024-09-16T07:36:03.858752+0000","last_fullsized":"2024-09-16T07:36:03.858752+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:58.216679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.847493+0000","last_change":"2024-09-16T07:36:03.847493+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.847493+0000","last_undegraded":"2024-09-16T07:36:03.847493+0000","last_fullsized":"2024-09-16T07:36:03.847493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.855023+0000","last_change":"2024-09-16T07:36:03.853595+0000","last_active":"2024-09-16T07:36:03.855023+0000","last_peered":"2024-09-16T07:36:03.855023+0000","last_clean":"2024-09-16T07:36:03.855023+0000","last_became_active":"2024-09-16T07:36:03.853315+0000","last_became_peered":"2024-09-16T07:36:03.853315+0000","last_unstale":"2024-09-16T07:36:03.855023+0000","last_undegraded":"2024-09-16T07:36:03.855023+0000","last_fullsized":"2024-09-16T07:36:03.855023+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:09.161606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.845779+0000","last_change":"2024-09-16T07:36:03.845779+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.845779+0000","last_undegraded":"2024-09-16T07:36:03.845779+0000","last_fullsized":"2024-09-16T07:36:03.845779+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856585+0000","last_change":"2024-09-16T07:36:03.849019+0000","last_active":"2024-09-16T07:36:05.856585+0000","last_peered":"2024-09-16T07:36:05.856585+0000","last_clean":"2024-09-16T07:36:05.856585+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.856585+0000","last_undegraded":"2024-09-16T07:36:05.856585+0000","last_fullsized":"2024-09-16T07:36:05.856585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:53.651818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063690+0000","last_change":"2024-09-16T07:36:03.853178+0000","last_active":"2024-09-16T07:36:06.063690+0000","last_peered":"2024-09-16T07:36:06.063690+0000","last_clean":"2024-09-16T07:36:06.063690+0000","last_became_active":"2024-09-16T07:36:03.852914+0000","last_became_peered":"2024-09-16T07:36:03.852914+0000","last_unstale":"2024-09-16T07:36:06.063690+0000","last_undegraded":"2024-09-16T07:36:06.063690+0000","last_fullsized":"2024-09-16T07:36:06.063690+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.868825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855929+0000","last_change":"2024-09-16T07:36:04.860375+0000","last_active":"2024-09-16T07:36:05.855929+0000","last_peered":"2024-09-16T07:36:05.855929+0000","last_clean":"2024-09-16T07:36:05.855929+0000","last_became_active":"2024-09-16T07:36:04.860110+0000","last_became_peered":"2024-09-16T07:36:04.860110+0000","last_unstale":"2024-09-16T07:36:05.855929+0000","last_undegraded":"2024-09-16T07:36:05.855929+0000","last_fullsized":"2024-09-16T07:36:05.855929+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:44.383231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063342+0000","last_change":"2024-09-16T07:36:03.848993+0000","last_active":"2024-09-16T07:36:06.063342+0000","last_peered":"2024-09-16T07:36:06.063342+0000","last_clean":"2024-09-16T07:36:06.063342+0000","last_became_active":"2024-09-16T07:36:03.848700+0000","last_became_peered":"2024-09-16T07:36:03.848700+0000","last_unstale":"2024-09-16T07:36:06.063342+0000","last_undegraded":"2024-09-16T07:36:06.063342+0000","last_fullsized":"2024-09-16T07:36:06.063342+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:54.220868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856372+0000","last_change":"2024-09-16T07:36:03.855637+0000","last_active":"2024-09-16T07:36:05.856372+0000","last_peered":"2024-09-16T07:36:05.856372+0000","last_clean":"2024-09-16T07:36:05.856372+0000","last_became_active":"2024-09-16T07:36:03.855357+0000","last_became_peered":"2024-09-16T07:36:03.855357+0000","last_unstale":"2024-09-16T07:36:05.856372+0000","last_undegraded":"2024-09-16T07:36:05.856372+0000","last_fullsized":"2024-09-16T07:36:05.856372+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.392485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858894+0000","last_change":"2024-09-16T07:36:03.858168+0000","last_active":"2024-09-16T07:36:03.858894+0000","last_peered":"2024-09-16T07:36:03.858894+0000","last_clean":"2024-09-16T07:36:03.858894+0000","last_became_active":"2024-09-16T07:36:03.857697+0000","last_became_peered":"2024-09-16T07:36:03.857697+0000","last_unstale":"2024-09-16T07:36:03.858894+0000","last_undegraded":"2024-09-16T07:36:03.858894+0000","last_fullsized":"2024-09-16T07:36:03.858894+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:36.388550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063343+0000","last_change":"2024-09-16T07:36:04.853058+0000","last_active":"2024-09-16T07:36:06.063343+0000","last_peered":"2024-09-16T07:36:06.063343+0000","last_clean":"2024-09-16T07:36:06.063343+0000","last_became_active":"2024-09-16T07:36:04.852535+0000","last_became_peered":"2024-09-16T07:36:04.852535+0000","last_unstale":"2024-09-16T07:36:06.063343+0000","last_undegraded":"2024-09-16T07:36:06.063343+0000","last_fullsized":"2024-09-16T07:36:06.063343+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:36.784633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063356+0000","last_change":"2024-09-16T07:36:03.846732+0000","last_active":"2024-09-16T07:36:06.063356+0000","last_peered":"2024-09-16T07:36:06.063356+0000","last_clean":"2024-09-16T07:36:06.063356+0000","last_became_active":"2024-09-16T07:36:03.845789+0000","last_became_peered":"2024-09-16T07:36:03.845789+0000","last_unstale":"2024-09-16T07:36:06.063356+0000","last_undegraded":"2024-09-16T07:36:06.063356+0000","last_fullsized":"2024-09-16T07:36:06.063356+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:53.374127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064066+0000","last_change":"2024-09-16T07:36:04.852160+0000","last_active":"2024-09-16T07:36:06.064066+0000","last_peered":"2024-09-16T07:36:06.064066+0000","last_clean":"2024-09-16T07:36:06.064066+0000","last_became_active":"2024-09-16T07:36:04.851903+0000","last_became_peered":"2024-09-16T07:36:04.851903+0000","last_unstale":"2024-09-16T07:36:06.064066+0000","last_undegraded":"2024-09-16T07:36:06.064066+0000","last_fullsized":"2024-09-16T07:36:06.064066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:32.387978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.832128+0000","last_change":"2024-09-16T07:36:02.832128+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.832128+0000","last_undegraded":"2024-09-16T07:36:02.832128+0000","last_fullsized":"2024-09-16T07:36:02.832128+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842513+0000","last_change":"2024-09-16T07:36:03.842513+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842513+0000","last_undegraded":"2024-09-16T07:36:03.842513+0000","last_fullsized":"2024-09-16T07:36:03.842513+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855092+0000","last_change":"2024-09-16T07:36:03.852706+0000","last_active":"2024-09-16T07:36:05.855092+0000","last_peered":"2024-09-16T07:36:05.855092+0000","last_clean":"2024-09-16T07:36:05.855092+0000","last_became_active":"2024-09-16T07:36:03.852295+0000","last_became_peered":"2024-09-16T07:36:03.852295+0000","last_unstale":"2024-09-16T07:36:05.855092+0000","last_undegraded":"2024-09-16T07:36:05.855092+0000","last_fullsized":"2024-09-16T07:36:05.855092+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:51.417749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.859083+0000","last_change":"2024-09-16T07:36:03.857233+0000","last_active":"2024-09-16T07:36:03.859083+0000","last_peered":"2024-09-16T07:36:03.859083+0000","last_clean":"2024-09-16T07:36:03.859083+0000","last_became_active":"2024-09-16T07:36:03.856817+0000","last_became_peered":"2024-09-16T07:36:03.856817+0000","last_unstale":"2024-09-16T07:36:03.859083+0000","last_undegraded":"2024-09-16T07:36:03.859083+0000","last_fullsized":"2024-09-16T07:36:03.859083+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:45.111727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837921+0000","last_change":"2024-09-16T07:36:03.837921+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837921+0000","last_undegraded":"2024-09-16T07:36:03.837921+0000","last_fullsized":"2024-09-16T07:36:03.837921+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.854803+0000","last_change":"2024-09-16T07:36:03.851395+0000","last_active":"2024-09-16T07:36:03.854803+0000","last_peered":"2024-09-16T07:36:03.854803+0000","last_clean":"2024-09-16T07:36:03.854803+0000","last_became_active":"2024-09-16T07:36:03.851127+0000","last_became_peered":"2024-09-16T07:36:03.851127+0000","last_unstale":"2024-09-16T07:36:03.854803+0000","last_undegraded":"2024-09-16T07:36:03.854803+0000","last_fullsized":"2024-09-16T07:36:03.854803+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:50.717571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856941+0000","last_change":"2024-09-16T07:36:04.863490+0000","last_active":"2024-09-16T07:36:05.856941+0000","last_peered":"2024-09-16T07:36:05.856941+0000","last_clean":"2024-09-16T07:36:05.856941+0000","last_became_active":"2024-09-16T07:36:04.862482+0000","last_became_peered":"2024-09-16T07:36:04.862482+0000","last_unstale":"2024-09-16T07:36:05.856941+0000","last_undegraded":"2024-09-16T07:36:05.856941+0000","last_fullsized":"2024-09-16T07:36:05.856941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:18.732185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856621+0000","last_change":"2024-09-16T07:36:03.852858+0000","last_active":"2024-09-16T07:36:05.856621+0000","last_peered":"2024-09-16T07:36:05.856621+0000","last_clean":"2024-09-16T07:36:05.856621+0000","last_became_active":"2024-09-16T07:36:03.852596+0000","last_became_peered":"2024-09-16T07:36:03.852596+0000","last_unstale":"2024-09-16T07:36:05.856621+0000","last_undegraded":"2024-09-16T07:36:05.856621+0000","last_fullsized":"2024-09-16T07:36:05.856621+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:29.307382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854662+0000","last_change":"2024-09-16T07:36:04.855631+0000","last_active":"2024-09-16T07:36:05.854662+0000","last_peered":"2024-09-16T07:36:05.854662+0000","last_clean":"2024-09-16T07:36:05.854662+0000","last_became_active":"2024-09-16T07:36:04.855341+0000","last_became_peered":"2024-09-16T07:36:04.855341+0000","last_unstale":"2024-09-16T07:36:05.854662+0000","last_undegraded":"2024-09-16T07:36:05.854662+0000","last_fullsized":"2024-09-16T07:36:05.854662+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:06.127741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854531+0000","last_change":"2024-09-16T07:36:03.854082+0000","last_active":"2024-09-16T07:36:05.854531+0000","last_peered":"2024-09-16T07:36:05.854531+0000","last_clean":"2024-09-16T07:36:05.854531+0000","last_became_active":"2024-09-16T07:36:03.853794+0000","last_became_peered":"2024-09-16T07:36:03.853794+0000","last_unstale":"2024-09-16T07:36:05.854531+0000","last_undegraded":"2024-09-16T07:36:05.854531+0000","last_fullsized":"2024-09-16T07:36:05.854531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:10.982614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.830540+0000","last_change":"2024-09-16T07:36:02.830540+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.830540+0000","last_undegraded":"2024-09-16T07:36:02.830540+0000","last_fullsized":"2024-09-16T07:36:02.830540+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.843486+0000","last_change":"2024-09-16T07:36:03.843486+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.843486+0000","last_undegraded":"2024-09-16T07:36:03.843486+0000","last_fullsized":"2024-09-16T07:36:03.843486+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.828755+0000","last_change":"2024-09-16T07:36:02.828755+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.828755+0000","last_undegraded":"2024-09-16T07:36:02.828755+0000","last_fullsized":"2024-09-16T07:36:02.828755+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.842116+0000","last_change":"2024-09-16T07:36:03.842116+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.842116+0000","last_undegraded":"2024-09-16T07:36:03.842116+0000","last_fullsized":"2024-09-16T07:36:03.842116+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856205+0000","last_change":"2024-09-16T07:36:03.850133+0000","last_active":"2024-09-16T07:36:05.856205+0000","last_peered":"2024-09-16T07:36:05.856205+0000","last_clean":"2024-09-16T07:36:05.856205+0000","last_became_active":"2024-09-16T07:36:03.849070+0000","last_became_peered":"2024-09-16T07:36:03.849070+0000","last_unstale":"2024-09-16T07:36:05.856205+0000","last_undegraded":"2024-09-16T07:36:05.856205+0000","last_fullsized":"2024-09-16T07:36:05.856205+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:35.166163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063646+0000","last_change":"2024-09-16T07:36:04.848685+0000","last_active":"2024-09-16T07:36:06.063646+0000","last_peered":"2024-09-16T07:36:06.063646+0000","last_clean":"2024-09-16T07:36:06.063646+0000","last_became_active":"2024-09-16T07:36:04.844479+0000","last_became_peered":"2024-09-16T07:36:04.844479+0000","last_unstale":"2024-09-16T07:36:06.063646+0000","last_undegraded":"2024-09-16T07:36:06.063646+0000","last_fullsized":"2024-09-16T07:36:06.063646+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:36.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.857074+0000","last_change":"2024-09-16T07:36:03.855122+0000","last_active":"2024-09-16T07:36:03.857074+0000","last_peered":"2024-09-16T07:36:03.857074+0000","last_clean":"2024-09-16T07:36:03.857074+0000","last_became_active":"2024-09-16T07:36:03.854786+0000","last_became_peered":"2024-09-16T07:36:03.854786+0000","last_unstale":"2024-09-16T07:36:03.857074+0000","last_undegraded":"2024-09-16T07:36:03.857074+0000","last_fullsized":"2024-09-16T07:36:03.857074+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:02.212316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063917+0000","last_change":"2024-09-16T07:36:04.853245+0000","last_active":"2024-09-16T07:36:06.063917+0000","last_peered":"2024-09-16T07:36:06.063917+0000","last_clean":"2024-09-16T07:36:06.063917+0000","last_became_active":"2024-09-16T07:36:04.852895+0000","last_became_peered":"2024-09-16T07:36:04.852895+0000","last_unstale":"2024-09-16T07:36:06.063917+0000","last_undegraded":"2024-09-16T07:36:06.063917+0000","last_fullsized":"2024-09-16T07:36:06.063917+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.199780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858884+0000","last_change":"2024-09-16T07:36:03.856130+0000","last_active":"2024-09-16T07:36:03.858884+0000","last_peered":"2024-09-16T07:36:03.858884+0000","last_clean":"2024-09-16T07:36:03.858884+0000","last_became_active":"2024-09-16T07:36:03.852405+0000","last_became_peered":"2024-09-16T07:36:03.852405+0000","last_unstale":"2024-09-16T07:36:03.858884+0000","last_undegraded":"2024-09-16T07:36:03.858884+0000","last_fullsized":"2024-09-16T07:36:03.858884+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:51.417749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064061+0000","last_change":"2024-09-16T07:36:04.847598+0000","last_active":"2024-09-16T07:36:06.064061+0000","last_peered":"2024-09-16T07:36:06.064061+0000","last_clean":"2024-09-16T07:36:06.064061+0000","last_became_active":"2024-09-16T07:36:04.847063+0000","last_became_peered":"2024-09-16T07:36:04.847063+0000","last_unstale":"2024-09-16T07:36:06.064061+0000","last_undegraded":"2024-09-16T07:36:06.064061+0000","last_fullsized":"2024-09-16T07:36:06.064061+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:50.838203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.858292+0000","last_change":"2024-09-16T07:36:03.856847+0000","last_active":"2024-09-16T07:36:03.858292+0000","last_peered":"2024-09-16T07:36:03.858292+0000","last_clean":"2024-09-16T07:36:03.858292+0000","last_became_active":"2024-09-16T07:36:03.856321+0000","last_became_peered":"2024-09-16T07:36:03.856321+0000","last_unstale":"2024-09-16T07:36:03.858292+0000","last_undegraded":"2024-09-16T07:36:03.858292+0000","last_fullsized":"2024-09-16T07:36:03.858292+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:57.138229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.837398+0000","last_change":"2024-09-16T07:36:03.837398+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.837398+0000","last_undegraded":"2024-09-16T07:36:03.837398+0000","last_fullsized":"2024-09-16T07:36:03.837398+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856080+0000","last_change":"2024-09-16T07:36:03.855412+0000","last_active":"2024-09-16T07:36:05.856080+0000","last_peered":"2024-09-16T07:36:05.856080+0000","last_clean":"2024-09-16T07:36:05.856080+0000","last_became_active":"2024-09-16T07:36:03.854933+0000","last_became_peered":"2024-09-16T07:36:03.854933+0000","last_unstale":"2024-09-16T07:36:05.856080+0000","last_undegraded":"2024-09-16T07:36:05.856080+0000","last_fullsized":"2024-09-16T07:36:05.856080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:58.216679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854581+0000","last_change":"2024-09-16T07:36:04.850083+0000","last_active":"2024-09-16T07:36:05.854581+0000","last_peered":"2024-09-16T07:36:05.854581+0000","last_clean":"2024-09-16T07:36:05.854581+0000","last_became_active":"2024-09-16T07:36:04.849731+0000","last_became_peered":"2024-09-16T07:36:04.849731+0000","last_unstale":"2024-09-16T07:36:05.854581+0000","last_undegraded":"2024-09-16T07:36:05.854581+0000","last_fullsized":"2024-09-16T07:36:05.854581+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:08.133005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855297+0000","last_change":"2024-09-16T07:36:03.848269+0000","last_active":"2024-09-16T07:36:05.855297+0000","last_peered":"2024-09-16T07:36:05.855297+0000","last_clean":"2024-09-16T07:36:05.855297+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.855297+0000","last_undegraded":"2024-09-16T07:36:05.855297+0000","last_fullsized":"2024-09-16T07:36:05.855297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:06.554515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856499+0000","last_change":"2024-09-16T07:36:03.847015+0000","last_active":"2024-09-16T07:36:05.856499+0000","last_peered":"2024-09-16T07:36:05.856499+0000","last_clean":"2024-09-16T07:36:05.856499+0000","last_became_active":"2024-09-16T07:36:03.846719+0000","last_became_peered":"2024-09-16T07:36:03.846719+0000","last_unstale":"2024-09-16T07:36:05.856499+0000","last_undegraded":"2024-09-16T07:36:05.856499+0000","last_fullsized":"2024-09-16T07:36:05.856499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:46.177351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.838152+0000","last_change":"2024-09-16T07:36:03.838152+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.838152+0000","last_undegraded":"2024-09-16T07:36:03.838152+0000","last_fullsized":"2024-09-16T07:36:03.838152+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856245+0000","last_change":"2024-09-16T07:36:03.854810+0000","last_active":"2024-09-16T07:36:05.856245+0000","last_peered":"2024-09-16T07:36:05.856245+0000","last_clean":"2024-09-16T07:36:05.856245+0000","last_became_active":"2024-09-16T07:36:03.854533+0000","last_became_peered":"2024-09-16T07:36:03.854533+0000","last_unstale":"2024-09-16T07:36:05.856245+0000","last_undegraded":"2024-09-16T07:36:05.856245+0000","last_fullsized":"2024-09-16T07:36:05.856245+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:23.851837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063875+0000","last_change":"2024-09-16T07:36:04.855496+0000","last_active":"2024-09-16T07:36:06.063875+0000","last_peered":"2024-09-16T07:36:06.063875+0000","last_clean":"2024-09-16T07:36:06.063875+0000","last_became_active":"2024-09-16T07:36:04.855167+0000","last_became_peered":"2024-09-16T07:36:04.855167+0000","last_unstale":"2024-09-16T07:36:06.063875+0000","last_undegraded":"2024-09-16T07:36:06.063875+0000","last_fullsized":"2024-09-16T07:36:06.063875+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:24.513277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.859106+0000","last_change":"2024-09-16T07:36:03.857828+0000","last_active":"2024-09-16T07:36:03.859106+0000","last_peered":"2024-09-16T07:36:03.859106+0000","last_clean":"2024-09-16T07:36:03.859106+0000","last_became_active":"2024-09-16T07:36:03.853109+0000","last_became_peered":"2024-09-16T07:36:03.853109+0000","last_unstale":"2024-09-16T07:36:03.859106+0000","last_undegraded":"2024-09-16T07:36:03.859106+0000","last_fullsized":"2024-09-16T07:36:03.859106+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:04.946096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857004+0000","last_change":"2024-09-16T07:36:04.852909+0000","last_active":"2024-09-16T07:36:05.857004+0000","last_peered":"2024-09-16T07:36:05.857004+0000","last_clean":"2024-09-16T07:36:05.857004+0000","last_became_active":"2024-09-16T07:36:04.852496+0000","last_became_peered":"2024-09-16T07:36:04.852496+0000","last_unstale":"2024-09-16T07:36:05.857004+0000","last_undegraded":"2024-09-16T07:36:05.857004+0000","last_fullsized":"2024-09-16T07:36:05.857004+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:43.843163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854718+0000","last_change":"2024-09-16T07:36:03.853605+0000","last_active":"2024-09-16T07:36:05.854718+0000","last_peered":"2024-09-16T07:36:05.854718+0000","last_clean":"2024-09-16T07:36:05.854718+0000","last_became_active":"2024-09-16T07:36:03.853349+0000","last_became_peered":"2024-09-16T07:36:03.853349+0000","last_unstale":"2024-09-16T07:36:05.854718+0000","last_undegraded":"2024-09-16T07:36:05.854718+0000","last_fullsized":"2024-09-16T07:36:05.854718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:29.592986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.846573+0000","last_change":"2024-09-16T07:36:03.846573+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.846573+0000","last_undegraded":"2024-09-16T07:36:03.846573+0000","last_fullsized":"2024-09-16T07:36:03.846573+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857065+0000","last_change":"2024-09-16T07:36:03.851730+0000","last_active":"2024-09-16T07:36:05.857065+0000","last_peered":"2024-09-16T07:36:05.857065+0000","last_clean":"2024-09-16T07:36:05.857065+0000","last_became_active":"2024-09-16T07:36:03.851457+0000","last_became_peered":"2024-09-16T07:36:03.851457+0000","last_unstale":"2024-09-16T07:36:05.857065+0000","last_undegraded":"2024-09-16T07:36:05.857065+0000","last_fullsized":"2024-09-16T07:36:05.857065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:37.222531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063468+0000","last_change":"2024-09-16T07:36:03.857845+0000","last_active":"2024-09-16T07:36:06.063468+0000","last_peered":"2024-09-16T07:36:06.063468+0000","last_clean":"2024-09-16T07:36:06.063468+0000","last_became_active":"2024-09-16T07:36:03.857366+0000","last_became_peered":"2024-09-16T07:36:03.857366+0000","last_unstale":"2024-09-16T07:36:06.063468+0000","last_undegraded":"2024-09-16T07:36:06.063468+0000","last_fullsized":"2024-09-16T07:36:06.063468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:36.413668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T07:36:03.818979+0000","last_change":"2024-09-16T07:36:03.818979+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.818979+0000","last_undegraded":"2024-09-16T07:36:03.818979+0000","last_fullsized":"2024-09-16T07:36:03.818979+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.830304+0000","last_change":"2024-09-16T07:36:02.830304+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.830304+0000","last_undegraded":"2024-09-16T07:36:02.830304+0000","last_fullsized":"2024-09-16T07:36:02.830304+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855281+0000","last_change":"2024-09-16T07:36:04.863145+0000","last_active":"2024-09-16T07:36:05.855281+0000","last_peered":"2024-09-16T07:36:05.855281+0000","last_clean":"2024-09-16T07:36:05.855281+0000","last_became_active":"2024-09-16T07:36:04.862893+0000","last_became_peered":"2024-09-16T07:36:04.862893+0000","last_unstale":"2024-09-16T07:36:05.855281+0000","last_undegraded":"2024-09-16T07:36:05.855281+0000","last_fullsized":"2024-09-16T07:36:05.855281+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:26.614434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854639+0000","last_change":"2024-09-16T07:36:03.851573+0000","last_active":"2024-09-16T07:36:05.854639+0000","last_peered":"2024-09-16T07:36:05.854639+0000","last_clean":"2024-09-16T07:36:05.854639+0000","last_became_active":"2024-09-16T07:36:03.851286+0000","last_became_peered":"2024-09-16T07:36:03.851286+0000","last_unstale":"2024-09-16T07:36:05.854639+0000","last_undegraded":"2024-09-16T07:36:05.854639+0000","last_fullsized":"2024-09-16T07:36:05.854639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.290737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856129+0000","last_change":"2024-09-16T07:36:04.858119+0000","last_active":"2024-09-16T07:36:05.856129+0000","last_peered":"2024-09-16T07:36:05.856129+0000","last_clean":"2024-09-16T07:36:05.856129+0000","last_became_active":"2024-09-16T07:36:04.855972+0000","last_became_peered":"2024-09-16T07:36:04.855972+0000","last_unstale":"2024-09-16T07:36:05.856129+0000","last_undegraded":"2024-09-16T07:36:05.856129+0000","last_fullsized":"2024-09-16T07:36:05.856129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:40.599143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854647+0000","last_change":"2024-09-16T07:36:03.854300+0000","last_active":"2024-09-16T07:36:05.854647+0000","last_peered":"2024-09-16T07:36:05.854647+0000","last_clean":"2024-09-16T07:36:05.854647+0000","last_became_active":"2024-09-16T07:36:03.853971+0000","last_became_peered":"2024-09-16T07:36:03.853971+0000","last_unstale":"2024-09-16T07:36:05.854647+0000","last_undegraded":"2024-09-16T07:36:05.854647+0000","last_fullsized":"2024-09-16T07:36:05.854647+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:25.287389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T07:36:03.844781+0000","last_change":"2024-09-16T07:36:03.844781+0000","last_active":"2024-09-16T07:36:03.818979+0000","last_peered":"2024-09-16T07:36:03.818979+0000","last_clean":"2024-09-16T07:36:03.818979+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:03.844781+0000","last_undegraded":"2024-09-16T07:36:03.844781+0000","last_fullsized":"2024-09-16T07:36:03.844781+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T07:36:02.828990+0000","last_change":"2024-09-16T07:36:02.828990+0000","last_active":"2024-09-16T07:36:02.811068+0000","last_peered":"2024-09-16T07:36:02.811068+0000","last_clean":"2024-09-16T07:36:02.811068+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T07:36:02.828990+0000","last_undegraded":"2024-09-16T07:36:02.828990+0000","last_fullsized":"2024-09-16T07:36:02.828990+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":310,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T07:36:03.839687+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:36:03.839687+0000","last_peered":"2024-09-16T07:36:03.839687+0000","last_clean":"2024-09-16T07:36:03.839687+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:36:03.839687+0000","last_undegraded":"2024-09-16T07:36:03.839687+0000","last_fullsized":"2024-09-16T07:36:03.839687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476902,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131260,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240580,"statfs":{"total":96636764160,"available":96502353920,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131256,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240584,"statfs":{"total":96636764160,"available":96502358016,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127676,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244164,"statfs":{"total":96636764160,"available":96506023936,"internally_reserved":0,"allocated":2203648,"data_stored":1295491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145144,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226696,"statfs":{"total":96636764160,"available":96488136704,"internally_reserved":0,"allocated":1609728,"data_stored":705123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127140,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244700,"statfs":{"total":96636764160,"available":96506572800,"internally_reserved":0,"allocated":1650688,"data_stored":724759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509606,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159076,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212764,"statfs":{"total":96636764160,"available":96473870336,"internally_reserved":0,"allocated":1646592,"data_stored":724725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161216966},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509606,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119404,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252436,"statfs":{"total":96636764160,"available":96514494464,"internally_reserved":0,"allocated":2252800,"data_stored":1315835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542308,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135440,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236400,"statfs":{"total":96636764160,"available":96498073600,"internally_reserved":0,"allocated":1634304,"data_stored":718341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":137034181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:36:09.880 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3d is not active+clean 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.847311+0000', 'last_change': '2024-09-16T07:36:03.847311+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.847311+0000', 'last_undegraded': '2024-09-16T07:36:03.847311+0000', 'last_fullsized': '2024-09-16T07:36:03.847311+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3b is not active+clean 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.843162+0000', 'last_change': '2024-09-16T07:36:03.843162+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.843162+0000', 'last_undegraded': '2024-09-16T07:36:03.843162+0000', 'last_fullsized': '2024-09-16T07:36:03.843162+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3a is not active+clean 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:PG 19.39 is not active+clean 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.841267+0000', 'last_change': '2024-09-16T07:36:03.841267+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.841267+0000', 'last_undegraded': '2024-09-16T07:36:03.841267+0000', 'last_fullsized': '2024-09-16T07:36:03.841267+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:PG 19.38 is not active+clean 2024-09-16T07:36:09.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.842354+0000', 'last_change': '2024-09-16T07:36:03.842354+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.842354+0000', 'last_undegraded': '2024-09-16T07:36:03.842354+0000', 'last_fullsized': '2024-09-16T07:36:03.842354+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:PG 19.37 is not active+clean 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.847905+0000', 'last_change': '2024-09-16T07:36:03.847905+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.847905+0000', 'last_undegraded': '2024-09-16T07:36:03.847905+0000', 'last_fullsized': '2024-09-16T07:36:03.847905+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:PG 19.34 is not active+clean 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2f is not active+clean 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.846503+0000', 'last_change': '2024-09-16T07:36:03.846503+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.846503+0000', 'last_undegraded': '2024-09-16T07:36:03.846503+0000', 'last_fullsized': '2024-09-16T07:36:03.846503+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2b is not active+clean 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:PG 19.29 is not active+clean 2024-09-16T07:36:09.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.837882+0000', 'last_change': '2024-09-16T07:36:03.837882+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.837882+0000', 'last_undegraded': '2024-09-16T07:36:03.837882+0000', 'last_fullsized': '2024-09-16T07:36:03.837882+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:PG 19.24 is not active+clean 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.837934+0000', 'last_change': '2024-09-16T07:36:03.837934+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.837934+0000', 'last_undegraded': '2024-09-16T07:36:03.837934+0000', 'last_fullsized': '2024-09-16T07:36:03.837934+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:PG 19.21 is not active+clean 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.837391+0000', 'last_change': '2024-09-16T07:36:03.837391+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.837391+0000', 'last_undegraded': '2024-09-16T07:36:03.837391+0000', 'last_fullsized': '2024-09-16T07:36:03.837391+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:PG 19.20 is not active+clean 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.842895+0000', 'last_change': '2024-09-16T07:36:03.842895+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.842895+0000', 'last_undegraded': '2024-09-16T07:36:03.842895+0000', 'last_fullsized': '2024-09-16T07:36:03.842895+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1f is not active+clean 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.847493+0000', 'last_change': '2024-09-16T07:36:03.847493+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.847493+0000', 'last_undegraded': '2024-09-16T07:36:03.847493+0000', 'last_fullsized': '2024-09-16T07:36:03.847493+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1c is not active+clean 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.845779+0000', 'last_change': '2024-09-16T07:36:03.845779+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.845779+0000', 'last_undegraded': '2024-09-16T07:36:03.845779+0000', 'last_fullsized': '2024-09-16T07:36:03.845779+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1d is not active+clean 2024-09-16T07:36:09.883 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1b is not active+clean 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:PG 19.18 is not active+clean 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:PG 18.17 is not active+clean 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:02.832128+0000', 'last_change': '2024-09-16T07:36:02.832128+0000', 'last_active': '2024-09-16T07:36:02.811068+0000', 'last_peered': '2024-09-16T07:36:02.811068+0000', 'last_clean': '2024-09-16T07:36:02.811068+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:02.832128+0000', 'last_undegraded': '2024-09-16T07:36:02.832128+0000', 'last_fullsized': '2024-09-16T07:36:02.832128+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:PG 19.17 is not active+clean 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.842513+0000', 'last_change': '2024-09-16T07:36:03.842513+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.842513+0000', 'last_undegraded': '2024-09-16T07:36:03.842513+0000', 'last_fullsized': '2024-09-16T07:36:03.842513+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:PG 19.14 is not active+clean 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.884 INFO:tasks.ceph.ceph_manager.ceph:PG 19.15 is not active+clean 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.837921+0000', 'last_change': '2024-09-16T07:36:03.837921+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.837921+0000', 'last_undegraded': '2024-09-16T07:36:03.837921+0000', 'last_fullsized': '2024-09-16T07:36:03.837921+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:PG 19.11 is not active+clean 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:PG 18.10 is not active+clean 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:02.830540+0000', 'last_change': '2024-09-16T07:36:02.830540+0000', 'last_active': '2024-09-16T07:36:02.811068+0000', 'last_peered': '2024-09-16T07:36:02.811068+0000', 'last_clean': '2024-09-16T07:36:02.811068+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:02.830540+0000', 'last_undegraded': '2024-09-16T07:36:02.830540+0000', 'last_fullsized': '2024-09-16T07:36:02.830540+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:PG 19.e is not active+clean 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.843486+0000', 'last_change': '2024-09-16T07:36:03.843486+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.843486+0000', 'last_undegraded': '2024-09-16T07:36:03.843486+0000', 'last_fullsized': '2024-09-16T07:36:03.843486+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:PG 18.f is not active+clean 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:02.828755+0000', 'last_change': '2024-09-16T07:36:02.828755+0000', 'last_active': '2024-09-16T07:36:02.811068+0000', 'last_peered': '2024-09-16T07:36:02.811068+0000', 'last_clean': '2024-09-16T07:36:02.811068+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:02.828755+0000', 'last_undegraded': '2024-09-16T07:36:02.828755+0000', 'last_fullsized': '2024-09-16T07:36:02.828755+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:36:09.885 INFO:tasks.ceph.ceph_manager.ceph:PG 19.f is not active+clean 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.842116+0000', 'last_change': '2024-09-16T07:36:03.842116+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.842116+0000', 'last_undegraded': '2024-09-16T07:36:03.842116+0000', 'last_fullsized': '2024-09-16T07:36:03.842116+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:PG 19.b is not active+clean 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.837398+0000', 'last_change': '2024-09-16T07:36:03.837398+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.837398+0000', 'last_undegraded': '2024-09-16T07:36:03.837398+0000', 'last_fullsized': '2024-09-16T07:36:03.837398+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:PG 19.9 is not active+clean 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:PG 19.6 is not active+clean 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.838152+0000', 'last_change': '2024-09-16T07:36:03.838152+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.838152+0000', 'last_undegraded': '2024-09-16T07:36:03.838152+0000', 'last_fullsized': '2024-09-16T07:36:03.838152+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:PG 19.5 is not active+clean 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.846573+0000', 'last_change': '2024-09-16T07:36:03.846573+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.846573+0000', 'last_undegraded': '2024-09-16T07:36:03.846573+0000', 'last_fullsized': '2024-09-16T07:36:03.846573+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:36:09.886 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2 is not active+clean 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3 is not active+clean 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T07:36:03.818979+0000', 'last_change': '2024-09-16T07:36:03.818979+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.818979+0000', 'last_undegraded': '2024-09-16T07:36:03.818979+0000', 'last_fullsized': '2024-09-16T07:36:03.818979+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2 is not active+clean 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:02.830304+0000', 'last_change': '2024-09-16T07:36:02.830304+0000', 'last_active': '2024-09-16T07:36:02.811068+0000', 'last_peered': '2024-09-16T07:36:02.811068+0000', 'last_clean': '2024-09-16T07:36:02.811068+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:02.830304+0000', 'last_undegraded': '2024-09-16T07:36:02.830304+0000', 'last_fullsized': '2024-09-16T07:36:02.830304+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:PG 19.12 is not active+clean 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 131, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:03.844781+0000', 'last_change': '2024-09-16T07:36:03.844781+0000', 'last_active': '2024-09-16T07:36:03.818979+0000', 'last_peered': '2024-09-16T07:36:03.818979+0000', 'last_clean': '2024-09-16T07:36:03.818979+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:03.844781+0000', 'last_undegraded': '2024-09-16T07:36:03.844781+0000', 'last_fullsized': '2024-09-16T07:36:03.844781+0000', 'mapping_epoch': 131, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 131, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:03.818979+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:PG 18.13 is not active+clean 2024-09-16T07:36:09.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-16T07:36:02.828990+0000', 'last_change': '2024-09-16T07:36:02.828990+0000', 'last_active': '2024-09-16T07:36:02.811068+0000', 'last_peered': '2024-09-16T07:36:02.811068+0000', 'last_clean': '2024-09-16T07:36:02.811068+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T07:36:02.828990+0000', 'last_undegraded': '2024-09-16T07:36:02.828990+0000', 'last_fullsized': '2024-09-16T07:36:02.828990+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'last_clean_scrub_stamp': '2024-09-16T07:36:02.811068+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T07:36:09.888 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T07:36:10.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.071+0000 7ff2cae00640 1 -- 172.21.15.71:0/2556919433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c415bd60 msgr2=0x7ff2c40cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.071+0000 7ff2cae00640 1 --2- 172.21.15.71:0/2556919433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c415bd60 0x7ff2c40cdad0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009830 tx=0x7ff2b402a510 comp rx=0 tx=0).stop 2024-09-16T07:36:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.071+0000 7ff2cae00640 1 -- 172.21.15.71:0/2556919433 shutdown_connections 2024-09-16T07:36:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.071+0000 7ff2cae00640 1 --2- 172.21.15.71:0/2556919433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2c4074460 0x7ff2c40cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.072+0000 7ff2cae00640 1 --2- 172.21.15.71:0/2556919433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c415bd60 0x7ff2c40cdad0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.072+0000 7ff2cae00640 1 -- 172.21.15.71:0/2556919433 >> 172.21.15.71:0/2556919433 conn(0x7ff2c40b18e0 msgr2=0x7ff2c40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.072+0000 7ff2cae00640 1 -- 172.21.15.71:0/2556919433 shutdown_connections 2024-09-16T07:36:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.072+0000 7ff2cae00640 1 -- 172.21.15.71:0/2556919433 wait complete. 2024-09-16T07:36:10.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.073+0000 7ff2cae00640 1 Processor -- start 2024-09-16T07:36:10.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.073+0000 7ff2cae00640 1 -- start start 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.074+0000 7ff2cae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c406e910 0x7ff2c41f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.074+0000 7ff2cae00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2c4074460 0x7ff2c41e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.074+0000 7ff2cae00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2c415bd60 0x7ff2c4156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.074+0000 7ff2cae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c40ccc70 con 0x7ff2c406e910 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.074+0000 7ff2cae00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c40ccaf0 con 0x7ff2c415bd60 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.074+0000 7ff2cae00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c40ccdf0 con 0x7ff2c4074460 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.074+0000 7ff2c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c406e910 0x7ff2c41f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c406e910 0x7ff2c41f8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40476/0 (socket says 172.21.15.71:40476) 2024-09-16T07:36:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3146224056 learned_addr learned my addr 172.21.15.71:0/3146224056 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2c4074460 msgr2=0x7ff2c41e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3400640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2c4074460 0x7ff2c41e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c8e00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2c415bd60 0x7ff2c4156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2c4074460 0x7ff2c41e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2c415bd60 msgr2=0x7ff2c4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2c415bd60 0x7ff2c4156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3146224056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c409cdd0 con 0x7ff2c406e910 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3400640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2c4074460 0x7ff2c41e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:36:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c8e00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2c415bd60 0x7ff2c4156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:36:10.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.075+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c406e910 0x7ff2c41f8f00 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009950 tx=0x7ff2b4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:10.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.076+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3146224056 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4005c90 con 0x7ff2c406e910 2024-09-16T07:36:10.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.076+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3146224056 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2b4031e50 con 0x7ff2c406e910 2024-09-16T07:36:10.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.076+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c409d0b0 con 0x7ff2c406e910 2024-09-16T07:36:10.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.076+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3146224056 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4005260 con 0x7ff2c406e910 2024-09-16T07:36:10.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.076+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c409d670 con 0x7ff2c406e910 2024-09-16T07:36:10.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.077+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c4069360 con 0x7ff2c406e910 2024-09-16T07:36:10.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.078+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3146224056 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7ff2b4005400 con 0x7ff2c406e910 2024-09-16T07:36:10.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.079+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2a007a6b0 0x7ff2a007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:10.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.079+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3146224056 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7ff2b40ba8b0 con 0x7ff2c406e910 2024-09-16T07:36:10.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.080+0000 7ff2c3400640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2a007a6b0 0x7ff2a007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:10.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.080+0000 7ff2c3400640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2a007a6b0 0x7ff2a007cb70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0013f0 tx=0x7ff2ac009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:10.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.083+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3146224056 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff2b403f030 con 0x7ff2c406e910 2024-09-16T07:36:10.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.334+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff2c40cd380 con 0x7ff2c406e910 2024-09-16T07:36:10.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.335+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3146224056 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1377 (secure 0 0 0) 0x7ff2b4084a20 con 0x7ff2c406e910 2024-09-16T07:36:10.335 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:10.335 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":827,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1726471357,"num_in_osds":8,"osd_in_since":1726471347,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":61},{"state_name":"creating+peering","count":25},{"state_name":"unknown","count":11}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":590368,"bytes_used":1102229504,"bytes_avail":771991883776,"bytes_total":773094113280,"unknown_pgs_ratio":0.11340206116437912,"inactive_pgs_ratio":0.25773194432258606,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":85,"btime":"2024-09-16T07:36:07:854566+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":8919}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-09-16T07:35:44.620542+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2a007a6b0 msgr2=0x7ff2a007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2a007a6b0 0x7ff2a007cb70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0013f0 tx=0x7ff2ac009040 comp rx=0 tx=0).stop 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c406e910 msgr2=0x7ff2c41f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c406e910 0x7ff2c41f8f00 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009950 tx=0x7ff2b4033c20 comp rx=0 tx=0).stop 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 shutdown_connections 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2a007a6b0 0x7ff2a007cb70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2c415bd60 0x7ff2c4156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2c4074460 0x7ff2c41e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 --2- 172.21.15.71:0/3146224056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c406e910 0x7ff2c41f8f00 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 >> 172.21.15.71:0/3146224056 conn(0x7ff2c40b18e0 msgr2=0x7ff2c40b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:10.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 shutdown_connections 2024-09-16T07:36:10.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.337+0000 7ff2cae00640 1 -- 172.21.15.71:0/3146224056 wait complete. 2024-09-16T07:36:10.353 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T07:36:10.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:36:10.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.510+0000 7fa30a200640 1 -- 172.21.15.71:0/3912044834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2e8003a00 msgr2=0x7fa2e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.510+0000 7fa30a200640 1 --2- 172.21.15.71:0/3912044834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2e8003a00 0x7fa2e8003e20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c2e0 tx=0x7fa2e8005830 comp rx=0 tx=0).stop 2024-09-16T07:36:10.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.510+0000 7fa303e00640 1 -- 172.21.15.71:0/3912044834 reap_dead start 2024-09-16T07:36:10.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.510+0000 7fa30a200640 1 -- 172.21.15.71:0/3912044834 shutdown_connections 2024-09-16T07:36:10.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.510+0000 7fa30a200640 1 -- 172.21.15.71:0/3912044834 >> 172.21.15.71:0/3912044834 conn(0x7fa3040b1950 msgr2=0x7fa3040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:10.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.510+0000 7fa30a200640 1 -- 172.21.15.71:0/3912044834 shutdown_connections 2024-09-16T07:36:10.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.510+0000 7fa30a200640 1 -- 172.21.15.71:0/3912044834 wait complete. 2024-09-16T07:36:10.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.512+0000 7fa30a200640 1 Processor -- start 2024-09-16T07:36:10.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.512+0000 7fa30a200640 1 -- start start 2024-09-16T07:36:10.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa30a200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2e8003a00 0x7fa3040b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:10.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa30a200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3040b0860 0x7fa3041f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa30a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3041f86a0 0x7fa3041595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa30a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa304161a70 con 0x7fa3041f86a0 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa30a200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa30406fdb0 con 0x7fa3040b0860 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa30a200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa304073280 con 0x7fa2e8003a00 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa303e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3041f86a0 0x7fa3041595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa303e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3041f86a0 0x7fa3041595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40500/0 (socket says 172.21.15.71:40500) 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa303e00640 1 -- 172.21.15.71:0/4084310851 learned_addr learned my addr 172.21.15.71:0/4084310851 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa303400640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2e8003a00 0x7fa3040b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.513+0000 7fa302a00640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3040b0860 0x7fa3041f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa303e00640 1 -- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2e8003a00 msgr2=0x7fa3040b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa303e00640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2e8003a00 0x7fa3040b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa303e00640 1 -- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3040b0860 msgr2=0x7fa3041f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa303e00640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3040b0860 0x7fa3041f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa303e00640 1 -- 172.21.15.71:0/4084310851 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3041f4c10 con 0x7fa3041f86a0 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa302a00640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3040b0860 0x7fa3041f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa303400640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2e8003a00 0x7fa3040b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa303e00640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3041f86a0 0x7fa3041595c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa3040af050 tx=0x7fa2fc010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa2fb400640 1 -- 172.21.15.71:0/4084310851 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc003970 con 0x7fa3041f86a0 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa2fb400640 1 -- 172.21.15.71:0/4084310851 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2fc015070 con 0x7fa3041f86a0 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3041f4ef0 con 0x7fa3041f86a0 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa2fb400640 1 -- 172.21.15.71:0/4084310851 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc004030 con 0x7fa3041f86a0 2024-09-16T07:36:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.514+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3041f54b0 con 0x7fa3041f86a0 2024-09-16T07:36:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.515+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3041568e0 con 0x7fa3041f86a0 2024-09-16T07:36:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.516+0000 7fa2fb400640 1 -- 172.21.15.71:0/4084310851 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fa2fc0041d0 con 0x7fa3041f86a0 2024-09-16T07:36:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.517+0000 7fa2fb400640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa2e407a6e0 0x7fa2e407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.517+0000 7fa2fb400640 1 -- 172.21.15.71:0/4084310851 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7fa2fc09a490 con 0x7fa3041f86a0 2024-09-16T07:36:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.518+0000 7fa303400640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa2e407a6e0 0x7fa2e407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:10.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.519+0000 7fa303400640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa2e407a6e0 0x7fa2e407cba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4033000 tx=0x7fa2f402d000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:10.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.522+0000 7fa2fb400640 1 -- 172.21.15.71:0/4084310851 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa2fc064560 con 0x7fa3041f86a0 2024-09-16T07:36:10.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.704+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3040b0c40 con 0x7fa2e407a6e0 2024-09-16T07:36:10.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.709+0000 7fa2fb400640 1 -- 172.21.15.71:0/4084310851 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252499 (secure 0 0 0) 0x7fa2f4081a70 con 0x7fa2e407a6e0 2024-09-16T07:36:10.712 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:10.713 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:36:10.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa2e407a6e0 msgr2=0x7fa2e407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa2e407a6e0 0x7fa2e407cba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4033000 tx=0x7fa2f402d000 comp rx=0 tx=0).stop 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3041f86a0 msgr2=0x7fa3041595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3041f86a0 0x7fa3041595c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa3040af050 tx=0x7fa2fc010f40 comp rx=0 tx=0).stop 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 shutdown_connections 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa2e407a6e0 0x7fa2e407cba0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3041f86a0 0x7fa3041595c0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3040b0860 0x7fa3041f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 --2- 172.21.15.71:0/4084310851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2e8003a00 0x7fa3040b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 >> 172.21.15.71:0/4084310851 conn(0x7fa3040b1950 msgr2=0x7fa3040acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 shutdown_connections 2024-09-16T07:36:10.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:10.713+0000 7fa30a200640 1 -- 172.21.15.71:0/4084310851 wait complete. 2024-09-16T07:36:10.730 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-09-16T07:36:10.626600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076584,"kb_used_data":14076,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898136,"statfs":{"total":773094113280,"available":771991691264,"internally_reserved":0,"allocated":14413824,"data_stored":6976972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1087819312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.793601"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.853420+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.853157+0000","last_became_peered":"2024-09-16T07:36:04.853157+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:29.795783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063845+0000","last_change":"2024-09-16T07:36:04.855589+0000","last_active":"2024-09-16T07:36:06.063845+0000","last_peered":"2024-09-16T07:36:06.063845+0000","last_clean":"2024-09-16T07:36:06.063845+0000","last_became_active":"2024-09-16T07:36:04.855323+0000","last_became_peered":"2024-09-16T07:36:04.855323+0000","last_unstale":"2024-09-16T07:36:06.063845+0000","last_undegraded":"2024-09-16T07:36:06.063845+0000","last_fullsized":"2024-09-16T07:36:06.063845+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:53.457746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978230+0000","last_change":"2024-09-16T07:36:04.852096+0000","last_active":"2024-09-16T07:36:06.978230+0000","last_peered":"2024-09-16T07:36:06.978230+0000","last_clean":"2024-09-16T07:36:06.978230+0000","last_became_active":"2024-09-16T07:36:04.851548+0000","last_became_peered":"2024-09-16T07:36:04.851548+0000","last_unstale":"2024-09-16T07:36:06.978230+0000","last_undegraded":"2024-09-16T07:36:06.978230+0000","last_fullsized":"2024-09-16T07:36:06.978230+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:35.503529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855470+0000","last_change":"2024-09-16T07:36:04.859510+0000","last_active":"2024-09-16T07:36:05.855470+0000","last_peered":"2024-09-16T07:36:05.855470+0000","last_clean":"2024-09-16T07:36:05.855470+0000","last_became_active":"2024-09-16T07:36:04.859175+0000","last_became_peered":"2024-09-16T07:36:04.859175+0000","last_unstale":"2024-09-16T07:36:05.855470+0000","last_undegraded":"2024-09-16T07:36:05.855470+0000","last_fullsized":"2024-09-16T07:36:05.855470+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:42:35.105087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858303+0000","last_change":"2024-09-16T07:36:04.855716+0000","last_active":"2024-09-16T07:36:05.858303+0000","last_peered":"2024-09-16T07:36:05.858303+0000","last_clean":"2024-09-16T07:36:05.858303+0000","last_became_active":"2024-09-16T07:36:04.853373+0000","last_became_peered":"2024-09-16T07:36:04.853373+0000","last_unstale":"2024-09-16T07:36:05.858303+0000","last_undegraded":"2024-09-16T07:36:05.858303+0000","last_fullsized":"2024-09-16T07:36:05.858303+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:39.830389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849450+0000","last_change":"2024-09-16T07:36:04.851280+0000","last_active":"2024-09-16T07:36:05.849450+0000","last_peered":"2024-09-16T07:36:05.849450+0000","last_clean":"2024-09-16T07:36:05.849450+0000","last_became_active":"2024-09-16T07:36:04.851020+0000","last_became_peered":"2024-09-16T07:36:04.851020+0000","last_unstale":"2024-09-16T07:36:05.849450+0000","last_undegraded":"2024-09-16T07:36:05.849450+0000","last_fullsized":"2024-09-16T07:36:05.849450+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:38.230442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858195+0000","last_change":"2024-09-16T07:36:04.844492+0000","last_active":"2024-09-16T07:36:05.858195+0000","last_peered":"2024-09-16T07:36:05.858195+0000","last_clean":"2024-09-16T07:36:05.858195+0000","last_became_active":"2024-09-16T07:36:04.843818+0000","last_became_peered":"2024-09-16T07:36:04.843818+0000","last_unstale":"2024-09-16T07:36:05.858195+0000","last_undegraded":"2024-09-16T07:36:05.858195+0000","last_fullsized":"2024-09-16T07:36:05.858195+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:38.924508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858071+0000","last_change":"2024-09-16T07:36:04.845663+0000","last_active":"2024-09-16T07:36:05.858071+0000","last_peered":"2024-09-16T07:36:05.858071+0000","last_clean":"2024-09-16T07:36:05.858071+0000","last_became_active":"2024-09-16T07:36:04.844565+0000","last_became_peered":"2024-09-16T07:36:04.844565+0000","last_unstale":"2024-09-16T07:36:05.858071+0000","last_undegraded":"2024-09-16T07:36:05.858071+0000","last_fullsized":"2024-09-16T07:36:05.858071+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:43.360504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978331+0000","last_change":"2024-09-16T07:36:04.859110+0000","last_active":"2024-09-16T07:36:06.978331+0000","last_peered":"2024-09-16T07:36:06.978331+0000","last_clean":"2024-09-16T07:36:06.978331+0000","last_became_active":"2024-09-16T07:36:04.858813+0000","last_became_peered":"2024-09-16T07:36:04.858813+0000","last_unstale":"2024-09-16T07:36:06.978331+0000","last_undegraded":"2024-09-16T07:36:06.978331+0000","last_fullsized":"2024-09-16T07:36:06.978331+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:54.655354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.851248+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.850709+0000","last_became_peered":"2024-09-16T07:36:04.850709+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:07.411898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855892+0000","last_change":"2024-09-16T07:36:04.854097+0000","last_active":"2024-09-16T07:36:05.855892+0000","last_peered":"2024-09-16T07:36:05.855892+0000","last_clean":"2024-09-16T07:36:05.855892+0000","last_became_active":"2024-09-16T07:36:04.853827+0000","last_became_peered":"2024-09-16T07:36:04.853827+0000","last_unstale":"2024-09-16T07:36:05.855892+0000","last_undegraded":"2024-09-16T07:36:05.855892+0000","last_fullsized":"2024-09-16T07:36:05.855892+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:21.875425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850041+0000","last_change":"2024-09-16T07:36:04.853331+0000","last_active":"2024-09-16T07:36:05.850041+0000","last_peered":"2024-09-16T07:36:05.850041+0000","last_clean":"2024-09-16T07:36:05.850041+0000","last_became_active":"2024-09-16T07:36:04.853085+0000","last_became_peered":"2024-09-16T07:36:04.853085+0000","last_unstale":"2024-09-16T07:36:05.850041+0000","last_undegraded":"2024-09-16T07:36:05.850041+0000","last_fullsized":"2024-09-16T07:36:05.850041+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:10.939637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855153+0000","last_change":"2024-09-16T07:36:04.848716+0000","last_active":"2024-09-16T07:36:05.855153+0000","last_peered":"2024-09-16T07:36:05.855153+0000","last_clean":"2024-09-16T07:36:05.855153+0000","last_became_active":"2024-09-16T07:36:04.848457+0000","last_became_peered":"2024-09-16T07:36:04.848457+0000","last_unstale":"2024-09-16T07:36:05.855153+0000","last_undegraded":"2024-09-16T07:36:05.855153+0000","last_fullsized":"2024-09-16T07:36:05.855153+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:21.475235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856556+0000","last_change":"2024-09-16T07:36:04.859096+0000","last_active":"2024-09-16T07:36:05.856556+0000","last_peered":"2024-09-16T07:36:05.856556+0000","last_clean":"2024-09-16T07:36:05.856556+0000","last_became_active":"2024-09-16T07:36:04.858812+0000","last_became_peered":"2024-09-16T07:36:04.858812+0000","last_unstale":"2024-09-16T07:36:05.856556+0000","last_undegraded":"2024-09-16T07:36:05.856556+0000","last_fullsized":"2024-09-16T07:36:05.856556+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:59.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855973+0000","last_change":"2024-09-16T07:36:04.862176+0000","last_active":"2024-09-16T07:36:05.855973+0000","last_peered":"2024-09-16T07:36:05.855973+0000","last_clean":"2024-09-16T07:36:05.855973+0000","last_became_active":"2024-09-16T07:36:04.860550+0000","last_became_peered":"2024-09-16T07:36:04.860550+0000","last_unstale":"2024-09-16T07:36:05.855973+0000","last_undegraded":"2024-09-16T07:36:05.855973+0000","last_fullsized":"2024-09-16T07:36:05.855973+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:10.040307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854370+0000","last_change":"2024-09-16T07:36:04.855363+0000","last_active":"2024-09-16T07:36:05.854370+0000","last_peered":"2024-09-16T07:36:05.854370+0000","last_clean":"2024-09-16T07:36:05.854370+0000","last_became_active":"2024-09-16T07:36:04.855013+0000","last_became_peered":"2024-09-16T07:36:04.855013+0000","last_unstale":"2024-09-16T07:36:05.854370+0000","last_undegraded":"2024-09-16T07:36:05.854370+0000","last_fullsized":"2024-09-16T07:36:05.854370+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:13.040194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978545+0000","last_change":"2024-09-16T07:36:04.859278+0000","last_active":"2024-09-16T07:36:06.978545+0000","last_peered":"2024-09-16T07:36:06.978545+0000","last_clean":"2024-09-16T07:36:06.978545+0000","last_became_active":"2024-09-16T07:36:04.858925+0000","last_became_peered":"2024-09-16T07:36:04.858925+0000","last_unstale":"2024-09-16T07:36:06.978545+0000","last_undegraded":"2024-09-16T07:36:06.978545+0000","last_fullsized":"2024-09-16T07:36:06.978545+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:38.241305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855025+0000","last_change":"2024-09-16T07:36:04.862436+0000","last_active":"2024-09-16T07:36:05.855025+0000","last_peered":"2024-09-16T07:36:05.855025+0000","last_clean":"2024-09-16T07:36:05.855025+0000","last_became_active":"2024-09-16T07:36:04.862125+0000","last_became_peered":"2024-09-16T07:36:04.862125+0000","last_unstale":"2024-09-16T07:36:05.855025+0000","last_undegraded":"2024-09-16T07:36:05.855025+0000","last_fullsized":"2024-09-16T07:36:05.855025+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:38.787658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063678+0000","last_change":"2024-09-16T07:36:04.857741+0000","last_active":"2024-09-16T07:36:06.063678+0000","last_peered":"2024-09-16T07:36:06.063678+0000","last_clean":"2024-09-16T07:36:06.063678+0000","last_became_active":"2024-09-16T07:36:04.854444+0000","last_became_peered":"2024-09-16T07:36:04.854444+0000","last_unstale":"2024-09-16T07:36:06.063678+0000","last_undegraded":"2024-09-16T07:36:06.063678+0000","last_fullsized":"2024-09-16T07:36:06.063678+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:15.100689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856475+0000","last_change":"2024-09-16T07:36:04.869720+0000","last_active":"2024-09-16T07:36:05.856475+0000","last_peered":"2024-09-16T07:36:05.856475+0000","last_clean":"2024-09-16T07:36:05.856475+0000","last_became_active":"2024-09-16T07:36:04.868956+0000","last_became_peered":"2024-09-16T07:36:04.868956+0000","last_unstale":"2024-09-16T07:36:05.856475+0000","last_undegraded":"2024-09-16T07:36:05.856475+0000","last_fullsized":"2024-09-16T07:36:05.856475+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:29.852710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849663+0000","last_change":"2024-09-16T07:36:04.855006+0000","last_active":"2024-09-16T07:36:05.849663+0000","last_peered":"2024-09-16T07:36:05.849663+0000","last_clean":"2024-09-16T07:36:05.849663+0000","last_became_active":"2024-09-16T07:36:04.854529+0000","last_became_peered":"2024-09-16T07:36:04.854529+0000","last_unstale":"2024-09-16T07:36:05.849663+0000","last_undegraded":"2024-09-16T07:36:05.849663+0000","last_fullsized":"2024-09-16T07:36:05.849663+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:04.807416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063836+0000","last_change":"2024-09-16T07:36:04.845962+0000","last_active":"2024-09-16T07:36:06.063836+0000","last_peered":"2024-09-16T07:36:06.063836+0000","last_clean":"2024-09-16T07:36:06.063836+0000","last_became_active":"2024-09-16T07:36:04.845664+0000","last_became_peered":"2024-09-16T07:36:04.845664+0000","last_unstale":"2024-09-16T07:36:06.063836+0000","last_undegraded":"2024-09-16T07:36:06.063836+0000","last_fullsized":"2024-09-16T07:36:06.063836+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:32.167063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974830+0000","last_change":"2024-09-16T07:36:04.853123+0000","last_active":"2024-09-16T07:36:06.974830+0000","last_peered":"2024-09-16T07:36:06.974830+0000","last_clean":"2024-09-16T07:36:06.974830+0000","last_became_active":"2024-09-16T07:36:04.852684+0000","last_became_peered":"2024-09-16T07:36:04.852684+0000","last_unstale":"2024-09-16T07:36:06.974830+0000","last_undegraded":"2024-09-16T07:36:06.974830+0000","last_fullsized":"2024-09-16T07:36:06.974830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:09.092561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854174+0000","last_change":"2024-09-16T07:36:04.865540+0000","last_active":"2024-09-16T07:36:05.854174+0000","last_peered":"2024-09-16T07:36:05.854174+0000","last_clean":"2024-09-16T07:36:05.854174+0000","last_became_active":"2024-09-16T07:36:04.865232+0000","last_became_peered":"2024-09-16T07:36:04.865232+0000","last_unstale":"2024-09-16T07:36:05.854174+0000","last_undegraded":"2024-09-16T07:36:05.854174+0000","last_fullsized":"2024-09-16T07:36:05.854174+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:24.118557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855774+0000","last_change":"2024-09-16T07:36:04.857205+0000","last_active":"2024-09-16T07:36:05.855774+0000","last_peered":"2024-09-16T07:36:05.855774+0000","last_clean":"2024-09-16T07:36:05.855774+0000","last_became_active":"2024-09-16T07:36:04.856823+0000","last_became_peered":"2024-09-16T07:36:04.856823+0000","last_unstale":"2024-09-16T07:36:05.855774+0000","last_undegraded":"2024-09-16T07:36:05.855774+0000","last_fullsized":"2024-09-16T07:36:05.855774+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:16.355699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064085+0000","last_change":"2024-09-16T07:36:04.850897+0000","last_active":"2024-09-16T07:36:06.064085+0000","last_peered":"2024-09-16T07:36:06.064085+0000","last_clean":"2024-09-16T07:36:06.064085+0000","last_became_active":"2024-09-16T07:36:04.850635+0000","last_became_peered":"2024-09-16T07:36:04.850635+0000","last_unstale":"2024-09-16T07:36:06.064085+0000","last_undegraded":"2024-09-16T07:36:06.064085+0000","last_fullsized":"2024-09-16T07:36:06.064085+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:38.194427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857379+0000","last_change":"2024-09-16T07:36:04.855205+0000","last_active":"2024-09-16T07:36:05.857379+0000","last_peered":"2024-09-16T07:36:05.857379+0000","last_clean":"2024-09-16T07:36:05.857379+0000","last_became_active":"2024-09-16T07:36:04.854356+0000","last_became_peered":"2024-09-16T07:36:04.854356+0000","last_unstale":"2024-09-16T07:36:05.857379+0000","last_undegraded":"2024-09-16T07:36:05.857379+0000","last_fullsized":"2024-09-16T07:36:05.857379+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:23.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974810+0000","last_change":"2024-09-16T07:36:04.854709+0000","last_active":"2024-09-16T07:36:06.974810+0000","last_peered":"2024-09-16T07:36:06.974810+0000","last_clean":"2024-09-16T07:36:06.974810+0000","last_became_active":"2024-09-16T07:36:04.854430+0000","last_became_peered":"2024-09-16T07:36:04.854430+0000","last_unstale":"2024-09-16T07:36:06.974810+0000","last_undegraded":"2024-09-16T07:36:06.974810+0000","last_fullsized":"2024-09-16T07:36:06.974810+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:33.380575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063430+0000","last_change":"2024-09-16T07:36:04.849621+0000","last_active":"2024-09-16T07:36:06.063430+0000","last_peered":"2024-09-16T07:36:06.063430+0000","last_clean":"2024-09-16T07:36:06.063430+0000","last_became_active":"2024-09-16T07:36:04.849325+0000","last_became_peered":"2024-09-16T07:36:04.849325+0000","last_unstale":"2024-09-16T07:36:06.063430+0000","last_undegraded":"2024-09-16T07:36:06.063430+0000","last_fullsized":"2024-09-16T07:36:06.063430+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:20.048326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063880+0000","last_change":"2024-09-16T07:36:04.840820+0000","last_active":"2024-09-16T07:36:06.063880+0000","last_peered":"2024-09-16T07:36:06.063880+0000","last_clean":"2024-09-16T07:36:06.063880+0000","last_became_active":"2024-09-16T07:36:04.839397+0000","last_became_peered":"2024-09-16T07:36:04.839397+0000","last_unstale":"2024-09-16T07:36:06.063880+0000","last_undegraded":"2024-09-16T07:36:06.063880+0000","last_fullsized":"2024-09-16T07:36:06.063880+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:01.349093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974555+0000","last_change":"2024-09-16T07:36:04.861314+0000","last_active":"2024-09-16T07:36:06.974555+0000","last_peered":"2024-09-16T07:36:06.974555+0000","last_clean":"2024-09-16T07:36:06.974555+0000","last_became_active":"2024-09-16T07:36:04.861052+0000","last_became_peered":"2024-09-16T07:36:04.861052+0000","last_unstale":"2024-09-16T07:36:06.974555+0000","last_undegraded":"2024-09-16T07:36:06.974555+0000","last_fullsized":"2024-09-16T07:36:06.974555+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:59.224590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978093+0000","last_change":"2024-09-16T07:36:04.841017+0000","last_active":"2024-09-16T07:36:06.978093+0000","last_peered":"2024-09-16T07:36:06.978093+0000","last_clean":"2024-09-16T07:36:06.978093+0000","last_became_active":"2024-09-16T07:36:04.840692+0000","last_became_peered":"2024-09-16T07:36:04.840692+0000","last_unstale":"2024-09-16T07:36:06.978093+0000","last_undegraded":"2024-09-16T07:36:06.978093+0000","last_fullsized":"2024-09-16T07:36:06.978093+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.829657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857460+0000","last_change":"2024-09-16T07:36:04.859595+0000","last_active":"2024-09-16T07:36:05.857460+0000","last_peered":"2024-09-16T07:36:05.857460+0000","last_clean":"2024-09-16T07:36:05.857460+0000","last_became_active":"2024-09-16T07:36:04.856512+0000","last_became_peered":"2024-09-16T07:36:04.856512+0000","last_unstale":"2024-09-16T07:36:05.857460+0000","last_undegraded":"2024-09-16T07:36:05.857460+0000","last_fullsized":"2024-09-16T07:36:05.857460+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:25.701980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"132'4","reported_seq":19,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858698+0000","last_change":"2024-09-16T07:36:03.855477+0000","last_active":"2024-09-16T07:36:05.858698+0000","last_peered":"2024-09-16T07:36:05.858698+0000","last_clean":"2024-09-16T07:36:05.858698+0000","last_became_active":"2024-09-16T07:36:03.852255+0000","last_became_peered":"2024-09-16T07:36:03.852255+0000","last_unstale":"2024-09-16T07:36:05.858698+0000","last_undegraded":"2024-09-16T07:36:05.858698+0000","last_fullsized":"2024-09-16T07:36:05.858698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:17.990182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978337+0000","last_change":"2024-09-16T07:36:04.853865+0000","last_active":"2024-09-16T07:36:06.978337+0000","last_peered":"2024-09-16T07:36:06.978337+0000","last_clean":"2024-09-16T07:36:06.978337+0000","last_became_active":"2024-09-16T07:36:04.852369+0000","last_became_peered":"2024-09-16T07:36:04.852369+0000","last_unstale":"2024-09-16T07:36:06.978337+0000","last_undegraded":"2024-09-16T07:36:06.978337+0000","last_fullsized":"2024-09-16T07:36:06.978337+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:11.213364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978359+0000","last_change":"2024-09-16T07:36:03.853595+0000","last_active":"2024-09-16T07:36:06.978359+0000","last_peered":"2024-09-16T07:36:06.978359+0000","last_clean":"2024-09-16T07:36:06.978359+0000","last_became_active":"2024-09-16T07:36:03.853315+0000","last_became_peered":"2024-09-16T07:36:03.853315+0000","last_unstale":"2024-09-16T07:36:06.978359+0000","last_undegraded":"2024-09-16T07:36:06.978359+0000","last_fullsized":"2024-09-16T07:36:06.978359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:39.036062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978237+0000","last_change":"2024-09-16T07:36:04.849253+0000","last_active":"2024-09-16T07:36:06.978237+0000","last_peered":"2024-09-16T07:36:06.978237+0000","last_clean":"2024-09-16T07:36:06.978237+0000","last_became_active":"2024-09-16T07:36:04.848990+0000","last_became_peered":"2024-09-16T07:36:04.848990+0000","last_unstale":"2024-09-16T07:36:06.978237+0000","last_undegraded":"2024-09-16T07:36:06.978237+0000","last_fullsized":"2024-09-16T07:36:06.978237+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:57.540368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856585+0000","last_change":"2024-09-16T07:36:03.849019+0000","last_active":"2024-09-16T07:36:05.856585+0000","last_peered":"2024-09-16T07:36:05.856585+0000","last_clean":"2024-09-16T07:36:05.856585+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.856585+0000","last_undegraded":"2024-09-16T07:36:05.856585+0000","last_fullsized":"2024-09-16T07:36:05.856585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:53.651818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850191+0000","last_change":"2024-09-16T07:36:04.861211+0000","last_active":"2024-09-16T07:36:05.850191+0000","last_peered":"2024-09-16T07:36:05.850191+0000","last_clean":"2024-09-16T07:36:05.850191+0000","last_became_active":"2024-09-16T07:36:04.860347+0000","last_became_peered":"2024-09-16T07:36:04.860347+0000","last_unstale":"2024-09-16T07:36:05.850191+0000","last_undegraded":"2024-09-16T07:36:05.850191+0000","last_fullsized":"2024-09-16T07:36:05.850191+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:29.976789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063690+0000","last_change":"2024-09-16T07:36:03.853178+0000","last_active":"2024-09-16T07:36:06.063690+0000","last_peered":"2024-09-16T07:36:06.063690+0000","last_clean":"2024-09-16T07:36:06.063690+0000","last_became_active":"2024-09-16T07:36:03.852914+0000","last_became_peered":"2024-09-16T07:36:03.852914+0000","last_unstale":"2024-09-16T07:36:06.063690+0000","last_undegraded":"2024-09-16T07:36:06.063690+0000","last_fullsized":"2024-09-16T07:36:06.063690+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.868825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855929+0000","last_change":"2024-09-16T07:36:04.860375+0000","last_active":"2024-09-16T07:36:05.855929+0000","last_peered":"2024-09-16T07:36:05.855929+0000","last_clean":"2024-09-16T07:36:05.855929+0000","last_became_active":"2024-09-16T07:36:04.860110+0000","last_became_peered":"2024-09-16T07:36:04.860110+0000","last_unstale":"2024-09-16T07:36:05.855929+0000","last_undegraded":"2024-09-16T07:36:05.855929+0000","last_fullsized":"2024-09-16T07:36:05.855929+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:44.383231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063342+0000","last_change":"2024-09-16T07:36:03.848993+0000","last_active":"2024-09-16T07:36:06.063342+0000","last_peered":"2024-09-16T07:36:06.063342+0000","last_clean":"2024-09-16T07:36:06.063342+0000","last_became_active":"2024-09-16T07:36:03.848700+0000","last_became_peered":"2024-09-16T07:36:03.848700+0000","last_unstale":"2024-09-16T07:36:06.063342+0000","last_undegraded":"2024-09-16T07:36:06.063342+0000","last_fullsized":"2024-09-16T07:36:06.063342+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:54.220868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849654+0000","last_change":"2024-09-16T07:36:04.850135+0000","last_active":"2024-09-16T07:36:05.849654+0000","last_peered":"2024-09-16T07:36:05.849654+0000","last_clean":"2024-09-16T07:36:05.849654+0000","last_became_active":"2024-09-16T07:36:04.849830+0000","last_became_peered":"2024-09-16T07:36:04.849830+0000","last_unstale":"2024-09-16T07:36:05.849654+0000","last_undegraded":"2024-09-16T07:36:05.849654+0000","last_fullsized":"2024-09-16T07:36:05.849654+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:40.566705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856372+0000","last_change":"2024-09-16T07:36:03.855637+0000","last_active":"2024-09-16T07:36:05.856372+0000","last_peered":"2024-09-16T07:36:05.856372+0000","last_clean":"2024-09-16T07:36:05.856372+0000","last_became_active":"2024-09-16T07:36:03.855357+0000","last_became_peered":"2024-09-16T07:36:03.855357+0000","last_unstale":"2024-09-16T07:36:05.856372+0000","last_undegraded":"2024-09-16T07:36:05.856372+0000","last_fullsized":"2024-09-16T07:36:05.856372+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.392485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849437+0000","last_change":"2024-09-16T07:36:04.852032+0000","last_active":"2024-09-16T07:36:05.849437+0000","last_peered":"2024-09-16T07:36:05.849437+0000","last_clean":"2024-09-16T07:36:05.849437+0000","last_became_active":"2024-09-16T07:36:04.851770+0000","last_became_peered":"2024-09-16T07:36:04.851770+0000","last_unstale":"2024-09-16T07:36:05.849437+0000","last_undegraded":"2024-09-16T07:36:05.849437+0000","last_fullsized":"2024-09-16T07:36:05.849437+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:10.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978019+0000","last_change":"2024-09-16T07:36:03.858168+0000","last_active":"2024-09-16T07:36:06.978019+0000","last_peered":"2024-09-16T07:36:06.978019+0000","last_clean":"2024-09-16T07:36:06.978019+0000","last_became_active":"2024-09-16T07:36:03.857697+0000","last_became_peered":"2024-09-16T07:36:03.857697+0000","last_unstale":"2024-09-16T07:36:06.978019+0000","last_undegraded":"2024-09-16T07:36:06.978019+0000","last_fullsized":"2024-09-16T07:36:06.978019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:53.424532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063343+0000","last_change":"2024-09-16T07:36:04.853058+0000","last_active":"2024-09-16T07:36:06.063343+0000","last_peered":"2024-09-16T07:36:06.063343+0000","last_clean":"2024-09-16T07:36:06.063343+0000","last_became_active":"2024-09-16T07:36:04.852535+0000","last_became_peered":"2024-09-16T07:36:04.852535+0000","last_unstale":"2024-09-16T07:36:06.063343+0000","last_undegraded":"2024-09-16T07:36:06.063343+0000","last_fullsized":"2024-09-16T07:36:06.063343+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:36.784633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063356+0000","last_change":"2024-09-16T07:36:03.846732+0000","last_active":"2024-09-16T07:36:06.063356+0000","last_peered":"2024-09-16T07:36:06.063356+0000","last_clean":"2024-09-16T07:36:06.063356+0000","last_became_active":"2024-09-16T07:36:03.845789+0000","last_became_peered":"2024-09-16T07:36:03.845789+0000","last_unstale":"2024-09-16T07:36:06.063356+0000","last_undegraded":"2024-09-16T07:36:06.063356+0000","last_fullsized":"2024-09-16T07:36:06.063356+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:53.374127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064066+0000","last_change":"2024-09-16T07:36:04.852160+0000","last_active":"2024-09-16T07:36:06.064066+0000","last_peered":"2024-09-16T07:36:06.064066+0000","last_clean":"2024-09-16T07:36:06.064066+0000","last_became_active":"2024-09-16T07:36:04.851903+0000","last_became_peered":"2024-09-16T07:36:04.851903+0000","last_unstale":"2024-09-16T07:36:06.064066+0000","last_undegraded":"2024-09-16T07:36:06.064066+0000","last_fullsized":"2024-09-16T07:36:06.064066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:32.387978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850176+0000","last_change":"2024-09-16T07:36:03.844377+0000","last_active":"2024-09-16T07:36:05.850176+0000","last_peered":"2024-09-16T07:36:05.850176+0000","last_clean":"2024-09-16T07:36:05.850176+0000","last_became_active":"2024-09-16T07:36:03.844005+0000","last_became_peered":"2024-09-16T07:36:03.844005+0000","last_unstale":"2024-09-16T07:36:05.850176+0000","last_undegraded":"2024-09-16T07:36:05.850176+0000","last_fullsized":"2024-09-16T07:36:05.850176+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:46.575242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858524+0000","last_change":"2024-09-16T07:36:04.855371+0000","last_active":"2024-09-16T07:36:05.858524+0000","last_peered":"2024-09-16T07:36:05.858524+0000","last_clean":"2024-09-16T07:36:05.858524+0000","last_became_active":"2024-09-16T07:36:04.854159+0000","last_became_peered":"2024-09-16T07:36:04.854159+0000","last_unstale":"2024-09-16T07:36:05.858524+0000","last_undegraded":"2024-09-16T07:36:05.858524+0000","last_fullsized":"2024-09-16T07:36:05.858524+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:53:57.092765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855092+0000","last_change":"2024-09-16T07:36:03.852706+0000","last_active":"2024-09-16T07:36:05.855092+0000","last_peered":"2024-09-16T07:36:05.855092+0000","last_clean":"2024-09-16T07:36:05.855092+0000","last_became_active":"2024-09-16T07:36:03.852295+0000","last_became_peered":"2024-09-16T07:36:03.852295+0000","last_unstale":"2024-09-16T07:36:05.855092+0000","last_undegraded":"2024-09-16T07:36:05.855092+0000","last_fullsized":"2024-09-16T07:36:05.855092+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:51.417749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850016+0000","last_change":"2024-09-16T07:36:04.851478+0000","last_active":"2024-09-16T07:36:05.850016+0000","last_peered":"2024-09-16T07:36:05.850016+0000","last_clean":"2024-09-16T07:36:05.850016+0000","last_became_active":"2024-09-16T07:36:04.851213+0000","last_became_peered":"2024-09-16T07:36:04.851213+0000","last_unstale":"2024-09-16T07:36:05.850016+0000","last_undegraded":"2024-09-16T07:36:05.850016+0000","last_fullsized":"2024-09-16T07:36:05.850016+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:09:51.497503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"132'3","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858392+0000","last_change":"2024-09-16T07:36:03.857233+0000","last_active":"2024-09-16T07:36:05.858392+0000","last_peered":"2024-09-16T07:36:05.858392+0000","last_clean":"2024-09-16T07:36:05.858392+0000","last_became_active":"2024-09-16T07:36:03.856817+0000","last_became_peered":"2024-09-16T07:36:03.856817+0000","last_unstale":"2024-09-16T07:36:05.858392+0000","last_undegraded":"2024-09-16T07:36:05.858392+0000","last_fullsized":"2024-09-16T07:36:05.858392+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:40:54.651864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974575+0000","last_change":"2024-09-16T07:36:04.855153+0000","last_active":"2024-09-16T07:36:06.974575+0000","last_peered":"2024-09-16T07:36:06.974575+0000","last_clean":"2024-09-16T07:36:06.974575+0000","last_became_active":"2024-09-16T07:36:04.854893+0000","last_became_peered":"2024-09-16T07:36:04.854893+0000","last_unstale":"2024-09-16T07:36:06.974575+0000","last_undegraded":"2024-09-16T07:36:06.974575+0000","last_fullsized":"2024-09-16T07:36:06.974575+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:00.759798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"132'6","reported_seq":21,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974642+0000","last_change":"2024-09-16T07:36:03.851395+0000","last_active":"2024-09-16T07:36:06.974642+0000","last_peered":"2024-09-16T07:36:06.974642+0000","last_clean":"2024-09-16T07:36:06.974642+0000","last_became_active":"2024-09-16T07:36:03.851127+0000","last_became_peered":"2024-09-16T07:36:03.851127+0000","last_unstale":"2024-09-16T07:36:06.974642+0000","last_undegraded":"2024-09-16T07:36:06.974642+0000","last_fullsized":"2024-09-16T07:36:06.974642+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:27.567738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856941+0000","last_change":"2024-09-16T07:36:04.863490+0000","last_active":"2024-09-16T07:36:05.856941+0000","last_peered":"2024-09-16T07:36:05.856941+0000","last_clean":"2024-09-16T07:36:05.856941+0000","last_became_active":"2024-09-16T07:36:04.862482+0000","last_became_peered":"2024-09-16T07:36:04.862482+0000","last_unstale":"2024-09-16T07:36:05.856941+0000","last_undegraded":"2024-09-16T07:36:05.856941+0000","last_fullsized":"2024-09-16T07:36:05.856941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:18.732185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856621+0000","last_change":"2024-09-16T07:36:03.852858+0000","last_active":"2024-09-16T07:36:05.856621+0000","last_peered":"2024-09-16T07:36:05.856621+0000","last_clean":"2024-09-16T07:36:05.856621+0000","last_became_active":"2024-09-16T07:36:03.852596+0000","last_became_peered":"2024-09-16T07:36:03.852596+0000","last_unstale":"2024-09-16T07:36:05.856621+0000","last_undegraded":"2024-09-16T07:36:05.856621+0000","last_fullsized":"2024-09-16T07:36:05.856621+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:29.307382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854662+0000","last_change":"2024-09-16T07:36:04.855631+0000","last_active":"2024-09-16T07:36:05.854662+0000","last_peered":"2024-09-16T07:36:05.854662+0000","last_clean":"2024-09-16T07:36:05.854662+0000","last_became_active":"2024-09-16T07:36:04.855341+0000","last_became_peered":"2024-09-16T07:36:04.855341+0000","last_unstale":"2024-09-16T07:36:05.854662+0000","last_undegraded":"2024-09-16T07:36:05.854662+0000","last_fullsized":"2024-09-16T07:36:05.854662+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:06.127741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854531+0000","last_change":"2024-09-16T07:36:03.854082+0000","last_active":"2024-09-16T07:36:05.854531+0000","last_peered":"2024-09-16T07:36:05.854531+0000","last_clean":"2024-09-16T07:36:05.854531+0000","last_became_active":"2024-09-16T07:36:03.853794+0000","last_became_peered":"2024-09-16T07:36:03.853794+0000","last_unstale":"2024-09-16T07:36:05.854531+0000","last_undegraded":"2024-09-16T07:36:05.854531+0000","last_fullsized":"2024-09-16T07:36:05.854531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:10.982614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849412+0000","last_change":"2024-09-16T07:36:04.852986+0000","last_active":"2024-09-16T07:36:05.849412+0000","last_peered":"2024-09-16T07:36:05.849412+0000","last_clean":"2024-09-16T07:36:05.849412+0000","last_became_active":"2024-09-16T07:36:04.851428+0000","last_became_peered":"2024-09-16T07:36:04.851428+0000","last_unstale":"2024-09-16T07:36:05.849412+0000","last_undegraded":"2024-09-16T07:36:05.849412+0000","last_fullsized":"2024-09-16T07:36:05.849412+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:58.294658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"132'8","reported_seq":23,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849259+0000","last_change":"2024-09-16T07:36:03.845321+0000","last_active":"2024-09-16T07:36:05.849259+0000","last_peered":"2024-09-16T07:36:05.849259+0000","last_clean":"2024-09-16T07:36:05.849259+0000","last_became_active":"2024-09-16T07:36:03.845024+0000","last_became_peered":"2024-09-16T07:36:03.845024+0000","last_unstale":"2024-09-16T07:36:05.849259+0000","last_undegraded":"2024-09-16T07:36:05.849259+0000","last_fullsized":"2024-09-16T07:36:05.849259+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:26.754316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858506+0000","last_change":"2024-09-16T07:36:04.850404+0000","last_active":"2024-09-16T07:36:05.858506+0000","last_peered":"2024-09-16T07:36:05.858506+0000","last_clean":"2024-09-16T07:36:05.858506+0000","last_became_active":"2024-09-16T07:36:04.850137+0000","last_became_peered":"2024-09-16T07:36:04.850137+0000","last_unstale":"2024-09-16T07:36:05.858506+0000","last_undegraded":"2024-09-16T07:36:05.858506+0000","last_fullsized":"2024-09-16T07:36:05.858506+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:35.706934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850284+0000","last_change":"2024-09-16T07:36:03.848724+0000","last_active":"2024-09-16T07:36:05.850284+0000","last_peered":"2024-09-16T07:36:05.850284+0000","last_clean":"2024-09-16T07:36:05.850284+0000","last_became_active":"2024-09-16T07:36:03.847881+0000","last_became_peered":"2024-09-16T07:36:03.847881+0000","last_unstale":"2024-09-16T07:36:05.850284+0000","last_undegraded":"2024-09-16T07:36:05.850284+0000","last_fullsized":"2024-09-16T07:36:05.850284+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:06.592607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858589+0000","last_change":"2024-09-16T07:36:04.860131+0000","last_active":"2024-09-16T07:36:05.858589+0000","last_peered":"2024-09-16T07:36:05.858589+0000","last_clean":"2024-09-16T07:36:05.858589+0000","last_became_active":"2024-09-16T07:36:04.859815+0000","last_became_peered":"2024-09-16T07:36:04.859815+0000","last_unstale":"2024-09-16T07:36:05.858589+0000","last_undegraded":"2024-09-16T07:36:05.858589+0000","last_fullsized":"2024-09-16T07:36:05.858589+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:31.769428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856205+0000","last_change":"2024-09-16T07:36:03.850133+0000","last_active":"2024-09-16T07:36:05.856205+0000","last_peered":"2024-09-16T07:36:05.856205+0000","last_clean":"2024-09-16T07:36:05.856205+0000","last_became_active":"2024-09-16T07:36:03.849070+0000","last_became_peered":"2024-09-16T07:36:03.849070+0000","last_unstale":"2024-09-16T07:36:05.856205+0000","last_undegraded":"2024-09-16T07:36:05.856205+0000","last_fullsized":"2024-09-16T07:36:05.856205+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:35.166163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063646+0000","last_change":"2024-09-16T07:36:04.848685+0000","last_active":"2024-09-16T07:36:06.063646+0000","last_peered":"2024-09-16T07:36:06.063646+0000","last_clean":"2024-09-16T07:36:06.063646+0000","last_became_active":"2024-09-16T07:36:04.844479+0000","last_became_peered":"2024-09-16T07:36:04.844479+0000","last_unstale":"2024-09-16T07:36:06.063646+0000","last_undegraded":"2024-09-16T07:36:06.063646+0000","last_fullsized":"2024-09-16T07:36:06.063646+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:36.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978385+0000","last_change":"2024-09-16T07:36:03.855122+0000","last_active":"2024-09-16T07:36:06.978385+0000","last_peered":"2024-09-16T07:36:06.978385+0000","last_clean":"2024-09-16T07:36:06.978385+0000","last_became_active":"2024-09-16T07:36:03.854786+0000","last_became_peered":"2024-09-16T07:36:03.854786+0000","last_unstale":"2024-09-16T07:36:06.978385+0000","last_undegraded":"2024-09-16T07:36:06.978385+0000","last_fullsized":"2024-09-16T07:36:06.978385+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:36.071792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063917+0000","last_change":"2024-09-16T07:36:04.853245+0000","last_active":"2024-09-16T07:36:06.063917+0000","last_peered":"2024-09-16T07:36:06.063917+0000","last_clean":"2024-09-16T07:36:06.063917+0000","last_became_active":"2024-09-16T07:36:04.852895+0000","last_became_peered":"2024-09-16T07:36:04.852895+0000","last_unstale":"2024-09-16T07:36:06.063917+0000","last_undegraded":"2024-09-16T07:36:06.063917+0000","last_fullsized":"2024-09-16T07:36:06.063917+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.199780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858362+0000","last_change":"2024-09-16T07:36:03.856130+0000","last_active":"2024-09-16T07:36:05.858362+0000","last_peered":"2024-09-16T07:36:05.858362+0000","last_clean":"2024-09-16T07:36:05.858362+0000","last_became_active":"2024-09-16T07:36:03.852405+0000","last_became_peered":"2024-09-16T07:36:03.852405+0000","last_unstale":"2024-09-16T07:36:05.858362+0000","last_undegraded":"2024-09-16T07:36:05.858362+0000","last_fullsized":"2024-09-16T07:36:05.858362+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:26.754316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064061+0000","last_change":"2024-09-16T07:36:04.847598+0000","last_active":"2024-09-16T07:36:06.064061+0000","last_peered":"2024-09-16T07:36:06.064061+0000","last_clean":"2024-09-16T07:36:06.064061+0000","last_became_active":"2024-09-16T07:36:04.847063+0000","last_became_peered":"2024-09-16T07:36:04.847063+0000","last_unstale":"2024-09-16T07:36:06.064061+0000","last_undegraded":"2024-09-16T07:36:06.064061+0000","last_fullsized":"2024-09-16T07:36:06.064061+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:50.838203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978132+0000","last_change":"2024-09-16T07:36:03.856847+0000","last_active":"2024-09-16T07:36:06.978132+0000","last_peered":"2024-09-16T07:36:06.978132+0000","last_clean":"2024-09-16T07:36:06.978132+0000","last_became_active":"2024-09-16T07:36:03.856321+0000","last_became_peered":"2024-09-16T07:36:03.856321+0000","last_unstale":"2024-09-16T07:36:06.978132+0000","last_undegraded":"2024-09-16T07:36:06.978132+0000","last_fullsized":"2024-09-16T07:36:06.978132+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:32.485232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974582+0000","last_change":"2024-09-16T07:36:04.848724+0000","last_active":"2024-09-16T07:36:06.974582+0000","last_peered":"2024-09-16T07:36:06.974582+0000","last_clean":"2024-09-16T07:36:06.974582+0000","last_became_active":"2024-09-16T07:36:04.848393+0000","last_became_peered":"2024-09-16T07:36:04.848393+0000","last_unstale":"2024-09-16T07:36:06.974582+0000","last_undegraded":"2024-09-16T07:36:06.974582+0000","last_fullsized":"2024-09-16T07:36:06.974582+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:12.890777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856080+0000","last_change":"2024-09-16T07:36:03.855412+0000","last_active":"2024-09-16T07:36:05.856080+0000","last_peered":"2024-09-16T07:36:05.856080+0000","last_clean":"2024-09-16T07:36:05.856080+0000","last_became_active":"2024-09-16T07:36:03.854933+0000","last_became_peered":"2024-09-16T07:36:03.854933+0000","last_unstale":"2024-09-16T07:36:05.856080+0000","last_undegraded":"2024-09-16T07:36:05.856080+0000","last_fullsized":"2024-09-16T07:36:05.856080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:58.216679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854581+0000","last_change":"2024-09-16T07:36:04.850083+0000","last_active":"2024-09-16T07:36:05.854581+0000","last_peered":"2024-09-16T07:36:05.854581+0000","last_clean":"2024-09-16T07:36:05.854581+0000","last_became_active":"2024-09-16T07:36:04.849731+0000","last_became_peered":"2024-09-16T07:36:04.849731+0000","last_unstale":"2024-09-16T07:36:05.854581+0000","last_undegraded":"2024-09-16T07:36:05.854581+0000","last_fullsized":"2024-09-16T07:36:05.854581+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:08.133005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855297+0000","last_change":"2024-09-16T07:36:03.848269+0000","last_active":"2024-09-16T07:36:05.855297+0000","last_peered":"2024-09-16T07:36:05.855297+0000","last_clean":"2024-09-16T07:36:05.855297+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.855297+0000","last_undegraded":"2024-09-16T07:36:05.855297+0000","last_fullsized":"2024-09-16T07:36:05.855297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:06.554515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849468+0000","last_change":"2024-09-16T07:36:04.859275+0000","last_active":"2024-09-16T07:36:05.849468+0000","last_peered":"2024-09-16T07:36:05.849468+0000","last_clean":"2024-09-16T07:36:05.849468+0000","last_became_active":"2024-09-16T07:36:04.859013+0000","last_became_peered":"2024-09-16T07:36:04.859013+0000","last_unstale":"2024-09-16T07:36:05.849468+0000","last_undegraded":"2024-09-16T07:36:05.849468+0000","last_fullsized":"2024-09-16T07:36:05.849468+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:42.011961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856499+0000","last_change":"2024-09-16T07:36:03.847015+0000","last_active":"2024-09-16T07:36:05.856499+0000","last_peered":"2024-09-16T07:36:05.856499+0000","last_clean":"2024-09-16T07:36:05.856499+0000","last_became_active":"2024-09-16T07:36:03.846719+0000","last_became_peered":"2024-09-16T07:36:03.846719+0000","last_unstale":"2024-09-16T07:36:05.856499+0000","last_undegraded":"2024-09-16T07:36:05.856499+0000","last_fullsized":"2024-09-16T07:36:05.856499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:46.177351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974797+0000","last_change":"2024-09-16T07:36:04.853837+0000","last_active":"2024-09-16T07:36:06.974797+0000","last_peered":"2024-09-16T07:36:06.974797+0000","last_clean":"2024-09-16T07:36:06.974797+0000","last_became_active":"2024-09-16T07:36:04.853492+0000","last_became_peered":"2024-09-16T07:36:04.853492+0000","last_unstale":"2024-09-16T07:36:06.974797+0000","last_undegraded":"2024-09-16T07:36:06.974797+0000","last_fullsized":"2024-09-16T07:36:06.974797+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:05.954007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856245+0000","last_change":"2024-09-16T07:36:03.854810+0000","last_active":"2024-09-16T07:36:05.856245+0000","last_peered":"2024-09-16T07:36:05.856245+0000","last_clean":"2024-09-16T07:36:05.856245+0000","last_became_active":"2024-09-16T07:36:03.854533+0000","last_became_peered":"2024-09-16T07:36:03.854533+0000","last_unstale":"2024-09-16T07:36:05.856245+0000","last_undegraded":"2024-09-16T07:36:05.856245+0000","last_fullsized":"2024-09-16T07:36:05.856245+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:23.851837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063875+0000","last_change":"2024-09-16T07:36:04.855496+0000","last_active":"2024-09-16T07:36:06.063875+0000","last_peered":"2024-09-16T07:36:06.063875+0000","last_clean":"2024-09-16T07:36:06.063875+0000","last_became_active":"2024-09-16T07:36:04.855167+0000","last_became_peered":"2024-09-16T07:36:04.855167+0000","last_unstale":"2024-09-16T07:36:06.063875+0000","last_undegraded":"2024-09-16T07:36:06.063875+0000","last_fullsized":"2024-09-16T07:36:06.063875+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:24.513277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"133'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858514+0000","last_change":"2024-09-16T07:36:03.857828+0000","last_active":"2024-09-16T07:36:05.858514+0000","last_peered":"2024-09-16T07:36:05.858514+0000","last_clean":"2024-09-16T07:36:05.858514+0000","last_became_active":"2024-09-16T07:36:03.853109+0000","last_became_peered":"2024-09-16T07:36:03.853109+0000","last_unstale":"2024-09-16T07:36:05.858514+0000","last_undegraded":"2024-09-16T07:36:05.858514+0000","last_fullsized":"2024-09-16T07:36:05.858514+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:47.645572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857004+0000","last_change":"2024-09-16T07:36:04.852909+0000","last_active":"2024-09-16T07:36:05.857004+0000","last_peered":"2024-09-16T07:36:05.857004+0000","last_clean":"2024-09-16T07:36:05.857004+0000","last_became_active":"2024-09-16T07:36:04.852496+0000","last_became_peered":"2024-09-16T07:36:04.852496+0000","last_unstale":"2024-09-16T07:36:05.857004+0000","last_undegraded":"2024-09-16T07:36:05.857004+0000","last_fullsized":"2024-09-16T07:36:05.857004+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:43.843163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854718+0000","last_change":"2024-09-16T07:36:03.853605+0000","last_active":"2024-09-16T07:36:05.854718+0000","last_peered":"2024-09-16T07:36:05.854718+0000","last_clean":"2024-09-16T07:36:05.854718+0000","last_became_active":"2024-09-16T07:36:03.853349+0000","last_became_peered":"2024-09-16T07:36:03.853349+0000","last_unstale":"2024-09-16T07:36:05.854718+0000","last_undegraded":"2024-09-16T07:36:05.854718+0000","last_fullsized":"2024-09-16T07:36:05.854718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:29.592986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978353+0000","last_change":"2024-09-16T07:36:04.851823+0000","last_active":"2024-09-16T07:36:06.978353+0000","last_peered":"2024-09-16T07:36:06.978353+0000","last_clean":"2024-09-16T07:36:06.978353+0000","last_became_active":"2024-09-16T07:36:04.851171+0000","last_became_peered":"2024-09-16T07:36:04.851171+0000","last_unstale":"2024-09-16T07:36:06.978353+0000","last_undegraded":"2024-09-16T07:36:06.978353+0000","last_fullsized":"2024-09-16T07:36:06.978353+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:40.887508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857065+0000","last_change":"2024-09-16T07:36:03.851730+0000","last_active":"2024-09-16T07:36:05.857065+0000","last_peered":"2024-09-16T07:36:05.857065+0000","last_clean":"2024-09-16T07:36:05.857065+0000","last_became_active":"2024-09-16T07:36:03.851457+0000","last_became_peered":"2024-09-16T07:36:03.851457+0000","last_unstale":"2024-09-16T07:36:05.857065+0000","last_undegraded":"2024-09-16T07:36:05.857065+0000","last_fullsized":"2024-09-16T07:36:05.857065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:37.222531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849662+0000","last_change":"2024-09-16T07:36:04.850230+0000","last_active":"2024-09-16T07:36:05.849662+0000","last_peered":"2024-09-16T07:36:05.849662+0000","last_clean":"2024-09-16T07:36:05.849662+0000","last_became_active":"2024-09-16T07:36:04.849632+0000","last_became_peered":"2024-09-16T07:36:04.849632+0000","last_unstale":"2024-09-16T07:36:05.849662+0000","last_undegraded":"2024-09-16T07:36:05.849662+0000","last_fullsized":"2024-09-16T07:36:05.849662+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:44:05.734690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063468+0000","last_change":"2024-09-16T07:36:03.857845+0000","last_active":"2024-09-16T07:36:06.063468+0000","last_peered":"2024-09-16T07:36:06.063468+0000","last_clean":"2024-09-16T07:36:06.063468+0000","last_became_active":"2024-09-16T07:36:03.857366+0000","last_became_peered":"2024-09-16T07:36:03.857366+0000","last_unstale":"2024-09-16T07:36:06.063468+0000","last_undegraded":"2024-09-16T07:36:06.063468+0000","last_fullsized":"2024-09-16T07:36:06.063468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:36.413668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849796+0000","last_change":"2024-09-16T07:36:04.866976+0000","last_active":"2024-09-16T07:36:05.849796+0000","last_peered":"2024-09-16T07:36:05.849796+0000","last_clean":"2024-09-16T07:36:05.849796+0000","last_became_active":"2024-09-16T07:36:04.866659+0000","last_became_peered":"2024-09-16T07:36:04.866659+0000","last_unstale":"2024-09-16T07:36:05.849796+0000","last_undegraded":"2024-09-16T07:36:05.849796+0000","last_fullsized":"2024-09-16T07:36:05.849796+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:21.257459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849573+0000","last_change":"2024-09-16T07:36:03.853643+0000","last_active":"2024-09-16T07:36:05.849573+0000","last_peered":"2024-09-16T07:36:05.849573+0000","last_clean":"2024-09-16T07:36:05.849573+0000","last_became_active":"2024-09-16T07:36:03.851881+0000","last_became_peered":"2024-09-16T07:36:03.851881+0000","last_unstale":"2024-09-16T07:36:05.849573+0000","last_undegraded":"2024-09-16T07:36:05.849573+0000","last_fullsized":"2024-09-16T07:36:05.849573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:47.645572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855281+0000","last_change":"2024-09-16T07:36:04.863145+0000","last_active":"2024-09-16T07:36:05.855281+0000","last_peered":"2024-09-16T07:36:05.855281+0000","last_clean":"2024-09-16T07:36:05.855281+0000","last_became_active":"2024-09-16T07:36:04.862893+0000","last_became_peered":"2024-09-16T07:36:04.862893+0000","last_unstale":"2024-09-16T07:36:05.855281+0000","last_undegraded":"2024-09-16T07:36:05.855281+0000","last_fullsized":"2024-09-16T07:36:05.855281+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:26.614434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854639+0000","last_change":"2024-09-16T07:36:03.851573+0000","last_active":"2024-09-16T07:36:05.854639+0000","last_peered":"2024-09-16T07:36:05.854639+0000","last_clean":"2024-09-16T07:36:05.854639+0000","last_became_active":"2024-09-16T07:36:03.851286+0000","last_became_peered":"2024-09-16T07:36:03.851286+0000","last_unstale":"2024-09-16T07:36:05.854639+0000","last_undegraded":"2024-09-16T07:36:05.854639+0000","last_fullsized":"2024-09-16T07:36:05.854639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.290737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856129+0000","last_change":"2024-09-16T07:36:04.858119+0000","last_active":"2024-09-16T07:36:05.856129+0000","last_peered":"2024-09-16T07:36:05.856129+0000","last_clean":"2024-09-16T07:36:05.856129+0000","last_became_active":"2024-09-16T07:36:04.855972+0000","last_became_peered":"2024-09-16T07:36:04.855972+0000","last_unstale":"2024-09-16T07:36:05.856129+0000","last_undegraded":"2024-09-16T07:36:05.856129+0000","last_fullsized":"2024-09-16T07:36:05.856129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:40.599143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854647+0000","last_change":"2024-09-16T07:36:03.854300+0000","last_active":"2024-09-16T07:36:05.854647+0000","last_peered":"2024-09-16T07:36:05.854647+0000","last_clean":"2024-09-16T07:36:05.854647+0000","last_became_active":"2024-09-16T07:36:03.853971+0000","last_became_peered":"2024-09-16T07:36:03.853971+0000","last_unstale":"2024-09-16T07:36:05.854647+0000","last_undegraded":"2024-09-16T07:36:05.854647+0000","last_fullsized":"2024-09-16T07:36:05.854647+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:25.287389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978598+0000","last_change":"2024-09-16T07:36:04.843292+0000","last_active":"2024-09-16T07:36:06.978598+0000","last_peered":"2024-09-16T07:36:06.978598+0000","last_clean":"2024-09-16T07:36:06.978598+0000","last_became_active":"2024-09-16T07:36:04.843022+0000","last_became_peered":"2024-09-16T07:36:04.843022+0000","last_unstale":"2024-09-16T07:36:06.978598+0000","last_undegraded":"2024-09-16T07:36:06.978598+0000","last_fullsized":"2024-09-16T07:36:06.978598+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:38.146552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849785+0000","last_change":"2024-09-16T07:36:03.855144+0000","last_active":"2024-09-16T07:36:05.849785+0000","last_peered":"2024-09-16T07:36:05.849785+0000","last_clean":"2024-09-16T07:36:05.849785+0000","last_became_active":"2024-09-16T07:36:03.854840+0000","last_became_peered":"2024-09-16T07:36:03.854840+0000","last_unstale":"2024-09-16T07:36:05.849785+0000","last_undegraded":"2024-09-16T07:36:05.849785+0000","last_fullsized":"2024-09-16T07:36:05.849785+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:17.990182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":314,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974366+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:36:06.974366+0000","last_peered":"2024-09-16T07:36:06.974366+0000","last_clean":"2024-09-16T07:36:06.974366+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:36:06.974366+0000","last_undegraded":"2024-09-16T07:36:06.974366+0000","last_fullsized":"2024-09-16T07:36:06.974366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476903,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131296,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240544,"statfs":{"total":96636764160,"available":96502317056,"internally_reserved":0,"allocated":1650688,"data_stored":724795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476903,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131300,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240540,"statfs":{"total":96636764160,"available":96502312960,"internally_reserved":0,"allocated":1654784,"data_stored":725397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476903,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127728,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244112,"statfs":{"total":96636764160,"available":96505970688,"internally_reserved":0,"allocated":2256896,"data_stored":1316727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476902,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145200,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226640,"statfs":{"total":96636764160,"available":96488079360,"internally_reserved":0,"allocated":1667072,"data_stored":726393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127140,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244700,"statfs":{"total":96636764160,"available":96506572800,"internally_reserved":0,"allocated":1650688,"data_stored":724759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509606,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159076,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212764,"statfs":{"total":96636764160,"available":96473870336,"internally_reserved":0,"allocated":1646592,"data_stored":724725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161216966},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509606,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119404,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252436,"statfs":{"total":96636764160,"available":96514494464,"internally_reserved":0,"allocated":2252800,"data_stored":1315835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542308,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135440,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236400,"statfs":{"total":96636764160,"available":96498073600,"internally_reserved":0,"allocated":1634304,"data_stored":718341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":137034181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:36:13.734 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 -- 172.21.15.71:0/1992889483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec15f5a0 msgr2=0x7f6dec0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 --2- 172.21.15.71:0/1992889483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec15f5a0 0x7f6dec0b2440 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0cfbb0 tx=0x7f6ddc02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 -- 172.21.15.71:0/1992889483 shutdown_connections 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 --2- 172.21.15.71:0/1992889483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dec0ad270 0x7f6dec073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 --2- 172.21.15.71:0/1992889483 >> v2:172.21.15.177:3300/0 conn(0x7f6dec0ccf00 0x7f6dec0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 --2- 172.21.15.71:0/1992889483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec15f5a0 0x7f6dec0b2440 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 -- 172.21.15.71:0/1992889483 >> 172.21.15.71:0/1992889483 conn(0x7f6dec0b1950 msgr2=0x7f6dec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 -- 172.21.15.71:0/1992889483 shutdown_connections 2024-09-16T07:36:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.903+0000 7f6df1c00640 1 -- 172.21.15.71:0/1992889483 wait complete. 2024-09-16T07:36:13.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.904+0000 7f6df1c00640 1 Processor -- start 2024-09-16T07:36:13.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.905+0000 7f6df1c00640 1 -- start start 2024-09-16T07:36:13.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6df1c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dec0ad270 0x7f6dec0d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6df1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec0ccf00 0x7f6dec146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6df1c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dec15f5a0 0x7f6dec0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6df1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec074880 con 0x7f6dec0ccf00 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6df1c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec074700 con 0x7f6dec15f5a0 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6df1c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6dec06f160 con 0x7f6dec0ad270 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6dea000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec0ccf00 0x7f6dec146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6dea000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec0ccf00 0x7f6dec146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40518/0 (socket says 172.21.15.71:40518) 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6dea000640 1 -- 172.21.15.71:0/789578770 learned_addr learned my addr 172.21.15.71:0/789578770 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6deaa00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dec0ad270 0x7f6dec0d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6deb400640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dec15f5a0 0x7f6dec0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6dea000640 1 -- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dec0ad270 msgr2=0x7f6dec0d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:13.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6dea000640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dec0ad270 0x7f6dec0d04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:13.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6dea000640 1 -- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dec15f5a0 msgr2=0x7f6dec0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:13.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6dea000640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dec15f5a0 0x7f6dec0cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:13.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6dea000640 1 -- 172.21.15.71:0/789578770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec0caa30 con 0x7f6dec0ccf00 2024-09-16T07:36:13.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.906+0000 7f6deb400640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dec15f5a0 0x7f6dec0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:36:13.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.907+0000 7f6dea000640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec0ccf00 0x7f6dec146530 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800d980 tx=0x7f6dd800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:13.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.907+0000 7f6de2a00640 1 -- 172.21.15.71:0/789578770 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8017070 con 0x7f6dec0ccf00 2024-09-16T07:36:13.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.907+0000 7f6de2a00640 1 -- 172.21.15.71:0/789578770 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dd8009d20 con 0x7f6dec0ccf00 2024-09-16T07:36:13.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.907+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec0cad10 con 0x7f6dec0ccf00 2024-09-16T07:36:13.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.907+0000 7f6de2a00640 1 -- 172.21.15.71:0/789578770 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80106c0 con 0x7f6dec0ccf00 2024-09-16T07:36:13.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.907+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec0cb2d0 con 0x7f6dec0ccf00 2024-09-16T07:36:13.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.908+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dec0cbaa0 con 0x7f6dec0ccf00 2024-09-16T07:36:13.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.909+0000 7f6de2a00640 1 -- 172.21.15.71:0/789578770 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f6dd8010860 con 0x7f6dec0ccf00 2024-09-16T07:36:13.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.911+0000 7f6de2a00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6dcc07a6b0 0x7f6dcc07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:13.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.911+0000 7f6de2a00640 1 -- 172.21.15.71:0/789578770 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f6dd8099df0 con 0x7f6dec0ccf00 2024-09-16T07:36:13.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.911+0000 7f6deaa00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6dcc07a6b0 0x7f6dcc07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:13.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.912+0000 7f6deaa00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6dcc07a6b0 0x7f6dcc07cb70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc033000 tx=0x7f6ddc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:13.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:13.915+0000 7f6de2a00640 1 -- 172.21.15.71:0/789578770 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6dd8063e30 con 0x7f6dec0ccf00 2024-09-16T07:36:14.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.098+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6dec155e50 con 0x7f6dcc07a6b0 2024-09-16T07:36:14.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.103+0000 7f6de2a00640 1 -- 172.21.15.71:0/789578770 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252487 (secure 0 0 0) 0x7f6dec155e50 con 0x7f6dcc07a6b0 2024-09-16T07:36:14.106 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:14.107 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6dcc07a6b0 msgr2=0x7f6dcc07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6dcc07a6b0 0x7f6dcc07cb70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc033000 tx=0x7f6ddc004730 comp rx=0 tx=0).stop 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec0ccf00 msgr2=0x7f6dec146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec0ccf00 0x7f6dec146530 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800d980 tx=0x7f6dd800de40 comp rx=0 tx=0).stop 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 shutdown_connections 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6dcc07a6b0 0x7f6dcc07cb70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dec15f5a0 0x7f6dec0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec0ccf00 0x7f6dec146530 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 --2- 172.21.15.71:0/789578770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dec0ad270 0x7f6dec0d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 >> 172.21.15.71:0/789578770 conn(0x7f6dec0b1950 msgr2=0x7f6dec09d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:14.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 shutdown_connections 2024-09-16T07:36:14.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.108+0000 7f6df1c00640 1 -- 172.21.15.71:0/789578770 wait complete. 2024-09-16T07:36:14.124 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-16T07:36:12.627228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076600,"kb_used_data":14088,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898120,"statfs":{"total":773094113280,"available":771991674880,"internally_reserved":0,"allocated":14426112,"data_stored":6983356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1087819312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002218"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.853420+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.853157+0000","last_became_peered":"2024-09-16T07:36:04.853157+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:29.795783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063845+0000","last_change":"2024-09-16T07:36:04.855589+0000","last_active":"2024-09-16T07:36:06.063845+0000","last_peered":"2024-09-16T07:36:06.063845+0000","last_clean":"2024-09-16T07:36:06.063845+0000","last_became_active":"2024-09-16T07:36:04.855323+0000","last_became_peered":"2024-09-16T07:36:04.855323+0000","last_unstale":"2024-09-16T07:36:06.063845+0000","last_undegraded":"2024-09-16T07:36:06.063845+0000","last_fullsized":"2024-09-16T07:36:06.063845+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:53.457746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978230+0000","last_change":"2024-09-16T07:36:04.852096+0000","last_active":"2024-09-16T07:36:06.978230+0000","last_peered":"2024-09-16T07:36:06.978230+0000","last_clean":"2024-09-16T07:36:06.978230+0000","last_became_active":"2024-09-16T07:36:04.851548+0000","last_became_peered":"2024-09-16T07:36:04.851548+0000","last_unstale":"2024-09-16T07:36:06.978230+0000","last_undegraded":"2024-09-16T07:36:06.978230+0000","last_fullsized":"2024-09-16T07:36:06.978230+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:35.503529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855470+0000","last_change":"2024-09-16T07:36:04.859510+0000","last_active":"2024-09-16T07:36:05.855470+0000","last_peered":"2024-09-16T07:36:05.855470+0000","last_clean":"2024-09-16T07:36:05.855470+0000","last_became_active":"2024-09-16T07:36:04.859175+0000","last_became_peered":"2024-09-16T07:36:04.859175+0000","last_unstale":"2024-09-16T07:36:05.855470+0000","last_undegraded":"2024-09-16T07:36:05.855470+0000","last_fullsized":"2024-09-16T07:36:05.855470+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:42:35.105087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858303+0000","last_change":"2024-09-16T07:36:04.855716+0000","last_active":"2024-09-16T07:36:05.858303+0000","last_peered":"2024-09-16T07:36:05.858303+0000","last_clean":"2024-09-16T07:36:05.858303+0000","last_became_active":"2024-09-16T07:36:04.853373+0000","last_became_peered":"2024-09-16T07:36:04.853373+0000","last_unstale":"2024-09-16T07:36:05.858303+0000","last_undegraded":"2024-09-16T07:36:05.858303+0000","last_fullsized":"2024-09-16T07:36:05.858303+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:39.830389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849450+0000","last_change":"2024-09-16T07:36:04.851280+0000","last_active":"2024-09-16T07:36:05.849450+0000","last_peered":"2024-09-16T07:36:05.849450+0000","last_clean":"2024-09-16T07:36:05.849450+0000","last_became_active":"2024-09-16T07:36:04.851020+0000","last_became_peered":"2024-09-16T07:36:04.851020+0000","last_unstale":"2024-09-16T07:36:05.849450+0000","last_undegraded":"2024-09-16T07:36:05.849450+0000","last_fullsized":"2024-09-16T07:36:05.849450+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:38.230442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858195+0000","last_change":"2024-09-16T07:36:04.844492+0000","last_active":"2024-09-16T07:36:05.858195+0000","last_peered":"2024-09-16T07:36:05.858195+0000","last_clean":"2024-09-16T07:36:05.858195+0000","last_became_active":"2024-09-16T07:36:04.843818+0000","last_became_peered":"2024-09-16T07:36:04.843818+0000","last_unstale":"2024-09-16T07:36:05.858195+0000","last_undegraded":"2024-09-16T07:36:05.858195+0000","last_fullsized":"2024-09-16T07:36:05.858195+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:38.924508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858071+0000","last_change":"2024-09-16T07:36:04.845663+0000","last_active":"2024-09-16T07:36:05.858071+0000","last_peered":"2024-09-16T07:36:05.858071+0000","last_clean":"2024-09-16T07:36:05.858071+0000","last_became_active":"2024-09-16T07:36:04.844565+0000","last_became_peered":"2024-09-16T07:36:04.844565+0000","last_unstale":"2024-09-16T07:36:05.858071+0000","last_undegraded":"2024-09-16T07:36:05.858071+0000","last_fullsized":"2024-09-16T07:36:05.858071+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:43.360504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978331+0000","last_change":"2024-09-16T07:36:04.859110+0000","last_active":"2024-09-16T07:36:06.978331+0000","last_peered":"2024-09-16T07:36:06.978331+0000","last_clean":"2024-09-16T07:36:06.978331+0000","last_became_active":"2024-09-16T07:36:04.858813+0000","last_became_peered":"2024-09-16T07:36:04.858813+0000","last_unstale":"2024-09-16T07:36:06.978331+0000","last_undegraded":"2024-09-16T07:36:06.978331+0000","last_fullsized":"2024-09-16T07:36:06.978331+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:54.655354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063831+0000","last_change":"2024-09-16T07:36:04.851248+0000","last_active":"2024-09-16T07:36:06.063831+0000","last_peered":"2024-09-16T07:36:06.063831+0000","last_clean":"2024-09-16T07:36:06.063831+0000","last_became_active":"2024-09-16T07:36:04.850709+0000","last_became_peered":"2024-09-16T07:36:04.850709+0000","last_unstale":"2024-09-16T07:36:06.063831+0000","last_undegraded":"2024-09-16T07:36:06.063831+0000","last_fullsized":"2024-09-16T07:36:06.063831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:07.411898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855892+0000","last_change":"2024-09-16T07:36:04.854097+0000","last_active":"2024-09-16T07:36:05.855892+0000","last_peered":"2024-09-16T07:36:05.855892+0000","last_clean":"2024-09-16T07:36:05.855892+0000","last_became_active":"2024-09-16T07:36:04.853827+0000","last_became_peered":"2024-09-16T07:36:04.853827+0000","last_unstale":"2024-09-16T07:36:05.855892+0000","last_undegraded":"2024-09-16T07:36:05.855892+0000","last_fullsized":"2024-09-16T07:36:05.855892+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:21.875425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850041+0000","last_change":"2024-09-16T07:36:04.853331+0000","last_active":"2024-09-16T07:36:05.850041+0000","last_peered":"2024-09-16T07:36:05.850041+0000","last_clean":"2024-09-16T07:36:05.850041+0000","last_became_active":"2024-09-16T07:36:04.853085+0000","last_became_peered":"2024-09-16T07:36:04.853085+0000","last_unstale":"2024-09-16T07:36:05.850041+0000","last_undegraded":"2024-09-16T07:36:05.850041+0000","last_fullsized":"2024-09-16T07:36:05.850041+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:10.939637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855153+0000","last_change":"2024-09-16T07:36:04.848716+0000","last_active":"2024-09-16T07:36:05.855153+0000","last_peered":"2024-09-16T07:36:05.855153+0000","last_clean":"2024-09-16T07:36:05.855153+0000","last_became_active":"2024-09-16T07:36:04.848457+0000","last_became_peered":"2024-09-16T07:36:04.848457+0000","last_unstale":"2024-09-16T07:36:05.855153+0000","last_undegraded":"2024-09-16T07:36:05.855153+0000","last_fullsized":"2024-09-16T07:36:05.855153+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:21.475235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856556+0000","last_change":"2024-09-16T07:36:04.859096+0000","last_active":"2024-09-16T07:36:05.856556+0000","last_peered":"2024-09-16T07:36:05.856556+0000","last_clean":"2024-09-16T07:36:05.856556+0000","last_became_active":"2024-09-16T07:36:04.858812+0000","last_became_peered":"2024-09-16T07:36:04.858812+0000","last_unstale":"2024-09-16T07:36:05.856556+0000","last_undegraded":"2024-09-16T07:36:05.856556+0000","last_fullsized":"2024-09-16T07:36:05.856556+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:59.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855973+0000","last_change":"2024-09-16T07:36:04.862176+0000","last_active":"2024-09-16T07:36:05.855973+0000","last_peered":"2024-09-16T07:36:05.855973+0000","last_clean":"2024-09-16T07:36:05.855973+0000","last_became_active":"2024-09-16T07:36:04.860550+0000","last_became_peered":"2024-09-16T07:36:04.860550+0000","last_unstale":"2024-09-16T07:36:05.855973+0000","last_undegraded":"2024-09-16T07:36:05.855973+0000","last_fullsized":"2024-09-16T07:36:05.855973+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:10.040307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854370+0000","last_change":"2024-09-16T07:36:04.855363+0000","last_active":"2024-09-16T07:36:05.854370+0000","last_peered":"2024-09-16T07:36:05.854370+0000","last_clean":"2024-09-16T07:36:05.854370+0000","last_became_active":"2024-09-16T07:36:04.855013+0000","last_became_peered":"2024-09-16T07:36:04.855013+0000","last_unstale":"2024-09-16T07:36:05.854370+0000","last_undegraded":"2024-09-16T07:36:05.854370+0000","last_fullsized":"2024-09-16T07:36:05.854370+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:13.040194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978545+0000","last_change":"2024-09-16T07:36:04.859278+0000","last_active":"2024-09-16T07:36:06.978545+0000","last_peered":"2024-09-16T07:36:06.978545+0000","last_clean":"2024-09-16T07:36:06.978545+0000","last_became_active":"2024-09-16T07:36:04.858925+0000","last_became_peered":"2024-09-16T07:36:04.858925+0000","last_unstale":"2024-09-16T07:36:06.978545+0000","last_undegraded":"2024-09-16T07:36:06.978545+0000","last_fullsized":"2024-09-16T07:36:06.978545+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:38.241305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855025+0000","last_change":"2024-09-16T07:36:04.862436+0000","last_active":"2024-09-16T07:36:05.855025+0000","last_peered":"2024-09-16T07:36:05.855025+0000","last_clean":"2024-09-16T07:36:05.855025+0000","last_became_active":"2024-09-16T07:36:04.862125+0000","last_became_peered":"2024-09-16T07:36:04.862125+0000","last_unstale":"2024-09-16T07:36:05.855025+0000","last_undegraded":"2024-09-16T07:36:05.855025+0000","last_fullsized":"2024-09-16T07:36:05.855025+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:38.787658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063678+0000","last_change":"2024-09-16T07:36:04.857741+0000","last_active":"2024-09-16T07:36:06.063678+0000","last_peered":"2024-09-16T07:36:06.063678+0000","last_clean":"2024-09-16T07:36:06.063678+0000","last_became_active":"2024-09-16T07:36:04.854444+0000","last_became_peered":"2024-09-16T07:36:04.854444+0000","last_unstale":"2024-09-16T07:36:06.063678+0000","last_undegraded":"2024-09-16T07:36:06.063678+0000","last_fullsized":"2024-09-16T07:36:06.063678+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:15.100689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856475+0000","last_change":"2024-09-16T07:36:04.869720+0000","last_active":"2024-09-16T07:36:05.856475+0000","last_peered":"2024-09-16T07:36:05.856475+0000","last_clean":"2024-09-16T07:36:05.856475+0000","last_became_active":"2024-09-16T07:36:04.868956+0000","last_became_peered":"2024-09-16T07:36:04.868956+0000","last_unstale":"2024-09-16T07:36:05.856475+0000","last_undegraded":"2024-09-16T07:36:05.856475+0000","last_fullsized":"2024-09-16T07:36:05.856475+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:29.852710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849663+0000","last_change":"2024-09-16T07:36:04.855006+0000","last_active":"2024-09-16T07:36:05.849663+0000","last_peered":"2024-09-16T07:36:05.849663+0000","last_clean":"2024-09-16T07:36:05.849663+0000","last_became_active":"2024-09-16T07:36:04.854529+0000","last_became_peered":"2024-09-16T07:36:04.854529+0000","last_unstale":"2024-09-16T07:36:05.849663+0000","last_undegraded":"2024-09-16T07:36:05.849663+0000","last_fullsized":"2024-09-16T07:36:05.849663+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:04.807416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063836+0000","last_change":"2024-09-16T07:36:04.845962+0000","last_active":"2024-09-16T07:36:06.063836+0000","last_peered":"2024-09-16T07:36:06.063836+0000","last_clean":"2024-09-16T07:36:06.063836+0000","last_became_active":"2024-09-16T07:36:04.845664+0000","last_became_peered":"2024-09-16T07:36:04.845664+0000","last_unstale":"2024-09-16T07:36:06.063836+0000","last_undegraded":"2024-09-16T07:36:06.063836+0000","last_fullsized":"2024-09-16T07:36:06.063836+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:32.167063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974830+0000","last_change":"2024-09-16T07:36:04.853123+0000","last_active":"2024-09-16T07:36:06.974830+0000","last_peered":"2024-09-16T07:36:06.974830+0000","last_clean":"2024-09-16T07:36:06.974830+0000","last_became_active":"2024-09-16T07:36:04.852684+0000","last_became_peered":"2024-09-16T07:36:04.852684+0000","last_unstale":"2024-09-16T07:36:06.974830+0000","last_undegraded":"2024-09-16T07:36:06.974830+0000","last_fullsized":"2024-09-16T07:36:06.974830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:09.092561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854174+0000","last_change":"2024-09-16T07:36:04.865540+0000","last_active":"2024-09-16T07:36:05.854174+0000","last_peered":"2024-09-16T07:36:05.854174+0000","last_clean":"2024-09-16T07:36:05.854174+0000","last_became_active":"2024-09-16T07:36:04.865232+0000","last_became_peered":"2024-09-16T07:36:04.865232+0000","last_unstale":"2024-09-16T07:36:05.854174+0000","last_undegraded":"2024-09-16T07:36:05.854174+0000","last_fullsized":"2024-09-16T07:36:05.854174+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:24.118557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855774+0000","last_change":"2024-09-16T07:36:04.857205+0000","last_active":"2024-09-16T07:36:05.855774+0000","last_peered":"2024-09-16T07:36:05.855774+0000","last_clean":"2024-09-16T07:36:05.855774+0000","last_became_active":"2024-09-16T07:36:04.856823+0000","last_became_peered":"2024-09-16T07:36:04.856823+0000","last_unstale":"2024-09-16T07:36:05.855774+0000","last_undegraded":"2024-09-16T07:36:05.855774+0000","last_fullsized":"2024-09-16T07:36:05.855774+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:16.355699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064085+0000","last_change":"2024-09-16T07:36:04.850897+0000","last_active":"2024-09-16T07:36:06.064085+0000","last_peered":"2024-09-16T07:36:06.064085+0000","last_clean":"2024-09-16T07:36:06.064085+0000","last_became_active":"2024-09-16T07:36:04.850635+0000","last_became_peered":"2024-09-16T07:36:04.850635+0000","last_unstale":"2024-09-16T07:36:06.064085+0000","last_undegraded":"2024-09-16T07:36:06.064085+0000","last_fullsized":"2024-09-16T07:36:06.064085+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:05:38.194427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857379+0000","last_change":"2024-09-16T07:36:04.855205+0000","last_active":"2024-09-16T07:36:05.857379+0000","last_peered":"2024-09-16T07:36:05.857379+0000","last_clean":"2024-09-16T07:36:05.857379+0000","last_became_active":"2024-09-16T07:36:04.854356+0000","last_became_peered":"2024-09-16T07:36:04.854356+0000","last_unstale":"2024-09-16T07:36:05.857379+0000","last_undegraded":"2024-09-16T07:36:05.857379+0000","last_fullsized":"2024-09-16T07:36:05.857379+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:23.776106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974810+0000","last_change":"2024-09-16T07:36:04.854709+0000","last_active":"2024-09-16T07:36:06.974810+0000","last_peered":"2024-09-16T07:36:06.974810+0000","last_clean":"2024-09-16T07:36:06.974810+0000","last_became_active":"2024-09-16T07:36:04.854430+0000","last_became_peered":"2024-09-16T07:36:04.854430+0000","last_unstale":"2024-09-16T07:36:06.974810+0000","last_undegraded":"2024-09-16T07:36:06.974810+0000","last_fullsized":"2024-09-16T07:36:06.974810+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:33.380575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063430+0000","last_change":"2024-09-16T07:36:04.849621+0000","last_active":"2024-09-16T07:36:06.063430+0000","last_peered":"2024-09-16T07:36:06.063430+0000","last_clean":"2024-09-16T07:36:06.063430+0000","last_became_active":"2024-09-16T07:36:04.849325+0000","last_became_peered":"2024-09-16T07:36:04.849325+0000","last_unstale":"2024-09-16T07:36:06.063430+0000","last_undegraded":"2024-09-16T07:36:06.063430+0000","last_fullsized":"2024-09-16T07:36:06.063430+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:20.048326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063880+0000","last_change":"2024-09-16T07:36:04.840820+0000","last_active":"2024-09-16T07:36:06.063880+0000","last_peered":"2024-09-16T07:36:06.063880+0000","last_clean":"2024-09-16T07:36:06.063880+0000","last_became_active":"2024-09-16T07:36:04.839397+0000","last_became_peered":"2024-09-16T07:36:04.839397+0000","last_unstale":"2024-09-16T07:36:06.063880+0000","last_undegraded":"2024-09-16T07:36:06.063880+0000","last_fullsized":"2024-09-16T07:36:06.063880+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:01.349093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974555+0000","last_change":"2024-09-16T07:36:04.861314+0000","last_active":"2024-09-16T07:36:06.974555+0000","last_peered":"2024-09-16T07:36:06.974555+0000","last_clean":"2024-09-16T07:36:06.974555+0000","last_became_active":"2024-09-16T07:36:04.861052+0000","last_became_peered":"2024-09-16T07:36:04.861052+0000","last_unstale":"2024-09-16T07:36:06.974555+0000","last_undegraded":"2024-09-16T07:36:06.974555+0000","last_fullsized":"2024-09-16T07:36:06.974555+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:59.224590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978093+0000","last_change":"2024-09-16T07:36:04.841017+0000","last_active":"2024-09-16T07:36:06.978093+0000","last_peered":"2024-09-16T07:36:06.978093+0000","last_clean":"2024-09-16T07:36:06.978093+0000","last_became_active":"2024-09-16T07:36:04.840692+0000","last_became_peered":"2024-09-16T07:36:04.840692+0000","last_unstale":"2024-09-16T07:36:06.978093+0000","last_undegraded":"2024-09-16T07:36:06.978093+0000","last_fullsized":"2024-09-16T07:36:06.978093+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.829657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857460+0000","last_change":"2024-09-16T07:36:04.859595+0000","last_active":"2024-09-16T07:36:05.857460+0000","last_peered":"2024-09-16T07:36:05.857460+0000","last_clean":"2024-09-16T07:36:05.857460+0000","last_became_active":"2024-09-16T07:36:04.856512+0000","last_became_peered":"2024-09-16T07:36:04.856512+0000","last_unstale":"2024-09-16T07:36:05.857460+0000","last_undegraded":"2024-09-16T07:36:05.857460+0000","last_fullsized":"2024-09-16T07:36:05.857460+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:25.701980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"132'4","reported_seq":19,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858698+0000","last_change":"2024-09-16T07:36:03.855477+0000","last_active":"2024-09-16T07:36:05.858698+0000","last_peered":"2024-09-16T07:36:05.858698+0000","last_clean":"2024-09-16T07:36:05.858698+0000","last_became_active":"2024-09-16T07:36:03.852255+0000","last_became_peered":"2024-09-16T07:36:03.852255+0000","last_unstale":"2024-09-16T07:36:05.858698+0000","last_undegraded":"2024-09-16T07:36:05.858698+0000","last_fullsized":"2024-09-16T07:36:05.858698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:17.990182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978337+0000","last_change":"2024-09-16T07:36:04.853865+0000","last_active":"2024-09-16T07:36:06.978337+0000","last_peered":"2024-09-16T07:36:06.978337+0000","last_clean":"2024-09-16T07:36:06.978337+0000","last_became_active":"2024-09-16T07:36:04.852369+0000","last_became_peered":"2024-09-16T07:36:04.852369+0000","last_unstale":"2024-09-16T07:36:06.978337+0000","last_undegraded":"2024-09-16T07:36:06.978337+0000","last_fullsized":"2024-09-16T07:36:06.978337+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:11.213364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978359+0000","last_change":"2024-09-16T07:36:03.853595+0000","last_active":"2024-09-16T07:36:06.978359+0000","last_peered":"2024-09-16T07:36:06.978359+0000","last_clean":"2024-09-16T07:36:06.978359+0000","last_became_active":"2024-09-16T07:36:03.853315+0000","last_became_peered":"2024-09-16T07:36:03.853315+0000","last_unstale":"2024-09-16T07:36:06.978359+0000","last_undegraded":"2024-09-16T07:36:06.978359+0000","last_fullsized":"2024-09-16T07:36:06.978359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:39.036062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978237+0000","last_change":"2024-09-16T07:36:04.849253+0000","last_active":"2024-09-16T07:36:06.978237+0000","last_peered":"2024-09-16T07:36:06.978237+0000","last_clean":"2024-09-16T07:36:06.978237+0000","last_became_active":"2024-09-16T07:36:04.848990+0000","last_became_peered":"2024-09-16T07:36:04.848990+0000","last_unstale":"2024-09-16T07:36:06.978237+0000","last_undegraded":"2024-09-16T07:36:06.978237+0000","last_fullsized":"2024-09-16T07:36:06.978237+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:57.540368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856585+0000","last_change":"2024-09-16T07:36:03.849019+0000","last_active":"2024-09-16T07:36:05.856585+0000","last_peered":"2024-09-16T07:36:05.856585+0000","last_clean":"2024-09-16T07:36:05.856585+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.856585+0000","last_undegraded":"2024-09-16T07:36:05.856585+0000","last_fullsized":"2024-09-16T07:36:05.856585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:53.651818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850191+0000","last_change":"2024-09-16T07:36:04.861211+0000","last_active":"2024-09-16T07:36:05.850191+0000","last_peered":"2024-09-16T07:36:05.850191+0000","last_clean":"2024-09-16T07:36:05.850191+0000","last_became_active":"2024-09-16T07:36:04.860347+0000","last_became_peered":"2024-09-16T07:36:04.860347+0000","last_unstale":"2024-09-16T07:36:05.850191+0000","last_undegraded":"2024-09-16T07:36:05.850191+0000","last_fullsized":"2024-09-16T07:36:05.850191+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:29.976789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063690+0000","last_change":"2024-09-16T07:36:03.853178+0000","last_active":"2024-09-16T07:36:06.063690+0000","last_peered":"2024-09-16T07:36:06.063690+0000","last_clean":"2024-09-16T07:36:06.063690+0000","last_became_active":"2024-09-16T07:36:03.852914+0000","last_became_peered":"2024-09-16T07:36:03.852914+0000","last_unstale":"2024-09-16T07:36:06.063690+0000","last_undegraded":"2024-09-16T07:36:06.063690+0000","last_fullsized":"2024-09-16T07:36:06.063690+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:09:30.868825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855929+0000","last_change":"2024-09-16T07:36:04.860375+0000","last_active":"2024-09-16T07:36:05.855929+0000","last_peered":"2024-09-16T07:36:05.855929+0000","last_clean":"2024-09-16T07:36:05.855929+0000","last_became_active":"2024-09-16T07:36:04.860110+0000","last_became_peered":"2024-09-16T07:36:04.860110+0000","last_unstale":"2024-09-16T07:36:05.855929+0000","last_undegraded":"2024-09-16T07:36:05.855929+0000","last_fullsized":"2024-09-16T07:36:05.855929+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:44.383231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063342+0000","last_change":"2024-09-16T07:36:03.848993+0000","last_active":"2024-09-16T07:36:06.063342+0000","last_peered":"2024-09-16T07:36:06.063342+0000","last_clean":"2024-09-16T07:36:06.063342+0000","last_became_active":"2024-09-16T07:36:03.848700+0000","last_became_peered":"2024-09-16T07:36:03.848700+0000","last_unstale":"2024-09-16T07:36:06.063342+0000","last_undegraded":"2024-09-16T07:36:06.063342+0000","last_fullsized":"2024-09-16T07:36:06.063342+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:54.220868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849654+0000","last_change":"2024-09-16T07:36:04.850135+0000","last_active":"2024-09-16T07:36:05.849654+0000","last_peered":"2024-09-16T07:36:05.849654+0000","last_clean":"2024-09-16T07:36:05.849654+0000","last_became_active":"2024-09-16T07:36:04.849830+0000","last_became_peered":"2024-09-16T07:36:04.849830+0000","last_unstale":"2024-09-16T07:36:05.849654+0000","last_undegraded":"2024-09-16T07:36:05.849654+0000","last_fullsized":"2024-09-16T07:36:05.849654+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:40.566705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856372+0000","last_change":"2024-09-16T07:36:03.855637+0000","last_active":"2024-09-16T07:36:05.856372+0000","last_peered":"2024-09-16T07:36:05.856372+0000","last_clean":"2024-09-16T07:36:05.856372+0000","last_became_active":"2024-09-16T07:36:03.855357+0000","last_became_peered":"2024-09-16T07:36:03.855357+0000","last_unstale":"2024-09-16T07:36:05.856372+0000","last_undegraded":"2024-09-16T07:36:05.856372+0000","last_fullsized":"2024-09-16T07:36:05.856372+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.392485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849437+0000","last_change":"2024-09-16T07:36:04.852032+0000","last_active":"2024-09-16T07:36:05.849437+0000","last_peered":"2024-09-16T07:36:05.849437+0000","last_clean":"2024-09-16T07:36:05.849437+0000","last_became_active":"2024-09-16T07:36:04.851770+0000","last_became_peered":"2024-09-16T07:36:04.851770+0000","last_unstale":"2024-09-16T07:36:05.849437+0000","last_undegraded":"2024-09-16T07:36:05.849437+0000","last_fullsized":"2024-09-16T07:36:05.849437+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:10.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978019+0000","last_change":"2024-09-16T07:36:03.858168+0000","last_active":"2024-09-16T07:36:06.978019+0000","last_peered":"2024-09-16T07:36:06.978019+0000","last_clean":"2024-09-16T07:36:06.978019+0000","last_became_active":"2024-09-16T07:36:03.857697+0000","last_became_peered":"2024-09-16T07:36:03.857697+0000","last_unstale":"2024-09-16T07:36:06.978019+0000","last_undegraded":"2024-09-16T07:36:06.978019+0000","last_fullsized":"2024-09-16T07:36:06.978019+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:53.424532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063343+0000","last_change":"2024-09-16T07:36:04.853058+0000","last_active":"2024-09-16T07:36:06.063343+0000","last_peered":"2024-09-16T07:36:06.063343+0000","last_clean":"2024-09-16T07:36:06.063343+0000","last_became_active":"2024-09-16T07:36:04.852535+0000","last_became_peered":"2024-09-16T07:36:04.852535+0000","last_unstale":"2024-09-16T07:36:06.063343+0000","last_undegraded":"2024-09-16T07:36:06.063343+0000","last_fullsized":"2024-09-16T07:36:06.063343+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:30:36.784633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063356+0000","last_change":"2024-09-16T07:36:03.846732+0000","last_active":"2024-09-16T07:36:06.063356+0000","last_peered":"2024-09-16T07:36:06.063356+0000","last_clean":"2024-09-16T07:36:06.063356+0000","last_became_active":"2024-09-16T07:36:03.845789+0000","last_became_peered":"2024-09-16T07:36:03.845789+0000","last_unstale":"2024-09-16T07:36:06.063356+0000","last_undegraded":"2024-09-16T07:36:06.063356+0000","last_fullsized":"2024-09-16T07:36:06.063356+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:53.374127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064066+0000","last_change":"2024-09-16T07:36:04.852160+0000","last_active":"2024-09-16T07:36:06.064066+0000","last_peered":"2024-09-16T07:36:06.064066+0000","last_clean":"2024-09-16T07:36:06.064066+0000","last_became_active":"2024-09-16T07:36:04.851903+0000","last_became_peered":"2024-09-16T07:36:04.851903+0000","last_unstale":"2024-09-16T07:36:06.064066+0000","last_undegraded":"2024-09-16T07:36:06.064066+0000","last_fullsized":"2024-09-16T07:36:06.064066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:32.387978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850176+0000","last_change":"2024-09-16T07:36:03.844377+0000","last_active":"2024-09-16T07:36:05.850176+0000","last_peered":"2024-09-16T07:36:05.850176+0000","last_clean":"2024-09-16T07:36:05.850176+0000","last_became_active":"2024-09-16T07:36:03.844005+0000","last_became_peered":"2024-09-16T07:36:03.844005+0000","last_unstale":"2024-09-16T07:36:05.850176+0000","last_undegraded":"2024-09-16T07:36:05.850176+0000","last_fullsized":"2024-09-16T07:36:05.850176+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:46.575242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858524+0000","last_change":"2024-09-16T07:36:04.855371+0000","last_active":"2024-09-16T07:36:05.858524+0000","last_peered":"2024-09-16T07:36:05.858524+0000","last_clean":"2024-09-16T07:36:05.858524+0000","last_became_active":"2024-09-16T07:36:04.854159+0000","last_became_peered":"2024-09-16T07:36:04.854159+0000","last_unstale":"2024-09-16T07:36:05.858524+0000","last_undegraded":"2024-09-16T07:36:05.858524+0000","last_fullsized":"2024-09-16T07:36:05.858524+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:53:57.092765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855092+0000","last_change":"2024-09-16T07:36:03.852706+0000","last_active":"2024-09-16T07:36:05.855092+0000","last_peered":"2024-09-16T07:36:05.855092+0000","last_clean":"2024-09-16T07:36:05.855092+0000","last_became_active":"2024-09-16T07:36:03.852295+0000","last_became_peered":"2024-09-16T07:36:03.852295+0000","last_unstale":"2024-09-16T07:36:05.855092+0000","last_undegraded":"2024-09-16T07:36:05.855092+0000","last_fullsized":"2024-09-16T07:36:05.855092+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:15:51.417749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850016+0000","last_change":"2024-09-16T07:36:04.851478+0000","last_active":"2024-09-16T07:36:05.850016+0000","last_peered":"2024-09-16T07:36:05.850016+0000","last_clean":"2024-09-16T07:36:05.850016+0000","last_became_active":"2024-09-16T07:36:04.851213+0000","last_became_peered":"2024-09-16T07:36:04.851213+0000","last_unstale":"2024-09-16T07:36:05.850016+0000","last_undegraded":"2024-09-16T07:36:05.850016+0000","last_fullsized":"2024-09-16T07:36:05.850016+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:09:51.497503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"132'3","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858392+0000","last_change":"2024-09-16T07:36:03.857233+0000","last_active":"2024-09-16T07:36:05.858392+0000","last_peered":"2024-09-16T07:36:05.858392+0000","last_clean":"2024-09-16T07:36:05.858392+0000","last_became_active":"2024-09-16T07:36:03.856817+0000","last_became_peered":"2024-09-16T07:36:03.856817+0000","last_unstale":"2024-09-16T07:36:05.858392+0000","last_undegraded":"2024-09-16T07:36:05.858392+0000","last_fullsized":"2024-09-16T07:36:05.858392+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:40:54.651864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974575+0000","last_change":"2024-09-16T07:36:04.855153+0000","last_active":"2024-09-16T07:36:06.974575+0000","last_peered":"2024-09-16T07:36:06.974575+0000","last_clean":"2024-09-16T07:36:06.974575+0000","last_became_active":"2024-09-16T07:36:04.854893+0000","last_became_peered":"2024-09-16T07:36:04.854893+0000","last_unstale":"2024-09-16T07:36:06.974575+0000","last_undegraded":"2024-09-16T07:36:06.974575+0000","last_fullsized":"2024-09-16T07:36:06.974575+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:00.759798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"132'6","reported_seq":21,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974642+0000","last_change":"2024-09-16T07:36:03.851395+0000","last_active":"2024-09-16T07:36:06.974642+0000","last_peered":"2024-09-16T07:36:06.974642+0000","last_clean":"2024-09-16T07:36:06.974642+0000","last_became_active":"2024-09-16T07:36:03.851127+0000","last_became_peered":"2024-09-16T07:36:03.851127+0000","last_unstale":"2024-09-16T07:36:06.974642+0000","last_undegraded":"2024-09-16T07:36:06.974642+0000","last_fullsized":"2024-09-16T07:36:06.974642+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:27.567738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856941+0000","last_change":"2024-09-16T07:36:04.863490+0000","last_active":"2024-09-16T07:36:05.856941+0000","last_peered":"2024-09-16T07:36:05.856941+0000","last_clean":"2024-09-16T07:36:05.856941+0000","last_became_active":"2024-09-16T07:36:04.862482+0000","last_became_peered":"2024-09-16T07:36:04.862482+0000","last_unstale":"2024-09-16T07:36:05.856941+0000","last_undegraded":"2024-09-16T07:36:05.856941+0000","last_fullsized":"2024-09-16T07:36:05.856941+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:18.732185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856621+0000","last_change":"2024-09-16T07:36:03.852858+0000","last_active":"2024-09-16T07:36:05.856621+0000","last_peered":"2024-09-16T07:36:05.856621+0000","last_clean":"2024-09-16T07:36:05.856621+0000","last_became_active":"2024-09-16T07:36:03.852596+0000","last_became_peered":"2024-09-16T07:36:03.852596+0000","last_unstale":"2024-09-16T07:36:05.856621+0000","last_undegraded":"2024-09-16T07:36:05.856621+0000","last_fullsized":"2024-09-16T07:36:05.856621+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:29.307382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854662+0000","last_change":"2024-09-16T07:36:04.855631+0000","last_active":"2024-09-16T07:36:05.854662+0000","last_peered":"2024-09-16T07:36:05.854662+0000","last_clean":"2024-09-16T07:36:05.854662+0000","last_became_active":"2024-09-16T07:36:04.855341+0000","last_became_peered":"2024-09-16T07:36:04.855341+0000","last_unstale":"2024-09-16T07:36:05.854662+0000","last_undegraded":"2024-09-16T07:36:05.854662+0000","last_fullsized":"2024-09-16T07:36:05.854662+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:06.127741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854531+0000","last_change":"2024-09-16T07:36:03.854082+0000","last_active":"2024-09-16T07:36:05.854531+0000","last_peered":"2024-09-16T07:36:05.854531+0000","last_clean":"2024-09-16T07:36:05.854531+0000","last_became_active":"2024-09-16T07:36:03.853794+0000","last_became_peered":"2024-09-16T07:36:03.853794+0000","last_unstale":"2024-09-16T07:36:05.854531+0000","last_undegraded":"2024-09-16T07:36:05.854531+0000","last_fullsized":"2024-09-16T07:36:05.854531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:50:10.982614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849412+0000","last_change":"2024-09-16T07:36:04.852986+0000","last_active":"2024-09-16T07:36:05.849412+0000","last_peered":"2024-09-16T07:36:05.849412+0000","last_clean":"2024-09-16T07:36:05.849412+0000","last_became_active":"2024-09-16T07:36:04.851428+0000","last_became_peered":"2024-09-16T07:36:04.851428+0000","last_unstale":"2024-09-16T07:36:05.849412+0000","last_undegraded":"2024-09-16T07:36:05.849412+0000","last_fullsized":"2024-09-16T07:36:05.849412+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:58.294658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"132'8","reported_seq":23,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849259+0000","last_change":"2024-09-16T07:36:03.845321+0000","last_active":"2024-09-16T07:36:05.849259+0000","last_peered":"2024-09-16T07:36:05.849259+0000","last_clean":"2024-09-16T07:36:05.849259+0000","last_became_active":"2024-09-16T07:36:03.845024+0000","last_became_peered":"2024-09-16T07:36:03.845024+0000","last_unstale":"2024-09-16T07:36:05.849259+0000","last_undegraded":"2024-09-16T07:36:05.849259+0000","last_fullsized":"2024-09-16T07:36:05.849259+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:26.754316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858506+0000","last_change":"2024-09-16T07:36:04.850404+0000","last_active":"2024-09-16T07:36:05.858506+0000","last_peered":"2024-09-16T07:36:05.858506+0000","last_clean":"2024-09-16T07:36:05.858506+0000","last_became_active":"2024-09-16T07:36:04.850137+0000","last_became_peered":"2024-09-16T07:36:04.850137+0000","last_unstale":"2024-09-16T07:36:05.858506+0000","last_undegraded":"2024-09-16T07:36:05.858506+0000","last_fullsized":"2024-09-16T07:36:05.858506+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:35.706934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.850284+0000","last_change":"2024-09-16T07:36:03.848724+0000","last_active":"2024-09-16T07:36:05.850284+0000","last_peered":"2024-09-16T07:36:05.850284+0000","last_clean":"2024-09-16T07:36:05.850284+0000","last_became_active":"2024-09-16T07:36:03.847881+0000","last_became_peered":"2024-09-16T07:36:03.847881+0000","last_unstale":"2024-09-16T07:36:05.850284+0000","last_undegraded":"2024-09-16T07:36:05.850284+0000","last_fullsized":"2024-09-16T07:36:05.850284+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:06.592607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858589+0000","last_change":"2024-09-16T07:36:04.860131+0000","last_active":"2024-09-16T07:36:05.858589+0000","last_peered":"2024-09-16T07:36:05.858589+0000","last_clean":"2024-09-16T07:36:05.858589+0000","last_became_active":"2024-09-16T07:36:04.859815+0000","last_became_peered":"2024-09-16T07:36:04.859815+0000","last_unstale":"2024-09-16T07:36:05.858589+0000","last_undegraded":"2024-09-16T07:36:05.858589+0000","last_fullsized":"2024-09-16T07:36:05.858589+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:31.769428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856205+0000","last_change":"2024-09-16T07:36:03.850133+0000","last_active":"2024-09-16T07:36:05.856205+0000","last_peered":"2024-09-16T07:36:05.856205+0000","last_clean":"2024-09-16T07:36:05.856205+0000","last_became_active":"2024-09-16T07:36:03.849070+0000","last_became_peered":"2024-09-16T07:36:03.849070+0000","last_unstale":"2024-09-16T07:36:05.856205+0000","last_undegraded":"2024-09-16T07:36:05.856205+0000","last_fullsized":"2024-09-16T07:36:05.856205+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:35.166163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063646+0000","last_change":"2024-09-16T07:36:04.848685+0000","last_active":"2024-09-16T07:36:06.063646+0000","last_peered":"2024-09-16T07:36:06.063646+0000","last_clean":"2024-09-16T07:36:06.063646+0000","last_became_active":"2024-09-16T07:36:04.844479+0000","last_became_peered":"2024-09-16T07:36:04.844479+0000","last_unstale":"2024-09-16T07:36:06.063646+0000","last_undegraded":"2024-09-16T07:36:06.063646+0000","last_fullsized":"2024-09-16T07:36:06.063646+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:36.534540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978385+0000","last_change":"2024-09-16T07:36:03.855122+0000","last_active":"2024-09-16T07:36:06.978385+0000","last_peered":"2024-09-16T07:36:06.978385+0000","last_clean":"2024-09-16T07:36:06.978385+0000","last_became_active":"2024-09-16T07:36:03.854786+0000","last_became_peered":"2024-09-16T07:36:03.854786+0000","last_unstale":"2024-09-16T07:36:06.978385+0000","last_undegraded":"2024-09-16T07:36:06.978385+0000","last_fullsized":"2024-09-16T07:36:06.978385+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:36.071792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063917+0000","last_change":"2024-09-16T07:36:04.853245+0000","last_active":"2024-09-16T07:36:06.063917+0000","last_peered":"2024-09-16T07:36:06.063917+0000","last_clean":"2024-09-16T07:36:06.063917+0000","last_became_active":"2024-09-16T07:36:04.852895+0000","last_became_peered":"2024-09-16T07:36:04.852895+0000","last_unstale":"2024-09-16T07:36:06.063917+0000","last_undegraded":"2024-09-16T07:36:06.063917+0000","last_fullsized":"2024-09-16T07:36:06.063917+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.199780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858362+0000","last_change":"2024-09-16T07:36:03.856130+0000","last_active":"2024-09-16T07:36:05.858362+0000","last_peered":"2024-09-16T07:36:05.858362+0000","last_clean":"2024-09-16T07:36:05.858362+0000","last_became_active":"2024-09-16T07:36:03.852405+0000","last_became_peered":"2024-09-16T07:36:03.852405+0000","last_unstale":"2024-09-16T07:36:05.858362+0000","last_undegraded":"2024-09-16T07:36:05.858362+0000","last_fullsized":"2024-09-16T07:36:05.858362+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:56:26.754316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.064061+0000","last_change":"2024-09-16T07:36:04.847598+0000","last_active":"2024-09-16T07:36:06.064061+0000","last_peered":"2024-09-16T07:36:06.064061+0000","last_clean":"2024-09-16T07:36:06.064061+0000","last_became_active":"2024-09-16T07:36:04.847063+0000","last_became_peered":"2024-09-16T07:36:04.847063+0000","last_unstale":"2024-09-16T07:36:06.064061+0000","last_undegraded":"2024-09-16T07:36:06.064061+0000","last_fullsized":"2024-09-16T07:36:06.064061+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:50.838203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978132+0000","last_change":"2024-09-16T07:36:03.856847+0000","last_active":"2024-09-16T07:36:06.978132+0000","last_peered":"2024-09-16T07:36:06.978132+0000","last_clean":"2024-09-16T07:36:06.978132+0000","last_became_active":"2024-09-16T07:36:03.856321+0000","last_became_peered":"2024-09-16T07:36:03.856321+0000","last_unstale":"2024-09-16T07:36:06.978132+0000","last_undegraded":"2024-09-16T07:36:06.978132+0000","last_fullsized":"2024-09-16T07:36:06.978132+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:32.485232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974582+0000","last_change":"2024-09-16T07:36:04.848724+0000","last_active":"2024-09-16T07:36:06.974582+0000","last_peered":"2024-09-16T07:36:06.974582+0000","last_clean":"2024-09-16T07:36:06.974582+0000","last_became_active":"2024-09-16T07:36:04.848393+0000","last_became_peered":"2024-09-16T07:36:04.848393+0000","last_unstale":"2024-09-16T07:36:06.974582+0000","last_undegraded":"2024-09-16T07:36:06.974582+0000","last_fullsized":"2024-09-16T07:36:06.974582+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:12.890777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856080+0000","last_change":"2024-09-16T07:36:03.855412+0000","last_active":"2024-09-16T07:36:05.856080+0000","last_peered":"2024-09-16T07:36:05.856080+0000","last_clean":"2024-09-16T07:36:05.856080+0000","last_became_active":"2024-09-16T07:36:03.854933+0000","last_became_peered":"2024-09-16T07:36:03.854933+0000","last_unstale":"2024-09-16T07:36:05.856080+0000","last_undegraded":"2024-09-16T07:36:05.856080+0000","last_fullsized":"2024-09-16T07:36:05.856080+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:58.216679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854581+0000","last_change":"2024-09-16T07:36:04.850083+0000","last_active":"2024-09-16T07:36:05.854581+0000","last_peered":"2024-09-16T07:36:05.854581+0000","last_clean":"2024-09-16T07:36:05.854581+0000","last_became_active":"2024-09-16T07:36:04.849731+0000","last_became_peered":"2024-09-16T07:36:04.849731+0000","last_unstale":"2024-09-16T07:36:05.854581+0000","last_undegraded":"2024-09-16T07:36:05.854581+0000","last_fullsized":"2024-09-16T07:36:05.854581+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:08.133005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855297+0000","last_change":"2024-09-16T07:36:03.848269+0000","last_active":"2024-09-16T07:36:05.855297+0000","last_peered":"2024-09-16T07:36:05.855297+0000","last_clean":"2024-09-16T07:36:05.855297+0000","last_became_active":"2024-09-16T07:36:03.847931+0000","last_became_peered":"2024-09-16T07:36:03.847931+0000","last_unstale":"2024-09-16T07:36:05.855297+0000","last_undegraded":"2024-09-16T07:36:05.855297+0000","last_fullsized":"2024-09-16T07:36:05.855297+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:06.554515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849468+0000","last_change":"2024-09-16T07:36:04.859275+0000","last_active":"2024-09-16T07:36:05.849468+0000","last_peered":"2024-09-16T07:36:05.849468+0000","last_clean":"2024-09-16T07:36:05.849468+0000","last_became_active":"2024-09-16T07:36:04.859013+0000","last_became_peered":"2024-09-16T07:36:04.859013+0000","last_unstale":"2024-09-16T07:36:05.849468+0000","last_undegraded":"2024-09-16T07:36:05.849468+0000","last_fullsized":"2024-09-16T07:36:05.849468+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:42.011961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856499+0000","last_change":"2024-09-16T07:36:03.847015+0000","last_active":"2024-09-16T07:36:05.856499+0000","last_peered":"2024-09-16T07:36:05.856499+0000","last_clean":"2024-09-16T07:36:05.856499+0000","last_became_active":"2024-09-16T07:36:03.846719+0000","last_became_peered":"2024-09-16T07:36:03.846719+0000","last_unstale":"2024-09-16T07:36:05.856499+0000","last_undegraded":"2024-09-16T07:36:05.856499+0000","last_fullsized":"2024-09-16T07:36:05.856499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:46.177351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974797+0000","last_change":"2024-09-16T07:36:04.853837+0000","last_active":"2024-09-16T07:36:06.974797+0000","last_peered":"2024-09-16T07:36:06.974797+0000","last_clean":"2024-09-16T07:36:06.974797+0000","last_became_active":"2024-09-16T07:36:04.853492+0000","last_became_peered":"2024-09-16T07:36:04.853492+0000","last_unstale":"2024-09-16T07:36:06.974797+0000","last_undegraded":"2024-09-16T07:36:06.974797+0000","last_fullsized":"2024-09-16T07:36:06.974797+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:05.954007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856245+0000","last_change":"2024-09-16T07:36:03.854810+0000","last_active":"2024-09-16T07:36:05.856245+0000","last_peered":"2024-09-16T07:36:05.856245+0000","last_clean":"2024-09-16T07:36:05.856245+0000","last_became_active":"2024-09-16T07:36:03.854533+0000","last_became_peered":"2024-09-16T07:36:03.854533+0000","last_unstale":"2024-09-16T07:36:05.856245+0000","last_undegraded":"2024-09-16T07:36:05.856245+0000","last_fullsized":"2024-09-16T07:36:05.856245+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:23.851837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063875+0000","last_change":"2024-09-16T07:36:04.855496+0000","last_active":"2024-09-16T07:36:06.063875+0000","last_peered":"2024-09-16T07:36:06.063875+0000","last_clean":"2024-09-16T07:36:06.063875+0000","last_became_active":"2024-09-16T07:36:04.855167+0000","last_became_peered":"2024-09-16T07:36:04.855167+0000","last_unstale":"2024-09-16T07:36:06.063875+0000","last_undegraded":"2024-09-16T07:36:06.063875+0000","last_fullsized":"2024-09-16T07:36:06.063875+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:24.513277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"133'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.858514+0000","last_change":"2024-09-16T07:36:03.857828+0000","last_active":"2024-09-16T07:36:05.858514+0000","last_peered":"2024-09-16T07:36:05.858514+0000","last_clean":"2024-09-16T07:36:05.858514+0000","last_became_active":"2024-09-16T07:36:03.853109+0000","last_became_peered":"2024-09-16T07:36:03.853109+0000","last_unstale":"2024-09-16T07:36:05.858514+0000","last_undegraded":"2024-09-16T07:36:05.858514+0000","last_fullsized":"2024-09-16T07:36:05.858514+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:47.645572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857004+0000","last_change":"2024-09-16T07:36:04.852909+0000","last_active":"2024-09-16T07:36:05.857004+0000","last_peered":"2024-09-16T07:36:05.857004+0000","last_clean":"2024-09-16T07:36:05.857004+0000","last_became_active":"2024-09-16T07:36:04.852496+0000","last_became_peered":"2024-09-16T07:36:04.852496+0000","last_unstale":"2024-09-16T07:36:05.857004+0000","last_undegraded":"2024-09-16T07:36:05.857004+0000","last_fullsized":"2024-09-16T07:36:05.857004+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:43.843163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854718+0000","last_change":"2024-09-16T07:36:03.853605+0000","last_active":"2024-09-16T07:36:05.854718+0000","last_peered":"2024-09-16T07:36:05.854718+0000","last_clean":"2024-09-16T07:36:05.854718+0000","last_became_active":"2024-09-16T07:36:03.853349+0000","last_became_peered":"2024-09-16T07:36:03.853349+0000","last_unstale":"2024-09-16T07:36:05.854718+0000","last_undegraded":"2024-09-16T07:36:05.854718+0000","last_fullsized":"2024-09-16T07:36:05.854718+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:29.592986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978353+0000","last_change":"2024-09-16T07:36:04.851823+0000","last_active":"2024-09-16T07:36:06.978353+0000","last_peered":"2024-09-16T07:36:06.978353+0000","last_clean":"2024-09-16T07:36:06.978353+0000","last_became_active":"2024-09-16T07:36:04.851171+0000","last_became_peered":"2024-09-16T07:36:04.851171+0000","last_unstale":"2024-09-16T07:36:06.978353+0000","last_undegraded":"2024-09-16T07:36:06.978353+0000","last_fullsized":"2024-09-16T07:36:06.978353+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:40.887508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.857065+0000","last_change":"2024-09-16T07:36:03.851730+0000","last_active":"2024-09-16T07:36:05.857065+0000","last_peered":"2024-09-16T07:36:05.857065+0000","last_clean":"2024-09-16T07:36:05.857065+0000","last_became_active":"2024-09-16T07:36:03.851457+0000","last_became_peered":"2024-09-16T07:36:03.851457+0000","last_unstale":"2024-09-16T07:36:05.857065+0000","last_undegraded":"2024-09-16T07:36:05.857065+0000","last_fullsized":"2024-09-16T07:36:05.857065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:37.222531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849662+0000","last_change":"2024-09-16T07:36:04.850230+0000","last_active":"2024-09-16T07:36:05.849662+0000","last_peered":"2024-09-16T07:36:05.849662+0000","last_clean":"2024-09-16T07:36:05.849662+0000","last_became_active":"2024-09-16T07:36:04.849632+0000","last_became_peered":"2024-09-16T07:36:04.849632+0000","last_unstale":"2024-09-16T07:36:05.849662+0000","last_undegraded":"2024-09-16T07:36:05.849662+0000","last_fullsized":"2024-09-16T07:36:05.849662+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:44:05.734690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.063468+0000","last_change":"2024-09-16T07:36:03.857845+0000","last_active":"2024-09-16T07:36:06.063468+0000","last_peered":"2024-09-16T07:36:06.063468+0000","last_clean":"2024-09-16T07:36:06.063468+0000","last_became_active":"2024-09-16T07:36:03.857366+0000","last_became_peered":"2024-09-16T07:36:03.857366+0000","last_unstale":"2024-09-16T07:36:06.063468+0000","last_undegraded":"2024-09-16T07:36:06.063468+0000","last_fullsized":"2024-09-16T07:36:06.063468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:36.413668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849796+0000","last_change":"2024-09-16T07:36:04.866976+0000","last_active":"2024-09-16T07:36:05.849796+0000","last_peered":"2024-09-16T07:36:05.849796+0000","last_clean":"2024-09-16T07:36:05.849796+0000","last_became_active":"2024-09-16T07:36:04.866659+0000","last_became_peered":"2024-09-16T07:36:04.866659+0000","last_unstale":"2024-09-16T07:36:05.849796+0000","last_undegraded":"2024-09-16T07:36:05.849796+0000","last_fullsized":"2024-09-16T07:36:05.849796+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:55:21.257459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849573+0000","last_change":"2024-09-16T07:36:03.853643+0000","last_active":"2024-09-16T07:36:05.849573+0000","last_peered":"2024-09-16T07:36:05.849573+0000","last_clean":"2024-09-16T07:36:05.849573+0000","last_became_active":"2024-09-16T07:36:03.851881+0000","last_became_peered":"2024-09-16T07:36:03.851881+0000","last_unstale":"2024-09-16T07:36:05.849573+0000","last_undegraded":"2024-09-16T07:36:05.849573+0000","last_fullsized":"2024-09-16T07:36:05.849573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:47.645572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.855281+0000","last_change":"2024-09-16T07:36:04.863145+0000","last_active":"2024-09-16T07:36:05.855281+0000","last_peered":"2024-09-16T07:36:05.855281+0000","last_clean":"2024-09-16T07:36:05.855281+0000","last_became_active":"2024-09-16T07:36:04.862893+0000","last_became_peered":"2024-09-16T07:36:04.862893+0000","last_unstale":"2024-09-16T07:36:05.855281+0000","last_undegraded":"2024-09-16T07:36:05.855281+0000","last_fullsized":"2024-09-16T07:36:05.855281+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:26.614434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854639+0000","last_change":"2024-09-16T07:36:03.851573+0000","last_active":"2024-09-16T07:36:05.854639+0000","last_peered":"2024-09-16T07:36:05.854639+0000","last_clean":"2024-09-16T07:36:05.854639+0000","last_became_active":"2024-09-16T07:36:03.851286+0000","last_became_peered":"2024-09-16T07:36:03.851286+0000","last_unstale":"2024-09-16T07:36:05.854639+0000","last_undegraded":"2024-09-16T07:36:05.854639+0000","last_fullsized":"2024-09-16T07:36:05.854639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.290737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.856129+0000","last_change":"2024-09-16T07:36:04.858119+0000","last_active":"2024-09-16T07:36:05.856129+0000","last_peered":"2024-09-16T07:36:05.856129+0000","last_clean":"2024-09-16T07:36:05.856129+0000","last_became_active":"2024-09-16T07:36:04.855972+0000","last_became_peered":"2024-09-16T07:36:04.855972+0000","last_unstale":"2024-09-16T07:36:05.856129+0000","last_undegraded":"2024-09-16T07:36:05.856129+0000","last_fullsized":"2024-09-16T07:36:05.856129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:40.599143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.854647+0000","last_change":"2024-09-16T07:36:03.854300+0000","last_active":"2024-09-16T07:36:05.854647+0000","last_peered":"2024-09-16T07:36:05.854647+0000","last_clean":"2024-09-16T07:36:05.854647+0000","last_became_active":"2024-09-16T07:36:03.853971+0000","last_became_peered":"2024-09-16T07:36:03.853971+0000","last_unstale":"2024-09-16T07:36:05.854647+0000","last_undegraded":"2024-09-16T07:36:05.854647+0000","last_fullsized":"2024-09-16T07:36:05.854647+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:25.287389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.978598+0000","last_change":"2024-09-16T07:36:04.843292+0000","last_active":"2024-09-16T07:36:06.978598+0000","last_peered":"2024-09-16T07:36:06.978598+0000","last_clean":"2024-09-16T07:36:06.978598+0000","last_became_active":"2024-09-16T07:36:04.843022+0000","last_became_peered":"2024-09-16T07:36:04.843022+0000","last_unstale":"2024-09-16T07:36:06.978598+0000","last_undegraded":"2024-09-16T07:36:06.978598+0000","last_fullsized":"2024-09-16T07:36:06.978598+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:03.818979+0000","last_clean_scrub_stamp":"2024-09-16T07:36:03.818979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:38.146552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:05.849785+0000","last_change":"2024-09-16T07:36:03.855144+0000","last_active":"2024-09-16T07:36:05.849785+0000","last_peered":"2024-09-16T07:36:05.849785+0000","last_clean":"2024-09-16T07:36:05.849785+0000","last_became_active":"2024-09-16T07:36:03.854840+0000","last_became_peered":"2024-09-16T07:36:03.854840+0000","last_unstale":"2024-09-16T07:36:05.849785+0000","last_undegraded":"2024-09-16T07:36:05.849785+0000","last_fullsized":"2024-09-16T07:36:05.849785+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:36:02.811068+0000","last_clean_scrub_stamp":"2024-09-16T07:36:02.811068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:17.990182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":314,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T07:36:06.974366+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:36:06.974366+0000","last_peered":"2024-09-16T07:36:06.974366+0000","last_clean":"2024-09-16T07:36:06.974366+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:36:06.974366+0000","last_undegraded":"2024-09-16T07:36:06.974366+0000","last_fullsized":"2024-09-16T07:36:06.974366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476903,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131296,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240544,"statfs":{"total":96636764160,"available":96502317056,"internally_reserved":0,"allocated":1650688,"data_stored":724795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476903,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131300,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240540,"statfs":{"total":96636764160,"available":96502312960,"internally_reserved":0,"allocated":1654784,"data_stored":725397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476903,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127728,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244112,"statfs":{"total":96636764160,"available":96505970688,"internally_reserved":0,"allocated":2256896,"data_stored":1316727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476902,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145200,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226640,"statfs":{"total":96636764160,"available":96488079360,"internally_reserved":0,"allocated":1667072,"data_stored":726393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127140,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244700,"statfs":{"total":96636764160,"available":96506572800,"internally_reserved":0,"allocated":1650688,"data_stored":724759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509606,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159076,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212764,"statfs":{"total":96636764160,"available":96473870336,"internally_reserved":0,"allocated":1646592,"data_stored":724725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161216966},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509606,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119404,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252436,"statfs":{"total":96636764160,"available":96514494464,"internally_reserved":0,"allocated":2252800,"data_stored":1315835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135456,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236384,"statfs":{"total":96636764160,"available":96498057216,"internally_reserved":0,"allocated":1646592,"data_stored":724725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":137034181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:36:14.128 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:36:14.128 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:36:14.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.330+0000 7f2b3c800640 1 -- 172.21.15.71:0/763790855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34145750 msgr2=0x7f2b34073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.330+0000 7f2b3c800640 1 --2- 172.21.15.71:0/763790855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34145750 0x7f2b34073440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009830 tx=0x7f2b2802a480 comp rx=0 tx=0).stop 2024-09-16T07:36:14.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.330+0000 7f2b3c800640 1 -- 172.21.15.71:0/763790855 shutdown_connections 2024-09-16T07:36:14.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.330+0000 7f2b3c800640 1 --2- 172.21.15.71:0/763790855 >> v2:172.21.15.177:3300/0 conn(0x7f2b3406efa0 0x7f2b340cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.330+0000 7f2b3c800640 1 --2- 172.21.15.71:0/763790855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34145750 0x7f2b34073440 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.330+0000 7f2b3c800640 1 -- 172.21.15.71:0/763790855 >> 172.21.15.71:0/763790855 conn(0x7f2b340b1840 msgr2=0x7f2b340b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:14.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.330+0000 7f2b3c800640 1 -- 172.21.15.71:0/763790855 shutdown_connections 2024-09-16T07:36:14.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.330+0000 7f2b3c800640 1 -- 172.21.15.71:0/763790855 wait complete. 2024-09-16T07:36:14.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.332+0000 7f2b3c800640 1 Processor -- start 2024-09-16T07:36:14.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.332+0000 7f2b3c800640 1 -- start start 2024-09-16T07:36:14.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3c800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b3406efa0 0x7f2b34141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:14.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3c800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340ccb00 0x7f2b3415e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:14.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3c800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b34145750 0x7f2b3415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:14.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3c800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b340b26b0 con 0x7f2b34145750 2024-09-16T07:36:14.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3c800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2b340cedd0 con 0x7f2b340ccb00 2024-09-16T07:36:14.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3c800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2b34160100 con 0x7f2b3406efa0 2024-09-16T07:36:14.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b34145750 0x7f2b3415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b34145750 0x7f2b3415eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40526/0 (socket says 172.21.15.71:40526) 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b3a800640 1 -- 172.21.15.71:0/3276720066 learned_addr learned my addr 172.21.15.71:0/3276720066 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b39e00640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b3406efa0 0x7f2b34141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.333+0000 7f2b39400640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340ccb00 0x7f2b3415e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b3a800640 1 -- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b3406efa0 msgr2=0x7f2b34141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b3a800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b3406efa0 0x7f2b34141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b3a800640 1 -- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340ccb00 msgr2=0x7f2b3415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b3a800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340ccb00 0x7f2b3415e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b3a800640 1 -- 172.21.15.71:0/3276720066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b34141c50 con 0x7f2b34145750 2024-09-16T07:36:14.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b3a800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b34145750 0x7f2b3415eb70 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c7a0 tx=0x7f2b3000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:14.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b2e000640 1 -- 172.21.15.71:0/3276720066 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30003b70 con 0x7f2b34145750 2024-09-16T07:36:14.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b2e000640 1 -- 172.21.15.71:0/3276720066 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b30003d10 con 0x7f2b34145750 2024-09-16T07:36:14.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b2e000640 1 -- 172.21.15.71:0/3276720066 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000aab0 con 0x7f2b34145750 2024-09-16T07:36:14.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b341f3080 con 0x7f2b34145750 2024-09-16T07:36:14.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.334+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b341f3620 con 0x7f2b34145750 2024-09-16T07:36:14.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.335+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b341605b0 con 0x7f2b34145750 2024-09-16T07:36:14.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.337+0000 7f2b2e000640 1 -- 172.21.15.71:0/3276720066 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f2b3000ac50 con 0x7f2b34145750 2024-09-16T07:36:14.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.337+0000 7f2b2e000640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b1807a6b0 0x7f2b1807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:14.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.338+0000 7f2b2e000640 1 -- 172.21.15.71:0/3276720066 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f2b30099660 con 0x7f2b34145750 2024-09-16T07:36:14.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.338+0000 7f2b39e00640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b1807a6b0 0x7f2b1807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:14.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.339+0000 7f2b39e00640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b1807a6b0 0x7f2b1807cb70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2b28034000 tx=0x7f2b28004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:14.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.342+0000 7f2b2e000640 1 -- 172.21.15.71:0/3276720066 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2b34141c50 con 0x7f2b34145750 2024-09-16T07:36:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.593+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2b340735a0 con 0x7f2b34145750 2024-09-16T07:36:14.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.596+0000 7f2b2e000640 1 -- 172.21.15.71:0/3276720066 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f2b30063180 con 0x7f2b34145750 2024-09-16T07:36:14.595 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:36:14.595 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:36:14.596 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:36:14.596 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:36:14.596 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:36:14.596 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:36:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b1807a6b0 msgr2=0x7f2b1807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b1807a6b0 0x7f2b1807cb70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2b28034000 tx=0x7f2b28004730 comp rx=0 tx=0).stop 2024-09-16T07:36:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b34145750 msgr2=0x7f2b3415eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b34145750 0x7f2b3415eb70 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c7a0 tx=0x7f2b3000cc60 comp rx=0 tx=0).stop 2024-09-16T07:36:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 shutdown_connections 2024-09-16T07:36:14.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2b1807a6b0 0x7f2b1807cb70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b34145750 0x7f2b3415eb70 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340ccb00 0x7f2b3415e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 --2- 172.21.15.71:0/3276720066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b3406efa0 0x7f2b34141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.598+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 >> 172.21.15.71:0/3276720066 conn(0x7f2b340b1840 msgr2=0x7f2b340b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.599+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 shutdown_connections 2024-09-16T07:36:14.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.599+0000 7f2b3c800640 1 -- 172.21.15.71:0/3276720066 wait complete. 2024-09-16T07:36:14.614 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:36:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.772+0000 7f0a38a00640 1 -- 172.21.15.71:0/337360273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340b2440 msgr2=0x7f0a3406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.772+0000 7f0a38a00640 1 --2- 172.21.15.71:0/337360273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340b2440 0x7f0a3406f160 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005fd0 tx=0x7f0a2c02a540 comp rx=0 tx=0).stop 2024-09-16T07:36:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.772+0000 7f0a38a00640 1 -- 172.21.15.71:0/337360273 shutdown_connections 2024-09-16T07:36:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.772+0000 7f0a38a00640 1 --2- 172.21.15.71:0/337360273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340b2440 0x7f0a3406f160 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.772+0000 7f0a38a00640 1 -- 172.21.15.71:0/337360273 >> 172.21.15.71:0/337360273 conn(0x7f0a340b1950 msgr2=0x7f0a340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.772+0000 7f0a38a00640 1 -- 172.21.15.71:0/337360273 shutdown_connections 2024-09-16T07:36:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.772+0000 7f0a38a00640 1 -- 172.21.15.71:0/337360273 wait complete. 2024-09-16T07:36:14.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.774+0000 7f0a38a00640 1 Processor -- start 2024-09-16T07:36:14.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.774+0000 7f0a38a00640 1 -- start start 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a38a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a340b2440 0x7f0a341f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a38a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340ccf00 0x7f0a3409c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a38a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a3415f5a0 0x7f0a340cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a38a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0a34074840 con 0x7f0a340ccf00 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a38a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a340746c0 con 0x7f0a3415f5a0 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a38a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3406f6a0 con 0x7f0a340b2440 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340ccf00 0x7f0a3409c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340ccf00 0x7f0a3409c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40184/0 (socket says 172.21.15.71:40184) 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31400640 1 -- 172.21.15.71:0/2191215116 learned_addr learned my addr 172.21.15.71:0/2191215116 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31e00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a340b2440 0x7f0a341f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a32800640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a3415f5a0 0x7f0a340cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31400640 1 -- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a340b2440 msgr2=0x7f0a341f8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31400640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a340b2440 0x7f0a341f8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31400640 1 -- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a3415f5a0 msgr2=0x7f0a340cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31400640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a3415f5a0 0x7f0a340cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.775+0000 7f0a31400640 1 -- 172.21.15.71:0/2191215116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a341f4b80 con 0x7f0a340ccf00 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.776+0000 7f0a31400640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340ccf00 0x7f0a3409c0d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c410 tx=0x7f0a2000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.776+0000 7f0a2a000640 1 -- 172.21.15.71:0/2191215116 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20017070 con 0x7f0a340ccf00 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.776+0000 7f0a2a000640 1 -- 172.21.15.71:0/2191215116 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a200035f0 con 0x7f0a340ccf00 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.776+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a341f4e60 con 0x7f0a340ccf00 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.776+0000 7f0a2a000640 1 -- 172.21.15.71:0/2191215116 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20003a80 con 0x7f0a340ccf00 2024-09-16T07:36:14.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.776+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a341f5420 con 0x7f0a340ccf00 2024-09-16T07:36:14.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.777+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3406f160 con 0x7f0a340ccf00 2024-09-16T07:36:14.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.778+0000 7f0a2a000640 1 -- 172.21.15.71:0/2191215116 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f0a2001a070 con 0x7f0a340ccf00 2024-09-16T07:36:14.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.779+0000 7f0a2a000640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0a1407a6e0 0x7f0a1407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:14.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.780+0000 7f0a2a000640 1 -- 172.21.15.71:0/2191215116 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f0a20098c60 con 0x7f0a340ccf00 2024-09-16T07:36:14.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.780+0000 7f0a31e00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0a1407a6e0 0x7f0a1407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:14.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.781+0000 7f0a31e00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0a1407a6e0 0x7f0a1407cba0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0a24001ce0 tx=0x7f0a24009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:14.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:14.783+0000 7f0a2a000640 1 -- 172.21.15.71:0/2191215116 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0a20062dd0 con 0x7f0a340ccf00 2024-09-16T07:36:15.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.004+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a3409d6b0 con 0x7f0a340ccf00 2024-09-16T07:36:15.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.005+0000 7f0a2a000640 1 -- 172.21.15.71:0/2191215116 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5231 (secure 0 0 0) 0x7f0a200010c0 con 0x7f0a340ccf00 2024-09-16T07:36:15.005 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:15.005 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":85,"btime":"2024-09-16T07:36:07:854566+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4275610322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":4275610322},{"type":"v1","addr":"172.21.15.71:6835","nonce":4275610322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/2559248159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2559248159},{"type":"v1","addr":"172.21.15.177:6836","nonce":2559248159}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/1515242906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1515242906},{"type":"v1","addr":"172.21.15.71:6834","nonce":1515242906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:36:04.253023+0000","modified":"2024-09-16T07:36:07.274434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2933333093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2933333093},{"type":"v1","addr":"172.21.15.177:6837","nonce":2933333093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":9}]} 2024-09-16T07:36:15.005 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 85 2024-09-16T07:36:15.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0a1407a6e0 msgr2=0x7f0a1407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0a1407a6e0 0x7f0a1407cba0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0a24001ce0 tx=0x7f0a24009040 comp rx=0 tx=0).stop 2024-09-16T07:36:15.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340ccf00 msgr2=0x7f0a3409c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340ccf00 0x7f0a3409c0d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c410 tx=0x7f0a2000c8d0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 shutdown_connections 2024-09-16T07:36:15.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0a1407a6e0 0x7f0a1407cba0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a3415f5a0 0x7f0a340cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a340ccf00 0x7f0a3409c0d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 --2- 172.21.15.71:0/2191215116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a340b2440 0x7f0a341f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 >> 172.21.15.71:0/2191215116 conn(0x7f0a340b1950 msgr2=0x7f0a340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:15.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 shutdown_connections 2024-09-16T07:36:15.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.007+0000 7f0a38a00640 1 -- 172.21.15.71:0/2191215116 wait complete. 2024-09-16T07:36:15.426 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:36:04.253023+0000', 'modified': '2024-09-16T07:36:07.274434+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8919}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8919': {'gid': 8919, 'name': 'd', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6837/2933333093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6835', 'nonce': 2933333093}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 2933333093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8919, 'qdb_cluster': [8919]} 2024-09-16T07:36:15.426 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:36:15.427 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-16T07:36:15.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.586+0000 7f830aa00640 1 -- 172.21.15.71:0/2742013123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ec004d40 msgr2=0x7f82ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.586+0000 7f830aa00640 1 --2- 172.21.15.71:0/2742013123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ec004d40 0x7f82ec0051c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f83040cc470 tx=0x7f82ec005830 comp rx=0 tx=0).stop 2024-09-16T07:36:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.587+0000 7f830a000640 1 -- 172.21.15.71:0/2742013123 reap_dead start 2024-09-16T07:36:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.587+0000 7f830aa00640 1 -- 172.21.15.71:0/2742013123 shutdown_connections 2024-09-16T07:36:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.587+0000 7f830aa00640 1 -- 172.21.15.71:0/2742013123 >> 172.21.15.71:0/2742013123 conn(0x7f83040cfa20 msgr2=0x7f83040cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.587+0000 7f830aa00640 1 -- 172.21.15.71:0/2742013123 shutdown_connections 2024-09-16T07:36:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.587+0000 7f830aa00640 1 -- 172.21.15.71:0/2742013123 wait complete. 2024-09-16T07:36:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.588+0000 7f830aa00640 1 Processor -- start 2024-09-16T07:36:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.588+0000 7f830aa00640 1 -- start start 2024-09-16T07:36:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.589+0000 7f830aa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82ec004d40 0x7f83041e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.589+0000 7f830aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83040c84b0 0x7f83041f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.589+0000 7f830aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83041f0760 0x7f83041f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.589+0000 7f830aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83040d0970 con 0x7f83040c84b0 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.589+0000 7f830aa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f83040ca750 con 0x7f83041f0760 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.589+0000 7f830aa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8304072a40 con 0x7f82ec004d40 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83040c84b0 0x7f83041f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83040c84b0 0x7f83041f0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40212/0 (socket says 172.21.15.71:40212) 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 -- 172.21.15.71:0/1067548490 learned_addr learned my addr 172.21.15.71:0/1067548490 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8309600640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82ec004d40 0x7f83041e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f830a000640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83041f0760 0x7f83041f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 -- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82ec004d40 msgr2=0x7f83041e1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82ec004d40 0x7f83041e1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 -- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83041f0760 msgr2=0x7f83041f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83041f0760 0x7f83041f0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 -- 172.21.15.71:0/1067548490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83041e23f0 con 0x7f83040c84b0 2024-09-16T07:36:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.590+0000 7f8308c00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83040c84b0 0x7f83041f0220 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f82f000e660 tx=0x7f82f000ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.591+0000 7f8301600640 1 -- 172.21.15.71:0/1067548490 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f00035f0 con 0x7f83040c84b0 2024-09-16T07:36:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.591+0000 7f8301600640 1 -- 172.21.15.71:0/1067548490 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82f0016070 con 0x7f83040c84b0 2024-09-16T07:36:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.591+0000 7f8301600640 1 -- 172.21.15.71:0/1067548490 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0003b00 con 0x7f83040c84b0 2024-09-16T07:36:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.591+0000 7f830aa00640 1 -- 172.21.15.71:0/1067548490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83040c8cf0 con 0x7f83040c84b0 2024-09-16T07:36:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.591+0000 7f830aa00640 1 -- 172.21.15.71:0/1067548490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83040c92b0 con 0x7f83040c84b0 2024-09-16T07:36:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.593+0000 7f8301600640 1 -- 172.21.15.71:0/1067548490 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f82f0003ca0 con 0x7f83040c84b0 2024-09-16T07:36:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.594+0000 7f8301600640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82e407a6b0 0x7f82e407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.594+0000 7f8301600640 1 -- 172.21.15.71:0/1067548490 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f82f0099d10 con 0x7f83040c84b0 2024-09-16T07:36:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.594+0000 7f830efc1740 1 Processor -- start 2024-09-16T07:36:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.594+0000 7f8309600640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82e407a6b0 0x7f82e407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.594+0000 7f830efc1740 1 -- start start 2024-09-16T07:36:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.595+0000 7f8309600640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82e407a6b0 0x7f82e407cb70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009c50 tx=0x7f82f8005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.595+0000 7f830efc1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af49095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.595+0000 7f830efc1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49050c10 0x55af49094580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.595+0000 7f830efc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.595+0000 7f830efc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55af48fd3430 con 0x55af49051190 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.595+0000 7f830efc1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55af48fc35d0 con 0x55af49095aa0 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.595+0000 7f830efc1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55af48fb2a00 con 0x55af49050c10 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.595+0000 7f830a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f8309600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af49095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f830a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40216/0 (socket says 172.21.15.71:40216) 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f830a000640 1 -- 172.21.15.71:0/2602916220 learned_addr learned my addr 172.21.15.71:0/2602916220 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f8308c00640 1 --2- 172.21.15.71:0/2602916220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49050c10 0x55af49094580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f830a000640 1 -- 172.21.15.71:0/2602916220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49050c10 msgr2=0x55af49094580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:36:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f830a000640 1 --2- 172.21.15.71:0/2602916220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49050c10 0x55af49094580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f830a000640 1 -- 172.21.15.71:0/2602916220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 msgr2=0x55af49095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:36:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f830a000640 1 --2- 172.21.15.71:0/2602916220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af49095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f830a000640 1 -- 172.21.15.71:0/2602916220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af48f7b520 con 0x55af49051190 2024-09-16T07:36:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.596+0000 7f830a000640 1 --2- 172.21.15.71:0/2602916220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094ac0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f82fc036040 tx=0x7f82fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996f3914445e665b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.597+0000 7f82f5c00640 1 -- 172.21.15.71:0/2602916220 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc038040 con 0x55af49051190 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.597+0000 7f82f5c00640 1 -- 172.21.15.71:0/2602916220 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82fc00ac10 con 0x55af49051190 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.597+0000 7f82f5c00640 1 -- 172.21.15.71:0/2602916220 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc004d40 con 0x55af49051190 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.597+0000 7f830efc1740 1 -- 172.21.15.71:0/2602916220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 msgr2=0x55af49094ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.597+0000 7f830efc1740 1 --2- 172.21.15.71:0/2602916220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094ac0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f82fc036040 tx=0x7f82fc004a40 comp rx=0 tx=0).stop 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.597+0000 7f830efc1740 1 -- 172.21.15.71:0/2602916220 shutdown_connections 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.597+0000 7f830efc1740 1 --2- 172.21.15.71:0/2602916220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094ac0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.597+0000 7f830efc1740 1 --2- 172.21.15.71:0/2602916220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49050c10 0x55af49094580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.598+0000 7f830efc1740 1 --2- 172.21.15.71:0/2602916220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af49095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.598+0000 7f830efc1740 1 -- 172.21.15.71:0/2602916220 >> 172.21.15.71:0/2602916220 conn(0x55af4904c1e0 msgr2=0x55af4904fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.598+0000 7f830efc1740 1 -- 172.21.15.71:0/2602916220 shutdown_connections 2024-09-16T07:36:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.598+0000 7f830efc1740 1 -- 172.21.15.71:0/2602916220 wait complete. 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.598+0000 7f830efc1740 1 -- start start 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.598+0000 7f830efc1740 1 Processor -- start 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.599+0000 7f830efc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.599+0000 7f830efc1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af490900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.599+0000 7f830efc1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 0x55af4905ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.599+0000 7f830efc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55af48f59330 con 0x55af49051190 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.599+0000 7f830efc1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55af48fb6810 con 0x55af49095aa0 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.599+0000 7f830efc1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55af48d23620 con 0x55af490905e0 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f830efc1740 10 client.0 _ll_register_callbacks cb 0x55af48f5e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40228/0 (socket says 172.21.15.71:40228) 2024-09-16T07:36:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 -- 172.21.15.71:0/3802207286 learned_addr learned my addr 172.21.15.71:0/3802207286 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8308c00640 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af490900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f830a000640 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 0x55af4905ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 -- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 msgr2=0x55af4905ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 0x55af4905ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 -- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 msgr2=0x55af490900a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af490900a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 -- 172.21.15.71:0/3802207286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af48d14850 con 0x55af49051190 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8308c00640 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af490900a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f8309600640 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094e70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f82f801da50 tx=0x7f82f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f1ca3ffa64fbd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.600+0000 7f82ea000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:36:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.601+0000 7f82ea000640 1 -- 172.21.15.71:0/3802207286 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f800f690 con 0x55af49051190 2024-09-16T07:36:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.601+0000 7f82ea000640 1 -- 172.21.15.71:0/3802207286 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f82f800f830 con 0x55af49051190 2024-09-16T07:36:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.601+0000 7f830efc1740 1 -- 172.21.15.71:0/3802207286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af48fc5520 con 0x55af49051190 2024-09-16T07:36:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.601+0000 7f82ea000640 1 -- 172.21.15.71:0/3802207286 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f82f800fa30 con 0x55af49051190 2024-09-16T07:36:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.601+0000 7f830efc1740 10 client.9054 fetch_fsmap learned FSMap version 85 2024-09-16T07:36:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.601+0000 7f830efc1740 1 -- 172.21.15.71:0/3802207286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55af48d8b6b0 con 0x55af49051190 2024-09-16T07:36:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.601+0000 7f82ea000640 1 -- 172.21.15.71:0/3802207286 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55af48fc5520 con 0x55af49051190 2024-09-16T07:36:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.601+0000 7f82ea000640 1 -- 172.21.15.71:0/3802207286 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f82f8038020 con 0x55af49051190 2024-09-16T07:36:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.602+0000 7f830efc1740 10 client.9054 fetch_fsmap finished waiting for FSMap version 85 2024-09-16T07:36:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.602+0000 7f830efc1740 10 client.9054 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:36:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.602+0000 7f830efc1740 20 client.9054 populate_metadata read hostname 'smithi071' 2024-09-16T07:36:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.602+0000 7f830efc1740 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] conn(0x55af49075c80 0x55af49076060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.602+0000 7f830efc1740 4 client.9054 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.602+0000 7f830efc1740 1 -- 172.21.15.71:0/3802207286 --> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af48fc0fd0 con 0x55af49075c80 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.602+0000 7f8308c00640 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] conn(0x55af49075c80 0x55af49076060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.603+0000 7f8308c00640 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] conn(0x55af49075c80 0x55af49076060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1445cc5313bbdf70 server_cookie=21101377489c2a9c in_seq=0 out_seq=0 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.603+0000 7f82ea000640 10 client.9054 ms_handle_connect on v2:172.21.15.177:6835/2933333093 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f82ea000640 1 -- 172.21.15.71:0/3802207286 <== mds.0 v2:172.21.15.177:6835/2933333093 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af48fc0fd0 con 0x55af49075c80 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f82ea000640 10 client.9054 handle_command_reply: tid=0 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 client.9054 shutdown 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 -- 172.21.15.71:0/3802207286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 msgr2=0x55af49094e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094e70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f82f801da50 tx=0x7f82f8004730 comp rx=0 tx=0).stop 2024-09-16T07:36:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 -- 172.21.15.71:0/3802207286 shutdown_connections 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] conn(0x55af49075c80 0x55af49076060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 0x55af4905ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af490900a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f82ea000640 0 client.9054 ms_handle_reset on v2:172.21.15.177:6835/2933333093 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 --2- 172.21.15.71:0/3802207286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af49094e70 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 -- 172.21.15.71:0/3802207286 >> 172.21.15.71:0/3802207286 conn(0x55af49050c10 msgr2=0x55af49049910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 -- 172.21.15.71:0/3802207286 shutdown_connections 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 1 -- 172.21.15.71:0/3802207286 wait complete. 2024-09-16T07:36:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.604+0000 7f830efc1740 20 client.9054 trim_cache size 0 max 16384 2024-09-16T07:36:15.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.623+0000 7f830efc1740 1 Processor -- start 2024-09-16T07:36:15.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.623+0000 7f830efc1740 1 -- start start 2024-09-16T07:36:15.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f830efc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af490bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f830efc1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f830efc1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f830efc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83041fd240 con 0x55af49051190 2024-09-16T07:36:15.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f830efc1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f83041fc2c0 con 0x55af49095aa0 2024-09-16T07:36:15.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f830efc1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f83040c46b0 con 0x55af490905e0 2024-09-16T07:36:15.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f8309600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af490bd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f8309600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af490bd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40242/0 (socket says 172.21.15.71:40242) 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f8309600640 1 -- 172.21.15.71:0/2316435433 learned_addr learned my addr 172.21.15.71:0/2316435433 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f8308c00640 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.624+0000 7f830a000640 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f8309600640 1 -- 172.21.15.71:0/2316435433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 msgr2=0x55af4904ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f8309600640 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f8309600640 1 -- 172.21.15.71:0/2316435433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 msgr2=0x55af4904b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f8309600640 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f8309600640 1 -- 172.21.15.71:0/2316435433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af490a3a10 con 0x55af49051190 2024-09-16T07:36:15.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f8309600640 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af490bd3b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f82f800fe40 tx=0x7f82f800fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896510f1eee5146e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:15.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f82f5200640 1 -- 172.21.15.71:0/2316435433 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f800f890 con 0x55af49051190 2024-09-16T07:36:15.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f82f5200640 1 -- 172.21.15.71:0/2316435433 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82f803e070 con 0x55af49051190 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.625+0000 7f82f5200640 1 -- 172.21.15.71:0/2316435433 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8028660 con 0x55af49051190 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 -- 172.21.15.71:0/2316435433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 msgr2=0x55af490bd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af490bd3b0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f82f800fe40 tx=0x7f82f800fe70 comp rx=0 tx=0).stop 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 -- 172.21.15.71:0/2316435433 shutdown_connections 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 --2- 172.21.15.71:0/2316435433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af490bd3b0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 -- 172.21.15.71:0/2316435433 >> 172.21.15.71:0/2316435433 conn(0x55af49050c10 msgr2=0x55af4908dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 -- 172.21.15.71:0/2316435433 shutdown_connections 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.626+0000 7f830efc1740 1 -- 172.21.15.71:0/2316435433 wait complete. 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.627+0000 7f830efc1740 1 -- start start 2024-09-16T07:36:15.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.627+0000 7f830efc1740 1 Processor -- start 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830efc1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af491110d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830efc1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830efc1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830efc1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55af490953b0 con 0x55af49051190 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830efc1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f83041fbc70 con 0x55af490905e0 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830efc1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55af490ac110 con 0x55af49095aa0 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830efc1740 10 client.0 _ll_register_callbacks cb 0x7f82f801d880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8309600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af491110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8309600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af491110d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40258/0 (socket says 172.21.15.71:40258) 2024-09-16T07:36:15.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8309600640 1 -- 172.21.15.71:0/467158308 learned_addr learned my addr 172.21.15.71:0/467158308 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8308c00640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830a000640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8309600640 1 -- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49095aa0 msgr2=0x55af4904b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8309600640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8309600640 1 -- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af490905e0 msgr2=0x55af4904ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8309600640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8309600640 1 -- 172.21.15.71:0/467158308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4911de10 con 0x55af49051190 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f830a000640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.628+0000 7f8308c00640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f8309600640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af491110d0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f82f801d850 tx=0x7f82f8028630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cc6151cee4818 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f82ea000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:36:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f82ea000640 1 -- 172.21.15.71:0/467158308 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8049060 con 0x55af49051190 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f830efc1740 1 -- 172.21.15.71:0/467158308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f803e070 con 0x55af49051190 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f82ea000640 1 -- 172.21.15.71:0/467158308 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f82f8028c30 con 0x55af49051190 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f82ea000640 1 -- 172.21.15.71:0/467158308 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f82f804f050 con 0x55af49051190 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f830efc1740 10 client.9060 fetch_fsmap learned FSMap version 85 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f830efc1740 1 -- 172.21.15.71:0/467158308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55af490b7de0 con 0x55af49051190 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f82ea000640 1 -- 172.21.15.71:0/467158308 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82f803e070 con 0x55af49051190 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f82ea000640 1 -- 172.21.15.71:0/467158308 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f82f8052020 con 0x55af49051190 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.629+0000 7f830efc1740 10 client.9060 fetch_fsmap finished waiting for FSMap version 85 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.630+0000 7f830efc1740 10 client.9060 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.630+0000 7f830efc1740 20 client.9060 populate_metadata read hostname 'smithi071' 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.630+0000 7f830efc1740 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] conn(0x55af4911b320 0x55af4908edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.630+0000 7f830efc1740 4 client.9060 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.630+0000 7f830efc1740 1 -- 172.21.15.71:0/467158308 --> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] -- command(tid 0: {"prefix": "status"}) -- 0x55af48fbe400 con 0x55af4911b320 2024-09-16T07:36:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.630+0000 7f8308c00640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] conn(0x55af4911b320 0x55af4908edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.630+0000 7f8308c00640 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] conn(0x55af4911b320 0x55af4908edb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3743d29fae9851e server_cookie=5778a0f1033b38fc in_seq=0 out_seq=0 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.630+0000 7f82ea000640 10 client.9060 ms_handle_connect on v2:172.21.15.177:6835/2933333093 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f82ea000640 1 -- 172.21.15.71:0/467158308 <== mds.0 v2:172.21.15.177:6835/2933333093 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55af48fbe400 con 0x55af4911b320 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f82ea000640 10 client.9060 handle_command_reply: tid=0 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 client.9060 shutdown 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 -- 172.21.15.71:0/467158308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 msgr2=0x55af491110d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af491110d0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f82f801d850 tx=0x7f82f8028630 comp rx=0 tx=0).stop 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 -- 172.21.15.71:0/467158308 shutdown_connections 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:6835/2933333093,v1:172.21.15.177:6837/2933333093] conn(0x55af4911b320 0x55af4908edb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55af49095aa0 0x55af4904b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af490905e0 0x55af4904ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f82ea000640 0 client.9060 ms_handle_reset on v2:172.21.15.177:6835/2933333093 2024-09-16T07:36:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 --2- 172.21.15.71:0/467158308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af49051190 0x55af491110d0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 -- 172.21.15.71:0/467158308 >> 172.21.15.71:0/467158308 conn(0x55af49050c10 msgr2=0x55af4908dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 -- 172.21.15.71:0/467158308 shutdown_connections 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 1 -- 172.21.15.71:0/467158308 wait complete. 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.631+0000 7f830efc1740 20 client.9060 trim_cache size 0 max 16384 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 8919, 2024-09-16T07:36:15.635 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:36:15.636 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:36:15.636 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:36:15.636 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 10.786666668000001, 2024-09-16T07:36:15.636 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 85, 2024-09-16T07:36:15.636 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 133, 2024-09-16T07:36:15.636 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 133, 2024-09-16T07:36:15.636 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 15.106906473 2024-09-16T07:36:15.636 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 -- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82e407a6b0 msgr2=0x7f82e407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82e407a6b0 0x7f82e407cb70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009c50 tx=0x7f82f8005f90 comp rx=0 tx=0).stop 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 -- 172.21.15.71:0/1067548490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83040c84b0 msgr2=0x7f83041f0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83040c84b0 0x7f83041f0220 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f82f000e660 tx=0x7f82f000ea60 comp rx=0 tx=0).stop 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 -- 172.21.15.71:0/1067548490 shutdown_connections 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f82e407a6b0 0x7f82e407cb70 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83041f0760 0x7f83041f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83040c84b0 0x7f83041f0220 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 --2- 172.21.15.71:0/1067548490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82ec004d40 0x7f83041e1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 -- 172.21.15.71:0/1067548490 >> 172.21.15.71:0/1067548490 conn(0x7f83040cfa20 msgr2=0x7f83040b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:15.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 -- 172.21.15.71:0/1067548490 shutdown_connections 2024-09-16T07:36:15.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.632+0000 7f830aa00640 1 -- 172.21.15.71:0/1067548490 wait complete. 2024-09-16T07:36:15.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:36:15.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 -- 172.21.15.71:0/74932203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c15f5a0 msgr2=0x7fed8c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 --2- 172.21.15.71:0/74932203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c15f5a0 0x7fed8c0b2440 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fed8c076720 tx=0x7fed7c02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 -- 172.21.15.71:0/74932203 shutdown_connections 2024-09-16T07:36:15.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 --2- 172.21.15.71:0/74932203 >> v2:172.21.15.177:3300/0 conn(0x7fed8c0ad270 0x7fed8c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 --2- 172.21.15.71:0/74932203 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed8c0ccf00 0x7fed8c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 --2- 172.21.15.71:0/74932203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c15f5a0 0x7fed8c0b2440 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 -- 172.21.15.71:0/74932203 >> 172.21.15.71:0/74932203 conn(0x7fed8c0b1950 msgr2=0x7fed8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:15.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 -- 172.21.15.71:0/74932203 shutdown_connections 2024-09-16T07:36:15.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.804+0000 7fed91a00640 1 -- 172.21.15.71:0/74932203 wait complete. 2024-09-16T07:36:15.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.806+0000 7fed91a00640 1 Processor -- start 2024-09-16T07:36:15.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.806+0000 7fed91a00640 1 -- start start 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed91a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c0ad270 0x7fed8c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed91a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed8c0ccf00 0x7fed8c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed91a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed8c15f5a0 0x7fed8c14a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed91a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c06f160 con 0x7fed8c0ad270 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed91a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c074780 con 0x7fed8c15f5a0 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed91a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fed8c06f2e0 con 0x7fed8c0ccf00 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c0ad270 0x7fed8c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c0ad270 0x7fed8c156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40280/0 (socket says 172.21.15.71:40280) 2024-09-16T07:36:15.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8aa00640 1 -- 172.21.15.71:0/1792172631 learned_addr learned my addr 172.21.15.71:0/1792172631 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8a000640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed8c0ccf00 0x7fed8c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8b400640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed8c15f5a0 0x7fed8c14a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8aa00640 1 -- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed8c0ccf00 msgr2=0x7fed8c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8aa00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed8c0ccf00 0x7fed8c0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8aa00640 1 -- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed8c15f5a0 msgr2=0x7fed8c14a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8aa00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed8c15f5a0 0x7fed8c14a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8aa00640 1 -- 172.21.15.71:0/1792172631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c156c50 con 0x7fed8c0ad270 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.807+0000 7fed8b400640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed8c15f5a0 0x7fed8c14a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.808+0000 7fed8aa00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c0ad270 0x7fed8c156090 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fed7c000c00 tx=0x7fed7c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.808+0000 7fed82a00640 1 -- 172.21.15.71:0/1792172631 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c03e030 con 0x7fed8c0ad270 2024-09-16T07:36:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.808+0000 7fed82a00640 1 -- 172.21.15.71:0/1792172631 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed7c0047a0 con 0x7fed8c0ad270 2024-09-16T07:36:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.808+0000 7fed82a00640 1 -- 172.21.15.71:0/1792172631 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c035040 con 0x7fed8c0ad270 2024-09-16T07:36:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.808+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c156f30 con 0x7fed8c0ad270 2024-09-16T07:36:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.808+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c0cf510 con 0x7fed8c0ad270 2024-09-16T07:36:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.809+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c157170 con 0x7fed8c0ad270 2024-09-16T07:36:15.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.810+0000 7fed82a00640 1 -- 172.21.15.71:0/1792172631 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fed7c041030 con 0x7fed8c0ad270 2024-09-16T07:36:15.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.811+0000 7fed82a00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fed6c07a6e0 0x7fed6c07cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:15.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.811+0000 7fed82a00640 1 -- 172.21.15.71:0/1792172631 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7fed7c0b9f30 con 0x7fed8c0ad270 2024-09-16T07:36:15.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.812+0000 7fed8a000640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fed6c07a6e0 0x7fed6c07cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:15.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.812+0000 7fed8a000640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fed6c07a6e0 0x7fed6c07cba0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fed8c14e1a0 tx=0x7fed78009110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:15.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:15.815+0000 7fed82a00640 1 -- 172.21.15.71:0/1792172631 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fed7c0840a0 con 0x7fed8c0ad270 2024-09-16T07:36:16.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.036+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fed8c06f810 con 0x7fed8c0ad270 2024-09-16T07:36:16.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.037+0000 7fed82a00640 1 -- 172.21.15.71:0/1792172631 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7fed7c0839a0 con 0x7fed8c0ad270 2024-09-16T07:36:16.037 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fed6c07a6e0 msgr2=0x7fed6c07cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fed6c07a6e0 0x7fed6c07cba0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fed8c14e1a0 tx=0x7fed78009110 comp rx=0 tx=0).stop 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c0ad270 msgr2=0x7fed8c156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c0ad270 0x7fed8c156090 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fed7c000c00 tx=0x7fed7c0039e0 comp rx=0 tx=0).stop 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 shutdown_connections 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fed6c07a6e0 0x7fed6c07cba0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed8c15f5a0 0x7fed8c14a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed8c0ccf00 0x7fed8c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:16.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 --2- 172.21.15.71:0/1792172631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed8c0ad270 0x7fed8c156090 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:16.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 >> 172.21.15.71:0/1792172631 conn(0x7fed8c0b1950 msgr2=0x7fed8c09d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:16.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 shutdown_connections 2024-09-16T07:36:16.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16.039+0000 7fed91a00640 1 -- 172.21.15.71:0/1792172631 wait complete. 2024-09-16T07:36:16.054 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:36:16.054 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:36:16.054 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:36:16.054 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:36:16.054 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:36:16.055 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:36:16.055 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:36:16.055 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:36:16.055 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:36:16.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:16.055 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:36:16.090 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:36:16.091 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 41901sec preferred_lft 41901sec 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:36:16.092 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:36:16.093 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:36:16.093 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:36:16.093 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:36:16.093 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:36:16.093 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:36:16.093 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:36:16.093 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:36:16.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:16.093 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:36:16.184 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:36:16.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:16.188 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:36:16.223 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:36:16.223 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:36:16.224 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:36:16.224 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:36:16.224 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:36:16.224 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:36:16.224 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:36:16.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:36:16.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:36:16.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:36:16.224 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:36:16.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:36:16.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:36:16.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:16.419 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:36:16.455 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:36:16.455 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:36:16.456 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:36:16.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:16.456 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:36:16.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:36:16.551 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:36:16.586 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:36:16.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:36:16.586 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:36:16.661 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:36:16.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:16.661 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:36:16.756 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:36:16.760 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:36:16.790 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:36:16.820 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:36:16.820 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:36:16.820 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:36:16.821 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:36:16.821 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:36:16.823 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:36:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:36:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:36:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:36:16.823 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:36:16.826 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:36:16.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:16.826 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:36:16.953 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:36:17.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:36:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:36:17.050 INFO:teuthology.orchestra.run.smithi071.stdout:client8200 2024-09-16T07:36:17.050 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:36:17.050 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:36:17.085 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:36:17.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.286+0000 7f7869400640 1 -- 172.21.15.71:0/3916411862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415aba0 msgr2=0x7f786415af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:17.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.286+0000 7f7869400640 1 --2- 172.21.15.71:0/3916411862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415aba0 0x7f786415af80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f7854009830 tx=0x7f785402a560 comp rx=0 tx=0).stop 2024-09-16T07:36:17.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.287+0000 7f7869400640 1 -- 172.21.15.71:0/3916411862 shutdown_connections 2024-09-16T07:36:17.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.287+0000 7f7869400640 1 --2- 172.21.15.71:0/3916411862 >> v2:172.21.15.177:3300/0 conn(0x7f7864073780 0x7f786406f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:17.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.287+0000 7f7869400640 1 --2- 172.21.15.71:0/3916411862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415aba0 0x7f786415af80 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:17.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.287+0000 7f7869400640 1 -- 172.21.15.71:0/3916411862 >> 172.21.15.71:0/3916411862 conn(0x7f786406cb70 msgr2=0x7f786406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:17.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.287+0000 7f7869400640 1 -- 172.21.15.71:0/3916411862 shutdown_connections 2024-09-16T07:36:17.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.287+0000 7f7869400640 1 -- 172.21.15.71:0/3916411862 wait complete. 2024-09-16T07:36:17.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.288+0000 7f7869400640 1 Processor -- start 2024-09-16T07:36:17.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.288+0000 7f7869400640 1 -- start start 2024-09-16T07:36:17.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.289+0000 7f7869400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7864073780 0x7f786414dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:17.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.289+0000 7f7869400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864073fd0 0x7f78641e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.289+0000 7f7869400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786415aba0 0x7f78641ea500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.289+0000 7f7869400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f786406f960 con 0x7f7864073780 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.289+0000 7f7869400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f786406f7e0 con 0x7f786415aba0 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.289+0000 7f7869400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f786406fae0 con 0x7f7864073fd0 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7864073780 0x7f786414dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7864073780 0x7f786414dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40310/0 (socket says 172.21.15.71:40310) 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 -- 172.21.15.71:0/1171930701 learned_addr learned my addr 172.21.15.71:0/1171930701 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f785be00640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864073fd0 0x7f78641e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7863200640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786415aba0 0x7f78641ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 -- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864073fd0 msgr2=0x7f78641e9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864073fd0 0x7f78641e9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:17.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 -- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786415aba0 msgr2=0x7f78641ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:17.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786415aba0 0x7f78641ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:17.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 -- 172.21.15.71:0/1171930701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78641e6100 con 0x7f7864073780 2024-09-16T07:36:17.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.290+0000 7f7862800640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7864073780 0x7f786414dca0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f7854009950 tx=0x7f7854033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:17.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.291+0000 7f785aa00640 1 -- 172.21.15.71:0/1171930701 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854005c90 con 0x7f7864073780 2024-09-16T07:36:17.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.291+0000 7f785aa00640 1 -- 172.21.15.71:0/1171930701 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7854031e50 con 0x7f7864073780 2024-09-16T07:36:17.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.291+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78641e63e0 con 0x7f7864073780 2024-09-16T07:36:17.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.291+0000 7f785aa00640 1 -- 172.21.15.71:0/1171930701 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78540332f0 con 0x7f7864073780 2024-09-16T07:36:17.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.291+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78641f5850 con 0x7f7864073780 2024-09-16T07:36:17.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.292+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786406ff90 con 0x7f7864073780 2024-09-16T07:36:17.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.293+0000 7f785aa00640 1 -- 172.21.15.71:0/1171930701 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f7854033490 con 0x7f7864073780 2024-09-16T07:36:17.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.294+0000 7f785aa00640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784007a6e0 0x7f784007cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:17.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.294+0000 7f785aa00640 1 -- 172.21.15.71:0/1171930701 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f78540bab00 con 0x7f7864073780 2024-09-16T07:36:17.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.294+0000 7f785be00640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784007a6e0 0x7f784007cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:17.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.295+0000 7f785be00640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784007a6e0 0x7f784007cba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f784c007fd0 tx=0x7f784c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.298+0000 7f785aa00640 1 -- 172.21.15.71:0/1171930701 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f785403f030 con 0x7f7864073780 2024-09-16T07:36:17.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.520+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f7864070b90 con 0x7f7864073780 2024-09-16T07:36:17.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.897+0000 7f785aa00640 1 -- 172.21.15.71:0/1171930701 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v86) ==== 136+0+0 (secure 0 0 0) 0x7f7854084620 con 0x7f7864073780 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784007a6e0 msgr2=0x7f784007cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784007a6e0 0x7f784007cba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f784c007fd0 tx=0x7f784c009040 comp rx=0 tx=0).stop 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7864073780 msgr2=0x7f786414dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7864073780 0x7f786414dca0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f7854009950 tx=0x7f7854033c90 comp rx=0 tx=0).stop 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 shutdown_connections 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784007a6e0 0x7f784007cba0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f786415aba0 0x7f78641ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7864073fd0 0x7f78641e9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 --2- 172.21.15.71:0/1171930701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7864073780 0x7f786414dca0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 >> 172.21.15.71:0/1171930701 conn(0x7f786406cb70 msgr2=0x7f7864075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 shutdown_connections 2024-09-16T07:36:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:17.900+0000 7f7869400640 1 -- 172.21.15.71:0/1171930701 wait complete. 2024-09-16T07:36:17.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:36:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.075+0000 7f8d62800640 1 -- 172.21.15.71:0/3403757667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c0ccf00 msgr2=0x7f8d5c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.075+0000 7f8d62800640 1 --2- 172.21.15.71:0/3403757667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c0ccf00 0x7f8d5c0b2440 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c06dac0 tx=0x7f8d4802a580 comp rx=0 tx=0).stop 2024-09-16T07:36:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.076+0000 7f8d62800640 1 -- 172.21.15.71:0/3403757667 shutdown_connections 2024-09-16T07:36:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.076+0000 7f8d62800640 1 --2- 172.21.15.71:0/3403757667 >> v2:172.21.15.177:3300/0 conn(0x7f8d5c073fb0 0x7f8d5c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.076+0000 7f8d62800640 1 --2- 172.21.15.71:0/3403757667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c0ccf00 0x7f8d5c0b2440 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.076+0000 7f8d62800640 1 -- 172.21.15.71:0/3403757667 >> 172.21.15.71:0/3403757667 conn(0x7f8d5c0b1950 msgr2=0x7f8d5c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:18.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.076+0000 7f8d62800640 1 -- 172.21.15.71:0/3403757667 shutdown_connections 2024-09-16T07:36:18.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.076+0000 7f8d62800640 1 -- 172.21.15.71:0/3403757667 wait complete. 2024-09-16T07:36:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.077+0000 7f8d62800640 1 Processor -- start 2024-09-16T07:36:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.077+0000 7f8d62800640 1 -- start start 2024-09-16T07:36:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.078+0000 7f8d62800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c073fb0 0x7f8d5c156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.078+0000 7f8d62800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d5c0ccf00 0x7f8d5c14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.078+0000 7f8d62800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d5c15f5a0 0x7f8d5c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.078+0000 7f8d62800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c06f2e0 con 0x7f8d5c073fb0 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.078+0000 7f8d62800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c06f160 con 0x7f8d5c15f5a0 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.078+0000 7f8d62800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c06f460 con 0x7f8d5c0ccf00 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.078+0000 7f8d5b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c073fb0 0x7f8d5c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.078+0000 7f8d5b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c073fb0 0x7f8d5c156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40338/0 (socket says 172.21.15.71:40338) 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5b400640 1 -- 172.21.15.71:0/2820225388 learned_addr learned my addr 172.21.15.71:0/2820225388 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5aa00640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d5c0ccf00 0x7f8d5c14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5be00640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d5c15f5a0 0x7f8d5c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5b400640 1 -- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d5c0ccf00 msgr2=0x7f8d5c14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5b400640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d5c0ccf00 0x7f8d5c14c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5b400640 1 -- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d5c15f5a0 msgr2=0x7f8d5c0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5b400640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d5c15f5a0 0x7f8d5c0cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5b400640 1 -- 172.21.15.71:0/2820225388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c156e50 con 0x7f8d5c073fb0 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5be00640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d5c15f5a0 0x7f8d5c0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5aa00640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d5c0ccf00 0x7f8d5c14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d5b400640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c073fb0 0x7f8d5c156910 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00c3a0 tx=0x7f8d4c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d53400640 1 -- 172.21.15.71:0/2820225388 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c017070 con 0x7f8d5c073fb0 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d53400640 1 -- 172.21.15.71:0/2820225388 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d4c0053c0 con 0x7f8d5c073fb0 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d53400640 1 -- 172.21.15.71:0/2820225388 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c003850 con 0x7f8d5c073fb0 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.079+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c1570f0 con 0x7f8d5c073fb0 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.080+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c15ad10 con 0x7f8d5c073fb0 2024-09-16T07:36:18.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.081+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c06f9f0 con 0x7f8d5c073fb0 2024-09-16T07:36:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.082+0000 7f8d53400640 1 -- 172.21.15.71:0/2820225388 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f8d4c01a070 con 0x7f8d5c073fb0 2024-09-16T07:36:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.083+0000 7f8d53400640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8d3c07a6e0 0x7f8d3c07cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.083+0000 7f8d53400640 1 -- 172.21.15.71:0/2820225388 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f8d4c098cb0 con 0x7f8d5c073fb0 2024-09-16T07:36:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.083+0000 7f8d5aa00640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8d3c07a6e0 0x7f8d3c07cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.084+0000 7f8d5aa00640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8d3c07a6e0 0x7f8d3c07cba0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8d48033000 tx=0x7f8d48004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:18.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.087+0000 7f8d53400640 1 -- 172.21.15.71:0/2820225388 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8d4c062da0 con 0x7f8d5c073fb0 2024-09-16T07:36:18.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.292+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f8d5c070930 con 0x7f8d3c07a6e0 2024-09-16T07:36:18.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.295+0000 7f8d53400640 1 -- 172.21.15.71:0/2820225388 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f8d5c070930 con 0x7f8d3c07a6e0 2024-09-16T07:36:18.294 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:36:18.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:18.295 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:36:18.295 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:18.295 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:36:18.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8d3c07a6e0 msgr2=0x7f8d3c07cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8d3c07a6e0 0x7f8d3c07cba0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8d48033000 tx=0x7f8d48004730 comp rx=0 tx=0).stop 2024-09-16T07:36:18.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c073fb0 msgr2=0x7f8d5c156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c073fb0 0x7f8d5c156910 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00c3a0 tx=0x7f8d4c00c860 comp rx=0 tx=0).stop 2024-09-16T07:36:18.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 shutdown_connections 2024-09-16T07:36:18.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8d3c07a6e0 0x7f8d3c07cba0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d5c15f5a0 0x7f8d5c0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d5c0ccf00 0x7f8d5c14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 --2- 172.21.15.71:0/2820225388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d5c073fb0 0x7f8d5c156910 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 >> 172.21.15.71:0/2820225388 conn(0x7f8d5c0b1950 msgr2=0x7f8d5c15cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:18.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 shutdown_connections 2024-09-16T07:36:18.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.297+0000 7f8d62800640 1 -- 172.21.15.71:0/2820225388 wait complete. 2024-09-16T07:36:18.314 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:36:18.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.474+0000 7fdc5b000640 1 -- 172.21.15.71:0/2068928364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc38004800 msgr2=0x7fdc38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.474+0000 7fdc5b000640 1 --2- 172.21.15.71:0/2068928364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc38004800 0x7fdc38004c80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009b50 tx=0x7fdc38005830 comp rx=0 tx=0).stop 2024-09-16T07:36:18.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.474+0000 7fdc59000640 1 -- 172.21.15.71:0/2068928364 reap_dead start 2024-09-16T07:36:18.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.474+0000 7fdc5b000640 1 -- 172.21.15.71:0/2068928364 shutdown_connections 2024-09-16T07:36:18.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.474+0000 7fdc5b000640 1 -- 172.21.15.71:0/2068928364 >> 172.21.15.71:0/2068928364 conn(0x7fdc540b1840 msgr2=0x7fdc540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:18.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.474+0000 7fdc5b000640 1 -- 172.21.15.71:0/2068928364 shutdown_connections 2024-09-16T07:36:18.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.474+0000 7fdc5b000640 1 -- 172.21.15.71:0/2068928364 wait complete. 2024-09-16T07:36:18.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.476+0000 7fdc5b000640 1 Processor -- start 2024-09-16T07:36:18.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.476+0000 7fdc5b000640 1 -- start start 2024-09-16T07:36:18.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc5b000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc38004800 0x7fdc541f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc5b000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc540cb5c0 0x7fdc541f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc5b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc541f7e10 0x7fdc541f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc5b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdc540ced40 con 0x7fdc541f7e10 2024-09-16T07:36:18.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc5b000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdc540cebc0 con 0x7fdc38004800 2024-09-16T07:36:18.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc5b000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdc540ceec0 con 0x7fdc540cb5c0 2024-09-16T07:36:18.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc59000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc541f7e10 0x7fdc541f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc59000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc541f7e10 0x7fdc541f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40346/0 (socket says 172.21.15.71:40346) 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc59000640 1 -- 172.21.15.71:0/3746463802 learned_addr learned my addr 172.21.15.71:0/3746463802 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc53400640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc540cb5c0 0x7fdc541f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.477+0000 7fdc53e00640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc38004800 0x7fdc541f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc59000640 1 -- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc540cb5c0 msgr2=0x7fdc541f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc59000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc540cb5c0 0x7fdc541f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc59000640 1 -- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc38004800 msgr2=0x7fdc541f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc59000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc38004800 0x7fdc541f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc59000640 1 -- 172.21.15.71:0/3746463802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc540cba40 con 0x7fdc541f7e10 2024-09-16T07:36:18.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc59000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc541f7e10 0x7fdc541f8290 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c001eb0 tx=0x7fdc4c00e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:18.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc4be00640 1 -- 172.21.15.71:0/3746463802 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c010890 con 0x7fdc541f7e10 2024-09-16T07:36:18.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc4be00640 1 -- 172.21.15.71:0/3746463802 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc4c017070 con 0x7fdc541f7e10 2024-09-16T07:36:18.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc4be00640 1 -- 172.21.15.71:0/3746463802 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c010de0 con 0x7fdc541f7e10 2024-09-16T07:36:18.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc541f50c0 con 0x7fdc541f7e10 2024-09-16T07:36:18.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.478+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc541f5680 con 0x7fdc541f7e10 2024-09-16T07:36:18.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.479+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc540cc5c0 con 0x7fdc541f7e10 2024-09-16T07:36:18.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.480+0000 7fdc4be00640 1 -- 172.21.15.71:0/3746463802 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7fdc4c01c460 con 0x7fdc541f7e10 2024-09-16T07:36:18.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.481+0000 7fdc4be00640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc3407a6e0 0x7fdc3407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.481+0000 7fdc4be00640 1 -- 172.21.15.71:0/3746463802 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7fdc4c099c60 con 0x7fdc541f7e10 2024-09-16T07:36:18.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.482+0000 7fdc53e00640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc3407a6e0 0x7fdc3407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.483+0000 7fdc53e00640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc3407a6e0 0x7fdc3407cba0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005fd0 tx=0x7fdc44009500 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:18.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.486+0000 7fdc4be00640 1 -- 172.21.15.71:0/3746463802 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdc540cba40 con 0x7fdc541f7e10 2024-09-16T07:36:18.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.670+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdc540afca0 con 0x7fdc541f7e10 2024-09-16T07:36:18.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.671+0000 7fdc4be00640 1 -- 172.21.15.71:0/3746463802 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7fdc4c0636e0 con 0x7fdc541f7e10 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc3407a6e0 msgr2=0x7fdc3407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc3407a6e0 0x7fdc3407cba0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005fd0 tx=0x7fdc44009500 comp rx=0 tx=0).stop 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc541f7e10 msgr2=0x7fdc541f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc541f7e10 0x7fdc541f8290 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c001eb0 tx=0x7fdc4c00e980 comp rx=0 tx=0).stop 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 shutdown_connections 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdc3407a6e0 0x7fdc3407cba0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc541f7e10 0x7fdc541f8290 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc540cb5c0 0x7fdc541f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 --2- 172.21.15.71:0/3746463802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc38004800 0x7fdc541f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 >> 172.21.15.71:0/3746463802 conn(0x7fdc540b1840 msgr2=0x7fdc54143dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:18.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 shutdown_connections 2024-09-16T07:36:18.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.673+0000 7fdc5b000640 1 -- 172.21.15.71:0/3746463802 wait complete. 2024-09-16T07:36:18.689 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:36:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.850+0000 7f911f600640 1 -- 172.21.15.71:0/2691786334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 msgr2=0x7f90fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.850+0000 7f911f600640 1 --2- 172.21.15.71:0/2691786334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 0x7f90fc004800 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f91040016c0 tx=0x7f90fc005830 comp rx=0 tx=0).stop 2024-09-16T07:36:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.850+0000 7f911d600640 1 -- 172.21.15.71:0/2691786334 reap_dead start 2024-09-16T07:36:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.850+0000 7f911f600640 1 -- 172.21.15.71:0/2691786334 shutdown_connections 2024-09-16T07:36:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.850+0000 7f911f600640 1 -- 172.21.15.71:0/2691786334 >> 172.21.15.71:0/2691786334 conn(0x7f91180b1840 msgr2=0x7f91180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.850+0000 7f911f600640 1 -- 172.21.15.71:0/2691786334 shutdown_connections 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.850+0000 7f911f600640 1 -- 172.21.15.71:0/2691786334 wait complete. 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.852+0000 7f911f600640 1 Processor -- start 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.852+0000 7f911f600640 1 -- start start 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911f600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 0x7f91181566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911f600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f911813c170 0x7f9118143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911f600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118156c00 0x7f9118144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911f600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91180ce2c0 con 0x7f90fc004000 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911f600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f91180736a0 con 0x7f9118156c00 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911f600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f91180ce440 con 0x7f911813c170 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 0x7f91181566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911cc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 0x7f91181566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40366/0 (socket says 172.21.15.71:40366) 2024-09-16T07:36:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911cc00640 1 -- 172.21.15.71:0/3444097375 learned_addr learned my addr 172.21.15.71:0/3444097375 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:18.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911d600640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118156c00 0x7f9118144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f9117e00640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f911813c170 0x7f9118143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911cc00640 1 -- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f911813c170 msgr2=0x7f9118143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911cc00640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f911813c170 0x7f9118143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911cc00640 1 -- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118156c00 msgr2=0x7f9118144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:18.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911cc00640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118156c00 0x7f9118144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:18.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911cc00640 1 -- 172.21.15.71:0/3444097375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9118155620 con 0x7f90fc004000 2024-09-16T07:36:18.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f911d600640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118156c00 0x7f9118144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:36:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.853+0000 7f9117e00640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f911813c170 0x7f9118143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:36:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.854+0000 7f911cc00640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 0x7f91181566c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9108011d90 tx=0x7f910800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.854+0000 7f9114c00640 1 -- 172.21.15.71:0/3444097375 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108003d80 con 0x7f90fc004000 2024-09-16T07:36:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.854+0000 7f9114c00640 1 -- 172.21.15.71:0/3444097375 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9108004b20 con 0x7f90fc004000 2024-09-16T07:36:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.854+0000 7f9114c00640 1 -- 172.21.15.71:0/3444097375 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108004050 con 0x7f90fc004000 2024-09-16T07:36:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.854+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9118155900 con 0x7f90fc004000 2024-09-16T07:36:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.854+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9118155ec0 con 0x7f90fc004000 2024-09-16T07:36:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.855+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91180ce9a0 con 0x7f90fc004000 2024-09-16T07:36:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.856+0000 7f9114c00640 1 -- 172.21.15.71:0/3444097375 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f91080041f0 con 0x7f90fc004000 2024-09-16T07:36:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.857+0000 7f9114c00640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90f807a6e0 0x7f90f807cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.857+0000 7f9114c00640 1 -- 172.21.15.71:0/3444097375 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f910809ab00 con 0x7f90fc004000 2024-09-16T07:36:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.858+0000 7f9117e00640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90f807a6e0 0x7f90f807cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.858+0000 7f9117e00640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90f807a6e0 0x7f90f807cba0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9104032000 tx=0x7f9104001190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:18.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:18.861+0000 7f9114c00640 1 -- 172.21.15.71:0/3444097375 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9108064bd0 con 0x7f90fc004000 2024-09-16T07:36:19.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.049+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f91181436f0 con 0x7f90fc004000 2024-09-16T07:36:19.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.049+0000 7f9114c00640 1 -- 172.21.15.71:0/3444097375 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v51) ==== 116+0+2 (secure 0 0 0) 0x7f91080644d0 con 0x7f90fc004000 2024-09-16T07:36:19.049 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:36:19.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90f807a6e0 msgr2=0x7f90f807cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90f807a6e0 0x7f90f807cba0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9104032000 tx=0x7f9104001190 comp rx=0 tx=0).stop 2024-09-16T07:36:19.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 msgr2=0x7f91181566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 0x7f91181566c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9108011d90 tx=0x7f910800f790 comp rx=0 tx=0).stop 2024-09-16T07:36:19.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 shutdown_connections 2024-09-16T07:36:19.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f90f807a6e0 0x7f90f807cba0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118156c00 0x7f9118144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f911813c170 0x7f9118143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.051+0000 7f911f600640 1 --2- 172.21.15.71:0/3444097375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90fc004000 0x7f91181566c0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.052+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 >> 172.21.15.71:0/3444097375 conn(0x7f91180b1840 msgr2=0x7f91180aa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:19.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.052+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 shutdown_connections 2024-09-16T07:36:19.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.052+0000 7f911f600640 1 -- 172.21.15.71:0/3444097375 wait complete. 2024-09-16T07:36:19.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:36:19.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.226+0000 7f8045800640 1 -- 172.21.15.71:0/2487433857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 msgr2=0x7f80400cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.226+0000 7f8045800640 1 --2- 172.21.15.71:0/2487433857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 0x7f80400cdbb0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f80400d0cd0 tx=0x7f803802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.226+0000 7f8045800640 1 -- 172.21.15.71:0/2487433857 shutdown_connections 2024-09-16T07:36:19.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.226+0000 7f8045800640 1 --2- 172.21.15.71:0/2487433857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 0x7f80400cdbb0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.226+0000 7f8045800640 1 --2- 172.21.15.71:0/2487433857 >> v2:172.21.15.177:3300/0 conn(0x7f8040145750 0x7f8040073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.226+0000 7f8045800640 1 -- 172.21.15.71:0/2487433857 >> 172.21.15.71:0/2487433857 conn(0x7f80400b1840 msgr2=0x7f80400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.226+0000 7f8045800640 1 -- 172.21.15.71:0/2487433857 shutdown_connections 2024-09-16T07:36:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.226+0000 7f8045800640 1 -- 172.21.15.71:0/2487433857 wait complete. 2024-09-16T07:36:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.228+0000 7f8045800640 1 Processor -- start 2024-09-16T07:36:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.228+0000 7f8045800640 1 -- start start 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f8045800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804006efa0 0x7f804013bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f8045800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 0x7f80401f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f8045800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8040145750 0x7f80400c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f8045800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f80400ce480 con 0x7f80400cccd0 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f8045800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f80400ce300 con 0x7f804006efa0 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f8045800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f80400ce600 con 0x7f8040145750 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f803e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 0x7f80401f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f803e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 0x7f80401f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40386/0 (socket says 172.21.15.71:40386) 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.229+0000 7f803e000640 1 -- 172.21.15.71:0/3105494493 learned_addr learned my addr 172.21.15.71:0/3105494493 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803ea00640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804006efa0 0x7f804013bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803f400640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8040145750 0x7f80400c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803e000640 1 -- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8040145750 msgr2=0x7f80400c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803e000640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8040145750 0x7f80400c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803e000640 1 -- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804006efa0 msgr2=0x7f804013bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803e000640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804006efa0 0x7f804013bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803e000640 1 -- 172.21.15.71:0/3105494493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80401f51c0 con 0x7f80400cccd0 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803ea00640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804006efa0 0x7f804013bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f803e000640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 0x7f80401f48c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c430 tx=0x7f802c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f8036a00640 1 -- 172.21.15.71:0/3105494493 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c011790 con 0x7f80400cccd0 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f8036a00640 1 -- 172.21.15.71:0/3105494493 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f802c017070 con 0x7f80400cccd0 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f8036a00640 1 -- 172.21.15.71:0/3105494493 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c011d00 con 0x7f80400cccd0 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80401f54a0 con 0x7f80400cccd0 2024-09-16T07:36:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.230+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80400b2c40 con 0x7f80400cccd0 2024-09-16T07:36:19.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.231+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80400cdbb0 con 0x7f80400cccd0 2024-09-16T07:36:19.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.233+0000 7f8036a00640 1 -- 172.21.15.71:0/3105494493 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f802c01c460 con 0x7f80400cccd0 2024-09-16T07:36:19.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.233+0000 7f8036a00640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f802007a6b0 0x7f802007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:19.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.234+0000 7f8036a00640 1 -- 172.21.15.71:0/3105494493 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f802c099860 con 0x7f80400cccd0 2024-09-16T07:36:19.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.234+0000 7f803ea00640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f802007a6b0 0x7f802007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:19.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.235+0000 7f803ea00640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f802007a6b0 0x7f802007cb70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f80400d06d0 tx=0x7f8030009480 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:19.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.238+0000 7f8036a00640 1 -- 172.21.15.71:0/3105494493 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f802c063930 con 0x7f80400cccd0 2024-09-16T07:36:19.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.421+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f80400cf590 con 0x7f80400cccd0 2024-09-16T07:36:19.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.425+0000 7f8036a00640 1 -- 172.21.15.71:0/3105494493 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7f802c063230 con 0x7f80400cccd0 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f802007a6b0 msgr2=0x7f802007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f802007a6b0 0x7f802007cb70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f80400d06d0 tx=0x7f8030009480 comp rx=0 tx=0).stop 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 msgr2=0x7f80401f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 0x7f80401f48c0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c430 tx=0x7f802c00c8f0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 shutdown_connections 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f802007a6b0 0x7f802007cb70 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8040145750 0x7f80400c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80400cccd0 0x7f80401f48c0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 --2- 172.21.15.71:0/3105494493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804006efa0 0x7f804013bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 >> 172.21.15.71:0/3105494493 conn(0x7f80400b1840 msgr2=0x7f80401429f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:19.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 shutdown_connections 2024-09-16T07:36:19.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.428+0000 7f8045800640 1 -- 172.21.15.71:0/3105494493 wait complete. 2024-09-16T07:36:19.444 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T07:36:19.449 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.449+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.449 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.449+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.449 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.449+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.449 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.449+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.449 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.449+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.606+0000 7f34d5e00640 1 -- 172.21.15.71:0/317175938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b4005210 msgr2=0x7f34b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.606+0000 7f34d5e00640 1 --2- 172.21.15.71:0/317175938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b4005210 0x7f34b400b9c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f34c8001d90 tx=0x7f34b4005830 comp rx=0 tx=0).stop 2024-09-16T07:36:19.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.606+0000 7f34d5e00640 1 -- 172.21.15.71:0/317175938 shutdown_connections 2024-09-16T07:36:19.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.606+0000 7f34d5e00640 1 --2- 172.21.15.71:0/317175938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b4005210 0x7f34b400b9c0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.606+0000 7f34d5e00640 1 -- 172.21.15.71:0/317175938 >> 172.21.15.71:0/317175938 conn(0x7f34d00b18e0 msgr2=0x7f34d00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:19.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.606+0000 7f34d5e00640 1 -- 172.21.15.71:0/317175938 shutdown_connections 2024-09-16T07:36:19.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.606+0000 7f34d5e00640 1 -- 172.21.15.71:0/317175938 wait complete. 2024-09-16T07:36:19.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.608+0000 7f34d5e00640 1 Processor -- start 2024-09-16T07:36:19.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.608+0000 7f34d5e00640 1 -- start start 2024-09-16T07:36:19.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34d5e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b4005210 0x7f34d015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:19.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34d5e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34d00c8f60 0x7f34d015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34d5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d01f7f00 0x7f34d01f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34d5e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34d006f560 con 0x7f34d01f7f00 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34d5e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34d006f3e0 con 0x7f34b4005210 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34d5e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f34d006f6e0 con 0x7f34d00c8f60 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d01f7f00 0x7f34d01f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d01f7f00 0x7f34d01f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40404/0 (socket says 172.21.15.71:40404) 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34cf400640 1 -- 172.21.15.71:0/577562857 learned_addr learned my addr 172.21.15.71:0/577562857 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34ce000640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34d00c8f60 0x7f34d015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.609+0000 7f34cea00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b4005210 0x7f34d015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34cf400640 1 -- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34d00c8f60 msgr2=0x7f34d015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34cf400640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34d00c8f60 0x7f34d015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34cf400640 1 -- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b4005210 msgr2=0x7f34d015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34cf400640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b4005210 0x7f34d015cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34cf400640 1 -- 172.21.15.71:0/577562857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d00cd4a0 con 0x7f34d01f7f00 2024-09-16T07:36:19.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34cf400640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d01f7f00 0x7f34d01f8380 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f34c802ac80 tx=0x7f34c8006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:19.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34c7400640 1 -- 172.21.15.71:0/577562857 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8003d80 con 0x7f34d01f7f00 2024-09-16T07:36:19.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34c7400640 1 -- 172.21.15.71:0/577562857 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34c8005c20 con 0x7f34d01f7f00 2024-09-16T07:36:19.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d00c9500 con 0x7f34d01f7f00 2024-09-16T07:36:19.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34c7400640 1 -- 172.21.15.71:0/577562857 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8006070 con 0x7f34d01f7f00 2024-09-16T07:36:19.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.610+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d00c9ac0 con 0x7f34d01f7f00 2024-09-16T07:36:19.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.611+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d0069360 con 0x7f34d01f7f00 2024-09-16T07:36:19.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.613+0000 7f34c7400640 1 -- 172.21.15.71:0/577562857 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f34c8005030 con 0x7f34d01f7f00 2024-09-16T07:36:19.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.614+0000 7f34c7400640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34b007a6b0 0x7f34b007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:19.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.614+0000 7f34c7400640 1 -- 172.21.15.71:0/577562857 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f34c80b9ac0 con 0x7f34d01f7f00 2024-09-16T07:36:19.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.614+0000 7f34cea00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34b007a6b0 0x7f34b007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:19.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.615+0000 7f34cea00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34b007a6b0 0x7f34b007cb70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f34d0143f40 tx=0x7f34c0006720 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:19.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.618+0000 7f34c7400640 1 -- 172.21.15.71:0/577562857 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f34c8043060 con 0x7f34d01f7f00 2024-09-16T07:36:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.826+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f34d0072ab0 con 0x7f34b007a6b0 2024-09-16T07:36:19.836 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.836+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.836 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.836+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.836 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.836+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.836 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.836+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.836 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:19.836+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.875+0000 7f34c7400640 1 -- 172.21.15.71:0/577562857 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34d0072ab0 con 0x7f34b007a6b0 2024-09-16T07:36:19.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.877+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34b007a6b0 msgr2=0x7f34b007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.877+0000 7f34d5e00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34b007a6b0 0x7f34b007cb70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f34d0143f40 tx=0x7f34c0006720 comp rx=0 tx=0).stop 2024-09-16T07:36:19.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.877+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d01f7f00 msgr2=0x7f34d01f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.877+0000 7f34d5e00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d01f7f00 0x7f34d01f8380 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f34c802ac80 tx=0x7f34c8006d80 comp rx=0 tx=0).stop 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.877+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 shutdown_connections 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.877+0000 7f34d5e00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f34b007a6b0 0x7f34b007cb70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.877+0000 7f34d5e00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d01f7f00 0x7f34d01f8380 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.877+0000 7f34d5e00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34d00c8f60 0x7f34d015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.878+0000 7f34d5e00640 1 --2- 172.21.15.71:0/577562857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b4005210 0x7f34d015cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.878+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 >> 172.21.15.71:0/577562857 conn(0x7f34d00b18e0 msgr2=0x7f34d00cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.878+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 shutdown_connections 2024-09-16T07:36:19.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:19.878+0000 7f34d5e00640 1 -- 172.21.15.71:0/577562857 wait complete. 2024-09-16T07:36:19.893 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:36:20.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.053+0000 7ff3f9600640 1 -- 172.21.15.71:0/359743265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f406efa0 msgr2=0x7ff3f40cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.053+0000 7ff3f9600640 1 --2- 172.21.15.71:0/359743265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f406efa0 0x7ff3f40cc5c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40d0cd0 tx=0x7ff3e002a560 comp rx=0 tx=0).stop 2024-09-16T07:36:20.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.053+0000 7ff3f9600640 1 -- 172.21.15.71:0/359743265 shutdown_connections 2024-09-16T07:36:20.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.053+0000 7ff3f9600640 1 --2- 172.21.15.71:0/359743265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f406efa0 0x7ff3f40cc5c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.053+0000 7ff3f9600640 1 --2- 172.21.15.71:0/359743265 >> v2:172.21.15.177:3300/0 conn(0x7ff3f4145750 0x7ff3f4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.053+0000 7ff3f9600640 1 -- 172.21.15.71:0/359743265 >> 172.21.15.71:0/359743265 conn(0x7ff3f40b1840 msgr2=0x7ff3f40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:20.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.054+0000 7ff3f9600640 1 -- 172.21.15.71:0/359743265 shutdown_connections 2024-09-16T07:36:20.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.054+0000 7ff3f9600640 1 -- 172.21.15.71:0/359743265 wait complete. 2024-09-16T07:36:20.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.055+0000 7ff3f9600640 1 Processor -- start 2024-09-16T07:36:20.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.055+0000 7ff3f9600640 1 -- start start 2024-09-16T07:36:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f9600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f406efa0 0x7ff3f41f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f9600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f40ccb00 0x7ff3f413a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f9600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f4145750 0x7ff3f4141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f9600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f40ced40 con 0x7ff3f40ccb00 2024-09-16T07:36:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f9600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f40cebc0 con 0x7ff3f4145750 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f9600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f40ceec0 con 0x7ff3f406efa0 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f40ccb00 0x7ff3f413a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f40ccb00 0x7ff3f413a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40426/0 (socket says 172.21.15.71:40426) 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f2000640 1 -- 172.21.15.71:0/170389323 learned_addr learned my addr 172.21.15.71:0/170389323 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f3400640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f4145750 0x7ff3f4141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.056+0000 7ff3f2a00640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f406efa0 0x7ff3f41f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3f2000640 1 -- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f406efa0 msgr2=0x7ff3f41f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3f2000640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f406efa0 0x7ff3f41f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3f2000640 1 -- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f4145750 msgr2=0x7ff3f4141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3f2000640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f4145750 0x7ff3f4141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3f2000640 1 -- 172.21.15.71:0/170389323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f414e3b0 con 0x7ff3f40ccb00 2024-09-16T07:36:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3f2000640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f40ccb00 0x7ff3f413a6d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0000c00 tx=0x7ff3e0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:20.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3eaa00640 1 -- 172.21.15.71:0/170389323 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e003f070 con 0x7ff3f40ccb00 2024-09-16T07:36:20.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3eaa00640 1 -- 172.21.15.71:0/170389323 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3e00047a0 con 0x7ff3f40ccb00 2024-09-16T07:36:20.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3eaa00640 1 -- 172.21.15.71:0/170389323 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0035030 con 0x7ff3f40ccb00 2024-09-16T07:36:20.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.057+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f413f4c0 con 0x7ff3f40ccb00 2024-09-16T07:36:20.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.058+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f41f5770 con 0x7ff3f40ccb00 2024-09-16T07:36:20.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.058+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f414e580 con 0x7ff3f40ccb00 2024-09-16T07:36:20.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.060+0000 7ff3eaa00640 1 -- 172.21.15.71:0/170389323 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7ff3e0042070 con 0x7ff3f40ccb00 2024-09-16T07:36:20.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.061+0000 7ff3eaa00640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff3d407a6b0 0x7ff3d407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:20.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.061+0000 7ff3eaa00640 1 -- 172.21.15.71:0/170389323 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7ff3e00ba7c0 con 0x7ff3f40ccb00 2024-09-16T07:36:20.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.061+0000 7ff3f2a00640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff3d407a6b0 0x7ff3d407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:20.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.062+0000 7ff3f2a00640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff3d407a6b0 0x7ff3d407cb70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400bb50 tx=0x7ff3e400b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:20.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.064+0000 7ff3eaa00640 1 -- 172.21.15.71:0/170389323 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff3e0084240 con 0x7ff3f40ccb00 2024-09-16T07:36:20.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.275+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff3f40cc5c0 con 0x7ff3d407a6b0 2024-09-16T07:36:20.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.281+0000 7ff3eaa00640 1 -- 172.21.15.71:0/170389323 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff3f40cc5c0 con 0x7ff3d407a6b0 2024-09-16T07:36:20.280 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff3d407a6b0 msgr2=0x7ff3d407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff3d407a6b0 0x7ff3d407cb70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400bb50 tx=0x7ff3e400b2e0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f40ccb00 msgr2=0x7ff3f413a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f40ccb00 0x7ff3f413a6d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0000c00 tx=0x7ff3e0003910 comp rx=0 tx=0).stop 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 shutdown_connections 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff3d407a6b0 0x7ff3d407cb70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f4145750 0x7ff3f4141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f40ccb00 0x7ff3f413a6d0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 --2- 172.21.15.71:0/170389323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f406efa0 0x7ff3f41f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 >> 172.21.15.71:0/170389323 conn(0x7ff3f40b1840 msgr2=0x7ff3f40aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:20.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 shutdown_connections 2024-09-16T07:36:20.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.283+0000 7ff3f9600640 1 -- 172.21.15.71:0/170389323 wait complete. 2024-09-16T07:36:20.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:36:20.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.458+0000 7f5474800640 1 -- 172.21.15.71:0/2570226921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c0cec60 msgr2=0x7f546c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.458+0000 7f5474800640 1 --2- 172.21.15.71:0/2570226921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c0cec60 0x7f546c0cf090 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f546c074ff0 tx=0x7f546802a980 comp rx=0 tx=0).stop 2024-09-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.459+0000 7f5474800640 1 -- 172.21.15.71:0/2570226921 shutdown_connections 2024-09-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.459+0000 7f5474800640 1 --2- 172.21.15.71:0/2570226921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c0cec60 0x7f546c0cf090 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.459+0000 7f5474800640 1 --2- 172.21.15.71:0/2570226921 >> v2:172.21.15.177:3300/0 conn(0x7f546c0af190 0x7f546c0b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.459+0000 7f5474800640 1 --2- 172.21.15.71:0/2570226921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f546c161760 0x7f546c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.459+0000 7f5474800640 1 -- 172.21.15.71:0/2570226921 >> 172.21.15.71:0/2570226921 conn(0x7f546c0b1840 msgr2=0x7f546c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.459+0000 7f5474800640 1 -- 172.21.15.71:0/2570226921 shutdown_connections 2024-09-16T07:36:20.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.459+0000 7f5474800640 1 -- 172.21.15.71:0/2570226921 wait complete. 2024-09-16T07:36:20.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.460+0000 7f5474800640 1 Processor -- start 2024-09-16T07:36:20.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.460+0000 7f5474800640 1 -- start start 2024-09-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5474800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f546c0af190 0x7f546c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5474800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f546c0cec60 0x7f546c1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5474800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c161760 0x7f546c1f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5474800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f546c0b3200 con 0x7f546c161760 2024-09-16T07:36:20.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5474800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f546c0b3080 con 0x7f546c0cec60 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5474800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f546c162190 con 0x7f546c0af190 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5472800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c161760 0x7f546c1f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5472800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c161760 0x7f546c1f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40440/0 (socket says 172.21.15.71:40440) 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.461+0000 7f5472800640 1 -- 172.21.15.71:0/1051702731 learned_addr learned my addr 172.21.15.71:0/1051702731 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5471400640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f546c0cec60 0x7f546c1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5471e00640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f546c0af190 0x7f546c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5472800640 1 -- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f546c0af190 msgr2=0x7f546c151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5472800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f546c0af190 0x7f546c151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5472800640 1 -- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f546c0cec60 msgr2=0x7f546c1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5472800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f546c0cec60 0x7f546c1609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5472800640 1 -- 172.21.15.71:0/1051702731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c1f5a90 con 0x7f546c161760 2024-09-16T07:36:20.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5472800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c161760 0x7f546c1f5010 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5468009d60 tx=0x7f5468003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:20.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5466000640 1 -- 172.21.15.71:0/1051702731 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468034050 con 0x7f546c161760 2024-09-16T07:36:20.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5466000640 1 -- 172.21.15.71:0/1051702731 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5468004450 con 0x7f546c161760 2024-09-16T07:36:20.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5466000640 1 -- 172.21.15.71:0/1051702731 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468043440 con 0x7f546c161760 2024-09-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.462+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c1f5d70 con 0x7f546c161760 2024-09-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.463+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c1604b0 con 0x7f546c161760 2024-09-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.464+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546c06a370 con 0x7f546c161760 2024-09-16T07:36:20.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.465+0000 7f5466000640 1 -- 172.21.15.71:0/1051702731 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 45) ==== 104041+0+0 (secure 0 0 0) 0x7f5468037020 con 0x7f546c161760 2024-09-16T07:36:20.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.466+0000 7f5466000640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f545007a6b0 0x7f545007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:20.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.466+0000 7f5466000640 1 -- 172.21.15.71:0/1051702731 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f54680bad60 con 0x7f546c161760 2024-09-16T07:36:20.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.466+0000 7f5471e00640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f545007a6b0 0x7f545007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:20.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.467+0000 7f5471e00640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f545007a6b0 0x7f545007cb70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5460005fd0 tx=0x7f5460005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:20.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.470+0000 7f5466000640 1 -- 172.21.15.71:0/1051702731 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f546c06a370 con 0x7f546c161760 2024-09-16T07:36:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.683+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f546c1f5f40 con 0x7f545007a6b0 2024-09-16T07:36:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.691+0000 7f5466000640 1 -- 172.21.15.71:0/1051702731 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f546c1f5f40 con 0x7f545007a6b0 2024-09-16T07:36:20.690 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:20.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.692+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f545007a6b0 msgr2=0x7f545007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.692+0000 7f5474800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f545007a6b0 0x7f545007cb70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5460005fd0 tx=0x7f5460005850 comp rx=0 tx=0).stop 2024-09-16T07:36:20.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.692+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c161760 msgr2=0x7f546c1f5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.692+0000 7f5474800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c161760 0x7f546c1f5010 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5468009d60 tx=0x7f5468003f10 comp rx=0 tx=0).stop 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.693+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 shutdown_connections 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.693+0000 7f5474800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f545007a6b0 0x7f545007cb70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.693+0000 7f5474800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f546c161760 0x7f546c1f5010 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.693+0000 7f5474800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f546c0cec60 0x7f546c1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.693+0000 7f5474800640 1 --2- 172.21.15.71:0/1051702731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f546c0af190 0x7f546c151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.693+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 >> 172.21.15.71:0/1051702731 conn(0x7f546c0b1840 msgr2=0x7f546c0ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.693+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 shutdown_connections 2024-09-16T07:36:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:20.693+0000 7f5474800640 1 -- 172.21.15.71:0/1051702731 wait complete. 2024-09-16T07:36:20.708 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:20.708 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:36:20.708 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:36:20.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:20.744 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:36:28.443 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:36:28.444 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:36:28.444 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.61643 s, 141 MB/s 2024-09-16T07:36:28.445 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:36:28.445 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:36:28.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:28.481 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:36:36.050 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:36:36.051 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:36:36.051 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.49157 s, 143 MB/s 2024-09-16T07:36:36.052 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:36:36.052 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:36:36.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:36.088 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:36:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:36:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:36:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.51765 s, 143 MB/s 2024-09-16T07:36:43.682 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T07:36:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.845+0000 7fa1ab000640 1 -- 172.21.15.71:0/764321234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 msgr2=0x7fa1a40741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.845+0000 7fa1ab000640 1 --2- 172.21.15.71:0/764321234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 0x7fa1a40741a0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa194005fd0 tx=0x7fa19402ab30 comp rx=0 tx=0).stop 2024-09-16T07:36:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.846+0000 7fa1ab000640 1 -- 172.21.15.71:0/764321234 shutdown_connections 2024-09-16T07:36:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.846+0000 7fa1ab000640 1 --2- 172.21.15.71:0/764321234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 0x7fa1a40741a0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.846+0000 7fa1ab000640 1 -- 172.21.15.71:0/764321234 >> 172.21.15.71:0/764321234 conn(0x7fa1a40b18e0 msgr2=0x7fa1a40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.846+0000 7fa1ab000640 1 -- 172.21.15.71:0/764321234 shutdown_connections 2024-09-16T07:36:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.846+0000 7fa1ab000640 1 -- 172.21.15.71:0/764321234 wait complete. 2024-09-16T07:36:43.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.847+0000 7fa1ab000640 1 Processor -- start 2024-09-16T07:36:43.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.847+0000 7fa1ab000640 1 -- start start 2024-09-16T07:36:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.848+0000 7fa1ab000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a406f220 0x7fa1a40cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.848+0000 7fa1ab000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 0x7fa1a41f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.848+0000 7fa1ab000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a415cff0 0x7fa1a409bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.848+0000 7fa1ab000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a406f6b0 con 0x7fa1a4159aa0 2024-09-16T07:36:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.848+0000 7fa1ab000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a40ccca0 con 0x7fa1a406f220 2024-09-16T07:36:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.848+0000 7fa1ab000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a406f830 con 0x7fa1a415cff0 2024-09-16T07:36:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 0x7fa1a41f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 0x7fa1a41f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43048/0 (socket says 172.21.15.71:43048) 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 -- 172.21.15.71:0/3792165661 learned_addr learned my addr 172.21.15.71:0/3792165661 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3e00640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a406f220 0x7fa1a40cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a9000640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a415cff0 0x7fa1a409bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 -- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a415cff0 msgr2=0x7fa1a409bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a415cff0 0x7fa1a409bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 -- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a406f220 msgr2=0x7fa1a40cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a406f220 0x7fa1a40cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 -- 172.21.15.71:0/3792165661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40ce010 con 0x7fa1a4159aa0 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa1a3400640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 0x7fa1a41f6970 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa19000c400 tx=0x7fa19000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.849+0000 7fa19be00640 1 -- 172.21.15.71:0/3792165661 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190017070 con 0x7fa1a4159aa0 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.850+0000 7fa19be00640 1 -- 172.21.15.71:0/3792165661 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1900053c0 con 0x7fa1a4159aa0 2024-09-16T07:36:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.850+0000 7fa19be00640 1 -- 172.21.15.71:0/3792165661 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190003850 con 0x7fa1a4159aa0 2024-09-16T07:36:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.850+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a40ce290 con 0x7fa1a4159aa0 2024-09-16T07:36:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.850+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a40ce850 con 0x7fa1a4159aa0 2024-09-16T07:36:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.851+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a4069360 con 0x7fa1a4159aa0 2024-09-16T07:36:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.853+0000 7fa19be00640 1 -- 172.21.15.71:0/3792165661 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 46) ==== 104143+0+0 (secure 0 0 0) 0x7fa19001a070 con 0x7fa1a4159aa0 2024-09-16T07:36:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.854+0000 7fa19be00640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa18407a7d0 0x7fa18407cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:43.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.854+0000 7fa19be00640 1 -- 172.21.15.71:0/3792165661 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7fa190098c40 con 0x7fa1a4159aa0 2024-09-16T07:36:43.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.854+0000 7fa1a3e00640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa18407a7d0 0x7fa18407cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:43.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.855+0000 7fa1a3e00640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa18407a7d0 0x7fa18407cc90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa194035000 tx=0x7fa194038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:43.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:43.858+0000 7fa19be00640 1 -- 172.21.15.71:0/3792165661 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa1a40ce010 con 0x7fa1a4159aa0 2024-09-16T07:36:44.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.074+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fa1a4155cd0 con 0x7fa18407a7d0 2024-09-16T07:36:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.088+0000 7fa19be00640 1 -- 172.21.15.71:0/3792165661 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa1a4155cd0 con 0x7fa18407a7d0 2024-09-16T07:36:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa18407a7d0 msgr2=0x7fa18407cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa18407a7d0 0x7fa18407cc90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa194035000 tx=0x7fa194038040 comp rx=0 tx=0).stop 2024-09-16T07:36:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 msgr2=0x7fa1a41f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 0x7fa1a41f6970 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa19000c400 tx=0x7fa19000c8c0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 shutdown_connections 2024-09-16T07:36:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa18407a7d0 0x7fa18407cc90 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a415cff0 0x7fa1a409bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1a4159aa0 0x7fa1a41f6970 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 --2- 172.21.15.71:0/3792165661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a406f220 0x7fa1a40cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 >> 172.21.15.71:0/3792165661 conn(0x7fa1a40b18e0 msgr2=0x7fa1a40aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 shutdown_connections 2024-09-16T07:36:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.090+0000 7fa1ab000640 1 -- 172.21.15.71:0/3792165661 wait complete. 2024-09-16T07:36:44.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:36:44.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.265+0000 7f12a6c00640 1 -- 172.21.15.71:0/2832995941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0074280 msgr2=0x7f12a006ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:44.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.265+0000 7f12a6c00640 1 --2- 172.21.15.71:0/2832995941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0074280 0x7f12a006ed60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1298005fd0 tx=0x7f129802aa90 comp rx=0 tx=0).stop 2024-09-16T07:36:44.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.265+0000 7f12a6c00640 1 -- 172.21.15.71:0/2832995941 shutdown_connections 2024-09-16T07:36:44.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.265+0000 7f12a6c00640 1 --2- 172.21.15.71:0/2832995941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0074280 0x7f12a006ed60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.265+0000 7f12a6c00640 1 --2- 172.21.15.71:0/2832995941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a00b2810 0x7f12a0073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.265+0000 7f12a6c00640 1 -- 172.21.15.71:0/2832995941 >> 172.21.15.71:0/2832995941 conn(0x7f12a00b1510 msgr2=0x7f12a00b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:44.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.265+0000 7f12a6c00640 1 -- 172.21.15.71:0/2832995941 shutdown_connections 2024-09-16T07:36:44.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.265+0000 7f12a6c00640 1 -- 172.21.15.71:0/2832995941 wait complete. 2024-09-16T07:36:44.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.267+0000 7f12a6c00640 1 Processor -- start 2024-09-16T07:36:44.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.267+0000 7f12a6c00640 1 -- start start 2024-09-16T07:36:44.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a6c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0074280 0x7f12a0158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:44.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a6c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a00b2810 0x7f12a00b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a6c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a015bb90 0x7f12a00b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a6c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12a006f2a0 con 0x7f12a00b2810 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a6c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f12a015bf70 con 0x7f12a0074280 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a6c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f12a006f420 con 0x7f12a015bb90 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a00b2810 0x7f12a00b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a00b2810 0x7f12a00b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43052/0 (socket says 172.21.15.71:43052) 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a4e00640 1 -- 172.21.15.71:0/1523129580 learned_addr learned my addr 172.21.15.71:0/1523129580 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a6200640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a015bb90 0x7f12a00b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a5800640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0074280 0x7f12a0158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a4e00640 1 -- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a015bb90 msgr2=0x7f12a00b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a4e00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a015bb90 0x7f12a00b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a4e00640 1 -- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0074280 msgr2=0x7f12a0158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a4e00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0074280 0x7f12a0158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.268+0000 7f12a4e00640 1 -- 172.21.15.71:0/1523129580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a01f6110 con 0x7f12a00b2810 2024-09-16T07:36:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.269+0000 7f12a4e00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a00b2810 0x7f12a00b4b20 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f128c00e9f0 tx=0x7f128c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.269+0000 7f129d600640 1 -- 172.21.15.71:0/1523129580 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c00cd80 con 0x7f12a00b2810 2024-09-16T07:36:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.269+0000 7f129d600640 1 -- 172.21.15.71:0/1523129580 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f128c010430 con 0x7f12a00b2810 2024-09-16T07:36:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.269+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a00c8d50 con 0x7f12a00b2810 2024-09-16T07:36:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.269+0000 7f129d600640 1 -- 172.21.15.71:0/1523129580 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c010840 con 0x7f12a00b2810 2024-09-16T07:36:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.269+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a0159130 con 0x7f12a00b2810 2024-09-16T07:36:44.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.270+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a006f8d0 con 0x7f12a00b2810 2024-09-16T07:36:44.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.271+0000 7f129d600640 1 -- 172.21.15.71:0/1523129580 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 46) ==== 104143+0+0 (secure 0 0 0) 0x7f128c0109e0 con 0x7f12a00b2810 2024-09-16T07:36:44.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.272+0000 7f129d600640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128007a7d0 0x7f128007cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:44.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.272+0000 7f129d600640 1 -- 172.21.15.71:0/1523129580 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f128c09a510 con 0x7f12a00b2810 2024-09-16T07:36:44.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.273+0000 7f12a5800640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128007a7d0 0x7f128007cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:44.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.273+0000 7f12a5800640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128007a7d0 0x7f128007cc90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1294000f60 tx=0x7f1294009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:44.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.276+0000 7f129d600640 1 -- 172.21.15.71:0/1523129580 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f128c064050 con 0x7f12a00b2810 2024-09-16T07:36:44.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.490+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f12a00c8b00 con 0x7f128007a7d0 2024-09-16T07:36:44.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.495+0000 7f129d600640 1 -- 172.21.15.71:0/1523129580 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f12a00c8b00 con 0x7f128007a7d0 2024-09-16T07:36:44.495 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128007a7d0 msgr2=0x7f128007cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128007a7d0 0x7f128007cc90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1294000f60 tx=0x7f1294009040 comp rx=0 tx=0).stop 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a00b2810 msgr2=0x7f12a00b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a00b2810 0x7f12a00b4b20 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f128c00e9f0 tx=0x7f128c00eeb0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 shutdown_connections 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128007a7d0 0x7f128007cc90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a015bb90 0x7f12a00b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a00b2810 0x7f12a00b4b20 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 --2- 172.21.15.71:0/1523129580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0074280 0x7f12a0158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 >> 172.21.15.71:0/1523129580 conn(0x7f12a00b1510 msgr2=0x7f12a00b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 shutdown_connections 2024-09-16T07:36:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:44.497+0000 7f12a6c00640 1 -- 172.21.15.71:0/1523129580 wait complete. 2024-09-16T07:36:44.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:44.514 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:44.514 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:36:44.550 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:44.550 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T07:36:44.550 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:44.551 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T07:36:46.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:46.552 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:46.552 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:36:46.593 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:46.593 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T07:36:46.593 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:46.594 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T07:36:46.594 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-16T07:36:46.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.805+0000 7f6fea600640 1 -- 172.21.15.71:0/4003489654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe4145750 msgr2=0x7f6fe4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:46.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.805+0000 7f6fea600640 1 --2- 172.21.15.71:0/4003489654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe4145750 0x7f6fe4073440 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4005fd0 tx=0x7f6fd402a820 comp rx=0 tx=0).stop 2024-09-16T07:36:46.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.805+0000 7f6fea600640 1 -- 172.21.15.71:0/4003489654 shutdown_connections 2024-09-16T07:36:46.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.805+0000 7f6fea600640 1 --2- 172.21.15.71:0/4003489654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe4145750 0x7f6fe4073440 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:46.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.805+0000 7f6fea600640 1 -- 172.21.15.71:0/4003489654 >> 172.21.15.71:0/4003489654 conn(0x7f6fe40b1840 msgr2=0x7f6fe40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:46.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.805+0000 7f6fea600640 1 -- 172.21.15.71:0/4003489654 shutdown_connections 2024-09-16T07:36:46.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.805+0000 7f6fea600640 1 -- 172.21.15.71:0/4003489654 wait complete. 2024-09-16T07:36:46.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.807+0000 7f6fea600640 1 Processor -- start 2024-09-16T07:36:46.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.807+0000 7f6fea600640 1 -- start start 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fea600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe406efa0 0x7f6fe41f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fea600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fe40cccd0 0x7f6fe413b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fea600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe4145750 0x7f6fe40b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fea600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe40ce6d0 con 0x7f6fe406efa0 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fea600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe40ce550 con 0x7f6fe4145750 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fea600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe40ce850 con 0x7f6fe40cccd0 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fe3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe406efa0 0x7f6fe41f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fe3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe406efa0 0x7f6fe41f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55148/0 (socket says 172.21.15.71:55148) 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fe3400640 1 -- 172.21.15.71:0/1661304255 learned_addr learned my addr 172.21.15.71:0/1661304255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fe2a00640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fe40cccd0 0x7f6fe413b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.808+0000 7f6fe3e00640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe4145750 0x7f6fe40b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fe3400640 1 -- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fe40cccd0 msgr2=0x7f6fe413b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fe3400640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fe40cccd0 0x7f6fe413b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fe3400640 1 -- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe4145750 msgr2=0x7f6fe40b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fe3400640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe4145750 0x7f6fe40b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fe3400640 1 -- 172.21.15.71:0/1661304255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe40b3c30 con 0x7f6fe406efa0 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fe2a00640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fe40cccd0 0x7f6fe413b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fe3400640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe406efa0 0x7f6fe41f4800 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4035040 tx=0x7f6fd4032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fdb400640 1 -- 172.21.15.71:0/1661304255 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4037040 con 0x7f6fe406efa0 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fdb400640 1 -- 172.21.15.71:0/1661304255 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fd4042070 con 0x7f6fe406efa0 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fdb400640 1 -- 172.21.15.71:0/1661304255 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40042a0 con 0x7f6fe406efa0 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe40b30b0 con 0x7f6fe406efa0 2024-09-16T07:36:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.809+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe40b3670 con 0x7f6fe406efa0 2024-09-16T07:36:46.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.810+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe40ced00 con 0x7f6fe406efa0 2024-09-16T07:36:46.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.812+0000 7f6fdb400640 1 -- 172.21.15.71:0/1661304255 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 46) ==== 104143+0+0 (secure 0 0 0) 0x7f6fd4004440 con 0x7f6fe406efa0 2024-09-16T07:36:46.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.812+0000 7f6fdb400640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fc407a7d0 0x7f6fc407cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:46.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.813+0000 7f6fdb400640 1 -- 172.21.15.71:0/1661304255 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f6fd40ba7e0 con 0x7f6fe406efa0 2024-09-16T07:36:46.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.813+0000 7f6fe2a00640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fc407a7d0 0x7f6fc407cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:46.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.814+0000 7f6fe2a00640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fc407a7d0 0x7f6fc407cc90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40af4f0 tx=0x7f6fd0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:46.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:46.817+0000 7f6fdb400640 1 -- 172.21.15.71:0/1661304255 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6fd40848b0 con 0x7f6fe406efa0 2024-09-16T07:36:46.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:36:46.864 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:36:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.042+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f6fe413d7c0 con 0x7f6fc407a7d0 2024-09-16T07:36:47.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.110+0000 7f6fdb400640 1 -- 172.21.15.71:0/1661304255 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6fe413d7c0 con 0x7f6fc407a7d0 2024-09-16T07:36:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fc407a7d0 msgr2=0x7f6fc407cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fc407a7d0 0x7f6fc407cc90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40af4f0 tx=0x7f6fd0009040 comp rx=0 tx=0).stop 2024-09-16T07:36:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe406efa0 msgr2=0x7f6fe41f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe406efa0 0x7f6fe41f4800 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4035040 tx=0x7f6fd4032c90 comp rx=0 tx=0).stop 2024-09-16T07:36:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 shutdown_connections 2024-09-16T07:36:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6fc407a7d0 0x7f6fc407cc90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe4145750 0x7f6fe40b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fe40cccd0 0x7f6fe413b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 --2- 172.21.15.71:0/1661304255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fe406efa0 0x7f6fe41f4800 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.112+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 >> 172.21.15.71:0/1661304255 conn(0x7f6fe40b1840 msgr2=0x7f6fe4143270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.113+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 shutdown_connections 2024-09-16T07:36:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:47.113+0000 7f6fea600640 1 -- 172.21.15.71:0/1661304255 wait complete. 2024-09-16T07:36:47.115 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.116+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.116 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.116+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.116 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.116+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.116 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.116+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.116 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.116+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.132+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.132+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.132+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.132+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:47.132 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:47.132+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:48.118 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:48.118+0000 7fe6e7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:48.118 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:48.118+0000 7fe6e7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:48.118 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:48.118+0000 7fe6e7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:48.118 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:48.118+0000 7fe6e7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:48.118 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:48.118+0000 7fe6e7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:48.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-16T07:36:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.360+0000 7f52fe200640 1 -- 172.21.15.71:0/3228084153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0004800 msgr2=0x7f52e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.360+0000 7f52fe200640 1 --2- 172.21.15.71:0/3228084153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0004800 0x7f52e0004c80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f52e400c6e0 tx=0x7f52e0005830 comp rx=0 tx=0).stop 2024-09-16T07:36:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.360+0000 7f52fe200640 1 -- 172.21.15.71:0/3228084153 shutdown_connections 2024-09-16T07:36:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.360+0000 7f52fe200640 1 --2- 172.21.15.71:0/3228084153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0004800 0x7f52e0004c80 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.360+0000 7f52fe200640 1 -- 172.21.15.71:0/3228084153 >> 172.21.15.71:0/3228084153 conn(0x7f52f80b1510 msgr2=0x7f52f80b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:48.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.361+0000 7f52fe200640 1 -- 172.21.15.71:0/3228084153 shutdown_connections 2024-09-16T07:36:48.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.361+0000 7f52fe200640 1 -- 172.21.15.71:0/3228084153 wait complete. 2024-09-16T07:36:48.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.363+0000 7f52fe200640 1 Processor -- start 2024-09-16T07:36:48.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.363+0000 7f52fe200640 1 -- start start 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fe200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52e0004800 0x7f52f8151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fe200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f80c6040 0x7f52f80c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fe200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52f8154310 0x7f52f80b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fe200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80b2ba0 con 0x7f52f8154310 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fe200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52f80b2a20 con 0x7f52f80c6040 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fe200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f52f8073d40 con 0x7f52e0004800 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52f8154310 0x7f52f80b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52f8154310 0x7f52f80b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55176/0 (socket says 172.21.15.71:55176) 2024-09-16T07:36:48.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.364+0000 7f52fd800640 1 -- 172.21.15.71:0/2648770696 learned_addr learned my addr 172.21.15.71:0/2648770696 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:48.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52f7e00640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f80c6040 0x7f52f80c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:48.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52fce00640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52e0004800 0x7f52f8151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:48.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52fd800640 1 -- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52e0004800 msgr2=0x7f52f8151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:48.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52fd800640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52e0004800 0x7f52f8151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:48.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52fd800640 1 -- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f80c6040 msgr2=0x7f52f80c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:48.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52fd800640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f80c6040 0x7f52f80c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:48.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52fd800640 1 -- 172.21.15.71:0/2648770696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f80c9980 con 0x7f52f8154310 2024-09-16T07:36:48.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52fd800640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52f8154310 0x7f52f80b3bd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f52f80ced80 tx=0x7f52f000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:48.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.365+0000 7f52f4c00640 1 -- 172.21.15.71:0/2648770696 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0017070 con 0x7f52f8154310 2024-09-16T07:36:48.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.366+0000 7f52f4c00640 1 -- 172.21.15.71:0/2648770696 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52f00050d0 con 0x7f52f8154310 2024-09-16T07:36:48.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.366+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f80b2e30 con 0x7f52f8154310 2024-09-16T07:36:48.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.366+0000 7f52f4c00640 1 -- 172.21.15.71:0/2648770696 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0005520 con 0x7f52f8154310 2024-09-16T07:36:48.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.366+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f80b33f0 con 0x7f52f8154310 2024-09-16T07:36:48.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.367+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f80742a0 con 0x7f52f8154310 2024-09-16T07:36:48.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.368+0000 7f52f4c00640 1 -- 172.21.15.71:0/2648770696 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 46) ==== 104143+0+0 (secure 0 0 0) 0x7f52f001a070 con 0x7f52f8154310 2024-09-16T07:36:48.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.369+0000 7f52f4c00640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f52d807a830 0x7f52d807ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:48.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.369+0000 7f52f4c00640 1 -- 172.21.15.71:0/2648770696 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f52f0098e90 con 0x7f52f8154310 2024-09-16T07:36:48.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.370+0000 7f52fce00640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f52d807a830 0x7f52d807ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:48.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.371+0000 7f52fce00640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f52d807a830 0x7f52d807ccf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00db80 tx=0x7f52ec00d560 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:48.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.374+0000 7f52f4c00640 1 -- 172.21.15.71:0/2648770696 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f52f0062910 con 0x7f52f8154310 2024-09-16T07:36:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.699+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f52f80ae400 con 0x7f52d807a830 2024-09-16T07:36:48.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.834+0000 7f52f4c00640 1 -- 172.21.15.71:0/2648770696 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f52f80ae400 con 0x7f52d807a830 2024-09-16T07:36:48.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f52d807a830 msgr2=0x7f52d807ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f52d807a830 0x7f52d807ccf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00db80 tx=0x7f52ec00d560 comp rx=0 tx=0).stop 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52f8154310 msgr2=0x7f52f80b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52f8154310 0x7f52f80b3bd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f52f80ced80 tx=0x7f52f000ccd0 comp rx=0 tx=0).stop 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 shutdown_connections 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f52d807a830 0x7f52d807ccf0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52f8154310 0x7f52f80b3bd0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f80c6040 0x7f52f80c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 --2- 172.21.15.71:0/2648770696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52e0004800 0x7f52f8151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 >> 172.21.15.71:0/2648770696 conn(0x7f52f80b1510 msgr2=0x7f52f80b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 shutdown_connections 2024-09-16T07:36:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:48.837+0000 7f52fe200640 1 -- 172.21.15.71:0/2648770696 wait complete. 2024-09-16T07:36:48.852 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T07:36:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.014+0000 7f3a3a800640 1 -- 172.21.15.71:0/2880166940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a340739d0 msgr2=0x7f3a3415e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.014+0000 7f3a3a800640 1 --2- 172.21.15.71:0/2880166940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a340739d0 0x7f3a3415e0f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c005fd0 tx=0x7f3a2c02b980 comp rx=0 tx=0).stop 2024-09-16T07:36:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.014+0000 7f3a3a800640 1 -- 172.21.15.71:0/2880166940 shutdown_connections 2024-09-16T07:36:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.014+0000 7f3a3a800640 1 --2- 172.21.15.71:0/2880166940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a340739d0 0x7f3a3415e0f0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.014+0000 7f3a3a800640 1 --2- 172.21.15.71:0/2880166940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a341517a0 0x7f3a340cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.014+0000 7f3a3a800640 1 -- 172.21.15.71:0/2880166940 >> 172.21.15.71:0/2880166940 conn(0x7f3a340b1840 msgr2=0x7f3a340b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.014+0000 7f3a3a800640 1 -- 172.21.15.71:0/2880166940 shutdown_connections 2024-09-16T07:36:49.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.014+0000 7f3a3a800640 1 -- 172.21.15.71:0/2880166940 wait complete. 2024-09-16T07:36:49.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.016+0000 7f3a3a800640 1 Processor -- start 2024-09-16T07:36:49.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.016+0000 7f3a3a800640 1 -- start start 2024-09-16T07:36:49.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a340739d0 0x7f3a3413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:49.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a340cd490 0x7f3a3414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:49.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a341517a0 0x7f3a341e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a3a800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340cc7d0 con 0x7f3a341517a0 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a3a800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340cc650 con 0x7f3a340cd490 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a3a800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3a340cc950 con 0x7f3a340739d0 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a341517a0 0x7f3a341e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a341517a0 0x7f3a341e4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55182/0 (socket says 172.21.15.71:55182) 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33e00640 1 -- 172.21.15.71:0/4119200883 learned_addr learned my addr 172.21.15.71:0/4119200883 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33400640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a340739d0 0x7f3a3413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a32a00640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a340cd490 0x7f3a3414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33e00640 1 -- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a340739d0 msgr2=0x7f3a3413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33e00640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a340739d0 0x7f3a3413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33e00640 1 -- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a340cd490 msgr2=0x7f3a3414fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33e00640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a340cd490 0x7f3a3414fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a33e00640 1 -- 172.21.15.71:0/4119200883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a341e3220 con 0x7f3a341517a0 2024-09-16T07:36:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.017+0000 7f3a32a00640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a340cd490 0x7f3a3414fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.018+0000 7f3a33e00640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a341517a0 0x7f3a341e4e20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c006010 tx=0x7f3a2c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.018+0000 7f3a2b400640 1 -- 172.21.15.71:0/4119200883 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c035040 con 0x7f3a341517a0 2024-09-16T07:36:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.018+0000 7f3a2b400640 1 -- 172.21.15.71:0/4119200883 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a2c004240 con 0x7f3a341517a0 2024-09-16T07:36:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.018+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a341e3500 con 0x7f3a341517a0 2024-09-16T07:36:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.018+0000 7f3a2b400640 1 -- 172.21.15.71:0/4119200883 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c004650 con 0x7f3a341517a0 2024-09-16T07:36:49.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.018+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a341e3ac0 con 0x7f3a341517a0 2024-09-16T07:36:49.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.019+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a340cdad0 con 0x7f3a341517a0 2024-09-16T07:36:49.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.020+0000 7f3a2b400640 1 -- 172.21.15.71:0/4119200883 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 46) ==== 104143+0+0 (secure 0 0 0) 0x7f3a2c038020 con 0x7f3a341517a0 2024-09-16T07:36:49.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.021+0000 7f3a2b400640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a1407a7d0 0x7f3a1407cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:49.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.021+0000 7f3a2b400640 1 -- 172.21.15.71:0/4119200883 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f3a2c0baea0 con 0x7f3a341517a0 2024-09-16T07:36:49.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.022+0000 7f3a33400640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a1407a7d0 0x7f3a1407cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:49.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.022+0000 7f3a33400640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a1407a7d0 0x7f3a1407cc90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009b30 tx=0x7f3a24005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:49.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.025+0000 7f3a2b400640 1 -- 172.21.15.71:0/4119200883 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3a2c084f70 con 0x7f3a341517a0 2024-09-16T07:36:49.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.249+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f3a340b2430 con 0x7f3a1407a7d0 2024-09-16T07:36:49.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:49.359+0000 7f3a2b400640 1 -- 172.21.15.71:0/4119200883 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7f3a2c080ae0 con 0x7f3a341517a0 2024-09-16T07:36:50.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.544+0000 7f3a2b400640 1 -- 172.21.15.71:0/4119200883 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f3a340b2430 con 0x7f3a1407a7d0 2024-09-16T07:36:50.544 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:36:50.544 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:36:50.544 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:36:50.544 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:36:50.544 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:36:50.544 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:36:50.544 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:36:50.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:50.545 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:36:50.545 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:36:50.545 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:36:50.545 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:50.545 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:50.545 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:36:50.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a1407a7d0 msgr2=0x7f3a1407cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a1407a7d0 0x7f3a1407cc90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009b30 tx=0x7f3a24005f90 comp rx=0 tx=0).stop 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a341517a0 msgr2=0x7f3a341e4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a341517a0 0x7f3a341e4e20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c006010 tx=0x7f3a2c003f10 comp rx=0 tx=0).stop 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 shutdown_connections 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a1407a7d0 0x7f3a1407cc90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a341517a0 0x7f3a341e4e20 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a340cd490 0x7f3a3414fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 --2- 172.21.15.71:0/4119200883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a340739d0 0x7f3a3413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 >> 172.21.15.71:0/4119200883 conn(0x7f3a340b1840 msgr2=0x7f3a34075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 shutdown_connections 2024-09-16T07:36:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.547+0000 7f3a3a800640 1 -- 172.21.15.71:0/4119200883 wait complete. 2024-09-16T07:36:50.565 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:36:50.565 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:36:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.727+0000 7f301d000640 1 -- 172.21.15.71:0/794567670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0053c0 msgr2=0x7f2ffc00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.727+0000 7f301d000640 1 --2- 172.21.15.71:0/794567670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0053c0 0x7f2ffc00dbd0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f301000c590 tx=0x7f2ffc005830 comp rx=0 tx=0).stop 2024-09-16T07:36:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.727+0000 7f301d000640 1 -- 172.21.15.71:0/794567670 shutdown_connections 2024-09-16T07:36:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.727+0000 7f301d000640 1 --2- 172.21.15.71:0/794567670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0053c0 0x7f2ffc00dbd0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.727+0000 7f301d000640 1 -- 172.21.15.71:0/794567670 >> 172.21.15.71:0/794567670 conn(0x7f30180b18e0 msgr2=0x7f30180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.727+0000 7f301d000640 1 -- 172.21.15.71:0/794567670 shutdown_connections 2024-09-16T07:36:50.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.727+0000 7f301d000640 1 -- 172.21.15.71:0/794567670 wait complete. 2024-09-16T07:36:50.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.729+0000 7f301d000640 1 Processor -- start 2024-09-16T07:36:50.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.729+0000 7f301d000640 1 -- start start 2024-09-16T07:36:50.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f301d000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ffc0053c0 0x7f301815c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:50.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f301d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f301815b410 0x7f30181512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f301d000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30181547a0 0x7f301809d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f301d000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f30180cc5f0 con 0x7f301815b410 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f301d000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30180b2f00 con 0x7f30181547a0 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f301d000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30180cc770 con 0x7f2ffc0053c0 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f300d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f301815b410 0x7f30181512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f300d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f301815b410 0x7f30181512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55196/0 (socket says 172.21.15.71:55196) 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f300d400640 1 -- 172.21.15.71:0/3833035400 learned_addr learned my addr 172.21.15.71:0/3833035400 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f3015e00640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ffc0053c0 0x7f301815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.730+0000 7f3016800640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30181547a0 0x7f301809d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300d400640 1 -- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ffc0053c0 msgr2=0x7f301815c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300d400640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ffc0053c0 0x7f301815c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300d400640 1 -- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30181547a0 msgr2=0x7f301809d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:50.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300d400640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30181547a0 0x7f301809d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300d400640 1 -- 172.21.15.71:0/3833035400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301815b8f0 con 0x7f301815b410 2024-09-16T07:36:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f3016800640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30181547a0 0x7f301809d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300d400640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f301815b410 0x7f30181512b0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f3000004d60 tx=0x7f3000001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300ea00640 1 -- 172.21.15.71:0/3833035400 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30000107d0 con 0x7f301815b410 2024-09-16T07:36:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300ea00640 1 -- 172.21.15.71:0/3833035400 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3000016070 con 0x7f301815b410 2024-09-16T07:36:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f300ea00640 1 -- 172.21.15.71:0/3833035400 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000010d20 con 0x7f301815b410 2024-09-16T07:36:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301815bbd0 con 0x7f301815b410 2024-09-16T07:36:50.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.731+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301815c190 con 0x7f301815b410 2024-09-16T07:36:50.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.732+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3018069360 con 0x7f301815b410 2024-09-16T07:36:50.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.734+0000 7f300ea00640 1 -- 172.21.15.71:0/3833035400 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7f300001b460 con 0x7f301815b410 2024-09-16T07:36:50.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.735+0000 7f300ea00640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ff407a980 0x7f2ff407ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:50.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.735+0000 7f300ea00640 1 -- 172.21.15.71:0/3833035400 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f300009a0b0 con 0x7f301815b410 2024-09-16T07:36:50.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.735+0000 7f3015e00640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ff407a980 0x7f2ff407ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:50.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.736+0000 7f3015e00640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ff407a980 0x7f2ff407ce40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f300800ac30 tx=0x7f3008010040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:50.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:50.739+0000 7f300ea00640 1 -- 172.21.15.71:0/3833035400 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3000064220 con 0x7f301815b410 2024-09-16T07:36:50.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.000+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f30180af2a0 con 0x7f301815b410 2024-09-16T07:36:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.000+0000 7f300ea00640 1 -- 172.21.15.71:0/3833035400 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2594 (secure 0 0 0) 0x7f3000063b20 con 0x7f301815b410 2024-09-16T07:36:51.000 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:51.000 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:36:51.000 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:36:51.000 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:36:51.000 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:36:51.000 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:36:51.001 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 867, 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 133, 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:36:51.002 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 793, 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 3221997607, 2024-09-16T07:36:51.003 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 7385559040, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 765708554240, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 349436, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 176471143, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 0, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 43 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 86, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:36:17:890799+0000", 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 9, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:36:51.004 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 9, 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 8919 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 20, 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:35:44.620542+0000", 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "services": { 2024-09-16T07:36:51.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "daemons": { 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "summary": "", 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "d": { 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:36:51.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": { 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": 0, 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: "add_to_ceph_s": true 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.008 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:36:51.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ff407a980 msgr2=0x7f2ff407ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:51.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ff407a980 0x7f2ff407ce40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f300800ac30 tx=0x7f3008010040 comp rx=0 tx=0).stop 2024-09-16T07:36:51.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f301815b410 msgr2=0x7f30181512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:51.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f301815b410 0x7f30181512b0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f3000004d60 tx=0x7f3000001630 comp rx=0 tx=0).stop 2024-09-16T07:36:51.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 shutdown_connections 2024-09-16T07:36:51.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2ff407a980 0x7f2ff407ce40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30181547a0 0x7f301809d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f301815b410 0x7f30181512b0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 --2- 172.21.15.71:0/3833035400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ffc0053c0 0x7f301815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 >> 172.21.15.71:0/3833035400 conn(0x7f30180b18e0 msgr2=0x7f30180cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:51.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 shutdown_connections 2024-09-16T07:36:51.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.003+0000 7f301d000640 1 -- 172.21.15.71:0/3833035400 wait complete. 2024-09-16T07:36:51.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:36:51.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.677+0000 7efdf4c00640 1 -- 172.21.15.71:0/3717467609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd4003a30 msgr2=0x7efdd4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:51.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.677+0000 7efdf4c00640 1 --2- 172.21.15.71:0/3717467609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdd4003a30 0x7efdd4003e50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7efde0009ed0 tx=0x7efdd4005830 comp rx=0 tx=0).stop 2024-09-16T07:36:51.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.677+0000 7efdee800640 1 -- 172.21.15.71:0/3717467609 reap_dead start 2024-09-16T07:36:51.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.677+0000 7efdf4c00640 1 -- 172.21.15.71:0/3717467609 shutdown_connections 2024-09-16T07:36:51.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.677+0000 7efdf4c00640 1 -- 172.21.15.71:0/3717467609 >> 172.21.15.71:0/3717467609 conn(0x7efdf00b18e0 msgr2=0x7efdf00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:51.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.677+0000 7efdf4c00640 1 -- 172.21.15.71:0/3717467609 shutdown_connections 2024-09-16T07:36:51.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.677+0000 7efdf4c00640 1 -- 172.21.15.71:0/3717467609 wait complete. 2024-09-16T07:36:51.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.678+0000 7efdf4c00640 1 Processor -- start 2024-09-16T07:36:51.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.679+0000 7efdf4c00640 1 -- start start 2024-09-16T07:36:51.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.679+0000 7efdf4c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd4003a30 0x7efdf01f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:51.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.679+0000 7efdf4c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf01f3480 0x7efdf01526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:51.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.679+0000 7efdf4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf009bbe0 0x7efdf009c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:51.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.679+0000 7efdf4c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efdf015fd80 con 0x7efdf009bbe0 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.679+0000 7efdf4c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efdf00cd5e0 con 0x7efdd4003a30 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.679+0000 7efdf4c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efdf015ff00 con 0x7efdf01f3480 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf009bbe0 0x7efdf009c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf009bbe0 0x7efdf009c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55222/0 (socket says 172.21.15.71:55222) 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 -- 172.21.15.71:0/341036620 learned_addr learned my addr 172.21.15.71:0/341036620 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdede00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd4003a30 0x7efdf01f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efded400640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf01f3480 0x7efdf01526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 -- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf01f3480 msgr2=0x7efdf01526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf01f3480 0x7efdf01526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 -- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd4003a30 msgr2=0x7efdf01f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd4003a30 0x7efdf01f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 -- 172.21.15.71:0/341036620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf009ab80 con 0x7efdf009bbe0 2024-09-16T07:36:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efded400640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf01f3480 0x7efdf01526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.680+0000 7efdee800640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf009bbe0 0x7efdf009c060 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7efdf01534a0 tx=0x7efde800e7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.681+0000 7efde6000640 1 -- 172.21.15.71:0/341036620 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde80107d0 con 0x7efdf009bbe0 2024-09-16T07:36:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.681+0000 7efde6000640 1 -- 172.21.15.71:0/341036620 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efde8016070 con 0x7efdf009bbe0 2024-09-16T07:36:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.681+0000 7efde6000640 1 -- 172.21.15.71:0/341036620 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8010d00 con 0x7efdf009bbe0 2024-09-16T07:36:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.681+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf01f6720 con 0x7efdf009bbe0 2024-09-16T07:36:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.681+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf01f6ce0 con 0x7efdf009bbe0 2024-09-16T07:36:51.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.682+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf0069360 con 0x7efdf009bbe0 2024-09-16T07:36:51.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.683+0000 7efde6000640 1 -- 172.21.15.71:0/341036620 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7efde801b460 con 0x7efdf009bbe0 2024-09-16T07:36:51.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.684+0000 7efde6000640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdd007a9b0 0x7efdd007ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:51.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.684+0000 7efde6000640 1 -- 172.21.15.71:0/341036620 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7efde8099c60 con 0x7efdf009bbe0 2024-09-16T07:36:51.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.684+0000 7efdede00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdd007a9b0 0x7efdd007ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:51.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.685+0000 7efdede00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdd007a9b0 0x7efdd007ce70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efde0035000 tx=0x7efde002d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:51.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.688+0000 7efde6000640 1 -- 172.21.15.71:0/341036620 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efde8063d30 con 0x7efdf009bbe0 2024-09-16T07:36:51.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.944+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efdf006f440 con 0x7efdf009bbe0 2024-09-16T07:36:51.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.945+0000 7efde6000640 1 -- 172.21.15.71:0/341036620 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2370 (secure 0 0 0) 0x7efde8063630 con 0x7efdf009bbe0 2024-09-16T07:36:51.945 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:51.945 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:36:51.945 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:36:51.945 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:36:51.945 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 868, 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:36:51.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 133, 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:36:51.948 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 824, 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 3335380906, 2024-09-16T07:36:51.949 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 7637377024, 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 765456736256, 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 13977126, 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 135245073, 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 3, 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 37 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 86, 2024-09-16T07:36:51.950 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:36:17:890799+0000", 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 9, 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 9, 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:36:51.951 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 8919 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:36:51.952 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 20, 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:35:44.620542+0000", 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "services": { 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "daemons": { 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "summary": "", 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "d": { 2024-09-16T07:36:51.953 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:51.954 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": {} 2024-09-16T07:36:51.955 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:36:51.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdd007a9b0 msgr2=0x7efdd007ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:51.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdd007a9b0 0x7efdd007ce70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efde0035000 tx=0x7efde002d040 comp rx=0 tx=0).stop 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf009bbe0 msgr2=0x7efdf009c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf009bbe0 0x7efdf009c060 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7efdf01534a0 tx=0x7efde800e7f0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 shutdown_connections 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efdd007a9b0 0x7efdd007ce70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf009bbe0 0x7efdf009c060 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdf01f3480 0x7efdf01526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 --2- 172.21.15.71:0/341036620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd4003a30 0x7efdf01f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 >> 172.21.15.71:0/341036620 conn(0x7efdf00b18e0 msgr2=0x7efdf015a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 shutdown_connections 2024-09-16T07:36:51.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:51.947+0000 7efdf4c00640 1 -- 172.21.15.71:0/341036620 wait complete. 2024-09-16T07:36:51.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:36:51.962 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/c23fda99-9481-44c2-8791-b710d82d0817 2024-09-16T07:36:51.962 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:36:51.999 INFO:teuthology.orchestra.run.smithi071.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/c23fda99-9481-44c2-8791-b710d82d0817': No such file or directory 2024-09-16T07:36:51.999 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T07:36:52.000 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:36:52.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.198+0000 7f0ae6a00640 1 -- 172.21.15.71:0/2193835241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 msgr2=0x7f0ac4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.198+0000 7f0ae6a00640 1 --2- 172.21.15.71:0/2193835241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 0x7f0ac4003e70 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0001690 tx=0x7f0ac4005830 comp rx=0 tx=0).stop 2024-09-16T07:36:52.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.198+0000 7f0ae6a00640 1 -- 172.21.15.71:0/2193835241 shutdown_connections 2024-09-16T07:36:52.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.198+0000 7f0ae6a00640 1 --2- 172.21.15.71:0/2193835241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 0x7f0ac4003e70 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.198+0000 7f0ae6a00640 1 -- 172.21.15.71:0/2193835241 >> 172.21.15.71:0/2193835241 conn(0x7f0ae00b1950 msgr2=0x7f0ae00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.198+0000 7f0ae6a00640 1 -- 172.21.15.71:0/2193835241 shutdown_connections 2024-09-16T07:36:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.198+0000 7f0ae6a00640 1 -- 172.21.15.71:0/2193835241 wait complete. 2024-09-16T07:36:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.200+0000 7f0ae6a00640 1 Processor -- start 2024-09-16T07:36:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.200+0000 7f0ae6a00640 1 -- start start 2024-09-16T07:36:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0ae6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 0x7f0ae015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0ae6a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae015b680 0x7f0ae0150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0ae6a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae0151400 0x7f0ae0151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0ae6a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0074420 con 0x7f0ac4003a50 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0ae6a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae00b2700 con 0x7f0ae0151400 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0ae6a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae00745a0 con 0x7f0ae015b680 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0adfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 0x7f0ae015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0adfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 0x7f0ae015a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55240/0 (socket says 172.21.15.71:55240) 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.201+0000 7f0adfe00640 1 -- 172.21.15.71:0/3976726117 learned_addr learned my addr 172.21.15.71:0/3976726117 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0adf400640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae015b680 0x7f0ae0150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0ae4a00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae0151400 0x7f0ae0151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0adfe00640 1 -- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae015b680 msgr2=0x7f0ae0150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0adfe00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae015b680 0x7f0ae0150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0adfe00640 1 -- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae0151400 msgr2=0x7f0ae0151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0adfe00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae0151400 0x7f0ae0151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0adfe00640 1 -- 172.21.15.71:0/3976726117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae015bed0 con 0x7f0ac4003a50 2024-09-16T07:36:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0adfe00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 0x7f0ae015a770 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0ad002a900 tx=0x7f0ad00063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:52.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0ad7e00640 1 -- 172.21.15.71:0/3976726117 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0003a00 con 0x7f0ac4003a50 2024-09-16T07:36:52.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0ad7e00640 1 -- 172.21.15.71:0/3976726117 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ad0039070 con 0x7f0ac4003a50 2024-09-16T07:36:52.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae015c1b0 con 0x7f0ac4003a50 2024-09-16T07:36:52.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.202+0000 7f0ad7e00640 1 -- 172.21.15.71:0/3976726117 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0033420 con 0x7f0ac4003a50 2024-09-16T07:36:52.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.203+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae0150c00 con 0x7f0ac4003a50 2024-09-16T07:36:52.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.204+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae00acd30 con 0x7f0ac4003a50 2024-09-16T07:36:52.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.205+0000 7f0ad7e00640 1 -- 172.21.15.71:0/3976726117 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7f0ad0044020 con 0x7f0ac4003a50 2024-09-16T07:36:52.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.206+0000 7f0ad7e00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac007a9b0 0x7f0ac007ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:52.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.206+0000 7f0ad7e00640 1 -- 172.21.15.71:0/3976726117 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f0ad00b8f60 con 0x7f0ac4003a50 2024-09-16T07:36:52.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.206+0000 7f0adf400640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac007a9b0 0x7f0ac007ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:52.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.207+0000 7f0adf400640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac007a9b0 0x7f0ac007ce70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0ae009cd10 tx=0x7f0acc005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:52.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.210+0000 7f0ad7e00640 1 -- 172.21.15.71:0/3976726117 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0ad00829e0 con 0x7f0ac4003a50 2024-09-16T07:36:52.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.417+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f0ae015df80 con 0x7f0ac007a9b0 2024-09-16T07:36:52.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.419+0000 7f0ad7e00640 1 -- 172.21.15.71:0/3976726117 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0ae015df80 con 0x7f0ac007a9b0 2024-09-16T07:36:52.419 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:36:52.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:52.419 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:36:52.419 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:52.419 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:36:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.421+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac007a9b0 msgr2=0x7f0ac007ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.421+0000 7f0ae6a00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac007a9b0 0x7f0ac007ce70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0ae009cd10 tx=0x7f0acc005f90 comp rx=0 tx=0).stop 2024-09-16T07:36:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.421+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 msgr2=0x7f0ae015a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.421+0000 7f0ae6a00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 0x7f0ae015a770 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0ad002a900 tx=0x7f0ad00063a0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.422+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 shutdown_connections 2024-09-16T07:36:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.422+0000 7f0ae6a00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f0ac007a9b0 0x7f0ac007ce70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.422+0000 7f0ae6a00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ae0151400 0x7f0ae0151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.422+0000 7f0ae6a00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ae015b680 0x7f0ae0150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.422+0000 7f0ae6a00640 1 --2- 172.21.15.71:0/3976726117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4003a50 0x7f0ae015a770 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.422+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 >> 172.21.15.71:0/3976726117 conn(0x7f0ae00b1950 msgr2=0x7f0ae015d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.422+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 shutdown_connections 2024-09-16T07:36:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.422+0000 7f0ae6a00640 1 -- 172.21.15.71:0/3976726117 wait complete. 2024-09-16T07:36:52.437 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:36:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.596+0000 7fb98ce00640 1 -- 172.21.15.71:0/1203435805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb96c004d90 msgr2=0x7fb96c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.596+0000 7fb98ce00640 1 --2- 172.21.15.71:0/1203435805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb96c004d90 0x7fb96c005210 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb98808d200 tx=0x7fb96c005830 comp rx=0 tx=0).stop 2024-09-16T07:36:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.596+0000 7fb987e00640 1 -- 172.21.15.71:0/1203435805 reap_dead start 2024-09-16T07:36:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.597+0000 7fb98ce00640 1 -- 172.21.15.71:0/1203435805 shutdown_connections 2024-09-16T07:36:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.597+0000 7fb98ce00640 1 -- 172.21.15.71:0/1203435805 >> 172.21.15.71:0/1203435805 conn(0x7fb98809d0b0 msgr2=0x7fb9880d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.597+0000 7fb98ce00640 1 -- 172.21.15.71:0/1203435805 shutdown_connections 2024-09-16T07:36:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.597+0000 7fb98ce00640 1 -- 172.21.15.71:0/1203435805 wait complete. 2024-09-16T07:36:52.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.598+0000 7fb98ce00640 1 Processor -- start 2024-09-16T07:36:52.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.598+0000 7fb98ce00640 1 -- start start 2024-09-16T07:36:52.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.599+0000 7fb98ce00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb96c004d90 0x7fb98815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:52.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.599+0000 7fb98ce00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880b7dc0 0x7fb9880b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:52.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.599+0000 7fb98ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9880b8240 0x7fb9880b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:52.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.599+0000 7fb98ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb9880b50c0 con 0x7fb9880b8240 2024-09-16T07:36:52.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.599+0000 7fb98ce00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb9880b4f40 con 0x7fb9880b7dc0 2024-09-16T07:36:52.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.599+0000 7fb98ce00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9880b3ac0 con 0x7fb96c004d90 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9880b8240 0x7fb9880b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9880b8240 0x7fb9880b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55254/0 (socket says 172.21.15.71:55254) 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 -- 172.21.15.71:0/1362445821 learned_addr learned my addr 172.21.15.71:0/1362445821 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb97ea00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880b7dc0 0x7fb9880b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987400640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb96c004d90 0x7fb98815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 -- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb96c004d90 msgr2=0x7fb98815c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb96c004d90 0x7fb98815c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 -- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880b7dc0 msgr2=0x7fb9880b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880b7dc0 0x7fb9880b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 -- 172.21.15.71:0/1362445821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9880b8740 con 0x7fb9880b8240 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb987e00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9880b8240 0x7fb9880b1d60 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb980036040 tx=0x7fb980034c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb97fe00640 1 -- 172.21.15.71:0/1362445821 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980043070 con 0x7fb9880b8240 2024-09-16T07:36:52.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb97fe00640 1 -- 172.21.15.71:0/1362445821 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb980034cf0 con 0x7fb9880b8240 2024-09-16T07:36:52.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb97fe00640 1 -- 172.21.15.71:0/1362445821 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980038040 con 0x7fb9880b8240 2024-09-16T07:36:52.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.600+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9880c8440 con 0x7fb9880b8240 2024-09-16T07:36:52.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.601+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9880c8a00 con 0x7fb9880b8240 2024-09-16T07:36:52.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.602+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9880b33b0 con 0x7fb9880b8240 2024-09-16T07:36:52.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.603+0000 7fb97fe00640 1 -- 172.21.15.71:0/1362445821 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7fb980003bd0 con 0x7fb9880b8240 2024-09-16T07:36:52.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.604+0000 7fb97fe00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb96407a9b0 0x7fb96407ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:52.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.604+0000 7fb97fe00640 1 -- 172.21.15.71:0/1362445821 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7fb9800b9850 con 0x7fb9880b8240 2024-09-16T07:36:52.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.604+0000 7fb987400640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb96407a9b0 0x7fb96407ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:52.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.605+0000 7fb987400640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb96407a9b0 0x7fb96407ce70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb978006bc0 tx=0x7fb978000ed0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:52.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.608+0000 7fb97fe00640 1 -- 172.21.15.71:0/1362445821 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb980083370 con 0x7fb9880b8240 2024-09-16T07:36:52.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.815+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb988069d80 con 0x7fb96407a9b0 2024-09-16T07:36:52.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.820+0000 7fb97fe00640 1 -- 172.21.15.71:0/1362445821 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fb9780035f0 con 0x7fb96407a9b0 2024-09-16T07:36:52.820 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:52.820 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:36:52.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:52.821 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sv1" 2024-09-16T07:36:52.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:36:52.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:52.821 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1clone1" 2024-09-16T07:36:52.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:52.821 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:36:52.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb96407a9b0 msgr2=0x7fb96407ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb96407a9b0 0x7fb96407ce70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb978006bc0 tx=0x7fb978000ed0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9880b8240 msgr2=0x7fb9880b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:52.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9880b8240 0x7fb9880b1d60 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb980036040 tx=0x7fb980034c00 comp rx=0 tx=0).stop 2024-09-16T07:36:52.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 shutdown_connections 2024-09-16T07:36:52.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb96407a9b0 0x7fb96407ce70 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9880b8240 0x7fb9880b1d60 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880b7dc0 0x7fb9880b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 --2- 172.21.15.71:0/1362445821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb96c004d90 0x7fb98815c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:52.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 >> 172.21.15.71:0/1362445821 conn(0x7fb98809d0b0 msgr2=0x7fb9880b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:52.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 shutdown_connections 2024-09-16T07:36:52.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:52.823+0000 7fb98ce00640 1 -- 172.21.15.71:0/1362445821 wait complete. 2024-09-16T07:36:52.838 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T07:36:53.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.001+0000 7fbc44e00640 1 -- 172.21.15.71:0/1856509495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc400b4770 msgr2=0x7fbc4006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.001+0000 7fbc44e00640 1 --2- 172.21.15.71:0/1856509495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc400b4770 0x7fbc4006e6f0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fbc400b2680 tx=0x7fbc2c02a560 comp rx=0 tx=0).stop 2024-09-16T07:36:53.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.001+0000 7fbc44e00640 1 -- 172.21.15.71:0/1856509495 shutdown_connections 2024-09-16T07:36:53.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.001+0000 7fbc44e00640 1 --2- 172.21.15.71:0/1856509495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc4006ec30 0x7fbc40161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.001+0000 7fbc44e00640 1 --2- 172.21.15.71:0/1856509495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc400b4770 0x7fbc4006e6f0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.001+0000 7fbc44e00640 1 --2- 172.21.15.71:0/1856509495 >> v2:172.21.15.177:3300/0 conn(0x7fbc4015dfc0 0x7fbc4006f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.001+0000 7fbc44e00640 1 -- 172.21.15.71:0/1856509495 >> 172.21.15.71:0/1856509495 conn(0x7fbc400b18e0 msgr2=0x7fbc400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:53.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.001+0000 7fbc44e00640 1 -- 172.21.15.71:0/1856509495 shutdown_connections 2024-09-16T07:36:53.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.002+0000 7fbc44e00640 1 -- 172.21.15.71:0/1856509495 wait complete. 2024-09-16T07:36:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.003+0000 7fbc44e00640 1 Processor -- start 2024-09-16T07:36:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.003+0000 7fbc44e00640 1 -- start start 2024-09-16T07:36:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc44e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc4006ec30 0x7fbc4015a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc44e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400b4770 0x7fbc4015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc44e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc4015dfc0 0x7fbc401f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc44e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40162480 con 0x7fbc4006ec30 2024-09-16T07:36:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc44e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40162300 con 0x7fbc4015dfc0 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc44e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbc40162600 con 0x7fbc400b4770 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc3de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc4006ec30 0x7fbc4015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc3de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc4006ec30 0x7fbc4015a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55264/0 (socket says 172.21.15.71:55264) 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc3de00640 1 -- 172.21.15.71:0/3851584073 learned_addr learned my addr 172.21.15.71:0/3851584073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.004+0000 7fbc3e800640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc4015dfc0 0x7fbc401f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc3d400640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400b4770 0x7fbc4015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc3de00640 1 -- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400b4770 msgr2=0x7fbc4015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc3de00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400b4770 0x7fbc4015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc3de00640 1 -- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc4015dfc0 msgr2=0x7fbc401f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc3de00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc4015dfc0 0x7fbc401f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc3de00640 1 -- 172.21.15.71:0/3851584073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4015c080 con 0x7fbc4006ec30 2024-09-16T07:36:53.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc3de00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc4006ec30 0x7fbc4015a480 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000eb30 tx=0x7fbc3000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:53.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc36000640 1 -- 172.21.15.71:0/3851584073 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30010430 con 0x7fbc4006ec30 2024-09-16T07:36:53.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc36000640 1 -- 172.21.15.71:0/3851584073 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc300105d0 con 0x7fbc4006ec30 2024-09-16T07:36:53.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4015c360 con 0x7fbc4006ec30 2024-09-16T07:36:53.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.005+0000 7fbc36000640 1 -- 172.21.15.71:0/3851584073 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30010a20 con 0x7fbc4006ec30 2024-09-16T07:36:53.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.006+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4015c920 con 0x7fbc4006ec30 2024-09-16T07:36:53.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.007+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc40069360 con 0x7fbc4006ec30 2024-09-16T07:36:53.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.008+0000 7fbc36000640 1 -- 172.21.15.71:0/3851584073 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7fbc30010bc0 con 0x7fbc4006ec30 2024-09-16T07:36:53.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.009+0000 7fbc36000640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc2007a9b0 0x7fbc2007ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.009+0000 7fbc36000640 1 -- 172.21.15.71:0/3851584073 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7fbc3009a6e0 con 0x7fbc4006ec30 2024-09-16T07:36:53.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.009+0000 7fbc3d400640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc2007a9b0 0x7fbc2007ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.010+0000 7fbc3d400640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc2007a9b0 0x7fbc2007ce70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c033000 tx=0x7fbc2c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:53.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.013+0000 7fbc36000640 1 -- 172.21.15.71:0/3851584073 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbc30064200 con 0x7fbc4006ec30 2024-09-16T07:36:53.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.226+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc4006e6f0 con 0x7fbc2007a9b0 2024-09-16T07:36:53.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.241+0000 7fbc36000640 1 -- 172.21.15.71:0/3851584073 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbc4006e6f0 con 0x7fbc2007a9b0 2024-09-16T07:36:53.241 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:53.241 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:36:53.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:36:53.241 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1" 2024-09-16T07:36:53.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:36:53.241 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:36:53.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc2007a9b0 msgr2=0x7fbc2007ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc2007a9b0 0x7fbc2007ce70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c033000 tx=0x7fbc2c004730 comp rx=0 tx=0).stop 2024-09-16T07:36:53.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc4006ec30 msgr2=0x7fbc4015a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc4006ec30 0x7fbc4015a480 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000eb30 tx=0x7fbc3000c2d0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 shutdown_connections 2024-09-16T07:36:53.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbc2007a9b0 0x7fbc2007ce70 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc4015dfc0 0x7fbc401f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400b4770 0x7fbc4015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 --2- 172.21.15.71:0/3851584073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc4006ec30 0x7fbc4015a480 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.243+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 >> 172.21.15.71:0/3851584073 conn(0x7fbc400b18e0 msgr2=0x7fbc400b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:53.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.244+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 shutdown_connections 2024-09-16T07:36:53.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.244+0000 7fbc44e00640 1 -- 172.21.15.71:0/3851584073 wait complete. 2024-09-16T07:36:53.258 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T07:36:53.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.428+0000 7f877ac00640 1 -- 172.21.15.71:0/1815206333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877406efa0 msgr2=0x7f87740cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.428+0000 7f877ac00640 1 --2- 172.21.15.71:0/1815206333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877406efa0 0x7f87740cdbb0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f87740af780 tx=0x7f876c02a4d0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.428+0000 7f877ac00640 1 -- 172.21.15.71:0/1815206333 shutdown_connections 2024-09-16T07:36:53.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.428+0000 7f877ac00640 1 --2- 172.21.15.71:0/1815206333 >> v2:172.21.15.177:3300/0 conn(0x7f87740cccd0 0x7f8774073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.428+0000 7f877ac00640 1 --2- 172.21.15.71:0/1815206333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f877406efa0 0x7f87740cdbb0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.428+0000 7f877ac00640 1 -- 172.21.15.71:0/1815206333 >> 172.21.15.71:0/1815206333 conn(0x7f87740b1840 msgr2=0x7f87740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:53.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.428+0000 7f877ac00640 1 -- 172.21.15.71:0/1815206333 shutdown_connections 2024-09-16T07:36:53.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.428+0000 7f877ac00640 1 -- 172.21.15.71:0/1815206333 wait complete. 2024-09-16T07:36:53.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.430+0000 7f877ac00640 1 Processor -- start 2024-09-16T07:36:53.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.430+0000 7f877ac00640 1 -- start start 2024-09-16T07:36:53.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f877ac00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f877406efa0 0x7f877412cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f877ac00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87740cccd0 0x7f87740b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f877ac00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8774145750 0x7f87740b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f877ac00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f87740ce480 con 0x7f87740cccd0 2024-09-16T07:36:53.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f877ac00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f87740ce300 con 0x7f877406efa0 2024-09-16T07:36:53.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f877ac00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f87740ce600 con 0x7f8774145750 2024-09-16T07:36:53.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87740cccd0 0x7f87740b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87740cccd0 0x7f87740b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55294/0 (socket says 172.21.15.71:55294) 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773400640 1 -- 172.21.15.71:0/1257822266 learned_addr learned my addr 172.21.15.71:0/1257822266 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773e00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f877406efa0 0x7f877412cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773400640 1 -- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8774145750 msgr2=0x7f87740b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8778c00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8774145750 0x7f87740b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773400640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8774145750 0x7f87740b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773400640 1 -- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f877406efa0 msgr2=0x7f877412cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773400640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f877406efa0 0x7f877412cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773400640 1 -- 172.21.15.71:0/1257822266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877412d950 con 0x7f87740cccd0 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8773e00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f877406efa0 0x7f877412cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.431+0000 7f8778c00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8774145750 0x7f87740b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:36:53.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.432+0000 7f8773400640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87740cccd0 0x7f87740b3bc0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f876000d770 tx=0x7f876000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:53.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.432+0000 7f876be00640 1 -- 172.21.15.71:0/1257822266 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760017070 con 0x7f87740cccd0 2024-09-16T07:36:53.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.432+0000 7f876be00640 1 -- 172.21.15.71:0/1257822266 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8760009d20 con 0x7f87740cccd0 2024-09-16T07:36:53.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.432+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877412dc30 con 0x7f87740cccd0 2024-09-16T07:36:53.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.432+0000 7f876be00640 1 -- 172.21.15.71:0/1257822266 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760005280 con 0x7f87740cccd0 2024-09-16T07:36:53.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.432+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877412e1f0 con 0x7f87740cccd0 2024-09-16T07:36:53.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.433+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87740cdbb0 con 0x7f87740cccd0 2024-09-16T07:36:53.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.434+0000 7f876be00640 1 -- 172.21.15.71:0/1257822266 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7f8760005420 con 0x7f87740cccd0 2024-09-16T07:36:53.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.435+0000 7f876be00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f875407a980 0x7f875407ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.435+0000 7f876be00640 1 -- 172.21.15.71:0/1257822266 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f8760099c70 con 0x7f87740cccd0 2024-09-16T07:36:53.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.436+0000 7f8773e00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f875407a980 0x7f875407ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.437+0000 7f8773e00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f875407a980 0x7f875407ce40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f87741f8150 tx=0x7f87640073c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:53.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.439+0000 7f876be00640 1 -- 172.21.15.71:0/1257822266 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8760063d40 con 0x7f87740cccd0 2024-09-16T07:36:53.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.660+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f877412e9f0 con 0x7f875407a980 2024-09-16T07:36:53.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.689+0000 7f876be00640 1 -- 172.21.15.71:0/1257822266 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f877412e9f0 con 0x7f875407a980 2024-09-16T07:36:53.689 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f875407a980 msgr2=0x7f875407ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f875407a980 0x7f875407ce40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f87741f8150 tx=0x7f87640073c0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87740cccd0 msgr2=0x7f87740b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87740cccd0 0x7f87740b3bc0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f876000d770 tx=0x7f876000dc30 comp rx=0 tx=0).stop 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 shutdown_connections 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f875407a980 0x7f875407ce40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8774145750 0x7f87740b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87740cccd0 0x7f87740b3bc0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 --2- 172.21.15.71:0/1257822266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f877406efa0 0x7f877412cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 >> 172.21.15.71:0/1257822266 conn(0x7f87740b1840 msgr2=0x7f87740a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:53.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 shutdown_connections 2024-09-16T07:36:53.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.692+0000 7f877ac00640 1 -- 172.21.15.71:0/1257822266 wait complete. 2024-09-16T07:36:53.709 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T07:36:53.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.869+0000 7f1a67800640 1 -- 172.21.15.71:0/2693330394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a6015c080 msgr2=0x7f1a6015eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.869+0000 7f1a67800640 1 --2- 172.21.15.71:0/2693330394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a6015c080 0x7f1a6015eec0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009830 tx=0x7f1a5802a520 comp rx=0 tx=0).stop 2024-09-16T07:36:53.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.870+0000 7f1a67800640 1 -- 172.21.15.71:0/2693330394 shutdown_connections 2024-09-16T07:36:53.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.870+0000 7f1a67800640 1 --2- 172.21.15.71:0/2693330394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a6015c080 0x7f1a6015eec0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.870+0000 7f1a67800640 1 --2- 172.21.15.71:0/2693330394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a600cf2f0 0x7f1a6015e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.870+0000 7f1a67800640 1 -- 172.21.15.71:0/2693330394 >> 172.21.15.71:0/2693330394 conn(0x7f1a600b1840 msgr2=0x7f1a600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:53.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.870+0000 7f1a67800640 1 -- 172.21.15.71:0/2693330394 shutdown_connections 2024-09-16T07:36:53.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.870+0000 7f1a67800640 1 -- 172.21.15.71:0/2693330394 wait complete. 2024-09-16T07:36:53.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.871+0000 7f1a67800640 1 Processor -- start 2024-09-16T07:36:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.871+0000 7f1a67800640 1 -- start start 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a67800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a600b3370 0x7f1a6013b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a67800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a600cf2f0 0x7f1a6015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a67800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a6015c080 0x7f1a6015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a67800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60164e20 con 0x7f1a600b3370 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a67800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60164ca0 con 0x7f1a600cf2f0 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a67800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1a60164fa0 con 0x7f1a6015c080 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a64e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a600b3370 0x7f1a6013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a64e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a600b3370 0x7f1a6013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55312/0 (socket says 172.21.15.71:55312) 2024-09-16T07:36:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.872+0000 7f1a64e00640 1 -- 172.21.15.71:0/3917841780 learned_addr learned my addr 172.21.15.71:0/3917841780 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a65800640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a6015c080 0x7f1a6015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a5fe00640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a600cf2f0 0x7f1a6015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a64e00640 1 -- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a6015c080 msgr2=0x7f1a6015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a64e00640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a6015c080 0x7f1a6015fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a64e00640 1 -- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a600cf2f0 msgr2=0x7f1a6015c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a64e00640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a600cf2f0 0x7f1a6015c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a64e00640 1 -- 172.21.15.71:0/3917841780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6015cee0 con 0x7f1a600b3370 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a64e00640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a600b3370 0x7f1a6013b160 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400cbd0 tx=0x7f1a5400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a5cc00640 1 -- 172.21.15.71:0/3917841780 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54016020 con 0x7f1a600b3370 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a5cc00640 1 -- 172.21.15.71:0/3917841780 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a5400ae30 con 0x7f1a600b3370 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a5cc00640 1 -- 172.21.15.71:0/3917841780 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54012660 con 0x7f1a600b3370 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.873+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a601f6170 con 0x7f1a600b3370 2024-09-16T07:36:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.874+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6013b9c0 con 0x7f1a600b3370 2024-09-16T07:36:53.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.874+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a600af880 con 0x7f1a600b3370 2024-09-16T07:36:53.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.876+0000 7f1a5cc00640 1 -- 172.21.15.71:0/3917841780 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7f1a54019020 con 0x7f1a600b3370 2024-09-16T07:36:53.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.876+0000 7f1a5cc00640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a4007a9b0 0x7f1a4007ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:53.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.877+0000 7f1a5cc00640 1 -- 172.21.15.71:0/3917841780 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f1a5409a200 con 0x7f1a600b3370 2024-09-16T07:36:53.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.877+0000 7f1a5fe00640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a4007a9b0 0x7f1a4007ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:53.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.878+0000 7f1a5fe00640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a4007a9b0 0x7f1a4007ce70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c005fd0 tx=0x7f1a4c005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:53.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:53.880+0000 7f1a5cc00640 1 -- 172.21.15.71:0/3917841780 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1a600af880 con 0x7f1a600b3370 2024-09-16T07:36:54.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.092+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f1a600cebe0 con 0x7f1a4007a9b0 2024-09-16T07:36:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.115+0000 7f1a5cc00640 1 -- 172.21.15.71:0/3917841780 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1a600cebe0 con 0x7f1a4007a9b0 2024-09-16T07:36:54.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a4007a9b0 msgr2=0x7f1a4007ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a4007a9b0 0x7f1a4007ce70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c005fd0 tx=0x7f1a4c005850 comp rx=0 tx=0).stop 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a600b3370 msgr2=0x7f1a6013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a600b3370 0x7f1a6013b160 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400cbd0 tx=0x7f1a5400a430 comp rx=0 tx=0).stop 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 shutdown_connections 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1a4007a9b0 0x7f1a4007ce70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a6015c080 0x7f1a6015fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a600cf2f0 0x7f1a6015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 --2- 172.21.15.71:0/3917841780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a600b3370 0x7f1a6013b160 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.117+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 >> 172.21.15.71:0/3917841780 conn(0x7f1a600b1840 msgr2=0x7f1a60073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.118+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 shutdown_connections 2024-09-16T07:36:54.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.118+0000 7f1a67800640 1 -- 172.21.15.71:0/3917841780 wait complete. 2024-09-16T07:36:54.129 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:54.129+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:54.129 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:54.129+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:54.129 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:54.129+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:54.129 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:54.129+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:54.129 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:36:54.129+0000 7fe76c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:36:54.135 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-16T07:36:54.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.299+0000 7f3378a00640 1 -- 172.21.15.71:0/1716194651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 msgr2=0x7f33740d48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:54.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.299+0000 7f3378a00640 1 --2- 172.21.15.71:0/1716194651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 0x7f33740d48f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f337408d580 tx=0x7f336402a540 comp rx=0 tx=0).stop 2024-09-16T07:36:54.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.299+0000 7f3378a00640 1 -- 172.21.15.71:0/1716194651 shutdown_connections 2024-09-16T07:36:54.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.299+0000 7f3378a00640 1 --2- 172.21.15.71:0/1716194651 >> v2:172.21.15.177:3300/0 conn(0x7f3374161580 0x7f33740b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.299+0000 7f3378a00640 1 --2- 172.21.15.71:0/1716194651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 0x7f33740d48f0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.299+0000 7f3378a00640 1 -- 172.21.15.71:0/1716194651 >> 172.21.15.71:0/1716194651 conn(0x7f33740d3dd0 msgr2=0x7f33740d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:54.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.299+0000 7f3378a00640 1 -- 172.21.15.71:0/1716194651 shutdown_connections 2024-09-16T07:36:54.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.299+0000 7f3378a00640 1 -- 172.21.15.71:0/1716194651 wait complete. 2024-09-16T07:36:54.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.301+0000 7f3378a00640 1 Processor -- start 2024-09-16T07:36:54.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.301+0000 7f3378a00640 1 -- start start 2024-09-16T07:36:54.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3378a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33740cfa90 0x7f33741f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:54.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3378a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 0x7f33740c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3378a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374161580 0x7f33740ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3378a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3374161f10 con 0x7f33741611a0 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3378a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3374161d90 con 0x7f3374161580 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3378a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3374162090 con 0x7f33740cfa90 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f336be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 0x7f33740c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f336be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 0x7f33740c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55326/0 (socket says 172.21.15.71:55326) 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f336be00640 1 -- 172.21.15.71:0/4094368829 learned_addr learned my addr 172.21.15.71:0/4094368829 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3373400640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33740cfa90 0x7f33741f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3373e00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374161580 0x7f33740ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f336be00640 1 -- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33740cfa90 msgr2=0x7f33741f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f336be00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33740cfa90 0x7f33741f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f336be00640 1 -- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374161580 msgr2=0x7f33740ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f336be00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374161580 0x7f33740ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f336be00640 1 -- 172.21.15.71:0/4094368829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33740c9c40 con 0x7f33741611a0 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.302+0000 7f3373400640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33740cfa90 0x7f33741f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.303+0000 7f336be00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 0x7f33740c9700 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f335c00b410 tx=0x7f335c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.303+0000 7f336b400640 1 -- 172.21.15.71:0/4094368829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c015020 con 0x7f33741611a0 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.303+0000 7f336b400640 1 -- 172.21.15.71:0/4094368829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f335c011940 con 0x7f33741611a0 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.303+0000 7f336b400640 1 -- 172.21.15.71:0/4094368829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c011d90 con 0x7f33741611a0 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.303+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741f81b0 con 0x7f33741611a0 2024-09-16T07:36:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.303+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741f8770 con 0x7f33741611a0 2024-09-16T07:36:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.304+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33740d48f0 con 0x7f33741611a0 2024-09-16T07:36:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.305+0000 7f336b400640 1 -- 172.21.15.71:0/4094368829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 47) ==== 104296+0+0 (secure 0 0 0) 0x7f335c018020 con 0x7f33741611a0 2024-09-16T07:36:54.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.307+0000 7f336b400640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335007a9b0 0x7f335007ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:36:54.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.307+0000 7f336b400640 1 -- 172.21.15.71:0/4094368829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5793+0+0 (secure 0 0 0) 0x7f335c099960 con 0x7f33741611a0 2024-09-16T07:36:54.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.307+0000 7f3373400640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335007a9b0 0x7f335007ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:36:54.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.308+0000 7f3373400640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335007a9b0 0x7f335007ce70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3364033000 tx=0x7f3364004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:36:54.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.311+0000 7f336b400640 1 -- 172.21.15.71:0/4094368829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f335c01f020 con 0x7f33741611a0 2024-09-16T07:36:54.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.532+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f33740b8ca0 con 0x7f335007a9b0 2024-09-16T07:36:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.843+0000 7f336b400640 1 -- 172.21.15.71:0/4094368829 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33740b8ca0 con 0x7f335007a9b0 2024-09-16T07:36:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335007a9b0 msgr2=0x7f335007ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335007a9b0 0x7f335007ce70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3364033000 tx=0x7f3364004730 comp rx=0 tx=0).stop 2024-09-16T07:36:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 msgr2=0x7f33740c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:36:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 0x7f33740c9700 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f335c00b410 tx=0x7f335c00b8d0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 shutdown_connections 2024-09-16T07:36:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f335007a9b0 0x7f335007ce70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374161580 0x7f33740ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33741611a0 0x7f33740c9700 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.845+0000 7f3378a00640 1 --2- 172.21.15.71:0/4094368829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33740cfa90 0x7f33741f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:36:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.846+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 >> 172.21.15.71:0/4094368829 conn(0x7f33740d3dd0 msgr2=0x7f3374160200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:36:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.846+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 shutdown_connections 2024-09-16T07:36:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54.846+0000 7f3378a00640 1 -- 172.21.15.71:0/4094368829 wait complete. 2024-09-16T07:36:54.861 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:36:54.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:36:54.946 INFO:teuthology.orchestra.run.smithi071.stdout:3 2024-09-16T07:36:54.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:36:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:36:59.956 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:37:00.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:37:00.046 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:37:00.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:37:00.055 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:37:00.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.216+0000 7fa4fee00640 1 -- 172.21.15.71:0/2234831094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 msgr2=0x7fa4f8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:00.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.216+0000 7fa4fee00640 1 --2- 172.21.15.71:0/2234831094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 0x7fa4f8073440 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80af750 tx=0x7fa4e802a580 comp rx=0 tx=0).stop 2024-09-16T07:37:00.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.216+0000 7fa4fee00640 1 -- 172.21.15.71:0/2234831094 shutdown_connections 2024-09-16T07:37:00.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.217+0000 7fa4fee00640 1 --2- 172.21.15.71:0/2234831094 >> v2:172.21.15.177:3300/0 conn(0x7fa4f806efa0 0x7fa4f80cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.217+0000 7fa4fee00640 1 --2- 172.21.15.71:0/2234831094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 0x7fa4f8073440 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.217+0000 7fa4fee00640 1 -- 172.21.15.71:0/2234831094 >> 172.21.15.71:0/2234831094 conn(0x7fa4f80b1840 msgr2=0x7fa4f80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:00.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.217+0000 7fa4fee00640 1 -- 172.21.15.71:0/2234831094 shutdown_connections 2024-09-16T07:37:00.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.217+0000 7fa4fee00640 1 -- 172.21.15.71:0/2234831094 wait complete. 2024-09-16T07:37:00.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.218+0000 7fa4fee00640 1 Processor -- start 2024-09-16T07:37:00.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.218+0000 7fa4fee00640 1 -- start start 2024-09-16T07:37:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4f806efa0 0x7fa4f81415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fee00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4f80ccb00 0x7fa4f80cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 0x7fa4f80cf580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f80ced40 con 0x7fa4f8145750 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fee00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f80cebc0 con 0x7fa4f806efa0 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fee00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f80ceec0 con 0x7fa4f80ccb00 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 0x7fa4f80cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 0x7fa4f80cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58946/0 (socket says 172.21.15.71:58946) 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.219+0000 7fa4fce00640 1 -- 172.21.15.71:0/4225520668 learned_addr learned my addr 172.21.15.71:0/4225520668 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4efe00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4f80ccb00 0x7fa4f80cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4f7e00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4f806efa0 0x7fa4f81415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4fce00640 1 -- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4f80ccb00 msgr2=0x7fa4f80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4fce00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4f80ccb00 0x7fa4f80cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4fce00640 1 -- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4f806efa0 msgr2=0x7fa4f81415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4fce00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4f806efa0 0x7fa4f81415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4fce00640 1 -- 172.21.15.71:0/4225520668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f81f8470 con 0x7fa4f8145750 2024-09-16T07:37:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4fce00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 0x7fa4f80cf580 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c7a0 tx=0x7fa4f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4f4c00640 1 -- 172.21.15.71:0/4225520668 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0003b70 con 0x7fa4f8145750 2024-09-16T07:37:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4f4c00640 1 -- 172.21.15.71:0/4225520668 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4f0003d30 con 0x7fa4f8145750 2024-09-16T07:37:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4f4c00640 1 -- 172.21.15.71:0/4225520668 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000aab0 con 0x7fa4f8145750 2024-09-16T07:37:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.220+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f81f8750 con 0x7fa4f8145750 2024-09-16T07:37:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.221+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f81f8d10 con 0x7fa4f8145750 2024-09-16T07:37:00.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.222+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f80cc5c0 con 0x7fa4f8145750 2024-09-16T07:37:00.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.223+0000 7fa4f4c00640 1 -- 172.21.15.71:0/4225520668 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fa4f000ac50 con 0x7fa4f8145750 2024-09-16T07:37:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.224+0000 7fa4f4c00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4d407aa10 0x7fa4d407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.224+0000 7fa4f4c00640 1 -- 172.21.15.71:0/4225520668 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5849+0+0 (secure 0 0 0) 0x7fa4f0099900 con 0x7fa4f8145750 2024-09-16T07:37:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.225+0000 7fa4f7e00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4d407aa10 0x7fa4d407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.226+0000 7fa4f7e00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4d407aa10 0x7fa4d407ced0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8033000 tx=0x7fa4e8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:00.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.229+0000 7fa4f4c00640 1 -- 172.21.15.71:0/4225520668 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa4f00639d0 con 0x7fa4f8145750 2024-09-16T07:37:00.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.413+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fa4f80ae1b0 con 0x7fa4f8145750 2024-09-16T07:37:00.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.414+0000 7fa4f4c00640 1 -- 172.21.15.71:0/4225520668 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7fa4f00632d0 con 0x7fa4f8145750 2024-09-16T07:37:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4d407aa10 msgr2=0x7fa4d407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4d407aa10 0x7fa4d407ced0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8033000 tx=0x7fa4e8004730 comp rx=0 tx=0).stop 2024-09-16T07:37:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 msgr2=0x7fa4f80cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 0x7fa4f80cf580 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c7a0 tx=0x7fa4f000cc60 comp rx=0 tx=0).stop 2024-09-16T07:37:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 shutdown_connections 2024-09-16T07:37:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa4d407aa10 0x7fa4d407ced0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f8145750 0x7fa4f80cf580 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4f80ccb00 0x7fa4f80cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 --2- 172.21.15.71:0/4225520668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4f806efa0 0x7fa4f81415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 >> 172.21.15.71:0/4225520668 conn(0x7fa4f80b1840 msgr2=0x7fa4f80aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 shutdown_connections 2024-09-16T07:37:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.416+0000 7fa4fee00640 1 -- 172.21.15.71:0/4225520668 wait complete. 2024-09-16T07:37:00.432 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:37:00.432 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:37:00.529 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:37:00.622 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:37:00.622 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:37:00.622 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:00.623 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:00.658 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:00.658 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:00.733 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:37:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.936+0000 7f8b07400640 1 -- 172.21.15.71:0/4264784390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b00073fb0 msgr2=0x7f8b00074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.936+0000 7f8b07400640 1 --2- 172.21.15.71:0/4264784390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b00073fb0 0x7f8b00074440 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8af8005fd0 tx=0x7f8af802a820 comp rx=0 tx=0).stop 2024-09-16T07:37:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.936+0000 7f8b07400640 1 -- 172.21.15.71:0/4264784390 shutdown_connections 2024-09-16T07:37:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.936+0000 7f8b07400640 1 --2- 172.21.15.71:0/4264784390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b00073fb0 0x7f8b00074440 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.936+0000 7f8b07400640 1 --2- 172.21.15.71:0/4264784390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000ccf00 0x7f8b000b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.936+0000 7f8b07400640 1 -- 172.21.15.71:0/4264784390 >> 172.21.15.71:0/4264784390 conn(0x7f8b000b1950 msgr2=0x7f8b000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.936+0000 7f8b07400640 1 -- 172.21.15.71:0/4264784390 shutdown_connections 2024-09-16T07:37:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.936+0000 7f8b07400640 1 -- 172.21.15.71:0/4264784390 wait complete. 2024-09-16T07:37:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.938+0000 7f8b07400640 1 Processor -- start 2024-09-16T07:37:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.938+0000 7f8b07400640 1 -- start start 2024-09-16T07:37:00.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.938+0000 7f8b07400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b00073fb0 0x7f8b00154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:00.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8b07400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b000ccf00 0x7f8b0015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8b07400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b0015f5a0 0x7f8b001e4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8b07400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0006f2e0 con 0x7f8b000ccf00 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8b07400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0006f160 con 0x7f8b00073fb0 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8b07400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0006f460 con 0x7f8b0015f5a0 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8affe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b000ccf00 0x7f8b0015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8affe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b000ccf00 0x7f8b0015a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58974/0 (socket says 172.21.15.71:58974) 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8affe00640 1 -- 172.21.15.71:0/66645829 learned_addr learned my addr 172.21.15.71:0/66645829 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8b05400640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b0015f5a0 0x7f8b001e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8b04a00640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b00073fb0 0x7f8b00154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8affe00640 1 -- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b0015f5a0 msgr2=0x7f8b001e4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8affe00640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b0015f5a0 0x7f8b001e4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8affe00640 1 -- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b00073fb0 msgr2=0x7f8b00154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8affe00640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b00073fb0 0x7f8b00154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8affe00640 1 -- 172.21.15.71:0/66645829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b000b35e0 con 0x7f8b000ccf00 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.939+0000 7f8b04a00640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b00073fb0 0x7f8b00154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.940+0000 7f8affe00640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b000ccf00 0x7f8b0015a780 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c920 tx=0x7f8aec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.940+0000 7f8afcc00640 1 -- 172.21.15.71:0/66645829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec007c70 con 0x7f8b000ccf00 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.940+0000 7f8afcc00640 1 -- 172.21.15.71:0/66645829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8aec00f450 con 0x7f8b000ccf00 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.940+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b000b38c0 con 0x7f8b000ccf00 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.940+0000 7f8afcc00640 1 -- 172.21.15.71:0/66645829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec00f8a0 con 0x7f8b000ccf00 2024-09-16T07:37:00.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.940+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0015b060 con 0x7f8b000ccf00 2024-09-16T07:37:00.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.941+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b000b3cb0 con 0x7f8b000ccf00 2024-09-16T07:37:00.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.942+0000 7f8afcc00640 1 -- 172.21.15.71:0/66645829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f8aec00fa40 con 0x7f8b000ccf00 2024-09-16T07:37:00.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.943+0000 7f8afcc00640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ae007aa10 0x7f8ae007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:00.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.943+0000 7f8afcc00640 1 -- 172.21.15.71:0/66645829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5849+0+0 (secure 0 0 0) 0x7f8aec09a330 con 0x7f8b000ccf00 2024-09-16T07:37:00.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.943+0000 7f8b04a00640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ae007aa10 0x7f8ae007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:00.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.944+0000 7f8b04a00640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ae007aa10 0x7f8ae007ced0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8af4001220 tx=0x7f8af4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:00.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:00.947+0000 7f8afcc00640 1 -- 172.21.15.71:0/66645829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8aec014030 con 0x7f8b000ccf00 2024-09-16T07:37:01.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.145+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b0006f9f0 con 0x7f8b000ccf00 2024-09-16T07:37:01.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.145+0000 7f8afcc00640 1 -- 172.21.15.71:0/66645829 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f8aec064400 con 0x7f8b000ccf00 2024-09-16T07:37:01.145 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:37:01.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ae007aa10 msgr2=0x7f8ae007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ae007aa10 0x7f8ae007ced0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8af4001220 tx=0x7f8af4009040 comp rx=0 tx=0).stop 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b000ccf00 msgr2=0x7f8b0015a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b000ccf00 0x7f8b0015a780 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c920 tx=0x7f8aec00cde0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 shutdown_connections 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8ae007aa10 0x7f8ae007ced0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b0015f5a0 0x7f8b001e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b000ccf00 0x7f8b0015a780 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 --2- 172.21.15.71:0/66645829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b00073fb0 0x7f8b00154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 >> 172.21.15.71:0/66645829 conn(0x7f8b000b1950 msgr2=0x7f8b00155340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 shutdown_connections 2024-09-16T07:37:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.148+0000 7f8b07400640 1 -- 172.21.15.71:0/66645829 wait complete. 2024-09-16T07:37:01.164 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:37:01.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:01.164 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:37:01.266 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:37:01.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:01.266 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:37:01.306 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:37:01.306 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:37:01.306 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:01.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:37:01.343 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:37:01.343 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:01.344 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:01.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:37:01.382 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:37:01.382 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:37:01.382 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:01.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:37:01.420 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:37:01.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.580+0000 7f29c2c00640 1 -- 172.21.15.71:0/2066095475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc0ccf00 msgr2=0x7f29bc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.580+0000 7f29c2c00640 1 --2- 172.21.15.71:0/2066095475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc0ccf00 0x7f29bc0b2440 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009830 tx=0x7f29a802a530 comp rx=0 tx=0).stop 2024-09-16T07:37:01.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.580+0000 7f29c2c00640 1 -- 172.21.15.71:0/2066095475 shutdown_connections 2024-09-16T07:37:01.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.580+0000 7f29c2c00640 1 --2- 172.21.15.71:0/2066095475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc0ccf00 0x7f29bc0b2440 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.580+0000 7f29c2c00640 1 -- 172.21.15.71:0/2066095475 >> 172.21.15.71:0/2066095475 conn(0x7f29bc0b1950 msgr2=0x7f29bc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:01.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.581+0000 7f29c2c00640 1 -- 172.21.15.71:0/2066095475 shutdown_connections 2024-09-16T07:37:01.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.581+0000 7f29c2c00640 1 -- 172.21.15.71:0/2066095475 wait complete. 2024-09-16T07:37:01.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.582+0000 7f29c2c00640 1 Processor -- start 2024-09-16T07:37:01.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.582+0000 7f29c2c00640 1 -- start start 2024-09-16T07:37:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c2c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc073fb0 0x7f29bc0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c2c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0ccf00 0x7f29bc0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c2c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29bc15f5a0 0x7f29bc0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c2c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc06f9f0 con 0x7f29bc15f5a0 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c2c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc06f870 con 0x7f29bc073fb0 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c2c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc074420 con 0x7f29bc0ccf00 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29bc15f5a0 0x7f29bc0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29bc15f5a0 0x7f29bc0b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58982/0 (socket says 172.21.15.71:58982) 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.583+0000 7f29c0c00640 1 -- 172.21.15.71:0/681761412 learned_addr learned my addr 172.21.15.71:0/681761412 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29bbe00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc073fb0 0x7f29bc0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29bb400640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0ccf00 0x7f29bc0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29c0c00640 1 -- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0ccf00 msgr2=0x7f29bc0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29c0c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0ccf00 0x7f29bc0c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29c0c00640 1 -- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc073fb0 msgr2=0x7f29bc0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29c0c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc073fb0 0x7f29bc0d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29c0c00640 1 -- 172.21.15.71:0/681761412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc0cffe0 con 0x7f29bc15f5a0 2024-09-16T07:37:01.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29c0c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29bc15f5a0 0x7f29bc0b45f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f29b400c7a0 tx=0x7f29b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:01.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29b3e00640 1 -- 172.21.15.71:0/681761412 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4003b70 con 0x7f29bc15f5a0 2024-09-16T07:37:01.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29b3e00640 1 -- 172.21.15.71:0/681761412 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29b4003d10 con 0x7f29bc15f5a0 2024-09-16T07:37:01.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc0d0820 con 0x7f29bc15f5a0 2024-09-16T07:37:01.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.584+0000 7f29b3e00640 1 -- 172.21.15.71:0/681761412 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b400aa50 con 0x7f29bc15f5a0 2024-09-16T07:37:01.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.585+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1f84c0 con 0x7f29bc15f5a0 2024-09-16T07:37:01.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.586+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc0b2440 con 0x7f29bc15f5a0 2024-09-16T07:37:01.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.587+0000 7f29b3e00640 1 -- 172.21.15.71:0/681761412 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f29b400abf0 con 0x7f29bc15f5a0 2024-09-16T07:37:01.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.588+0000 7f29b3e00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f299c07aa10 0x7f299c07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:01.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.589+0000 7f29b3e00640 1 -- 172.21.15.71:0/681761412 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5849+0+0 (secure 0 0 0) 0x7f29b4099810 con 0x7f29bc15f5a0 2024-09-16T07:37:01.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.589+0000 7f29bbe00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f299c07aa10 0x7f299c07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:01.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.590+0000 7f29bbe00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f299c07aa10 0x7f299c07ced0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0046b0 tx=0x7f29ac007cc0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:01.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.593+0000 7f29b3e00640 1 -- 172.21.15.71:0/681761412 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f29bc0cffe0 con 0x7f29bc15f5a0 2024-09-16T07:37:01.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.812+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29bc0cc060 con 0x7f29bc15f5a0 2024-09-16T07:37:01.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.812+0000 7f29b3e00640 1 -- 172.21.15.71:0/681761412 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5231 (secure 0 0 0) 0x7f29b4063290 con 0x7f29bc15f5a0 2024-09-16T07:37:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:01.812 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":86,"btime":"2024-09-16T07:36:17:890799+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4275610322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":4275610322},{"type":"v1","addr":"172.21.15.71:6835","nonce":4275610322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/2559248159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2559248159},{"type":"v1","addr":"172.21.15.177:6836","nonce":2559248159}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/1515242906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1515242906},{"type":"v1","addr":"172.21.15.71:6834","nonce":1515242906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:36:04.253023+0000","modified":"2024-09-16T07:36:07.274434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2933333093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2933333093},{"type":"v1","addr":"172.21.15.177:6837","nonce":2933333093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":9}]} 2024-09-16T07:37:01.812 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 86 2024-09-16T07:37:01.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f299c07aa10 msgr2=0x7f299c07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f299c07aa10 0x7f299c07ced0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0046b0 tx=0x7f29ac007cc0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29bc15f5a0 msgr2=0x7f29bc0b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29bc15f5a0 0x7f29bc0b45f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f29b400c7a0 tx=0x7f29b400cc60 comp rx=0 tx=0).stop 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 shutdown_connections 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f299c07aa10 0x7f299c07ced0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29bc15f5a0 0x7f29bc0b45f0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0ccf00 0x7f29bc0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 --2- 172.21.15.71:0/681761412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc073fb0 0x7f29bc0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 >> 172.21.15.71:0/681761412 conn(0x7f29bc0b1950 msgr2=0x7f29bc09ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 shutdown_connections 2024-09-16T07:37:01.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.815+0000 7f29c2c00640 1 -- 172.21.15.71:0/681761412 wait complete. 2024-09-16T07:37:01.831 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:37:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.988+0000 7f52a2a00640 1 -- 172.21.15.71:0/3707370704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f529c06f850 msgr2=0x7f529c06fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.988+0000 7f52a2a00640 1 --2- 172.21.15.71:0/3707370704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f529c06f850 0x7f529c06fcb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5288009830 tx=0x7f528802a500 comp rx=0 tx=0).stop 2024-09-16T07:37:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.988+0000 7f52a2a00640 1 -- 172.21.15.71:0/3707370704 shutdown_connections 2024-09-16T07:37:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.988+0000 7f52a2a00640 1 --2- 172.21.15.71:0/3707370704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f529c06f850 0x7f529c06fcb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.988+0000 7f52a2a00640 1 --2- 172.21.15.71:0/3707370704 >> v2:172.21.15.177:3300/0 conn(0x7f529c070920 0x7f529c1438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.988+0000 7f52a2a00640 1 -- 172.21.15.71:0/3707370704 >> 172.21.15.71:0/3707370704 conn(0x7f529c0b1950 msgr2=0x7f529c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.988+0000 7f52a2a00640 1 -- 172.21.15.71:0/3707370704 shutdown_connections 2024-09-16T07:37:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.988+0000 7f52a2a00640 1 -- 172.21.15.71:0/3707370704 wait complete. 2024-09-16T07:37:01.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.989+0000 7f52a2a00640 1 Processor -- start 2024-09-16T07:37:01.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.990+0000 7f52a2a00640 1 -- start start 2024-09-16T07:37:01.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.990+0000 7f52a2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c06f850 0x7f529c1f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.990+0000 7f52a2a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f529c070920 0x7f529c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.990+0000 7f52a2a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f529c15dbf0 0x7f529c13f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f52a2a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f529c15e930 con 0x7f529c06f850 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f52a2a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f529c15e7b0 con 0x7f529c070920 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f52a2a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f529c0cc5f0 con 0x7f529c15dbf0 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c06f850 0x7f529c1f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c06f850 0x7f529c1f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58988/0 (socket says 172.21.15.71:58988) 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 -- 172.21.15.71:0/4224861557 learned_addr learned my addr 172.21.15.71:0/4224861557 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529b400640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f529c070920 0x7f529c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f52a0a00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f529c15dbf0 0x7f529c13f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 -- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f529c15dbf0 msgr2=0x7f529c13f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f529c15dbf0 0x7f529c13f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 -- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f529c070920 msgr2=0x7f529c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f529c070920 0x7f529c0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 -- 172.21.15.71:0/4224861557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c0b12a0 con 0x7f529c06f850 2024-09-16T07:37:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529b400640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f529c070920 0x7f529c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.991+0000 7f529be00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c06f850 0x7f529c1f8e10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f528c00ed40 tx=0x7f528c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.992+0000 7f5293e00640 1 -- 172.21.15.71:0/4224861557 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c00b770 con 0x7f529c06f850 2024-09-16T07:37:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.992+0000 7f5293e00640 1 -- 172.21.15.71:0/4224861557 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f528c00b910 con 0x7f529c06f850 2024-09-16T07:37:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.992+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529c0b1580 con 0x7f529c06f850 2024-09-16T07:37:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.992+0000 7f5293e00640 1 -- 172.21.15.71:0/4224861557 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c00bd00 con 0x7f529c06f850 2024-09-16T07:37:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.992+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529c1f88f0 con 0x7f529c06f850 2024-09-16T07:37:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.993+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529c06fcb0 con 0x7f529c06f850 2024-09-16T07:37:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.994+0000 7f5293e00640 1 -- 172.21.15.71:0/4224861557 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f528c020440 con 0x7f529c06f850 2024-09-16T07:37:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.995+0000 7f5293e00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f527c07aa10 0x7f527c07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.996+0000 7f5293e00640 1 -- 172.21.15.71:0/4224861557 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5849+0+0 (secure 0 0 0) 0x7f528c09aa40 con 0x7f529c06f850 2024-09-16T07:37:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.996+0000 7f529b400640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f527c07aa10 0x7f527c07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:01.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:01.997+0000 7f529b400640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f527c07aa10 0x7f527c07ced0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5288034000 tx=0x7f5288004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:01.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.000+0000 7f5293e00640 1 -- 172.21.15.71:0/4224861557 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f528c064b10 con 0x7f529c06f850 2024-09-16T07:37:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.220+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f529c0cd430 con 0x7f529c06f850 2024-09-16T07:37:02.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.221+0000 7f5293e00640 1 -- 172.21.15.71:0/4224861557 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5231 (secure 0 0 0) 0x7f528c064410 con 0x7f529c06f850 2024-09-16T07:37:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:02.220 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":86,"btime":"2024-09-16T07:36:17:890799+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4275610322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":4275610322},{"type":"v1","addr":"172.21.15.71:6835","nonce":4275610322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/2559248159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2559248159},{"type":"v1","addr":"172.21.15.177:6836","nonce":2559248159}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/1515242906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1515242906},{"type":"v1","addr":"172.21.15.71:6834","nonce":1515242906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:36:04.253023+0000","modified":"2024-09-16T07:36:07.274434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2933333093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2933333093},{"type":"v1","addr":"172.21.15.177:6837","nonce":2933333093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":9}]} 2024-09-16T07:37:02.221 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 86 2024-09-16T07:37:02.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f527c07aa10 msgr2=0x7f527c07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f527c07aa10 0x7f527c07ced0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5288034000 tx=0x7f5288004730 comp rx=0 tx=0).stop 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c06f850 msgr2=0x7f529c1f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c06f850 0x7f529c1f8e10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f528c00ed40 tx=0x7f528c00c7a0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 shutdown_connections 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f527c07aa10 0x7f527c07ced0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f529c15dbf0 0x7f529c13f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f529c070920 0x7f529c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 --2- 172.21.15.71:0/4224861557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c06f850 0x7f529c1f8e10 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 >> 172.21.15.71:0/4224861557 conn(0x7f529c0b1950 msgr2=0x7f529c0cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 shutdown_connections 2024-09-16T07:37:02.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.223+0000 7f52a2a00640 1 -- 172.21.15.71:0/4224861557 wait complete. 2024-09-16T07:37:02.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:37:02.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.394+0000 7f8f4f800640 1 -- 172.21.15.71:0/3692814979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b2440 msgr2=0x7f8f4806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.394+0000 7f8f4f800640 1 --2- 172.21.15.71:0/3692814979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b2440 0x7f8f4806f160 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005fd0 tx=0x7f8f4002a8b0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.394+0000 7f8f4f800640 1 -- 172.21.15.71:0/3692814979 shutdown_connections 2024-09-16T07:37:02.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.394+0000 7f8f4f800640 1 --2- 172.21.15.71:0/3692814979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480b2440 0x7f8f4806f160 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.394+0000 7f8f4f800640 1 -- 172.21.15.71:0/3692814979 >> 172.21.15.71:0/3692814979 conn(0x7f8f480b1950 msgr2=0x7f8f480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.394+0000 7f8f4f800640 1 -- 172.21.15.71:0/3692814979 shutdown_connections 2024-09-16T07:37:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.394+0000 7f8f4f800640 1 -- 172.21.15.71:0/3692814979 wait complete. 2024-09-16T07:37:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.396+0000 7f8f4f800640 1 Processor -- start 2024-09-16T07:37:02.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.396+0000 7f8f4f800640 1 -- start start 2024-09-16T07:37:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f4f800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b2440 0x7f8f4809d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f4f800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480ccf00 0x7f8f481486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f4f800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f4815f5a0 0x7f8f4809b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f4f800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48074840 con 0x7f8f480ccf00 2024-09-16T07:37:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f4f800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f480746c0 con 0x7f8f4815f5a0 2024-09-16T07:37:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f4f800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4806f6a0 con 0x7f8f480b2440 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480ccf00 0x7f8f481486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480ccf00 0x7f8f481486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59018/0 (socket says 172.21.15.71:59018) 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f47e00640 1 -- 172.21.15.71:0/3655621294 learned_addr learned my addr 172.21.15.71:0/3655621294 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f4d800640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f4815f5a0 0x7f8f4809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.397+0000 7f8f4ce00640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b2440 0x7f8f4809d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f47e00640 1 -- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b2440 msgr2=0x7f8f4809d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f47e00640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b2440 0x7f8f4809d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f47e00640 1 -- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f4815f5a0 msgr2=0x7f8f4809b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f47e00640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f4815f5a0 0x7f8f4809b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f47e00640 1 -- 172.21.15.71:0/3655621294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4815aeb0 con 0x7f8f480ccf00 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f4d800640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f4815f5a0 0x7f8f4809b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f4ce00640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b2440 0x7f8f4809d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f47e00640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480ccf00 0x7f8f481486f0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400c490 tx=0x7f8f3400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f44c00640 1 -- 172.21.15.71:0/3655621294 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34011790 con 0x7f8f480ccf00 2024-09-16T07:37:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f44c00640 1 -- 172.21.15.71:0/3655621294 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f34017070 con 0x7f8f480ccf00 2024-09-16T07:37:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f44c00640 1 -- 172.21.15.71:0/3655621294 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34011ce0 con 0x7f8f480ccf00 2024-09-16T07:37:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f48154e70 con 0x7f8f480ccf00 2024-09-16T07:37:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.398+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4815b430 con 0x7f8f480ccf00 2024-09-16T07:37:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.399+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4806f160 con 0x7f8f480ccf00 2024-09-16T07:37:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.401+0000 7f8f44c00640 1 -- 172.21.15.71:0/3655621294 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f8f3401c460 con 0x7f8f480ccf00 2024-09-16T07:37:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.401+0000 7f8f44c00640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f2807aa10 0x7f8f2807ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.402+0000 7f8f44c00640 1 -- 172.21.15.71:0/3655621294 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5849+0+0 (secure 0 0 0) 0x7f8f340998e0 con 0x7f8f480ccf00 2024-09-16T07:37:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.402+0000 7f8f4ce00640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f2807aa10 0x7f8f2807ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.403+0000 7f8f4ce00640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f2807aa10 0x7f8f2807ced0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8f4814f1c0 tx=0x7f8f38009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:02.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.405+0000 7f8f44c00640 1 -- 172.21.15.71:0/3655621294 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8f340639b0 con 0x7f8f480ccf00 2024-09-16T07:37:02.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.592+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f480b0d50 con 0x7f8f480ccf00 2024-09-16T07:37:02.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.593+0000 7f8f44c00640 1 -- 172.21.15.71:0/3655621294 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16944 (secure 0 0 0) 0x7f8f340632b0 con 0x7f8f480ccf00 2024-09-16T07:37:02.593 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:02.593 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":134,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:36:56.470220+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:36:02.011721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T07:36:03.220374+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":134,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:37:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f2807aa10 msgr2=0x7f8f2807ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f2807aa10 0x7f8f2807ced0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8f4814f1c0 tx=0x7f8f38009040 comp rx=0 tx=0).stop 2024-09-16T07:37:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480ccf00 msgr2=0x7f8f481486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480ccf00 0x7f8f481486f0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400c490 tx=0x7f8f3400c950 comp rx=0 tx=0).stop 2024-09-16T07:37:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 shutdown_connections 2024-09-16T07:37:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8f2807aa10 0x7f8f2807ced0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f4815f5a0 0x7f8f4809b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f480ccf00 0x7f8f481486f0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 --2- 172.21.15.71:0/3655621294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f480b2440 0x7f8f4809d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 >> 172.21.15.71:0/3655621294 conn(0x7f8f480b1950 msgr2=0x7f8f48073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 shutdown_connections 2024-09-16T07:37:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.595+0000 7f8f4f800640 1 -- 172.21.15.71:0/3655621294 wait complete. 2024-09-16T07:37:02.611 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:37:02.612 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:37:02.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 -- 172.21.15.71:0/3550794368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ad270 msgr2=0x7fc4d4073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 --2- 172.21.15.71:0/3550794368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ad270 0x7fc4d4073fb0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4075130 tx=0x7fc4d002a980 comp rx=0 tx=0).stop 2024-09-16T07:37:02.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 -- 172.21.15.71:0/3550794368 shutdown_connections 2024-09-16T07:37:02.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 --2- 172.21.15.71:0/3550794368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ad270 0x7fc4d4073fb0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 --2- 172.21.15.71:0/3550794368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4d40ccf00 0x7fc4d40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 --2- 172.21.15.71:0/3550794368 >> v2:172.21.15.177:3300/0 conn(0x7fc4d415f5a0 0x7fc4d40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 -- 172.21.15.71:0/3550794368 >> 172.21.15.71:0/3550794368 conn(0x7fc4d40b1950 msgr2=0x7fc4d40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:02.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 -- 172.21.15.71:0/3550794368 shutdown_connections 2024-09-16T07:37:02.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.769+0000 7fc4dc800640 1 -- 172.21.15.71:0/3550794368 wait complete. 2024-09-16T07:37:02.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.771+0000 7fc4dc800640 1 Processor -- start 2024-09-16T07:37:02.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.771+0000 7fc4dc800640 1 -- start start 2024-09-16T07:37:02.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4dc800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4d40ad270 0x7fc4d4154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:02.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4dc800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ccf00 0x7fc4d41f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:02.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4dc800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4d415f5a0 0x7fc4d415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:02.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4dc800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d406f160 con 0x7fc4d40ccf00 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4dc800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4074780 con 0x7fc4d40ad270 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4dc800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d406f2e0 con 0x7fc4d415f5a0 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4d9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ccf00 0x7fc4d41f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4d9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ccf00 0x7fc4d41f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59034/0 (socket says 172.21.15.71:59034) 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.772+0000 7fc4d9400640 1 -- 172.21.15.71:0/2290378946 learned_addr learned my addr 172.21.15.71:0/2290378946 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4d9e00640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4d40ad270 0x7fc4d4154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4da800640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4d415f5a0 0x7fc4d415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4d9400640 1 -- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4d415f5a0 msgr2=0x7fc4d415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4d9400640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4d415f5a0 0x7fc4d415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4d9400640 1 -- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4d40ad270 msgr2=0x7fc4d4154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4d9400640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4d40ad270 0x7fc4d4154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4d9400640 1 -- 172.21.15.71:0/2290378946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d41569e0 con 0x7fc4d40ccf00 2024-09-16T07:37:02.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4da800640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4d415f5a0 0x7fc4d415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:02.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4d9400640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ccf00 0x7fc4d41f1da0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400d780 tx=0x7fc4c400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:02.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4ce000640 1 -- 172.21.15.71:0/2290378946 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4017070 con 0x7fc4d40ccf00 2024-09-16T07:37:02.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4ce000640 1 -- 172.21.15.71:0/2290378946 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4c400de90 con 0x7fc4d40ccf00 2024-09-16T07:37:02.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4ce000640 1 -- 172.21.15.71:0/2290378946 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4005020 con 0x7fc4d40ccf00 2024-09-16T07:37:02.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d4156cc0 con 0x7fc4d40ccf00 2024-09-16T07:37:02.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.773+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d4157280 con 0x7fc4d40ccf00 2024-09-16T07:37:02.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.774+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d406f870 con 0x7fc4d40ccf00 2024-09-16T07:37:02.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.776+0000 7fc4ce000640 1 -- 172.21.15.71:0/2290378946 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fc4c40051c0 con 0x7fc4d40ccf00 2024-09-16T07:37:02.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.777+0000 7fc4ce000640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4b807aa10 0x7fc4b807ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:02.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.777+0000 7fc4ce000640 1 -- 172.21.15.71:0/2290378946 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5849+0+0 (secure 0 0 0) 0x7fc4c4099bb0 con 0x7fc4d40ccf00 2024-09-16T07:37:02.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.777+0000 7fc4d9e00640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4b807aa10 0x7fc4b807ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:02.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.778+0000 7fc4d9e00640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4b807aa10 0x7fc4b807ced0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8005fd0 tx=0x7fc4c8005850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:02.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.781+0000 7fc4ce000640 1 -- 172.21.15.71:0/2290378946 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc4c4063c80 con 0x7fc4d40ccf00 2024-09-16T07:37:02.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:02.999+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4d40b03f0 con 0x7fc4d40ccf00 2024-09-16T07:37:02.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.000+0000 7fc4ce000640 1 -- 172.21.15.71:0/2290378946 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5231 (secure 0 0 0) 0x7fc4c4063580 con 0x7fc4d40ccf00 2024-09-16T07:37:03.000 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":86,"btime":"2024-09-16T07:36:17:890799+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4275610322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":4275610322},{"type":"v1","addr":"172.21.15.71:6835","nonce":4275610322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/2559248159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2559248159},{"type":"v1","addr":"172.21.15.177:6836","nonce":2559248159}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/1515242906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1515242906},{"type":"v1","addr":"172.21.15.71:6834","nonce":1515242906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:36:04.253023+0000","modified":"2024-09-16T07:36:07.274434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2933333093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2933333093},{"type":"v1","addr":"172.21.15.177:6837","nonce":2933333093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":9}]} 2024-09-16T07:37:03.000 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 86 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4b807aa10 msgr2=0x7fc4b807ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4b807aa10 0x7fc4b807ced0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8005fd0 tx=0x7fc4c8005850 comp rx=0 tx=0).stop 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ccf00 msgr2=0x7fc4d41f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ccf00 0x7fc4d41f1da0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400d780 tx=0x7fc4c400dc40 comp rx=0 tx=0).stop 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 shutdown_connections 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc4b807aa10 0x7fc4b807ced0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4d415f5a0 0x7fc4d415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d40ccf00 0x7fc4d41f1da0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 --2- 172.21.15.71:0/2290378946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4d40ad270 0x7fc4d4154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 >> 172.21.15.71:0/2290378946 conn(0x7fc4d40b1950 msgr2=0x7fc4d409cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:03.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 shutdown_connections 2024-09-16T07:37:03.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.003+0000 7fc4dc800640 1 -- 172.21.15.71:0/2290378946 wait complete. 2024-09-16T07:37:03.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:37:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.175+0000 7f1456200640 1 -- 172.21.15.71:0/2029899070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14500ccfd0 msgr2=0x7f14500730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.175+0000 7f1456200640 1 --2- 172.21.15.71:0/2029899070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14500ccfd0 0x7f14500730b0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f1448009be0 tx=0x7f144802a8e0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.175+0000 7f1456200640 1 -- 172.21.15.71:0/2029899070 shutdown_connections 2024-09-16T07:37:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.175+0000 7f1456200640 1 --2- 172.21.15.71:0/2029899070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14500ccfd0 0x7f14500730b0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.175+0000 7f1456200640 1 -- 172.21.15.71:0/2029899070 >> 172.21.15.71:0/2029899070 conn(0x7f14500b1950 msgr2=0x7f14500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.175+0000 7f1456200640 1 -- 172.21.15.71:0/2029899070 shutdown_connections 2024-09-16T07:37:03.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.175+0000 7f1456200640 1 -- 172.21.15.71:0/2029899070 wait complete. 2024-09-16T07:37:03.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.177+0000 7f1456200640 1 Processor -- start 2024-09-16T07:37:03.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.177+0000 7f1456200640 1 -- start start 2024-09-16T07:37:03.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f1456200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14500ccfd0 0x7f1450141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f1456200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14500cf300 0x7f14501e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f1456200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14501445d0 0x7f14501e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f1456200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14500735f0 con 0x7f14501445d0 2024-09-16T07:37:03.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f1456200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f14500b4f60 con 0x7f14500ccfd0 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f1456200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1450073770 con 0x7f14500cf300 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f144fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14501445d0 0x7f14501e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f144fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14501445d0 0x7f14501e4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59044/0 (socket says 172.21.15.71:59044) 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f144fe00640 1 -- 172.21.15.71:0/2790735881 learned_addr learned my addr 172.21.15.71:0/2790735881 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f144f400640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14500ccfd0 0x7f1450141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f1447e00640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14500cf300 0x7f14501e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.178+0000 7f144fe00640 1 -- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14500cf300 msgr2=0x7f14501e47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f144fe00640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14500cf300 0x7f14501e47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f144fe00640 1 -- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14500ccfd0 msgr2=0x7f1450141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f144fe00640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14500ccfd0 0x7f1450141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f144fe00640 1 -- 172.21.15.71:0/2790735881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1450141670 con 0x7f14501445d0 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f144f400640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14500ccfd0 0x7f1450141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f1447e00640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14500cf300 0x7f14501e47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f144fe00640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14501445d0 0x7f14501e4cf0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f1448006010 tx=0x7f1448032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f1447400640 1 -- 172.21.15.71:0/2790735881 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448031430 con 0x7f14501445d0 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f1447400640 1 -- 172.21.15.71:0/2790735881 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1448004450 con 0x7f14501445d0 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f1447400640 1 -- 172.21.15.71:0/2790735881 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448004840 con 0x7f14501445d0 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14501e6c60 con 0x7f14501445d0 2024-09-16T07:37:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.179+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14501e7220 con 0x7f14501445d0 2024-09-16T07:37:03.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.180+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1450070a20 con 0x7f14501445d0 2024-09-16T07:37:03.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.181+0000 7f1447400640 1 -- 172.21.15.71:0/2790735881 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f1448043020 con 0x7f14501445d0 2024-09-16T07:37:03.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.182+0000 7f1447400640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f142c07aa40 0x7f142c07cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.182+0000 7f1447400640 1 -- 172.21.15.71:0/2790735881 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5849+0+0 (secure 0 0 0) 0x7f144802e030 con 0x7f14501445d0 2024-09-16T07:37:03.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.183+0000 7f144f400640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f142c07aa40 0x7f142c07cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.184+0000 7f144f400640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f142c07aa40 0x7f142c07cf00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f145009d280 tx=0x7f1440009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:03.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.186+0000 7f1447400640 1 -- 172.21.15.71:0/2790735881 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f144804a060 con 0x7f14501445d0 2024-09-16T07:37:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.410+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1450074110 con 0x7f14501445d0 2024-09-16T07:37:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.411+0000 7f1447400640 1 -- 172.21.15.71:0/2790735881 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5231 (secure 0 0 0) 0x7f1448085410 con 0x7f14501445d0 2024-09-16T07:37:03.410 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:03.411 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":86,"btime":"2024-09-16T07:36:17:890799+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4275610322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":4275610322},{"type":"v1","addr":"172.21.15.71:6835","nonce":4275610322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/2559248159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2559248159},{"type":"v1","addr":"172.21.15.177:6836","nonce":2559248159}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/1515242906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1515242906},{"type":"v1","addr":"172.21.15.71:6834","nonce":1515242906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:36:04.253023+0000","modified":"2024-09-16T07:36:07.274434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"d","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/2933333093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":2933333093},{"type":"v1","addr":"172.21.15.177:6837","nonce":2933333093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8919,"qdb_cluster":[8919]},"id":9}]} 2024-09-16T07:37:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 86 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f142c07aa40 msgr2=0x7f142c07cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f142c07aa40 0x7f142c07cf00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f145009d280 tx=0x7f1440009040 comp rx=0 tx=0).stop 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14501445d0 msgr2=0x7f14501e4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14501445d0 0x7f14501e4cf0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f1448006010 tx=0x7f1448032d10 comp rx=0 tx=0).stop 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 shutdown_connections 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f142c07aa40 0x7f142c07cf00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14501445d0 0x7f14501e4cf0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14500cf300 0x7f14501e47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 --2- 172.21.15.71:0/2790735881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14500ccfd0 0x7f1450141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 >> 172.21.15.71:0/2790735881 conn(0x7f14500b1950 msgr2=0x7f14500ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 shutdown_connections 2024-09-16T07:37:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.413+0000 7f1456200640 1 -- 172.21.15.71:0/2790735881 wait complete. 2024-09-16T07:37:03.430 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:37:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.588+0000 7fb42fe00640 1 -- 172.21.15.71:0/1426717207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300ce100 msgr2=0x7fb4300cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.588+0000 7fb42fe00640 1 --2- 172.21.15.71:0/1426717207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300ce100 0x7fb4300cc860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb420009860 tx=0x7fb42002a5a0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.588+0000 7fb42fe00640 1 -- 172.21.15.71:0/1426717207 shutdown_connections 2024-09-16T07:37:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.588+0000 7fb42fe00640 1 --2- 172.21.15.71:0/1426717207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300ce100 0x7fb4300cc860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.588+0000 7fb42fe00640 1 -- 172.21.15.71:0/1426717207 >> 172.21.15.71:0/1426717207 conn(0x7fb4300b1e90 msgr2=0x7fb4300b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.588+0000 7fb42fe00640 1 -- 172.21.15.71:0/1426717207 shutdown_connections 2024-09-16T07:37:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.588+0000 7fb42fe00640 1 -- 172.21.15.71:0/1426717207 wait complete. 2024-09-16T07:37:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.590+0000 7fb42fe00640 1 Processor -- start 2024-09-16T07:37:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.590+0000 7fb42fe00640 1 -- start start 2024-09-16T07:37:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb430074ca0 0x7fb430156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300afde0 0x7fb4301566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4300ce100 0x7fb430157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4300b1000 con 0x7fb430074ca0 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4300b0e80 con 0x7fb4300ce100 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb4301662b0 con 0x7fb4300afde0 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb430074ca0 0x7fb430156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb430074ca0 0x7fb430156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59046/0 (socket says 172.21.15.71:59046) 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42ea00640 1 -- 172.21.15.71:0/784308859 learned_addr learned my addr 172.21.15.71:0/784308859 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42f400640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4300ce100 0x7fb430157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42ea00640 1 -- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300afde0 msgr2=0x7fb4301566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42e000640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300afde0 0x7fb4301566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42ea00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300afde0 0x7fb4301566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42ea00640 1 -- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4300ce100 msgr2=0x7fb430157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42ea00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4300ce100 0x7fb430157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42ea00640 1 -- 172.21.15.71:0/784308859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4300ae530 con 0x7fb430074ca0 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42f400640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4300ce100 0x7fb430157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.591+0000 7fb42e000640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300afde0 0x7fb4301566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.592+0000 7fb42ea00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb430074ca0 0x7fb430156d50 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fb42002ab90 tx=0x7fb420032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.592+0000 7fb426a00640 1 -- 172.21.15.71:0/784308859 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420040070 con 0x7fb430074ca0 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.592+0000 7fb426a00640 1 -- 172.21.15.71:0/784308859 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4200038e0 con 0x7fb430074ca0 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.592+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4300ccb10 con 0x7fb430074ca0 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.592+0000 7fb426a00640 1 -- 172.21.15.71:0/784308859 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420003d30 con 0x7fb430074ca0 2024-09-16T07:37:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.592+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4300cd0d0 con 0x7fb430074ca0 2024-09-16T07:37:03.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.593+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4300cc860 con 0x7fb430074ca0 2024-09-16T07:37:03.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.595+0000 7fb426a00640 1 -- 172.21.15.71:0/784308859 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fb420039040 con 0x7fb430074ca0 2024-09-16T07:37:03.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.597+0000 7fb426a00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb41007aa10 0x7fb41007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.597+0000 7fb426a00640 1 -- 172.21.15.71:0/784308859 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5821+0+0 (secure 0 0 0) 0x7fb4200baed0 con 0x7fb430074ca0 2024-09-16T07:37:03.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.598+0000 7fb42e000640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb41007aa10 0x7fb41007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.600+0000 7fb42e000640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb41007aa10 0x7fb41007ced0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb4301f7c10 tx=0x7fb41c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:03.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.604+0000 7fb426a00640 1 -- 172.21.15.71:0/784308859 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb420084fa0 con 0x7fb430074ca0 2024-09-16T07:37:03.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.789+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4300aedb0 con 0x7fb430074ca0 2024-09-16T07:37:03.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.790+0000 7fb426a00640 1 -- 172.21.15.71:0/784308859 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+16899 (secure 0 0 0) 0x7fb4200848a0 con 0x7fb430074ca0 2024-09-16T07:37:03.789 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:03.790 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":135,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:37:03.263845+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:36:02.011721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T07:36:03.220374+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":134,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:37:03.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb41007aa10 msgr2=0x7fb41007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb41007aa10 0x7fb41007ced0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb4301f7c10 tx=0x7fb41c009040 comp rx=0 tx=0).stop 2024-09-16T07:37:03.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb430074ca0 msgr2=0x7fb430156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb430074ca0 0x7fb430156d50 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fb42002ab90 tx=0x7fb420032b90 comp rx=0 tx=0).stop 2024-09-16T07:37:03.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 shutdown_connections 2024-09-16T07:37:03.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb41007aa10 0x7fb41007ced0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4300ce100 0x7fb430157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4300afde0 0x7fb4301566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 --2- 172.21.15.71:0/784308859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb430074ca0 0x7fb430156d50 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 >> 172.21.15.71:0/784308859 conn(0x7fb4300b1e90 msgr2=0x7fb4300b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.792+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 shutdown_connections 2024-09-16T07:37:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.793+0000 7fb42fe00640 1 -- 172.21.15.71:0/784308859 wait complete. 2024-09-16T07:37:03.808 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:37:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.967+0000 7f64dac00640 1 -- 172.21.15.71:0/2300702071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 msgr2=0x7f64d40b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.967+0000 7f64dac00640 1 --2- 172.21.15.71:0/2300702071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 0x7f64d40b5540 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f64c8005fd0 tx=0x7f64c802a4b0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.967+0000 7f64dac00640 1 -- 172.21.15.71:0/2300702071 shutdown_connections 2024-09-16T07:37:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.967+0000 7f64dac00640 1 --2- 172.21.15.71:0/2300702071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 0x7f64d40b5540 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.967+0000 7f64dac00640 1 -- 172.21.15.71:0/2300702071 >> 172.21.15.71:0/2300702071 conn(0x7f64d40d3dc0 msgr2=0x7f64d40d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.967+0000 7f64dac00640 1 -- 172.21.15.71:0/2300702071 shutdown_connections 2024-09-16T07:37:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.967+0000 7f64dac00640 1 -- 172.21.15.71:0/2300702071 wait complete. 2024-09-16T07:37:03.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.968+0000 7f64dac00640 1 Processor -- start 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.968+0000 7f64dac00640 1 -- start start 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.969+0000 7f64dac00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d40b4aa0 0x7f64d4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.969+0000 7f64dac00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d40cff50 0x7f64d40cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.969+0000 7f64dac00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 0x7f64d40cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.969+0000 7f64dac00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f64d40b3f30 con 0x7f64d4161430 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.969+0000 7f64dac00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64d40b3db0 con 0x7f64d40cff50 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.969+0000 7f64dac00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64d40b40b0 con 0x7f64d40b4aa0 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 0x7f64d40cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 0x7f64d40cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59060/0 (socket says 172.21.15.71:59060) 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 -- 172.21.15.71:0/2312156969 learned_addr learned my addr 172.21.15.71:0/2312156969 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:03.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64d8e00640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d40cff50 0x7f64d40cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64d9800640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d40b4aa0 0x7f64d4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 -- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d40b4aa0 msgr2=0x7f64d4157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d40b4aa0 0x7f64d4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 -- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d40cff50 msgr2=0x7f64d40cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d40cff50 0x7f64d40cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 -- 172.21.15.71:0/2312156969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d40cc0f0 con 0x7f64d4161430 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64d8e00640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d40cff50 0x7f64d40cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.970+0000 7f64da200640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 0x7f64d40cccc0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00c7b0 tx=0x7f64cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:03.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.971+0000 7f64d1600640 1 -- 172.21.15.71:0/2312156969 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc004b60 con 0x7f64d4161430 2024-09-16T07:37:03.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.971+0000 7f64d1600640 1 -- 172.21.15.71:0/2312156969 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64cc004df0 con 0x7f64d4161430 2024-09-16T07:37:03.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.971+0000 7f64d1600640 1 -- 172.21.15.71:0/2312156969 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00aa90 con 0x7f64d4161430 2024-09-16T07:37:03.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.971+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d40cb570 con 0x7f64d4161430 2024-09-16T07:37:03.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.971+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d40cbb30 con 0x7f64d4161430 2024-09-16T07:37:03.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.972+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d40b4600 con 0x7f64d4161430 2024-09-16T07:37:03.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.973+0000 7f64d1600640 1 -- 172.21.15.71:0/2312156969 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f64cc00ac30 con 0x7f64d4161430 2024-09-16T07:37:03.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.974+0000 7f64d1600640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64b407aa10 0x7f64b407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:03.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.975+0000 7f64d1600640 1 -- 172.21.15.71:0/2312156969 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5821+0+0 (secure 0 0 0) 0x7f64cc099d40 con 0x7f64d4161430 2024-09-16T07:37:03.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.975+0000 7f64d9800640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64b407aa10 0x7f64b407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:03.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.976+0000 7f64d9800640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64b407aa10 0x7f64b407ced0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f64c8033000 tx=0x7f64c8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:03.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:03.979+0000 7f64d1600640 1 -- 172.21.15.71:0/2312156969 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f64cc020070 con 0x7f64d4161430 2024-09-16T07:37:04.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.199+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f64d40b5010 con 0x7f64d4161430 2024-09-16T07:37:04.199 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:37:04.200+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:37:04.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.202+0000 7f64d1600640 1 -- 172.21.15.71:0/2312156969 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) ==== 161+0+0 (secure 0 0 0) 0x7f64cc063e10 con 0x7f64d4161430 2024-09-16T07:37:04.201 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64b407aa10 msgr2=0x7f64b407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64b407aa10 0x7f64b407ced0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f64c8033000 tx=0x7f64c8004730 comp rx=0 tx=0).stop 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 msgr2=0x7f64d40cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 0x7f64d40cccc0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00c7b0 tx=0x7f64cc00cc70 comp rx=0 tx=0).stop 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 shutdown_connections 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f64b407aa10 0x7f64b407ced0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64d4161430 0x7f64d40cccc0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64d40cff50 0x7f64d40cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 --2- 172.21.15.71:0/2312156969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d40b4aa0 0x7f64d4157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 >> 172.21.15.71:0/2312156969 conn(0x7f64d40d3dc0 msgr2=0x7f64d40b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 shutdown_connections 2024-09-16T07:37:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.205+0000 7f64dac00640 1 -- 172.21.15.71:0/2312156969 wait complete. 2024-09-16T07:37:04.221 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:37:04.273 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:37:04.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.382+0000 7fa12ec00640 1 -- 172.21.15.71:0/391329120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c004800 msgr2=0x7fa10c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:04.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.382+0000 7fa12ec00640 1 --2- 172.21.15.71:0/391329120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c004800 0x7fa10c004c80 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c260 tx=0x7fa10c005830 comp rx=0 tx=0).stop 2024-09-16T07:37:04.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.382+0000 7fa12ec00640 1 -- 172.21.15.71:0/391329120 shutdown_connections 2024-09-16T07:37:04.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.382+0000 7fa12ec00640 1 --2- 172.21.15.71:0/391329120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c004800 0x7fa10c004c80 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:04.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.382+0000 7fa12ec00640 1 -- 172.21.15.71:0/391329120 >> 172.21.15.71:0/391329120 conn(0x7fa1280b1840 msgr2=0x7fa1280b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:04.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.382+0000 7fa12ec00640 1 -- 172.21.15.71:0/391329120 shutdown_connections 2024-09-16T07:37:04.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.382+0000 7fa12ec00640 1 -- 172.21.15.71:0/391329120 wait complete. 2024-09-16T07:37:04.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.384+0000 7fa12ec00640 1 Processor -- start 2024-09-16T07:37:04.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.384+0000 7fa12ec00640 1 -- start start 2024-09-16T07:37:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa12ec00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa10c004800 0x7fa128146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa12ec00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1281f5710 0x7fa1280d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa12ec00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1281603f0 0x7fa1281608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa12ec00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa1280ce620 con 0x7fa1281f5710 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa12ec00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1280ce4a0 con 0x7fa10c004800 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa12ec00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1280ce7a0 con 0x7fa1281603f0 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1281f5710 0x7fa1280d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1281f5710 0x7fa1280d0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59076/0 (socket says 172.21.15.71:59076) 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127400640 1 -- 172.21.15.71:0/2209179802 learned_addr learned my addr 172.21.15.71:0/2209179802 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127e00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa10c004800 0x7fa128146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa12cc00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1281603f0 0x7fa1281608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127400640 1 -- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1281603f0 msgr2=0x7fa1281608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127400640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1281603f0 0x7fa1281608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127400640 1 -- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa10c004800 msgr2=0x7fa128146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127400640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa10c004800 0x7fa128146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.385+0000 7fa127400640 1 -- 172.21.15.71:0/2209179802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1281f5b90 con 0x7fa1281f5710 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.386+0000 7fa127400640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1281f5710 0x7fa1280d0060 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c380 tx=0x7fa114001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.386+0000 7fa11fe00640 1 -- 172.21.15.71:0/2209179802 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114031040 con 0x7fa1281f5710 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.386+0000 7fa11fe00640 1 -- 172.21.15.71:0/2209179802 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa114003610 con 0x7fa1281f5710 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.386+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa128147370 con 0x7fa1281f5710 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.386+0000 7fa11fe00640 1 -- 172.21.15.71:0/2209179802 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114003a60 con 0x7fa1281f5710 2024-09-16T07:37:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.386+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa128147930 con 0x7fa1281f5710 2024-09-16T07:37:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.387+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12815fcc0 con 0x7fa1281f5710 2024-09-16T07:37:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.389+0000 7fa11fe00640 1 -- 172.21.15.71:0/2209179802 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fa114034070 con 0x7fa1281f5710 2024-09-16T07:37:04.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.390+0000 7fa11fe00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa10807aa40 0x7fa10807cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:04.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.390+0000 7fa11fe00640 1 -- 172.21.15.71:0/2209179802 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5879+0+0 (secure 0 0 0) 0x7fa1140b99b0 con 0x7fa1281f5710 2024-09-16T07:37:04.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.390+0000 7fa127e00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa10807aa40 0x7fa10807cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:04.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.391+0000 7fa127e00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa10807aa40 0x7fa10807cf00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa1180097f0 tx=0x7fa118009500 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:04.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.394+0000 7fa11fe00640 1 -- 172.21.15.71:0/2209179802 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa114083b00 con 0x7fa1281f5710 2024-09-16T07:37:04.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:04.617+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa1280ae3f0 con 0x7fa1281f5710 2024-09-16T07:37:05.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.204+0000 7fa11fe00640 1 -- 172.21.15.71:0/2209179802 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) ==== 104+0+0 (secure 0 0 0) 0x7fa114083400 con 0x7fa1281f5710 2024-09-16T07:37:05.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa10807aa40 msgr2=0x7fa10807cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:05.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa10807aa40 0x7fa10807cf00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa1180097f0 tx=0x7fa118009500 comp rx=0 tx=0).stop 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1281f5710 msgr2=0x7fa1280d0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1281f5710 0x7fa1280d0060 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fa11400c380 tx=0x7fa114001cd0 comp rx=0 tx=0).stop 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 shutdown_connections 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa10807aa40 0x7fa10807cf00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1281603f0 0x7fa1281608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1281f5710 0x7fa1280d0060 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 --2- 172.21.15.71:0/2209179802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa10c004800 0x7fa128146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 >> 172.21.15.71:0/2209179802 conn(0x7fa1280b1840 msgr2=0x7fa1280b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 shutdown_connections 2024-09-16T07:37:05.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.207+0000 7fa12ec00640 1 -- 172.21.15.71:0/2209179802 wait complete. 2024-09-16T07:37:05.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:37:05.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.388+0000 7f3924800640 1 -- 172.21.15.71:0/3753379004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3904003fd0 msgr2=0x7f3904004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:05.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.388+0000 7f3924800640 1 --2- 172.21.15.71:0/3753379004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3904003fd0 0x7f3904004800 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f390c00ac30 tx=0x7f3904005830 comp rx=0 tx=0).stop 2024-09-16T07:37:05.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.388+0000 7f3922800640 1 -- 172.21.15.71:0/3753379004 reap_dead start 2024-09-16T07:37:05.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.388+0000 7f3924800640 1 -- 172.21.15.71:0/3753379004 shutdown_connections 2024-09-16T07:37:05.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.388+0000 7f3924800640 1 -- 172.21.15.71:0/3753379004 >> 172.21.15.71:0/3753379004 conn(0x7f391c0b18e0 msgr2=0x7f391c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.388+0000 7f3924800640 1 -- 172.21.15.71:0/3753379004 shutdown_connections 2024-09-16T07:37:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.388+0000 7f3924800640 1 -- 172.21.15.71:0/3753379004 wait complete. 2024-09-16T07:37:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.389+0000 7f3924800640 1 Processor -- start 2024-09-16T07:37:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.390+0000 7f3924800640 1 -- start start 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.390+0000 7f3924800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3904003fd0 0x7f391c1446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.390+0000 7f3924800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f391c0c9a90 0x7f391c1f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.390+0000 7f3924800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f391c0cb230 0x7f391c0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.390+0000 7f3924800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f391c0b43a0 con 0x7f391c0cb230 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.390+0000 7f3924800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f391c073c40 con 0x7f391c0c9a90 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.390+0000 7f3924800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f391c0b4520 con 0x7f3904003fd0 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f391c0cb230 0x7f391c0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f391c0cb230 0x7f391c0cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33448/0 (socket says 172.21.15.71:33448) 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 -- 172.21.15.71:0/3420190550 learned_addr learned my addr 172.21.15.71:0/3420190550 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3921400640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f391c0c9a90 0x7f391c1f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3921e00640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3904003fd0 0x7f391c1446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 -- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3904003fd0 msgr2=0x7f391c1446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3904003fd0 0x7f391c1446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 -- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f391c0c9a90 msgr2=0x7f391c1f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f391c0c9a90 0x7f391c1f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 -- 172.21.15.71:0/3420190550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c1f9080 con 0x7f391c0cb230 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.391+0000 7f3922800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f391c0cb230 0x7f391c0cb6b0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f391c0cd7d0 tx=0x7f391800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.392+0000 7f3916000640 1 -- 172.21.15.71:0/3420190550 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918015020 con 0x7f391c0cb230 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.392+0000 7f3916000640 1 -- 172.21.15.71:0/3420190550 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f391800cc50 con 0x7f391c0cb230 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.392+0000 7f3916000640 1 -- 172.21.15.71:0/3420190550 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391800ee60 con 0x7f391c0cb230 2024-09-16T07:37:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.392+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c1f8ac0 con 0x7f391c0cb230 2024-09-16T07:37:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.392+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c15b150 con 0x7f391c0cb230 2024-09-16T07:37:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.393+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391c069860 con 0x7f391c0cb230 2024-09-16T07:37:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.394+0000 7f3916000640 1 -- 172.21.15.71:0/3420190550 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f3918018020 con 0x7f391c0cb230 2024-09-16T07:37:05.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.396+0000 7f3916000640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f390007aa40 0x7f390007cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:05.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.396+0000 7f3916000640 1 -- 172.21.15.71:0/3420190550 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5879+0+0 (secure 0 0 0) 0x7f3918099cc0 con 0x7f391c0cb230 2024-09-16T07:37:05.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.396+0000 7f3921e00640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f390007aa40 0x7f390007cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:05.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.397+0000 7f3921e00640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f390007aa40 0x7f390007cf00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3910009ab0 tx=0x7f3910005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.400+0000 7f3916000640 1 -- 172.21.15.71:0/3420190550 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f391801f030 con 0x7f391c0cb230 2024-09-16T07:37:05.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:05.585+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f391c069360 con 0x7f391c0cb230 2024-09-16T07:37:06.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.510+0000 7f3916000640 1 -- 172.21.15.71:0/3420190550 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v137) ==== 181+0+0 (secure 0 0 0) 0x7f3918063e30 con 0x7f391c0cb230 2024-09-16T07:37:06.509 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f390007aa40 msgr2=0x7f390007cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f390007aa40 0x7f390007cf00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3910009ab0 tx=0x7f3910005f90 comp rx=0 tx=0).stop 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f391c0cb230 msgr2=0x7f391c0cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f391c0cb230 0x7f391c0cb6b0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f391c0cd7d0 tx=0x7f391800ed30 comp rx=0 tx=0).stop 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 shutdown_connections 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f390007aa40 0x7f390007cf00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f391c0cb230 0x7f391c0cb6b0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f391c0c9a90 0x7f391c1f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 --2- 172.21.15.71:0/3420190550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3904003fd0 0x7f391c1446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 >> 172.21.15.71:0/3420190550 conn(0x7f391c0b18e0 msgr2=0x7f391c0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 shutdown_connections 2024-09-16T07:37:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.513+0000 7f3924800640 1 -- 172.21.15.71:0/3420190550 wait complete. 2024-09-16T07:37:06.533 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:37:06.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.694+0000 7f18ece00640 1 -- 172.21.15.71:0/284549872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 msgr2=0x7f18e815e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:06.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.694+0000 7f18ece00640 1 --2- 172.21.15.71:0/284549872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 0x7f18e815e490 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f18e80cfba0 tx=0x7f18e002a8b0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.694+0000 7f18ece00640 1 -- 172.21.15.71:0/284549872 shutdown_connections 2024-09-16T07:37:06.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.694+0000 7f18ece00640 1 --2- 172.21.15.71:0/284549872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 0x7f18e815e490 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.694+0000 7f18ece00640 1 --2- 172.21.15.71:0/284549872 >> v2:172.21.15.177:3300/0 conn(0x7f18e80b4350 0x7f18e806f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.694+0000 7f18ece00640 1 -- 172.21.15.71:0/284549872 >> 172.21.15.71:0/284549872 conn(0x7f18e80b18e0 msgr2=0x7f18e80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:06.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.694+0000 7f18ece00640 1 -- 172.21.15.71:0/284549872 shutdown_connections 2024-09-16T07:37:06.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.694+0000 7f18ece00640 1 -- 172.21.15.71:0/284549872 wait complete. 2024-09-16T07:37:06.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.696+0000 7f18ece00640 1 Processor -- start 2024-09-16T07:37:06.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.696+0000 7f18ece00640 1 -- start start 2024-09-16T07:37:06.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18ece00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 0x7f18e8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:06.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18ece00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80b4350 0x7f18e80cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:06.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18ece00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e81550c0 0x7f18e81f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:06.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18ece00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18e815eb50 con 0x7f18e806f920 2024-09-16T07:37:06.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18ece00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18e815e9d0 con 0x7f18e80b4350 2024-09-16T07:37:06.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18ece00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f18e815ecd0 con 0x7f18e81550c0 2024-09-16T07:37:06.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18e5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 0x7f18e8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:06.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18e5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 0x7f18e8157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33464/0 (socket says 172.21.15.71:33464) 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.697+0000 7f18e5e00640 1 -- 172.21.15.71:0/2008088027 learned_addr learned my addr 172.21.15.71:0/2008088027 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18e5400640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80b4350 0x7f18e80cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18e6800640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e81550c0 0x7f18e81f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18e5e00640 1 -- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e81550c0 msgr2=0x7f18e81f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18e5e00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e81550c0 0x7f18e81f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18e5e00640 1 -- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80b4350 msgr2=0x7f18e80cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18e5e00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80b4350 0x7f18e80cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18e5e00640 1 -- 172.21.15.71:0/2008088027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e8158690 con 0x7f18e806f920 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18e5e00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 0x7f18e8157f70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c430 tx=0x7f18d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18de000640 1 -- 172.21.15.71:0/2008088027 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8011790 con 0x7f18e806f920 2024-09-16T07:37:06.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18de000640 1 -- 172.21.15.71:0/2008088027 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18d8017070 con 0x7f18e806f920 2024-09-16T07:37:06.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18de000640 1 -- 172.21.15.71:0/2008088027 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8011ce0 con 0x7f18e806f920 2024-09-16T07:37:06.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e80ce010 con 0x7f18e806f920 2024-09-16T07:37:06.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.698+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e80d0080 con 0x7f18e806f920 2024-09-16T07:37:06.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.700+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e8069360 con 0x7f18e806f920 2024-09-16T07:37:06.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.701+0000 7f18de000640 1 -- 172.21.15.71:0/2008088027 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f18d801c460 con 0x7f18e806f920 2024-09-16T07:37:06.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.702+0000 7f18de000640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c807aa10 0x7f18c807ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:06.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.702+0000 7f18de000640 1 -- 172.21.15.71:0/2008088027 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5441+0+0 (secure 0 0 0) 0x7f18d8099850 con 0x7f18e806f920 2024-09-16T07:37:06.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.702+0000 7f18e5400640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c807aa10 0x7f18c807ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:06.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.703+0000 7f18e5400640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c807aa10 0x7f18c807ced0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f18e80cd230 tx=0x7f18d400b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:06.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.706+0000 7f18de000640 1 -- 172.21.15.71:0/2008088027 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f18d8063920 con 0x7f18e806f920 2024-09-16T07:37:06.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:06.891+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f18e806f3f0 con 0x7f18e806f920 2024-09-16T07:37:07.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.522+0000 7f18de000640 1 -- 172.21.15.71:0/2008088027 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v138) ==== 169+0+0 (secure 0 0 0) 0x7f18d8063220 con 0x7f18e806f920 2024-09-16T07:37:07.522 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:37:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.526+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c807aa10 msgr2=0x7f18c807ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.526+0000 7f18ece00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c807aa10 0x7f18c807ced0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f18e80cd230 tx=0x7f18d400b390 comp rx=0 tx=0).stop 2024-09-16T07:37:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.526+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 msgr2=0x7f18e8157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.526+0000 7f18ece00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 0x7f18e8157f70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c430 tx=0x7f18d800c8f0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.527+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 shutdown_connections 2024-09-16T07:37:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.527+0000 7f18ece00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f18c807aa10 0x7f18c807ced0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.527+0000 7f18ece00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e81550c0 0x7f18e81f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.527+0000 7f18ece00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e80b4350 0x7f18e80cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.527+0000 7f18ece00640 1 --2- 172.21.15.71:0/2008088027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18e806f920 0x7f18e8157f70 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.527+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 >> 172.21.15.71:0/2008088027 conn(0x7f18e80b18e0 msgr2=0x7f18e80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.527+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 shutdown_connections 2024-09-16T07:37:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.527+0000 7f18ece00640 1 -- 172.21.15.71:0/2008088027 wait complete. 2024-09-16T07:37:07.545 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:37:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.704+0000 7fa8e8000640 1 -- 172.21.15.71:0/503410708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80041c0 msgr2=0x7fa8c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.704+0000 7fa8e8000640 1 --2- 172.21.15.71:0/503410708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80041c0 0x7fa8c8004800 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009b50 tx=0x7fa8c8005830 comp rx=0 tx=0).stop 2024-09-16T07:37:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.704+0000 7fa8e6000640 1 -- 172.21.15.71:0/503410708 reap_dead start 2024-09-16T07:37:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.704+0000 7fa8e8000640 1 -- 172.21.15.71:0/503410708 shutdown_connections 2024-09-16T07:37:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.704+0000 7fa8e8000640 1 -- 172.21.15.71:0/503410708 >> 172.21.15.71:0/503410708 conn(0x7fa8e00b1950 msgr2=0x7fa8e00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:07.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.704+0000 7fa8e8000640 1 -- 172.21.15.71:0/503410708 shutdown_connections 2024-09-16T07:37:07.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.704+0000 7fa8e8000640 1 -- 172.21.15.71:0/503410708 wait complete. 2024-09-16T07:37:07.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.706+0000 7fa8e8000640 1 Processor -- start 2024-09-16T07:37:07.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.706+0000 7fa8e8000640 1 -- start start 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e8000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c80041c0 0x7fa8e01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e8000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e01f2080 0x7fa8e01f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e8000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8e01f5490 0x7fa8e0128ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e8000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e006f9f0 con 0x7fa8e01f2080 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e8000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e006f870 con 0x7fa8c80041c0 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e8000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e00b2440 con 0x7fa8e01f5490 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e01f2080 0x7fa8e01f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e01f2080 0x7fa8e01f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33474/0 (socket says 172.21.15.71:33474) 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e4c00640 1 -- 172.21.15.71:0/986048194 learned_addr learned my addr 172.21.15.71:0/986048194 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e5600640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c80041c0 0x7fa8e01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:07.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.707+0000 7fa8e6000640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8e01f5490 0x7fa8e0128ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e4c00640 1 -- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8e01f5490 msgr2=0x7fa8e0128ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e4c00640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8e01f5490 0x7fa8e0128ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e4c00640 1 -- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c80041c0 msgr2=0x7fa8e01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e4c00640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c80041c0 0x7fa8e01562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e4c00640 1 -- 172.21.15.71:0/986048194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e01f5950 con 0x7fa8e01f2080 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e6000640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8e01f5490 0x7fa8e0128ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e4c00640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e01f2080 0x7fa8e01f4f50 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc036040 tx=0x7fa8cc033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8dd600640 1 -- 172.21.15.71:0/986048194 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc038040 con 0x7fa8e01f2080 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8dd600640 1 -- 172.21.15.71:0/986048194 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8cc041070 con 0x7fa8e01f2080 2024-09-16T07:37:07.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8dd600640 1 -- 172.21.15.71:0/986048194 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc033e10 con 0x7fa8e01f2080 2024-09-16T07:37:07.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e01f5c30 con 0x7fa8e01f2080 2024-09-16T07:37:07.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.708+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e01f61f0 con 0x7fa8e01f2080 2024-09-16T07:37:07.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.710+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e0074440 con 0x7fa8e01f2080 2024-09-16T07:37:07.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.711+0000 7fa8dd600640 1 -- 172.21.15.71:0/986048194 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fa8cc046460 con 0x7fa8e01f2080 2024-09-16T07:37:07.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.712+0000 7fa8dd600640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8c007aa10 0x7fa8c007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:07.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.712+0000 7fa8dd600640 1 -- 172.21.15.71:0/986048194 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7fa8cc0b9fd0 con 0x7fa8e01f2080 2024-09-16T07:37:07.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.713+0000 7fa8e5600640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8c007aa10 0x7fa8c007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:07.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.713+0000 7fa8e5600640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8c007aa10 0x7fa8c007ced0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0070020 tx=0x7fa8d4006b10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:07.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.716+0000 7fa8dd600640 1 -- 172.21.15.71:0/986048194 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8cc04b020 con 0x7fa8e01f2080 2024-09-16T07:37:07.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.900+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa8e0156800 con 0x7fa8e01f2080 2024-09-16T07:37:07.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.904+0000 7fa8dd600640 1 -- 172.21.15.71:0/986048194 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v53) ==== 71+0+0 (secure 0 0 0) 0x7fa8cc083a50 con 0x7fa8e01f2080 2024-09-16T07:37:07.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8c007aa10 msgr2=0x7fa8c007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:07.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8c007aa10 0x7fa8c007ced0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0070020 tx=0x7fa8d4006b10 comp rx=0 tx=0).stop 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e01f2080 msgr2=0x7fa8e01f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e01f2080 0x7fa8e01f4f50 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc036040 tx=0x7fa8cc033670 comp rx=0 tx=0).stop 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 shutdown_connections 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa8c007aa10 0x7fa8c007ced0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8e01f5490 0x7fa8e0128ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e01f2080 0x7fa8e01f4f50 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.906+0000 7fa8e8000640 1 --2- 172.21.15.71:0/986048194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c80041c0 0x7fa8e01562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.907+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 >> 172.21.15.71:0/986048194 conn(0x7fa8e00b1950 msgr2=0x7fa8e00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.907+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 shutdown_connections 2024-09-16T07:37:07.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:07.907+0000 7fa8e8000640 1 -- 172.21.15.71:0/986048194 wait complete. 2024-09-16T07:37:07.923 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-16T07:37:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.081+0000 7f88bc000640 1 -- 172.21.15.71:0/14370209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f889c003a00 msgr2=0x7f889c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.081+0000 7f88bc000640 1 --2- 172.21.15.71:0/14370209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f889c003a00 0x7f889c003e20 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009ed0 tx=0x7f889c005830 comp rx=0 tx=0).stop 2024-09-16T07:37:08.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.081+0000 7f88ba000640 1 -- 172.21.15.71:0/14370209 reap_dead start 2024-09-16T07:37:08.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.081+0000 7f88bc000640 1 -- 172.21.15.71:0/14370209 shutdown_connections 2024-09-16T07:37:08.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.081+0000 7f88bc000640 1 -- 172.21.15.71:0/14370209 >> 172.21.15.71:0/14370209 conn(0x7f88b40b1950 msgr2=0x7f88b40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:08.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.081+0000 7f88bc000640 1 -- 172.21.15.71:0/14370209 shutdown_connections 2024-09-16T07:37:08.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.081+0000 7f88bc000640 1 -- 172.21.15.71:0/14370209 wait complete. 2024-09-16T07:37:08.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.083+0000 7f88bc000640 1 Processor -- start 2024-09-16T07:37:08.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.083+0000 7f88bc000640 1 -- start start 2024-09-16T07:37:08.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88bc000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f889c003a00 0x7f88b414b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:08.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88bc000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b4154d80 0x7f88b4159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:08.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88bc000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b4159db0 0x7f88b415b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:08.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88bc000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88b40b25c0 con 0x7f88b4159db0 2024-09-16T07:37:08.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88bc000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88b40b2440 con 0x7f88b4154d80 2024-09-16T07:37:08.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88bc000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f88b40b2740 con 0x7f889c003a00 2024-09-16T07:37:08.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88ba000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b4159db0 0x7f88b415b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:08.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88ba000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b4159db0 0x7f88b415b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33488/0 (socket says 172.21.15.71:33488) 2024-09-16T07:37:08.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88ba000640 1 -- 172.21.15.71:0/1572181098 learned_addr learned my addr 172.21.15.71:0/1572181098 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:08.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88b8c00640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b4154d80 0x7f88b4159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:08.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88b9600640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f889c003a00 0x7f88b414b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:08.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88ba000640 1 -- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f889c003a00 msgr2=0x7f88b414b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88ba000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f889c003a00 0x7f88b414b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88ba000640 1 -- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b4154d80 msgr2=0x7f88b4159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88ba000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b4154d80 0x7f88b4159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.084+0000 7f88ba000640 1 -- 172.21.15.71:0/1572181098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b414bbd0 con 0x7f88b4159db0 2024-09-16T07:37:08.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.085+0000 7f88ba000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b4159db0 0x7f88b415b430 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f88b4075130 tx=0x7f88ac011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:08.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.085+0000 7f88b1600640 1 -- 172.21.15.71:0/1572181098 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac004440 con 0x7f88b4159db0 2024-09-16T07:37:08.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.085+0000 7f88b1600640 1 -- 172.21.15.71:0/1572181098 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88ac016070 con 0x7f88b4159db0 2024-09-16T07:37:08.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.085+0000 7f88b1600640 1 -- 172.21.15.71:0/1572181098 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac005280 con 0x7f88b4159db0 2024-09-16T07:37:08.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.085+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b415c190 con 0x7f88b4159db0 2024-09-16T07:37:08.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.085+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b415c750 con 0x7f88b4159db0 2024-09-16T07:37:08.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.086+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b4074460 con 0x7f88b4159db0 2024-09-16T07:37:08.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.087+0000 7f88b1600640 1 -- 172.21.15.71:0/1572181098 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f88ac005420 con 0x7f88b4159db0 2024-09-16T07:37:08.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.088+0000 7f88b1600640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f889407aa10 0x7f889407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:08.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.088+0000 7f88b1600640 1 -- 172.21.15.71:0/1572181098 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7f88ac099fb0 con 0x7f88b4159db0 2024-09-16T07:37:08.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.089+0000 7f88b9600640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f889407aa10 0x7f889407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:08.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.089+0000 7f88b9600640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f889407aa10 0x7f889407ced0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f88a8035000 tx=0x7f88a802d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:08.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.092+0000 7f88b1600640 1 -- 172.21.15.71:0/1572181098 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f88ac063a30 con 0x7f88b4159db0 2024-09-16T07:37:08.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.354+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7f88b4156d30 con 0x7f88b4159db0 2024-09-16T07:37:08.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.524+0000 7f88b1600640 1 -- 172.21.15.71:0/1572181098 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v671) ==== 186+0+0 (secure 0 0 0) 0x7f88ac068460 con 0x7f88b4159db0 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f889407aa10 msgr2=0x7f889407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f889407aa10 0x7f889407ced0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f88a8035000 tx=0x7f88a802d040 comp rx=0 tx=0).stop 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b4159db0 msgr2=0x7f88b415b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b4159db0 0x7f88b415b430 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f88b4075130 tx=0x7f88ac011e10 comp rx=0 tx=0).stop 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 shutdown_connections 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f889407aa10 0x7f889407ced0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88b4159db0 0x7f88b415b430 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88b4154d80 0x7f88b4159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 --2- 172.21.15.71:0/1572181098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f889c003a00 0x7f88b414b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 >> 172.21.15.71:0/1572181098 conn(0x7f88b40b1950 msgr2=0x7f88b406f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 shutdown_connections 2024-09-16T07:37:08.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.527+0000 7f88bc000640 1 -- 172.21.15.71:0/1572181098 wait complete. 2024-09-16T07:37:08.544 INFO:tasks.cephfs_test_runner:Test that the progress bar that is printed for 1 ongoing clone job is ... ok 2024-09-16T07:37:08.544 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-09-16T07:37:08.544 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T07:37:08.545 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:37:08.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.701+0000 7fb026a00640 1 -- 172.21.15.71:0/2952290855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0200d0540 msgr2=0x7fb02006f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.701+0000 7fb026a00640 1 --2- 172.21.15.71:0/2952290855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0200d0540 0x7fb02006f320 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb020073b00 tx=0x7fb01802a8b0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.701+0000 7fb026a00640 1 -- 172.21.15.71:0/2952290855 shutdown_connections 2024-09-16T07:37:08.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.701+0000 7fb026a00640 1 --2- 172.21.15.71:0/2952290855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0200d0540 0x7fb02006f320 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.701+0000 7fb026a00640 1 --2- 172.21.15.71:0/2952290855 >> v2:172.21.15.177:3300/0 conn(0x7fb0200725d0 0x7fb0200b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.701+0000 7fb026a00640 1 -- 172.21.15.71:0/2952290855 >> 172.21.15.71:0/2952290855 conn(0x7fb0200cfa20 msgr2=0x7fb0200cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:08.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.701+0000 7fb026a00640 1 -- 172.21.15.71:0/2952290855 shutdown_connections 2024-09-16T07:37:08.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.701+0000 7fb026a00640 1 -- 172.21.15.71:0/2952290855 wait complete. 2024-09-16T07:37:08.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.703+0000 7fb026a00640 1 Processor -- start 2024-09-16T07:37:08.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.703+0000 7fb026a00640 1 -- start start 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0200d0540 0x7fb0201515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02015bc30 0x7fb020157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0201f8b80 0x7fb020158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb020074660 con 0x7fb0201f8b80 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb02006f860 con 0x7fb0200d0540 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0201478d0 con 0x7fb02015bc30 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0201f8b80 0x7fb020158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0201f8b80 0x7fb020158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33500/0 (socket says 172.21.15.71:33500) 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb026000640 1 -- 172.21.15.71:0/369567325 learned_addr learned my addr 172.21.15.71:0/369567325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:08.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb024c00640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02015bc30 0x7fb020157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.704+0000 7fb025600640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0200d0540 0x7fb0201515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb026000640 1 -- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02015bc30 msgr2=0x7fb020157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb026000640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02015bc30 0x7fb020157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb026000640 1 -- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0200d0540 msgr2=0x7fb0201515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb026000640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0200d0540 0x7fb0201515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb026000640 1 -- 172.21.15.71:0/369567325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0200b13e0 con 0x7fb0201f8b80 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb025600640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0200d0540 0x7fb0201515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb024c00640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02015bc30 0x7fb020157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb026000640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0201f8b80 0x7fb020158080 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb01802aea0 tx=0x7fb0180039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb01d600640 1 -- 172.21.15.71:0/369567325 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018040070 con 0x7fb0201f8b80 2024-09-16T07:37:08.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb01d600640 1 -- 172.21.15.71:0/369567325 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb018004150 con 0x7fb0201f8b80 2024-09-16T07:37:08.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb01d600640 1 -- 172.21.15.71:0/369567325 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0180045a0 con 0x7fb0201f8b80 2024-09-16T07:37:08.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.705+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb020151ec0 con 0x7fb0201f8b80 2024-09-16T07:37:08.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.706+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb020147e00 con 0x7fb0201f8b80 2024-09-16T07:37:08.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.707+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02006fdf0 con 0x7fb0201f8b80 2024-09-16T07:37:08.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.708+0000 7fb01d600640 1 -- 172.21.15.71:0/369567325 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fb018039040 con 0x7fb0201f8b80 2024-09-16T07:37:08.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.709+0000 7fb01d600640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00007aa10 0x7fb00007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:08.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.709+0000 7fb01d600640 1 -- 172.21.15.71:0/369567325 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7fb0180bb260 con 0x7fb0201f8b80 2024-09-16T07:37:08.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.709+0000 7fb025600640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00007aa10 0x7fb00007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:08.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.710+0000 7fb025600640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00007aa10 0x7fb00007ced0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb0140016a0 tx=0x7fb014009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.713+0000 7fb01d600640 1 -- 172.21.15.71:0/369567325 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb018085330 con 0x7fb0201f8b80 2024-09-16T07:37:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.900+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb02015b200 con 0x7fb0201f8b80 2024-09-16T07:37:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.900+0000 7fb01d600640 1 -- 172.21.15.71:0/369567325 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+13189 (secure 0 0 0) 0x7fb018084c30 con 0x7fb0201f8b80 2024-09-16T07:37:08.900 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:08.900 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":138,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:37:07.516744+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6837/2933333093":"2024-09-17T07:37:04.200966+0000","172.21.15.177:6835/2933333093":"2024-09-17T07:37:04.200966+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00007aa10 msgr2=0x7fb00007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00007aa10 0x7fb00007ced0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb0140016a0 tx=0x7fb014009040 comp rx=0 tx=0).stop 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0201f8b80 msgr2=0x7fb020158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0201f8b80 0x7fb020158080 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb01802aea0 tx=0x7fb0180039f0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 shutdown_connections 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb00007aa10 0x7fb00007ced0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0201f8b80 0x7fb020158080 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb02015bc30 0x7fb020157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 --2- 172.21.15.71:0/369567325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0200d0540 0x7fb0201515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:08.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 >> 172.21.15.71:0/369567325 conn(0x7fb0200cfa20 msgr2=0x7fb02015a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:08.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 shutdown_connections 2024-09-16T07:37:08.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:08.903+0000 7fb026a00640 1 -- 172.21.15.71:0/369567325 wait complete. 2024-09-16T07:37:08.918 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T07:22:38.319708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T07:37:08.919 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T07:37:09.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.078+0000 7fddf8400640 1 -- 172.21.15.71:0/1384261499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf00b3370 msgr2=0x7fddf00cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:09.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.078+0000 7fddf8400640 1 --2- 172.21.15.71:0/1384261499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf00b3370 0x7fddf00cebe0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fdde4005fd0 tx=0x7fdde402a8b0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.078+0000 7fddf8400640 1 -- 172.21.15.71:0/1384261499 shutdown_connections 2024-09-16T07:37:09.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.078+0000 7fddf8400640 1 --2- 172.21.15.71:0/1384261499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf00b3370 0x7fddf00cebe0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.078+0000 7fddf8400640 1 -- 172.21.15.71:0/1384261499 >> 172.21.15.71:0/1384261499 conn(0x7fddf00b1840 msgr2=0x7fddf00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:09.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.078+0000 7fddf8400640 1 -- 172.21.15.71:0/1384261499 shutdown_connections 2024-09-16T07:37:09.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.078+0000 7fddf8400640 1 -- 172.21.15.71:0/1384261499 wait complete. 2024-09-16T07:37:09.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.080+0000 7fddf8400640 1 Processor -- start 2024-09-16T07:37:09.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.080+0000 7fddf8400640 1 -- start start 2024-09-16T07:37:09.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf8400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf00b3370 0x7fddf01f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:09.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf8400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf015c080 0x7fddf01559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:09.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf8400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf015f090 0x7fddf015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:09.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf8400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fddf00cf590 con 0x7fddf015c080 2024-09-16T07:37:09.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf8400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fddf00cf410 con 0x7fddf015f090 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf8400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fddf01661b0 con 0x7fddf00b3370 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf015c080 0x7fddf01559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf015c080 0x7fddf01559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33530/0 (socket says 172.21.15.71:33530) 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.081+0000 7fddf5000640 1 -- 172.21.15.71:0/2525543972 learned_addr learned my addr 172.21.15.71:0/2525543972 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf5a00640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf00b3370 0x7fddf01f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf6400640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf015f090 0x7fddf015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf5000640 1 -- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf00b3370 msgr2=0x7fddf01f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf5000640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf00b3370 0x7fddf01f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf5000640 1 -- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf015f090 msgr2=0x7fddf015a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf5000640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf015f090 0x7fddf015a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf5000640 1 -- 172.21.15.71:0/2525543972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf01f6580 con 0x7fddf015c080 2024-09-16T07:37:09.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf5000640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf015c080 0x7fddf01559a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00c3f0 tx=0x7fdddc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fdded600640 1 -- 172.21.15.71:0/2525543972 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc017070 con 0x7fddf015c080 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fdded600640 1 -- 172.21.15.71:0/2525543972 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdddc0035f0 con 0x7fddf015c080 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fdded600640 1 -- 172.21.15.71:0/2525543972 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc003a40 con 0x7fddf015c080 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf01f6860 con 0x7fddf015c080 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.082+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf015a850 con 0x7fddf015c080 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.083+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf01f6d80 con 0x7fddf015c080 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.085+0000 7fdded600640 1 -- 172.21.15.71:0/2525543972 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fdddc003be0 con 0x7fddf015c080 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.086+0000 7fdded600640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fddd007aa40 0x7fddd007cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.086+0000 7fdded600640 1 -- 172.21.15.71:0/2525543972 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7fdddc098d90 con 0x7fddf015c080 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.086+0000 7fddf5a00640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fddd007aa40 0x7fddd007cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:09.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.087+0000 7fddf5a00640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fddd007aa40 0x7fddd007cf00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdde4034000 tx=0x7fdde4004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:09.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.090+0000 7fdded600640 1 -- 172.21.15.71:0/2525543972 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fddf01f6d80 con 0x7fddf015c080 2024-09-16T07:37:09.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.272+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fddf00cebe0 con 0x7fddf015c080 2024-09-16T07:37:09.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.273+0000 7fdded600640 1 -- 172.21.15.71:0/2525543972 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) ==== 93+0+10 (secure 0 0 0) 0x7fdddc062e60 con 0x7fddf015c080 2024-09-16T07:37:09.273 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fddd007aa40 msgr2=0x7fddd007cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fddd007aa40 0x7fddd007cf00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdde4034000 tx=0x7fdde4004ab0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf015c080 msgr2=0x7fddf01559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf015c080 0x7fddf01559a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00c3f0 tx=0x7fdddc00c8b0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 shutdown_connections 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fddd007aa40 0x7fddd007cf00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf015f090 0x7fddf015a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf015c080 0x7fddf01559a0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 --2- 172.21.15.71:0/2525543972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf00b3370 0x7fddf01f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 >> 172.21.15.71:0/2525543972 conn(0x7fddf00b1840 msgr2=0x7fddf00735e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:09.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 shutdown_connections 2024-09-16T07:37:09.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.275+0000 7fddf8400640 1 -- 172.21.15.71:0/2525543972 wait complete. 2024-09-16T07:37:09.290 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-16T07:37:09.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.447+0000 7fae74e00640 1 -- 172.21.15.71:0/910704764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae70073e00 msgr2=0x7fae70168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:09.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.447+0000 7fae74e00640 1 --2- 172.21.15.71:0/910704764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae70073e00 0x7fae70168e10 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fae700b4890 tx=0x7fae6802a480 comp rx=0 tx=0).stop 2024-09-16T07:37:09.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.447+0000 7fae74e00640 1 -- 172.21.15.71:0/910704764 shutdown_connections 2024-09-16T07:37:09.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.447+0000 7fae74e00640 1 --2- 172.21.15.71:0/910704764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae70073e00 0x7fae70168e10 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.447+0000 7fae74e00640 1 -- 172.21.15.71:0/910704764 >> 172.21.15.71:0/910704764 conn(0x7fae700b1950 msgr2=0x7fae700b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:09.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.448+0000 7fae74e00640 1 -- 172.21.15.71:0/910704764 shutdown_connections 2024-09-16T07:37:09.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.448+0000 7fae74e00640 1 -- 172.21.15.71:0/910704764 wait complete. 2024-09-16T07:37:09.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.449+0000 7fae74e00640 1 Processor -- start 2024-09-16T07:37:09.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.449+0000 7fae74e00640 1 -- start start 2024-09-16T07:37:09.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae74e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae70073e00 0x7fae700cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:09.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae74e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae700af3d0 0x7fae701f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:09.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae74e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae70161550 0x7fae701f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:09.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae74e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fae701694d0 con 0x7fae700af3d0 2024-09-16T07:37:09.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae74e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fae70169350 con 0x7fae70161550 2024-09-16T07:37:09.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae74e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fae70169650 con 0x7fae70073e00 2024-09-16T07:37:09.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae6d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae700af3d0 0x7fae701f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:09.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae6d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae700af3d0 0x7fae701f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33546/0 (socket says 172.21.15.71:33546) 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae6d400640 1 -- 172.21.15.71:0/2251201076 learned_addr learned my addr 172.21.15.71:0/2251201076 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae6de00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae70073e00 0x7fae700cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.450+0000 7fae6e800640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae70161550 0x7fae701f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae6d400640 1 -- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae70073e00 msgr2=0x7fae700cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae6d400640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae70073e00 0x7fae700cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae6d400640 1 -- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae70161550 msgr2=0x7fae701f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae6d400640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae70161550 0x7fae701f8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae6d400640 1 -- 172.21.15.71:0/2251201076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae700cfa10 con 0x7fae700af3d0 2024-09-16T07:37:09.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae6e800640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae70161550 0x7fae701f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:09.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae6d400640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae700af3d0 0x7fae701f84c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c480 tx=0x7fae5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:09.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae66000640 1 -- 172.21.15.71:0/2251201076 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c016020 con 0x7fae700af3d0 2024-09-16T07:37:09.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae66000640 1 -- 172.21.15.71:0/2251201076 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae5c012920 con 0x7fae700af3d0 2024-09-16T07:37:09.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae66000640 1 -- 172.21.15.71:0/2251201076 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c012d70 con 0x7fae700af3d0 2024-09-16T07:37:09.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae700cfcf0 con 0x7fae700af3d0 2024-09-16T07:37:09.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.451+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae700d02b0 con 0x7fae700af3d0 2024-09-16T07:37:09.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.452+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7006ef50 con 0x7fae700af3d0 2024-09-16T07:37:09.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.454+0000 7fae66000640 1 -- 172.21.15.71:0/2251201076 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fae5c019020 con 0x7fae700af3d0 2024-09-16T07:37:09.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.454+0000 7fae66000640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae5007aa10 0x7fae5007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:09.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.455+0000 7fae66000640 1 -- 172.21.15.71:0/2251201076 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7fae5c099ab0 con 0x7fae700af3d0 2024-09-16T07:37:09.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.455+0000 7fae6de00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae5007aa10 0x7fae5007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:09.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.456+0000 7fae6de00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae5007aa10 0x7fae5007ced0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fae7015a620 tx=0x7fae6000d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:09.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.459+0000 7fae66000640 1 -- 172.21.15.71:0/2251201076 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fae5c020020 con 0x7fae700af3d0 2024-09-16T07:37:09.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:09.713+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7fae70154e90 con 0x7fae700af3d0 2024-09-16T07:37:10.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.534+0000 7fae66000640 1 -- 172.21.15.71:0/2251201076 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v673) ==== 189+0+0 (secure 0 0 0) 0x7fae5c063b80 con 0x7fae700af3d0 2024-09-16T07:37:10.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.536+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae5007aa10 msgr2=0x7fae5007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae5007aa10 0x7fae5007ced0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fae7015a620 tx=0x7fae6000d040 comp rx=0 tx=0).stop 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae700af3d0 msgr2=0x7fae701f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae700af3d0 0x7fae701f84c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c480 tx=0x7fae5c00c940 comp rx=0 tx=0).stop 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 shutdown_connections 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fae5007aa10 0x7fae5007ced0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae70161550 0x7fae701f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae700af3d0 0x7fae701f84c0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 --2- 172.21.15.71:0/2251201076 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae70073e00 0x7fae700cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 >> 172.21.15.71:0/2251201076 conn(0x7fae700b1950 msgr2=0x7fae7015f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 shutdown_connections 2024-09-16T07:37:10.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.537+0000 7fae74e00640 1 -- 172.21.15.71:0/2251201076 wait complete. 2024-09-16T07:37:10.555 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T07:37:10.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.716+0000 7fbd6a800640 1 -- 172.21.15.71:0/2435698723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415b0c0 msgr2=0x7fbd6415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:10.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.716+0000 7fbd6a800640 1 --2- 172.21.15.71:0/2435698723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415b0c0 0x7fbd6415b4c0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fbd64076770 tx=0x7fbd5c02aa10 comp rx=0 tx=0).stop 2024-09-16T07:37:10.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.717+0000 7fbd6a800640 1 -- 172.21.15.71:0/2435698723 shutdown_connections 2024-09-16T07:37:10.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.717+0000 7fbd6a800640 1 --2- 172.21.15.71:0/2435698723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415b0c0 0x7fbd6415b4c0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.717+0000 7fbd6a800640 1 --2- 172.21.15.71:0/2435698723 >> v2:172.21.15.177:3300/0 conn(0x7fbd6415f220 0x7fbd640cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.717+0000 7fbd6a800640 1 -- 172.21.15.71:0/2435698723 >> 172.21.15.71:0/2435698723 conn(0x7fbd640b2b90 msgr2=0x7fbd640b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:10.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.717+0000 7fbd6a800640 1 -- 172.21.15.71:0/2435698723 shutdown_connections 2024-09-16T07:37:10.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.717+0000 7fbd6a800640 1 -- 172.21.15.71:0/2435698723 wait complete. 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.718+0000 7fbd6a800640 1 Processor -- start 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.718+0000 7fbd6a800640 1 -- start start 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.719+0000 7fbd6a800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd640cd8a0 0x7fbd64152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.719+0000 7fbd6a800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd6415b0c0 0x7fbd64159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.719+0000 7fbd6a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415f220 0x7fbd640b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.719+0000 7fbd6a800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbd640708c0 con 0x7fbd6415f220 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.719+0000 7fbd6a800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64070740 con 0x7fbd6415b0c0 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.719+0000 7fbd6a800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbd64070a40 con 0x7fbd640cd8a0 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415f220 0x7fbd640b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415f220 0x7fbd640b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33572/0 (socket says 172.21.15.71:33572) 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 -- 172.21.15.71:0/3975497644 learned_addr learned my addr 172.21.15.71:0/3975497644 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd62a00640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd6415b0c0 0x7fbd64159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63400640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd640cd8a0 0x7fbd64152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 -- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd640cd8a0 msgr2=0x7fbd64152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd640cd8a0 0x7fbd64152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 -- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd6415b0c0 msgr2=0x7fbd64159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd6415b0c0 0x7fbd64159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 -- 172.21.15.71:0/3975497644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd64152120 con 0x7fbd6415f220 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.720+0000 7fbd63e00640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415f220 0x7fbd640b1ff0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009d60 tx=0x7fbd5c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.721+0000 7fbd5b400640 1 -- 172.21.15.71:0/3975497644 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c047070 con 0x7fbd6415f220 2024-09-16T07:37:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.721+0000 7fbd5b400640 1 -- 172.21.15.71:0/3975497644 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd5c004250 con 0x7fbd6415f220 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.721+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd64152400 con 0x7fbd6415f220 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.721+0000 7fbd5b400640 1 -- 172.21.15.71:0/3975497644 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c033050 con 0x7fbd6415f220 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.721+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641529c0 con 0x7fbd6415f220 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.722+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd64070fb0 con 0x7fbd6415f220 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.723+0000 7fbd5b400640 1 -- 172.21.15.71:0/3975497644 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fbd5c04a070 con 0x7fbd6415f220 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.724+0000 7fbd5b400640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbd4407aa10 0x7fbd4407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.724+0000 7fbd5b400640 1 -- 172.21.15.71:0/3975497644 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7fbd5c0bae10 con 0x7fbd6415f220 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.724+0000 7fbd63400640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbd4407aa10 0x7fbd4407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.725+0000 7fbd63400640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbd4407aa10 0x7fbd4407ced0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400b810 tx=0x7fbd54008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:10.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.728+0000 7fbd5b400640 1 -- 172.21.15.71:0/3975497644 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbd5c084ee0 con 0x7fbd6415f220 2024-09-16T07:37:10.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.914+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbd64152b90 con 0x7fbd6415f220 2024-09-16T07:37:10.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.915+0000 7fbd5b400640 1 -- 172.21.15.71:0/3975497644 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v53) ==== 59+0+1998 (secure 0 0 0) 0x7fbd5c0847e0 con 0x7fbd6415f220 2024-09-16T07:37:10.914 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T07:37:10.914 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T07:37:10.914 INFO:teuthology.orchestra.run.smithi071.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4115.312358 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3960.654452 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3785.239405 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3844.335278 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6790.055234 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 6229.050005 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5174.871357 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5062.231770 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 2/2 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2024-09-16T07:37:10.915 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T07:37:10.916 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T07:37:10.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbd4407aa10 msgr2=0x7fbd4407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:10.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbd4407aa10 0x7fbd4407ced0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400b810 tx=0x7fbd54008040 comp rx=0 tx=0).stop 2024-09-16T07:37:10.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415f220 msgr2=0x7fbd640b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415f220 0x7fbd640b1ff0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009d60 tx=0x7fbd5c003f10 comp rx=0 tx=0).stop 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 shutdown_connections 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbd4407aa10 0x7fbd4407ced0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6415f220 0x7fbd640b1ff0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd6415b0c0 0x7fbd64159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 --2- 172.21.15.71:0/3975497644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd640cd8a0 0x7fbd64152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 >> 172.21.15.71:0/3975497644 conn(0x7fbd640b2b90 msgr2=0x7fbd640711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 shutdown_connections 2024-09-16T07:37:10.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:10.917+0000 7fbd6a800640 1 -- 172.21.15.71:0/3975497644 wait complete. 2024-09-16T07:37:10.934 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:37:11.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.094+0000 7fe28f200640 1 -- 172.21.15.71:0/175652234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe274003a50 msgr2=0x7fe274003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:11.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.094+0000 7fe28f200640 1 --2- 172.21.15.71:0/175652234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe274003a50 0x7fe274003e70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe280001eb0 tx=0x7fe274005830 comp rx=0 tx=0).stop 2024-09-16T07:37:11.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.095+0000 7fe28f200640 1 -- 172.21.15.71:0/175652234 shutdown_connections 2024-09-16T07:37:11.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.095+0000 7fe28f200640 1 --2- 172.21.15.71:0/175652234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe274003a50 0x7fe274003e70 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.095+0000 7fe28f200640 1 -- 172.21.15.71:0/175652234 >> 172.21.15.71:0/175652234 conn(0x7fe29009d0b0 msgr2=0x7fe2900d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:11.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.095+0000 7fe28f200640 1 -- 172.21.15.71:0/175652234 shutdown_connections 2024-09-16T07:37:11.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.095+0000 7fe28f200640 1 -- 172.21.15.71:0/175652234 wait complete. 2024-09-16T07:37:11.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.096+0000 7fe28f200640 1 Processor -- start 2024-09-16T07:37:11.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.096+0000 7fe28f200640 1 -- start start 2024-09-16T07:37:11.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.097+0000 7fe28f200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe274003a50 0x7fe2901f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:11.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.097+0000 7fe28f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2901f4c40 0x7fe2900b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:11.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.097+0000 7fe28f200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2901f50c0 0x7fe2900b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:11.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.097+0000 7fe28f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe2900b50c0 con 0x7fe2901f4c40 2024-09-16T07:37:11.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.097+0000 7fe28f200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe2900b4f40 con 0x7fe274003a50 2024-09-16T07:37:11.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.097+0000 7fe28f200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe2900d22f0 con 0x7fe2901f50c0 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2901f4c40 0x7fe2900b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2901f4c40 0x7fe2900b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33580/0 (socket says 172.21.15.71:33580) 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 -- 172.21.15.71:0/2909060231 learned_addr learned my addr 172.21.15.71:0/2909060231 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28de00640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe274003a50 0x7fe2901f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28e800640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2901f50c0 0x7fe2900b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 -- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2901f50c0 msgr2=0x7fe2900b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2901f50c0 0x7fe2900b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 -- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe274003a50 msgr2=0x7fe2901f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe274003a50 0x7fe2901f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 -- 172.21.15.71:0/2909060231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2901f9180 con 0x7fe2901f4c40 2024-09-16T07:37:11.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28e800640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2901f50c0 0x7fe2900b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:11.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28de00640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe274003a50 0x7fe2901f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:11.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.098+0000 7fe28d400640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2901f4c40 0x7fe2900b3090 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe27c004d90 tx=0x7fe27c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:11.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.099+0000 7fe286000640 1 -- 172.21.15.71:0/2909060231 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c017070 con 0x7fe2901f4c40 2024-09-16T07:37:11.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.099+0000 7fe286000640 1 -- 172.21.15.71:0/2909060231 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe27c0108c0 con 0x7fe2901f4c40 2024-09-16T07:37:11.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.099+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2900b79e0 con 0x7fe2901f4c40 2024-09-16T07:37:11.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.099+0000 7fe286000640 1 -- 172.21.15.71:0/2909060231 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c001d40 con 0x7fe2901f4c40 2024-09-16T07:37:11.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.099+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2901f5950 con 0x7fe2901f4c40 2024-09-16T07:37:11.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.100+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2900ce820 con 0x7fe2901f4c40 2024-09-16T07:37:11.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.101+0000 7fe286000640 1 -- 172.21.15.71:0/2909060231 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fe27c01a070 con 0x7fe2901f4c40 2024-09-16T07:37:11.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.102+0000 7fe286000640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe27007aa40 0x7fe27007cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:11.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.102+0000 7fe286000640 1 -- 172.21.15.71:0/2909060231 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7fe27c0997c0 con 0x7fe2901f4c40 2024-09-16T07:37:11.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.103+0000 7fe28de00640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe27007aa40 0x7fe27007cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:11.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.104+0000 7fe28de00640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe27007aa40 0x7fe27007cf00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe280033000 tx=0x7fe2800019f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:11.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.106+0000 7fe286000640 1 -- 172.21.15.71:0/2909060231 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe2901f9180 con 0x7fe2901f4c40 2024-09-16T07:37:11.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.287+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe29015d8b0 con 0x7fe2901f4c40 2024-09-16T07:37:11.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.292+0000 7fe286000640 1 -- 172.21.15.71:0/2909060231 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7fe27c063890 con 0x7fe2901f4c40 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe27007aa40 msgr2=0x7fe27007cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe27007aa40 0x7fe27007cf00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe280033000 tx=0x7fe2800019f0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2901f4c40 msgr2=0x7fe2900b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2901f4c40 0x7fe2900b3090 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe27c004d90 tx=0x7fe27c001630 comp rx=0 tx=0).stop 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 shutdown_connections 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe27007aa40 0x7fe27007cf00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2901f50c0 0x7fe2900b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2901f4c40 0x7fe2900b3090 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 --2- 172.21.15.71:0/2909060231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe274003a50 0x7fe2901f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 >> 172.21.15.71:0/2909060231 conn(0x7fe29009d0b0 msgr2=0x7fe2900cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:11.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 shutdown_connections 2024-09-16T07:37:11.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.294+0000 7fe28f200640 1 -- 172.21.15.71:0/2909060231 wait complete. 2024-09-16T07:37:11.309 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:37:11.309 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:37:11.396 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:37:11.487 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:37:11.487 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:11.487 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:11.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:37:11.522 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:37:11.522 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:11.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:37:11.561 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:11.561 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:11.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:37:11.597 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:37:11.597 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:11.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:37:11.637 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T07:37:11.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:37:11.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.794+0000 7f9cf3600640 1 -- 172.21.15.71:0/2922972567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec0af3d0 msgr2=0x7f9cec06f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.794+0000 7f9cf3600640 1 --2- 172.21.15.71:0/2922972567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec0af3d0 0x7f9cec06f660 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8005fd0 tx=0x7f9cd802a5f0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.795+0000 7f9cf3600640 1 -- 172.21.15.71:0/2922972567 shutdown_connections 2024-09-16T07:37:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.795+0000 7f9cf3600640 1 --2- 172.21.15.71:0/2922972567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec0af3d0 0x7f9cec06f660 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.795+0000 7f9cf3600640 1 -- 172.21.15.71:0/2922972567 >> 172.21.15.71:0/2922972567 conn(0x7f9cec0b1950 msgr2=0x7f9cec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.795+0000 7f9cf3600640 1 -- 172.21.15.71:0/2922972567 shutdown_connections 2024-09-16T07:37:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.795+0000 7f9cf3600640 1 -- 172.21.15.71:0/2922972567 wait complete. 2024-09-16T07:37:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.796+0000 7f9cf3600640 1 Processor -- start 2024-09-16T07:37:11.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.796+0000 7f9cf3600640 1 -- start start 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.797+0000 7f9cf3600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec072df0 0x7f9cec0b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.797+0000 7f9cf3600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cec161550 0x7f9cec09b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.797+0000 7f9cf3600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cec09b9e0 0x7f9cec1f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.797+0000 7f9cf3600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec073950 con 0x7f9cec072df0 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.797+0000 7f9cf3600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec06fdb0 con 0x7f9cec09b9e0 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.797+0000 7f9cf3600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9cec073ad0 con 0x7f9cec161550 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.797+0000 7f9cf0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec072df0 0x7f9cec0b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec072df0 0x7f9cec0b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33592/0 (socket says 172.21.15.71:33592) 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf0c00640 1 -- 172.21.15.71:0/403758032 learned_addr learned my addr 172.21.15.71:0/403758032 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cebe00640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cec161550 0x7f9cec09b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf1600640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cec09b9e0 0x7f9cec1f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf0c00640 1 -- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cec161550 msgr2=0x7f9cec09b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf0c00640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cec161550 0x7f9cec09b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf0c00640 1 -- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cec09b9e0 msgr2=0x7f9cec1f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf0c00640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cec09b9e0 0x7f9cec1f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf0c00640 1 -- 172.21.15.71:0/403758032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cec09aa00 con 0x7f9cec072df0 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cebe00640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cec161550 0x7f9cec09b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf0c00640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec072df0 0x7f9cec0b0120 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00c780 tx=0x7f9cdc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9ce8c00640 1 -- 172.21.15.71:0/403758032 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc003b70 con 0x7f9cec072df0 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9ce8c00640 1 -- 172.21.15.71:0/403758032 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cdc003d10 con 0x7f9cec072df0 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9ce8c00640 1 -- 172.21.15.71:0/403758032 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00aab0 con 0x7f9cec072df0 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.798+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cec1f5b20 con 0x7f9cec072df0 2024-09-16T07:37:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.799+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cec09af80 con 0x7f9cec072df0 2024-09-16T07:37:11.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.800+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cec073fc0 con 0x7f9cec072df0 2024-09-16T07:37:11.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.801+0000 7f9ce8c00640 1 -- 172.21.15.71:0/403758032 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f9cdc00ac50 con 0x7f9cec072df0 2024-09-16T07:37:11.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.802+0000 7f9ce8c00640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9ccc07aa10 0x7f9ccc07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:11.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.802+0000 7f9ce8c00640 1 -- 172.21.15.71:0/403758032 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7f9cdc099880 con 0x7f9cec072df0 2024-09-16T07:37:11.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.802+0000 7f9cebe00640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9ccc07aa10 0x7f9ccc07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:11.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.803+0000 7f9cebe00640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9ccc07aa10 0x7f9ccc07ced0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8033000 tx=0x7f9cd8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:11.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:11.806+0000 7f9ce8c00640 1 -- 172.21.15.71:0/403758032 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9cdc0639f0 con 0x7f9cec072df0 2024-09-16T07:37:12.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.026+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cec0acde0 con 0x7f9cec072df0 2024-09-16T07:37:12.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.026+0000 7f9ce8c00640 1 -- 172.21.15.71:0/403758032 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+3917 (secure 0 0 0) 0x7f9cdc0632f0 con 0x7f9cec072df0 2024-09-16T07:37:12.026 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:12.026 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":88,"btime":"2024-09-16T07:37:05:203129+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7283,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4275610322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":4275610322},{"type":"v1","addr":"172.21.15.71:6835","nonce":4275610322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/2559248159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2559248159},{"type":"v1","addr":"172.21.15.177:6836","nonce":2559248159}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1689772423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1689772423},{"type":"v1","addr":"172.21.15.177:6837","nonce":1689772423}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":88},{"gid":8904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/1515242906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1515242906},{"type":"v1","addr":"172.21.15.71:6834","nonce":1515242906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[]} 2024-09-16T07:37:12.026 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 88 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9ccc07aa10 msgr2=0x7f9ccc07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9ccc07aa10 0x7f9ccc07ced0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8033000 tx=0x7f9cd8004730 comp rx=0 tx=0).stop 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec072df0 msgr2=0x7f9cec0b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec072df0 0x7f9cec0b0120 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00c780 tx=0x7f9cdc00cc40 comp rx=0 tx=0).stop 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 shutdown_connections 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9ccc07aa10 0x7f9ccc07ced0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cec09b9e0 0x7f9cec1f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cec161550 0x7f9cec09b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 --2- 172.21.15.71:0/403758032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cec072df0 0x7f9cec0b0120 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 >> 172.21.15.71:0/403758032 conn(0x7f9cec0b1950 msgr2=0x7f9cec15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 shutdown_connections 2024-09-16T07:37:12.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.029+0000 7f9cf3600640 1 -- 172.21.15.71:0/403758032 wait complete. 2024-09-16T07:37:12.045 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T07:37:12.045 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T07:37:12.045 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:37:12.045 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:37:12.046 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T07:37:12.046 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T07:37:12.046 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:37:12.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:37:12.046 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T07:37:12.046 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T07:37:12.047 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:37:12.047 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:37:12.047 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T07:37:12.047 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T07:37:12.047 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:37:12.047 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:37:12.060 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:37:12.060 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T07:37:12.061 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:37:12.062 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T07:37:12.064 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:37:12.064 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T07:37:12.065 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:37:12.065 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T07:37:12.106 INFO:tasks.ceph.mds.a:Started 2024-09-16T07:37:12.106 DEBUG:teuthology.parallel:result is None 2024-09-16T07:37:12.108 INFO:tasks.ceph.mds.b:Started 2024-09-16T07:37:12.108 DEBUG:teuthology.parallel:result is None 2024-09-16T07:37:12.109 INFO:tasks.ceph.mds.c:Started 2024-09-16T07:37:12.109 DEBUG:teuthology.parallel:result is None 2024-09-16T07:37:12.110 INFO:tasks.ceph.mds.d:Started 2024-09-16T07:37:12.110 DEBUG:teuthology.parallel:result is None 2024-09-16T07:37:12.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T07:37:12.234 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-09-16T07:37:12.235 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-09-16T07:37:12.240 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-09-16T07:37:12.241 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:37:12.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.277+0000 7fda87200640 1 -- 172.21.15.71:0/1355253584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda800b3e20 msgr2=0x7fda800688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.277+0000 7fda87200640 1 --2- 172.21.15.71:0/1355253584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda800b3e20 0x7fda800688b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda70005fd0 tx=0x7fda7002a780 comp rx=0 tx=0).stop 2024-09-16T07:37:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.277+0000 7fda87200640 1 -- 172.21.15.71:0/1355253584 shutdown_connections 2024-09-16T07:37:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.277+0000 7fda87200640 1 --2- 172.21.15.71:0/1355253584 >> v2:172.21.15.71:3300/0 conn(0x7fda8015e6d0 0x7fda8006cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.277+0000 7fda87200640 1 --2- 172.21.15.71:0/1355253584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda800b3e20 0x7fda800688b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.277+0000 7fda87200640 1 -- 172.21.15.71:0/1355253584 >> 172.21.15.71:0/1355253584 conn(0x7fda800b2230 msgr2=0x7fda800b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.277+0000 7fda87200640 1 -- 172.21.15.71:0/1355253584 shutdown_connections 2024-09-16T07:37:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.277+0000 7fda87200640 1 -- 172.21.15.71:0/1355253584 wait complete. 2024-09-16T07:37:12.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.279+0000 7fda87200640 1 Processor -- start 2024-09-16T07:37:12.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.279+0000 7fda87200640 1 -- start start 2024-09-16T07:37:12.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda87200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda800b3e20 0x7fda801539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda87200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8015e6d0 0x7fda800b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda87200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8015eb30 0x7fda800b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda87200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fda801629e0 con 0x7fda800b3e20 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda87200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda80162860 con 0x7fda8015e6d0 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda87200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda80162b60 con 0x7fda8015eb30 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda7fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda800b3e20 0x7fda801539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda7fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda800b3e20 0x7fda801539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33636/0 (socket says 172.21.15.71:33636) 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda7fe00640 1 -- 172.21.15.71:0/3088597541 learned_addr learned my addr 172.21.15.71:0/3088597541 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda85200640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8015eb30 0x7fda800b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.280+0000 7fda7f400640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8015e6d0 0x7fda800b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:12.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda7fe00640 1 -- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8015eb30 msgr2=0x7fda800b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda7fe00640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8015eb30 0x7fda800b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda7fe00640 1 -- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8015e6d0 msgr2=0x7fda800b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda7fe00640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8015e6d0 0x7fda800b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda7fe00640 1 -- 172.21.15.71:0/3088597541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda80153f30 con 0x7fda800b3e20 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda7fe00640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda800b3e20 0x7fda801539f0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fda70006010 tx=0x7fda70003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda77e00640 1 -- 172.21.15.71:0/3088597541 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70040070 con 0x7fda800b3e20 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda77e00640 1 -- 172.21.15.71:0/3088597541 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda70004540 con 0x7fda800b3e20 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda801562e0 con 0x7fda800b3e20 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda77e00640 1 -- 172.21.15.71:0/3088597541 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70031480 con 0x7fda800b3e20 2024-09-16T07:37:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.281+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda801f86f0 con 0x7fda800b3e20 2024-09-16T07:37:12.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.283+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8006d350 con 0x7fda800b3e20 2024-09-16T07:37:12.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.284+0000 7fda77e00640 1 -- 172.21.15.71:0/3088597541 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fda70043070 con 0x7fda800b3e20 2024-09-16T07:37:12.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.285+0000 7fda77e00640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fda6007aa10 0x7fda6007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:12.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.285+0000 7fda77e00640 1 -- 172.21.15.71:0/3088597541 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5027+0+0 (secure 0 0 0) 0x7fda700baaf0 con 0x7fda800b3e20 2024-09-16T07:37:12.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.285+0000 7fda7f400640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fda6007aa10 0x7fda6007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:12.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.286+0000 7fda7f400640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fda6007aa10 0x7fda6007ced0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fda80155260 tx=0x7fda6c009260 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:12.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.289+0000 7fda77e00640 1 -- 172.21.15.71:0/3088597541 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fda70084610 con 0x7fda800b3e20 2024-09-16T07:37:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.485+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fda800d0270 con 0x7fda800b3e20 2024-09-16T07:37:12.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.553+0000 7fda77e00640 1 -- 172.21.15.71:0/3088597541 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v140) ==== 97+0+0 (secure 0 0 0) 0x7fda70089040 con 0x7fda800b3e20 2024-09-16T07:37:12.553 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-09-16T07:37:12.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fda6007aa10 msgr2=0x7fda6007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fda6007aa10 0x7fda6007ced0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fda80155260 tx=0x7fda6c009260 comp rx=0 tx=0).stop 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda800b3e20 msgr2=0x7fda801539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda800b3e20 0x7fda801539f0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fda70006010 tx=0x7fda70003c90 comp rx=0 tx=0).stop 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 shutdown_connections 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fda6007aa10 0x7fda6007ced0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8015eb30 0x7fda800b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8015e6d0 0x7fda800b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 --2- 172.21.15.71:0/3088597541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda800b3e20 0x7fda801539f0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 >> 172.21.15.71:0/3088597541 conn(0x7fda800b2230 msgr2=0x7fda801620c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 shutdown_connections 2024-09-16T07:37:12.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.556+0000 7fda87200640 1 -- 172.21.15.71:0/3088597541 wait complete. 2024-09-16T07:37:12.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T07:37:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.732+0000 7f9dc5a00640 1 -- 172.21.15.71:0/4252988060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 msgr2=0x7f9da400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.732+0000 7f9dc5a00640 1 --2- 172.21.15.71:0/4252988060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 0x7f9da400b9c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c310 tx=0x7f9da4005830 comp rx=0 tx=0).stop 2024-09-16T07:37:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.732+0000 7f9dc5a00640 1 -- 172.21.15.71:0/4252988060 shutdown_connections 2024-09-16T07:37:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.732+0000 7f9dc5a00640 1 --2- 172.21.15.71:0/4252988060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 0x7f9da400b9c0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.732+0000 7f9dc5a00640 1 -- 172.21.15.71:0/4252988060 >> 172.21.15.71:0/4252988060 conn(0x7f9dc00b2b90 msgr2=0x7f9dc00b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.732+0000 7f9dc5a00640 1 -- 172.21.15.71:0/4252988060 shutdown_connections 2024-09-16T07:37:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.732+0000 7f9dc5a00640 1 -- 172.21.15.71:0/4252988060 wait complete. 2024-09-16T07:37:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.734+0000 7f9dc5a00640 1 Processor -- start 2024-09-16T07:37:12.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.734+0000 7f9dc5a00640 1 -- start start 2024-09-16T07:37:12.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dc5a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 0x7f9dc0157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:12.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dc5a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc0157810 0x7f9dc013bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:12.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dc5a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc00b6670 0x7f9dc01451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:12.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dc5a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0141da0 con 0x7f9da40051c0 2024-09-16T07:37:12.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dc5a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0141c20 con 0x7f9dc00b6670 2024-09-16T07:37:12.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dc5a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc0141f20 con 0x7f9dc0157810 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 0x7f9dc0157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 0x7f9dc0157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33654/0 (socket says 172.21.15.71:33654) 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dbea00640 1 -- 172.21.15.71:0/2889456477 learned_addr learned my addr 172.21.15.71:0/2889456477 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dbe000640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc0157810 0x7f9dc013bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.735+0000 7f9dbf400640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc00b6670 0x7f9dc01451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dbea00640 1 -- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc0157810 msgr2=0x7f9dc013bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dbea00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc0157810 0x7f9dc013bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dbea00640 1 -- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc00b6670 msgr2=0x7f9dc01451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dbea00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc00b6670 0x7f9dc01451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dbea00640 1 -- 172.21.15.71:0/2889456477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc00b6b70 con 0x7f9da40051c0 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dbe000640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc0157810 0x7f9dc013bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dbf400640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc00b6670 0x7f9dc01451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:12.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dbea00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 0x7f9dc0157170 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9db0003fd0 tx=0x7f9db00064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:12.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9db6a00640 1 -- 172.21.15.71:0/2889456477 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0015070 con 0x7f9da40051c0 2024-09-16T07:37:12.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9db6a00640 1 -- 172.21.15.71:0/2889456477 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9db000fa60 con 0x7f9da40051c0 2024-09-16T07:37:12.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc01446f0 con 0x7f9da40051c0 2024-09-16T07:37:12.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9db6a00640 1 -- 172.21.15.71:0/2889456477 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011430 con 0x7f9da40051c0 2024-09-16T07:37:12.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.736+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc0144cb0 con 0x7f9da40051c0 2024-09-16T07:37:12.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.737+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc00cc0c0 con 0x7f9da40051c0 2024-09-16T07:37:12.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.738+0000 7f9db6a00640 1 -- 172.21.15.71:0/2889456477 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f9db00115d0 con 0x7f9da40051c0 2024-09-16T07:37:12.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.739+0000 7f9db6a00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9da007aa40 0x7f9da007cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:12.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.740+0000 7f9db6a00640 1 -- 172.21.15.71:0/2889456477 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4941+0+0 (secure 0 0 0) 0x7f9db009a8f0 con 0x7f9da40051c0 2024-09-16T07:37:12.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.740+0000 7f9dbe000640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9da007aa40 0x7f9da007cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:12.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.740+0000 7f9dbe000640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9da007aa40 0x7f9da007cf00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0157120 tx=0x7f9dac005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:12.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.743+0000 7f9db6a00640 1 -- 172.21.15.71:0/2889456477 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9dc00cc0c0 con 0x7f9da40051c0 2024-09-16T07:37:12.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.997+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9dc015f7d0 con 0x7f9da40051c0 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:12.997+0000 7f9db6a00640 1 -- 172.21.15.71:0/2889456477 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f9db00649c0 con 0x7f9da40051c0 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odmcz1THhAAvWWczIpYY6CJKi8/UjGvUg==", 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:12.997 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmJjTDMRAAtpL5coLyOvS0EgdVorOGag==", 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:37:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmMa37JxAA3wjcjgnDl9wBRfhB0Wjlmw==", 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:12.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odml7OTOxAAW1OX8ZomSEcOacLTUzJuBA==", 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-09-16T07:37:13.000 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCO3OdmdsUdFxAAiJZ3yiFcO9hRpaPXk+2H6w==", 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCR3OdmtPuMIhAAvHxSqhYId1VixWMu4fTGXQ==", 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:37:13.001 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCU3OdmpNGfLxAAteeR7oc9kfOQ4lZSKAp9SQ==", 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.002 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCX3Odmx4k/OhAAj9RUa+23BpICOXJsVCLeQQ==", 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCd3Odm0qMcORAAwKvvagvx9HdeSpZEDjIMpw==", 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:37:13.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCh3Odmz6vhCBAAn1lYHjaaONmRKmpwxGwpCg==", 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCk3OdmRzvEEhAAXmKH6SJT2lpAWeiMFWrmnw==", 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCn3OdmCyAJKxAAPdtyXMoge68b2e8jq07Aqg==", 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g==", 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odm1dV0FRAAu2tWgrh5yH0lfSEhejipAg==", 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3OdmWVP0DhAAJOgWU4Ba5sOtExBYI2BoTg==", 2024-09-16T07:37:13.007 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCL3Odmr5vpGxAAWxZMoZe6WqLDRQ48ya+QcA==", 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-09-16T07:37:13.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCJ3Odmgi+GKhAA4CFZHXgPtlBDUHF3PlYIJg==", 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3Odms1fJFxAAgtzEgPYIjgHGQEV8JOr87Q==", 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCK3OdmOqzMEBAACOStS6lX0t4tvZBWDAjhKA==", 2024-09-16T07:37:13.010 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-09-16T07:37:13.011 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-09-16T07:37:13.011 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-09-16T07:37:13.011 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-09-16T07:37:13.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.011 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.011 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-16T07:37:13.011 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:37:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9da007aa40 msgr2=0x7f9da007cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9da007aa40 0x7f9da007cf00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0157120 tx=0x7f9dac005f90 comp rx=0 tx=0).stop 2024-09-16T07:37:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 msgr2=0x7f9dc0157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 0x7f9dc0157170 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9db0003fd0 tx=0x7f9db00064f0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 shutdown_connections 2024-09-16T07:37:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9da007aa40 0x7f9da007cf00 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc00b6670 0x7f9dc01451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc0157810 0x7f9dc013bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 --2- 172.21.15.71:0/2889456477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9da40051c0 0x7f9dc0157170 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 >> 172.21.15.71:0/2889456477 conn(0x7f9dc00b2b90 msgr2=0x7f9dc00b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 shutdown_connections 2024-09-16T07:37:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.000+0000 7f9dc5a00640 1 -- 172.21.15.71:0/2889456477 wait complete. 2024-09-16T07:37:13.016 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T07:37:13.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.173+0000 7fb237400640 1 -- 172.21.15.71:0/3997545233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230072df0 msgr2=0x7fb230073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.173+0000 7fb237400640 1 --2- 172.21.15.71:0/3997545233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230072df0 0x7fb230073240 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b4890 tx=0x7fb22802a4a0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.173+0000 7fb237400640 1 -- 172.21.15.71:0/3997545233 shutdown_connections 2024-09-16T07:37:13.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.173+0000 7fb237400640 1 --2- 172.21.15.71:0/3997545233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230072df0 0x7fb230073240 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.173+0000 7fb237400640 1 --2- 172.21.15.71:0/3997545233 >> v2:172.21.15.177:3300/0 conn(0x7fb2300af3d0 0x7fb23006f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.173+0000 7fb237400640 1 -- 172.21.15.71:0/3997545233 >> 172.21.15.71:0/3997545233 conn(0x7fb2300b1950 msgr2=0x7fb2300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.173+0000 7fb237400640 1 -- 172.21.15.71:0/3997545233 shutdown_connections 2024-09-16T07:37:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.173+0000 7fb237400640 1 -- 172.21.15.71:0/3997545233 wait complete. 2024-09-16T07:37:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.175+0000 7fb237400640 1 Processor -- start 2024-09-16T07:37:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.175+0000 7fb237400640 1 -- start start 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb237400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb230072df0 0x7fb230156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb237400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300af3d0 0x7fb2300cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb237400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230161550 0x7fb2300b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb237400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb230073780 con 0x7fb230161550 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb237400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb23006fdb0 con 0x7fb2300af3d0 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb237400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb230073900 con 0x7fb230072df0 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb235400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230161550 0x7fb2300b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb235400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230161550 0x7fb2300b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33660/0 (socket says 172.21.15.71:33660) 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb235400640 1 -- 172.21.15.71:0/1557659222 learned_addr learned my addr 172.21.15.71:0/1557659222 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb234a00640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb230072df0 0x7fb230156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.176+0000 7fb22fe00640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300af3d0 0x7fb2300cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:13.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb235400640 1 -- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb230072df0 msgr2=0x7fb230156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb235400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb230072df0 0x7fb230156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb235400640 1 -- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300af3d0 msgr2=0x7fb2300cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb235400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300af3d0 0x7fb2300cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb235400640 1 -- 172.21.15.71:0/1557659222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2301f8390 con 0x7fb230161550 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb235400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230161550 0x7fb2300b08c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb22802aa90 tx=0x7fb228032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb22cc00640 1 -- 172.21.15.71:0/1557659222 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228040070 con 0x7fb230161550 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb22cc00640 1 -- 172.21.15.71:0/1557659222 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb228005330 con 0x7fb230161550 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2301f9110 con 0x7fb230161550 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.177+0000 7fb22cc00640 1 -- 172.21.15.71:0/1557659222 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228003800 con 0x7fb230161550 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.178+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2301f8910 con 0x7fb230161550 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.179+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb230070330 con 0x7fb230161550 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.180+0000 7fb22cc00640 1 -- 172.21.15.71:0/1557659222 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fb228039040 con 0x7fb230161550 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.180+0000 7fb22cc00640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb21007aa40 0x7fb21007cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:13.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.181+0000 7fb22cc00640 1 -- 172.21.15.71:0/1557659222 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4941+0+0 (secure 0 0 0) 0x7fb2280bac00 con 0x7fb230161550 2024-09-16T07:37:13.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.181+0000 7fb234a00640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb21007aa40 0x7fb21007cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:13.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.181+0000 7fb234a00640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb21007aa40 0x7fb21007cf00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb2200012e0 tx=0x7fb220009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:13.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.184+0000 7fb22cc00640 1 -- 172.21.15.71:0/1557659222 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb230070330 con 0x7fb230161550 2024-09-16T07:37:13.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.370+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb230073e10 con 0x7fb230161550 2024-09-16T07:37:13.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.370+0000 7fb22cc00640 1 -- 172.21.15.71:0/1557659222 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v142) ==== 84+0+65 (secure 0 0 0) 0x7fb228084cd0 con 0x7fb230161550 2024-09-16T07:37:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:13.370 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:37:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-16T07:37:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-16T07:37:13.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:13.370 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:37:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.372+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb21007aa40 msgr2=0x7fb21007cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.372+0000 7fb237400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb21007aa40 0x7fb21007cf00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb2200012e0 tx=0x7fb220009040 comp rx=0 tx=0).stop 2024-09-16T07:37:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230161550 msgr2=0x7fb2300b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230161550 0x7fb2300b08c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb22802aa90 tx=0x7fb228032cb0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 shutdown_connections 2024-09-16T07:37:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb21007aa40 0x7fb21007cf00 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb230161550 0x7fb2300b08c0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300af3d0 0x7fb2300cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 --2- 172.21.15.71:0/1557659222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb230072df0 0x7fb230156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 >> 172.21.15.71:0/1557659222 conn(0x7fb2300b1950 msgr2=0x7fb23015f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 shutdown_connections 2024-09-16T07:37:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.373+0000 7fb237400640 1 -- 172.21.15.71:0/1557659222 wait complete. 2024-09-16T07:37:13.389 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T07:37:13.390 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T07:37:13.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.551+0000 7f8f1be00640 1 -- 172.21.15.71:0/2911780698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f00003c60 msgr2=0x7f8f00004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.551+0000 7f8f1be00640 1 --2- 172.21.15.71:0/2911780698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f00003c60 0x7f8f00004060 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009b50 tx=0x7f8f00005830 comp rx=0 tx=0).stop 2024-09-16T07:37:13.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.551+0000 7f8f1b400640 1 -- 172.21.15.71:0/2911780698 reap_dead start 2024-09-16T07:37:13.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.551+0000 7f8f1be00640 1 -- 172.21.15.71:0/2911780698 shutdown_connections 2024-09-16T07:37:13.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.551+0000 7f8f1be00640 1 -- 172.21.15.71:0/2911780698 >> 172.21.15.71:0/2911780698 conn(0x7f8f1c0cfa70 msgr2=0x7f8f1c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:13.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.551+0000 7f8f1be00640 1 -- 172.21.15.71:0/2911780698 shutdown_connections 2024-09-16T07:37:13.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.551+0000 7f8f1be00640 1 -- 172.21.15.71:0/2911780698 wait complete. 2024-09-16T07:37:13.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.553+0000 7f8f1be00640 1 Processor -- start 2024-09-16T07:37:13.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.553+0000 7f8f1be00640 1 -- start start 2024-09-16T07:37:13.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f00003c60 0x7f8f1c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:13.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f1c0c92b0 0x7f8f1c1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:13.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1c1f4c10 0x7f8f1c1f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:13.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c0734e0 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c073360 con 0x7f8f1c0c92b0 2024-09-16T07:37:13.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1c0cb8d0 con 0x7f8f00003c60 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1c1f4c10 0x7f8f1c1f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1c1f4c10 0x7f8f1c1f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33676/0 (socket says 172.21.15.71:33676) 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1b400640 1 -- 172.21.15.71:0/299550165 learned_addr learned my addr 172.21.15.71:0/299550165 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1aa00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f00003c60 0x7f8f1c15c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.554+0000 7f8f1a000640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f1c0c92b0 0x7f8f1c1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.555+0000 7f8f1b400640 1 -- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f00003c60 msgr2=0x7f8f1c15c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.555+0000 7f8f1b400640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f00003c60 0x7f8f1c15c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.555+0000 7f8f1b400640 1 -- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f1c0c92b0 msgr2=0x7f8f1c1f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.555+0000 7f8f1b400640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f1c0c92b0 0x7f8f1c1f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.555+0000 7f8f1b400640 1 -- 172.21.15.71:0/299550165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c15ead0 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.555+0000 7f8f1b400640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1c1f4c10 0x7f8f1c1f50c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0cf9f0 tx=0x7f8f14006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:13.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.556+0000 7f8f12a00640 1 -- 172.21.15.71:0/299550165 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14015070 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.556+0000 7f8f12a00640 1 -- 172.21.15.71:0/299550165 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f14011900 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.556+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c15edb0 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.556+0000 7f8f12a00640 1 -- 172.21.15.71:0/299550165 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14011d30 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.556+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c0c96f0 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.557+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1c0cbe60 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.558+0000 7f8f12a00640 1 -- 172.21.15.71:0/299550165 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f8f140103c0 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.560+0000 7f8f12a00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8efc07aa10 0x7f8efc07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:13.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.560+0000 7f8f12a00640 1 -- 172.21.15.71:0/299550165 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4941+0+0 (secure 0 0 0) 0x7f8f1409a5f0 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.561+0000 7f8f1aa00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8efc07aa10 0x7f8efc07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:13.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.562+0000 7f8f1aa00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8efc07aa10 0x7f8efc07ced0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c035000 tx=0x7f8f0c02d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:13.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.569+0000 7f8f12a00640 1 -- 172.21.15.71:0/299550165 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8f14064110 con 0x7f8f1c1f4c10 2024-09-16T07:37:13.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:13.755+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8f1c0cd0e0 con 0x7f8f1c1f4c10 2024-09-16T07:37:14.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.572+0000 7f8f12a00640 1 -- 172.21.15.71:0/299550165 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v143) ==== 138+0+0 (secure 0 0 0) 0x7f8f14068b40 con 0x7f8f1c1f4c10 2024-09-16T07:37:14.572 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-16T07:37:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8efc07aa10 msgr2=0x7f8efc07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8efc07aa10 0x7f8efc07ced0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c035000 tx=0x7f8f0c02d040 comp rx=0 tx=0).stop 2024-09-16T07:37:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1c1f4c10 msgr2=0x7f8f1c1f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1c1f4c10 0x7f8f1c1f50c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0cf9f0 tx=0x7f8f14006430 comp rx=0 tx=0).stop 2024-09-16T07:37:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 shutdown_connections 2024-09-16T07:37:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8efc07aa10 0x7f8efc07ced0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f1c1f4c10 0x7f8f1c1f50c0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f1c0c92b0 0x7f8f1c1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 --2- 172.21.15.71:0/299550165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f00003c60 0x7f8f1c15c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.575+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 >> 172.21.15.71:0/299550165 conn(0x7f8f1c0cfa70 msgr2=0x7f8f1c072450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:14.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.576+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 shutdown_connections 2024-09-16T07:37:14.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.576+0000 7f8f1be00640 1 -- 172.21.15.71:0/299550165 wait complete. 2024-09-16T07:37:14.596 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T07:37:14.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 -- 172.21.15.71:0/2816222833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e80b4400 msgr2=0x7fe9e815db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2816222833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e80b4400 0x7fe9e815db50 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80d0cd0 tx=0x7fe9e002aa30 comp rx=0 tx=0).stop 2024-09-16T07:37:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 -- 172.21.15.71:0/2816222833 shutdown_connections 2024-09-16T07:37:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2816222833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e80b4400 0x7fe9e815db50 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2816222833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9e8070870 0x7fe9e80b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2816222833 >> v2:172.21.15.177:3300/0 conn(0x7fe9e80b4bd0 0x7fe9e81437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 -- 172.21.15.71:0/2816222833 >> 172.21.15.71:0/2816222833 conn(0x7fe9e80b1840 msgr2=0x7fe9e80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 -- 172.21.15.71:0/2816222833 shutdown_connections 2024-09-16T07:37:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.763+0000 7fe9eea00640 1 -- 172.21.15.71:0/2816222833 wait complete. 2024-09-16T07:37:14.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.764+0000 7fe9eea00640 1 Processor -- start 2024-09-16T07:37:14.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.764+0000 7fe9eea00640 1 -- start start 2024-09-16T07:37:14.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.765+0000 7fe9eea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8070870 0x7fe9e8138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:14.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.765+0000 7fe9eea00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9e80b4400 0x7fe9e80aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:14.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.765+0000 7fe9eea00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9e80b4bd0 0x7fe9e806f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:14.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.765+0000 7fe9eea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e815e090 con 0x7fe9e8070870 2024-09-16T07:37:14.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.765+0000 7fe9eea00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e80ce080 con 0x7fe9e80b4400 2024-09-16T07:37:14.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.765+0000 7fe9eea00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e815e210 con 0x7fe9e80b4bd0 2024-09-16T07:37:14.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8070870 0x7fe9e8138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8070870 0x7fe9e8138360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57000/0 (socket says 172.21.15.71:57000) 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 -- 172.21.15.71:0/2173874292 learned_addr learned my addr 172.21.15.71:0/2173874292 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7400640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9e80b4400 0x7fe9e80aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9eca00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9e80b4bd0 0x7fe9e806f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 -- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9e80b4bd0 msgr2=0x7fe9e806f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9e80b4bd0 0x7fe9e806f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 -- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9e80b4400 msgr2=0x7fe9e80aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9e80b4400 0x7fe9e80aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 -- 172.21.15.71:0/2173874292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e8138d90 con 0x7fe9e8070870 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7400640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9e80b4400 0x7fe9e80aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.766+0000 7fe9e7e00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8070870 0x7fe9e8138360 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800e790 tx=0x7fe9d800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.767+0000 7fe9dfe00640 1 -- 172.21.15.71:0/2173874292 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d800eea0 con 0x7fe9e8070870 2024-09-16T07:37:14.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.767+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e8139070 con 0x7fe9e8070870 2024-09-16T07:37:14.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.767+0000 7fe9dfe00640 1 -- 172.21.15.71:0/2173874292 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9d800cd50 con 0x7fe9e8070870 2024-09-16T07:37:14.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.767+0000 7fe9dfe00640 1 -- 172.21.15.71:0/2173874292 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8010690 con 0x7fe9e8070870 2024-09-16T07:37:14.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.767+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e8139630 con 0x7fe9e8070870 2024-09-16T07:37:14.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.768+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e8139a60 con 0x7fe9e8070870 2024-09-16T07:37:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.770+0000 7fe9dfe00640 1 -- 172.21.15.71:0/2173874292 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fe9d8010830 con 0x7fe9e8070870 2024-09-16T07:37:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.771+0000 7fe9dfe00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9c807aa10 0x7fe9c807ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.771+0000 7fe9dfe00640 1 -- 172.21.15.71:0/2173874292 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5311+0+0 (secure 0 0 0) 0x7fe9d809acb0 con 0x7fe9e8070870 2024-09-16T07:37:14.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.771+0000 7fe9e7400640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9c807aa10 0x7fe9c807ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.772+0000 7fe9e7400640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9c807aa10 0x7fe9c807ced0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8070460 tx=0x7fe9d4005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:14.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.774+0000 7fe9dfe00640 1 -- 172.21.15.71:0/2173874292 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe9d8064e00 con 0x7fe9e8070870 2024-09-16T07:37:14.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:14.958+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe9e815e6c0 con 0x7fe9e8070870 2024-09-16T07:37:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.579+0000 7fe9dfe00640 1 -- 172.21.15.71:0/2173874292 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v144) ==== 170+0+0 (secure 0 0 0) 0x7fe9d8064700 con 0x7fe9e8070870 2024-09-16T07:37:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-16T07:37:15.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.582+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9c807aa10 msgr2=0x7fe9c807ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:15.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.582+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9c807aa10 0x7fe9c807ced0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8070460 tx=0x7fe9d4005f90 comp rx=0 tx=0).stop 2024-09-16T07:37:15.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.582+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8070870 msgr2=0x7fe9e8138360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.582+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8070870 0x7fe9e8138360 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800e790 tx=0x7fe9d800ec50 comp rx=0 tx=0).stop 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.583+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 shutdown_connections 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.583+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe9c807aa10 0x7fe9c807ced0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.583+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9e80b4bd0 0x7fe9e806f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.583+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9e80b4400 0x7fe9e80aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.583+0000 7fe9eea00640 1 --2- 172.21.15.71:0/2173874292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8070870 0x7fe9e8138360 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.583+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 >> 172.21.15.71:0/2173874292 conn(0x7fe9e80b1840 msgr2=0x7fe9e81429c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.583+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 shutdown_connections 2024-09-16T07:37:15.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.583+0000 7fe9eea00640 1 -- 172.21.15.71:0/2173874292 wait complete. 2024-09-16T07:37:15.614 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 -- 172.21.15.71:0/1417677266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c06efa0 msgr2=0x7f4a1c0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 --2- 172.21.15.71:0/1417677266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c06efa0 0x7f4a1c0cc5c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0d0cd0 tx=0x7f4a0402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 -- 172.21.15.71:0/1417677266 shutdown_connections 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 --2- 172.21.15.71:0/1417677266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c0ccb00 0x7f4a1c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 --2- 172.21.15.71:0/1417677266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c06efa0 0x7f4a1c0cc5c0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 --2- 172.21.15.71:0/1417677266 >> v2:172.21.15.177:3300/0 conn(0x7f4a1c145750 0x7f4a1c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 -- 172.21.15.71:0/1417677266 >> 172.21.15.71:0/1417677266 conn(0x7f4a1c0b1840 msgr2=0x7f4a1c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 -- 172.21.15.71:0/1417677266 shutdown_connections 2024-09-16T07:37:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.775+0000 7f4a21200640 1 -- 172.21.15.71:0/1417677266 wait complete. 2024-09-16T07:37:15.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.776+0000 7f4a21200640 1 Processor -- start 2024-09-16T07:37:15.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.777+0000 7f4a21200640 1 -- start start 2024-09-16T07:37:15.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.777+0000 7f4a21200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c06efa0 0x7f4a1c1f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:15.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.777+0000 7f4a21200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c0ccb00 0x7f4a1c1438b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:15.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.777+0000 7f4a21200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c145750 0x7f4a1c143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.777+0000 7f4a21200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1602c0 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.777+0000 7f4a21200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c160140 con 0x7f4a1c06efa0 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.777+0000 7f4a21200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c160440 con 0x7f4a1c145750 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c0ccb00 0x7f4a1c1438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a13e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c0ccb00 0x7f4a1c1438b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57014/0 (socket says 172.21.15.71:57014) 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a13e00640 1 -- 172.21.15.71:0/193521891 learned_addr learned my addr 172.21.15.71:0/193521891 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a19e00640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c06efa0 0x7f4a1c1f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a1a800640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c145750 0x7f4a1c143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a13e00640 1 -- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c145750 msgr2=0x7f4a1c143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a13e00640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c145750 0x7f4a1c143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a13e00640 1 -- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c06efa0 msgr2=0x7f4a1c1f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a13e00640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c06efa0 0x7f4a1c1f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a13e00640 1 -- 172.21.15.71:0/193521891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c0cfa10 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.778+0000 7f4a19e00640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c06efa0 0x7f4a1c1f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.779+0000 7f4a13e00640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c0ccb00 0x7f4a1c1438b0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009850 tx=0x7f4a040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.779+0000 7f4a12000640 1 -- 172.21.15.71:0/193521891 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0403f070 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.779+0000 7f4a12000640 1 -- 172.21.15.71:0/193521891 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a040047a0 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.779+0000 7f4a12000640 1 -- 172.21.15.71:0/193521891 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04035030 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.779+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c0cfcf0 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.779+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c0d02b0 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.780+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c15f9f0 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.781+0000 7f4a12000640 1 -- 172.21.15.71:0/193521891 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f4a04042070 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.782+0000 7f4a12000640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49f807aa40 0x7f49f807cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:15.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.782+0000 7f4a12000640 1 -- 172.21.15.71:0/193521891 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5693+0+0 (secure 0 0 0) 0x7f4a040ba620 con 0x7f4a1c0ccb00 2024-09-16T07:37:15.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.783+0000 7f4a19e00640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49f807aa40 0x7f49f807cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:15.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.784+0000 7f4a19e00640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49f807aa40 0x7f49f807cf00 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c06ed70 tx=0x7f4a0c005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:15.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:15.787+0000 7f4a12000640 1 -- 172.21.15.71:0/193521891 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4a040840a0 con 0x7f4a1c0ccb00 2024-09-16T07:37:16.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.009+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4a1c0af260 con 0x7f4a1c0ccb00 2024-09-16T07:37:16.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.582+0000 7f4a12000640 1 -- 172.21.15.71:0/193521891 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v91) ==== 174+0+0 (secure 0 0 0) 0x7f4a04088ad0 con 0x7f4a1c0ccb00 2024-09-16T07:37:16.581 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 20 and data pool 21 2024-09-16T07:37:16.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49f807aa40 msgr2=0x7f49f807cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49f807aa40 0x7f49f807cf00 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c06ed70 tx=0x7f4a0c005f90 comp rx=0 tx=0).stop 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c0ccb00 msgr2=0x7f4a1c1438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c0ccb00 0x7f4a1c1438b0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009850 tx=0x7f4a040038a0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 shutdown_connections 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49f807aa40 0x7f49f807cf00 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c145750 0x7f4a1c143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a1c0ccb00 0x7f4a1c1438b0 secure :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009850 tx=0x7f4a040038a0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 --2- 172.21.15.71:0/193521891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c06efa0 0x7f4a1c1f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.587+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 >> 172.21.15.71:0/193521891 conn(0x7f4a1c0b1840 msgr2=0x7f4a1c0b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:16.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.588+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 shutdown_connections 2024-09-16T07:37:16.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.590+0000 7f4a21200640 1 -- 172.21.15.71:0/193521891 wait complete. 2024-09-16T07:37:16.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:37:16.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.785+0000 7efe8fe00640 1 -- 172.21.15.71:0/3774179944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe9006f770 msgr2=0x7efe9006fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:16.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.785+0000 7efe8fe00640 1 --2- 172.21.15.71:0/3774179944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe9006f770 0x7efe9006fba0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efe88005fd0 tx=0x7efe8802a940 comp rx=0 tx=0).stop 2024-09-16T07:37:16.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.785+0000 7efe8fe00640 1 -- 172.21.15.71:0/3774179944 shutdown_connections 2024-09-16T07:37:16.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.785+0000 7efe8fe00640 1 --2- 172.21.15.71:0/3774179944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe9006f770 0x7efe9006fba0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.785+0000 7efe8fe00640 1 -- 172.21.15.71:0/3774179944 >> 172.21.15.71:0/3774179944 conn(0x7efe900cfa20 msgr2=0x7efe900cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:16.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.785+0000 7efe8fe00640 1 -- 172.21.15.71:0/3774179944 shutdown_connections 2024-09-16T07:37:16.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.785+0000 7efe8fe00640 1 -- 172.21.15.71:0/3774179944 wait complete. 2024-09-16T07:37:16.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.787+0000 7efe8fe00640 1 Processor -- start 2024-09-16T07:37:16.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.787+0000 7efe8fe00640 1 -- start start 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8fe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe9006f770 0x7efe900a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8fe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe900cbdc0 0x7efe900c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9015a620 0x7efe901f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efe90070840 con 0x7efe9015a620 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8fe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efe900706c0 con 0x7efe9006f770 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8fe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efe900cd5f0 con 0x7efe900cbdc0 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9015a620 0x7efe901f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9015a620 0x7efe901f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57030/0 (socket says 172.21.15.71:57030) 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8f400640 1 -- 172.21.15.71:0/2784199498 learned_addr learned my addr 172.21.15.71:0/2784199498 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:16.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8ea00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe9006f770 0x7efe900a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:16.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.788+0000 7efe8e000640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe900cbdc0 0x7efe900c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe8f400640 1 -- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe900cbdc0 msgr2=0x7efe900c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe8f400640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe900cbdc0 0x7efe900c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe8f400640 1 -- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe9006f770 msgr2=0x7efe900a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe8f400640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe9006f770 0x7efe900a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe8f400640 1 -- 172.21.15.71:0/2784199498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe901f3570 con 0x7efe9015a620 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe8f400640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9015a620 0x7efe901f5cc0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7efe88006010 tx=0x7efe88031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe86a00640 1 -- 172.21.15.71:0/2784199498 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88030430 con 0x7efe9015a620 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe86a00640 1 -- 172.21.15.71:0/2784199498 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe88004240 con 0x7efe9015a620 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe86a00640 1 -- 172.21.15.71:0/2784199498 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88004650 con 0x7efe9015a620 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.789+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe900c8cf0 con 0x7efe9015a620 2024-09-16T07:37:16.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.790+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe900c92b0 con 0x7efe9015a620 2024-09-16T07:37:16.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.791+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe900cda90 con 0x7efe9015a620 2024-09-16T07:37:16.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.792+0000 7efe86a00640 1 -- 172.21.15.71:0/2784199498 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7efe88039440 con 0x7efe9015a620 2024-09-16T07:37:16.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.793+0000 7efe86a00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe7007aa10 0x7efe7007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:16.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.793+0000 7efe86a00640 1 -- 172.21.15.71:0/2784199498 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7efe880bb1c0 con 0x7efe9015a620 2024-09-16T07:37:16.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.793+0000 7efe8ea00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe7007aa10 0x7efe7007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:16.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.794+0000 7efe8ea00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe7007aa10 0x7efe7007ced0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efe80000f60 tx=0x7efe80009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:16.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.797+0000 7efe86a00640 1 -- 172.21.15.71:0/2784199498 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efe88084ce0 con 0x7efe9015a620 2024-09-16T07:37:16.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:37:16.962 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:37:16.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.982+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe900af010 con 0x7efe9015a620 2024-09-16T07:37:16.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.983+0000 7efe86a00640 1 -- 172.21.15.71:0/2784199498 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16897 (secure 0 0 0) 0x7efe88089710 con 0x7efe9015a620 2024-09-16T07:37:16.984 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:16.984 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":145,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:37:16.010498+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:37:13.765661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T07:37:14.968618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:37:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe7007aa10 msgr2=0x7efe7007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe7007aa10 0x7efe7007ced0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efe80000f60 tx=0x7efe80009040 comp rx=0 tx=0).stop 2024-09-16T07:37:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9015a620 msgr2=0x7efe901f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9015a620 0x7efe901f5cc0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7efe88006010 tx=0x7efe88031e20 comp rx=0 tx=0).stop 2024-09-16T07:37:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 shutdown_connections 2024-09-16T07:37:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7efe7007aa10 0x7efe7007ced0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9015a620 0x7efe901f5cc0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe900cbdc0 0x7efe900c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 --2- 172.21.15.71:0/2784199498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe9006f770 0x7efe900a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 >> 172.21.15.71:0/2784199498 conn(0x7efe900cfa20 msgr2=0x7efe900c6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 shutdown_connections 2024-09-16T07:37:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:16.986+0000 7efe8fe00640 1 -- 172.21.15.71:0/2784199498 wait complete. 2024-09-16T07:37:17.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:37:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.199+0000 7ff436c00640 1 -- 172.21.15.71:0/1986496980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff414004d40 msgr2=0x7ff4140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.199+0000 7ff436c00640 1 --2- 172.21.15.71:0/1986496980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff414004d40 0x7ff4140051c0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff428009ed0 tx=0x7ff414005830 comp rx=0 tx=0).stop 2024-09-16T07:37:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.199+0000 7ff434c00640 1 -- 172.21.15.71:0/1986496980 reap_dead start 2024-09-16T07:37:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.199+0000 7ff436c00640 1 -- 172.21.15.71:0/1986496980 shutdown_connections 2024-09-16T07:37:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.199+0000 7ff436c00640 1 -- 172.21.15.71:0/1986496980 >> 172.21.15.71:0/1986496980 conn(0x7ff4300b1950 msgr2=0x7ff4300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.199+0000 7ff436c00640 1 -- 172.21.15.71:0/1986496980 shutdown_connections 2024-09-16T07:37:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.199+0000 7ff436c00640 1 -- 172.21.15.71:0/1986496980 wait complete. 2024-09-16T07:37:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.201+0000 7ff436c00640 1 Processor -- start 2024-09-16T07:37:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.201+0000 7ff436c00640 1 -- start start 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff436c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff414004d40 0x7ff4301fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff436c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff430154690 0x7ff43014c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff436c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4301f8fc0 0x7ff430150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff436c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff430166f80 con 0x7ff4301f8fc0 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff436c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff430166e00 con 0x7ff430154690 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff436c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff430167100 con 0x7ff414004d40 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff434c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4301f8fc0 0x7ff430150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff434c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4301f8fc0 0x7ff430150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57042/0 (socket says 172.21.15.71:57042) 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff434c00640 1 -- 172.21.15.71:0/2841385488 learned_addr learned my addr 172.21.15.71:0/2841385488 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff42fe00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff414004d40 0x7ff4301fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff427400640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff430154690 0x7ff43014c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff434c00640 1 -- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff414004d40 msgr2=0x7ff4301fbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff434c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff414004d40 0x7ff4301fbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff434c00640 1 -- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff430154690 msgr2=0x7ff43014c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff434c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff430154690 0x7ff43014c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.202+0000 7ff434c00640 1 -- 172.21.15.71:0/2841385488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff430155830 con 0x7ff4301f8fc0 2024-09-16T07:37:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.203+0000 7ff434c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4301f8fc0 0x7ff430150740 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff428036040 tx=0x7ff428033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.203+0000 7ff42cc00640 1 -- 172.21.15.71:0/2841385488 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428038040 con 0x7ff4301f8fc0 2024-09-16T07:37:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.203+0000 7ff42cc00640 1 -- 172.21.15.71:0/2841385488 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff42802bc30 con 0x7ff4301f8fc0 2024-09-16T07:37:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.203+0000 7ff42cc00640 1 -- 172.21.15.71:0/2841385488 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428004d80 con 0x7ff4301f8fc0 2024-09-16T07:37:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.203+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4301fc510 con 0x7ff4301f8fc0 2024-09-16T07:37:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.203+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4301fcad0 con 0x7ff4301f8fc0 2024-09-16T07:37:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.204+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43006ef50 con 0x7ff4301f8fc0 2024-09-16T07:37:17.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.205+0000 7ff42cc00640 1 -- 172.21.15.71:0/2841385488 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7ff428043030 con 0x7ff4301f8fc0 2024-09-16T07:37:17.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.206+0000 7ff42cc00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40c07aa10 0x7ff40c07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:17.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.206+0000 7ff42cc00640 1 -- 172.21.15.71:0/2841385488 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7ff4280b9f70 con 0x7ff4301f8fc0 2024-09-16T07:37:17.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.207+0000 7ff42fe00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40c07aa10 0x7ff40c07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:17.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.208+0000 7ff42fe00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40c07aa10 0x7ff40c07ced0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff420006fd0 tx=0x7ff420006b10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:17.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.210+0000 7ff42cc00640 1 -- 172.21.15.71:0/2841385488 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff42804a020 con 0x7ff4301f8fc0 2024-09-16T07:37:17.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.395+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4300ad5d0 con 0x7ff4301f8fc0 2024-09-16T07:37:17.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.396+0000 7ff42cc00640 1 -- 172.21.15.71:0/2841385488 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16897 (secure 0 0 0) 0x7ff4280839f0 con 0x7ff4301f8fc0 2024-09-16T07:37:17.396 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:17.396 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":145,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:37:16.010498+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:37:13.765661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T07:37:14.968618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:37:17.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40c07aa10 msgr2=0x7ff40c07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40c07aa10 0x7ff40c07ced0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff420006fd0 tx=0x7ff420006b10 comp rx=0 tx=0).stop 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4301f8fc0 msgr2=0x7ff430150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4301f8fc0 0x7ff430150740 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff428036040 tx=0x7ff428033c30 comp rx=0 tx=0).stop 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 shutdown_connections 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40c07aa10 0x7ff40c07ced0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4301f8fc0 0x7ff430150740 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff430154690 0x7ff43014c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 --2- 172.21.15.71:0/2841385488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff414004d40 0x7ff4301fbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 >> 172.21.15.71:0/2841385488 conn(0x7ff4300b1950 msgr2=0x7ff43014c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 shutdown_connections 2024-09-16T07:37:17.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.398+0000 7ff436c00640 1 -- 172.21.15.71:0/2841385488 wait complete. 2024-09-16T07:37:17.414 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T07:37:17.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.574+0000 7f5519200640 1 -- 172.21.15.71:0/753821120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f8006320 msgr2=0x7f54f800cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:17.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.574+0000 7f5519200640 1 --2- 172.21.15.71:0/753821120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f8006320 0x7f54f800cad0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f550c005fd0 tx=0x7f54f8006940 comp rx=0 tx=0).stop 2024-09-16T07:37:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.574+0000 7f5519200640 1 -- 172.21.15.71:0/753821120 shutdown_connections 2024-09-16T07:37:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.574+0000 7f5519200640 1 --2- 172.21.15.71:0/753821120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f8006320 0x7f54f800cad0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.574+0000 7f5519200640 1 -- 172.21.15.71:0/753821120 >> 172.21.15.71:0/753821120 conn(0x7f55140b1840 msgr2=0x7f55140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.574+0000 7f5519200640 1 -- 172.21.15.71:0/753821120 shutdown_connections 2024-09-16T07:37:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.574+0000 7f5519200640 1 -- 172.21.15.71:0/753821120 wait complete. 2024-09-16T07:37:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.576+0000 7f5519200640 1 Processor -- start 2024-09-16T07:37:17.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.576+0000 7f5519200640 1 -- start start 2024-09-16T07:37:17.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5519200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8006320 0x7f55141476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:17.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5519200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415c590 0x7f551415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5519200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55140ce080 0x7f55140ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5519200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55140cc9e0 con 0x7f55140ce080 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5519200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55140cc860 con 0x7f54f8006320 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5519200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55140ccb60 con 0x7f551415c590 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5512800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55140ce080 0x7f55140ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5512800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55140ce080 0x7f55140ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57062/0 (socket says 172.21.15.71:57062) 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.577+0000 7f5512800640 1 -- 172.21.15.71:0/4221488031 learned_addr learned my addr 172.21.15.71:0/4221488031 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f5512800640 1 -- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415c590 msgr2=0x7f551415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f5512800640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415c590 0x7f551415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f5512800640 1 -- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8006320 msgr2=0x7f55141476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f5512800640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8006320 0x7f55141476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f5512800640 1 -- 172.21.15.71:0/4221488031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551414f6a0 con 0x7f55140ce080 2024-09-16T07:37:17.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f5512800640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55140ce080 0x7f55140ce500 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f550c009e20 tx=0x7f550c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:17.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f550a000640 1 -- 172.21.15.71:0/4221488031 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c009200 con 0x7f55140ce080 2024-09-16T07:37:17.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f550a000640 1 -- 172.21.15.71:0/4221488031 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f550c041070 con 0x7f55140ce080 2024-09-16T07:37:17.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f550a000640 1 -- 172.21.15.71:0/4221488031 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c009ad0 con 0x7f55140ce080 2024-09-16T07:37:17.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551415d100 con 0x7f55140ce080 2024-09-16T07:37:17.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.578+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551415d6c0 con 0x7f55140ce080 2024-09-16T07:37:17.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.579+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5514073430 con 0x7f55140ce080 2024-09-16T07:37:17.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.581+0000 7f550a000640 1 -- 172.21.15.71:0/4221488031 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f550c046460 con 0x7f55140ce080 2024-09-16T07:37:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.581+0000 7f550a000640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f407aa40 0x7f54f407cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.582+0000 7f550a000640 1 -- 172.21.15.71:0/4221488031 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f550c0baa20 con 0x7f55140ce080 2024-09-16T07:37:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.582+0000 7f5511e00640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f407aa40 0x7f54f407cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:17.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.583+0000 7f5511e00640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f407aa40 0x7f54f407cf00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5504006b30 tx=0x7f5504000f10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:17.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.585+0000 7f550a000640 1 -- 172.21.15.71:0/4221488031 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f550c084a60 con 0x7f55140ce080 2024-09-16T07:37:17.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:17.815+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f55140cd190 con 0x7f55140ce080 2024-09-16T07:37:18.561 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-09-16T07:37:18.561+0000 7fd303000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T07:37:18.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.607+0000 7f550a000640 1 -- 172.21.15.71:0/4221488031 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v94) ==== 118+0+0 (secure 0 0 0) 0x7f550c084360 con 0x7f55140ce080 2024-09-16T07:37:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.609+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f407aa40 msgr2=0x7f54f407cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.609+0000 7f5519200640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f407aa40 0x7f54f407cf00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5504006b30 tx=0x7f5504000f10 comp rx=0 tx=0).stop 2024-09-16T07:37:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.609+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55140ce080 msgr2=0x7f55140ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.609+0000 7f5519200640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55140ce080 0x7f55140ce500 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f550c009e20 tx=0x7f550c0042d0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.610+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 shutdown_connections 2024-09-16T07:37:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.610+0000 7f5519200640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f54f407aa40 0x7f54f407cf00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.610+0000 7f5519200640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55140ce080 0x7f55140ce500 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.610+0000 7f5519200640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415c590 0x7f551415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.610+0000 7f5519200640 1 --2- 172.21.15.71:0/4221488031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54f8006320 0x7f55141476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.610+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 >> 172.21.15.71:0/4221488031 conn(0x7f55140b1840 msgr2=0x7f551414b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.610+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 shutdown_connections 2024-09-16T07:37:18.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.610+0000 7f5519200640 1 -- 172.21.15.71:0/4221488031 wait complete. 2024-09-16T07:37:18.627 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T07:37:18.627 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T07:37:18.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.788+0000 7f33bd000640 1 -- 172.21.15.71:0/9045885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33b8161760 msgr2=0x7f33b80ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.788+0000 7f33bd000640 1 --2- 172.21.15.71:0/9045885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33b8161760 0x7f33b80ce4a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005fd0 tx=0x7f33a802a540 comp rx=0 tx=0).stop 2024-09-16T07:37:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.789+0000 7f33bd000640 1 -- 172.21.15.71:0/9045885 shutdown_connections 2024-09-16T07:37:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.789+0000 7f33bd000640 1 --2- 172.21.15.71:0/9045885 >> v2:172.21.15.177:3300/0 conn(0x7f33b80cec60 0x7f33b80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.789+0000 7f33bd000640 1 --2- 172.21.15.71:0/9045885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33b8161760 0x7f33b80ce4a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.789+0000 7f33bd000640 1 -- 172.21.15.71:0/9045885 >> 172.21.15.71:0/9045885 conn(0x7f33b80b1840 msgr2=0x7f33b80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.789+0000 7f33bd000640 1 -- 172.21.15.71:0/9045885 shutdown_connections 2024-09-16T07:37:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.789+0000 7f33bd000640 1 -- 172.21.15.71:0/9045885 wait complete. 2024-09-16T07:37:18.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.790+0000 7f33bd000640 1 Processor -- start 2024-09-16T07:37:18.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.790+0000 7f33bd000640 1 -- start start 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33bd000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33b80af190 0x7f33b8147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33bd000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33b80cec60 0x7f33b81f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33bd000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33b8161760 0x7f33b815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33bd000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8162190 con 0x7f33b80af190 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33bd000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8162310 con 0x7f33b8161760 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33bd000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f33b8162490 con 0x7f33b80cec60 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33b5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33b80af190 0x7f33b8147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33b5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33b80af190 0x7f33b8147de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57076/0 (socket says 172.21.15.71:57076) 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.791+0000 7f33b5e00640 1 -- 172.21.15.71:0/1349974683 learned_addr learned my addr 172.21.15.71:0/1349974683 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33b6800640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33b8161760 0x7f33b815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33b5400640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33b80cec60 0x7f33b81f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33b5e00640 1 -- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33b80cec60 msgr2=0x7f33b81f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33b5e00640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33b80cec60 0x7f33b81f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33b5e00640 1 -- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33b8161760 msgr2=0x7f33b815aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33b5e00640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33b8161760 0x7f33b815aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33b5e00640 1 -- 172.21.15.71:0/1349974683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b8148320 con 0x7f33b80af190 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33b5e00640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33b80af190 0x7f33b8147de0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f33a8034040 tx=0x7f33a80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.792+0000 7f33ae000640 1 -- 172.21.15.71:0/1349974683 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a803e030 con 0x7f33b80af190 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.793+0000 7f33ae000640 1 -- 172.21.15.71:0/1349974683 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33a8003d90 con 0x7f33b80af190 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.793+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b81f23b0 con 0x7f33b80af190 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.793+0000 7f33ae000640 1 -- 172.21.15.71:0/1349974683 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8035040 con 0x7f33b80af190 2024-09-16T07:37:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.793+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b81f51f0 con 0x7f33b80af190 2024-09-16T07:37:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.794+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b8162910 con 0x7f33b80af190 2024-09-16T07:37:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.795+0000 7f33ae000640 1 -- 172.21.15.71:0/1349974683 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f33a8041030 con 0x7f33b80af190 2024-09-16T07:37:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.796+0000 7f33ae000640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f339807aa40 0x7f339807cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.796+0000 7f33ae000640 1 -- 172.21.15.71:0/1349974683 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f33a80ba3d0 con 0x7f33b80af190 2024-09-16T07:37:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.796+0000 7f33b5400640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f339807aa40 0x7f339807cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.797+0000 7f33b5400640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f339807aa40 0x7f339807cf00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f33b81f7ec0 tx=0x7f33a4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:18.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:18.800+0000 7f33ae000640 1 -- 172.21.15.71:0/1349974683 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f33b81f23b0 con 0x7f33b80af190 2024-09-16T07:37:19.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.021+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f33b815d650 con 0x7f33b80af190 2024-09-16T07:37:19.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.613+0000 7f33ae000640 1 -- 172.21.15.71:0/1349974683 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v95) ==== 122+0+0 (secure 0 0 0) 0x7f33b8148320 con 0x7f33b80af190 2024-09-16T07:37:19.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f339807aa40 msgr2=0x7f339807cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:19.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f339807aa40 0x7f339807cf00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f33b81f7ec0 tx=0x7f33a4009040 comp rx=0 tx=0).stop 2024-09-16T07:37:19.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33b80af190 msgr2=0x7f33b8147de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:19.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33b80af190 0x7f33b8147de0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f33a8034040 tx=0x7f33a80038a0 comp rx=0 tx=0).stop 2024-09-16T07:37:19.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 shutdown_connections 2024-09-16T07:37:19.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f339807aa40 0x7f339807cf00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:19.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33b8161760 0x7f33b815aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:19.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33b80cec60 0x7f33b81f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:19.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 --2- 172.21.15.71:0/1349974683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33b80af190 0x7f33b8147de0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:19.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 >> 172.21.15.71:0/1349974683 conn(0x7f33b80b1840 msgr2=0x7f33b80afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:19.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 shutdown_connections 2024-09-16T07:37:19.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.616+0000 7f33bd000640 1 -- 172.21.15.71:0/1349974683 wait complete. 2024-09-16T07:37:19.633 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T07:37:19.634 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:37:19.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.792+0000 7ff072e00640 1 -- 172.21.15.71:0/3140419263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 msgr2=0x7ff050003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:19.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.792+0000 7ff072e00640 1 --2- 172.21.15.71:0/3140419263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 0x7ff050003ec0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff05c001da0 tx=0x7ff050005830 comp rx=0 tx=0).stop 2024-09-16T07:37:19.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.792+0000 7ff072e00640 1 -- 172.21.15.71:0/3140419263 shutdown_connections 2024-09-16T07:37:19.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.792+0000 7ff072e00640 1 --2- 172.21.15.71:0/3140419263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 0x7ff050003ec0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.792+0000 7ff072e00640 1 -- 172.21.15.71:0/3140419263 >> 172.21.15.71:0/3140419263 conn(0x7ff06c0b1950 msgr2=0x7ff06c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.792+0000 7ff072e00640 1 -- 172.21.15.71:0/3140419263 shutdown_connections 2024-09-16T07:37:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.792+0000 7ff072e00640 1 -- 172.21.15.71:0/3140419263 wait complete. 2024-09-16T07:37:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.794+0000 7ff072e00640 1 Processor -- start 2024-09-16T07:37:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.794+0000 7ff072e00640 1 -- start start 2024-09-16T07:37:19.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff072e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 0x7ff06c150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:19.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff072e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff06c152760 0x7ff06c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff072e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff06c1f7dc0 0x7ff06c1f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff072e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c074420 con 0x7ff050003aa0 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff072e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c0b2700 con 0x7ff06c152760 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff072e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff06c0745a0 con 0x7ff06c1f7dc0 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 0x7ff06c150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 0x7ff06c150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57096/0 (socket says 172.21.15.71:57096) 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06be00640 1 -- 172.21.15.71:0/4182543108 learned_addr learned my addr 172.21.15.71:0/4182543108 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff070e00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff06c1f7dc0 0x7ff06c1f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06b400640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff06c152760 0x7ff06c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06be00640 1 -- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff06c1f7dc0 msgr2=0x7ff06c1f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06be00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff06c1f7dc0 0x7ff06c1f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06be00640 1 -- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff06c152760 msgr2=0x7ff06c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06be00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff06c152760 0x7ff06c152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.795+0000 7ff06be00640 1 -- 172.21.15.71:0/4182543108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c153300 con 0x7ff050003aa0 2024-09-16T07:37:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.796+0000 7ff06be00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 0x7ff06c150e70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff05c000f80 tx=0x7ff05c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.796+0000 7ff063e00640 1 -- 172.21.15.71:0/4182543108 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c006460 con 0x7ff050003aa0 2024-09-16T07:37:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.796+0000 7ff063e00640 1 -- 172.21.15.71:0/4182543108 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff05c006600 con 0x7ff050003aa0 2024-09-16T07:37:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.796+0000 7ff063e00640 1 -- 172.21.15.71:0/4182543108 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c003e20 con 0x7ff050003aa0 2024-09-16T07:37:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.796+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c1535e0 con 0x7ff050003aa0 2024-09-16T07:37:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.796+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c153ba0 con 0x7ff050003aa0 2024-09-16T07:37:19.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.797+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06c0b0840 con 0x7ff050003aa0 2024-09-16T07:37:19.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.798+0000 7ff063e00640 1 -- 172.21.15.71:0/4182543108 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7ff05c03d070 con 0x7ff050003aa0 2024-09-16T07:37:19.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.799+0000 7ff063e00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff04c07aa40 0x7ff04c07cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:19.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.799+0000 7ff063e00640 1 -- 172.21.15.71:0/4182543108 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7ff05c0b9c70 con 0x7ff050003aa0 2024-09-16T07:37:19.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.800+0000 7ff06b400640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff04c07aa40 0x7ff04c07cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:19.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.800+0000 7ff06b400640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff04c07aa40 0x7ff04c07cf00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff06c152550 tx=0x7ff058005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:19.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:19.803+0000 7ff063e00640 1 -- 172.21.15.71:0/4182543108 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff06c0b0840 con 0x7ff050003aa0 2024-09-16T07:37:20.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.020+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff06c0adeb0 con 0x7ff050003aa0 2024-09-16T07:37:20.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.021+0000 7ff063e00640 1 -- 172.21.15.71:0/4182543108 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5230 (secure 0 0 0) 0x7ff05c083800 con 0x7ff050003aa0 2024-09-16T07:37:20.021 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:20.021 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":95,"btime":"2024-09-16T07:37:19:608461+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/343939477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":343939477},{"type":"v1","addr":"172.21.15.71:6835","nonce":343939477}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1796926972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1796926972},{"type":"v1","addr":"172.21.15.177:6836","nonce":1796926972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3638818185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3638818185},{"type":"v1","addr":"172.21.15.71:6834","nonce":3638818185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:37:16.010465+0000","modified":"2024-09-16T07:37:19.022287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1486935311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1486935311},{"type":"v1","addr":"172.21.15.177:6837","nonce":1486935311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9420,"qdb_cluster":[9420]},"id":10}]} 2024-09-16T07:37:20.022 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 95 2024-09-16T07:37:20.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff04c07aa40 msgr2=0x7ff04c07cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff04c07aa40 0x7ff04c07cf00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff06c152550 tx=0x7ff058005f90 comp rx=0 tx=0).stop 2024-09-16T07:37:20.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 msgr2=0x7ff06c150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 0x7ff06c150e70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff05c000f80 tx=0x7ff05c003a80 comp rx=0 tx=0).stop 2024-09-16T07:37:20.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 shutdown_connections 2024-09-16T07:37:20.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff04c07aa40 0x7ff04c07cf00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff06c1f7dc0 0x7ff06c1f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff06c152760 0x7ff06c152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 --2- 172.21.15.71:0/4182543108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050003aa0 0x7ff06c150e70 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.023+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 >> 172.21.15.71:0/4182543108 conn(0x7ff06c0b1950 msgr2=0x7ff06c0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:20.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.024+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 shutdown_connections 2024-09-16T07:37:20.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.024+0000 7ff072e00640 1 -- 172.21.15.71:0/4182543108 wait complete. 2024-09-16T07:37:20.040 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 -- 172.21.15.71:0/4032188568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc0ccf00 msgr2=0x7ffadc0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 --2- 172.21.15.71:0/4032188568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc0ccf00 0x7ffadc0acd30 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ffadc075130 tx=0x7ffac802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 -- 172.21.15.71:0/4032188568 shutdown_connections 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 --2- 172.21.15.71:0/4032188568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffadc0ad270 0x7ffadc073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 --2- 172.21.15.71:0/4032188568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc0ccf00 0x7ffadc0acd30 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 --2- 172.21.15.71:0/4032188568 >> v2:172.21.15.177:3300/0 conn(0x7ffadc15f5a0 0x7ffadc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 -- 172.21.15.71:0/4032188568 >> 172.21.15.71:0/4032188568 conn(0x7ffadc0b1950 msgr2=0x7ffadc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 -- 172.21.15.71:0/4032188568 shutdown_connections 2024-09-16T07:37:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.198+0000 7ffae1a00640 1 -- 172.21.15.71:0/4032188568 wait complete. 2024-09-16T07:37:20.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.199+0000 7ffae1a00640 1 Processor -- start 2024-09-16T07:37:20.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.200+0000 7ffae1a00640 1 -- start start 2024-09-16T07:37:20.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.200+0000 7ffae1a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc0ad270 0x7ffadc1f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.200+0000 7ffae1a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffadc0ccf00 0x7ffadc15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffae1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc15f5a0 0x7ffadc15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffae1a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc074880 con 0x7ffadc15f5a0 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffae1a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc074700 con 0x7ffadc0ad270 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffae1a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffadc06f160 con 0x7ffadc0ccf00 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc15f5a0 0x7ffadc15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc15f5a0 0x7ffadc15af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57106/0 (socket says 172.21.15.71:57106) 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadb400640 1 -- 172.21.15.71:0/64267897 learned_addr learned my addr 172.21.15.71:0/64267897 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffada000640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffadc0ccf00 0x7ffadc15bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadaa00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc0ad270 0x7ffadc1f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadb400640 1 -- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffadc0ccf00 msgr2=0x7ffadc15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadb400640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffadc0ccf00 0x7ffadc15bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadb400640 1 -- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc0ad270 msgr2=0x7ffadc1f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadb400640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc0ad270 0x7ffadc1f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.201+0000 7ffadb400640 1 -- 172.21.15.71:0/64267897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc1f4930 con 0x7ffadc15f5a0 2024-09-16T07:37:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.202+0000 7ffadb400640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc15f5a0 0x7ffadc15af80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ffad400eaa0 tx=0x7ffad400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.202+0000 7ffad2a00640 1 -- 172.21.15.71:0/64267897 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad400ce10 con 0x7ffadc15f5a0 2024-09-16T07:37:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.202+0000 7ffad2a00640 1 -- 172.21.15.71:0/64267897 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffad4004c10 con 0x7ffadc15f5a0 2024-09-16T07:37:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.202+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc15b4c0 con 0x7ffadc15f5a0 2024-09-16T07:37:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.202+0000 7ffad2a00640 1 -- 172.21.15.71:0/64267897 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4005020 con 0x7ffadc15f5a0 2024-09-16T07:37:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.202+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc1f4bd0 con 0x7ffadc15f5a0 2024-09-16T07:37:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.203+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc073fb0 con 0x7ffadc15f5a0 2024-09-16T07:37:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.204+0000 7ffad2a00640 1 -- 172.21.15.71:0/64267897 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7ffad40051c0 con 0x7ffadc15f5a0 2024-09-16T07:37:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.205+0000 7ffad2a00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc07aa10 0x7ffabc07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.205+0000 7ffad2a00640 1 -- 172.21.15.71:0/64267897 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7ffad409a760 con 0x7ffadc15f5a0 2024-09-16T07:37:20.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.205+0000 7ffadaa00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc07aa10 0x7ffabc07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.206+0000 7ffadaa00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc07aa10 0x7ffabc07ced0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009b30 tx=0x7ffacc005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.209+0000 7ffad2a00640 1 -- 172.21.15.71:0/64267897 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffad40641e0 con 0x7ffadc15f5a0 2024-09-16T07:37:20.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.395+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffadc0af950 con 0x7ffadc15f5a0 2024-09-16T07:37:20.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.396+0000 7ffad2a00640 1 -- 172.21.15.71:0/64267897 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16884 (secure 0 0 0) 0x7ffad4068c10 con 0x7ffadc15f5a0 2024-09-16T07:37:20.396 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:20.396 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":146,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:37:17.594029+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:37:13.765661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T07:37:14.968618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc07aa10 msgr2=0x7ffabc07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc07aa10 0x7ffabc07ced0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009b30 tx=0x7ffacc005f90 comp rx=0 tx=0).stop 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc15f5a0 msgr2=0x7ffadc15af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc15f5a0 0x7ffadc15af80 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ffad400eaa0 tx=0x7ffad400ef60 comp rx=0 tx=0).stop 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 shutdown_connections 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffabc07aa10 0x7ffabc07ced0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc15f5a0 0x7ffadc15af80 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffadc0ccf00 0x7ffadc15bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 --2- 172.21.15.71:0/64267897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc0ad270 0x7ffadc1f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 >> 172.21.15.71:0/64267897 conn(0x7ffadc0b1950 msgr2=0x7ffadc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:20.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 shutdown_connections 2024-09-16T07:37:20.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.399+0000 7ffae1a00640 1 -- 172.21.15.71:0/64267897 wait complete. 2024-09-16T07:37:20.414 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:37:20.415 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:37:20.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.572+0000 7f7869200640 1 -- 172.21.15.71:0/1643120671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7848004800 msgr2=0x7f7848004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.572+0000 7f7869200640 1 --2- 172.21.15.71:0/1643120671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7848004800 0x7f7848004c80 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f78500016c0 tx=0x7f7848005830 comp rx=0 tx=0).stop 2024-09-16T07:37:20.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.572+0000 7f7862800640 1 -- 172.21.15.71:0/1643120671 reap_dead start 2024-09-16T07:37:20.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.573+0000 7f7869200640 1 -- 172.21.15.71:0/1643120671 shutdown_connections 2024-09-16T07:37:20.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.573+0000 7f7869200640 1 -- 172.21.15.71:0/1643120671 >> 172.21.15.71:0/1643120671 conn(0x7f78640b1950 msgr2=0x7f78640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:20.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.573+0000 7f7869200640 1 -- 172.21.15.71:0/1643120671 shutdown_connections 2024-09-16T07:37:20.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.573+0000 7f7869200640 1 -- 172.21.15.71:0/1643120671 wait complete. 2024-09-16T07:37:20.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.574+0000 7f7869200640 1 Processor -- start 2024-09-16T07:37:20.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.574+0000 7f7869200640 1 -- start start 2024-09-16T07:37:20.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.575+0000 7f7869200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7848004800 0x7f78640d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.575+0000 7f7869200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7864145d70 0x7f78641461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.575+0000 7f7869200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415cbd0 0x7f786415d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.575+0000 7f7869200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7864074840 con 0x7f786415cbd0 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.575+0000 7f7869200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f78640746c0 con 0x7f7864145d70 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.575+0000 7f7869200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f786406f160 con 0x7f7848004800 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415cbd0 0x7f786415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415cbd0 0x7f786415d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57134/0 (socket says 172.21.15.71:57134) 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 -- 172.21.15.71:0/3612484875 learned_addr learned my addr 172.21.15.71:0/3612484875 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7861400640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7864145d70 0x7f78641461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7861e00640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7848004800 0x7f78640d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 -- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7848004800 msgr2=0x7f78640d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7848004800 0x7f78640d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 -- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7864145d70 msgr2=0x7f78641461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7864145d70 0x7f78641461f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 -- 172.21.15.71:0/3612484875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7864148c00 con 0x7f786415cbd0 2024-09-16T07:37:20.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7861e00640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7848004800 0x7f78640d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:20.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.576+0000 7f7862800640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415cbd0 0x7f786415d030 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f785c011b20 tx=0x7f785c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:20.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.577+0000 7f785a000640 1 -- 172.21.15.71:0/3612484875 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c004b20 con 0x7f786415cbd0 2024-09-16T07:37:20.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.577+0000 7f785a000640 1 -- 172.21.15.71:0/3612484875 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f785c004cc0 con 0x7f786415cbd0 2024-09-16T07:37:20.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.577+0000 7f785a000640 1 -- 172.21.15.71:0/3612484875 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c003bb0 con 0x7f786415cbd0 2024-09-16T07:37:20.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.577+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7864148ee0 con 0x7f786415cbd0 2024-09-16T07:37:20.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.577+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78641494a0 con 0x7f786415cbd0 2024-09-16T07:37:20.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.578+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786406f6c0 con 0x7f786415cbd0 2024-09-16T07:37:20.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.579+0000 7f785a000640 1 -- 172.21.15.71:0/3612484875 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f785c004030 con 0x7f786415cbd0 2024-09-16T07:37:20.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.580+0000 7f785a000640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784407aa10 0x7f784407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.580+0000 7f785a000640 1 -- 172.21.15.71:0/3612484875 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f785c09a5a0 con 0x7f786415cbd0 2024-09-16T07:37:20.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.580+0000 7f7861e00640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784407aa10 0x7f784407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.581+0000 7f7861e00640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784407aa10 0x7f784407ced0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7854000fb0 tx=0x7f78540067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:20.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.585+0000 7f785a000640 1 -- 172.21.15.71:0/3612484875 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f785c064670 con 0x7f786415cbd0 2024-09-16T07:37:20.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.767+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78640b13c0 con 0x7f784407aa10 2024-09-16T07:37:20.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.773+0000 7f785a000640 1 -- 172.21.15.71:0/3612484875 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252495 (secure 0 0 0) 0x7f78640b13c0 con 0x7f784407aa10 2024-09-16T07:37:20.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:20.777 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:37:20.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784407aa10 msgr2=0x7f784407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784407aa10 0x7f784407ced0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7854000fb0 tx=0x7f78540067a0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415cbd0 msgr2=0x7f786415d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415cbd0 0x7f786415d030 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f785c011b20 tx=0x7f785c00f3c0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 shutdown_connections 2024-09-16T07:37:20.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f784407aa10 0x7f784407ced0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f786415cbd0 0x7f786415d030 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7864145d70 0x7f78641461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 --2- 172.21.15.71:0/3612484875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7848004800 0x7f78640d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 >> 172.21.15.71:0/3612484875 conn(0x7f78640b1950 msgr2=0x7f7864154100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:20.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 shutdown_connections 2024-09-16T07:37:20.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.777+0000 7f7869200640 1 -- 172.21.15.71:0/3612484875 wait complete. 2024-09-16T07:37:20.793 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":581,"stamp":"2024-09-16T07:37:20.645444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1082040,"kb_used_data":15344,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753892680,"statfs":{"total":773094113280,"available":771986104320,"internally_reserved":0,"allocated":15712256,"data_stored":7574641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1092079152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001323"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.616647+0000","last_change":"2024-09-16T07:37:16.612115+0000","last_active":"2024-09-16T07:37:16.616647+0000","last_peered":"2024-09-16T07:37:16.616647+0000","last_clean":"2024-09-16T07:37:16.616647+0000","last_became_active":"2024-09-16T07:37:16.608130+0000","last_became_peered":"2024-09-16T07:37:16.608130+0000","last_unstale":"2024-09-16T07:37:16.616647+0000","last_undegraded":"2024-09-16T07:37:16.616647+0000","last_fullsized":"2024-09-16T07:37:16.616647+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:04.216562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686059+0000","last_change":"2024-09-16T07:37:16.622991+0000","last_active":"2024-09-16T07:37:17.686059+0000","last_peered":"2024-09-16T07:37:17.686059+0000","last_clean":"2024-09-16T07:37:17.686059+0000","last_became_active":"2024-09-16T07:37:16.622051+0000","last_became_peered":"2024-09-16T07:37:16.622051+0000","last_unstale":"2024-09-16T07:37:17.686059+0000","last_undegraded":"2024-09-16T07:37:17.686059+0000","last_fullsized":"2024-09-16T07:37:17.686059+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:50.480562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685732+0000","last_change":"2024-09-16T07:37:16.603985+0000","last_active":"2024-09-16T07:37:17.685732+0000","last_peered":"2024-09-16T07:37:17.685732+0000","last_clean":"2024-09-16T07:37:17.685732+0000","last_became_active":"2024-09-16T07:37:16.603103+0000","last_became_peered":"2024-09-16T07:37:16.603103+0000","last_unstale":"2024-09-16T07:37:17.685732+0000","last_undegraded":"2024-09-16T07:37:17.685732+0000","last_fullsized":"2024-09-16T07:37:17.685732+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:29.032286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685566+0000","last_change":"2024-09-16T07:37:16.611845+0000","last_active":"2024-09-16T07:37:17.685566+0000","last_peered":"2024-09-16T07:37:17.685566+0000","last_clean":"2024-09-16T07:37:17.685566+0000","last_became_active":"2024-09-16T07:37:16.611110+0000","last_became_peered":"2024-09-16T07:37:16.611110+0000","last_unstale":"2024-09-16T07:37:17.685566+0000","last_undegraded":"2024-09-16T07:37:17.685566+0000","last_fullsized":"2024-09-16T07:37:17.685566+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:13:15.329105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601935+0000","last_change":"2024-09-16T07:37:16.599113+0000","last_active":"2024-09-16T07:37:17.601935+0000","last_peered":"2024-09-16T07:37:17.601935+0000","last_clean":"2024-09-16T07:37:17.601935+0000","last_became_active":"2024-09-16T07:37:16.598622+0000","last_became_peered":"2024-09-16T07:37:16.598622+0000","last_unstale":"2024-09-16T07:37:17.601935+0000","last_undegraded":"2024-09-16T07:37:17.601935+0000","last_fullsized":"2024-09-16T07:37:17.601935+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:26:52.294335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602813+0000","last_change":"2024-09-16T07:37:16.612001+0000","last_active":"2024-09-16T07:37:17.602813+0000","last_peered":"2024-09-16T07:37:17.602813+0000","last_clean":"2024-09-16T07:37:17.602813+0000","last_became_active":"2024-09-16T07:37:16.611749+0000","last_became_peered":"2024-09-16T07:37:16.611749+0000","last_unstale":"2024-09-16T07:37:17.602813+0000","last_undegraded":"2024-09-16T07:37:17.602813+0000","last_fullsized":"2024-09-16T07:37:17.602813+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:13:57.251738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603787+0000","last_change":"2024-09-16T07:37:16.599337+0000","last_active":"2024-09-16T07:37:17.603787+0000","last_peered":"2024-09-16T07:37:17.603787+0000","last_clean":"2024-09-16T07:37:17.603787+0000","last_became_active":"2024-09-16T07:37:16.599022+0000","last_became_peered":"2024-09-16T07:37:16.599022+0000","last_unstale":"2024-09-16T07:37:17.603787+0000","last_undegraded":"2024-09-16T07:37:17.603787+0000","last_fullsized":"2024-09-16T07:37:17.603787+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:59.704586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.613852+0000","last_change":"2024-09-16T07:37:16.604546+0000","last_active":"2024-09-16T07:37:16.613852+0000","last_peered":"2024-09-16T07:37:16.613852+0000","last_clean":"2024-09-16T07:37:16.613852+0000","last_became_active":"2024-09-16T07:37:16.604198+0000","last_became_peered":"2024-09-16T07:37:16.604198+0000","last_unstale":"2024-09-16T07:37:16.613852+0000","last_undegraded":"2024-09-16T07:37:16.613852+0000","last_fullsized":"2024-09-16T07:37:16.613852+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:09.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602853+0000","last_change":"2024-09-16T07:37:16.603290+0000","last_active":"2024-09-16T07:37:17.602853+0000","last_peered":"2024-09-16T07:37:17.602853+0000","last_clean":"2024-09-16T07:37:17.602853+0000","last_became_active":"2024-09-16T07:37:16.602468+0000","last_became_peered":"2024-09-16T07:37:16.602468+0000","last_unstale":"2024-09-16T07:37:17.602853+0000","last_undegraded":"2024-09-16T07:37:17.602853+0000","last_fullsized":"2024-09-16T07:37:17.602853+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:21.924648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685735+0000","last_change":"2024-09-16T07:37:16.604257+0000","last_active":"2024-09-16T07:37:17.685735+0000","last_peered":"2024-09-16T07:37:17.685735+0000","last_clean":"2024-09-16T07:37:17.685735+0000","last_became_active":"2024-09-16T07:37:16.603950+0000","last_became_peered":"2024-09-16T07:37:16.603950+0000","last_unstale":"2024-09-16T07:37:17.685735+0000","last_undegraded":"2024-09-16T07:37:17.685735+0000","last_fullsized":"2024-09-16T07:37:17.685735+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:31.650789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603309+0000","last_change":"2024-09-16T07:37:16.609966+0000","last_active":"2024-09-16T07:37:17.603309+0000","last_peered":"2024-09-16T07:37:17.603309+0000","last_clean":"2024-09-16T07:37:17.603309+0000","last_became_active":"2024-09-16T07:37:16.608454+0000","last_became_peered":"2024-09-16T07:37:16.608454+0000","last_unstale":"2024-09-16T07:37:17.603309+0000","last_undegraded":"2024-09-16T07:37:17.603309+0000","last_fullsized":"2024-09-16T07:37:17.603309+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:09.901271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654312+0000","last_change":"2024-09-16T07:37:16.603062+0000","last_active":"2024-09-16T07:37:17.654312+0000","last_peered":"2024-09-16T07:37:17.654312+0000","last_clean":"2024-09-16T07:37:17.654312+0000","last_became_active":"2024-09-16T07:37:16.602212+0000","last_became_peered":"2024-09-16T07:37:16.602212+0000","last_unstale":"2024-09-16T07:37:17.654312+0000","last_undegraded":"2024-09-16T07:37:17.654312+0000","last_fullsized":"2024-09-16T07:37:17.654312+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:28.614057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601902+0000","last_change":"2024-09-16T07:37:16.600584+0000","last_active":"2024-09-16T07:37:17.601902+0000","last_peered":"2024-09-16T07:37:17.601902+0000","last_clean":"2024-09-16T07:37:17.601902+0000","last_became_active":"2024-09-16T07:37:16.598273+0000","last_became_peered":"2024-09-16T07:37:16.598273+0000","last_unstale":"2024-09-16T07:37:17.601902+0000","last_undegraded":"2024-09-16T07:37:17.601902+0000","last_fullsized":"2024-09-16T07:37:17.601902+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:23.631603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654118+0000","last_change":"2024-09-16T07:37:16.604047+0000","last_active":"2024-09-16T07:37:17.654118+0000","last_peered":"2024-09-16T07:37:17.654118+0000","last_clean":"2024-09-16T07:37:17.654118+0000","last_became_active":"2024-09-16T07:37:16.603784+0000","last_became_peered":"2024-09-16T07:37:16.603784+0000","last_unstale":"2024-09-16T07:37:17.654118+0000","last_undegraded":"2024-09-16T07:37:17.654118+0000","last_fullsized":"2024-09-16T07:37:17.654118+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:00.251509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601741+0000","last_change":"2024-09-16T07:37:16.616467+0000","last_active":"2024-09-16T07:37:17.601741+0000","last_peered":"2024-09-16T07:37:17.601741+0000","last_clean":"2024-09-16T07:37:17.601741+0000","last_became_active":"2024-09-16T07:37:16.616127+0000","last_became_peered":"2024-09-16T07:37:16.616127+0000","last_unstale":"2024-09-16T07:37:17.601741+0000","last_undegraded":"2024-09-16T07:37:17.601741+0000","last_fullsized":"2024-09-16T07:37:17.601741+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:46.157557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604530+0000","last_change":"2024-09-16T07:37:16.596723+0000","last_active":"2024-09-16T07:37:17.604530+0000","last_peered":"2024-09-16T07:37:17.604530+0000","last_clean":"2024-09-16T07:37:17.604530+0000","last_became_active":"2024-09-16T07:37:16.596382+0000","last_became_peered":"2024-09-16T07:37:16.596382+0000","last_unstale":"2024-09-16T07:37:17.604530+0000","last_undegraded":"2024-09-16T07:37:17.604530+0000","last_fullsized":"2024-09-16T07:37:17.604530+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:10.792519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685660+0000","last_change":"2024-09-16T07:37:16.605843+0000","last_active":"2024-09-16T07:37:17.685660+0000","last_peered":"2024-09-16T07:37:17.685660+0000","last_clean":"2024-09-16T07:37:17.685660+0000","last_became_active":"2024-09-16T07:37:16.605575+0000","last_became_peered":"2024-09-16T07:37:16.605575+0000","last_unstale":"2024-09-16T07:37:17.685660+0000","last_undegraded":"2024-09-16T07:37:17.685660+0000","last_fullsized":"2024-09-16T07:37:17.685660+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:23.527726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602120+0000","last_change":"2024-09-16T07:37:16.596469+0000","last_active":"2024-09-16T07:37:17.602120+0000","last_peered":"2024-09-16T07:37:17.602120+0000","last_clean":"2024-09-16T07:37:17.602120+0000","last_became_active":"2024-09-16T07:37:16.596220+0000","last_became_peered":"2024-09-16T07:37:16.596220+0000","last_unstale":"2024-09-16T07:37:17.602120+0000","last_undegraded":"2024-09-16T07:37:17.602120+0000","last_fullsized":"2024-09-16T07:37:17.602120+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:33.615526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.618542+0000","last_change":"2024-09-16T07:37:16.616585+0000","last_active":"2024-09-16T07:37:16.618542+0000","last_peered":"2024-09-16T07:37:16.618542+0000","last_clean":"2024-09-16T07:37:16.618542+0000","last_became_active":"2024-09-16T07:37:16.615817+0000","last_became_peered":"2024-09-16T07:37:16.615817+0000","last_unstale":"2024-09-16T07:37:16.618542+0000","last_undegraded":"2024-09-16T07:37:16.618542+0000","last_fullsized":"2024-09-16T07:37:16.618542+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:30:45.044712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.618742+0000","last_change":"2024-09-16T07:37:16.615823+0000","last_active":"2024-09-16T07:37:16.618742+0000","last_peered":"2024-09-16T07:37:16.618742+0000","last_clean":"2024-09-16T07:37:16.618742+0000","last_became_active":"2024-09-16T07:37:16.614368+0000","last_became_peered":"2024-09-16T07:37:16.614368+0000","last_unstale":"2024-09-16T07:37:16.618742+0000","last_undegraded":"2024-09-16T07:37:16.618742+0000","last_fullsized":"2024-09-16T07:37:16.618742+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:23.217079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686169+0000","last_change":"2024-09-16T07:37:16.608203+0000","last_active":"2024-09-16T07:37:17.686169+0000","last_peered":"2024-09-16T07:37:17.686169+0000","last_clean":"2024-09-16T07:37:17.686169+0000","last_became_active":"2024-09-16T07:37:16.607881+0000","last_became_peered":"2024-09-16T07:37:16.607881+0000","last_unstale":"2024-09-16T07:37:17.686169+0000","last_undegraded":"2024-09-16T07:37:17.686169+0000","last_fullsized":"2024-09-16T07:37:17.686169+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:06.217492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654122+0000","last_change":"2024-09-16T07:37:16.597912+0000","last_active":"2024-09-16T07:37:17.654122+0000","last_peered":"2024-09-16T07:37:17.654122+0000","last_clean":"2024-09-16T07:37:17.654122+0000","last_became_active":"2024-09-16T07:37:16.597642+0000","last_became_peered":"2024-09-16T07:37:16.597642+0000","last_unstale":"2024-09-16T07:37:17.654122+0000","last_undegraded":"2024-09-16T07:37:17.654122+0000","last_fullsized":"2024-09-16T07:37:17.654122+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:46.975361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685094+0000","last_change":"2024-09-16T07:37:16.597799+0000","last_active":"2024-09-16T07:37:17.685094+0000","last_peered":"2024-09-16T07:37:17.685094+0000","last_clean":"2024-09-16T07:37:17.685094+0000","last_became_active":"2024-09-16T07:37:16.597507+0000","last_became_peered":"2024-09-16T07:37:16.597507+0000","last_unstale":"2024-09-16T07:37:17.685094+0000","last_undegraded":"2024-09-16T07:37:17.685094+0000","last_fullsized":"2024-09-16T07:37:17.685094+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:51.743308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601601+0000","last_change":"2024-09-16T07:37:16.602247+0000","last_active":"2024-09-16T07:37:17.601601+0000","last_peered":"2024-09-16T07:37:17.601601+0000","last_clean":"2024-09-16T07:37:17.601601+0000","last_became_active":"2024-09-16T07:37:16.598358+0000","last_became_peered":"2024-09-16T07:37:16.598358+0000","last_unstale":"2024-09-16T07:37:17.601601+0000","last_undegraded":"2024-09-16T07:37:17.601601+0000","last_fullsized":"2024-09-16T07:37:17.601601+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:23.407213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685953+0000","last_change":"2024-09-16T07:37:16.621878+0000","last_active":"2024-09-16T07:37:17.685953+0000","last_peered":"2024-09-16T07:37:17.685953+0000","last_clean":"2024-09-16T07:37:17.685953+0000","last_became_active":"2024-09-16T07:37:16.621088+0000","last_became_peered":"2024-09-16T07:37:16.621088+0000","last_unstale":"2024-09-16T07:37:17.685953+0000","last_undegraded":"2024-09-16T07:37:17.685953+0000","last_fullsized":"2024-09-16T07:37:17.685953+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:38:57.572392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.621478+0000","last_change":"2024-09-16T07:37:16.618890+0000","last_active":"2024-09-16T07:37:16.621478+0000","last_peered":"2024-09-16T07:37:16.621478+0000","last_clean":"2024-09-16T07:37:16.621478+0000","last_became_active":"2024-09-16T07:37:16.618099+0000","last_became_peered":"2024-09-16T07:37:16.618099+0000","last_unstale":"2024-09-16T07:37:16.621478+0000","last_undegraded":"2024-09-16T07:37:16.621478+0000","last_fullsized":"2024-09-16T07:37:16.621478+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:33.280212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685743+0000","last_change":"2024-09-16T07:37:16.615559+0000","last_active":"2024-09-16T07:37:17.685743+0000","last_peered":"2024-09-16T07:37:17.685743+0000","last_clean":"2024-09-16T07:37:17.685743+0000","last_became_active":"2024-09-16T07:37:16.615229+0000","last_became_peered":"2024-09-16T07:37:16.615229+0000","last_unstale":"2024-09-16T07:37:17.685743+0000","last_undegraded":"2024-09-16T07:37:17.685743+0000","last_fullsized":"2024-09-16T07:37:17.685743+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:34.699450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685692+0000","last_change":"2024-09-16T07:37:16.623386+0000","last_active":"2024-09-16T07:37:17.685692+0000","last_peered":"2024-09-16T07:37:17.685692+0000","last_clean":"2024-09-16T07:37:17.685692+0000","last_became_active":"2024-09-16T07:37:16.623116+0000","last_became_peered":"2024-09-16T07:37:16.623116+0000","last_unstale":"2024-09-16T07:37:17.685692+0000","last_undegraded":"2024-09-16T07:37:17.685692+0000","last_fullsized":"2024-09-16T07:37:17.685692+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:32:09.521160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.620611+0000","last_change":"2024-09-16T07:37:16.612904+0000","last_active":"2024-09-16T07:37:16.620611+0000","last_peered":"2024-09-16T07:37:16.620611+0000","last_clean":"2024-09-16T07:37:16.620611+0000","last_became_active":"2024-09-16T07:37:16.612651+0000","last_became_peered":"2024-09-16T07:37:16.612651+0000","last_unstale":"2024-09-16T07:37:16.620611+0000","last_undegraded":"2024-09-16T07:37:16.620611+0000","last_fullsized":"2024-09-16T07:37:16.620611+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:41.123149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654332+0000","last_change":"2024-09-16T07:37:16.599877+0000","last_active":"2024-09-16T07:37:17.654332+0000","last_peered":"2024-09-16T07:37:17.654332+0000","last_clean":"2024-09-16T07:37:17.654332+0000","last_became_active":"2024-09-16T07:37:16.599587+0000","last_became_peered":"2024-09-16T07:37:16.599587+0000","last_unstale":"2024-09-16T07:37:17.654332+0000","last_undegraded":"2024-09-16T07:37:17.654332+0000","last_fullsized":"2024-09-16T07:37:17.654332+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:25.522308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685182+0000","last_change":"2024-09-16T07:37:16.619414+0000","last_active":"2024-09-16T07:37:17.685182+0000","last_peered":"2024-09-16T07:37:17.685182+0000","last_clean":"2024-09-16T07:37:17.685182+0000","last_became_active":"2024-09-16T07:37:16.618102+0000","last_became_peered":"2024-09-16T07:37:16.618102+0000","last_unstale":"2024-09-16T07:37:17.685182+0000","last_undegraded":"2024-09-16T07:37:17.685182+0000","last_fullsized":"2024-09-16T07:37:17.685182+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:28.117165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685153+0000","last_change":"2024-09-16T07:37:16.617056+0000","last_active":"2024-09-16T07:37:17.685153+0000","last_peered":"2024-09-16T07:37:17.685153+0000","last_clean":"2024-09-16T07:37:17.685153+0000","last_became_active":"2024-09-16T07:37:16.616489+0000","last_became_peered":"2024-09-16T07:37:16.616489+0000","last_unstale":"2024-09-16T07:37:17.685153+0000","last_undegraded":"2024-09-16T07:37:17.685153+0000","last_fullsized":"2024-09-16T07:37:17.685153+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:51.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602305+0000","last_change":"2024-09-16T07:37:16.601844+0000","last_active":"2024-09-16T07:37:17.602305+0000","last_peered":"2024-09-16T07:37:17.602305+0000","last_clean":"2024-09-16T07:37:17.602305+0000","last_became_active":"2024-09-16T07:37:16.600150+0000","last_became_peered":"2024-09-16T07:37:16.600150+0000","last_unstale":"2024-09-16T07:37:17.602305+0000","last_undegraded":"2024-09-16T07:37:17.602305+0000","last_fullsized":"2024-09-16T07:37:17.602305+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:31.475733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685757+0000","last_change":"2024-09-16T07:37:15.600312+0000","last_active":"2024-09-16T07:37:17.685757+0000","last_peered":"2024-09-16T07:37:17.685757+0000","last_clean":"2024-09-16T07:37:17.685757+0000","last_became_active":"2024-09-16T07:37:15.600061+0000","last_became_peered":"2024-09-16T07:37:15.600061+0000","last_unstale":"2024-09-16T07:37:17.685757+0000","last_undegraded":"2024-09-16T07:37:17.685757+0000","last_fullsized":"2024-09-16T07:37:17.685757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:36.953048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604744+0000","last_change":"2024-09-16T07:37:16.605795+0000","last_active":"2024-09-16T07:37:17.604744+0000","last_peered":"2024-09-16T07:37:17.604744+0000","last_clean":"2024-09-16T07:37:17.604744+0000","last_became_active":"2024-09-16T07:37:16.605514+0000","last_became_peered":"2024-09-16T07:37:16.605514+0000","last_unstale":"2024-09-16T07:37:17.604744+0000","last_undegraded":"2024-09-16T07:37:17.604744+0000","last_fullsized":"2024-09-16T07:37:17.604744+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:54:20.004547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603384+0000","last_change":"2024-09-16T07:37:15.600366+0000","last_active":"2024-09-16T07:37:17.603384+0000","last_peered":"2024-09-16T07:37:17.603384+0000","last_clean":"2024-09-16T07:37:17.603384+0000","last_became_active":"2024-09-16T07:37:15.600098+0000","last_became_peered":"2024-09-16T07:37:15.600098+0000","last_unstale":"2024-09-16T07:37:17.603384+0000","last_undegraded":"2024-09-16T07:37:17.603384+0000","last_fullsized":"2024-09-16T07:37:17.603384+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:39:34.281801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602979+0000","last_change":"2024-09-16T07:37:16.605178+0000","last_active":"2024-09-16T07:37:17.602979+0000","last_peered":"2024-09-16T07:37:17.602979+0000","last_clean":"2024-09-16T07:37:17.602979+0000","last_became_active":"2024-09-16T07:37:16.604894+0000","last_became_peered":"2024-09-16T07:37:16.604894+0000","last_unstale":"2024-09-16T07:37:17.602979+0000","last_undegraded":"2024-09-16T07:37:17.602979+0000","last_fullsized":"2024-09-16T07:37:17.602979+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:14.975358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685826+0000","last_change":"2024-09-16T07:37:15.595761+0000","last_active":"2024-09-16T07:37:17.685826+0000","last_peered":"2024-09-16T07:37:17.685826+0000","last_clean":"2024-09-16T07:37:17.685826+0000","last_became_active":"2024-09-16T07:37:15.595387+0000","last_became_peered":"2024-09-16T07:37:15.595387+0000","last_unstale":"2024-09-16T07:37:17.685826+0000","last_undegraded":"2024-09-16T07:37:17.685826+0000","last_fullsized":"2024-09-16T07:37:17.685826+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:06.596941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685746+0000","last_change":"2024-09-16T07:37:16.604298+0000","last_active":"2024-09-16T07:37:17.685746+0000","last_peered":"2024-09-16T07:37:17.685746+0000","last_clean":"2024-09-16T07:37:17.685746+0000","last_became_active":"2024-09-16T07:37:16.601596+0000","last_became_peered":"2024-09-16T07:37:16.601596+0000","last_unstale":"2024-09-16T07:37:17.685746+0000","last_undegraded":"2024-09-16T07:37:17.685746+0000","last_fullsized":"2024-09-16T07:37:17.685746+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:12.794564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686427+0000","last_change":"2024-09-16T07:37:15.597770+0000","last_active":"2024-09-16T07:37:17.686427+0000","last_peered":"2024-09-16T07:37:17.686427+0000","last_clean":"2024-09-16T07:37:17.686427+0000","last_became_active":"2024-09-16T07:37:15.596291+0000","last_became_peered":"2024-09-16T07:37:15.596291+0000","last_unstale":"2024-09-16T07:37:17.686427+0000","last_undegraded":"2024-09-16T07:37:17.686427+0000","last_fullsized":"2024-09-16T07:37:17.686427+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:05.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686169+0000","last_change":"2024-09-16T07:37:16.619485+0000","last_active":"2024-09-16T07:37:17.686169+0000","last_peered":"2024-09-16T07:37:17.686169+0000","last_clean":"2024-09-16T07:37:17.686169+0000","last_became_active":"2024-09-16T07:37:16.619172+0000","last_became_peered":"2024-09-16T07:37:16.619172+0000","last_unstale":"2024-09-16T07:37:17.686169+0000","last_undegraded":"2024-09-16T07:37:17.686169+0000","last_fullsized":"2024-09-16T07:37:17.686169+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:28.449555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"145'1","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.612127+0000","last_change":"2024-09-16T07:37:15.600241+0000","last_active":"2024-09-16T07:37:16.612127+0000","last_peered":"2024-09-16T07:37:16.612127+0000","last_clean":"2024-09-16T07:37:16.612127+0000","last_became_active":"2024-09-16T07:37:15.599973+0000","last_became_peered":"2024-09-16T07:37:15.599973+0000","last_unstale":"2024-09-16T07:37:16.612127+0000","last_undegraded":"2024-09-16T07:37:16.612127+0000","last_fullsized":"2024-09-16T07:37:16.612127+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:41.517289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686016+0000","last_change":"2024-09-16T07:37:16.605737+0000","last_active":"2024-09-16T07:37:17.686016+0000","last_peered":"2024-09-16T07:37:17.686016+0000","last_clean":"2024-09-16T07:37:17.686016+0000","last_became_active":"2024-09-16T07:37:16.604805+0000","last_became_peered":"2024-09-16T07:37:16.604805+0000","last_unstale":"2024-09-16T07:37:17.686016+0000","last_undegraded":"2024-09-16T07:37:17.686016+0000","last_fullsized":"2024-09-16T07:37:17.686016+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:25.522308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601572+0000","last_change":"2024-09-16T07:37:15.601710+0000","last_active":"2024-09-16T07:37:17.601572+0000","last_peered":"2024-09-16T07:37:17.601572+0000","last_clean":"2024-09-16T07:37:17.601572+0000","last_became_active":"2024-09-16T07:37:15.601456+0000","last_became_peered":"2024-09-16T07:37:15.601456+0000","last_unstale":"2024-09-16T07:37:17.601572+0000","last_undegraded":"2024-09-16T07:37:17.601572+0000","last_fullsized":"2024-09-16T07:37:17.601572+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:51.317745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685137+0000","last_change":"2024-09-16T07:37:16.617921+0000","last_active":"2024-09-16T07:37:17.685137+0000","last_peered":"2024-09-16T07:37:17.685137+0000","last_clean":"2024-09-16T07:37:17.685137+0000","last_became_active":"2024-09-16T07:37:16.616812+0000","last_became_peered":"2024-09-16T07:37:16.616812+0000","last_unstale":"2024-09-16T07:37:17.685137+0000","last_undegraded":"2024-09-16T07:37:17.685137+0000","last_fullsized":"2024-09-16T07:37:17.685137+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:24.855633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602930+0000","last_change":"2024-09-16T07:37:15.604158+0000","last_active":"2024-09-16T07:37:17.602930+0000","last_peered":"2024-09-16T07:37:17.602930+0000","last_clean":"2024-09-16T07:37:17.602930+0000","last_became_active":"2024-09-16T07:37:15.603895+0000","last_became_peered":"2024-09-16T07:37:15.603895+0000","last_unstale":"2024-09-16T07:37:17.602930+0000","last_undegraded":"2024-09-16T07:37:17.602930+0000","last_fullsized":"2024-09-16T07:37:17.602930+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:27.609967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685281+0000","last_change":"2024-09-16T07:37:16.614133+0000","last_active":"2024-09-16T07:37:17.685281+0000","last_peered":"2024-09-16T07:37:17.685281+0000","last_clean":"2024-09-16T07:37:17.685281+0000","last_became_active":"2024-09-16T07:37:16.613846+0000","last_became_peered":"2024-09-16T07:37:16.613846+0000","last_unstale":"2024-09-16T07:37:17.685281+0000","last_undegraded":"2024-09-16T07:37:17.685281+0000","last_fullsized":"2024-09-16T07:37:17.685281+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:43.046079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685309+0000","last_change":"2024-09-16T07:37:15.608141+0000","last_active":"2024-09-16T07:37:17.685309+0000","last_peered":"2024-09-16T07:37:17.685309+0000","last_clean":"2024-09-16T07:37:17.685309+0000","last_became_active":"2024-09-16T07:37:15.607286+0000","last_became_peered":"2024-09-16T07:37:15.607286+0000","last_unstale":"2024-09-16T07:37:17.685309+0000","last_undegraded":"2024-09-16T07:37:17.685309+0000","last_fullsized":"2024-09-16T07:37:17.685309+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:22.497851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685607+0000","last_change":"2024-09-16T07:37:16.599443+0000","last_active":"2024-09-16T07:37:17.685607+0000","last_peered":"2024-09-16T07:37:17.685607+0000","last_clean":"2024-09-16T07:37:17.685607+0000","last_became_active":"2024-09-16T07:37:16.599176+0000","last_became_peered":"2024-09-16T07:37:16.599176+0000","last_unstale":"2024-09-16T07:37:17.685607+0000","last_undegraded":"2024-09-16T07:37:17.685607+0000","last_fullsized":"2024-09-16T07:37:17.685607+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:46.975361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602963+0000","last_change":"2024-09-16T07:37:15.607600+0000","last_active":"2024-09-16T07:37:17.602963+0000","last_peered":"2024-09-16T07:37:17.602963+0000","last_clean":"2024-09-16T07:37:17.602963+0000","last_became_active":"2024-09-16T07:37:15.606766+0000","last_became_peered":"2024-09-16T07:37:15.606766+0000","last_unstale":"2024-09-16T07:37:17.602963+0000","last_undegraded":"2024-09-16T07:37:17.602963+0000","last_fullsized":"2024-09-16T07:37:17.602963+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:24.518217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654608+0000","last_change":"2024-09-16T07:37:16.608093+0000","last_active":"2024-09-16T07:37:17.654608+0000","last_peered":"2024-09-16T07:37:17.654608+0000","last_clean":"2024-09-16T07:37:17.654608+0000","last_became_active":"2024-09-16T07:37:16.607818+0000","last_became_peered":"2024-09-16T07:37:16.607818+0000","last_unstale":"2024-09-16T07:37:17.654608+0000","last_undegraded":"2024-09-16T07:37:17.654608+0000","last_fullsized":"2024-09-16T07:37:17.654608+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:17.637595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602366+0000","last_change":"2024-09-16T07:37:15.599779+0000","last_active":"2024-09-16T07:37:17.602366+0000","last_peered":"2024-09-16T07:37:17.602366+0000","last_clean":"2024-09-16T07:37:17.602366+0000","last_became_active":"2024-09-16T07:37:15.599464+0000","last_became_peered":"2024-09-16T07:37:15.599464+0000","last_unstale":"2024-09-16T07:37:17.602366+0000","last_undegraded":"2024-09-16T07:37:17.602366+0000","last_fullsized":"2024-09-16T07:37:17.602366+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:16.485730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604699+0000","last_change":"2024-09-16T07:37:16.604052+0000","last_active":"2024-09-16T07:37:17.604699+0000","last_peered":"2024-09-16T07:37:17.604699+0000","last_clean":"2024-09-16T07:37:17.604699+0000","last_became_active":"2024-09-16T07:37:16.603638+0000","last_became_peered":"2024-09-16T07:37:16.603638+0000","last_unstale":"2024-09-16T07:37:17.604699+0000","last_undegraded":"2024-09-16T07:37:17.604699+0000","last_fullsized":"2024-09-16T07:37:17.604699+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:19:17.755613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604290+0000","last_change":"2024-09-16T07:37:15.596420+0000","last_active":"2024-09-16T07:37:17.604290+0000","last_peered":"2024-09-16T07:37:17.604290+0000","last_clean":"2024-09-16T07:37:17.604290+0000","last_became_active":"2024-09-16T07:37:15.596131+0000","last_became_peered":"2024-09-16T07:37:15.596131+0000","last_unstale":"2024-09-16T07:37:17.604290+0000","last_undegraded":"2024-09-16T07:37:17.604290+0000","last_fullsized":"2024-09-16T07:37:17.604290+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:27.606954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685155+0000","last_change":"2024-09-16T07:37:16.605491+0000","last_active":"2024-09-16T07:37:17.685155+0000","last_peered":"2024-09-16T07:37:17.685155+0000","last_clean":"2024-09-16T07:37:17.685155+0000","last_became_active":"2024-09-16T07:37:16.605236+0000","last_became_peered":"2024-09-16T07:37:16.605236+0000","last_unstale":"2024-09-16T07:37:17.685155+0000","last_undegraded":"2024-09-16T07:37:17.685155+0000","last_fullsized":"2024-09-16T07:37:17.685155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:42:23.350926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686263+0000","last_change":"2024-09-16T07:37:15.596066+0000","last_active":"2024-09-16T07:37:17.686263+0000","last_peered":"2024-09-16T07:37:17.686263+0000","last_clean":"2024-09-16T07:37:17.686263+0000","last_became_active":"2024-09-16T07:37:15.595812+0000","last_became_peered":"2024-09-16T07:37:15.595812+0000","last_unstale":"2024-09-16T07:37:17.686263+0000","last_undegraded":"2024-09-16T07:37:17.686263+0000","last_fullsized":"2024-09-16T07:37:17.686263+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:15.895320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.620584+0000","last_change":"2024-09-16T07:37:16.613421+0000","last_active":"2024-09-16T07:37:16.620584+0000","last_peered":"2024-09-16T07:37:16.620584+0000","last_clean":"2024-09-16T07:37:16.620584+0000","last_became_active":"2024-09-16T07:37:16.612333+0000","last_became_peered":"2024-09-16T07:37:16.612333+0000","last_unstale":"2024-09-16T07:37:16.620584+0000","last_undegraded":"2024-09-16T07:37:16.620584+0000","last_fullsized":"2024-09-16T07:37:16.620584+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:20:07.837786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654425+0000","last_change":"2024-09-16T07:37:15.593786+0000","last_active":"2024-09-16T07:37:17.654425+0000","last_peered":"2024-09-16T07:37:17.654425+0000","last_clean":"2024-09-16T07:37:17.654425+0000","last_became_active":"2024-09-16T07:37:15.593082+0000","last_became_peered":"2024-09-16T07:37:15.593082+0000","last_unstale":"2024-09-16T07:37:17.654425+0000","last_undegraded":"2024-09-16T07:37:17.654425+0000","last_fullsized":"2024-09-16T07:37:17.654425+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:07.749839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685069+0000","last_change":"2024-09-16T07:37:16.611529+0000","last_active":"2024-09-16T07:37:17.685069+0000","last_peered":"2024-09-16T07:37:17.685069+0000","last_clean":"2024-09-16T07:37:17.685069+0000","last_became_active":"2024-09-16T07:37:16.611200+0000","last_became_peered":"2024-09-16T07:37:16.611200+0000","last_unstale":"2024-09-16T07:37:17.685069+0000","last_undegraded":"2024-09-16T07:37:17.685069+0000","last_fullsized":"2024-09-16T07:37:17.685069+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:28.449555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685994+0000","last_change":"2024-09-16T07:37:15.608181+0000","last_active":"2024-09-16T07:37:17.685994+0000","last_peered":"2024-09-16T07:37:17.685994+0000","last_clean":"2024-09-16T07:37:17.685994+0000","last_became_active":"2024-09-16T07:37:15.607890+0000","last_became_peered":"2024-09-16T07:37:15.607890+0000","last_unstale":"2024-09-16T07:37:17.685994+0000","last_undegraded":"2024-09-16T07:37:17.685994+0000","last_fullsized":"2024-09-16T07:37:17.685994+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:22.627513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685274+0000","last_change":"2024-09-16T07:37:16.601574+0000","last_active":"2024-09-16T07:37:17.685274+0000","last_peered":"2024-09-16T07:37:17.685274+0000","last_clean":"2024-09-16T07:37:17.685274+0000","last_became_active":"2024-09-16T07:37:16.598772+0000","last_became_peered":"2024-09-16T07:37:16.598772+0000","last_unstale":"2024-09-16T07:37:17.685274+0000","last_undegraded":"2024-09-16T07:37:17.685274+0000","last_fullsized":"2024-09-16T07:37:17.685274+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:26.103572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"145'8","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.621959+0000","last_change":"2024-09-16T07:37:15.604167+0000","last_active":"2024-09-16T07:37:16.621959+0000","last_peered":"2024-09-16T07:37:16.621959+0000","last_clean":"2024-09-16T07:37:16.621959+0000","last_became_active":"2024-09-16T07:37:15.603883+0000","last_became_peered":"2024-09-16T07:37:15.603883+0000","last_unstale":"2024-09-16T07:37:16.621959+0000","last_undegraded":"2024-09-16T07:37:16.621959+0000","last_fullsized":"2024-09-16T07:37:16.621959+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:56.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602365+0000","last_change":"2024-09-16T07:37:16.605149+0000","last_active":"2024-09-16T07:37:17.602365+0000","last_peered":"2024-09-16T07:37:17.602365+0000","last_clean":"2024-09-16T07:37:17.602365+0000","last_became_active":"2024-09-16T07:37:16.604651+0000","last_became_peered":"2024-09-16T07:37:16.604651+0000","last_unstale":"2024-09-16T07:37:17.602365+0000","last_undegraded":"2024-09-16T07:37:17.602365+0000","last_fullsized":"2024-09-16T07:37:17.602365+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:12.910252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685941+0000","last_change":"2024-09-16T07:37:15.609324+0000","last_active":"2024-09-16T07:37:17.685941+0000","last_peered":"2024-09-16T07:37:17.685941+0000","last_clean":"2024-09-16T07:37:17.685941+0000","last_became_active":"2024-09-16T07:37:15.608788+0000","last_became_peered":"2024-09-16T07:37:15.608788+0000","last_unstale":"2024-09-16T07:37:17.685941+0000","last_undegraded":"2024-09-16T07:37:17.685941+0000","last_fullsized":"2024-09-16T07:37:17.685941+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:11.906162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685903+0000","last_change":"2024-09-16T07:37:16.619759+0000","last_active":"2024-09-16T07:37:17.685903+0000","last_peered":"2024-09-16T07:37:17.685903+0000","last_clean":"2024-09-16T07:37:17.685903+0000","last_became_active":"2024-09-16T07:37:16.619489+0000","last_became_peered":"2024-09-16T07:37:16.619489+0000","last_unstale":"2024-09-16T07:37:17.685903+0000","last_undegraded":"2024-09-16T07:37:17.685903+0000","last_fullsized":"2024-09-16T07:37:17.685903+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:08.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.615586+0000","last_change":"2024-09-16T07:37:15.595980+0000","last_active":"2024-09-16T07:37:16.615586+0000","last_peered":"2024-09-16T07:37:16.615586+0000","last_clean":"2024-09-16T07:37:16.615586+0000","last_became_active":"2024-09-16T07:37:15.595714+0000","last_became_peered":"2024-09-16T07:37:15.595714+0000","last_unstale":"2024-09-16T07:37:16.615586+0000","last_undegraded":"2024-09-16T07:37:16.615586+0000","last_fullsized":"2024-09-16T07:37:16.615586+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602006+0000","last_change":"2024-09-16T07:37:16.601387+0000","last_active":"2024-09-16T07:37:17.602006+0000","last_peered":"2024-09-16T07:37:17.602006+0000","last_clean":"2024-09-16T07:37:17.602006+0000","last_became_active":"2024-09-16T07:37:16.601026+0000","last_became_peered":"2024-09-16T07:37:16.601026+0000","last_unstale":"2024-09-16T07:37:17.602006+0000","last_undegraded":"2024-09-16T07:37:17.602006+0000","last_fullsized":"2024-09-16T07:37:17.602006+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:18.850568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604818+0000","last_change":"2024-09-16T07:37:15.601467+0000","last_active":"2024-09-16T07:37:17.604818+0000","last_peered":"2024-09-16T07:37:17.604818+0000","last_clean":"2024-09-16T07:37:17.604818+0000","last_became_active":"2024-09-16T07:37:15.601138+0000","last_became_peered":"2024-09-16T07:37:15.601138+0000","last_unstale":"2024-09-16T07:37:17.604818+0000","last_undegraded":"2024-09-16T07:37:17.604818+0000","last_fullsized":"2024-09-16T07:37:17.604818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:05.895984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603497+0000","last_change":"2024-09-16T07:37:16.608026+0000","last_active":"2024-09-16T07:37:17.603497+0000","last_peered":"2024-09-16T07:37:17.603497+0000","last_clean":"2024-09-16T07:37:17.603497+0000","last_became_active":"2024-09-16T07:37:16.606029+0000","last_became_peered":"2024-09-16T07:37:16.606029+0000","last_unstale":"2024-09-16T07:37:17.603497+0000","last_undegraded":"2024-09-16T07:37:17.603497+0000","last_fullsized":"2024-09-16T07:37:17.603497+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:01:49.151592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685682+0000","last_change":"2024-09-16T07:37:15.604913+0000","last_active":"2024-09-16T07:37:17.685682+0000","last_peered":"2024-09-16T07:37:17.685682+0000","last_clean":"2024-09-16T07:37:17.685682+0000","last_became_active":"2024-09-16T07:37:15.604587+0000","last_became_peered":"2024-09-16T07:37:15.604587+0000","last_unstale":"2024-09-16T07:37:17.685682+0000","last_undegraded":"2024-09-16T07:37:17.685682+0000","last_fullsized":"2024-09-16T07:37:17.685682+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:45.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654339+0000","last_change":"2024-09-16T07:37:16.613535+0000","last_active":"2024-09-16T07:37:17.654339+0000","last_peered":"2024-09-16T07:37:17.654339+0000","last_clean":"2024-09-16T07:37:17.654339+0000","last_became_active":"2024-09-16T07:37:16.613239+0000","last_became_peered":"2024-09-16T07:37:16.613239+0000","last_unstale":"2024-09-16T07:37:17.654339+0000","last_undegraded":"2024-09-16T07:37:17.654339+0000","last_fullsized":"2024-09-16T07:37:17.654339+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:24.457755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603125+0000","last_change":"2024-09-16T07:37:15.598775+0000","last_active":"2024-09-16T07:37:17.603125+0000","last_peered":"2024-09-16T07:37:17.603125+0000","last_clean":"2024-09-16T07:37:17.603125+0000","last_became_active":"2024-09-16T07:37:15.598352+0000","last_became_peered":"2024-09-16T07:37:15.598352+0000","last_unstale":"2024-09-16T07:37:17.603125+0000","last_undegraded":"2024-09-16T07:37:17.603125+0000","last_fullsized":"2024-09-16T07:37:17.603125+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:54.665554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685486+0000","last_change":"2024-09-16T07:37:16.615046+0000","last_active":"2024-09-16T07:37:17.685486+0000","last_peered":"2024-09-16T07:37:17.685486+0000","last_clean":"2024-09-16T07:37:17.685486+0000","last_became_active":"2024-09-16T07:37:16.614753+0000","last_became_peered":"2024-09-16T07:37:16.614753+0000","last_unstale":"2024-09-16T07:37:17.685486+0000","last_undegraded":"2024-09-16T07:37:17.685486+0000","last_fullsized":"2024-09-16T07:37:17.685486+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:31.393623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686122+0000","last_change":"2024-09-16T07:37:15.605799+0000","last_active":"2024-09-16T07:37:17.686122+0000","last_peered":"2024-09-16T07:37:17.686122+0000","last_clean":"2024-09-16T07:37:17.686122+0000","last_became_active":"2024-09-16T07:37:15.605538+0000","last_became_peered":"2024-09-16T07:37:15.605538+0000","last_unstale":"2024-09-16T07:37:17.686122+0000","last_undegraded":"2024-09-16T07:37:17.686122+0000","last_fullsized":"2024-09-16T07:37:17.686122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:29.344622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654123+0000","last_change":"2024-09-16T07:37:16.596498+0000","last_active":"2024-09-16T07:37:17.654123+0000","last_peered":"2024-09-16T07:37:17.654123+0000","last_clean":"2024-09-16T07:37:17.654123+0000","last_became_active":"2024-09-16T07:37:16.596227+0000","last_became_peered":"2024-09-16T07:37:16.596227+0000","last_unstale":"2024-09-16T07:37:17.654123+0000","last_undegraded":"2024-09-16T07:37:17.654123+0000","last_fullsized":"2024-09-16T07:37:17.654123+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:55.669644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685370+0000","last_change":"2024-09-16T07:37:15.601619+0000","last_active":"2024-09-16T07:37:17.685370+0000","last_peered":"2024-09-16T07:37:17.685370+0000","last_clean":"2024-09-16T07:37:17.685370+0000","last_became_active":"2024-09-16T07:37:15.601344+0000","last_became_peered":"2024-09-16T07:37:15.601344+0000","last_unstale":"2024-09-16T07:37:17.685370+0000","last_undegraded":"2024-09-16T07:37:17.685370+0000","last_fullsized":"2024-09-16T07:37:17.685370+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:36.453021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685177+0000","last_change":"2024-09-16T07:37:16.618459+0000","last_active":"2024-09-16T07:37:17.685177+0000","last_peered":"2024-09-16T07:37:17.685177+0000","last_clean":"2024-09-16T07:37:17.685177+0000","last_became_active":"2024-09-16T07:37:16.618146+0000","last_became_peered":"2024-09-16T07:37:16.618146+0000","last_unstale":"2024-09-16T07:37:17.685177+0000","last_undegraded":"2024-09-16T07:37:17.685177+0000","last_fullsized":"2024-09-16T07:37:17.685177+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.914629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685190+0000","last_change":"2024-09-16T07:37:15.605579+0000","last_active":"2024-09-16T07:37:17.685190+0000","last_peered":"2024-09-16T07:37:17.685190+0000","last_clean":"2024-09-16T07:37:17.685190+0000","last_became_active":"2024-09-16T07:37:15.605107+0000","last_became_peered":"2024-09-16T07:37:15.605107+0000","last_unstale":"2024-09-16T07:37:17.685190+0000","last_undegraded":"2024-09-16T07:37:17.685190+0000","last_fullsized":"2024-09-16T07:37:17.685190+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:52.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685792+0000","last_change":"2024-09-16T07:37:16.612546+0000","last_active":"2024-09-16T07:37:17.685792+0000","last_peered":"2024-09-16T07:37:17.685792+0000","last_clean":"2024-09-16T07:37:17.685792+0000","last_became_active":"2024-09-16T07:37:16.612301+0000","last_became_peered":"2024-09-16T07:37:16.612301+0000","last_unstale":"2024-09-16T07:37:17.685792+0000","last_undegraded":"2024-09-16T07:37:17.685792+0000","last_fullsized":"2024-09-16T07:37:17.685792+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:31.575258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686145+0000","last_change":"2024-09-16T07:37:15.608951+0000","last_active":"2024-09-16T07:37:17.686145+0000","last_peered":"2024-09-16T07:37:17.686145+0000","last_clean":"2024-09-16T07:37:17.686145+0000","last_became_active":"2024-09-16T07:37:15.608682+0000","last_became_peered":"2024-09-16T07:37:15.608682+0000","last_unstale":"2024-09-16T07:37:17.686145+0000","last_undegraded":"2024-09-16T07:37:17.686145+0000","last_fullsized":"2024-09-16T07:37:17.686145+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:19.491042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686016+0000","last_change":"2024-09-16T07:37:16.613996+0000","last_active":"2024-09-16T07:37:17.686016+0000","last_peered":"2024-09-16T07:37:17.686016+0000","last_clean":"2024-09-16T07:37:17.686016+0000","last_became_active":"2024-09-16T07:37:16.613625+0000","last_became_peered":"2024-09-16T07:37:16.613625+0000","last_unstale":"2024-09-16T07:37:17.686016+0000","last_undegraded":"2024-09-16T07:37:17.686016+0000","last_fullsized":"2024-09-16T07:37:17.686016+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:38.509608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686218+0000","last_change":"2024-09-16T07:37:15.607833+0000","last_active":"2024-09-16T07:37:17.686218+0000","last_peered":"2024-09-16T07:37:17.686218+0000","last_clean":"2024-09-16T07:37:17.686218+0000","last_became_active":"2024-09-16T07:37:15.607243+0000","last_became_peered":"2024-09-16T07:37:15.607243+0000","last_unstale":"2024-09-16T07:37:17.686218+0000","last_undegraded":"2024-09-16T07:37:17.686218+0000","last_fullsized":"2024-09-16T07:37:17.686218+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:26:51.290244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.618516+0000","last_change":"2024-09-16T07:37:16.614649+0000","last_active":"2024-09-16T07:37:16.618516+0000","last_peered":"2024-09-16T07:37:16.618516+0000","last_clean":"2024-09-16T07:37:16.618516+0000","last_became_active":"2024-09-16T07:37:16.614200+0000","last_became_peered":"2024-09-16T07:37:16.614200+0000","last_unstale":"2024-09-16T07:37:16.618516+0000","last_undegraded":"2024-09-16T07:37:16.618516+0000","last_fullsized":"2024-09-16T07:37:16.618516+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:26:12.262888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604878+0000","last_change":"2024-09-16T07:37:15.599948+0000","last_active":"2024-09-16T07:37:17.604878+0000","last_peered":"2024-09-16T07:37:17.604878+0000","last_clean":"2024-09-16T07:37:17.604878+0000","last_became_active":"2024-09-16T07:37:15.598850+0000","last_became_peered":"2024-09-16T07:37:15.598850+0000","last_unstale":"2024-09-16T07:37:17.604878+0000","last_undegraded":"2024-09-16T07:37:17.604878+0000","last_fullsized":"2024-09-16T07:37:17.604878+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:50.418816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.618476+0000","last_change":"2024-09-16T07:37:16.612993+0000","last_active":"2024-09-16T07:37:16.618476+0000","last_peered":"2024-09-16T07:37:16.618476+0000","last_clean":"2024-09-16T07:37:16.618476+0000","last_became_active":"2024-09-16T07:37:16.612744+0000","last_became_peered":"2024-09-16T07:37:16.612744+0000","last_unstale":"2024-09-16T07:37:16.618476+0000","last_undegraded":"2024-09-16T07:37:16.618476+0000","last_fullsized":"2024-09-16T07:37:16.618476+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:25.165694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604355+0000","last_change":"2024-09-16T07:37:15.602241+0000","last_active":"2024-09-16T07:37:17.604355+0000","last_peered":"2024-09-16T07:37:17.604355+0000","last_clean":"2024-09-16T07:37:17.604355+0000","last_became_active":"2024-09-16T07:37:15.601952+0000","last_became_peered":"2024-09-16T07:37:15.601952+0000","last_unstale":"2024-09-16T07:37:17.604355+0000","last_undegraded":"2024-09-16T07:37:17.604355+0000","last_fullsized":"2024-09-16T07:37:17.604355+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:20.185672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604121+0000","last_change":"2024-09-16T07:37:16.603932+0000","last_active":"2024-09-16T07:37:17.604121+0000","last_peered":"2024-09-16T07:37:17.604121+0000","last_clean":"2024-09-16T07:37:17.604121+0000","last_became_active":"2024-09-16T07:37:16.603324+0000","last_became_peered":"2024-09-16T07:37:16.603324+0000","last_unstale":"2024-09-16T07:37:17.604121+0000","last_undegraded":"2024-09-16T07:37:17.604121+0000","last_fullsized":"2024-09-16T07:37:17.604121+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:09.855435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685596+0000","last_change":"2024-09-16T07:37:15.605860+0000","last_active":"2024-09-16T07:37:17.685596+0000","last_peered":"2024-09-16T07:37:17.685596+0000","last_clean":"2024-09-16T07:37:17.685596+0000","last_became_active":"2024-09-16T07:37:15.605593+0000","last_became_peered":"2024-09-16T07:37:15.605593+0000","last_unstale":"2024-09-16T07:37:17.685596+0000","last_undegraded":"2024-09-16T07:37:17.685596+0000","last_fullsized":"2024-09-16T07:37:17.685596+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:22.403123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686193+0000","last_change":"2024-09-16T07:37:16.597229+0000","last_active":"2024-09-16T07:37:17.686193+0000","last_peered":"2024-09-16T07:37:17.686193+0000","last_clean":"2024-09-16T07:37:17.686193+0000","last_became_active":"2024-09-16T07:37:16.596969+0000","last_became_peered":"2024-09-16T07:37:16.596969+0000","last_unstale":"2024-09-16T07:37:17.686193+0000","last_undegraded":"2024-09-16T07:37:17.686193+0000","last_fullsized":"2024-09-16T07:37:17.686193+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:39.426464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686091+0000","last_change":"2024-09-16T07:37:15.601520+0000","last_active":"2024-09-16T07:37:17.686091+0000","last_peered":"2024-09-16T07:37:17.686091+0000","last_clean":"2024-09-16T07:37:17.686091+0000","last_became_active":"2024-09-16T07:37:15.601276+0000","last_became_peered":"2024-09-16T07:37:15.601276+0000","last_unstale":"2024-09-16T07:37:17.686091+0000","last_undegraded":"2024-09-16T07:37:17.686091+0000","last_fullsized":"2024-09-16T07:37:17.686091+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:42:48.022898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685229+0000","last_change":"2024-09-16T07:37:16.609120+0000","last_active":"2024-09-16T07:37:17.685229+0000","last_peered":"2024-09-16T07:37:17.685229+0000","last_clean":"2024-09-16T07:37:17.685229+0000","last_became_active":"2024-09-16T07:37:16.608814+0000","last_became_peered":"2024-09-16T07:37:16.608814+0000","last_unstale":"2024-09-16T07:37:17.685229+0000","last_undegraded":"2024-09-16T07:37:17.685229+0000","last_fullsized":"2024-09-16T07:37:17.685229+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:15.077059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.653882+0000","last_change":"2024-09-16T07:37:15.591807+0000","last_active":"2024-09-16T07:37:17.653882+0000","last_peered":"2024-09-16T07:37:17.653882+0000","last_clean":"2024-09-16T07:37:17.653882+0000","last_became_active":"2024-09-16T07:37:15.591560+0000","last_became_peered":"2024-09-16T07:37:15.591560+0000","last_unstale":"2024-09-16T07:37:17.653882+0000","last_undegraded":"2024-09-16T07:37:17.653882+0000","last_fullsized":"2024-09-16T07:37:17.653882+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:33.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685464+0000","last_change":"2024-09-16T07:37:16.615455+0000","last_active":"2024-09-16T07:37:17.685464+0000","last_peered":"2024-09-16T07:37:17.685464+0000","last_clean":"2024-09-16T07:37:17.685464+0000","last_became_active":"2024-09-16T07:37:16.614884+0000","last_became_peered":"2024-09-16T07:37:16.614884+0000","last_unstale":"2024-09-16T07:37:17.685464+0000","last_undegraded":"2024-09-16T07:37:17.685464+0000","last_fullsized":"2024-09-16T07:37:17.685464+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:57.493246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602605+0000","last_change":"2024-09-16T07:37:15.601484+0000","last_active":"2024-09-16T07:37:17.602605+0000","last_peered":"2024-09-16T07:37:17.602605+0000","last_clean":"2024-09-16T07:37:17.602605+0000","last_became_active":"2024-09-16T07:37:15.600178+0000","last_became_peered":"2024-09-16T07:37:15.600178+0000","last_unstale":"2024-09-16T07:37:17.602605+0000","last_undegraded":"2024-09-16T07:37:17.602605+0000","last_fullsized":"2024-09-16T07:37:17.602605+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:56.630838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604247+0000","last_change":"2024-09-16T07:37:16.606688+0000","last_active":"2024-09-16T07:37:17.604247+0000","last_peered":"2024-09-16T07:37:17.604247+0000","last_clean":"2024-09-16T07:37:17.604247+0000","last_became_active":"2024-09-16T07:37:16.603710+0000","last_became_peered":"2024-09-16T07:37:16.603710+0000","last_unstale":"2024-09-16T07:37:17.604247+0000","last_undegraded":"2024-09-16T07:37:17.604247+0000","last_fullsized":"2024-09-16T07:37:17.604247+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:59.951624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654475+0000","last_change":"2024-09-16T07:37:15.591979+0000","last_active":"2024-09-16T07:37:17.654475+0000","last_peered":"2024-09-16T07:37:17.654475+0000","last_clean":"2024-09-16T07:37:17.654475+0000","last_became_active":"2024-09-16T07:37:15.591522+0000","last_became_peered":"2024-09-16T07:37:15.591522+0000","last_unstale":"2024-09-16T07:37:17.654475+0000","last_undegraded":"2024-09-16T07:37:17.654475+0000","last_fullsized":"2024-09-16T07:37:17.654475+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:38:56.568301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":340,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601449+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:37:17.601449+0000","last_peered":"2024-09-16T07:37:17.601449+0000","last_clean":"2024-09-16T07:37:17.601449+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:37:17.601449+0000","last_undegraded":"2024-09-16T07:37:17.601449+0000","last_fullsized":"2024-09-16T07:37:17.601449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476917,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135632,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236208,"statfs":{"total":96636764160,"available":96497876992,"internally_reserved":0,"allocated":1822720,"data_stored":801982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476917,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131468,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240372,"statfs":{"total":96636764160,"available":96502140928,"internally_reserved":0,"allocated":1818624,"data_stored":801388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476917,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132048,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239792,"statfs":{"total":96636764160,"available":96501547008,"internally_reserved":0,"allocated":2412544,"data_stored":1391804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149508,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222332,"statfs":{"total":96636764160,"available":96483667968,"internally_reserved":0,"allocated":1818624,"data_stored":801456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509618,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131444,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240396,"statfs":{"total":96636764160,"available":96502165504,"internally_reserved":0,"allocated":1802240,"data_stored":794982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509620,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159236,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212604,"statfs":{"total":96636764160,"available":96473706496,"internally_reserved":0,"allocated":1814528,"data_stored":801366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161216966},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509620,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123748,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248092,"statfs":{"total":96636764160,"available":96510046208,"internally_reserved":0,"allocated":2433024,"data_stored":1393540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542322,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118956,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252884,"statfs":{"total":96636764160,"available":96514953216,"internally_reserved":0,"allocated":1789952,"data_stored":788123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:37:20.798 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:37:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.992+0000 7f7e31a00640 1 -- 172.21.15.71:0/2914412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0725d0 msgr2=0x7f7e2c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.992+0000 7f7e31a00640 1 --2- 172.21.15.71:0/2914412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0725d0 0x7f7e2c0b28f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009860 tx=0x7f7e1802a5a0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.992+0000 7f7e31a00640 1 -- 172.21.15.71:0/2914412185 shutdown_connections 2024-09-16T07:37:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.992+0000 7f7e31a00640 1 --2- 172.21.15.71:0/2914412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0725d0 0x7f7e2c0b28f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.992+0000 7f7e31a00640 1 -- 172.21.15.71:0/2914412185 >> 172.21.15.71:0/2914412185 conn(0x7f7e2c0cfa20 msgr2=0x7f7e2c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.992+0000 7f7e31a00640 1 -- 172.21.15.71:0/2914412185 shutdown_connections 2024-09-16T07:37:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.992+0000 7f7e31a00640 1 -- 172.21.15.71:0/2914412185 wait complete. 2024-09-16T07:37:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.993+0000 7f7e31a00640 1 Processor -- start 2024-09-16T07:37:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.994+0000 7f7e31a00640 1 -- start start 2024-09-16T07:37:20.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.994+0000 7f7e31a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0d0540 0x7f7e2c1f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.994+0000 7f7e31a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2c15bc30 0x7f7e2c15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.994+0000 7f7e31a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e2c149df0 0x7f7e2c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.994+0000 7f7e31a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c074660 con 0x7f7e2c149df0 2024-09-16T07:37:20.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.994+0000 7f7e31a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c06fa30 con 0x7f7e2c15bc30 2024-09-16T07:37:20.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.994+0000 7f7e31a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c15d980 con 0x7f7e2c0d0540 2024-09-16T07:37:20.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e2c149df0 0x7f7e2c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e2c149df0 0x7f7e2c0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57140/0 (socket says 172.21.15.71:57140) 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 -- 172.21.15.71:0/1694879603 learned_addr learned my addr 172.21.15.71:0/1694879603 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e2b400640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2c15bc30 0x7f7e2c15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e2be00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0d0540 0x7f7e2c1f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 -- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0d0540 msgr2=0x7f7e2c1f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0d0540 0x7f7e2c1f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 -- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2c15bc30 msgr2=0x7f7e2c15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2c15bc30 0x7f7e2c15d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 -- 172.21.15.71:0/1694879603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c1f83b0 con 0x7f7e2c149df0 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e2be00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0d0540 0x7f7e2c1f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e2b400640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2c15bc30 0x7f7e2c15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.995+0000 7f7e31000640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e2c149df0 0x7f7e2c0cd900 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c780 tx=0x7f7e2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.996+0000 7f7e23e00640 1 -- 172.21.15.71:0/1694879603 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24003b70 con 0x7f7e2c149df0 2024-09-16T07:37:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.996+0000 7f7e23e00640 1 -- 172.21.15.71:0/1694879603 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e24003d10 con 0x7f7e2c149df0 2024-09-16T07:37:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.996+0000 7f7e23e00640 1 -- 172.21.15.71:0/1694879603 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400aab0 con 0x7f7e2c149df0 2024-09-16T07:37:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.996+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c1f8690 con 0x7f7e2c149df0 2024-09-16T07:37:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.996+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c1f8c50 con 0x7f7e2c149df0 2024-09-16T07:37:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.997+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c0b28f0 con 0x7f7e2c149df0 2024-09-16T07:37:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.998+0000 7f7e23e00640 1 -- 172.21.15.71:0/1694879603 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f7e2400ac50 con 0x7f7e2c149df0 2024-09-16T07:37:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.999+0000 7f7e23e00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e0c07aa10 0x7f7e0c07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:20.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.999+0000 7f7e23e00640 1 -- 172.21.15.71:0/1694879603 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f7e240999d0 con 0x7f7e2c149df0 2024-09-16T07:37:20.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:20.999+0000 7f7e2be00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e0c07aa10 0x7f7e0c07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:20.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.000+0000 7f7e2be00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e0c07aa10 0x7f7e0c07ced0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c001220 tx=0x7f7e1c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:21.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.003+0000 7f7e23e00640 1 -- 172.21.15.71:0/1694879603 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7e24063ac0 con 0x7f7e2c149df0 2024-09-16T07:37:21.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.183+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e2c0cd330 con 0x7f7e0c07aa10 2024-09-16T07:37:21.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.188+0000 7f7e23e00640 1 -- 172.21.15.71:0/1694879603 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252495 (secure 0 0 0) 0x7f7e2c0cd330 con 0x7f7e0c07aa10 2024-09-16T07:37:21.191 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:21.192 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:37:21.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e0c07aa10 msgr2=0x7f7e0c07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e0c07aa10 0x7f7e0c07ced0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c001220 tx=0x7f7e1c009040 comp rx=0 tx=0).stop 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e2c149df0 msgr2=0x7f7e2c0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e2c149df0 0x7f7e2c0cd900 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c780 tx=0x7f7e2400cc40 comp rx=0 tx=0).stop 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 shutdown_connections 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f7e0c07aa10 0x7f7e0c07ced0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e2c149df0 0x7f7e2c0cd900 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2c15bc30 0x7f7e2c15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 --2- 172.21.15.71:0/1694879603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e2c0d0540 0x7f7e2c1f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 >> 172.21.15.71:0/1694879603 conn(0x7f7e2c0cfa20 msgr2=0x7f7e2c0ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 shutdown_connections 2024-09-16T07:37:21.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.192+0000 7f7e31a00640 1 -- 172.21.15.71:0/1694879603 wait complete. 2024-09-16T07:37:21.207 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":581,"stamp":"2024-09-16T07:37:20.645444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1082040,"kb_used_data":15344,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753892680,"statfs":{"total":773094113280,"available":771986104320,"internally_reserved":0,"allocated":15712256,"data_stored":7574641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1092079152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001323"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.616647+0000","last_change":"2024-09-16T07:37:16.612115+0000","last_active":"2024-09-16T07:37:16.616647+0000","last_peered":"2024-09-16T07:37:16.616647+0000","last_clean":"2024-09-16T07:37:16.616647+0000","last_became_active":"2024-09-16T07:37:16.608130+0000","last_became_peered":"2024-09-16T07:37:16.608130+0000","last_unstale":"2024-09-16T07:37:16.616647+0000","last_undegraded":"2024-09-16T07:37:16.616647+0000","last_fullsized":"2024-09-16T07:37:16.616647+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:19:04.216562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686059+0000","last_change":"2024-09-16T07:37:16.622991+0000","last_active":"2024-09-16T07:37:17.686059+0000","last_peered":"2024-09-16T07:37:17.686059+0000","last_clean":"2024-09-16T07:37:17.686059+0000","last_became_active":"2024-09-16T07:37:16.622051+0000","last_became_peered":"2024-09-16T07:37:16.622051+0000","last_unstale":"2024-09-16T07:37:17.686059+0000","last_undegraded":"2024-09-16T07:37:17.686059+0000","last_fullsized":"2024-09-16T07:37:17.686059+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:05:50.480562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685732+0000","last_change":"2024-09-16T07:37:16.603985+0000","last_active":"2024-09-16T07:37:17.685732+0000","last_peered":"2024-09-16T07:37:17.685732+0000","last_clean":"2024-09-16T07:37:17.685732+0000","last_became_active":"2024-09-16T07:37:16.603103+0000","last_became_peered":"2024-09-16T07:37:16.603103+0000","last_unstale":"2024-09-16T07:37:17.685732+0000","last_undegraded":"2024-09-16T07:37:17.685732+0000","last_fullsized":"2024-09-16T07:37:17.685732+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:29.032286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685566+0000","last_change":"2024-09-16T07:37:16.611845+0000","last_active":"2024-09-16T07:37:17.685566+0000","last_peered":"2024-09-16T07:37:17.685566+0000","last_clean":"2024-09-16T07:37:17.685566+0000","last_became_active":"2024-09-16T07:37:16.611110+0000","last_became_peered":"2024-09-16T07:37:16.611110+0000","last_unstale":"2024-09-16T07:37:17.685566+0000","last_undegraded":"2024-09-16T07:37:17.685566+0000","last_fullsized":"2024-09-16T07:37:17.685566+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:13:15.329105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601935+0000","last_change":"2024-09-16T07:37:16.599113+0000","last_active":"2024-09-16T07:37:17.601935+0000","last_peered":"2024-09-16T07:37:17.601935+0000","last_clean":"2024-09-16T07:37:17.601935+0000","last_became_active":"2024-09-16T07:37:16.598622+0000","last_became_peered":"2024-09-16T07:37:16.598622+0000","last_unstale":"2024-09-16T07:37:17.601935+0000","last_undegraded":"2024-09-16T07:37:17.601935+0000","last_fullsized":"2024-09-16T07:37:17.601935+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:26:52.294335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602813+0000","last_change":"2024-09-16T07:37:16.612001+0000","last_active":"2024-09-16T07:37:17.602813+0000","last_peered":"2024-09-16T07:37:17.602813+0000","last_clean":"2024-09-16T07:37:17.602813+0000","last_became_active":"2024-09-16T07:37:16.611749+0000","last_became_peered":"2024-09-16T07:37:16.611749+0000","last_unstale":"2024-09-16T07:37:17.602813+0000","last_undegraded":"2024-09-16T07:37:17.602813+0000","last_fullsized":"2024-09-16T07:37:17.602813+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:13:57.251738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603787+0000","last_change":"2024-09-16T07:37:16.599337+0000","last_active":"2024-09-16T07:37:17.603787+0000","last_peered":"2024-09-16T07:37:17.603787+0000","last_clean":"2024-09-16T07:37:17.603787+0000","last_became_active":"2024-09-16T07:37:16.599022+0000","last_became_peered":"2024-09-16T07:37:16.599022+0000","last_unstale":"2024-09-16T07:37:17.603787+0000","last_undegraded":"2024-09-16T07:37:17.603787+0000","last_fullsized":"2024-09-16T07:37:17.603787+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:59.704586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.613852+0000","last_change":"2024-09-16T07:37:16.604546+0000","last_active":"2024-09-16T07:37:16.613852+0000","last_peered":"2024-09-16T07:37:16.613852+0000","last_clean":"2024-09-16T07:37:16.613852+0000","last_became_active":"2024-09-16T07:37:16.604198+0000","last_became_peered":"2024-09-16T07:37:16.604198+0000","last_unstale":"2024-09-16T07:37:16.613852+0000","last_undegraded":"2024-09-16T07:37:16.613852+0000","last_fullsized":"2024-09-16T07:37:16.613852+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:09.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602853+0000","last_change":"2024-09-16T07:37:16.603290+0000","last_active":"2024-09-16T07:37:17.602853+0000","last_peered":"2024-09-16T07:37:17.602853+0000","last_clean":"2024-09-16T07:37:17.602853+0000","last_became_active":"2024-09-16T07:37:16.602468+0000","last_became_peered":"2024-09-16T07:37:16.602468+0000","last_unstale":"2024-09-16T07:37:17.602853+0000","last_undegraded":"2024-09-16T07:37:17.602853+0000","last_fullsized":"2024-09-16T07:37:17.602853+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:21.924648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685735+0000","last_change":"2024-09-16T07:37:16.604257+0000","last_active":"2024-09-16T07:37:17.685735+0000","last_peered":"2024-09-16T07:37:17.685735+0000","last_clean":"2024-09-16T07:37:17.685735+0000","last_became_active":"2024-09-16T07:37:16.603950+0000","last_became_peered":"2024-09-16T07:37:16.603950+0000","last_unstale":"2024-09-16T07:37:17.685735+0000","last_undegraded":"2024-09-16T07:37:17.685735+0000","last_fullsized":"2024-09-16T07:37:17.685735+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:31.650789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603309+0000","last_change":"2024-09-16T07:37:16.609966+0000","last_active":"2024-09-16T07:37:17.603309+0000","last_peered":"2024-09-16T07:37:17.603309+0000","last_clean":"2024-09-16T07:37:17.603309+0000","last_became_active":"2024-09-16T07:37:16.608454+0000","last_became_peered":"2024-09-16T07:37:16.608454+0000","last_unstale":"2024-09-16T07:37:17.603309+0000","last_undegraded":"2024-09-16T07:37:17.603309+0000","last_fullsized":"2024-09-16T07:37:17.603309+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:09.901271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654312+0000","last_change":"2024-09-16T07:37:16.603062+0000","last_active":"2024-09-16T07:37:17.654312+0000","last_peered":"2024-09-16T07:37:17.654312+0000","last_clean":"2024-09-16T07:37:17.654312+0000","last_became_active":"2024-09-16T07:37:16.602212+0000","last_became_peered":"2024-09-16T07:37:16.602212+0000","last_unstale":"2024-09-16T07:37:17.654312+0000","last_undegraded":"2024-09-16T07:37:17.654312+0000","last_fullsized":"2024-09-16T07:37:17.654312+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:28.614057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601902+0000","last_change":"2024-09-16T07:37:16.600584+0000","last_active":"2024-09-16T07:37:17.601902+0000","last_peered":"2024-09-16T07:37:17.601902+0000","last_clean":"2024-09-16T07:37:17.601902+0000","last_became_active":"2024-09-16T07:37:16.598273+0000","last_became_peered":"2024-09-16T07:37:16.598273+0000","last_unstale":"2024-09-16T07:37:17.601902+0000","last_undegraded":"2024-09-16T07:37:17.601902+0000","last_fullsized":"2024-09-16T07:37:17.601902+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:23.631603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654118+0000","last_change":"2024-09-16T07:37:16.604047+0000","last_active":"2024-09-16T07:37:17.654118+0000","last_peered":"2024-09-16T07:37:17.654118+0000","last_clean":"2024-09-16T07:37:17.654118+0000","last_became_active":"2024-09-16T07:37:16.603784+0000","last_became_peered":"2024-09-16T07:37:16.603784+0000","last_unstale":"2024-09-16T07:37:17.654118+0000","last_undegraded":"2024-09-16T07:37:17.654118+0000","last_fullsized":"2024-09-16T07:37:17.654118+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:00.251509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601741+0000","last_change":"2024-09-16T07:37:16.616467+0000","last_active":"2024-09-16T07:37:17.601741+0000","last_peered":"2024-09-16T07:37:17.601741+0000","last_clean":"2024-09-16T07:37:17.601741+0000","last_became_active":"2024-09-16T07:37:16.616127+0000","last_became_peered":"2024-09-16T07:37:16.616127+0000","last_unstale":"2024-09-16T07:37:17.601741+0000","last_undegraded":"2024-09-16T07:37:17.601741+0000","last_fullsized":"2024-09-16T07:37:17.601741+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:46.157557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604530+0000","last_change":"2024-09-16T07:37:16.596723+0000","last_active":"2024-09-16T07:37:17.604530+0000","last_peered":"2024-09-16T07:37:17.604530+0000","last_clean":"2024-09-16T07:37:17.604530+0000","last_became_active":"2024-09-16T07:37:16.596382+0000","last_became_peered":"2024-09-16T07:37:16.596382+0000","last_unstale":"2024-09-16T07:37:17.604530+0000","last_undegraded":"2024-09-16T07:37:17.604530+0000","last_fullsized":"2024-09-16T07:37:17.604530+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:18:10.792519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685660+0000","last_change":"2024-09-16T07:37:16.605843+0000","last_active":"2024-09-16T07:37:17.685660+0000","last_peered":"2024-09-16T07:37:17.685660+0000","last_clean":"2024-09-16T07:37:17.685660+0000","last_became_active":"2024-09-16T07:37:16.605575+0000","last_became_peered":"2024-09-16T07:37:16.605575+0000","last_unstale":"2024-09-16T07:37:17.685660+0000","last_undegraded":"2024-09-16T07:37:17.685660+0000","last_fullsized":"2024-09-16T07:37:17.685660+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:23.527726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602120+0000","last_change":"2024-09-16T07:37:16.596469+0000","last_active":"2024-09-16T07:37:17.602120+0000","last_peered":"2024-09-16T07:37:17.602120+0000","last_clean":"2024-09-16T07:37:17.602120+0000","last_became_active":"2024-09-16T07:37:16.596220+0000","last_became_peered":"2024-09-16T07:37:16.596220+0000","last_unstale":"2024-09-16T07:37:17.602120+0000","last_undegraded":"2024-09-16T07:37:17.602120+0000","last_fullsized":"2024-09-16T07:37:17.602120+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:33.615526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.618542+0000","last_change":"2024-09-16T07:37:16.616585+0000","last_active":"2024-09-16T07:37:16.618542+0000","last_peered":"2024-09-16T07:37:16.618542+0000","last_clean":"2024-09-16T07:37:16.618542+0000","last_became_active":"2024-09-16T07:37:16.615817+0000","last_became_peered":"2024-09-16T07:37:16.615817+0000","last_unstale":"2024-09-16T07:37:16.618542+0000","last_undegraded":"2024-09-16T07:37:16.618542+0000","last_fullsized":"2024-09-16T07:37:16.618542+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:30:45.044712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.618742+0000","last_change":"2024-09-16T07:37:16.615823+0000","last_active":"2024-09-16T07:37:16.618742+0000","last_peered":"2024-09-16T07:37:16.618742+0000","last_clean":"2024-09-16T07:37:16.618742+0000","last_became_active":"2024-09-16T07:37:16.614368+0000","last_became_peered":"2024-09-16T07:37:16.614368+0000","last_unstale":"2024-09-16T07:37:16.618742+0000","last_undegraded":"2024-09-16T07:37:16.618742+0000","last_fullsized":"2024-09-16T07:37:16.618742+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:38:23.217079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686169+0000","last_change":"2024-09-16T07:37:16.608203+0000","last_active":"2024-09-16T07:37:17.686169+0000","last_peered":"2024-09-16T07:37:17.686169+0000","last_clean":"2024-09-16T07:37:17.686169+0000","last_became_active":"2024-09-16T07:37:16.607881+0000","last_became_peered":"2024-09-16T07:37:16.607881+0000","last_unstale":"2024-09-16T07:37:17.686169+0000","last_undegraded":"2024-09-16T07:37:17.686169+0000","last_fullsized":"2024-09-16T07:37:17.686169+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:06.217492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654122+0000","last_change":"2024-09-16T07:37:16.597912+0000","last_active":"2024-09-16T07:37:17.654122+0000","last_peered":"2024-09-16T07:37:17.654122+0000","last_clean":"2024-09-16T07:37:17.654122+0000","last_became_active":"2024-09-16T07:37:16.597642+0000","last_became_peered":"2024-09-16T07:37:16.597642+0000","last_unstale":"2024-09-16T07:37:17.654122+0000","last_undegraded":"2024-09-16T07:37:17.654122+0000","last_fullsized":"2024-09-16T07:37:17.654122+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:46.975361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685094+0000","last_change":"2024-09-16T07:37:16.597799+0000","last_active":"2024-09-16T07:37:17.685094+0000","last_peered":"2024-09-16T07:37:17.685094+0000","last_clean":"2024-09-16T07:37:17.685094+0000","last_became_active":"2024-09-16T07:37:16.597507+0000","last_became_peered":"2024-09-16T07:37:16.597507+0000","last_unstale":"2024-09-16T07:37:17.685094+0000","last_undegraded":"2024-09-16T07:37:17.685094+0000","last_fullsized":"2024-09-16T07:37:17.685094+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:51.743308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601601+0000","last_change":"2024-09-16T07:37:16.602247+0000","last_active":"2024-09-16T07:37:17.601601+0000","last_peered":"2024-09-16T07:37:17.601601+0000","last_clean":"2024-09-16T07:37:17.601601+0000","last_became_active":"2024-09-16T07:37:16.598358+0000","last_became_peered":"2024-09-16T07:37:16.598358+0000","last_unstale":"2024-09-16T07:37:17.601601+0000","last_undegraded":"2024-09-16T07:37:17.601601+0000","last_fullsized":"2024-09-16T07:37:17.601601+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:23.407213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685953+0000","last_change":"2024-09-16T07:37:16.621878+0000","last_active":"2024-09-16T07:37:17.685953+0000","last_peered":"2024-09-16T07:37:17.685953+0000","last_clean":"2024-09-16T07:37:17.685953+0000","last_became_active":"2024-09-16T07:37:16.621088+0000","last_became_peered":"2024-09-16T07:37:16.621088+0000","last_unstale":"2024-09-16T07:37:17.685953+0000","last_undegraded":"2024-09-16T07:37:17.685953+0000","last_fullsized":"2024-09-16T07:37:17.685953+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:38:57.572392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.621478+0000","last_change":"2024-09-16T07:37:16.618890+0000","last_active":"2024-09-16T07:37:16.621478+0000","last_peered":"2024-09-16T07:37:16.621478+0000","last_clean":"2024-09-16T07:37:16.621478+0000","last_became_active":"2024-09-16T07:37:16.618099+0000","last_became_peered":"2024-09-16T07:37:16.618099+0000","last_unstale":"2024-09-16T07:37:16.621478+0000","last_undegraded":"2024-09-16T07:37:16.621478+0000","last_fullsized":"2024-09-16T07:37:16.621478+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:33.280212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685743+0000","last_change":"2024-09-16T07:37:16.615559+0000","last_active":"2024-09-16T07:37:17.685743+0000","last_peered":"2024-09-16T07:37:17.685743+0000","last_clean":"2024-09-16T07:37:17.685743+0000","last_became_active":"2024-09-16T07:37:16.615229+0000","last_became_peered":"2024-09-16T07:37:16.615229+0000","last_unstale":"2024-09-16T07:37:17.685743+0000","last_undegraded":"2024-09-16T07:37:17.685743+0000","last_fullsized":"2024-09-16T07:37:17.685743+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:34.699450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685692+0000","last_change":"2024-09-16T07:37:16.623386+0000","last_active":"2024-09-16T07:37:17.685692+0000","last_peered":"2024-09-16T07:37:17.685692+0000","last_clean":"2024-09-16T07:37:17.685692+0000","last_became_active":"2024-09-16T07:37:16.623116+0000","last_became_peered":"2024-09-16T07:37:16.623116+0000","last_unstale":"2024-09-16T07:37:17.685692+0000","last_undegraded":"2024-09-16T07:37:17.685692+0000","last_fullsized":"2024-09-16T07:37:17.685692+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:32:09.521160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.620611+0000","last_change":"2024-09-16T07:37:16.612904+0000","last_active":"2024-09-16T07:37:16.620611+0000","last_peered":"2024-09-16T07:37:16.620611+0000","last_clean":"2024-09-16T07:37:16.620611+0000","last_became_active":"2024-09-16T07:37:16.612651+0000","last_became_peered":"2024-09-16T07:37:16.612651+0000","last_unstale":"2024-09-16T07:37:16.620611+0000","last_undegraded":"2024-09-16T07:37:16.620611+0000","last_fullsized":"2024-09-16T07:37:16.620611+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:41.123149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654332+0000","last_change":"2024-09-16T07:37:16.599877+0000","last_active":"2024-09-16T07:37:17.654332+0000","last_peered":"2024-09-16T07:37:17.654332+0000","last_clean":"2024-09-16T07:37:17.654332+0000","last_became_active":"2024-09-16T07:37:16.599587+0000","last_became_peered":"2024-09-16T07:37:16.599587+0000","last_unstale":"2024-09-16T07:37:17.654332+0000","last_undegraded":"2024-09-16T07:37:17.654332+0000","last_fullsized":"2024-09-16T07:37:17.654332+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:25.522308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685182+0000","last_change":"2024-09-16T07:37:16.619414+0000","last_active":"2024-09-16T07:37:17.685182+0000","last_peered":"2024-09-16T07:37:17.685182+0000","last_clean":"2024-09-16T07:37:17.685182+0000","last_became_active":"2024-09-16T07:37:16.618102+0000","last_became_peered":"2024-09-16T07:37:16.618102+0000","last_unstale":"2024-09-16T07:37:17.685182+0000","last_undegraded":"2024-09-16T07:37:17.685182+0000","last_fullsized":"2024-09-16T07:37:17.685182+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:28.117165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685153+0000","last_change":"2024-09-16T07:37:16.617056+0000","last_active":"2024-09-16T07:37:17.685153+0000","last_peered":"2024-09-16T07:37:17.685153+0000","last_clean":"2024-09-16T07:37:17.685153+0000","last_became_active":"2024-09-16T07:37:16.616489+0000","last_became_peered":"2024-09-16T07:37:16.616489+0000","last_unstale":"2024-09-16T07:37:17.685153+0000","last_undegraded":"2024-09-16T07:37:17.685153+0000","last_fullsized":"2024-09-16T07:37:17.685153+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:51.337001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602305+0000","last_change":"2024-09-16T07:37:16.601844+0000","last_active":"2024-09-16T07:37:17.602305+0000","last_peered":"2024-09-16T07:37:17.602305+0000","last_clean":"2024-09-16T07:37:17.602305+0000","last_became_active":"2024-09-16T07:37:16.600150+0000","last_became_peered":"2024-09-16T07:37:16.600150+0000","last_unstale":"2024-09-16T07:37:17.602305+0000","last_undegraded":"2024-09-16T07:37:17.602305+0000","last_fullsized":"2024-09-16T07:37:17.602305+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:31.475733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"145'4","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685757+0000","last_change":"2024-09-16T07:37:15.600312+0000","last_active":"2024-09-16T07:37:17.685757+0000","last_peered":"2024-09-16T07:37:17.685757+0000","last_clean":"2024-09-16T07:37:17.685757+0000","last_became_active":"2024-09-16T07:37:15.600061+0000","last_became_peered":"2024-09-16T07:37:15.600061+0000","last_unstale":"2024-09-16T07:37:17.685757+0000","last_undegraded":"2024-09-16T07:37:17.685757+0000","last_fullsized":"2024-09-16T07:37:17.685757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:36.953048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604744+0000","last_change":"2024-09-16T07:37:16.605795+0000","last_active":"2024-09-16T07:37:17.604744+0000","last_peered":"2024-09-16T07:37:17.604744+0000","last_clean":"2024-09-16T07:37:17.604744+0000","last_became_active":"2024-09-16T07:37:16.605514+0000","last_became_peered":"2024-09-16T07:37:16.605514+0000","last_unstale":"2024-09-16T07:37:17.604744+0000","last_undegraded":"2024-09-16T07:37:17.604744+0000","last_fullsized":"2024-09-16T07:37:17.604744+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:54:20.004547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603384+0000","last_change":"2024-09-16T07:37:15.600366+0000","last_active":"2024-09-16T07:37:17.603384+0000","last_peered":"2024-09-16T07:37:17.603384+0000","last_clean":"2024-09-16T07:37:17.603384+0000","last_became_active":"2024-09-16T07:37:15.600098+0000","last_became_peered":"2024-09-16T07:37:15.600098+0000","last_unstale":"2024-09-16T07:37:17.603384+0000","last_undegraded":"2024-09-16T07:37:17.603384+0000","last_fullsized":"2024-09-16T07:37:17.603384+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:39:34.281801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602979+0000","last_change":"2024-09-16T07:37:16.605178+0000","last_active":"2024-09-16T07:37:17.602979+0000","last_peered":"2024-09-16T07:37:17.602979+0000","last_clean":"2024-09-16T07:37:17.602979+0000","last_became_active":"2024-09-16T07:37:16.604894+0000","last_became_peered":"2024-09-16T07:37:16.604894+0000","last_unstale":"2024-09-16T07:37:17.602979+0000","last_undegraded":"2024-09-16T07:37:17.602979+0000","last_fullsized":"2024-09-16T07:37:17.602979+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:14.975358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685826+0000","last_change":"2024-09-16T07:37:15.595761+0000","last_active":"2024-09-16T07:37:17.685826+0000","last_peered":"2024-09-16T07:37:17.685826+0000","last_clean":"2024-09-16T07:37:17.685826+0000","last_became_active":"2024-09-16T07:37:15.595387+0000","last_became_peered":"2024-09-16T07:37:15.595387+0000","last_unstale":"2024-09-16T07:37:17.685826+0000","last_undegraded":"2024-09-16T07:37:17.685826+0000","last_fullsized":"2024-09-16T07:37:17.685826+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:06.596941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685746+0000","last_change":"2024-09-16T07:37:16.604298+0000","last_active":"2024-09-16T07:37:17.685746+0000","last_peered":"2024-09-16T07:37:17.685746+0000","last_clean":"2024-09-16T07:37:17.685746+0000","last_became_active":"2024-09-16T07:37:16.601596+0000","last_became_peered":"2024-09-16T07:37:16.601596+0000","last_unstale":"2024-09-16T07:37:17.685746+0000","last_undegraded":"2024-09-16T07:37:17.685746+0000","last_fullsized":"2024-09-16T07:37:17.685746+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:12.794564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686427+0000","last_change":"2024-09-16T07:37:15.597770+0000","last_active":"2024-09-16T07:37:17.686427+0000","last_peered":"2024-09-16T07:37:17.686427+0000","last_clean":"2024-09-16T07:37:17.686427+0000","last_became_active":"2024-09-16T07:37:15.596291+0000","last_became_peered":"2024-09-16T07:37:15.596291+0000","last_unstale":"2024-09-16T07:37:17.686427+0000","last_undegraded":"2024-09-16T07:37:17.686427+0000","last_fullsized":"2024-09-16T07:37:17.686427+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:05.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686169+0000","last_change":"2024-09-16T07:37:16.619485+0000","last_active":"2024-09-16T07:37:17.686169+0000","last_peered":"2024-09-16T07:37:17.686169+0000","last_clean":"2024-09-16T07:37:17.686169+0000","last_became_active":"2024-09-16T07:37:16.619172+0000","last_became_peered":"2024-09-16T07:37:16.619172+0000","last_unstale":"2024-09-16T07:37:17.686169+0000","last_undegraded":"2024-09-16T07:37:17.686169+0000","last_fullsized":"2024-09-16T07:37:17.686169+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:28.449555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"145'1","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.612127+0000","last_change":"2024-09-16T07:37:15.600241+0000","last_active":"2024-09-16T07:37:16.612127+0000","last_peered":"2024-09-16T07:37:16.612127+0000","last_clean":"2024-09-16T07:37:16.612127+0000","last_became_active":"2024-09-16T07:37:15.599973+0000","last_became_peered":"2024-09-16T07:37:15.599973+0000","last_unstale":"2024-09-16T07:37:16.612127+0000","last_undegraded":"2024-09-16T07:37:16.612127+0000","last_fullsized":"2024-09-16T07:37:16.612127+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:41.517289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686016+0000","last_change":"2024-09-16T07:37:16.605737+0000","last_active":"2024-09-16T07:37:17.686016+0000","last_peered":"2024-09-16T07:37:17.686016+0000","last_clean":"2024-09-16T07:37:17.686016+0000","last_became_active":"2024-09-16T07:37:16.604805+0000","last_became_peered":"2024-09-16T07:37:16.604805+0000","last_unstale":"2024-09-16T07:37:17.686016+0000","last_undegraded":"2024-09-16T07:37:17.686016+0000","last_fullsized":"2024-09-16T07:37:17.686016+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:25.522308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601572+0000","last_change":"2024-09-16T07:37:15.601710+0000","last_active":"2024-09-16T07:37:17.601572+0000","last_peered":"2024-09-16T07:37:17.601572+0000","last_clean":"2024-09-16T07:37:17.601572+0000","last_became_active":"2024-09-16T07:37:15.601456+0000","last_became_peered":"2024-09-16T07:37:15.601456+0000","last_unstale":"2024-09-16T07:37:17.601572+0000","last_undegraded":"2024-09-16T07:37:17.601572+0000","last_fullsized":"2024-09-16T07:37:17.601572+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:51.317745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685137+0000","last_change":"2024-09-16T07:37:16.617921+0000","last_active":"2024-09-16T07:37:17.685137+0000","last_peered":"2024-09-16T07:37:17.685137+0000","last_clean":"2024-09-16T07:37:17.685137+0000","last_became_active":"2024-09-16T07:37:16.616812+0000","last_became_peered":"2024-09-16T07:37:16.616812+0000","last_unstale":"2024-09-16T07:37:17.685137+0000","last_undegraded":"2024-09-16T07:37:17.685137+0000","last_fullsized":"2024-09-16T07:37:17.685137+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:24.855633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602930+0000","last_change":"2024-09-16T07:37:15.604158+0000","last_active":"2024-09-16T07:37:17.602930+0000","last_peered":"2024-09-16T07:37:17.602930+0000","last_clean":"2024-09-16T07:37:17.602930+0000","last_became_active":"2024-09-16T07:37:15.603895+0000","last_became_peered":"2024-09-16T07:37:15.603895+0000","last_unstale":"2024-09-16T07:37:17.602930+0000","last_undegraded":"2024-09-16T07:37:17.602930+0000","last_fullsized":"2024-09-16T07:37:17.602930+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:27.609967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685281+0000","last_change":"2024-09-16T07:37:16.614133+0000","last_active":"2024-09-16T07:37:17.685281+0000","last_peered":"2024-09-16T07:37:17.685281+0000","last_clean":"2024-09-16T07:37:17.685281+0000","last_became_active":"2024-09-16T07:37:16.613846+0000","last_became_peered":"2024-09-16T07:37:16.613846+0000","last_unstale":"2024-09-16T07:37:17.685281+0000","last_undegraded":"2024-09-16T07:37:17.685281+0000","last_fullsized":"2024-09-16T07:37:17.685281+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:43.046079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685309+0000","last_change":"2024-09-16T07:37:15.608141+0000","last_active":"2024-09-16T07:37:17.685309+0000","last_peered":"2024-09-16T07:37:17.685309+0000","last_clean":"2024-09-16T07:37:17.685309+0000","last_became_active":"2024-09-16T07:37:15.607286+0000","last_became_peered":"2024-09-16T07:37:15.607286+0000","last_unstale":"2024-09-16T07:37:17.685309+0000","last_undegraded":"2024-09-16T07:37:17.685309+0000","last_fullsized":"2024-09-16T07:37:17.685309+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:22.497851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685607+0000","last_change":"2024-09-16T07:37:16.599443+0000","last_active":"2024-09-16T07:37:17.685607+0000","last_peered":"2024-09-16T07:37:17.685607+0000","last_clean":"2024-09-16T07:37:17.685607+0000","last_became_active":"2024-09-16T07:37:16.599176+0000","last_became_peered":"2024-09-16T07:37:16.599176+0000","last_unstale":"2024-09-16T07:37:17.685607+0000","last_undegraded":"2024-09-16T07:37:17.685607+0000","last_fullsized":"2024-09-16T07:37:17.685607+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:46.975361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602963+0000","last_change":"2024-09-16T07:37:15.607600+0000","last_active":"2024-09-16T07:37:17.602963+0000","last_peered":"2024-09-16T07:37:17.602963+0000","last_clean":"2024-09-16T07:37:17.602963+0000","last_became_active":"2024-09-16T07:37:15.606766+0000","last_became_peered":"2024-09-16T07:37:15.606766+0000","last_unstale":"2024-09-16T07:37:17.602963+0000","last_undegraded":"2024-09-16T07:37:17.602963+0000","last_fullsized":"2024-09-16T07:37:17.602963+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:24.518217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654608+0000","last_change":"2024-09-16T07:37:16.608093+0000","last_active":"2024-09-16T07:37:17.654608+0000","last_peered":"2024-09-16T07:37:17.654608+0000","last_clean":"2024-09-16T07:37:17.654608+0000","last_became_active":"2024-09-16T07:37:16.607818+0000","last_became_peered":"2024-09-16T07:37:16.607818+0000","last_unstale":"2024-09-16T07:37:17.654608+0000","last_undegraded":"2024-09-16T07:37:17.654608+0000","last_fullsized":"2024-09-16T07:37:17.654608+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:17.637595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602366+0000","last_change":"2024-09-16T07:37:15.599779+0000","last_active":"2024-09-16T07:37:17.602366+0000","last_peered":"2024-09-16T07:37:17.602366+0000","last_clean":"2024-09-16T07:37:17.602366+0000","last_became_active":"2024-09-16T07:37:15.599464+0000","last_became_peered":"2024-09-16T07:37:15.599464+0000","last_unstale":"2024-09-16T07:37:17.602366+0000","last_undegraded":"2024-09-16T07:37:17.602366+0000","last_fullsized":"2024-09-16T07:37:17.602366+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:16.485730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604699+0000","last_change":"2024-09-16T07:37:16.604052+0000","last_active":"2024-09-16T07:37:17.604699+0000","last_peered":"2024-09-16T07:37:17.604699+0000","last_clean":"2024-09-16T07:37:17.604699+0000","last_became_active":"2024-09-16T07:37:16.603638+0000","last_became_peered":"2024-09-16T07:37:16.603638+0000","last_unstale":"2024-09-16T07:37:17.604699+0000","last_undegraded":"2024-09-16T07:37:17.604699+0000","last_fullsized":"2024-09-16T07:37:17.604699+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:19:17.755613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"145'6","reported_seq":21,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604290+0000","last_change":"2024-09-16T07:37:15.596420+0000","last_active":"2024-09-16T07:37:17.604290+0000","last_peered":"2024-09-16T07:37:17.604290+0000","last_clean":"2024-09-16T07:37:17.604290+0000","last_became_active":"2024-09-16T07:37:15.596131+0000","last_became_peered":"2024-09-16T07:37:15.596131+0000","last_unstale":"2024-09-16T07:37:17.604290+0000","last_undegraded":"2024-09-16T07:37:17.604290+0000","last_fullsized":"2024-09-16T07:37:17.604290+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:27.606954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685155+0000","last_change":"2024-09-16T07:37:16.605491+0000","last_active":"2024-09-16T07:37:17.685155+0000","last_peered":"2024-09-16T07:37:17.685155+0000","last_clean":"2024-09-16T07:37:17.685155+0000","last_became_active":"2024-09-16T07:37:16.605236+0000","last_became_peered":"2024-09-16T07:37:16.605236+0000","last_unstale":"2024-09-16T07:37:17.685155+0000","last_undegraded":"2024-09-16T07:37:17.685155+0000","last_fullsized":"2024-09-16T07:37:17.685155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:42:23.350926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686263+0000","last_change":"2024-09-16T07:37:15.596066+0000","last_active":"2024-09-16T07:37:17.686263+0000","last_peered":"2024-09-16T07:37:17.686263+0000","last_clean":"2024-09-16T07:37:17.686263+0000","last_became_active":"2024-09-16T07:37:15.595812+0000","last_became_peered":"2024-09-16T07:37:15.595812+0000","last_unstale":"2024-09-16T07:37:17.686263+0000","last_undegraded":"2024-09-16T07:37:17.686263+0000","last_fullsized":"2024-09-16T07:37:17.686263+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:15.895320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.620584+0000","last_change":"2024-09-16T07:37:16.613421+0000","last_active":"2024-09-16T07:37:16.620584+0000","last_peered":"2024-09-16T07:37:16.620584+0000","last_clean":"2024-09-16T07:37:16.620584+0000","last_became_active":"2024-09-16T07:37:16.612333+0000","last_became_peered":"2024-09-16T07:37:16.612333+0000","last_unstale":"2024-09-16T07:37:16.620584+0000","last_undegraded":"2024-09-16T07:37:16.620584+0000","last_fullsized":"2024-09-16T07:37:16.620584+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:20:07.837786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654425+0000","last_change":"2024-09-16T07:37:15.593786+0000","last_active":"2024-09-16T07:37:17.654425+0000","last_peered":"2024-09-16T07:37:17.654425+0000","last_clean":"2024-09-16T07:37:17.654425+0000","last_became_active":"2024-09-16T07:37:15.593082+0000","last_became_peered":"2024-09-16T07:37:15.593082+0000","last_unstale":"2024-09-16T07:37:17.654425+0000","last_undegraded":"2024-09-16T07:37:17.654425+0000","last_fullsized":"2024-09-16T07:37:17.654425+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:07.749839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685069+0000","last_change":"2024-09-16T07:37:16.611529+0000","last_active":"2024-09-16T07:37:17.685069+0000","last_peered":"2024-09-16T07:37:17.685069+0000","last_clean":"2024-09-16T07:37:17.685069+0000","last_became_active":"2024-09-16T07:37:16.611200+0000","last_became_peered":"2024-09-16T07:37:16.611200+0000","last_unstale":"2024-09-16T07:37:17.685069+0000","last_undegraded":"2024-09-16T07:37:17.685069+0000","last_fullsized":"2024-09-16T07:37:17.685069+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:28.449555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685994+0000","last_change":"2024-09-16T07:37:15.608181+0000","last_active":"2024-09-16T07:37:17.685994+0000","last_peered":"2024-09-16T07:37:17.685994+0000","last_clean":"2024-09-16T07:37:17.685994+0000","last_became_active":"2024-09-16T07:37:15.607890+0000","last_became_peered":"2024-09-16T07:37:15.607890+0000","last_unstale":"2024-09-16T07:37:17.685994+0000","last_undegraded":"2024-09-16T07:37:17.685994+0000","last_fullsized":"2024-09-16T07:37:17.685994+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:22.627513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685274+0000","last_change":"2024-09-16T07:37:16.601574+0000","last_active":"2024-09-16T07:37:17.685274+0000","last_peered":"2024-09-16T07:37:17.685274+0000","last_clean":"2024-09-16T07:37:17.685274+0000","last_became_active":"2024-09-16T07:37:16.598772+0000","last_became_peered":"2024-09-16T07:37:16.598772+0000","last_unstale":"2024-09-16T07:37:17.685274+0000","last_undegraded":"2024-09-16T07:37:17.685274+0000","last_fullsized":"2024-09-16T07:37:17.685274+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:06:26.103572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"145'8","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.621959+0000","last_change":"2024-09-16T07:37:15.604167+0000","last_active":"2024-09-16T07:37:16.621959+0000","last_peered":"2024-09-16T07:37:16.621959+0000","last_clean":"2024-09-16T07:37:16.621959+0000","last_became_active":"2024-09-16T07:37:15.603883+0000","last_became_peered":"2024-09-16T07:37:15.603883+0000","last_unstale":"2024-09-16T07:37:16.621959+0000","last_undegraded":"2024-09-16T07:37:16.621959+0000","last_fullsized":"2024-09-16T07:37:16.621959+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:27:56.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602365+0000","last_change":"2024-09-16T07:37:16.605149+0000","last_active":"2024-09-16T07:37:17.602365+0000","last_peered":"2024-09-16T07:37:17.602365+0000","last_clean":"2024-09-16T07:37:17.602365+0000","last_became_active":"2024-09-16T07:37:16.604651+0000","last_became_peered":"2024-09-16T07:37:16.604651+0000","last_unstale":"2024-09-16T07:37:17.602365+0000","last_undegraded":"2024-09-16T07:37:17.602365+0000","last_fullsized":"2024-09-16T07:37:17.602365+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:12.910252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685941+0000","last_change":"2024-09-16T07:37:15.609324+0000","last_active":"2024-09-16T07:37:17.685941+0000","last_peered":"2024-09-16T07:37:17.685941+0000","last_clean":"2024-09-16T07:37:17.685941+0000","last_became_active":"2024-09-16T07:37:15.608788+0000","last_became_peered":"2024-09-16T07:37:15.608788+0000","last_unstale":"2024-09-16T07:37:17.685941+0000","last_undegraded":"2024-09-16T07:37:17.685941+0000","last_fullsized":"2024-09-16T07:37:17.685941+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:29:11.906162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685903+0000","last_change":"2024-09-16T07:37:16.619759+0000","last_active":"2024-09-16T07:37:17.685903+0000","last_peered":"2024-09-16T07:37:17.685903+0000","last_clean":"2024-09-16T07:37:17.685903+0000","last_became_active":"2024-09-16T07:37:16.619489+0000","last_became_peered":"2024-09-16T07:37:16.619489+0000","last_unstale":"2024-09-16T07:37:17.685903+0000","last_undegraded":"2024-09-16T07:37:17.685903+0000","last_fullsized":"2024-09-16T07:37:17.685903+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:08.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"145'2","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.615586+0000","last_change":"2024-09-16T07:37:15.595980+0000","last_active":"2024-09-16T07:37:16.615586+0000","last_peered":"2024-09-16T07:37:16.615586+0000","last_clean":"2024-09-16T07:37:16.615586+0000","last_became_active":"2024-09-16T07:37:15.595714+0000","last_became_peered":"2024-09-16T07:37:15.595714+0000","last_unstale":"2024-09-16T07:37:16.615586+0000","last_undegraded":"2024-09-16T07:37:16.615586+0000","last_fullsized":"2024-09-16T07:37:16.615586+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.934535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602006+0000","last_change":"2024-09-16T07:37:16.601387+0000","last_active":"2024-09-16T07:37:17.602006+0000","last_peered":"2024-09-16T07:37:17.602006+0000","last_clean":"2024-09-16T07:37:17.602006+0000","last_became_active":"2024-09-16T07:37:16.601026+0000","last_became_peered":"2024-09-16T07:37:16.601026+0000","last_unstale":"2024-09-16T07:37:17.602006+0000","last_undegraded":"2024-09-16T07:37:17.602006+0000","last_fullsized":"2024-09-16T07:37:17.602006+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:18.850568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"145'1","reported_seq":16,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604818+0000","last_change":"2024-09-16T07:37:15.601467+0000","last_active":"2024-09-16T07:37:17.604818+0000","last_peered":"2024-09-16T07:37:17.604818+0000","last_clean":"2024-09-16T07:37:17.604818+0000","last_became_active":"2024-09-16T07:37:15.601138+0000","last_became_peered":"2024-09-16T07:37:15.601138+0000","last_unstale":"2024-09-16T07:37:17.604818+0000","last_undegraded":"2024-09-16T07:37:17.604818+0000","last_fullsized":"2024-09-16T07:37:17.604818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:05.895984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603497+0000","last_change":"2024-09-16T07:37:16.608026+0000","last_active":"2024-09-16T07:37:17.603497+0000","last_peered":"2024-09-16T07:37:17.603497+0000","last_clean":"2024-09-16T07:37:17.603497+0000","last_became_active":"2024-09-16T07:37:16.606029+0000","last_became_peered":"2024-09-16T07:37:16.606029+0000","last_unstale":"2024-09-16T07:37:17.603497+0000","last_undegraded":"2024-09-16T07:37:17.603497+0000","last_fullsized":"2024-09-16T07:37:17.603497+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:01:49.151592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685682+0000","last_change":"2024-09-16T07:37:15.604913+0000","last_active":"2024-09-16T07:37:17.685682+0000","last_peered":"2024-09-16T07:37:17.685682+0000","last_clean":"2024-09-16T07:37:17.685682+0000","last_became_active":"2024-09-16T07:37:15.604587+0000","last_became_peered":"2024-09-16T07:37:15.604587+0000","last_unstale":"2024-09-16T07:37:17.685682+0000","last_undegraded":"2024-09-16T07:37:17.685682+0000","last_fullsized":"2024-09-16T07:37:17.685682+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:19:45.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654339+0000","last_change":"2024-09-16T07:37:16.613535+0000","last_active":"2024-09-16T07:37:17.654339+0000","last_peered":"2024-09-16T07:37:17.654339+0000","last_clean":"2024-09-16T07:37:17.654339+0000","last_became_active":"2024-09-16T07:37:16.613239+0000","last_became_peered":"2024-09-16T07:37:16.613239+0000","last_unstale":"2024-09-16T07:37:17.654339+0000","last_undegraded":"2024-09-16T07:37:17.654339+0000","last_fullsized":"2024-09-16T07:37:17.654339+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:24.457755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.603125+0000","last_change":"2024-09-16T07:37:15.598775+0000","last_active":"2024-09-16T07:37:17.603125+0000","last_peered":"2024-09-16T07:37:17.603125+0000","last_clean":"2024-09-16T07:37:17.603125+0000","last_became_active":"2024-09-16T07:37:15.598352+0000","last_became_peered":"2024-09-16T07:37:15.598352+0000","last_unstale":"2024-09-16T07:37:17.603125+0000","last_undegraded":"2024-09-16T07:37:17.603125+0000","last_fullsized":"2024-09-16T07:37:17.603125+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:54.665554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685486+0000","last_change":"2024-09-16T07:37:16.615046+0000","last_active":"2024-09-16T07:37:17.685486+0000","last_peered":"2024-09-16T07:37:17.685486+0000","last_clean":"2024-09-16T07:37:17.685486+0000","last_became_active":"2024-09-16T07:37:16.614753+0000","last_became_peered":"2024-09-16T07:37:16.614753+0000","last_unstale":"2024-09-16T07:37:17.685486+0000","last_undegraded":"2024-09-16T07:37:17.685486+0000","last_fullsized":"2024-09-16T07:37:17.685486+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:31.393623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686122+0000","last_change":"2024-09-16T07:37:15.605799+0000","last_active":"2024-09-16T07:37:17.686122+0000","last_peered":"2024-09-16T07:37:17.686122+0000","last_clean":"2024-09-16T07:37:17.686122+0000","last_became_active":"2024-09-16T07:37:15.605538+0000","last_became_peered":"2024-09-16T07:37:15.605538+0000","last_unstale":"2024-09-16T07:37:17.686122+0000","last_undegraded":"2024-09-16T07:37:17.686122+0000","last_fullsized":"2024-09-16T07:37:17.686122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:29.344622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654123+0000","last_change":"2024-09-16T07:37:16.596498+0000","last_active":"2024-09-16T07:37:17.654123+0000","last_peered":"2024-09-16T07:37:17.654123+0000","last_clean":"2024-09-16T07:37:17.654123+0000","last_became_active":"2024-09-16T07:37:16.596227+0000","last_became_peered":"2024-09-16T07:37:16.596227+0000","last_unstale":"2024-09-16T07:37:17.654123+0000","last_undegraded":"2024-09-16T07:37:17.654123+0000","last_fullsized":"2024-09-16T07:37:17.654123+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:55.669644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685370+0000","last_change":"2024-09-16T07:37:15.601619+0000","last_active":"2024-09-16T07:37:17.685370+0000","last_peered":"2024-09-16T07:37:17.685370+0000","last_clean":"2024-09-16T07:37:17.685370+0000","last_became_active":"2024-09-16T07:37:15.601344+0000","last_became_peered":"2024-09-16T07:37:15.601344+0000","last_unstale":"2024-09-16T07:37:17.685370+0000","last_undegraded":"2024-09-16T07:37:17.685370+0000","last_fullsized":"2024-09-16T07:37:17.685370+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:21:36.453021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685177+0000","last_change":"2024-09-16T07:37:16.618459+0000","last_active":"2024-09-16T07:37:17.685177+0000","last_peered":"2024-09-16T07:37:17.685177+0000","last_clean":"2024-09-16T07:37:17.685177+0000","last_became_active":"2024-09-16T07:37:16.618146+0000","last_became_peered":"2024-09-16T07:37:16.618146+0000","last_unstale":"2024-09-16T07:37:17.685177+0000","last_undegraded":"2024-09-16T07:37:17.685177+0000","last_fullsized":"2024-09-16T07:37:17.685177+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.914629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685190+0000","last_change":"2024-09-16T07:37:15.605579+0000","last_active":"2024-09-16T07:37:17.685190+0000","last_peered":"2024-09-16T07:37:17.685190+0000","last_clean":"2024-09-16T07:37:17.685190+0000","last_became_active":"2024-09-16T07:37:15.605107+0000","last_became_peered":"2024-09-16T07:37:15.605107+0000","last_unstale":"2024-09-16T07:37:17.685190+0000","last_undegraded":"2024-09-16T07:37:17.685190+0000","last_fullsized":"2024-09-16T07:37:17.685190+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:52.968288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685792+0000","last_change":"2024-09-16T07:37:16.612546+0000","last_active":"2024-09-16T07:37:17.685792+0000","last_peered":"2024-09-16T07:37:17.685792+0000","last_clean":"2024-09-16T07:37:17.685792+0000","last_became_active":"2024-09-16T07:37:16.612301+0000","last_became_peered":"2024-09-16T07:37:16.612301+0000","last_unstale":"2024-09-16T07:37:17.685792+0000","last_undegraded":"2024-09-16T07:37:17.685792+0000","last_fullsized":"2024-09-16T07:37:17.685792+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:31.575258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"145'2","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686145+0000","last_change":"2024-09-16T07:37:15.608951+0000","last_active":"2024-09-16T07:37:17.686145+0000","last_peered":"2024-09-16T07:37:17.686145+0000","last_clean":"2024-09-16T07:37:17.686145+0000","last_became_active":"2024-09-16T07:37:15.608682+0000","last_became_peered":"2024-09-16T07:37:15.608682+0000","last_unstale":"2024-09-16T07:37:17.686145+0000","last_undegraded":"2024-09-16T07:37:17.686145+0000","last_fullsized":"2024-09-16T07:37:17.686145+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:19.491042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686016+0000","last_change":"2024-09-16T07:37:16.613996+0000","last_active":"2024-09-16T07:37:17.686016+0000","last_peered":"2024-09-16T07:37:17.686016+0000","last_clean":"2024-09-16T07:37:17.686016+0000","last_became_active":"2024-09-16T07:37:16.613625+0000","last_became_peered":"2024-09-16T07:37:16.613625+0000","last_unstale":"2024-09-16T07:37:17.686016+0000","last_undegraded":"2024-09-16T07:37:17.686016+0000","last_fullsized":"2024-09-16T07:37:17.686016+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:38.509608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686218+0000","last_change":"2024-09-16T07:37:15.607833+0000","last_active":"2024-09-16T07:37:17.686218+0000","last_peered":"2024-09-16T07:37:17.686218+0000","last_clean":"2024-09-16T07:37:17.686218+0000","last_became_active":"2024-09-16T07:37:15.607243+0000","last_became_peered":"2024-09-16T07:37:15.607243+0000","last_unstale":"2024-09-16T07:37:17.686218+0000","last_undegraded":"2024-09-16T07:37:17.686218+0000","last_fullsized":"2024-09-16T07:37:17.686218+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:26:51.290244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.618516+0000","last_change":"2024-09-16T07:37:16.614649+0000","last_active":"2024-09-16T07:37:16.618516+0000","last_peered":"2024-09-16T07:37:16.618516+0000","last_clean":"2024-09-16T07:37:16.618516+0000","last_became_active":"2024-09-16T07:37:16.614200+0000","last_became_peered":"2024-09-16T07:37:16.614200+0000","last_unstale":"2024-09-16T07:37:16.618516+0000","last_undegraded":"2024-09-16T07:37:16.618516+0000","last_fullsized":"2024-09-16T07:37:16.618516+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:26:12.262888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604878+0000","last_change":"2024-09-16T07:37:15.599948+0000","last_active":"2024-09-16T07:37:17.604878+0000","last_peered":"2024-09-16T07:37:17.604878+0000","last_clean":"2024-09-16T07:37:17.604878+0000","last_became_active":"2024-09-16T07:37:15.598850+0000","last_became_peered":"2024-09-16T07:37:15.598850+0000","last_unstale":"2024-09-16T07:37:17.604878+0000","last_undegraded":"2024-09-16T07:37:17.604878+0000","last_fullsized":"2024-09-16T07:37:17.604878+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:30:50.418816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T07:37:16.618476+0000","last_change":"2024-09-16T07:37:16.612993+0000","last_active":"2024-09-16T07:37:16.618476+0000","last_peered":"2024-09-16T07:37:16.618476+0000","last_clean":"2024-09-16T07:37:16.618476+0000","last_became_active":"2024-09-16T07:37:16.612744+0000","last_became_peered":"2024-09-16T07:37:16.612744+0000","last_unstale":"2024-09-16T07:37:16.618476+0000","last_undegraded":"2024-09-16T07:37:16.618476+0000","last_fullsized":"2024-09-16T07:37:16.618476+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:25.165694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604355+0000","last_change":"2024-09-16T07:37:15.602241+0000","last_active":"2024-09-16T07:37:17.604355+0000","last_peered":"2024-09-16T07:37:17.604355+0000","last_clean":"2024-09-16T07:37:17.604355+0000","last_became_active":"2024-09-16T07:37:15.601952+0000","last_became_peered":"2024-09-16T07:37:15.601952+0000","last_unstale":"2024-09-16T07:37:17.604355+0000","last_undegraded":"2024-09-16T07:37:17.604355+0000","last_fullsized":"2024-09-16T07:37:17.604355+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:20.185672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604121+0000","last_change":"2024-09-16T07:37:16.603932+0000","last_active":"2024-09-16T07:37:17.604121+0000","last_peered":"2024-09-16T07:37:17.604121+0000","last_clean":"2024-09-16T07:37:17.604121+0000","last_became_active":"2024-09-16T07:37:16.603324+0000","last_became_peered":"2024-09-16T07:37:16.603324+0000","last_unstale":"2024-09-16T07:37:17.604121+0000","last_undegraded":"2024-09-16T07:37:17.604121+0000","last_fullsized":"2024-09-16T07:37:17.604121+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:09.855435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685596+0000","last_change":"2024-09-16T07:37:15.605860+0000","last_active":"2024-09-16T07:37:17.685596+0000","last_peered":"2024-09-16T07:37:17.685596+0000","last_clean":"2024-09-16T07:37:17.685596+0000","last_became_active":"2024-09-16T07:37:15.605593+0000","last_became_peered":"2024-09-16T07:37:15.605593+0000","last_unstale":"2024-09-16T07:37:17.685596+0000","last_undegraded":"2024-09-16T07:37:17.685596+0000","last_fullsized":"2024-09-16T07:37:17.685596+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:22.403123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686193+0000","last_change":"2024-09-16T07:37:16.597229+0000","last_active":"2024-09-16T07:37:17.686193+0000","last_peered":"2024-09-16T07:37:17.686193+0000","last_clean":"2024-09-16T07:37:17.686193+0000","last_became_active":"2024-09-16T07:37:16.596969+0000","last_became_peered":"2024-09-16T07:37:16.596969+0000","last_unstale":"2024-09-16T07:37:17.686193+0000","last_undegraded":"2024-09-16T07:37:17.686193+0000","last_fullsized":"2024-09-16T07:37:17.686193+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:58:39.426464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.686091+0000","last_change":"2024-09-16T07:37:15.601520+0000","last_active":"2024-09-16T07:37:17.686091+0000","last_peered":"2024-09-16T07:37:17.686091+0000","last_clean":"2024-09-16T07:37:17.686091+0000","last_became_active":"2024-09-16T07:37:15.601276+0000","last_became_peered":"2024-09-16T07:37:15.601276+0000","last_unstale":"2024-09-16T07:37:17.686091+0000","last_undegraded":"2024-09-16T07:37:17.686091+0000","last_fullsized":"2024-09-16T07:37:17.686091+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:42:48.022898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685229+0000","last_change":"2024-09-16T07:37:16.609120+0000","last_active":"2024-09-16T07:37:17.685229+0000","last_peered":"2024-09-16T07:37:17.685229+0000","last_clean":"2024-09-16T07:37:17.685229+0000","last_became_active":"2024-09-16T07:37:16.608814+0000","last_became_peered":"2024-09-16T07:37:16.608814+0000","last_unstale":"2024-09-16T07:37:17.685229+0000","last_undegraded":"2024-09-16T07:37:17.685229+0000","last_fullsized":"2024-09-16T07:37:17.685229+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:08:15.077059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.653882+0000","last_change":"2024-09-16T07:37:15.591807+0000","last_active":"2024-09-16T07:37:17.653882+0000","last_peered":"2024-09-16T07:37:17.653882+0000","last_clean":"2024-09-16T07:37:17.653882+0000","last_became_active":"2024-09-16T07:37:15.591560+0000","last_became_peered":"2024-09-16T07:37:15.591560+0000","last_unstale":"2024-09-16T07:37:17.653882+0000","last_undegraded":"2024-09-16T07:37:17.653882+0000","last_fullsized":"2024-09-16T07:37:17.653882+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:33.695360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.685464+0000","last_change":"2024-09-16T07:37:16.615455+0000","last_active":"2024-09-16T07:37:17.685464+0000","last_peered":"2024-09-16T07:37:17.685464+0000","last_clean":"2024-09-16T07:37:17.685464+0000","last_became_active":"2024-09-16T07:37:16.614884+0000","last_became_peered":"2024-09-16T07:37:16.614884+0000","last_unstale":"2024-09-16T07:37:17.685464+0000","last_undegraded":"2024-09-16T07:37:17.685464+0000","last_fullsized":"2024-09-16T07:37:17.685464+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:31:57.493246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.602605+0000","last_change":"2024-09-16T07:37:15.601484+0000","last_active":"2024-09-16T07:37:17.602605+0000","last_peered":"2024-09-16T07:37:17.602605+0000","last_clean":"2024-09-16T07:37:17.602605+0000","last_became_active":"2024-09-16T07:37:15.600178+0000","last_became_peered":"2024-09-16T07:37:15.600178+0000","last_unstale":"2024-09-16T07:37:17.602605+0000","last_undegraded":"2024-09-16T07:37:17.602605+0000","last_fullsized":"2024-09-16T07:37:17.602605+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:56.630838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.604247+0000","last_change":"2024-09-16T07:37:16.606688+0000","last_active":"2024-09-16T07:37:17.604247+0000","last_peered":"2024-09-16T07:37:17.604247+0000","last_clean":"2024-09-16T07:37:17.604247+0000","last_became_active":"2024-09-16T07:37:16.603710+0000","last_became_peered":"2024-09-16T07:37:16.603710+0000","last_unstale":"2024-09-16T07:37:17.604247+0000","last_undegraded":"2024-09-16T07:37:17.604247+0000","last_fullsized":"2024-09-16T07:37:17.604247+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:15.574110+0000","last_clean_scrub_stamp":"2024-09-16T07:37:15.574110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:59.951624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"145'3","reported_seq":18,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.654475+0000","last_change":"2024-09-16T07:37:15.591979+0000","last_active":"2024-09-16T07:37:17.654475+0000","last_peered":"2024-09-16T07:37:17.654475+0000","last_clean":"2024-09-16T07:37:17.654475+0000","last_became_active":"2024-09-16T07:37:15.591522+0000","last_became_peered":"2024-09-16T07:37:15.591522+0000","last_unstale":"2024-09-16T07:37:17.654475+0000","last_undegraded":"2024-09-16T07:37:17.654475+0000","last_fullsized":"2024-09-16T07:37:17.654475+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:37:14.570020+0000","last_clean_scrub_stamp":"2024-09-16T07:37:14.570020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:38:56.568301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":340,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T07:37:17.601449+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:37:17.601449+0000","last_peered":"2024-09-16T07:37:17.601449+0000","last_clean":"2024-09-16T07:37:17.601449+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:37:17.601449+0000","last_undegraded":"2024-09-16T07:37:17.601449+0000","last_fullsized":"2024-09-16T07:37:17.601449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476917,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135632,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236208,"statfs":{"total":96636764160,"available":96497876992,"internally_reserved":0,"allocated":1822720,"data_stored":801982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476917,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131468,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240372,"statfs":{"total":96636764160,"available":96502140928,"internally_reserved":0,"allocated":1818624,"data_stored":801388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476917,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132048,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239792,"statfs":{"total":96636764160,"available":96501547008,"internally_reserved":0,"allocated":2412544,"data_stored":1391804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149508,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222332,"statfs":{"total":96636764160,"available":96483667968,"internally_reserved":0,"allocated":1818624,"data_stored":801456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509618,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131444,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240396,"statfs":{"total":96636764160,"available":96502165504,"internally_reserved":0,"allocated":1802240,"data_stored":794982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":132774346},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509620,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159236,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212604,"statfs":{"total":96636764160,"available":96473706496,"internally_reserved":0,"allocated":1814528,"data_stored":801366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161216966},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509620,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123748,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248092,"statfs":{"total":96636764160,"available":96510046208,"internally_reserved":0,"allocated":2433024,"data_stored":1393540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542322,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118956,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252884,"statfs":{"total":96636764160,"available":96514953216,"internally_reserved":0,"allocated":1789952,"data_stored":788123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:37:21.212 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:37:21.212 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T07:37:21.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.412+0000 7f327e600640 1 -- 172.21.15.71:0/3134899925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 msgr2=0x7f32780c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.412+0000 7f327e600640 1 --2- 172.21.15.71:0/3134899925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 0x7f32780c6430 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f3278075410 tx=0x7f326402a560 comp rx=0 tx=0).stop 2024-09-16T07:37:21.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.412+0000 7f327e600640 1 -- 172.21.15.71:0/3134899925 shutdown_connections 2024-09-16T07:37:21.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.412+0000 7f327e600640 1 --2- 172.21.15.71:0/3134899925 >> v2:172.21.15.177:3300/0 conn(0x7f327806e660 0x7f327806eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.412+0000 7f327e600640 1 --2- 172.21.15.71:0/3134899925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 0x7f32780c6430 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.412+0000 7f327e600640 1 -- 172.21.15.71:0/3134899925 >> 172.21.15.71:0/3134899925 conn(0x7f32780cc490 msgr2=0x7f32780cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:21.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.412+0000 7f327e600640 1 -- 172.21.15.71:0/3134899925 shutdown_connections 2024-09-16T07:37:21.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.412+0000 7f327e600640 1 -- 172.21.15.71:0/3134899925 wait complete. 2024-09-16T07:37:21.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.414+0000 7f327e600640 1 Processor -- start 2024-09-16T07:37:21.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.414+0000 7f327e600640 1 -- start start 2024-09-16T07:37:21.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f327e600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f327806e660 0x7f32780b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f327e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 0x7f32780b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f327e600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327815f670 0x7f327814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f327e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f32780ca4f0 con 0x7f32780c5730 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f327e600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32780ca370 con 0x7f327815f670 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f327e600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f327806f000 con 0x7f327806e660 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f3277e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 0x7f32780b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f3277e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 0x7f32780b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57150/0 (socket says 172.21.15.71:57150) 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f3277e00640 1 -- 172.21.15.71:0/3842117422 learned_addr learned my addr 172.21.15.71:0/3842117422 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f327d200640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f327806e660 0x7f32780b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.415+0000 7f327dc00640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327815f670 0x7f327814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3277e00640 1 -- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f327806e660 msgr2=0x7f32780b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3277e00640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f327806e660 0x7f32780b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3277e00640 1 -- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327815f670 msgr2=0x7f327814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3277e00640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327815f670 0x7f327814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3277e00640 1 -- 172.21.15.71:0/3842117422 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32780b3030 con 0x7f32780c5730 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3277e00640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 0x7f32780b3df0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3264009850 tx=0x7f32640039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3274c00640 1 -- 172.21.15.71:0/3842117422 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326403f070 con 0x7f32780c5730 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3274c00640 1 -- 172.21.15.71:0/3842117422 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3264004890 con 0x7f32780c5730 2024-09-16T07:37:21.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f3274c00640 1 -- 172.21.15.71:0/3842117422 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264035030 con 0x7f32780c5730 2024-09-16T07:37:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327814ac40 con 0x7f32780c5730 2024-09-16T07:37:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.416+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32780af9f0 con 0x7f32780c5730 2024-09-16T07:37:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.417+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327806eac0 con 0x7f32780c5730 2024-09-16T07:37:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.419+0000 7f3274c00640 1 -- 172.21.15.71:0/3842117422 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f3264042070 con 0x7f32780c5730 2024-09-16T07:37:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.420+0000 7f3274c00640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f325807aa10 0x7f325807ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.420+0000 7f3274c00640 1 -- 172.21.15.71:0/3842117422 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f32640ba990 con 0x7f32780c5730 2024-09-16T07:37:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.421+0000 7f327d200640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f325807aa10 0x7f325807ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.421+0000 7f327d200640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f325807aa10 0x7f325807ced0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3268009fd0 tx=0x7f326800d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:21.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.424+0000 7f3274c00640 1 -- 172.21.15.71:0/3842117422 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f32780b3030 con 0x7f32780c5730 2024-09-16T07:37:21.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.678+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f327815fb80 con 0x7f32780c5730 2024-09-16T07:37:21.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.680+0000 7f3274c00640 1 -- 172.21.15.71:0/3842117422 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f3264084410 con 0x7f32780c5730 2024-09-16T07:37:21.680 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-16T07:37:21.680 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCL3OdmOuZ3CBAAUAt4jAQBzM2WHBau9gVu+g== 2024-09-16T07:37:21.680 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-09-16T07:37:21.680 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-09-16T07:37:21.680 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T07:37:21.680 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-09-16T07:37:21.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f325807aa10 msgr2=0x7f325807ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f325807aa10 0x7f325807ced0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3268009fd0 tx=0x7f326800d040 comp rx=0 tx=0).stop 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 msgr2=0x7f32780b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 0x7f32780b3df0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3264009850 tx=0x7f32640039e0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 shutdown_connections 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f325807aa10 0x7f325807ced0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f327815f670 0x7f327814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32780c5730 0x7f32780b3df0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 --2- 172.21.15.71:0/3842117422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f327806e660 0x7f32780b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 >> 172.21.15.71:0/3842117422 conn(0x7f32780cc490 msgr2=0x7f32780ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 shutdown_connections 2024-09-16T07:37:21.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.683+0000 7f327e600640 1 -- 172.21.15.71:0/3842117422 wait complete. 2024-09-16T07:37:21.698 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:37:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.856+0000 7f7439a00640 1 -- 172.21.15.71:0/1451230170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f743415f5a0 msgr2=0x7f74340b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.856+0000 7f7439a00640 1 --2- 172.21.15.71:0/1451230170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f743415f5a0 0x7f74340b2440 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7424009830 tx=0x7f742402a480 comp rx=0 tx=0).stop 2024-09-16T07:37:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.856+0000 7f7439a00640 1 -- 172.21.15.71:0/1451230170 shutdown_connections 2024-09-16T07:37:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.856+0000 7f7439a00640 1 --2- 172.21.15.71:0/1451230170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f743415f5a0 0x7f74340b2440 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.856+0000 7f7439a00640 1 -- 172.21.15.71:0/1451230170 >> 172.21.15.71:0/1451230170 conn(0x7f74340b1950 msgr2=0x7f74340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.856+0000 7f7439a00640 1 -- 172.21.15.71:0/1451230170 shutdown_connections 2024-09-16T07:37:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.856+0000 7f7439a00640 1 -- 172.21.15.71:0/1451230170 wait complete. 2024-09-16T07:37:21.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.857+0000 7f7439a00640 1 Processor -- start 2024-09-16T07:37:21.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.857+0000 7f7439a00640 1 -- start start 2024-09-16T07:37:21.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.858+0000 7f7439a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7434073fb0 0x7f7434146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:21.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.858+0000 7f7439a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74340ccf00 0x7f74340cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:21.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.858+0000 7f7439a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f743415f5a0 0x7f74340cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:21.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.858+0000 7f7439a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f743406f9f0 con 0x7f74340ccf00 2024-09-16T07:37:21.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.858+0000 7f7439a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f743406f870 con 0x7f7434073fb0 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.858+0000 7f7439a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7434074420 con 0x7f743415f5a0 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74340ccf00 0x7f74340cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74340ccf00 0x7f74340cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57162/0 (socket says 172.21.15.71:57162) 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 -- 172.21.15.71:0/3587998929 learned_addr learned my addr 172.21.15.71:0/3587998929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7434073fb0 0x7f7434146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7433400640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f743415f5a0 0x7f74340cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 -- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f743415f5a0 msgr2=0x7f74340cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f743415f5a0 0x7f74340cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 -- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7434073fb0 msgr2=0x7f7434146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7434073fb0 0x7f7434146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 -- 172.21.15.71:0/3587998929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743414d8f0 con 0x7f74340ccf00 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.859+0000 7f7432000640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74340ccf00 0x7f74340cb6c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f742000c430 tx=0x7f742000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:21.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.860+0000 7f742aa00640 1 -- 172.21.15.71:0/3587998929 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420011790 con 0x7f74340ccf00 2024-09-16T07:37:21.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.860+0000 7f742aa00640 1 -- 172.21.15.71:0/3587998929 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7420017070 con 0x7f74340ccf00 2024-09-16T07:37:21.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.860+0000 7f742aa00640 1 -- 172.21.15.71:0/3587998929 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420011d00 con 0x7f74340ccf00 2024-09-16T07:37:21.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.860+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7434156600 con 0x7f74340ccf00 2024-09-16T07:37:21.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.860+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74340cf820 con 0x7f74340ccf00 2024-09-16T07:37:21.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.861+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74340b2440 con 0x7f74340ccf00 2024-09-16T07:37:21.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.862+0000 7f742aa00640 1 -- 172.21.15.71:0/3587998929 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f742001c460 con 0x7f74340ccf00 2024-09-16T07:37:21.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.863+0000 7f742aa00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f741407aa10 0x7f741407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:21.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.863+0000 7f742aa00640 1 -- 172.21.15.71:0/3587998929 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f7420099820 con 0x7f74340ccf00 2024-09-16T07:37:21.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.863+0000 7f7432a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f741407aa10 0x7f741407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:21.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.864+0000 7f7432a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f741407aa10 0x7f741407ced0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7424034000 tx=0x7f7424004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:21.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:21.867+0000 7f742aa00640 1 -- 172.21.15.71:0/3587998929 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f74200632a0 con 0x7f74340ccf00 2024-09-16T07:37:22.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.086+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f743415e5d0 con 0x7f74340ccf00 2024-09-16T07:37:22.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.087+0000 7f742aa00640 1 -- 172.21.15.71:0/3587998929 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5230 (secure 0 0 0) 0x7f7420067cd0 con 0x7f74340ccf00 2024-09-16T07:37:22.087 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:22.087 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":95,"btime":"2024-09-16T07:37:19:608461+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/343939477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":343939477},{"type":"v1","addr":"172.21.15.71:6835","nonce":343939477}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1796926972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1796926972},{"type":"v1","addr":"172.21.15.177:6836","nonce":1796926972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3638818185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3638818185},{"type":"v1","addr":"172.21.15.71:6834","nonce":3638818185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:37:16.010465+0000","modified":"2024-09-16T07:37:19.022287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1486935311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1486935311},{"type":"v1","addr":"172.21.15.177:6837","nonce":1486935311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9420,"qdb_cluster":[9420]},"id":10}]} 2024-09-16T07:37:22.087 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 95 2024-09-16T07:37:22.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f741407aa10 msgr2=0x7f741407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f741407aa10 0x7f741407ced0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7424034000 tx=0x7f7424004730 comp rx=0 tx=0).stop 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74340ccf00 msgr2=0x7f74340cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74340ccf00 0x7f74340cb6c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f742000c430 tx=0x7f742000c8f0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 shutdown_connections 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f741407aa10 0x7f741407ced0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f743415f5a0 0x7f74340cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74340ccf00 0x7f74340cb6c0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 --2- 172.21.15.71:0/3587998929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7434073fb0 0x7f7434146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 >> 172.21.15.71:0/3587998929 conn(0x7f74340b1950 msgr2=0x7f74340b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 shutdown_connections 2024-09-16T07:37:22.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.090+0000 7f7439a00640 1 -- 172.21.15.71:0/3587998929 wait complete. 2024-09-16T07:37:22.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T07:37:16.010465+0000', 'modified': '2024-09-16T07:37:19.022287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9420}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9420': {'gid': 9420, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6837/1486935311', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6835', 'nonce': 1486935311}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 1486935311}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9420, 'qdb_cluster': [9420]} 2024-09-16T07:37:22.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T07:37:22.509 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-16T07:37:22.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.671+0000 7fad97800640 1 -- 172.21.15.71:0/4097505324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 msgr2=0x7fad740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.671+0000 7fad97800640 1 --2- 172.21.15.71:0/4097505324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 0x7fad740051c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fad88001660 tx=0x7fad74005830 comp rx=0 tx=0).stop 2024-09-16T07:37:22.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.671+0000 7fad95800640 1 -- 172.21.15.71:0/4097505324 reap_dead start 2024-09-16T07:37:22.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.671+0000 7fad97800640 1 -- 172.21.15.71:0/4097505324 shutdown_connections 2024-09-16T07:37:22.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.671+0000 7fad97800640 1 -- 172.21.15.71:0/4097505324 >> 172.21.15.71:0/4097505324 conn(0x7fad900b1950 msgr2=0x7fad900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:22.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.671+0000 7fad97800640 1 -- 172.21.15.71:0/4097505324 shutdown_connections 2024-09-16T07:37:22.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.671+0000 7fad97800640 1 -- 172.21.15.71:0/4097505324 wait complete. 2024-09-16T07:37:22.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.672+0000 7fad97800640 1 Processor -- start 2024-09-16T07:37:22.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.673+0000 7fad97800640 1 -- start start 2024-09-16T07:37:22.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.673+0000 7fad97800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 0x7fad901f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.673+0000 7fad97800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad901298c0 0x7fad901509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.673+0000 7fad97800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad90128ba0 0x7fad90129020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.673+0000 7fad97800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fad9006f9f0 con 0x7fad74004d40 2024-09-16T07:37:22.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.673+0000 7fad97800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad9006f870 con 0x7fad901298c0 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.673+0000 7fad97800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fad900b2440 con 0x7fad90128ba0 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 0x7fad901f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 0x7fad901f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57182/0 (socket says 172.21.15.71:57182) 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 -- 172.21.15.71:0/2631151673 learned_addr learned my addr 172.21.15.71:0/2631151673 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad8fe00640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad901298c0 0x7fad901509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad95800640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad90128ba0 0x7fad90129020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 -- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad90128ba0 msgr2=0x7fad90129020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad90128ba0 0x7fad90129020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 -- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad901298c0 msgr2=0x7fad901509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad901298c0 0x7fad901509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 -- 172.21.15.71:0/2631151673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9012a0a0 con 0x7fad74004d40 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad95800640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad90128ba0 0x7fad90129020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:22.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.674+0000 7fad94e00640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 0x7fad901f5260 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fad80011d20 tx=0x7fad8000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:22.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.675+0000 7fad8cc00640 1 -- 172.21.15.71:0/2631151673 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80003e40 con 0x7fad74004d40 2024-09-16T07:37:22.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.675+0000 7fad8cc00640 1 -- 172.21.15.71:0/2631151673 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad80004b20 con 0x7fad74004d40 2024-09-16T07:37:22.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.675+0000 7fad8cc00640 1 -- 172.21.15.71:0/2631151673 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80004050 con 0x7fad74004d40 2024-09-16T07:37:22.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.675+0000 7fad97800640 1 -- 172.21.15.71:0/2631151673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9012a380 con 0x7fad74004d40 2024-09-16T07:37:22.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.675+0000 7fad97800640 1 -- 172.21.15.71:0/2631151673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad901f5c10 con 0x7fad74004d40 2024-09-16T07:37:22.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.677+0000 7fad8cc00640 1 -- 172.21.15.71:0/2631151673 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fad800041f0 con 0x7fad74004d40 2024-09-16T07:37:22.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.678+0000 7fad8cc00640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad7007aa40 0x7fad7007cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.678+0000 7fad9a838740 1 Processor -- start 2024-09-16T07:37:22.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.678+0000 7fad8cc00640 1 -- 172.21.15.71:0/2631151673 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7fad8009add0 con 0x7fad74004d40 2024-09-16T07:37:22.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.678+0000 7fad9a838740 1 -- start start 2024-09-16T07:37:22.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.679+0000 7fad8fe00640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad7007aa40 0x7fad7007cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.679+0000 7fad9a838740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8594c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.679+0000 7fad9a838740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854fa20 0x5610a8593390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.679+0000 7fad9a838740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a854ffa0 0x5610a85938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.679+0000 7fad8fe00640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad7007aa40 0x7fad7007cf00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000fb0 tx=0x7fad7c0067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.679+0000 7fad9a838740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5610a84d2430 con 0x5610a854ffa0 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.679+0000 7fad9a838740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5610a84c25d0 con 0x5610a854fa20 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.679+0000 7fad9a838740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5610a84b1a00 con 0x5610a85948b0 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a854ffa0 0x5610a85938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a854ffa0 0x5610a85938d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57184/0 (socket says 172.21.15.71:57184) 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 -- 172.21.15.71:0/1257714943 learned_addr learned my addr 172.21.15.71:0/1257714943 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad94e00640 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8594c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad8fe00640 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854fa20 0x5610a8593390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 -- 172.21.15.71:0/1257714943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 msgr2=0x5610a8594c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8594c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 -- 172.21.15.71:0/1257714943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854fa20 msgr2=0x5610a8593390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:37:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854fa20 0x5610a8593390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 -- 172.21.15.71:0/1257714943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a847a520 con 0x5610a854ffa0 2024-09-16T07:37:22.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad95800640 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a854ffa0 0x5610a85938d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fad8802a930 tx=0x7fad88001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4880ff7978065697 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:22.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.680+0000 7fad85600640 1 -- 172.21.15.71:0/1257714943 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad880062b0 con 0x5610a854ffa0 2024-09-16T07:37:22.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad85600640 1 -- 172.21.15.71:0/1257714943 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad8802e050 con 0x5610a854ffa0 2024-09-16T07:37:22.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad85600640 1 -- 172.21.15.71:0/1257714943 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88003850 con 0x5610a854ffa0 2024-09-16T07:37:22.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 -- 172.21.15.71:0/1257714943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a854ffa0 msgr2=0x5610a85938d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a854ffa0 0x5610a85938d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fad8802a930 tx=0x7fad88001190 comp rx=0 tx=0).stop 2024-09-16T07:37:22.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 -- 172.21.15.71:0/1257714943 shutdown_connections 2024-09-16T07:37:22.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a854ffa0 0x5610a85938d0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854fa20 0x5610a8593390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 --2- 172.21.15.71:0/1257714943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8594c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 -- 172.21.15.71:0/1257714943 >> 172.21.15.71:0/1257714943 conn(0x5610a854aff0 msgr2=0x5610a854ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:22.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 -- 172.21.15.71:0/1257714943 shutdown_connections 2024-09-16T07:37:22.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.681+0000 7fad9a838740 1 -- 172.21.15.71:0/1257714943 wait complete. 2024-09-16T07:37:22.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.682+0000 7fad9a838740 1 -- start start 2024-09-16T07:37:22.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.682+0000 7fad9a838740 1 Processor -- start 2024-09-16T07:37:22.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad9a838740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a8564a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad9a838740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85948b0 0x5610a8593ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad9a838740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85923a0 0x5610a8578220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad9a838740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5610a8458330 con 0x5610a85948b0 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad9a838740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5610a84b5810 con 0x5610a854ffa0 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad9a838740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5610a8222620 con 0x5610a85923a0 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad9a838740 10 client.0 _ll_register_callbacks cb 0x5610a845d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85948b0 0x5610a8593ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85948b0 0x5610a8593ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57190/0 (socket says 172.21.15.71:57190) 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.683+0000 7fad8fe00640 1 -- 172.21.15.71:0/1012209933 learned_addr learned my addr 172.21.15.71:0/1012209933 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad95800640 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85923a0 0x5610a8578220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad94e00640 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a8564a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad8fe00640 1 -- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85923a0 msgr2=0x5610a8578220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad8fe00640 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85923a0 0x5610a8578220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad8fe00640 1 -- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 msgr2=0x5610a8564a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad8fe00640 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a8564a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad8fe00640 1 -- 172.21.15.71:0/1012209933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a8213850 con 0x5610a85948b0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad94e00640 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a8564a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad8fe00640 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85948b0 0x5610a8593ee0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fad7c004db0 tx=0x7fad7c0019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec179980d2632699 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad7a000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad7a000640 1 -- 172.21.15.71:0/1012209933 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c012e10 con 0x5610a85948b0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad9a838740 1 -- 172.21.15.71:0/1012209933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad8802e050 con 0x5610a85948b0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.684+0000 7fad7a000640 1 -- 172.21.15.71:0/1012209933 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7fad7c01e410 con 0x5610a85948b0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad7a000640 1 -- 172.21.15.71:0/1012209933 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fad7c02f050 con 0x5610a85948b0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad9a838740 10 client.9534 fetch_fsmap learned FSMap version 95 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad9a838740 1 -- 172.21.15.71:0/1012209933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x5610a828a6b0 con 0x5610a85948b0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad7a000640 1 -- 172.21.15.71:0/1012209933 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad8802e050 con 0x5610a85948b0 2024-09-16T07:37:22.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad7a000640 1 -- 172.21.15.71:0/1012209933 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 95) ==== 3502+0+0 (secure 0 0 0) 0x7fad7c032020 con 0x5610a85948b0 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad9a838740 10 client.9534 fetch_fsmap finished waiting for FSMap version 95 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad9a838740 10 client.9534 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad9a838740 20 client.9534 populate_metadata read hostname 'smithi071' 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad9a838740 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] conn(0x5610a858e090 0x5610a858e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad9a838740 4 client.9534 mds_command: new command op to 9420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.685+0000 7fad9a838740 1 -- 172.21.15.71:0/1012209933 --> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610a84c4520 con 0x5610a858e090 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.686+0000 7fad94e00640 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] conn(0x5610a858e090 0x5610a858e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.686+0000 7fad94e00640 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] conn(0x5610a858e090 0x5610a858e470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ace5d79f9b81da server_cookie=afcb4f765a715d44 in_seq=0 out_seq=0 2024-09-16T07:37:22.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.686+0000 7fad7a000640 10 client.9534 ms_handle_connect on v2:172.21.15.177:6835/1486935311 2024-09-16T07:37:22.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.687+0000 7fad7a000640 1 -- 172.21.15.71:0/1012209933 <== mds.0 v2:172.21.15.177:6835/1486935311 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5610a84c4520 con 0x5610a858e090 2024-09-16T07:37:22.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.687+0000 7fad7a000640 10 client.9534 handle_command_reply: tid=0 2024-09-16T07:37:22.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 client.9534 shutdown 2024-09-16T07:37:22.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 -- 172.21.15.71:0/1012209933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85948b0 msgr2=0x5610a8593ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85948b0 0x5610a8593ee0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fad7c004db0 tx=0x7fad7c0019f0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 -- 172.21.15.71:0/1012209933 shutdown_connections 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] conn(0x5610a858e090 0x5610a858e470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85923a0 0x5610a8578220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85948b0 0x5610a8593ee0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 --2- 172.21.15.71:0/1012209933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a8564a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad7a000640 0 client.9534 ms_handle_reset on v2:172.21.15.177:6835/1486935311 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 -- 172.21.15.71:0/1012209933 >> 172.21.15.71:0/1012209933 conn(0x5610a854fa20 msgr2=0x5610a858d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 -- 172.21.15.71:0/1012209933 shutdown_connections 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 1 -- 172.21.15.71:0/1012209933 wait complete. 2024-09-16T07:37:22.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.688+0000 7fad9a838740 20 client.9534 trim_cache size 0 max 16384 2024-09-16T07:37:22.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.707+0000 7fad9a838740 1 Processor -- start 2024-09-16T07:37:22.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.707+0000 7fad9a838740 1 -- start start 2024-09-16T07:37:22.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad9a838740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a85936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad9a838740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a854d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad9a838740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8589750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad9a838740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fad90200bf0 con 0x5610a85923a0 2024-09-16T07:37:22.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad9a838740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad90200050 con 0x5610a854ffa0 2024-09-16T07:37:22.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad9a838740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fad902005a0 con 0x5610a85948b0 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a854d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a854d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57202/0 (socket says 172.21.15.71:57202) 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad8fe00640 1 -- 172.21.15.71:0/3860902172 learned_addr learned my addr 172.21.15.71:0/3860902172 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad94e00640 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a85936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad95800640 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8589750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad8fe00640 1 -- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 msgr2=0x5610a8589750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad8fe00640 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8589750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad8fe00640 1 -- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 msgr2=0x5610a85936c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad8fe00640 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a85936c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad8fe00640 1 -- 172.21.15.71:0/3860902172 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a8594250 con 0x5610a85923a0 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.708+0000 7fad95800640 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8589750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad8fe00640 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a854d1c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fad7c01e870 tx=0x7fad900af580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd5be5600327ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad84c00640 1 -- 172.21.15.71:0/3860902172 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c012c10 con 0x5610a85923a0 2024-09-16T07:37:22.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad84c00640 1 -- 172.21.15.71:0/3860902172 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad7c01e9e0 con 0x5610a85923a0 2024-09-16T07:37:22.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad84c00640 1 -- 172.21.15.71:0/3860902172 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c01ee30 con 0x5610a85923a0 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 -- 172.21.15.71:0/3860902172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 msgr2=0x5610a854d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a854d1c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fad7c01e870 tx=0x7fad900af580 comp rx=0 tx=0).stop 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 -- 172.21.15.71:0/3860902172 shutdown_connections 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a85948b0 0x5610a8589750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a854d1c0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 --2- 172.21.15.71:0/3860902172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a854ffa0 0x5610a85936c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 -- 172.21.15.71:0/3860902172 >> 172.21.15.71:0/3860902172 conn(0x5610a854fa20 msgr2=0x5610a8549bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 -- 172.21.15.71:0/3860902172 shutdown_connections 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.709+0000 7fad9a838740 1 -- 172.21.15.71:0/3860902172 wait complete. 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.710+0000 7fad9a838740 1 -- start start 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.710+0000 7fad9a838740 1 Processor -- start 2024-09-16T07:37:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.711+0000 7fad9a838740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a854ffa0 0x5610a854ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.711+0000 7fad9a838740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a8613910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.711+0000 7fad9a838740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a85948b0 0x5610a8592fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.711+0000 7fad9a838740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5610a856ad10 con 0x5610a85923a0 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.711+0000 7fad9a838740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5610a8567490 con 0x5610a85948b0 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.711+0000 7fad9a838740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5610a85a7d70 con 0x5610a854ffa0 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.711+0000 7fad9a838740 10 client.0 _ll_register_callbacks cb 0x7fad7c004c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.711+0000 7fad8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a8613910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad8fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a8613910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57206/0 (socket says 172.21.15.71:57206) 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad8fe00640 1 -- 172.21.15.71:0/223191557 learned_addr learned my addr 172.21.15.71:0/223191557 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad95800640 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a85948b0 0x5610a8592fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad94e00640 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a854ffa0 0x5610a854ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad8fe00640 1 -- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a854ffa0 msgr2=0x5610a854ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:37:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad8fe00640 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a854ffa0 0x5610a854ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad8fe00640 1 -- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a85948b0 msgr2=0x5610a8592fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad8fe00640 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a85948b0 0x5610a8592fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad8fe00640 1 -- 172.21.15.71:0/223191557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a8589750 con 0x5610a85923a0 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad94e00640 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a854ffa0 0x5610a854ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad8fe00640 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a8613910 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006770 tx=0x7fad7c01ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcad7ea89201b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad7a000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad7a000640 1 -- 172.21.15.71:0/223191557 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c041060 con 0x5610a85923a0 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad7a000640 1 -- 172.21.15.71:0/223191557 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7fad7c029b00 con 0x5610a85923a0 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.712+0000 7fad9a838740 1 -- 172.21.15.71:0/223191557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad7c01e9e0 con 0x5610a85923a0 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad7a000640 1 -- 172.21.15.71:0/223191557 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fad7c012c10 con 0x5610a85923a0 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad9a838740 10 client.9540 fetch_fsmap learned FSMap version 95 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad9a838740 1 -- 172.21.15.71:0/223191557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) -- 0x5610a8613f10 con 0x5610a85923a0 2024-09-16T07:37:22.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad7a000640 1 -- 172.21.15.71:0/223191557 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad7c01e9e0 con 0x5610a85923a0 2024-09-16T07:37:22.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad7a000640 1 -- 172.21.15.71:0/223191557 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 95) ==== 3502+0+0 (secure 0 0 0) 0x7fad7c032040 con 0x5610a85923a0 2024-09-16T07:37:22.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad9a838740 10 client.9540 fetch_fsmap finished waiting for FSMap version 95 2024-09-16T07:37:22.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad9a838740 10 client.9540 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T07:37:22.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad9a838740 20 client.9540 populate_metadata read hostname 'smithi071' 2024-09-16T07:37:22.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad9a838740 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] conn(0x5610a861da50 0x5610a85a8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad9a838740 4 client.9540 mds_command: new command op to 9420 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T07:37:22.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad9a838740 1 -- 172.21.15.71:0/223191557 --> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] -- command(tid 0: {"prefix": "status"}) -- 0x5610a84bffd0 con 0x5610a861da50 2024-09-16T07:37:22.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.713+0000 7fad94e00640 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] conn(0x5610a861da50 0x5610a85a8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad94e00640 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] conn(0x5610a861da50 0x5610a85a8560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b4d16b8b71e9f0 server_cookie=b929b7c4df1bb911 in_seq=0 out_seq=0 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad7a000640 10 client.9540 ms_handle_connect on v2:172.21.15.177:6835/1486935311 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad7a000640 1 -- 172.21.15.71:0/223191557 <== mds.0 v2:172.21.15.177:6835/1486935311 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5610a84bffd0 con 0x5610a861da50 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad7a000640 10 client.9540 handle_command_reply: tid=0 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 client.9540 shutdown 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 -- 172.21.15.71:0/223191557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 msgr2=0x5610a8613910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a8613910 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006770 tx=0x7fad7c01ee40 comp rx=0 tx=0).stop 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 -- 172.21.15.71:0/223191557 shutdown_connections 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:6835/1486935311,v1:172.21.15.177:6837/1486935311] conn(0x5610a861da50 0x5610a85a8560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a85948b0 0x5610a8592fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610a85923a0 0x5610a8613910 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 --2- 172.21.15.71:0/223191557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a854ffa0 0x5610a854ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad7a000640 0 client.9540 ms_handle_reset on v2:172.21.15.177:6835/1486935311 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.714+0000 7fad9a838740 1 -- 172.21.15.71:0/223191557 >> 172.21.15.71:0/223191557 conn(0x5610a854fa20 msgr2=0x5610a858e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad9a838740 1 -- 172.21.15.71:0/223191557 shutdown_connections 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad9a838740 1 -- 172.21.15.71:0/223191557 wait complete. 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad9a838740 20 client.9540 trim_cache size 0 max 16384 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-16T07:37:22.719 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 9420, 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 6.131284044, 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 95, 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 146, 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 146, 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 10.468069653000001 2024-09-16T07:37:22.720 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 -- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad7007aa40 msgr2=0x7fad7007cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad7007aa40 0x7fad7007cf00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000fb0 tx=0x7fad7c0067a0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 -- 172.21.15.71:0/2631151673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 msgr2=0x7fad901f5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 0x7fad901f5260 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fad80011d20 tx=0x7fad8000f790 comp rx=0 tx=0).stop 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 -- 172.21.15.71:0/2631151673 shutdown_connections 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad7007aa40 0x7fad7007cf00 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad90128ba0 0x7fad90129020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad901298c0 0x7fad901509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 --2- 172.21.15.71:0/2631151673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74004d40 0x7fad901f5260 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 -- 172.21.15.71:0/2631151673 >> 172.21.15.71:0/2631151673 conn(0x7fad900b1950 msgr2=0x7fad900afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:22.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.715+0000 7fad97800640 1 -- 172.21.15.71:0/2631151673 shutdown_connections 2024-09-16T07:37:22.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.716+0000 7fad97800640 1 -- 172.21.15.71:0/2631151673 wait complete. 2024-09-16T07:37:22.730 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T07:37:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.888+0000 7f4091600640 1 -- 172.21.15.71:0/4156418559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 msgr2=0x7f407000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.888+0000 7f4091600640 1 --2- 172.21.15.71:0/4156418559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 0x7f407000b9c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4084001e40 tx=0x7f4070005830 comp rx=0 tx=0).stop 2024-09-16T07:37:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.888+0000 7f4091600640 1 -- 172.21.15.71:0/4156418559 shutdown_connections 2024-09-16T07:37:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.888+0000 7f4091600640 1 --2- 172.21.15.71:0/4156418559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 0x7f407000b9c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.888+0000 7f4091600640 1 -- 172.21.15.71:0/4156418559 >> 172.21.15.71:0/4156418559 conn(0x7f408c0b1950 msgr2=0x7f408c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.888+0000 7f4091600640 1 -- 172.21.15.71:0/4156418559 shutdown_connections 2024-09-16T07:37:22.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.888+0000 7f4091600640 1 -- 172.21.15.71:0/4156418559 wait complete. 2024-09-16T07:37:22.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.890+0000 7f4091600640 1 Processor -- start 2024-09-16T07:37:22.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.890+0000 7f4091600640 1 -- start start 2024-09-16T07:37:22.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f4091600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 0x7f408c1f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f4091600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f408c1f81b0 0x7f408c09bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f4091600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f408c15de50 0x7f408c1f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f4091600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f408c074420 con 0x7f40700051c0 2024-09-16T07:37:22.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f4091600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f408c0b2700 con 0x7f408c15de50 2024-09-16T07:37:22.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f4091600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f408c0745a0 con 0x7f408c1f81b0 2024-09-16T07:37:22.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 0x7f408c1f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 0x7f408c1f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57224/0 (socket says 172.21.15.71:57224) 2024-09-16T07:37:22.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408aa00640 1 -- 172.21.15.71:0/2397731834 learned_addr learned my addr 172.21.15.71:0/2397731834 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:22.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408b400640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f408c15de50 0x7f408c1f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f4083e00640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f408c1f81b0 0x7f408c09bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408aa00640 1 -- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f408c1f81b0 msgr2=0x7f408c09bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408aa00640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f408c1f81b0 0x7f408c09bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408aa00640 1 -- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f408c15de50 msgr2=0x7f408c1f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:22.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408aa00640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f408c15de50 0x7f408c1f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:22.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408aa00640 1 -- 172.21.15.71:0/2397731834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c1f8770 con 0x7f40700051c0 2024-09-16T07:37:22.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.891+0000 7f408b400640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f408c15de50 0x7f408c1f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:22.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.892+0000 7f4083e00640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f408c1f81b0 0x7f408c09bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.892+0000 7f408aa00640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 0x7f408c1f5660 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f407c0063d0 tx=0x7f407c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.892+0000 7f4082a00640 1 -- 172.21.15.71:0/2397731834 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c015070 con 0x7f40700051c0 2024-09-16T07:37:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.892+0000 7f4082a00640 1 -- 172.21.15.71:0/2397731834 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f407c011860 con 0x7f40700051c0 2024-09-16T07:37:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.892+0000 7f4082a00640 1 -- 172.21.15.71:0/2397731834 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c011cb0 con 0x7f40700051c0 2024-09-16T07:37:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.892+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c1f8a50 con 0x7f40700051c0 2024-09-16T07:37:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.892+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c1f9010 con 0x7f40700051c0 2024-09-16T07:37:22.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.893+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c0acd30 con 0x7f40700051c0 2024-09-16T07:37:22.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.894+0000 7f4082a00640 1 -- 172.21.15.71:0/2397731834 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f407c018070 con 0x7f40700051c0 2024-09-16T07:37:22.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.895+0000 7f4082a00640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f406807aa40 0x7f406807cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:22.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.895+0000 7f4082a00640 1 -- 172.21.15.71:0/2397731834 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f407c09a670 con 0x7f40700051c0 2024-09-16T07:37:22.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.896+0000 7f4083e00640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f406807aa40 0x7f406807cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:22.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.897+0000 7f4083e00640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f406807aa40 0x7f406807cf00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4074009460 tx=0x7f4074005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:22.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:22.900+0000 7f4082a00640 1 -- 172.21.15.71:0/2397731834 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f407c064760 con 0x7f40700051c0 2024-09-16T07:37:23.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.120+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f408c0b02b0 con 0x7f40700051c0 2024-09-16T07:37:23.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.120+0000 7f4082a00640 1 -- 172.21.15.71:0/2397731834 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) ==== 53+0+73 (secure 0 0 0) 0x7f407c064060 con 0x7f40700051c0 2024-09-16T07:37:23.120 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T07:37:23.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f406807aa40 msgr2=0x7f406807cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f406807aa40 0x7f406807cf00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4074009460 tx=0x7f4074005f90 comp rx=0 tx=0).stop 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 msgr2=0x7f408c1f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 0x7f408c1f5660 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f407c0063d0 tx=0x7f407c0039c0 comp rx=0 tx=0).stop 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 shutdown_connections 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f406807aa40 0x7f406807cf00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f408c15de50 0x7f408c1f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f408c1f81b0 0x7f408c09bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 --2- 172.21.15.71:0/2397731834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40700051c0 0x7f408c1f5660 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 >> 172.21.15.71:0/2397731834 conn(0x7f408c0b1950 msgr2=0x7f408c0afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 shutdown_connections 2024-09-16T07:37:23.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23.123+0000 7f4091600640 1 -- 172.21.15.71:0/2397731834 wait complete. 2024-09-16T07:37:23.138 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T07:37:23.139 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T07:37:23.139 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-16T07:37:23.139 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T07:37:23.139 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T07:37:23.139 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T07:37:23.139 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T07:37:23.139 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T07:37:23.139 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:37:23.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:23.140 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-16T07:37:23.175 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T07:37:23.175 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T07:37:23.175 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T07:37:23.175 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:37:23.176 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 41834sec preferred_lft 41834sec 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::44f2:f3ff:feee:ea8/64 scope link 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether f6:f3:f4:28:4b:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::f4f3:f4ff:fe28:4b13/64 scope link 2024-09-16T07:37:23.177 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:37:23.178 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T07:37:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 56:04:6a:fb:d7:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:37:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5404:6aff:fefb:d701/64 scope link 2024-09-16T07:37:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-16T07:37:23.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:23.178 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T07:37:23.270 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-16T07:37:23.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:23.274 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-16T07:37:23.310 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-16T07:37:23.311 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T07:37:23.311 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T07:37:23.311 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T07:37:23.311 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T07:37:23.312 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T07:37:23.312 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-16T07:37:23.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T07:37:23.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T07:37:23.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T07:37:23.312 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-16T07:37:23.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T07:37:23.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:37:23.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:23.506 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:37:23.541 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:37:23.541 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:37:23.541 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:37:23.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:23.541 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-16T07:37:23.639 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:37:23.639 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T07:37:23.674 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T07:37:23.674 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T07:37:23.674 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:37:23.750 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T07:37:23.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:23.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:37:23.879 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:37:23.879 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:37:23.907 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T07:37:23.907 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T07:37:23.907 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-09-16T07:37:23.907 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@5821e025-6a94-4c05-a5f1-d6094eac5c68.cephfs=/ 2024-09-16T07:37:23.907 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.71:3300/172.21.15.177:3300/172.21.15.177:3301" will pass to kernel 2024-09-16T07:37:23.909 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T07:37:23.910 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-16T07:37:23.910 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T07:37:23.910 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T07:37:23.910 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T07:37:23.913 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T07:37:23.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:23.913 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T07:37:23.969 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T07:37:24.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:37:24.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:37:24.067 INFO:teuthology.orchestra.run.smithi071.stdout:client9549 2024-09-16T07:37:24.067 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:24.068 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:24.104 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T07:37:24.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.308+0000 7f3abfe00640 1 -- 172.21.15.71:0/2332170300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 msgr2=0x7f3aa000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:24.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.308+0000 7f3abfe00640 1 --2- 172.21.15.71:0/2332170300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 0x7f3aa000b9c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0001f30 tx=0x7f3aa0005830 comp rx=0 tx=0).stop 2024-09-16T07:37:24.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.308+0000 7f3abde00640 1 -- 172.21.15.71:0/2332170300 reap_dead start 2024-09-16T07:37:24.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.308+0000 7f3abfe00640 1 -- 172.21.15.71:0/2332170300 shutdown_connections 2024-09-16T07:37:24.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.308+0000 7f3abfe00640 1 -- 172.21.15.71:0/2332170300 >> 172.21.15.71:0/2332170300 conn(0x7f3ab80b18e0 msgr2=0x7f3ab80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:24.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.308+0000 7f3abfe00640 1 -- 172.21.15.71:0/2332170300 shutdown_connections 2024-09-16T07:37:24.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.308+0000 7f3abfe00640 1 -- 172.21.15.71:0/2332170300 wait complete. 2024-09-16T07:37:24.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.310+0000 7f3abfe00640 1 Processor -- start 2024-09-16T07:37:24.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.310+0000 7f3abfe00640 1 -- start start 2024-09-16T07:37:24.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 0x7f3ab809c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:24.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab809d570 0x7f3ab81f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:24.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab81f8b10 0x7f3ab80cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abfe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab806f6d0 con 0x7f3aa00051c0 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abfe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab80744f0 con 0x7f3ab81f8b10 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abfe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab806f850 con 0x7f3ab809d570 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 0x7f3ab809c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 0x7f3ab809c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57256/0 (socket says 172.21.15.71:57256) 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abd400640 1 -- 172.21.15.71:0/1408642005 learned_addr learned my addr 172.21.15.71:0/1408642005 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abca00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab809d570 0x7f3ab81f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abde00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab81f8b10 0x7f3ab80cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abd400640 1 -- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab809d570 msgr2=0x7f3ab81f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abd400640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab809d570 0x7f3ab81f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abd400640 1 -- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab81f8b10 msgr2=0x7f3ab80cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:24.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abd400640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab81f8b10 0x7f3ab80cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.311+0000 7f3abd400640 1 -- 172.21.15.71:0/1408642005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab80b4730 con 0x7f3aa00051c0 2024-09-16T07:37:24.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.312+0000 7f3abd400640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 0x7f3ab809c240 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f3aac011930 tx=0x7f3aac011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:24.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.312+0000 7f3ab5600640 1 -- 172.21.15.71:0/1408642005 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00fe10 con 0x7f3aa00051c0 2024-09-16T07:37:24.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.312+0000 7f3ab5600640 1 -- 172.21.15.71:0/1408642005 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3aac004030 con 0x7f3aa00051c0 2024-09-16T07:37:24.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.312+0000 7f3ab5600640 1 -- 172.21.15.71:0/1408642005 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004480 con 0x7f3aa00051c0 2024-09-16T07:37:24.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.312+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab80b3970 con 0x7f3aa00051c0 2024-09-16T07:37:24.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.312+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab80b3f30 con 0x7f3aa00051c0 2024-09-16T07:37:24.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.313+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab8069360 con 0x7f3aa00051c0 2024-09-16T07:37:24.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.314+0000 7f3ab5600640 1 -- 172.21.15.71:0/1408642005 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f3aac0035f0 con 0x7f3aa00051c0 2024-09-16T07:37:24.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.315+0000 7f3ab5600640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a9807aa40 0x7f3a9807cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:24.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.316+0000 7f3ab5600640 1 -- 172.21.15.71:0/1408642005 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f3aac09a890 con 0x7f3aa00051c0 2024-09-16T07:37:24.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.316+0000 7f3abca00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a9807aa40 0x7f3a9807cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:24.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.317+0000 7f3abca00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a9807aa40 0x7f3a9807cf00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4001ca0 tx=0x7f3aa4001e30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:24.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.320+0000 7f3ab5600640 1 -- 172.21.15.71:0/1408642005 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3aac064960 con 0x7f3aa00051c0 2024-09-16T07:37:24.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.542+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f3ab806ad00 con 0x7f3aa00051c0 2024-09-16T07:37:24.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.639+0000 7f3ab5600640 1 -- 172.21.15.71:0/1408642005 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v96) ==== 136+0+0 (secure 0 0 0) 0x7f3ab806ad00 con 0x7f3aa00051c0 2024-09-16T07:37:24.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.641+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a9807aa40 msgr2=0x7f3a9807cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:24.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.641+0000 7f3abfe00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a9807aa40 0x7f3a9807cf00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4001ca0 tx=0x7f3aa4001e30 comp rx=0 tx=0).stop 2024-09-16T07:37:24.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 msgr2=0x7f3ab809c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:24.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 0x7f3ab809c240 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f3aac011930 tx=0x7f3aac011df0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 shutdown_connections 2024-09-16T07:37:24.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3a9807aa40 0x7f3a9807cf00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ab81f8b10 0x7f3ab80cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ab809d570 0x7f3ab81f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 --2- 172.21.15.71:0/1408642005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa00051c0 0x7f3ab809c240 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 >> 172.21.15.71:0/1408642005 conn(0x7f3ab80b18e0 msgr2=0x7f3ab80af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:24.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 shutdown_connections 2024-09-16T07:37:24.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.642+0000 7f3abfe00640 1 -- 172.21.15.71:0/1408642005 wait complete. 2024-09-16T07:37:24.658 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:37:24.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.817+0000 7f36ce800640 1 -- 172.21.15.71:0/727750245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c80d0540 msgr2=0x7f36c806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:24.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.817+0000 7f36ce800640 1 --2- 172.21.15.71:0/727750245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c80d0540 0x7f36c806f320 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f36c0005fd0 tx=0x7f36c002aaa0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.817+0000 7f36ce800640 1 -- 172.21.15.71:0/727750245 shutdown_connections 2024-09-16T07:37:24.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.817+0000 7f36ce800640 1 --2- 172.21.15.71:0/727750245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c80725d0 0x7f36c80b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.817+0000 7f36ce800640 1 --2- 172.21.15.71:0/727750245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c80d0540 0x7f36c806f320 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.817+0000 7f36ce800640 1 -- 172.21.15.71:0/727750245 >> 172.21.15.71:0/727750245 conn(0x7f36c80cfa20 msgr2=0x7f36c80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:24.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.817+0000 7f36ce800640 1 -- 172.21.15.71:0/727750245 shutdown_connections 2024-09-16T07:37:24.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.817+0000 7f36ce800640 1 -- 172.21.15.71:0/727750245 wait complete. 2024-09-16T07:37:24.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.819+0000 7f36ce800640 1 Processor -- start 2024-09-16T07:37:24.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.819+0000 7f36ce800640 1 -- start start 2024-09-16T07:37:24.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.819+0000 7f36ce800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c80d0540 0x7f36c80c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:24.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.819+0000 7f36ce800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c815bc30 0x7f36c8137450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:24.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36ce800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c8150330 0x7f36c81507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:24.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36ce800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8074660 con 0x7f36c8150330 2024-09-16T07:37:24.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36ce800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36c806f860 con 0x7f36c80d0540 2024-09-16T07:37:24.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36ce800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36c80b4260 con 0x7f36c815bc30 2024-09-16T07:37:24.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c8150330 0x7f36c81507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:24.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c8150330 0x7f36c81507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60402/0 (socket says 172.21.15.71:60402) 2024-09-16T07:37:24.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 -- 172.21.15.71:0/1802874209 learned_addr learned my addr 172.21.15.71:0/1802874209 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cca00640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c815bc30 0x7f36c8137450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cd400640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c80d0540 0x7f36c80c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 -- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c815bc30 msgr2=0x7f36c8137450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c815bc30 0x7f36c8137450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 -- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c80d0540 msgr2=0x7f36c80c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c80d0540 0x7f36c80c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 -- 172.21.15.71:0/1802874209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c80c77b0 con 0x7f36c8150330 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cca00640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c815bc30 0x7f36c8137450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cd400640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c80d0540 0x7f36c80c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.820+0000 7f36cde00640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c8150330 0x7f36c81507c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f36c0006010 tx=0x7f36c0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.821+0000 7f36c5600640 1 -- 172.21.15.71:0/1802874209 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0046070 con 0x7f36c8150330 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.821+0000 7f36c5600640 1 -- 172.21.15.71:0/1802874209 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36c002e800 con 0x7f36c8150330 2024-09-16T07:37:24.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.821+0000 7f36c5600640 1 -- 172.21.15.71:0/1802874209 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c00044b0 con 0x7f36c8150330 2024-09-16T07:37:24.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.821+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c80c7a90 con 0x7f36c8150330 2024-09-16T07:37:24.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.821+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c80c5fc0 con 0x7f36c8150330 2024-09-16T07:37:24.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.822+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c806fdf0 con 0x7f36c8150330 2024-09-16T07:37:24.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.823+0000 7f36c5600640 1 -- 172.21.15.71:0/1802874209 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f36c0049070 con 0x7f36c8150330 2024-09-16T07:37:24.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.824+0000 7f36c5600640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a807aa40 0x7f36a807cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:24.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.824+0000 7f36c5600640 1 -- 172.21.15.71:0/1802874209 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f36c00ba890 con 0x7f36c8150330 2024-09-16T07:37:24.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.825+0000 7f36cd400640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a807aa40 0x7f36a807cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:24.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.825+0000 7f36cd400640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a807aa40 0x7f36a807cf00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0016a0 tx=0x7f36bc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:24.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:24.828+0000 7f36c5600640 1 -- 172.21.15.71:0/1802874209 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f36c0032030 con 0x7f36c8150330 2024-09-16T07:37:25.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.035+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f36c80cd840 con 0x7f36a807aa40 2024-09-16T07:37:25.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.037+0000 7f36c5600640 1 -- 172.21.15.71:0/1802874209 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f36c80cd840 con 0x7f36a807aa40 2024-09-16T07:37:25.036 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:37:25.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:37:25.037 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:37:25.037 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:37:25.037 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a807aa40 msgr2=0x7f36a807cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a807aa40 0x7f36a807cf00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0016a0 tx=0x7f36bc009040 comp rx=0 tx=0).stop 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c8150330 msgr2=0x7f36c81507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c8150330 0x7f36c81507c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f36c0006010 tx=0x7f36c0003f10 comp rx=0 tx=0).stop 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 shutdown_connections 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f36a807aa40 0x7f36a807cf00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36c8150330 0x7f36c81507c0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c815bc30 0x7f36c8137450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 --2- 172.21.15.71:0/1802874209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c80d0540 0x7f36c80c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 >> 172.21.15.71:0/1802874209 conn(0x7f36c80cfa20 msgr2=0x7f36c815ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 shutdown_connections 2024-09-16T07:37:25.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.039+0000 7f36ce800640 1 -- 172.21.15.71:0/1802874209 wait complete. 2024-09-16T07:37:25.054 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T07:37:25.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.215+0000 7f28a2200640 1 -- 172.21.15.71:0/1264589164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2880003c60 msgr2=0x7f2880004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.215+0000 7f28a2200640 1 --2- 172.21.15.71:0/1264589164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2880003c60 0x7f2880004080 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f288c001830 tx=0x7f2880005830 comp rx=0 tx=0).stop 2024-09-16T07:37:25.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.215+0000 7f28a1800640 1 -- 172.21.15.71:0/1264589164 reap_dead start 2024-09-16T07:37:25.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.215+0000 7f28a2200640 1 -- 172.21.15.71:0/1264589164 shutdown_connections 2024-09-16T07:37:25.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.215+0000 7f28a2200640 1 -- 172.21.15.71:0/1264589164 >> 172.21.15.71:0/1264589164 conn(0x7f289c0cf9f0 msgr2=0x7f289c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:25.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.215+0000 7f28a2200640 1 -- 172.21.15.71:0/1264589164 shutdown_connections 2024-09-16T07:37:25.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.215+0000 7f28a2200640 1 -- 172.21.15.71:0/1264589164 wait complete. 2024-09-16T07:37:25.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.217+0000 7f28a2200640 1 Processor -- start 2024-09-16T07:37:25.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.217+0000 7f28a2200640 1 -- start start 2024-09-16T07:37:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.218+0000 7f28a2200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880003c60 0x7f289c14a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.218+0000 7f28a2200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f289c1f8910 0x7f289c15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.218+0000 7f28a2200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f289c15e290 0x7f289c1f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.218+0000 7f28a2200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f289c074070 con 0x7f289c15e290 2024-09-16T07:37:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.218+0000 7f28a2200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f289c073ef0 con 0x7f2880003c60 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.218+0000 7f28a2200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f289c0741f0 con 0x7f289c1f8910 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.218+0000 7f28a1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f289c15e290 0x7f289c1f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f289c15e290 0x7f289c1f7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60424/0 (socket says 172.21.15.71:60424) 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a1800640 1 -- 172.21.15.71:0/2950934683 learned_addr learned my addr 172.21.15.71:0/2950934683 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a0e00640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880003c60 0x7f289c14a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f289be00640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f289c1f8910 0x7f289c15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a1800640 1 -- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f289c1f8910 msgr2=0x7f289c15dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a1800640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f289c1f8910 0x7f289c15dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a1800640 1 -- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880003c60 msgr2=0x7f289c14a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a1800640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880003c60 0x7f289c14a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a1800640 1 -- 172.21.15.71:0/2950934683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c15efd0 con 0x7f289c15e290 2024-09-16T07:37:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a1800640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f289c15e290 0x7f289c1f7ce0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f289c15f850 tx=0x7f2894006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:25.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f2898c00640 1 -- 172.21.15.71:0/2950934683 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894015070 con 0x7f289c15e290 2024-09-16T07:37:25.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f2898c00640 1 -- 172.21.15.71:0/2950934683 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2894011900 con 0x7f289c15e290 2024-09-16T07:37:25.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c15e730 con 0x7f289c15e290 2024-09-16T07:37:25.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.219+0000 7f2898c00640 1 -- 172.21.15.71:0/2950934683 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894011d50 con 0x7f289c15e290 2024-09-16T07:37:25.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.220+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c15ecf0 con 0x7f289c15e290 2024-09-16T07:37:25.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.221+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c0b0010 con 0x7f289c15e290 2024-09-16T07:37:25.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.222+0000 7f2898c00640 1 -- 172.21.15.71:0/2950934683 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f2894018070 con 0x7f289c15e290 2024-09-16T07:37:25.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.223+0000 7f2898c00640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f287c07aa40 0x7f287c07cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.223+0000 7f2898c00640 1 -- 172.21.15.71:0/2950934683 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f289409a7b0 con 0x7f289c15e290 2024-09-16T07:37:25.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.223+0000 7f28a0e00640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f287c07aa40 0x7f287c07cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.224+0000 7f28a0e00640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f287c07aa40 0x7f287c07cf00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f288c032000 tx=0x7f288c001360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:25.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.227+0000 7f2898c00640 1 -- 172.21.15.71:0/2950934683 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f289c0b0010 con 0x7f289c15e290 2024-09-16T07:37:25.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.411+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f289c0aed90 con 0x7f289c15e290 2024-09-16T07:37:25.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.411+0000 7f2898c00640 1 -- 172.21.15.71:0/2950934683 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f2894064230 con 0x7f289c15e290 2024-09-16T07:37:25.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f287c07aa40 msgr2=0x7f287c07cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f287c07aa40 0x7f287c07cf00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f288c032000 tx=0x7f288c001360 comp rx=0 tx=0).stop 2024-09-16T07:37:25.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f289c15e290 msgr2=0x7f289c1f7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f289c15e290 0x7f289c1f7ce0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f289c15f850 tx=0x7f2894006430 comp rx=0 tx=0).stop 2024-09-16T07:37:25.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 shutdown_connections 2024-09-16T07:37:25.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f287c07aa40 0x7f287c07cf00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f289c15e290 0x7f289c1f7ce0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f289c1f8910 0x7f289c15dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 --2- 172.21.15.71:0/2950934683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880003c60 0x7f289c14a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 >> 172.21.15.71:0/2950934683 conn(0x7f289c0cf9f0 msgr2=0x7f289c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:25.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 shutdown_connections 2024-09-16T07:37:25.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.413+0000 7f28a2200640 1 -- 172.21.15.71:0/2950934683 wait complete. 2024-09-16T07:37:25.429 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T07:37:25.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.589+0000 7faf52a00640 1 -- 172.21.15.71:0/3522658940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf34004d40 msgr2=0x7faf340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.589+0000 7faf52a00640 1 --2- 172.21.15.71:0/3522658940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf34004d40 0x7faf340051c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7faf440016c0 tx=0x7faf34005830 comp rx=0 tx=0).stop 2024-09-16T07:37:25.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.589+0000 7faf52000640 1 -- 172.21.15.71:0/3522658940 reap_dead start 2024-09-16T07:37:25.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.589+0000 7faf52a00640 1 -- 172.21.15.71:0/3522658940 shutdown_connections 2024-09-16T07:37:25.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.589+0000 7faf52a00640 1 -- 172.21.15.71:0/3522658940 >> 172.21.15.71:0/3522658940 conn(0x7faf4c0cf9f0 msgr2=0x7faf4c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:25.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.589+0000 7faf52a00640 1 -- 172.21.15.71:0/3522658940 shutdown_connections 2024-09-16T07:37:25.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.589+0000 7faf52a00640 1 -- 172.21.15.71:0/3522658940 wait complete. 2024-09-16T07:37:25.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.591+0000 7faf52a00640 1 Processor -- start 2024-09-16T07:37:25.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.591+0000 7faf52a00640 1 -- start start 2024-09-16T07:37:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf34004d40 0x7faf4c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf4c151ee0 0x7faf4c0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf4c1f6150 0x7faf4c0c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c0cde60 con 0x7faf4c1f6150 2024-09-16T07:37:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c0cdce0 con 0x7faf34004d40 2024-09-16T07:37:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c0ca2b0 con 0x7faf4c151ee0 2024-09-16T07:37:25.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf4c1f6150 0x7faf4c0c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf4c1f6150 0x7faf4c0c3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60440/0 (socket says 172.21.15.71:60440) 2024-09-16T07:37:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf52000640 1 -- 172.21.15.71:0/2037190526 learned_addr learned my addr 172.21.15.71:0/2037190526 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf51600640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf34004d40 0x7faf4c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.592+0000 7faf50c00640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf4c151ee0 0x7faf4c0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf52000640 1 -- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf4c151ee0 msgr2=0x7faf4c0c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf52000640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf4c151ee0 0x7faf4c0c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf52000640 1 -- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf34004d40 msgr2=0x7faf4c0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf52000640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf34004d40 0x7faf4c0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf52000640 1 -- 172.21.15.71:0/2037190526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c0725e0 con 0x7faf4c1f6150 2024-09-16T07:37:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf52000640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf4c1f6150 0x7faf4c0c3520 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7faf440017e0 tx=0x7faf4402ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf49600640 1 -- 172.21.15.71:0/2037190526 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44038070 con 0x7faf4c1f6150 2024-09-16T07:37:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf49600640 1 -- 172.21.15.71:0/2037190526 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf440035f0 con 0x7faf4c1f6150 2024-09-16T07:37:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf49600640 1 -- 172.21.15.71:0/2037190526 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44003a20 con 0x7faf4c1f6150 2024-09-16T07:37:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c0728a0 con 0x7faf4c1f6150 2024-09-16T07:37:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.593+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c0c9940 con 0x7faf4c1f6150 2024-09-16T07:37:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.594+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c0ce540 con 0x7faf4c1f6150 2024-09-16T07:37:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.595+0000 7faf49600640 1 -- 172.21.15.71:0/2037190526 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7faf44044020 con 0x7faf4c1f6150 2024-09-16T07:37:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.597+0000 7faf49600640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faf2c07aa40 0x7faf2c07cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.597+0000 7faf49600640 1 -- 172.21.15.71:0/2037190526 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7faf440b8e90 con 0x7faf4c1f6150 2024-09-16T07:37:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.598+0000 7faf51600640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faf2c07aa40 0x7faf2c07cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.598+0000 7faf51600640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faf2c07aa40 0x7faf2c07cf00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7faf40005fd0 tx=0x7faf4000a610 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:25.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.601+0000 7faf49600640 1 -- 172.21.15.71:0/2037190526 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faf4c0ce540 con 0x7faf4c1f6150 2024-09-16T07:37:25.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.789+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7faf4c0ca810 con 0x7faf4c1f6150 2024-09-16T07:37:25.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.790+0000 7faf49600640 1 -- 172.21.15.71:0/2037190526 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v54) ==== 116+0+2 (secure 0 0 0) 0x7faf44082910 con 0x7faf4c1f6150 2024-09-16T07:37:25.789 INFO:teuthology.orchestra.run.smithi071.stdout:4 2024-09-16T07:37:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.792+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faf2c07aa40 msgr2=0x7faf2c07cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.792+0000 7faf52a00640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faf2c07aa40 0x7faf2c07cf00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7faf40005fd0 tx=0x7faf4000a610 comp rx=0 tx=0).stop 2024-09-16T07:37:25.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.792+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf4c1f6150 msgr2=0x7faf4c0c3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.792+0000 7faf52a00640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf4c1f6150 0x7faf4c0c3520 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7faf440017e0 tx=0x7faf4402ae90 comp rx=0 tx=0).stop 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.792+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 shutdown_connections 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.792+0000 7faf52a00640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faf2c07aa40 0x7faf2c07cf00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.792+0000 7faf52a00640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf4c1f6150 0x7faf4c0c3520 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.792+0000 7faf52a00640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf4c151ee0 0x7faf4c0c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.793+0000 7faf52a00640 1 --2- 172.21.15.71:0/2037190526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf34004d40 0x7faf4c0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.793+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 >> 172.21.15.71:0/2037190526 conn(0x7faf4c0cf9f0 msgr2=0x7faf4c15d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.793+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 shutdown_connections 2024-09-16T07:37:25.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.793+0000 7faf52a00640 1 -- 172.21.15.71:0/2037190526 wait complete. 2024-09-16T07:37:25.808 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 -- 172.21.15.71:0/2262328564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 msgr2=0x7f1304074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 --2- 172.21.15.71:0/2262328564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 0x7f1304074280 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f1304070740 tx=0x7f12fc02aa30 comp rx=0 tx=0).stop 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 -- 172.21.15.71:0/2262328564 shutdown_connections 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 --2- 172.21.15.71:0/2262328564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 0x7f1304074280 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 --2- 172.21.15.71:0/2262328564 >> v2:172.21.15.177:3300/0 conn(0x7f13040cefc0 0x7f1304073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 --2- 172.21.15.71:0/2262328564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f130415bb90 0x7f13040b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 -- 172.21.15.71:0/2262328564 >> 172.21.15.71:0/2262328564 conn(0x7f13040b1510 msgr2=0x7f13040b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 -- 172.21.15.71:0/2262328564 shutdown_connections 2024-09-16T07:37:25.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.969+0000 7f1303e00640 1 -- 172.21.15.71:0/2262328564 wait complete. 2024-09-16T07:37:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.970+0000 7f1303e00640 1 Processor -- start 2024-09-16T07:37:25.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.971+0000 7f1303e00640 1 -- start start 2024-09-16T07:37:25.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1303e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13040cefc0 0x7f13041527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1303e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 0x7f13040caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1303e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f130415bb90 0x7f13040c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1303e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f130406eee0 con 0x7f13040cf3e0 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1303e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f130406ed60 con 0x7f130415bb90 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1303e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f130406f060 con 0x7f13040cefc0 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 0x7f13040caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 0x7f13040caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60456/0 (socket says 172.21.15.71:60456) 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302000640 1 -- 172.21.15.71:0/2005260829 learned_addr learned my addr 172.21.15.71:0/2005260829 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302a00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13040cefc0 0x7f13041527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1303400640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f130415bb90 0x7f13040c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302000640 1 -- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13040cefc0 msgr2=0x7f13041527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302000640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13040cefc0 0x7f13041527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302000640 1 -- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f130415bb90 msgr2=0x7f13040c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302000640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f130415bb90 0x7f13040c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302000640 1 -- 172.21.15.71:0/2005260829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13040c8e40 con 0x7f13040cf3e0 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.972+0000 7f1302a00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13040cefc0 0x7f13041527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.973+0000 7f1302000640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 0x7f13040caa00 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f12f000d7a0 tx=0x7f12f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.973+0000 7f12faa00640 1 -- 172.21.15.71:0/2005260829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0017070 con 0x7f13040cf3e0 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.973+0000 7f12faa00640 1 -- 172.21.15.71:0/2005260829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12f0009d20 con 0x7f13040cf3e0 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.973+0000 7f12faa00640 1 -- 172.21.15.71:0/2005260829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0004280 con 0x7f13040cf3e0 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.973+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13040c9120 con 0x7f13040cf3e0 2024-09-16T07:37:25.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.973+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13040ca4e0 con 0x7f13040cf3e0 2024-09-16T07:37:25.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.974+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13040b2810 con 0x7f13040cf3e0 2024-09-16T07:37:25.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.975+0000 7f12faa00640 1 -- 172.21.15.71:0/2005260829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f12f00035f0 con 0x7f13040cf3e0 2024-09-16T07:37:25.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.976+0000 7f12faa00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12e407aa10 0x7f12e407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:25.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.976+0000 7f12faa00640 1 -- 172.21.15.71:0/2005260829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f12f0099b60 con 0x7f13040cf3e0 2024-09-16T07:37:25.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.976+0000 7f1302a00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12e407aa10 0x7f12e407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:25.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.977+0000 7f1302a00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12e407aa10 0x7f12e407ced0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f12f4005fd0 tx=0x7f12f4005cf0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:25.980+0000 7f12faa00640 1 -- 172.21.15.71:0/2005260829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f12f00635e0 con 0x7f13040cf3e0 2024-09-16T07:37:26.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.165+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f1304069d20 con 0x7f13040cf3e0 2024-09-16T07:37:26.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.170+0000 7f12faa00640 1 -- 172.21.15.71:0/2005260829 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v55)=0 v55) ==== 131+0+0 (secure 0 0 0) 0x7f12f0068010 con 0x7f13040cf3e0 2024-09-16T07:37:26.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.172+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12e407aa10 msgr2=0x7f12e407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.172+0000 7f1303e00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12e407aa10 0x7f12e407ced0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f12f4005fd0 tx=0x7f12f4005cf0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.172+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 msgr2=0x7f13040caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.172+0000 7f1303e00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 0x7f13040caa00 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f12f000d7a0 tx=0x7f12f000dc60 comp rx=0 tx=0).stop 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.172+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 shutdown_connections 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.172+0000 7f1303e00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12e407aa10 0x7f12e407ced0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.172+0000 7f1303e00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f130415bb90 0x7f13040c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.173+0000 7f1303e00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13040cf3e0 0x7f13040caa00 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.173+0000 7f1303e00640 1 --2- 172.21.15.71:0/2005260829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13040cefc0 0x7f13041527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.173+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 >> 172.21.15.71:0/2005260829 conn(0x7f13040b1510 msgr2=0x7f13040b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.173+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 shutdown_connections 2024-09-16T07:37:26.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.173+0000 7f1303e00640 1 -- 172.21.15.71:0/2005260829 wait complete. 2024-09-16T07:37:26.188 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-16T07:37:26.194 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.194+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.194 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.194+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.194 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.194+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.194 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.194+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.194 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.194+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.348+0000 7f988ee00640 1 -- 172.21.15.71:0/3628329116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 msgr2=0x7f98881604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.348+0000 7f988ee00640 1 --2- 172.21.15.71:0/3628329116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 0x7f98881604b0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f98880cc400 tx=0x7f987802aac0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.349+0000 7f988ee00640 1 -- 172.21.15.71:0/3628329116 shutdown_connections 2024-09-16T07:37:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.349+0000 7f988ee00640 1 --2- 172.21.15.71:0/3628329116 >> v2:172.21.15.177:3300/0 conn(0x7f98880cd6f0 0x7f98880d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.349+0000 7f988ee00640 1 --2- 172.21.15.71:0/3628329116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 0x7f98881604b0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.349+0000 7f988ee00640 1 -- 172.21.15.71:0/3628329116 >> 172.21.15.71:0/3628329116 conn(0x7f98880d1e20 msgr2=0x7f98880d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.349+0000 7f988ee00640 1 -- 172.21.15.71:0/3628329116 shutdown_connections 2024-09-16T07:37:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.349+0000 7f988ee00640 1 -- 172.21.15.71:0/3628329116 wait complete. 2024-09-16T07:37:26.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.350+0000 7f988ee00640 1 Processor -- start 2024-09-16T07:37:26.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.350+0000 7f988ee00640 1 -- start start 2024-09-16T07:37:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.351+0000 7f988ee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98880cd6f0 0x7f988815e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.351+0000 7f988ee00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98880d0710 0x7f988809f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.351+0000 7f988ee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 0x7f98880c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.351+0000 7f988ee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f98880cdce0 con 0x7f988815bf10 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.351+0000 7f988ee00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98880cdb60 con 0x7f98880cd6f0 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.351+0000 7f988ee00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f98880cf9f0 con 0x7f98880d0710 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 0x7f98880c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f9887e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98880cd6f0 0x7f988815e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f9887400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98880d0710 0x7f988809f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 0x7f98880c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60462/0 (socket says 172.21.15.71:60462) 2024-09-16T07:37:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 -- 172.21.15.71:0/1363898949 learned_addr learned my addr 172.21.15.71:0/1363898949 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 -- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98880d0710 msgr2=0x7f988809f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98880d0710 0x7f988809f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 -- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98880cd6f0 msgr2=0x7f988815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98880cd6f0 0x7f988815e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 -- 172.21.15.71:0/1363898949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988809e820 con 0x7f988815bf10 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.352+0000 7f988ce00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 0x7f98880c6ab0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f988000c3c0 tx=0x7f988000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.353+0000 7f987fe00640 1 -- 172.21.15.71:0/1363898949 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880017070 con 0x7f988815bf10 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.353+0000 7f987fe00640 1 -- 172.21.15.71:0/1363898949 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98800053c0 con 0x7f988815bf10 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.353+0000 7f987fe00640 1 -- 172.21.15.71:0/1363898949 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880003850 con 0x7f988815bf10 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.353+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988809eb00 con 0x7f988815bf10 2024-09-16T07:37:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.353+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f988809e1f0 con 0x7f988815bf10 2024-09-16T07:37:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.354+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98880d01d0 con 0x7f988815bf10 2024-09-16T07:37:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.355+0000 7f987fe00640 1 -- 172.21.15.71:0/1363898949 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f988001a070 con 0x7f988815bf10 2024-09-16T07:37:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.356+0000 7f987fe00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f986807aa10 0x7f986807ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.356+0000 7f987fe00640 1 -- 172.21.15.71:0/1363898949 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f9880098d70 con 0x7f988815bf10 2024-09-16T07:37:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.357+0000 7f9887e00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f986807aa10 0x7f986807ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:26.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.357+0000 7f9887e00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f986807aa10 0x7f986807ced0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9878035000 tx=0x7f9878038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:26.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.360+0000 7f987fe00640 1 -- 172.21.15.71:0/1363898949 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f98800627f0 con 0x7f988815bf10 2024-09-16T07:37:26.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.545+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f98880b7cb0 con 0x7f988815bf10 2024-09-16T07:37:26.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.550+0000 7f987fe00640 1 -- 172.21.15.71:0/1363898949 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v56)=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7f9880067220 con 0x7f988815bf10 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f986807aa10 msgr2=0x7f986807ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f986807aa10 0x7f986807ced0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9878035000 tx=0x7f9878038040 comp rx=0 tx=0).stop 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 msgr2=0x7f98880c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 0x7f98880c6ab0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f988000c3c0 tx=0x7f988000c880 comp rx=0 tx=0).stop 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 shutdown_connections 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f986807aa10 0x7f986807ced0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f988815bf10 0x7f98880c6ab0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98880d0710 0x7f988809f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 --2- 172.21.15.71:0/1363898949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98880cd6f0 0x7f988815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 >> 172.21.15.71:0/1363898949 conn(0x7f98880d1e20 msgr2=0x7f98880709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:26.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 shutdown_connections 2024-09-16T07:37:26.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.552+0000 7f988ee00640 1 -- 172.21.15.71:0/1363898949 wait complete. 2024-09-16T07:37:26.567 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T07:37:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.730+0000 7ff130c00640 1 -- 172.21.15.71:0/2129973917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 msgr2=0x7ff12c0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.730+0000 7ff130c00640 1 --2- 172.21.15.71:0/2129973917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 0x7ff12c0741a0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff11c005fd0 tx=0x7ff11c02a680 comp rx=0 tx=0).stop 2024-09-16T07:37:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.731+0000 7ff130c00640 1 -- 172.21.15.71:0/2129973917 shutdown_connections 2024-09-16T07:37:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.731+0000 7ff130c00640 1 --2- 172.21.15.71:0/2129973917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 0x7ff12c0741a0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.731+0000 7ff130c00640 1 -- 172.21.15.71:0/2129973917 >> 172.21.15.71:0/2129973917 conn(0x7ff12c0b18e0 msgr2=0x7ff12c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.731+0000 7ff130c00640 1 -- 172.21.15.71:0/2129973917 shutdown_connections 2024-09-16T07:37:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.731+0000 7ff130c00640 1 -- 172.21.15.71:0/2129973917 wait complete. 2024-09-16T07:37:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.732+0000 7ff130c00640 1 Processor -- start 2024-09-16T07:37:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.732+0000 7ff130c00640 1 -- start start 2024-09-16T07:37:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.733+0000 7ff130c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff12c06f220 0x7ff12c1f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.733+0000 7ff130c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 0x7ff12c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.733+0000 7ff130c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff12c15cff0 0x7ff12c0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.733+0000 7ff130c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c06f6b0 con 0x7ff12c159aa0 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.733+0000 7ff130c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0ccc50 con 0x7ff12c15cff0 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.733+0000 7ff130c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c06f830 con 0x7ff12c06f220 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.733+0000 7ff129400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 0x7ff12c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 0x7ff12c0b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60478/0 (socket says 172.21.15.71:60478) 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129400640 1 -- 172.21.15.71:0/2842116441 learned_addr learned my addr 172.21.15.71:0/2842116441 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129e00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff12c06f220 0x7ff12c1f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff12a800640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff12c15cff0 0x7ff12c0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129400640 1 -- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff12c06f220 msgr2=0x7ff12c1f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129400640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff12c06f220 0x7ff12c1f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129400640 1 -- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff12c15cff0 msgr2=0x7ff12c0ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129400640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff12c15cff0 0x7ff12c0ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129400640 1 -- 172.21.15.71:0/2842116441 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0b3c10 con 0x7ff12c159aa0 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff12a800640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff12c15cff0 0x7ff12c0ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff129400640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 0x7ff12c0b2ce0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff11800c400 tx=0x7ff11800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.734+0000 7ff122000640 1 -- 172.21.15.71:0/2842116441 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118017070 con 0x7ff12c159aa0 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.735+0000 7ff122000640 1 -- 172.21.15.71:0/2842116441 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1180053c0 con 0x7ff12c159aa0 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.735+0000 7ff122000640 1 -- 172.21.15.71:0/2842116441 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118003850 con 0x7ff12c159aa0 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.735+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c0b3650 con 0x7ff12c159aa0 2024-09-16T07:37:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.735+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c0cdb70 con 0x7ff12c159aa0 2024-09-16T07:37:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.736+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c069360 con 0x7ff12c159aa0 2024-09-16T07:37:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.737+0000 7ff122000640 1 -- 172.21.15.71:0/2842116441 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7ff11801a070 con 0x7ff12c159aa0 2024-09-16T07:37:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.738+0000 7ff122000640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff10c07aa40 0x7ff10c07cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.738+0000 7ff122000640 1 -- 172.21.15.71:0/2842116441 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7ff118098e40 con 0x7ff12c159aa0 2024-09-16T07:37:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.738+0000 7ff129e00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff10c07aa40 0x7ff10c07cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.739+0000 7ff129e00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff10c07aa40 0x7ff10c07cf00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff11c033000 tx=0x7ff11c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.742+0000 7ff122000640 1 -- 172.21.15.71:0/2842116441 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff118062f10 con 0x7ff12c159aa0 2024-09-16T07:37:26.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:26.950+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7ff12c0741a0 con 0x7ff10c07aa40 2024-09-16T07:37:26.961 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.961+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.961 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.961+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.961 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.961+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.961 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.961+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.961 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:26.961+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:26.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.000+0000 7ff122000640 1 -- 172.21.15.71:0/2842116441 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff12c0741a0 con 0x7ff10c07aa40 2024-09-16T07:37:27.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff10c07aa40 msgr2=0x7ff10c07cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff10c07aa40 0x7ff10c07cf00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff11c033000 tx=0x7ff11c004730 comp rx=0 tx=0).stop 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 msgr2=0x7ff12c0b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 0x7ff12c0b2ce0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff11800c400 tx=0x7ff11800c8c0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 shutdown_connections 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff10c07aa40 0x7ff10c07cf00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff12c15cff0 0x7ff12c0ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c159aa0 0x7ff12c0b2ce0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 --2- 172.21.15.71:0/2842116441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff12c06f220 0x7ff12c1f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 >> 172.21.15.71:0/2842116441 conn(0x7ff12c0b18e0 msgr2=0x7ff12c159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 shutdown_connections 2024-09-16T07:37:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.003+0000 7ff130c00640 1 -- 172.21.15.71:0/2842116441 wait complete. 2024-09-16T07:37:27.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:37:27.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.179+0000 7f966dc00640 1 -- 172.21.15.71:0/3733695363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f964c005210 msgr2=0x7f964c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.179+0000 7f966dc00640 1 --2- 172.21.15.71:0/3733695363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f964c005210 0x7f964c00b9c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f966000c730 tx=0x7f964c005830 comp rx=0 tx=0).stop 2024-09-16T07:37:27.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.179+0000 7f966dc00640 1 -- 172.21.15.71:0/3733695363 shutdown_connections 2024-09-16T07:37:27.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.179+0000 7f966dc00640 1 --2- 172.21.15.71:0/3733695363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f964c005210 0x7f964c00b9c0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.179+0000 7f966dc00640 1 -- 172.21.15.71:0/3733695363 >> 172.21.15.71:0/3733695363 conn(0x7f96680b1840 msgr2=0x7f96680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:27.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.179+0000 7f966dc00640 1 -- 172.21.15.71:0/3733695363 shutdown_connections 2024-09-16T07:37:27.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.179+0000 7f966dc00640 1 -- 172.21.15.71:0/3733695363 wait complete. 2024-09-16T07:37:27.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.181+0000 7f966dc00640 1 Processor -- start 2024-09-16T07:37:27.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.181+0000 7f966dc00640 1 -- start start 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f966dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f964c005210 0x7f96681f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f966dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96681511e0 0x7f966812efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f966dc00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9668151660 0x7f966812f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f966dc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f966815edc0 con 0x7f96681511e0 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f966dc00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f966815ec40 con 0x7f964c005210 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f966dc00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96680cdad0 con 0x7f9668151660 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96681511e0 0x7f966812efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96681511e0 0x7f966812efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60504/0 (socket says 172.21.15.71:60504) 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666000640 1 -- 172.21.15.71:0/3905417778 learned_addr learned my addr 172.21.15.71:0/3905417778 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666a00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f964c005210 0x7f96681f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9667400640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9668151660 0x7f966812f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:27.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666000640 1 -- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9668151660 msgr2=0x7f966812f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666000640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9668151660 0x7f966812f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666000640 1 -- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f964c005210 msgr2=0x7f96681f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666000640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f964c005210 0x7f96681f1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.182+0000 7f9666000640 1 -- 172.21.15.71:0/3905417778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96680af460 con 0x7f96681511e0 2024-09-16T07:37:27.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.183+0000 7f9666000640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96681511e0 0x7f966812efa0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f9654004c40 tx=0x7f9654001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:27.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.183+0000 7f965ea00640 1 -- 172.21.15.71:0/3905417778 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654015020 con 0x7f96681511e0 2024-09-16T07:37:27.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.183+0000 7f965ea00640 1 -- 172.21.15.71:0/3905417778 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9654004e50 con 0x7f96681511e0 2024-09-16T07:37:27.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.183+0000 7f965ea00640 1 -- 172.21.15.71:0/3905417778 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654011be0 con 0x7f96681511e0 2024-09-16T07:37:27.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.183+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966812fb10 con 0x7f96681511e0 2024-09-16T07:37:27.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.183+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96681f0c60 con 0x7f96681511e0 2024-09-16T07:37:27.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.184+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96680ce7f0 con 0x7f96681511e0 2024-09-16T07:37:27.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.185+0000 7f965ea00640 1 -- 172.21.15.71:0/3905417778 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7f9654018020 con 0x7f96681511e0 2024-09-16T07:37:27.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.186+0000 7f965ea00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f964807aa10 0x7f964807ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:27.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.186+0000 7f965ea00640 1 -- 172.21.15.71:0/3905417778 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7f965409a490 con 0x7f96681511e0 2024-09-16T07:37:27.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.187+0000 7f9666a00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f964807aa10 0x7f964807ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:27.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.187+0000 7f9666a00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f964807aa10 0x7f964807ced0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9658009b10 tx=0x7f9658005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:27.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.190+0000 7f965ea00640 1 -- 172.21.15.71:0/3905417778 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f965401f020 con 0x7f96681511e0 2024-09-16T07:37:27.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.402+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9668074600 con 0x7f964807aa10 2024-09-16T07:37:27.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.408+0000 7f965ea00640 1 -- 172.21.15.71:0/3905417778 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f9668074600 con 0x7f964807aa10 2024-09-16T07:37:27.407 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f964807aa10 msgr2=0x7f964807ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f964807aa10 0x7f964807ced0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9658009b10 tx=0x7f9658005f90 comp rx=0 tx=0).stop 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96681511e0 msgr2=0x7f966812efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96681511e0 0x7f966812efa0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f9654004c40 tx=0x7f9654001800 comp rx=0 tx=0).stop 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 shutdown_connections 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f964807aa10 0x7f964807ced0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9668151660 0x7f966812f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96681511e0 0x7f966812efa0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 --2- 172.21.15.71:0/3905417778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f964c005210 0x7f96681f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 >> 172.21.15.71:0/3905417778 conn(0x7f96680b1840 msgr2=0x7f96680736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 shutdown_connections 2024-09-16T07:37:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.410+0000 7f966dc00640 1 -- 172.21.15.71:0/3905417778 wait complete. 2024-09-16T07:37:27.425 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:37:27.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.587+0000 7fb424c00640 1 -- 172.21.15.71:0/4243285730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb420145750 msgr2=0x7fb420073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.587+0000 7fb424c00640 1 --2- 172.21.15.71:0/4243285730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb420145750 0x7fb420073440 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb410005fd0 tx=0x7fb41002a840 comp rx=0 tx=0).stop 2024-09-16T07:37:27.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.587+0000 7fb424c00640 1 -- 172.21.15.71:0/4243285730 shutdown_connections 2024-09-16T07:37:27.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.587+0000 7fb424c00640 1 --2- 172.21.15.71:0/4243285730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb420145750 0x7fb420073440 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.587+0000 7fb424c00640 1 -- 172.21.15.71:0/4243285730 >> 172.21.15.71:0/4243285730 conn(0x7fb4200b1840 msgr2=0x7fb4200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:27.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.587+0000 7fb424c00640 1 -- 172.21.15.71:0/4243285730 shutdown_connections 2024-09-16T07:37:27.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.587+0000 7fb424c00640 1 -- 172.21.15.71:0/4243285730 wait complete. 2024-09-16T07:37:27.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.588+0000 7fb424c00640 1 Processor -- start 2024-09-16T07:37:27.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.588+0000 7fb424c00640 1 -- start start 2024-09-16T07:37:27.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.589+0000 7fb424c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42006efa0 0x7fb4200a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:27.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.589+0000 7fb424c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cccd0 0x7fb42015e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.589+0000 7fb424c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420145750 0x7fb42015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.589+0000 7fb424c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4200ce6a0 con 0x7fb42006efa0 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.589+0000 7fb424c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4200ce520 con 0x7fb4200cccd0 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.589+0000 7fb424c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb4200ce820 con 0x7fb420145750 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.589+0000 7fb41de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42006efa0 0x7fb4200a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42006efa0 0x7fb4200a9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60512/0 (socket says 172.21.15.71:60512) 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41de00640 1 -- 172.21.15.71:0/295051787 learned_addr learned my addr 172.21.15.71:0/295051787 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41d400640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cccd0 0x7fb42015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41e800640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420145750 0x7fb42015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41de00640 1 -- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420145750 msgr2=0x7fb42015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41de00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420145750 0x7fb42015ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41de00640 1 -- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cccd0 msgr2=0x7fb42015e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41de00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cccd0 0x7fb42015e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41de00640 1 -- 172.21.15.71:0/295051787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb420138f20 con 0x7fb42006efa0 2024-09-16T07:37:27.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb41de00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42006efa0 0x7fb4200a9a60 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb410034040 tx=0x7fb410031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:27.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.590+0000 7fb416000640 1 -- 172.21.15.71:0/295051787 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410042030 con 0x7fb42006efa0 2024-09-16T07:37:27.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.591+0000 7fb416000640 1 -- 172.21.15.71:0/295051787 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb410031b80 con 0x7fb42006efa0 2024-09-16T07:37:27.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.591+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb420138160 con 0x7fb42006efa0 2024-09-16T07:37:27.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.591+0000 7fb416000640 1 -- 172.21.15.71:0/295051787 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410003970 con 0x7fb42006efa0 2024-09-16T07:37:27.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.591+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb420138720 con 0x7fb42006efa0 2024-09-16T07:37:27.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.592+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4200cecd0 con 0x7fb42006efa0 2024-09-16T07:37:27.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.593+0000 7fb416000640 1 -- 172.21.15.71:0/295051787 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 48) ==== 104347+0+0 (secure 0 0 0) 0x7fb410045070 con 0x7fb42006efa0 2024-09-16T07:37:27.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.594+0000 7fb416000640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40007aa10 0x7fb40007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:27.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.594+0000 7fb416000640 1 -- 172.21.15.71:0/295051787 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5793+0+0 (secure 0 0 0) 0x7fb4100bad70 con 0x7fb42006efa0 2024-09-16T07:37:27.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.595+0000 7fb41d400640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40007aa10 0x7fb40007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:27.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.595+0000 7fb41d400640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40007aa10 0x7fb40007ced0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb420146cc0 tx=0x7fb40c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.599+0000 7fb416000640 1 -- 172.21.15.71:0/295051787 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb410084e60 con 0x7fb42006efa0 2024-09-16T07:37:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.661+0000 7fb416000640 1 -- 172.21.15.71:0/295051787 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 49) ==== 104398+0+0 (secure 0 0 0) 0x7fb4100809d0 con 0x7fb42006efa0 2024-09-16T07:37:27.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.814+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fb420144e70 con 0x7fb40007aa10 2024-09-16T07:37:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.820+0000 7fb416000640 1 -- 172.21.15.71:0/295051787 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fb420144e70 con 0x7fb40007aa10 2024-09-16T07:37:27.819 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:37:27.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40007aa10 msgr2=0x7fb40007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40007aa10 0x7fb40007ced0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb420146cc0 tx=0x7fb40c009040 comp rx=0 tx=0).stop 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42006efa0 msgr2=0x7fb4200a9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42006efa0 0x7fb4200a9a60 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb410034040 tx=0x7fb410031690 comp rx=0 tx=0).stop 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 shutdown_connections 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40007aa10 0x7fb40007ced0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420145750 0x7fb42015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4200cccd0 0x7fb42015e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 --2- 172.21.15.71:0/295051787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42006efa0 0x7fb4200a9a60 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 >> 172.21.15.71:0/295051787 conn(0x7fb4200b1840 msgr2=0x7fb420143a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 shutdown_connections 2024-09-16T07:37:27.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:27.822+0000 7fb424c00640 1 -- 172.21.15.71:0/295051787 wait complete. 2024-09-16T07:37:27.837 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:37:27.838 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:27.838 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:27.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:27.873 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:37:35.239 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:37:35.239 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:37:35.239 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.28393 s, 147 MB/s 2024-09-16T07:37:35.240 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:35.241 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:35.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:35.276 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:37:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:37:42.741 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:37:42.741 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.38808 s, 145 MB/s 2024-09-16T07:37:42.742 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:37:42.742 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:37:42.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:42.778 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T07:37:47.055 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:37:47.059 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:37:50.635 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records in 2024-09-16T07:37:50.635 INFO:teuthology.orchestra.run.smithi071.stderr:1024+0 records out 2024-09-16T07:37:50.635 INFO:teuthology.orchestra.run.smithi071.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.78056 s, 138 MB/s 2024-09-16T07:37:50.636 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T07:37:50.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.800+0000 7ff42b600640 1 -- 172.21.15.71:0/2114452258 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff42406ee00 msgr2=0x7ff42406f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.800+0000 7ff42b600640 1 --2- 172.21.15.71:0/2114452258 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff42406ee00 0x7ff42406f300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff410009830 tx=0x7ff41002a4b0 comp rx=0 tx=0).stop 2024-09-16T07:37:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.800+0000 7ff42b600640 1 -- 172.21.15.71:0/2114452258 shutdown_connections 2024-09-16T07:37:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.800+0000 7ff42b600640 1 --2- 172.21.15.71:0/2114452258 >> v2:172.21.15.177:3300/0 conn(0x7ff4240b4770 0x7ff4240ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.800+0000 7ff42b600640 1 --2- 172.21.15.71:0/2114452258 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff42406ee00 0x7ff42406f300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.800+0000 7ff42b600640 1 -- 172.21.15.71:0/2114452258 >> 172.21.15.71:0/2114452258 conn(0x7ff4240b18e0 msgr2=0x7ff4240b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.800+0000 7ff42b600640 1 -- 172.21.15.71:0/2114452258 shutdown_connections 2024-09-16T07:37:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.800+0000 7ff42b600640 1 -- 172.21.15.71:0/2114452258 wait complete. 2024-09-16T07:37:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.801+0000 7ff42b600640 1 Processor -- start 2024-09-16T07:37:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.801+0000 7ff42b600640 1 -- start start 2024-09-16T07:37:50.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.802+0000 7ff42b600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff42406ee00 0x7ff4240b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:50.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.802+0000 7ff42b600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff4240b4770 0x7ff4241ea020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:50.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.802+0000 7ff42b600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff42415dfc0 0x7ff4241f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.802+0000 7ff42b600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240ae0e0 con 0x7ff42415dfc0 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.802+0000 7ff42b600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240adf60 con 0x7ff42406ee00 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.802+0000 7ff42b600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff4240ae260 con 0x7ff4240b4770 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.802+0000 7ff429600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff42415dfc0 0x7ff4241f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff429600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff42415dfc0 0x7ff4241f7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39476/0 (socket says 172.21.15.71:39476) 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff429600640 1 -- 172.21.15.71:0/249220433 learned_addr learned my addr 172.21.15.71:0/249220433 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff423e00640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff4240b4770 0x7ff4241ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff428c00640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff42406ee00 0x7ff4240b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff429600640 1 -- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff4240b4770 msgr2=0x7ff4241ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff429600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff4240b4770 0x7ff4241ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff429600640 1 -- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff42406ee00 msgr2=0x7ff4240b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff429600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff42406ee00 0x7ff4240b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff429600640 1 -- 172.21.15.71:0/249220433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4241ea9c0 con 0x7ff42415dfc0 2024-09-16T07:37:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff429600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff42415dfc0 0x7ff4241f7290 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9e0 tx=0x7ff41c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.803+0000 7ff420c00640 1 -- 172.21.15.71:0/249220433 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c00cdb0 con 0x7ff42415dfc0 2024-09-16T07:37:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.804+0000 7ff420c00640 1 -- 172.21.15.71:0/249220433 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff41c004020 con 0x7ff42415dfc0 2024-09-16T07:37:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.804+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4241eaca0 con 0x7ff42415dfc0 2024-09-16T07:37:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.804+0000 7ff420c00640 1 -- 172.21.15.71:0/249220433 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c004450 con 0x7ff42415dfc0 2024-09-16T07:37:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.804+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4241eb540 con 0x7ff42415dfc0 2024-09-16T07:37:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.805+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff424069360 con 0x7ff42415dfc0 2024-09-16T07:37:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.806+0000 7ff420c00640 1 -- 172.21.15.71:0/249220433 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 50) ==== 104449+0+0 (secure 0 0 0) 0x7ff41c0035f0 con 0x7ff42415dfc0 2024-09-16T07:37:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.807+0000 7ff420c00640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40407ab30 0x7ff40407cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.807+0000 7ff420c00640 1 -- 172.21.15.71:0/249220433 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7ff41c09a3a0 con 0x7ff42415dfc0 2024-09-16T07:37:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.807+0000 7ff428c00640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40407ab30 0x7ff40407cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.808+0000 7ff428c00640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40407ab30 0x7ff40407cff0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff41400f290 tx=0x7ff4140020d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:50.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:50.811+0000 7ff420c00640 1 -- 172.21.15.71:0/249220433 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff41c063e20 con 0x7ff42415dfc0 2024-09-16T07:37:51.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.024+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7ff4240ae710 con 0x7ff40407ab30 2024-09-16T07:37:51.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.037+0000 7ff420c00640 1 -- 172.21.15.71:0/249220433 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff4240ae710 con 0x7ff40407ab30 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40407ab30 msgr2=0x7ff40407cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40407ab30 0x7ff40407cff0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff41400f290 tx=0x7ff4140020d0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff42415dfc0 msgr2=0x7ff4241f7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff42415dfc0 0x7ff4241f7290 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00e9e0 tx=0x7ff41c00eea0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 shutdown_connections 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff40407ab30 0x7ff40407cff0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff42415dfc0 0x7ff4241f7290 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff4240b4770 0x7ff4241ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 --2- 172.21.15.71:0/249220433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff42406ee00 0x7ff4240b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 >> 172.21.15.71:0/249220433 conn(0x7ff4240b18e0 msgr2=0x7ff4241591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:51.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 shutdown_connections 2024-09-16T07:37:51.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.040+0000 7ff42b600640 1 -- 172.21.15.71:0/249220433 wait complete. 2024-09-16T07:37:51.055 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T07:37:51.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 -- 172.21.15.71:0/1389031627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f4145750 msgr2=0x7f70f4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:51.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 --2- 172.21.15.71:0/1389031627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f4145750 0x7f70f4073440 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f70f40af750 tx=0x7f70e402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 -- 172.21.15.71:0/1389031627 shutdown_connections 2024-09-16T07:37:51.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 --2- 172.21.15.71:0/1389031627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70f40ccb00 0x7f70f415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 --2- 172.21.15.71:0/1389031627 >> v2:172.21.15.177:3300/0 conn(0x7f70f406efa0 0x7f70f40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 --2- 172.21.15.71:0/1389031627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f4145750 0x7f70f4073440 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 -- 172.21.15.71:0/1389031627 >> 172.21.15.71:0/1389031627 conn(0x7f70f40b1840 msgr2=0x7f70f40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:51.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 -- 172.21.15.71:0/1389031627 shutdown_connections 2024-09-16T07:37:51.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.215+0000 7f70fac00640 1 -- 172.21.15.71:0/1389031627 wait complete. 2024-09-16T07:37:51.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.216+0000 7f70fac00640 1 Processor -- start 2024-09-16T07:37:51.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.217+0000 7f70fac00640 1 -- start start 2024-09-16T07:37:51.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70fac00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f406efa0 0x7f70f4138dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:51.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70fac00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70f40ccb00 0x7f70f415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70fac00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70f4145750 0x7f70f40cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70fac00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41602c0 con 0x7f70f406efa0 2024-09-16T07:37:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70fac00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70f4160140 con 0x7f70f40ccb00 2024-09-16T07:37:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70fac00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f70f4160440 con 0x7f70f4145750 2024-09-16T07:37:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f406efa0 0x7f70f4138dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f406efa0 0x7f70f4138dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39494/0 (socket says 172.21.15.71:39494) 2024-09-16T07:37:51.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3e00640 1 -- 172.21.15.71:0/1078837800 learned_addr learned my addr 172.21.15.71:0/1078837800 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:51.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f8c00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70f4145750 0x7f70f40cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:51.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3400640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70f40ccb00 0x7f70f415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:51.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3e00640 1 -- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70f4145750 msgr2=0x7f70f40cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:51.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3e00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70f4145750 0x7f70f40cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3e00640 1 -- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70f40ccb00 msgr2=0x7f70f415ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3e00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70f40ccb00 0x7f70f415ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3e00640 1 -- 172.21.15.71:0/1078837800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f40c6030 con 0x7f70f406efa0 2024-09-16T07:37:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.218+0000 7f70f3400640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70f40ccb00 0x7f70f415ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:37:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.219+0000 7f70f3e00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f406efa0 0x7f70f4138dc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f70e4000c00 tx=0x7f70e4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.219+0000 7f70ebe00640 1 -- 172.21.15.71:0/1078837800 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e403e030 con 0x7f70f406efa0 2024-09-16T07:37:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.219+0000 7f70ebe00640 1 -- 172.21.15.71:0/1078837800 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70e4003e00 con 0x7f70f406efa0 2024-09-16T07:37:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.219+0000 7f70ebe00640 1 -- 172.21.15.71:0/1078837800 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4035040 con 0x7f70f406efa0 2024-09-16T07:37:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.219+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f40c6310 con 0x7f70f406efa0 2024-09-16T07:37:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.219+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f40c68d0 con 0x7f70f406efa0 2024-09-16T07:37:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.220+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f415f9f0 con 0x7f70f406efa0 2024-09-16T07:37:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.221+0000 7f70ebe00640 1 -- 172.21.15.71:0/1078837800 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 50) ==== 104449+0+0 (secure 0 0 0) 0x7f70e4041030 con 0x7f70f406efa0 2024-09-16T07:37:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.222+0000 7f70ebe00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70d407ab30 0x7f70d407cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.222+0000 7f70ebe00640 1 -- 172.21.15.71:0/1078837800 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f70e40ba1c0 con 0x7f70f406efa0 2024-09-16T07:37:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.223+0000 7f70f3400640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70d407ab30 0x7f70d407cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.223+0000 7f70f3400640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70d407ab30 0x7f70d407cff0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f70e0005e00 tx=0x7f70e0005d50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.226+0000 7f70ebe00640 1 -- 172.21.15.71:0/1078837800 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f70e4083bc0 con 0x7f70f406efa0 2024-09-16T07:37:51.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.441+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f70f40c6b10 con 0x7f70d407ab30 2024-09-16T07:37:51.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.447+0000 7f70ebe00640 1 -- 172.21.15.71:0/1078837800 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f70f40c6b10 con 0x7f70d407ab30 2024-09-16T07:37:51.447 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:37:51.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70d407ab30 msgr2=0x7f70d407cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:51.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70d407ab30 0x7f70d407cff0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f70e0005e00 tx=0x7f70e0005d50 comp rx=0 tx=0).stop 2024-09-16T07:37:51.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f406efa0 msgr2=0x7f70f4138dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:51.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f406efa0 0x7f70f4138dc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f70e4000c00 tx=0x7f70e4003910 comp rx=0 tx=0).stop 2024-09-16T07:37:51.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 shutdown_connections 2024-09-16T07:37:51.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f70d407ab30 0x7f70d407cff0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f70f4145750 0x7f70f40cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70f40ccb00 0x7f70f415ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 --2- 172.21.15.71:0/1078837800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f406efa0 0x7f70f4138dc0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:51.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 >> 172.21.15.71:0/1078837800 conn(0x7f70f40b1840 msgr2=0x7f70f40d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:51.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 shutdown_connections 2024-09-16T07:37:51.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:51.449+0000 7f70fac00640 1 -- 172.21.15.71:0/1078837800 wait complete. 2024-09-16T07:37:51.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:51.466 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:37:51.466 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:37:51.502 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:37:51.504 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T07:37:51.504 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:51.505 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T07:37:53.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:37:53.506 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:37:53.506 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:37:53.548 INFO:teuthology.orchestra.run.smithi071.stdout:# file: volumes/_nogroup/sv1/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:37:53.548 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T07:37:53.548 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:37:53.549 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T07:37:53.549 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone18140 2024-09-16T07:37:53.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.760+0000 7ff117600640 1 -- 172.21.15.71:0/3615974928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110145750 msgr2=0x7ff110073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.760+0000 7ff117600640 1 --2- 172.21.15.71:0/3615974928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110145750 0x7ff110073440 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff1100cfb70 tx=0x7ff10402a4d0 comp rx=0 tx=0).stop 2024-09-16T07:37:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.760+0000 7ff117600640 1 -- 172.21.15.71:0/3615974928 shutdown_connections 2024-09-16T07:37:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.760+0000 7ff117600640 1 --2- 172.21.15.71:0/3615974928 >> v2:172.21.15.177:3300/0 conn(0x7ff1100cccd0 0x7ff1100cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.760+0000 7ff117600640 1 --2- 172.21.15.71:0/3615974928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff110145750 0x7ff110073440 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.760+0000 7ff117600640 1 -- 172.21.15.71:0/3615974928 >> 172.21.15.71:0/3615974928 conn(0x7ff1100b1840 msgr2=0x7ff1100b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.760+0000 7ff117600640 1 -- 172.21.15.71:0/3615974928 shutdown_connections 2024-09-16T07:37:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.760+0000 7ff117600640 1 -- 172.21.15.71:0/3615974928 wait complete. 2024-09-16T07:37:53.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.761+0000 7ff117600640 1 Processor -- start 2024-09-16T07:37:53.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.761+0000 7ff117600640 1 -- start start 2024-09-16T07:37:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.762+0000 7ff117600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11006efa0 0x7ff110140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.762+0000 7ff117600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1100cccd0 0x7ff1101e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.762+0000 7ff117600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff110145750 0x7ff1101e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.762+0000 7ff117600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100ce480 con 0x7ff11006efa0 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.762+0000 7ff117600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100ce300 con 0x7ff110145750 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.762+0000 7ff117600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff1100ce600 con 0x7ff1100cccd0 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11006efa0 0x7ff110140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11006efa0 0x7ff110140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39506/0 (socket says 172.21.15.71:39506) 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 -- 172.21.15.71:0/1069835096 learned_addr learned my addr 172.21.15.71:0/1069835096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff115600640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff110145750 0x7ff1101e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 -- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1100cccd0 msgr2=0x7ff1101e65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff10fe00640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1100cccd0 0x7ff1101e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1100cccd0 0x7ff1101e65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:53.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 -- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff110145750 msgr2=0x7ff1101e6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:53.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff110145750 0x7ff1101e6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:53.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 -- 172.21.15.71:0/1069835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1101f8dc0 con 0x7ff11006efa0 2024-09-16T07:37:53.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff115600640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff110145750 0x7ff1101e6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:53.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff10fe00640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1100cccd0 0x7ff1101e65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:37:53.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.763+0000 7ff114c00640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11006efa0 0x7ff110140ec0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ff1040099c0 tx=0x7ff1040039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:53.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.764+0000 7ff10cc00640 1 -- 172.21.15.71:0/1069835096 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10403e030 con 0x7ff11006efa0 2024-09-16T07:37:53.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.764+0000 7ff10cc00640 1 -- 172.21.15.71:0/1069835096 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1040047a0 con 0x7ff11006efa0 2024-09-16T07:37:53.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.764+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1101f4d60 con 0x7ff11006efa0 2024-09-16T07:37:53.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.764+0000 7ff10cc00640 1 -- 172.21.15.71:0/1069835096 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104035040 con 0x7ff11006efa0 2024-09-16T07:37:53.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.764+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1101f4a10 con 0x7ff11006efa0 2024-09-16T07:37:53.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.765+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1100cdbb0 con 0x7ff11006efa0 2024-09-16T07:37:53.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.766+0000 7ff10cc00640 1 -- 172.21.15.71:0/1069835096 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 50) ==== 104449+0+0 (secure 0 0 0) 0x7ff104041030 con 0x7ff11006efa0 2024-09-16T07:37:53.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.767+0000 7ff10cc00640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff0f007ab30 0x7ff0f007cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:53.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.767+0000 7ff10cc00640 1 -- 172.21.15.71:0/1069835096 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7ff1040ba1d0 con 0x7ff11006efa0 2024-09-16T07:37:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.767+0000 7ff10fe00640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff0f007ab30 0x7ff0f007cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.768+0000 7ff10fe00640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff0f007ab30 0x7ff0f007cff0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc007fd0 tx=0x7ff0fc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:53.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.771+0000 7ff10cc00640 1 -- 172.21.15.71:0/1069835096 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff104084340 con 0x7ff11006efa0 2024-09-16T07:37:53.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:53.990+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone18140", "target": ["mon-mgr", ""]}) -- 0x7ff11015f480 con 0x7ff0f007ab30 2024-09-16T07:37:54.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.060+0000 7ff10cc00640 1 -- 172.21.15.71:0/1069835096 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff11015f480 con 0x7ff0f007ab30 2024-09-16T07:37:54.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff0f007ab30 msgr2=0x7ff0f007cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff0f007ab30 0x7ff0f007cff0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc007fd0 tx=0x7ff0fc009040 comp rx=0 tx=0).stop 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11006efa0 msgr2=0x7ff110140ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11006efa0 0x7ff110140ec0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ff1040099c0 tx=0x7ff1040039e0 comp rx=0 tx=0).stop 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 shutdown_connections 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff0f007ab30 0x7ff0f007cff0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff110145750 0x7ff1101e6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1100cccd0 0x7ff1101e65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 --2- 172.21.15.71:0/1069835096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11006efa0 0x7ff110140ec0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 >> 172.21.15.71:0/1069835096 conn(0x7ff1100b1840 msgr2=0x7ff1100aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 shutdown_connections 2024-09-16T07:37:54.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.063+0000 7ff117600640 1 -- 172.21.15.71:0/1069835096 wait complete. 2024-09-16T07:37:54.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.065+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.065+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.065+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.065+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.065+0000 7fe76f200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone33361 2024-09-16T07:37:54.082 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.082+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.082 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.082+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.082 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.082+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.082 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.082+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.082 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:54.082+0000 7fe771000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:54.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.244+0000 7fadc2000640 1 -- 172.21.15.71:0/2285748619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadbc0d1360 msgr2=0x7fada4003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:54.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.244+0000 7fadc2000640 1 --2- 172.21.15.71:0/2285748619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadbc0d1360 0x7fada4003a00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c2e0 tx=0x7fada4005830 comp rx=0 tx=0).stop 2024-09-16T07:37:54.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.244+0000 7fadc1600640 1 -- 172.21.15.71:0/2285748619 reap_dead start 2024-09-16T07:37:54.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.244+0000 7fadc2000640 1 -- 172.21.15.71:0/2285748619 shutdown_connections 2024-09-16T07:37:54.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.244+0000 7fadc2000640 1 -- 172.21.15.71:0/2285748619 >> 172.21.15.71:0/2285748619 conn(0x7fadbc0d3e60 msgr2=0x7fadbc0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:54.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.244+0000 7fadc2000640 1 -- 172.21.15.71:0/2285748619 shutdown_connections 2024-09-16T07:37:54.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.244+0000 7fadc2000640 1 -- 172.21.15.71:0/2285748619 wait complete. 2024-09-16T07:37:54.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.245+0000 7fadc2000640 1 Processor -- start 2024-09-16T07:37:54.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.246+0000 7fadc2000640 1 -- start start 2024-09-16T07:37:54.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.246+0000 7fadc2000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadbc0d1360 0x7fadbc0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:54.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.246+0000 7fadc2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1f5660 0x7fadbc09e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.246+0000 7fadc2000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadbc09e810 0x7fadbc1f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.246+0000 7fadc2000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0b8110 con 0x7fadbc1f5660 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.246+0000 7fadc2000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0b7f90 con 0x7fadbc09e810 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.246+0000 7fadc2000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fadbc0b8290 con 0x7fadbc0d1360 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1f5660 0x7fadbc09e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1f5660 0x7fadbc09e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39530/0 (socket says 172.21.15.71:39530) 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 -- 172.21.15.71:0/2539861799 learned_addr learned my addr 172.21.15.71:0/2539861799 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 -- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadbc0d1360 msgr2=0x7fadbc0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadbc0d1360 0x7fadbc0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:54.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 -- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadbc09e810 msgr2=0x7fadbc1f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:54.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadbc09e810 0x7fadbc1f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:54.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 -- 172.21.15.71:0/2539861799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc1f8be0 con 0x7fadbc1f5660 2024-09-16T07:37:54.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadbbe00640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1f5660 0x7fadbc09e2d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fada8011ae0 tx=0x7fada8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:54.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.247+0000 7fadb8c00640 1 -- 172.21.15.71:0/2539861799 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada801e070 con 0x7fadbc1f5660 2024-09-16T07:37:54.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.248+0000 7fadb8c00640 1 -- 172.21.15.71:0/2539861799 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada800fb90 con 0x7fadbc1f5660 2024-09-16T07:37:54.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.248+0000 7fadb8c00640 1 -- 172.21.15.71:0/2539861799 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8019410 con 0x7fadbc1f5660 2024-09-16T07:37:54.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.248+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc1f4b80 con 0x7fadbc1f5660 2024-09-16T07:37:54.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.248+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc0cdbe0 con 0x7fadbc1f5660 2024-09-16T07:37:54.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.249+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadbc0d3260 con 0x7fadbc1f5660 2024-09-16T07:37:54.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.250+0000 7fadb8c00640 1 -- 172.21.15.71:0/2539861799 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 50) ==== 104449+0+0 (secure 0 0 0) 0x7fada8021070 con 0x7fadbc1f5660 2024-09-16T07:37:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.251+0000 7fadb8c00640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad9c07ab60 0x7fad9c07d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:54.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.251+0000 7fadb8c00640 1 -- 172.21.15.71:0/2539861799 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fada809a130 con 0x7fadbc1f5660 2024-09-16T07:37:54.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.251+0000 7fadc0c00640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad9c07ab60 0x7fad9c07d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:54.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.252+0000 7fadc0c00640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad9c07ab60 0x7fad9c07d020 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fadb0033000 tx=0x7fadb0001a30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.255+0000 7fadb8c00640 1 -- 172.21.15.71:0/2539861799 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fadbc0d3260 con 0x7fadbc1f5660 2024-09-16T07:37:54.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:54.474+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone33361", "target": ["mon-mgr", ""]}) -- 0x7fadbc09ec40 con 0x7fad9c07ab60 2024-09-16T07:37:55.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:55.065+0000 7fe6d6a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:55.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:55.065+0000 7fe6d6a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:55.065 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:55.065+0000 7fe6d6a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:55.066 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:55.065+0000 7fe6d6a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:55.066 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:55.065+0000 7fe6d6a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:55.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.157+0000 7fadb8c00640 1 -- 172.21.15.71:0/2539861799 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fadb00014b0 con 0x7fad9c07ab60 2024-09-16T07:37:55.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad9c07ab60 msgr2=0x7fad9c07d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad9c07ab60 0x7fad9c07d020 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fadb0033000 tx=0x7fadb0001a30 comp rx=0 tx=0).stop 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1f5660 msgr2=0x7fadbc09e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1f5660 0x7fadbc09e2d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fada8011ae0 tx=0x7fada8009eb0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 shutdown_connections 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fad9c07ab60 0x7fad9c07d020 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadbc09e810 0x7fadbc1f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1f5660 0x7fadbc09e2d0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 --2- 172.21.15.71:0/2539861799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadbc0d1360 0x7fadbc0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 >> 172.21.15.71:0/2539861799 conn(0x7fadbc0d3e60 msgr2=0x7fadbc0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:55.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 shutdown_connections 2024-09-16T07:37:55.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.162+0000 7fadc2000640 1 -- 172.21.15.71:0/2539861799 wait complete. 2024-09-16T07:37:55.189 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone39912 2024-09-16T07:37:55.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 -- 172.21.15.71:0/1822239095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15c080 msgr2=0x7fd43c0cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:55.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 --2- 172.21.15.71:0/1822239095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15c080 0x7fd43c0cf120 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fd43c15b4d0 tx=0x7fd42402aa00 comp rx=0 tx=0).stop 2024-09-16T07:37:55.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 -- 172.21.15.71:0/1822239095 shutdown_connections 2024-09-16T07:37:55.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 --2- 172.21.15.71:0/1822239095 >> v2:172.21.15.177:3300/0 conn(0x7fd43c15e980 0x7fd43c15ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 --2- 172.21.15.71:0/1822239095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15c080 0x7fd43c0cf120 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 --2- 172.21.15.71:0/1822239095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd43c0b3370 0x7fd43c0cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 -- 172.21.15.71:0/1822239095 >> 172.21.15.71:0/1822239095 conn(0x7fd43c0b1840 msgr2=0x7fd43c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:55.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 -- 172.21.15.71:0/1822239095 shutdown_connections 2024-09-16T07:37:55.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.413+0000 7fd443000640 1 -- 172.21.15.71:0/1822239095 wait complete. 2024-09-16T07:37:55.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.415+0000 7fd443000640 1 Processor -- start 2024-09-16T07:37:55.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.415+0000 7fd443000640 1 -- start start 2024-09-16T07:37:55.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd443000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43c0b3370 0x7fd43c1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd443000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd43c15c080 0x7fd43c157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd443000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15e980 0x7fd43c153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd443000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c140770 con 0x7fd43c15e980 2024-09-16T07:37:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd443000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1405f0 con 0x7fd43c0b3370 2024-09-16T07:37:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd443000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c1408f0 con 0x7fd43c15c080 2024-09-16T07:37:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd441000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15e980 0x7fd43c153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd441000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15e980 0x7fd43c153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38096/0 (socket says 172.21.15.71:38096) 2024-09-16T07:37:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.416+0000 7fd441000640 1 -- 172.21.15.71:0/2901246036 learned_addr learned my addr 172.21.15.71:0/2901246036 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:55.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd441000640 1 -- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd43c15c080 msgr2=0x7fd43c157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:55.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd441000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd43c15c080 0x7fd43c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd441000640 1 -- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43c0b3370 msgr2=0x7fd43c1568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:55.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd441000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43c0b3370 0x7fd43c1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd441000640 1 -- 172.21.15.71:0/2901246036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c155af0 con 0x7fd43c15e980 2024-09-16T07:37:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd441000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15e980 0x7fd43c153410 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fd43400b440 tx=0x7fd43400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd438c00640 1 -- 172.21.15.71:0/2901246036 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434015020 con 0x7fd43c15e980 2024-09-16T07:37:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd438c00640 1 -- 172.21.15.71:0/2901246036 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd434011920 con 0x7fd43c15e980 2024-09-16T07:37:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd438c00640 1 -- 172.21.15.71:0/2901246036 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434011d70 con 0x7fd43c15e980 2024-09-16T07:37:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c155dd0 con 0x7fd43c15e980 2024-09-16T07:37:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.417+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c156390 con 0x7fd43c15e980 2024-09-16T07:37:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.419+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c0cd100 con 0x7fd43c15e980 2024-09-16T07:37:55.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.420+0000 7fd438c00640 1 -- 172.21.15.71:0/2901246036 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 50) ==== 104449+0+0 (secure 0 0 0) 0x7fd434018020 con 0x7fd43c15e980 2024-09-16T07:37:55.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.421+0000 7fd438c00640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd41807ab30 0x7fd41807cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:55.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.421+0000 7fd438c00640 1 -- 172.21.15.71:0/2901246036 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fd434099a80 con 0x7fd43c15e980 2024-09-16T07:37:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.426+0000 7fd43be00640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd41807ab30 0x7fd41807cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.428+0000 7fd438c00640 1 -- 172.21.15.71:0/2901246036 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd43c0cd100 con 0x7fd43c15e980 2024-09-16T07:37:55.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.437+0000 7fd43be00640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd41807ab30 0x7fd41807cff0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009710 tx=0x7fd42c006c90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:55.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.758+0000 7fd438c00640 1 -- 172.21.15.71:0/2901246036 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 51) ==== 104551+0+0 (secure 0 0 0) 0x7fd43405f6c0 con 0x7fd43c15e980 2024-09-16T07:37:55.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:55.815+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone39912", "target": ["mon-mgr", ""]}) -- 0x7fd43c156590 con 0x7fd41807ab30 2024-09-16T07:37:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.529+0000 7fd438c00640 1 -- 172.21.15.71:0/2901246036 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd43c156590 con 0x7fd41807ab30 2024-09-16T07:37:56.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd41807ab30 msgr2=0x7fd41807cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:56.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd41807ab30 0x7fd41807cff0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009710 tx=0x7fd42c006c90 comp rx=0 tx=0).stop 2024-09-16T07:37:56.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15e980 msgr2=0x7fd43c153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:56.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15e980 0x7fd43c153410 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fd43400b440 tx=0x7fd43400b900 comp rx=0 tx=0).stop 2024-09-16T07:37:56.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 shutdown_connections 2024-09-16T07:37:56.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd41807ab30 0x7fd41807cff0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:56.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c15e980 0x7fd43c153410 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:56.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd43c15c080 0x7fd43c157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:56.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.533+0000 7fd443000640 1 --2- 172.21.15.71:0/2901246036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43c0b3370 0x7fd43c1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:56.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.534+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 >> 172.21.15.71:0/2901246036 conn(0x7fd43c0b1840 msgr2=0x7fd43c15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:56.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.534+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 shutdown_connections 2024-09-16T07:37:56.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.534+0000 7fd443000640 1 -- 172.21.15.71:0/2901246036 wait complete. 2024-09-16T07:37:56.553 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone87673 2024-09-16T07:37:56.642 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:56.642+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:56.642 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:56.642+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:56.642 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:56.642+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:56.642 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:56.642+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:56.642 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:56.642+0000 7fe770600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:56.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.816+0000 7f35ff200640 1 -- 172.21.15.71:0/151911607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 msgr2=0x7f36000d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.816+0000 7f35ff200640 1 --2- 172.21.15.71:0/151911607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 0x7f36000d0c50 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009830 tx=0x7f35f002a500 comp rx=0 tx=0).stop 2024-09-16T07:37:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.816+0000 7f35ff200640 1 -- 172.21.15.71:0/151911607 shutdown_connections 2024-09-16T07:37:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.816+0000 7f35ff200640 1 --2- 172.21.15.71:0/151911607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36000b8a60 0x7f36000d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.816+0000 7f35ff200640 1 --2- 172.21.15.71:0/151911607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 0x7f36000d0c50 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.816+0000 7f35ff200640 1 -- 172.21.15.71:0/151911607 >> 172.21.15.71:0/151911607 conn(0x7f36000d3e60 msgr2=0x7f36000d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.817+0000 7f35ff200640 1 -- 172.21.15.71:0/151911607 shutdown_connections 2024-09-16T07:37:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.817+0000 7f35ff200640 1 -- 172.21.15.71:0/151911607 wait complete. 2024-09-16T07:37:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.820+0000 7f35ff200640 1 Processor -- start 2024-09-16T07:37:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.820+0000 7f35ff200640 1 -- start start 2024-09-16T07:37:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.821+0000 7f35ff200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36000b8a60 0x7f36001511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.821+0000 7f35ff200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 0x7f360015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.821+0000 7f35ff200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36000d1f00 0x7f36000cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.821+0000 7f35ff200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36000cf190 con 0x7f36000d0870 2024-09-16T07:37:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.821+0000 7f35ff200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36000cf010 con 0x7f36000b8a60 2024-09-16T07:37:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.821+0000 7f35ff200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36000cf310 con 0x7f36000d1f00 2024-09-16T07:37:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.822+0000 7f35f7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 0x7f360015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.822+0000 7f35f7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 0x7f360015a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38102/0 (socket says 172.21.15.71:38102) 2024-09-16T07:37:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.822+0000 7f35f7e00640 1 -- 172.21.15.71:0/4143636436 learned_addr learned my addr 172.21.15.71:0/4143636436 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.822+0000 7f35f7e00640 1 -- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36000d1f00 msgr2=0x7f36000cd100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T07:37:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.822+0000 7f35f7e00640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36000d1f00 0x7f36000cd100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.822+0000 7f35f7e00640 1 -- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36000b8a60 msgr2=0x7f36001511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.822+0000 7f35f7e00640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36000b8a60 0x7f36001511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.822+0000 7f35f7e00640 1 -- 172.21.15.71:0/4143636436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360015b390 con 0x7f36000d0870 2024-09-16T07:37:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.823+0000 7f35f7e00640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 0x7f360015a6d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f35e800c3d0 tx=0x7f35e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:56.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.824+0000 7f35f6000640 1 -- 172.21.15.71:0/4143636436 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8011790 con 0x7f36000d0870 2024-09-16T07:37:56.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.824+0000 7f35f6000640 1 -- 172.21.15.71:0/4143636436 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35e8017070 con 0x7f36000d0870 2024-09-16T07:37:56.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.825+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360015b0b0 con 0x7f36000d0870 2024-09-16T07:37:56.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.825+0000 7f35f6000640 1 -- 172.21.15.71:0/4143636436 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8011cc0 con 0x7f36000d0870 2024-09-16T07:37:56.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.825+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36000c9220 con 0x7f36000d0870 2024-09-16T07:37:56.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.826+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36000ce900 con 0x7f36000d0870 2024-09-16T07:37:56.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.830+0000 7f35f6000640 1 -- 172.21.15.71:0/4143636436 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 51) ==== 104551+0+0 (secure 0 0 0) 0x7f35e801c460 con 0x7f36000d0870 2024-09-16T07:37:56.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.831+0000 7f35f6000640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f35dc07ac50 0x7f35dc07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:56.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.831+0000 7f35f6000640 1 -- 172.21.15.71:0/4143636436 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f35e809a350 con 0x7f36000d0870 2024-09-16T07:37:56.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.834+0000 7f35f6000640 1 -- 172.21.15.71:0/4143636436 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f35e80644a0 con 0x7f36000d0870 2024-09-16T07:37:56.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.841+0000 7f35fde00640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f35dc07ac50 0x7f35dc07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:56.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.883+0000 7f35fde00640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f35dc07ac50 0x7f35dc07d110 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f35f0034000 tx=0x7f35f00046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:56.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:56.912+0000 7f35f6000640 1 -- 172.21.15.71:0/4143636436 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 52) ==== 104653+0+0 (secure 0 0 0) 0x7f35e8060010 con 0x7f36000d0870 2024-09-16T07:37:57.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:57.170+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone87673", "target": ["mon-mgr", ""]}) -- 0x7f36000d0650 con 0x7f35dc07ac50 2024-09-16T07:37:58.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.531+0000 7f35f6000640 1 -- 172.21.15.71:0/4143636436 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f36000d0650 con 0x7f35dc07ac50 2024-09-16T07:37:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f35dc07ac50 msgr2=0x7f35dc07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f35dc07ac50 0x7f35dc07d110 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f35f0034000 tx=0x7f35f00046c0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 msgr2=0x7f360015a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 0x7f360015a6d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f35e800c3d0 tx=0x7f35e800c890 comp rx=0 tx=0).stop 2024-09-16T07:37:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 shutdown_connections 2024-09-16T07:37:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f35dc07ac50 0x7f35dc07d110 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36000d1f00 0x7f36000cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36000d0870 0x7f360015a6d0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 --2- 172.21.15.71:0/4143636436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36000b8a60 0x7f36001511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.534+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 >> 172.21.15.71:0/4143636436 conn(0x7f36000d3e60 msgr2=0x7f360006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:58.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.535+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 shutdown_connections 2024-09-16T07:37:58.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.535+0000 7f35ff200640 1 -- 172.21.15.71:0/4143636436 wait complete. 2024-09-16T07:37:58.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone9524 2024-09-16T07:37:58.560 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:58.560+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:58.560 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:58.560+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:58.560 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:58.560+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:58.560 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:58.560+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:58.560 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:37:58.560+0000 7fe76fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:37:58.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.791+0000 7f1204a00640 1 -- 172.21.15.71:0/381361307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000af190 msgr2=0x7f12000b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:58.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.791+0000 7f1204a00640 1 --2- 172.21.15.71:0/381361307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000af190 0x7f12000b2b40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f12000b2860 tx=0x7f11f002aa00 comp rx=0 tx=0).stop 2024-09-16T07:37:58.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.791+0000 7f1204a00640 1 -- 172.21.15.71:0/381361307 shutdown_connections 2024-09-16T07:37:58.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.791+0000 7f1204a00640 1 --2- 172.21.15.71:0/381361307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000af190 0x7f12000b2b40 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.791+0000 7f1204a00640 1 --2- 172.21.15.71:0/381361307 >> v2:172.21.15.177:3300/0 conn(0x7f1200161760 0x7f12000ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.791+0000 7f1204a00640 1 -- 172.21.15.71:0/381361307 >> 172.21.15.71:0/381361307 conn(0x7f12000b1840 msgr2=0x7f12000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:37:58.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.791+0000 7f1204a00640 1 -- 172.21.15.71:0/381361307 shutdown_connections 2024-09-16T07:37:58.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.791+0000 7f1204a00640 1 -- 172.21.15.71:0/381361307 wait complete. 2024-09-16T07:37:58.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.793+0000 7f1204a00640 1 Processor -- start 2024-09-16T07:37:58.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.793+0000 7f1204a00640 1 -- start start 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.794+0000 7f1204a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12000af190 0x7f120015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.794+0000 7f1204a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000cec60 0x7f12001593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.794+0000 7f1204a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1200161760 0x7f120014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.794+0000 7f1204a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1200162310 con 0x7f12000cec60 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.794+0000 7f1204a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1200162190 con 0x7f12000af190 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.794+0000 7f1204a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1200162490 con 0x7f1200161760 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.794+0000 7f11fd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000cec60 0x7f12001593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.795+0000 7f11fd400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000cec60 0x7f12001593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38116/0 (socket says 172.21.15.71:38116) 2024-09-16T07:37:58.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.795+0000 7f11fd400640 1 -- 172.21.15.71:0/3730701671 learned_addr learned my addr 172.21.15.71:0/3730701671 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:37:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.795+0000 7f11fde00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12000af190 0x7f120015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.795+0000 7f11fd400640 1 -- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1200161760 msgr2=0x7f120014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.795+0000 7f11fd400640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1200161760 0x7f120014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.795+0000 7f11fd400640 1 -- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12000af190 msgr2=0x7f120015ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:37:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.795+0000 7f11fd400640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12000af190 0x7f120015ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:37:58.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.795+0000 7f11fd400640 1 -- 172.21.15.71:0/3730701671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1200159b60 con 0x7f12000cec60 2024-09-16T07:37:58.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.796+0000 7f11fd400640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000cec60 0x7f12001593f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f11f0000f80 tx=0x7f11f0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:58.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.796+0000 7f11f6000640 1 -- 172.21.15.71:0/3730701671 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0046070 con 0x7f12000cec60 2024-09-16T07:37:58.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.796+0000 7f11f6000640 1 -- 172.21.15.71:0/3730701671 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11f00042d0 con 0x7f12000cec60 2024-09-16T07:37:58.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.796+0000 7f11f6000640 1 -- 172.21.15.71:0/3730701671 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0033050 con 0x7f12000cec60 2024-09-16T07:37:58.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.796+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1200159e40 con 0x7f12000cec60 2024-09-16T07:37:58.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.797+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120015a400 con 0x7f12000cec60 2024-09-16T07:37:58.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.798+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1200162950 con 0x7f12000cec60 2024-09-16T07:37:58.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.799+0000 7f11f6000640 1 -- 172.21.15.71:0/3730701671 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 52) ==== 104653+0+0 (secure 0 0 0) 0x7f11f0049070 con 0x7f12000cec60 2024-09-16T07:37:58.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.800+0000 7f11f6000640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11dc07ad70 0x7f11dc07d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:37:58.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.800+0000 7f11f6000640 1 -- 172.21.15.71:0/3730701671 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f11f00bae70 con 0x7f12000cec60 2024-09-16T07:37:58.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.806+0000 7f11fde00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11dc07ad70 0x7f11dc07d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:37:58.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.809+0000 7f11f6000640 1 -- 172.21.15.71:0/3730701671 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f11f0084fc0 con 0x7f12000cec60 2024-09-16T07:37:58.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.815+0000 7f11fde00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11dc07ad70 0x7f11dc07d230 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f11e800b810 tx=0x7f11e8008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:37:58.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:58.998+0000 7f11f6000640 1 -- 172.21.15.71:0/3730701671 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 53) ==== 104704+0+0 (secure 0 0 0) 0x7f11f0036070 con 0x7f12000cec60 2024-09-16T07:37:59.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:37:59.155+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone9524", "target": ["mon-mgr", ""]}) -- 0x7f12000b3780 con 0x7f11dc07ad70 2024-09-16T07:38:01.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.461+0000 7f11f6000640 1 -- 172.21.15.71:0/3730701671 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12000b3780 con 0x7f11dc07ad70 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11dc07ad70 msgr2=0x7f11dc07d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11dc07ad70 0x7f11dc07d230 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f11e800b810 tx=0x7f11e8008040 comp rx=0 tx=0).stop 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000cec60 msgr2=0x7f12001593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000cec60 0x7f12001593f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f11f0000f80 tx=0x7f11f0003f10 comp rx=0 tx=0).stop 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 shutdown_connections 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f11dc07ad70 0x7f11dc07d230 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1200161760 0x7f120014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12000cec60 0x7f12001593f0 secure :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f11f0000f80 tx=0x7f11f0003f10 comp rx=0 tx=0).stop 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 --2- 172.21.15.71:0/3730701671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12000af190 0x7f120015ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 >> 172.21.15.71:0/3730701671 conn(0x7f12000b1840 msgr2=0x7f12000ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 shutdown_connections 2024-09-16T07:38:01.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.464+0000 7f1204a00640 1 -- 172.21.15.71:0/3730701671 wait complete. 2024-09-16T07:38:01.485 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone13325 2024-09-16T07:38:01.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.738+0000 7f3b86800640 1 -- 172.21.15.71:0/635996151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b800cbc00 msgr2=0x7f3b8006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:01.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.738+0000 7f3b86800640 1 --2- 172.21.15.71:0/635996151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b800cbc00 0x7f3b8006fce0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009830 tx=0x7f3b6c02a520 comp rx=0 tx=0).stop 2024-09-16T07:38:01.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.738+0000 7f3b86800640 1 -- 172.21.15.71:0/635996151 shutdown_connections 2024-09-16T07:38:01.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.738+0000 7f3b86800640 1 --2- 172.21.15.71:0/635996151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b80072560 0x7f3b80073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:01.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.738+0000 7f3b86800640 1 --2- 172.21.15.71:0/635996151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b800cbc00 0x7f3b8006fce0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:01.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.738+0000 7f3b86800640 1 -- 172.21.15.71:0/635996151 >> 172.21.15.71:0/635996151 conn(0x7f3b800cf9f0 msgr2=0x7f3b800cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:01.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.739+0000 7f3b86800640 1 -- 172.21.15.71:0/635996151 shutdown_connections 2024-09-16T07:38:01.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.739+0000 7f3b86800640 1 -- 172.21.15.71:0/635996151 wait complete. 2024-09-16T07:38:01.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.742+0000 7f3b86800640 1 Processor -- start 2024-09-16T07:38:01.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.742+0000 7f3b86800640 1 -- start start 2024-09-16T07:38:01.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.744+0000 7f3b86800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80072560 0x7f3b8015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:01.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.744+0000 7f3b86800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b800cbc00 0x7f3b8014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:01.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.744+0000 7f3b86800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b8015bec0 0x7f3b801fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:01.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.744+0000 7f3b86800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3b800ae960 con 0x7f3b80072560 2024-09-16T07:38:01.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.744+0000 7f3b86800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3b800ae7e0 con 0x7f3b8015bec0 2024-09-16T07:38:01.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.744+0000 7f3b86800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3b800aeae0 con 0x7f3b800cbc00 2024-09-16T07:38:01.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.746+0000 7f3b85e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b8015bec0 0x7f3b801fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:01.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.746+0000 7f3b85400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80072560 0x7f3b8015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:01.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.746+0000 7f3b85e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b8015bec0 0x7f3b801fc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:52430/0 (socket says 172.21.15.71:52430) 2024-09-16T07:38:01.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.746+0000 7f3b85e00640 1 -- 172.21.15.71:0/3893665729 learned_addr learned my addr 172.21.15.71:0/3893665729 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:01.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.747+0000 7f3b85400640 1 -- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b800cbc00 msgr2=0x7f3b8014d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:01.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.747+0000 7f3b85400640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b800cbc00 0x7f3b8014d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:01.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.747+0000 7f3b85400640 1 -- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b8015bec0 msgr2=0x7f3b801fc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:01.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.747+0000 7f3b85400640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b8015bec0 0x7f3b801fc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:01.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.747+0000 7f3b85400640 1 -- 172.21.15.71:0/3893665729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b800c8730 con 0x7f3b80072560 2024-09-16T07:38:01.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.747+0000 7f3b85400640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80072560 0x7f3b8015d9e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000c390 tx=0x7f3b7000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:01.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.748+0000 7f3b7d600640 1 -- 172.21.15.71:0/3893665729 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70006890 con 0x7f3b80072560 2024-09-16T07:38:01.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.748+0000 7f3b7d600640 1 -- 172.21.15.71:0/3893665729 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b70006b40 con 0x7f3b80072560 2024-09-16T07:38:01.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.748+0000 7f3b7d600640 1 -- 172.21.15.71:0/3893665729 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70007020 con 0x7f3b80072560 2024-09-16T07:38:01.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.748+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b800c72d0 con 0x7f3b80072560 2024-09-16T07:38:01.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.748+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b800c6ff0 con 0x7f3b80072560 2024-09-16T07:38:01.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.750+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b58005290 con 0x7f3b80072560 2024-09-16T07:38:01.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.752+0000 7f3b7d600640 1 -- 172.21.15.71:0/3893665729 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 53) ==== 104704+0+0 (secure 0 0 0) 0x7f3b70019020 con 0x7f3b80072560 2024-09-16T07:38:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.753+0000 7f3b7d600640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b6007aed0 0x7f3b6007d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.754+0000 7f3b7d600640 1 -- 172.21.15.71:0/3893665729 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f3b70098f10 con 0x7f3b80072560 2024-09-16T07:38:01.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.759+0000 7f3b7d600640 1 -- 172.21.15.71:0/3893665729 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b70020020 con 0x7f3b80072560 2024-09-16T07:38:01.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.791+0000 7f3b84a00640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b6007aed0 0x7f3b6007d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:01.877+0000 7f3b84a00640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b6007aed0 0x7f3b6007d390 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c034000 tx=0x7f3b6c0046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:02.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:02.150+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone13325", "target": ["mon-mgr", ""]}) -- 0x7f3b58000c80 con 0x7f3b6007aed0 2024-09-16T07:38:03.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.892+0000 7f3b7d600640 1 -- 172.21.15.71:0/3893665729 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b58000c80 con 0x7f3b6007aed0 2024-09-16T07:38:03.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b6007aed0 msgr2=0x7f3b6007d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:03.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b6007aed0 0x7f3b6007d390 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c034000 tx=0x7f3b6c0046c0 comp rx=0 tx=0).stop 2024-09-16T07:38:03.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80072560 msgr2=0x7f3b8015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:03.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80072560 0x7f3b8015d9e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000c390 tx=0x7f3b7000c850 comp rx=0 tx=0).stop 2024-09-16T07:38:03.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 shutdown_connections 2024-09-16T07:38:03.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f3b6007aed0 0x7f3b6007d390 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:03.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b8015bec0 0x7f3b801fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:03.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b800cbc00 0x7f3b8014d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:03.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.896+0000 7f3b86800640 1 --2- 172.21.15.71:0/3893665729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80072560 0x7f3b8015d9e0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:03.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.897+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 >> 172.21.15.71:0/3893665729 conn(0x7f3b800cf9f0 msgr2=0x7f3b800b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:03.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.897+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 shutdown_connections 2024-09-16T07:38:03.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:03.897+0000 7f3b86800640 1 -- 172.21.15.71:0/3893665729 wait complete. 2024-09-16T07:38:03.920 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone89816 2024-09-16T07:38:04.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.188+0000 7f58e0600640 1 -- 172.21.15.71:0/1482315521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c0003a50 msgr2=0x7f58c0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:04.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.188+0000 7f58e0600640 1 --2- 172.21.15.71:0/1482315521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c0003a50 0x7f58c0003e50 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f58cc006fb0 tx=0x7f58c0005830 comp rx=0 tx=0).stop 2024-09-16T07:38:04.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.188+0000 7f58e0600640 1 -- 172.21.15.71:0/1482315521 shutdown_connections 2024-09-16T07:38:04.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.189+0000 7f58e0600640 1 --2- 172.21.15.71:0/1482315521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c0003a50 0x7f58c0003e50 secure :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f58cc006fb0 tx=0x7f58c0005830 comp rx=0 tx=0).stop 2024-09-16T07:38:04.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.189+0000 7f58e0600640 1 -- 172.21.15.71:0/1482315521 >> 172.21.15.71:0/1482315521 conn(0x7f58d80b2130 msgr2=0x7f58d80b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:04.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.189+0000 7f58e0600640 1 -- 172.21.15.71:0/1482315521 shutdown_connections 2024-09-16T07:38:04.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.189+0000 7f58e0600640 1 -- 172.21.15.71:0/1482315521 wait complete. 2024-09-16T07:38:04.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.192+0000 7f58e0600640 1 Processor -- start 2024-09-16T07:38:04.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.192+0000 7f58e0600640 1 -- start start 2024-09-16T07:38:04.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.194+0000 7f58e0600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d8155830 0x7f58d81f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:04.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.194+0000 7f58e0600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d81600a0 0x7f58d815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:04.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.194+0000 7f58e0600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d815ecc0 0x7f58d8153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:04.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.194+0000 7f58e0600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f58d80cd240 con 0x7f58d8155830 2024-09-16T07:38:04.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.194+0000 7f58e0600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f58d80cd0c0 con 0x7f58d815ecc0 2024-09-16T07:38:04.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.194+0000 7f58e0600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f58d80cd3c0 con 0x7f58d81600a0 2024-09-16T07:38:04.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.195+0000 7f58ddc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d8155830 0x7f58d81f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:04.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.195+0000 7f58de600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d815ecc0 0x7f58d8153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:04.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.195+0000 7f58de600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d815ecc0 0x7f58d8153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:52456/0 (socket says 172.21.15.71:52456) 2024-09-16T07:38:04.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.195+0000 7f58de600640 1 -- 172.21.15.71:0/2708898231 learned_addr learned my addr 172.21.15.71:0/2708898231 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:04.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.196+0000 7f58ddc00640 1 -- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d81600a0 msgr2=0x7f58d815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:04.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.196+0000 7f58ddc00640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d81600a0 0x7f58d815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:04.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.196+0000 7f58ddc00640 1 -- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d815ecc0 msgr2=0x7f58d8153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:04.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.196+0000 7f58ddc00640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d815ecc0 0x7f58d8153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:04.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.196+0000 7f58ddc00640 1 -- 172.21.15.71:0/2708898231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d81f50a0 con 0x7f58d8155830 2024-09-16T07:38:04.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.197+0000 7f58ddc00640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d8155830 0x7f58d81f49a0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f58cc030000 tx=0x7f58cc004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:04.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.198+0000 7f58d5600640 1 -- 172.21.15.71:0/2708898231 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc044070 con 0x7f58d8155830 2024-09-16T07:38:04.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.199+0000 7f58d5600640 1 -- 172.21.15.71:0/2708898231 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58cc004cf0 con 0x7f58d8155830 2024-09-16T07:38:04.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.199+0000 7f58d5600640 1 -- 172.21.15.71:0/2708898231 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc031050 con 0x7f58d8155830 2024-09-16T07:38:04.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.199+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d81f5380 con 0x7f58d8155830 2024-09-16T07:38:04.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.199+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d8153500 con 0x7f58d8155830 2024-09-16T07:38:04.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.201+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d8073320 con 0x7f58d8155830 2024-09-16T07:38:04.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.203+0000 7f58d5600640 1 -- 172.21.15.71:0/2708898231 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 53) ==== 104704+0+0 (secure 0 0 0) 0x7f58cc034070 con 0x7f58d8155830 2024-09-16T07:38:04.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.204+0000 7f58d5600640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f58bc07ae30 0x7f58bc07d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:04.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.204+0000 7f58d5600640 1 -- 172.21.15.71:0/2708898231 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f58cc0b9310 con 0x7f58d8155830 2024-09-16T07:38:04.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.212+0000 7f58d5600640 1 -- 172.21.15.71:0/2708898231 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f58cc082ed0 con 0x7f58d8155830 2024-09-16T07:38:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.237+0000 7f58dd200640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f58bc07ae30 0x7f58bc07d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:04.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.344+0000 7f58dd200640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f58bc07ae30 0x7f58bc07d2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f58d815fd50 tx=0x7f58c8006b00 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:04.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:04.560+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone89816", "target": ["mon-mgr", ""]}) -- 0x7f58d815aa30 con 0x7f58bc07ae30 2024-09-16T07:38:05.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.991+0000 7f58d5600640 1 -- 172.21.15.71:0/2708898231 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f58d815aa30 con 0x7f58bc07ae30 2024-09-16T07:38:05.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.994+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f58bc07ae30 msgr2=0x7f58bc07d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:05.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.994+0000 7f58e0600640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f58bc07ae30 0x7f58bc07d2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f58d815fd50 tx=0x7f58c8006b00 comp rx=0 tx=0).stop 2024-09-16T07:38:05.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.994+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d8155830 msgr2=0x7f58d81f49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:05.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.994+0000 7f58e0600640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d8155830 0x7f58d81f49a0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f58cc030000 tx=0x7f58cc004c00 comp rx=0 tx=0).stop 2024-09-16T07:38:05.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.994+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 shutdown_connections 2024-09-16T07:38:05.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.995+0000 7f58e0600640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f58bc07ae30 0x7f58bc07d2f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:05.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.995+0000 7f58e0600640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d815ecc0 0x7f58d8153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:05.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.995+0000 7f58e0600640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d81600a0 0x7f58d815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:05.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.995+0000 7f58e0600640 1 --2- 172.21.15.71:0/2708898231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d8155830 0x7f58d81f49a0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:05.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.995+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 >> 172.21.15.71:0/2708898231 conn(0x7f58d80b2130 msgr2=0x7f58d806faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:05.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.995+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 shutdown_connections 2024-09-16T07:38:05.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:05.995+0000 7f58e0600640 1 -- 172.21.15.71:0/2708898231 wait complete. 2024-09-16T07:38:07.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone18140 2024-09-16T07:38:07.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 -- 172.21.15.71:0/2906284715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc288161590 msgr2=0x7fc2880b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:07.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 --2- 172.21.15.71:0/2906284715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc288161590 0x7fc2880b6e00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fc2880d1fc0 tx=0x7fc27802a560 comp rx=0 tx=0).stop 2024-09-16T07:38:07.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 -- 172.21.15.71:0/2906284715 shutdown_connections 2024-09-16T07:38:07.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 --2- 172.21.15.71:0/2906284715 >> v2:172.21.15.177:3300/0 conn(0x7fc2880b7340 0x7fc2880cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:07.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 --2- 172.21.15.71:0/2906284715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2880b5630 0x7fc2880cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:07.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 --2- 172.21.15.71:0/2906284715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc288161590 0x7fc2880b6e00 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:07.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 -- 172.21.15.71:0/2906284715 >> 172.21.15.71:0/2906284715 conn(0x7fc2880d3fb0 msgr2=0x7fc2880d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:07.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 -- 172.21.15.71:0/2906284715 shutdown_connections 2024-09-16T07:38:07.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.298+0000 7fc28e800640 1 -- 172.21.15.71:0/2906284715 wait complete. 2024-09-16T07:38:07.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.299+0000 7fc28e800640 1 Processor -- start 2024-09-16T07:38:07.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.300+0000 7fc28e800640 1 -- start start 2024-09-16T07:38:07.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc28e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2880b5630 0x7fc2881f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc28e800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2880b7340 0x7fc288159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc28e800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc288161590 0x7fc2881576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc28e800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881620b0 con 0x7fc2880b5630 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc28e800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc288161f30 con 0x7fc288161590 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc28e800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc288162230 con 0x7fc2880b7340 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc287400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2880b5630 0x7fc2881f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc287400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2880b5630 0x7fc2881f21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42508/0 (socket says 172.21.15.71:42508) 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc287400640 1 -- 172.21.15.71:0/384051819 learned_addr learned my addr 172.21.15.71:0/384051819 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:07.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.301+0000 7fc286a00640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2880b7340 0x7fc288159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:07.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc287400640 1 -- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2880b7340 msgr2=0x7fc288159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:07.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc287400640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2880b7340 0x7fc288159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:07.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc287400640 1 -- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc288161590 msgr2=0x7fc2881576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:07.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc287400640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc288161590 0x7fc2881576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:07.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc287400640 1 -- 172.21.15.71:0/384051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc288159fa0 con 0x7fc2880b5630 2024-09-16T07:38:07.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc287400640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2880b5630 0x7fc2881f21f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fc278034040 tx=0x7fc2780039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:07.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc27f400640 1 -- 172.21.15.71:0/384051819 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27803e030 con 0x7fc2880b5630 2024-09-16T07:38:07.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc27f400640 1 -- 172.21.15.71:0/384051819 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2780047a0 con 0x7fc2880b5630 2024-09-16T07:38:07.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.302+0000 7fc27f400640 1 -- 172.21.15.71:0/384051819 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278035040 con 0x7fc2880b5630 2024-09-16T07:38:07.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.303+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28815a280 con 0x7fc2880b5630 2024-09-16T07:38:07.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.303+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28815a840 con 0x7fc2880b5630 2024-09-16T07:38:07.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.305+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2880b6e00 con 0x7fc2880b5630 2024-09-16T07:38:07.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.306+0000 7fc27f400640 1 -- 172.21.15.71:0/384051819 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 53) ==== 104704+0+0 (secure 0 0 0) 0x7fc278041030 con 0x7fc2880b5630 2024-09-16T07:38:07.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.307+0000 7fc27f400640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26807ae00 0x7fc26807d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:07.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.307+0000 7fc27f400640 1 -- 172.21.15.71:0/384051819 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fc2780ba2d0 con 0x7fc2880b5630 2024-09-16T07:38:07.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.316+0000 7fc27f400640 1 -- 172.21.15.71:0/384051819 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc278084440 con 0x7fc2880b5630 2024-09-16T07:38:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.349+0000 7fc286a00640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26807ae00 0x7fc26807d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.361+0000 7fc286a00640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26807ae00 0x7fc26807d2c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc288159400 tx=0x7fc274008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:07.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:07.660+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone18140", "target": ["mon-mgr", ""]}) -- 0x7fc2881f2730 con 0x7fc26807ae00 2024-09-16T07:38:07.933 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:07.933+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:07.933 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:07.933+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:07.933 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:07.933+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:07.933 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:07.933+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:07.934 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:07.933+0000 7fe769800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:09.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.099+0000 7fc27f400640 1 -- 172.21.15.71:0/384051819 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc2881f2730 con 0x7fc26807ae00 2024-09-16T07:38:09.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.104+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26807ae00 msgr2=0x7fc26807d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:09.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.104+0000 7fc28e800640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26807ae00 0x7fc26807d2c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc288159400 tx=0x7fc274008040 comp rx=0 tx=0).stop 2024-09-16T07:38:09.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.104+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2880b5630 msgr2=0x7fc2881f21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:09.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.104+0000 7fc28e800640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2880b5630 0x7fc2881f21f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fc278034040 tx=0x7fc2780039e0 comp rx=0 tx=0).stop 2024-09-16T07:38:09.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.105+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 shutdown_connections 2024-09-16T07:38:09.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.105+0000 7fc28e800640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc26807ae00 0x7fc26807d2c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:09.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.105+0000 7fc28e800640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc288161590 0x7fc2881576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:09.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.105+0000 7fc28e800640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2880b7340 0x7fc288159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:09.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.105+0000 7fc28e800640 1 --2- 172.21.15.71:0/384051819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2880b5630 0x7fc2881f21f0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:09.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.105+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 >> 172.21.15.71:0/384051819 conn(0x7fc2880d3fb0 msgr2=0x7fc2880b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:09.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.107+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 shutdown_connections 2024-09-16T07:38:09.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.107+0000 7fc28e800640 1 -- 172.21.15.71:0/384051819 wait complete. 2024-09-16T07:38:09.142 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone33361 2024-09-16T07:38:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.398+0000 7f464ca00640 1 -- 172.21.15.71:0/494073702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f462c003c10 msgr2=0x7f462c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.398+0000 7f464ca00640 1 --2- 172.21.15.71:0/494073702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f462c003c10 0x7f462c004010 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4638001830 tx=0x7f462c005830 comp rx=0 tx=0).stop 2024-09-16T07:38:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.398+0000 7f4646800640 1 -- 172.21.15.71:0/494073702 reap_dead start 2024-09-16T07:38:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.398+0000 7f464ca00640 1 -- 172.21.15.71:0/494073702 shutdown_connections 2024-09-16T07:38:09.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.398+0000 7f464ca00640 1 -- 172.21.15.71:0/494073702 >> 172.21.15.71:0/494073702 conn(0x7f46480b1840 msgr2=0x7f46480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:09.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.399+0000 7f464ca00640 1 -- 172.21.15.71:0/494073702 shutdown_connections 2024-09-16T07:38:09.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.399+0000 7f464ca00640 1 -- 172.21.15.71:0/494073702 wait complete. 2024-09-16T07:38:09.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.402+0000 7f464ca00640 1 Processor -- start 2024-09-16T07:38:09.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.402+0000 7f464ca00640 1 -- start start 2024-09-16T07:38:09.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.403+0000 7f464ca00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f462c003c10 0x7f4648128ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.404+0000 7f464ca00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4648129960 0x7f464815eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.404+0000 7f464ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4648143b10 0x7f4648141910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.404+0000 7f464ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f46480ce2c0 con 0x7f4648143b10 2024-09-16T07:38:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.404+0000 7f464ca00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46480736a0 con 0x7f4648129960 2024-09-16T07:38:09.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.405+0000 7f464ca00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f46480ce440 con 0x7f462c003c10 2024-09-16T07:38:09.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.405+0000 7f4646800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4648143b10 0x7f4648141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:09.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.405+0000 7f4645e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f462c003c10 0x7f4648128ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:09.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.405+0000 7f4645e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f462c003c10 0x7f4648128ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:58614/0 (socket says 172.21.15.71:58614) 2024-09-16T07:38:09.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.405+0000 7f4646800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4648143b10 0x7f4648141910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42526/0 (socket says 172.21.15.71:42526) 2024-09-16T07:38:09.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.405+0000 7f4645e00640 1 -- 172.21.15.71:0/593977204 learned_addr learned my addr 172.21.15.71:0/593977204 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:09.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.406+0000 7f4646800640 1 -- 172.21.15.71:0/593977204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f462c003c10 msgr2=0x7f4648128ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:09.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.406+0000 7f4646800640 1 --2- 172.21.15.71:0/593977204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f462c003c10 0x7f4648128ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.406+0000 7f4646800640 1 -- 172.21.15.71:0/593977204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4648129960 msgr2=0x7f464815eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.406+0000 7f4646800640 1 --2- 172.21.15.71:0/593977204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4648129960 0x7f464815eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.406+0000 7f4646800640 1 -- 172.21.15.71:0/593977204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4648129f20 con 0x7f4648143b10 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.406+0000 7f4646800640 1 --2- 172.21.15.71:0/593977204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4648143b10 0x7f4648141910 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f464806dbc0 tx=0x7f464000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.407+0000 7f463e000640 1 -- 172.21.15.71:0/593977204 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640017070 con 0x7f4648143b10 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.407+0000 7f464ca00640 1 -- 172.21.15.71:0/593977204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464812a200 con 0x7f4648143b10 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.407+0000 7f464ca00640 1 -- 172.21.15.71:0/593977204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4648143310 con 0x7f4648143b10 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.407+0000 7f463e000640 1 -- 172.21.15.71:0/593977204 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4640015070 con 0x7f4648143b10 2024-09-16T07:38:09.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.408+0000 7f463e000640 1 -- 172.21.15.71:0/593977204 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464000eea0 con 0x7f4648143b10 2024-09-16T07:38:09.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.410+0000 7f464ca00640 1 -- 172.21.15.71:0/593977204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4620005290 con 0x7f4648143b10 2024-09-16T07:38:09.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.418+0000 7f463e000640 1 -- 172.21.15.71:0/593977204 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 53) ==== 104704+0+0 (secure 0 0 0) 0x7f4640003e30 con 0x7f4648143b10 2024-09-16T07:38:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.420+0000 7f463e000640 1 --2- 172.21.15.71:0/593977204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f462807ae30 0x7f462807d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.420+0000 7f463e000640 1 -- 172.21.15.71:0/593977204 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f464009a380 con 0x7f4648143b10 2024-09-16T07:38:09.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.426+0000 7f463e000640 1 -- 172.21.15.71:0/593977204 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4640022070 con 0x7f4648143b10 2024-09-16T07:38:09.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.440+0000 7f4645e00640 1 --2- 172.21.15.71:0/593977204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f462807ae30 0x7f462807d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:09.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.458+0000 7f4645e00640 1 --2- 172.21.15.71:0/593977204 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f462807ae30 0x7f462807d2f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4638001830 tx=0x7f4638001360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:09.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.589+0000 7f463e000640 1 -- 172.21.15.71:0/593977204 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f4640060140 con 0x7f4648143b10 2024-09-16T07:38:09.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:09.807+0000 7f464ca00640 1 -- 172.21.15.71:0/593977204 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone33361", "target": ["mon-mgr", ""]}) -- 0x7f4620000bc0 con 0x7f462807ae30 2024-09-16T07:38:10.018 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:10.018+0000 7fe769800640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T07:38:10.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.019+0000 7f463e000640 1 -- 172.21.15.71:0/593977204 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f4620000bc0 con 0x7f462807ae30 2024-09-16T07:38:10.019 INFO:teuthology.orchestra.run.smithi071.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T07:38:10.029 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T07:38:10.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone39912 2024-09-16T07:38:10.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.295+0000 7fc132800640 1 -- 172.21.15.71:0/2900806780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc114004040 msgr2=0x7fc114004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:10.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.295+0000 7fc132800640 1 --2- 172.21.15.71:0/2900806780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc114004040 0x7fc114004800 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c6f0 tx=0x7fc114005830 comp rx=0 tx=0).stop 2024-09-16T07:38:10.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.295+0000 7fc131e00640 1 -- 172.21.15.71:0/2900806780 reap_dead start 2024-09-16T07:38:10.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.295+0000 7fc132800640 1 -- 172.21.15.71:0/2900806780 shutdown_connections 2024-09-16T07:38:10.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.295+0000 7fc132800640 1 -- 172.21.15.71:0/2900806780 >> 172.21.15.71:0/2900806780 conn(0x7fc12c09d0b0 msgr2=0x7fc12c0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:10.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.295+0000 7fc132800640 1 -- 172.21.15.71:0/2900806780 shutdown_connections 2024-09-16T07:38:10.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.295+0000 7fc132800640 1 -- 172.21.15.71:0/2900806780 wait complete. 2024-09-16T07:38:10.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.297+0000 7fc132800640 1 Processor -- start 2024-09-16T07:38:10.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.298+0000 7fc132800640 1 -- start start 2024-09-16T07:38:10.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc132800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc12c15d0f0 0x7fc12c0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:10.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc132800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc12c0ca9a0 0x7fc12c0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:10.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc132800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc12c15d4d0 0x7fc12c0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:10.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc132800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c0b50c0 con 0x7fc12c15d0f0 2024-09-16T07:38:10.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc132800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c0b4f40 con 0x7fc12c0ca9a0 2024-09-16T07:38:10.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc132800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c0b3ac0 con 0x7fc12c15d4d0 2024-09-16T07:38:10.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc131400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc12c15d0f0 0x7fc12c0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:10.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc131400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc12c15d0f0 0x7fc12c0b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42552/0 (socket says 172.21.15.71:42552) 2024-09-16T07:38:10.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc131400640 1 -- 172.21.15.71:0/774427490 learned_addr learned my addr 172.21.15.71:0/774427490 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:10.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc130a00640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc12c0ca9a0 0x7fc12c0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:10.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.299+0000 7fc131e00640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc12c15d4d0 0x7fc12c0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc131400640 1 -- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc12c15d4d0 msgr2=0x7fc12c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc131400640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc12c15d4d0 0x7fc12c0b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc131400640 1 -- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc12c0ca9a0 msgr2=0x7fc12c0b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc131400640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc12c0ca9a0 0x7fc12c0b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc131400640 1 -- 172.21.15.71:0/774427490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c0c85c0 con 0x7fc12c15d0f0 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc130a00640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc12c0ca9a0 0x7fc12c0b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc131400640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc12c15d0f0 0x7fc12c0b2520 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c840 tx=0x7fc11800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc129600640 1 -- 172.21.15.71:0/774427490 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118017070 con 0x7fc12c15d0f0 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.300+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c0c88a0 con 0x7fc12c15d0f0 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.301+0000 7fc129600640 1 -- 172.21.15.71:0/774427490 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1180036d0 con 0x7fc12c15d0f0 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.301+0000 7fc129600640 1 -- 172.21.15.71:0/774427490 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118003aa0 con 0x7fc12c15d0f0 2024-09-16T07:38:10.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.301+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c0b7fb0 con 0x7fc12c15d0f0 2024-09-16T07:38:10.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.302+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c0b33b0 con 0x7fc12c15d0f0 2024-09-16T07:38:10.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.303+0000 7fc129600640 1 -- 172.21.15.71:0/774427490 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7fc11801a070 con 0x7fc12c15d0f0 2024-09-16T07:38:10.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.304+0000 7fc129600640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc10c07ae90 0x7fc10c07d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:10.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.304+0000 7fc129600640 1 -- 172.21.15.71:0/774427490 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fc1180991d0 con 0x7fc12c15d0f0 2024-09-16T07:38:10.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.311+0000 7fc129600640 1 -- 172.21.15.71:0/774427490 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc118062c70 con 0x7fc12c15d0f0 2024-09-16T07:38:10.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.323+0000 7fc130a00640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc10c07ae90 0x7fc10c07d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:10.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.357+0000 7fc130a00640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc10c07ae90 0x7fc10c07d350 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc120032000 tx=0x7fc120001020 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:10.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:10.662+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone39912", "target": ["mon-mgr", ""]}) -- 0x7fc12c1609a0 con 0x7fc10c07ae90 2024-09-16T07:38:12.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.253+0000 7fc129600640 1 -- 172.21.15.71:0/774427490 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc12c1609a0 con 0x7fc10c07ae90 2024-09-16T07:38:12.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc10c07ae90 msgr2=0x7fc10c07d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:12.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc10c07ae90 0x7fc10c07d350 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc120032000 tx=0x7fc120001020 comp rx=0 tx=0).stop 2024-09-16T07:38:12.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc12c15d0f0 msgr2=0x7fc12c0b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc12c15d0f0 0x7fc12c0b2520 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c840 tx=0x7fc11800cd00 comp rx=0 tx=0).stop 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 shutdown_connections 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc10c07ae90 0x7fc10c07d350 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc12c15d4d0 0x7fc12c0b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc12c0ca9a0 0x7fc12c0b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 --2- 172.21.15.71:0/774427490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc12c15d0f0 0x7fc12c0b2520 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 >> 172.21.15.71:0/774427490 conn(0x7fc12c09d0b0 msgr2=0x7fc12c0d3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 shutdown_connections 2024-09-16T07:38:12.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.257+0000 7fc132800640 1 -- 172.21.15.71:0/774427490 wait complete. 2024-09-16T07:38:12.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone87673 2024-09-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.564+0000 7f469d600640 1 -- 172.21.15.71:0/724635772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f469815dc00 msgr2=0x7f46980cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.564+0000 7f469d600640 1 --2- 172.21.15.71:0/724635772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f469815dc00 0x7f46980cdad0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f469806aa00 tx=0x7f468002a4a0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.564+0000 7f469d600640 1 -- 172.21.15.71:0/724635772 shutdown_connections 2024-09-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.564+0000 7f469d600640 1 --2- 172.21.15.71:0/724635772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f469815dc00 0x7f46980cdad0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.564+0000 7f469d600640 1 --2- 172.21.15.71:0/724635772 >> v2:172.21.15.177:3300/0 conn(0x7f46980ca2b0 0x7f46980ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.564+0000 7f469d600640 1 -- 172.21.15.71:0/724635772 >> 172.21.15.71:0/724635772 conn(0x7f46980cf9f0 msgr2=0x7f46980cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.564+0000 7f469d600640 1 -- 172.21.15.71:0/724635772 shutdown_connections 2024-09-16T07:38:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.564+0000 7f469d600640 1 -- 172.21.15.71:0/724635772 wait complete. 2024-09-16T07:38:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.567+0000 7f469d600640 1 Processor -- start 2024-09-16T07:38:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.567+0000 7f469d600640 1 -- start start 2024-09-16T07:38:12.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.568+0000 7f469d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4698070650 0x7f469814ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:12.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.568+0000 7f469d600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46980ca2b0 0x7f4698157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:12.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.568+0000 7f469d600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f469815dc00 0x7f46980c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.568+0000 7f469d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f46980ce5f0 con 0x7f4698070650 2024-09-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.568+0000 7f469d600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46980ce470 con 0x7f469815dc00 2024-09-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.568+0000 7f469d600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f46980ce770 con 0x7f46980ca2b0 2024-09-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.568+0000 7f4697e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4698070650 0x7f469814ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.569+0000 7f4697e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4698070650 0x7f469814ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42582/0 (socket says 172.21.15.71:42582) 2024-09-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.569+0000 7f4697e00640 1 -- 172.21.15.71:0/109096798 learned_addr learned my addr 172.21.15.71:0/109096798 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.569+0000 7f469cc00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f469815dc00 0x7f46980c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.569+0000 7f468fe00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46980ca2b0 0x7f4698157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.570+0000 7f4697e00640 1 -- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46980ca2b0 msgr2=0x7f4698157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.570+0000 7f4697e00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46980ca2b0 0x7f4698157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.570+0000 7f4697e00640 1 -- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f469815dc00 msgr2=0x7f46980c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.570+0000 7f4697e00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f469815dc00 0x7f46980c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.570+0000 7f4697e00640 1 -- 172.21.15.71:0/109096798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46980c6490 con 0x7f4698070650 2024-09-16T07:38:12.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.570+0000 7f468fe00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46980ca2b0 0x7f4698157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.571+0000 7f4697e00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4698070650 0x7f469814ec30 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f468800d950 tx=0x7f468800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.571+0000 7f4694c00640 1 -- 172.21.15.71:0/109096798 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688017070 con 0x7f4698070650 2024-09-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.571+0000 7f4694c00640 1 -- 172.21.15.71:0/109096798 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4688009d20 con 0x7f4698070650 2024-09-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.571+0000 7f4694c00640 1 -- 172.21.15.71:0/109096798 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46880106c0 con 0x7f4698070650 2024-09-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.571+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46980c5ed0 con 0x7f4698070650 2024-09-16T07:38:12.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.571+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469814f520 con 0x7f4698070650 2024-09-16T07:38:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.573+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469806a110 con 0x7f4698070650 2024-09-16T07:38:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.574+0000 7f4694c00640 1 -- 172.21.15.71:0/109096798 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f4688010860 con 0x7f4698070650 2024-09-16T07:38:12.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.575+0000 7f4694c00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f467407ae90 0x7f467407d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:12.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.575+0000 7f4694c00640 1 -- 172.21.15.71:0/109096798 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f4688099ee0 con 0x7f4698070650 2024-09-16T07:38:12.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.581+0000 7f468fe00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f467407ae90 0x7f467407d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:12.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.584+0000 7f4694c00640 1 -- 172.21.15.71:0/109096798 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f469806a110 con 0x7f4698070650 2024-09-16T07:38:12.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.596+0000 7f468fe00640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f467407ae90 0x7f467407d350 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4680034000 tx=0x7f4680004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:12.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:12.956+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone87673", "target": ["mon-mgr", ""]}) -- 0x7f469814f950 con 0x7f467407ae90 2024-09-16T07:38:14.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.250+0000 7f4694c00640 1 -- 172.21.15.71:0/109096798 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f469814f950 con 0x7f467407ae90 2024-09-16T07:38:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f467407ae90 msgr2=0x7f467407d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f467407ae90 0x7f467407d350 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4680034000 tx=0x7f4680004730 comp rx=0 tx=0).stop 2024-09-16T07:38:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4698070650 msgr2=0x7f469814ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4698070650 0x7f469814ec30 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f468800d950 tx=0x7f468800de10 comp rx=0 tx=0).stop 2024-09-16T07:38:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 shutdown_connections 2024-09-16T07:38:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f467407ae90 0x7f467407d350 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f469815dc00 0x7f46980c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:14.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46980ca2b0 0x7f4698157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:14.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 --2- 172.21.15.71:0/109096798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4698070650 0x7f469814ec30 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:14.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 >> 172.21.15.71:0/109096798 conn(0x7f46980cf9f0 msgr2=0x7f4698072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:14.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.254+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 shutdown_connections 2024-09-16T07:38:14.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.255+0000 7f469d600640 1 -- 172.21.15.71:0/109096798 wait complete. 2024-09-16T07:38:14.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone9524 2024-09-16T07:38:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.554+0000 7fa779e00640 1 -- 172.21.15.71:0/671132414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa774072560 msgr2=0x7fa77406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.554+0000 7fa779e00640 1 --2- 172.21.15.71:0/671132414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa774072560 0x7fa77406fce0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa76c005fd0 tx=0x7fa76c02a940 comp rx=0 tx=0).stop 2024-09-16T07:38:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.554+0000 7fa779e00640 1 -- 172.21.15.71:0/671132414 shutdown_connections 2024-09-16T07:38:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.554+0000 7fa779e00640 1 --2- 172.21.15.71:0/671132414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa774072560 0x7fa77406fce0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.554+0000 7fa779e00640 1 -- 172.21.15.71:0/671132414 >> 172.21.15.71:0/671132414 conn(0x7fa7740cf9f0 msgr2=0x7fa7740cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.554+0000 7fa779e00640 1 -- 172.21.15.71:0/671132414 shutdown_connections 2024-09-16T07:38:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.554+0000 7fa779e00640 1 -- 172.21.15.71:0/671132414 wait complete. 2024-09-16T07:38:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.557+0000 7fa779e00640 1 Processor -- start 2024-09-16T07:38:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.557+0000 7fa779e00640 1 -- start start 2024-09-16T07:38:14.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.558+0000 7fa779e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa774072560 0x7fa7741e5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:14.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.558+0000 7fa779e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7740cbc00 0x7fa7741e4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:14.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.558+0000 7fa779e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa77415bec0 0x7fa77414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:14.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.558+0000 7fa779e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7740aebd0 con 0x7fa7740cbc00 2024-09-16T07:38:14.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.558+0000 7fa779e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa774074410 con 0x7fa774072560 2024-09-16T07:38:14.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.558+0000 7fa779e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa7740aed50 con 0x7fa77415bec0 2024-09-16T07:38:14.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa76be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7740cbc00 0x7fa7741e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:14.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa76be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7740cbc00 0x7fa7741e4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42602/0 (socket says 172.21.15.71:42602) 2024-09-16T07:38:14.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa76be00640 1 -- 172.21.15.71:0/724831399 learned_addr learned my addr 172.21.15.71:0/724831399 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:14.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa778a00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa774072560 0x7fa7741e5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:14.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa779400640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa77415bec0 0x7fa77414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:14.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa76be00640 1 -- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa77415bec0 msgr2=0x7fa77414b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:14.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa76be00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa77415bec0 0x7fa77414b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:14.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa76be00640 1 -- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa774072560 msgr2=0x7fa7741e5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:14.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa76be00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa774072560 0x7fa7741e5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:14.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.559+0000 7fa76be00640 1 -- 172.21.15.71:0/724831399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7741e5090 con 0x7fa7740cbc00 2024-09-16T07:38:14.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.560+0000 7fa76be00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7740cbc00 0x7fa7741e4590 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c450 tx=0x7fa75c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:14.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.560+0000 7fa771600640 1 -- 172.21.15.71:0/724831399 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0052d0 con 0x7fa7740cbc00 2024-09-16T07:38:14.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.560+0000 7fa771600640 1 -- 172.21.15.71:0/724831399 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa75c005470 con 0x7fa7740cbc00 2024-09-16T07:38:14.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.560+0000 7fa771600640 1 -- 172.21.15.71:0/724831399 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c012870 con 0x7fa7740cbc00 2024-09-16T07:38:14.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.561+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7741e53d0 con 0x7fa7740cbc00 2024-09-16T07:38:14.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.562+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7740c6b60 con 0x7fa7740cbc00 2024-09-16T07:38:14.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.563+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7740c6d60 con 0x7fa7740cbc00 2024-09-16T07:38:14.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.564+0000 7fa771600640 1 -- 172.21.15.71:0/724831399 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7fa75c012a10 con 0x7fa7740cbc00 2024-09-16T07:38:14.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.565+0000 7fa771600640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa75007ae90 0x7fa75007d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:14.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.566+0000 7fa771600640 1 -- 172.21.15.71:0/724831399 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fa75c099600 con 0x7fa7740cbc00 2024-09-16T07:38:14.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.573+0000 7fa778a00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa75007ae90 0x7fa75007d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:14.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.576+0000 7fa771600640 1 -- 172.21.15.71:0/724831399 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa7740c6d60 con 0x7fa7740cbc00 2024-09-16T07:38:14.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.577+0000 7fa778a00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa75007ae90 0x7fa75007d350 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa7640046b0 tx=0x7fa764009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:14.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:14.957+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone9524", "target": ["mon-mgr", ""]}) -- 0x7fa77406fce0 con 0x7fa75007ae90 2024-09-16T07:38:15.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.616+0000 7fa771600640 1 -- 172.21.15.71:0/724831399 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa77406fce0 con 0x7fa75007ae90 2024-09-16T07:38:15.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa75007ae90 msgr2=0x7fa75007d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:15.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa75007ae90 0x7fa75007d350 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa7640046b0 tx=0x7fa764009040 comp rx=0 tx=0).stop 2024-09-16T07:38:15.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7740cbc00 msgr2=0x7fa7741e4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:15.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7740cbc00 0x7fa7741e4590 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c450 tx=0x7fa75c00c910 comp rx=0 tx=0).stop 2024-09-16T07:38:15.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 shutdown_connections 2024-09-16T07:38:15.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa75007ae90 0x7fa75007d350 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa77415bec0 0x7fa77414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7740cbc00 0x7fa7741e4590 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 --2- 172.21.15.71:0/724831399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa774072560 0x7fa7741e5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 >> 172.21.15.71:0/724831399 conn(0x7fa7740cf9f0 msgr2=0x7fa7740b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.619+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 shutdown_connections 2024-09-16T07:38:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.620+0000 7fa779e00640 1 -- 172.21.15.71:0/724831399 wait complete. 2024-09-16T07:38:15.640 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone13325 2024-09-16T07:38:15.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.921+0000 7f60eda00640 1 -- 172.21.15.71:0/1985779840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e8145750 msgr2=0x7f60e8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:15.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.921+0000 7f60eda00640 1 --2- 172.21.15.71:0/1985779840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e8145750 0x7f60e8073440 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f60e80cfb70 tx=0x7f60d802a980 comp rx=0 tx=0).stop 2024-09-16T07:38:15.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.922+0000 7f60eda00640 1 -- 172.21.15.71:0/1985779840 shutdown_connections 2024-09-16T07:38:15.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.922+0000 7f60eda00640 1 --2- 172.21.15.71:0/1985779840 >> v2:172.21.15.177:3300/0 conn(0x7f60e80cccd0 0x7f60e80cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:15.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.922+0000 7f60eda00640 1 --2- 172.21.15.71:0/1985779840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e8145750 0x7f60e8073440 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:15.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.922+0000 7f60eda00640 1 -- 172.21.15.71:0/1985779840 >> 172.21.15.71:0/1985779840 conn(0x7f60e80b1840 msgr2=0x7f60e80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:15.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.922+0000 7f60eda00640 1 -- 172.21.15.71:0/1985779840 shutdown_connections 2024-09-16T07:38:15.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.922+0000 7f60eda00640 1 -- 172.21.15.71:0/1985779840 wait complete. 2024-09-16T07:38:15.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.923+0000 7f60eda00640 1 Processor -- start 2024-09-16T07:38:15.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.923+0000 7f60eda00640 1 -- start start 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.924+0000 7f60eda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e806efa0 0x7f60e8140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.924+0000 7f60eda00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60e80cccd0 0x7f60e80a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.924+0000 7f60eda00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60e8145750 0x7f60e80ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.924+0000 7f60eda00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60e80ce480 con 0x7f60e806efa0 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.924+0000 7f60eda00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60e80ce300 con 0x7f60e8145750 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.924+0000 7f60eda00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60e80ce600 con 0x7f60e80cccd0 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e806efa0 0x7f60e8140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e806efa0 0x7f60e8140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48652/0 (socket says 172.21.15.71:48652) 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6a00640 1 -- 172.21.15.71:0/1299527834 learned_addr learned my addr 172.21.15.71:0/1299527834 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6000640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60e80cccd0 0x7f60e80a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:15.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6a00640 1 -- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60e80cccd0 msgr2=0x7f60e80a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6a00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60e80cccd0 0x7f60e80a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6a00640 1 -- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60e8145750 msgr2=0x7f60e80ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6a00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60e8145750 0x7f60e80ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.925+0000 7f60e6a00640 1 -- 172.21.15.71:0/1299527834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e81411f0 con 0x7f60e806efa0 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.926+0000 7f60e6a00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e806efa0 0x7f60e8140cb0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f60d8000f80 tx=0x7f60d8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.926+0000 7f60dea00640 1 -- 172.21.15.71:0/1299527834 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8034040 con 0x7f60e806efa0 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.926+0000 7f60dea00640 1 -- 172.21.15.71:0/1299527834 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60d80046c0 con 0x7f60e806efa0 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.926+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e8141520 con 0x7f60e806efa0 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.926+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e80abff0 con 0x7f60e806efa0 2024-09-16T07:38:15.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.926+0000 7f60dea00640 1 -- 172.21.15.71:0/1299527834 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d80456e0 con 0x7f60e806efa0 2024-09-16T07:38:15.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.928+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c0005290 con 0x7f60e806efa0 2024-09-16T07:38:15.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.929+0000 7f60dea00640 1 -- 172.21.15.71:0/1299527834 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f60d8037020 con 0x7f60e806efa0 2024-09-16T07:38:15.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.930+0000 7f60dea00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60c807ae90 0x7f60c807d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:15.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.930+0000 7f60dea00640 1 -- 172.21.15.71:0/1299527834 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f60d80bb430 con 0x7f60e806efa0 2024-09-16T07:38:15.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.934+0000 7f60e6000640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60c807ae90 0x7f60c807d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:15.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.935+0000 7f60dea00640 1 -- 172.21.15.71:0/1299527834 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f60e8141520 con 0x7f60e806efa0 2024-09-16T07:38:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:15.939+0000 7f60e6000640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60c807ae90 0x7f60c807d350 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f60d4007fd0 tx=0x7f60d4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.278+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone13325", "target": ["mon-mgr", ""]}) -- 0x7f60c0000bc0 con 0x7f60c807ae90 2024-09-16T07:38:16.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.794+0000 7f60dea00640 1 -- 172.21.15.71:0/1299527834 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f60c0000bc0 con 0x7f60c807ae90 2024-09-16T07:38:16.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60c807ae90 msgr2=0x7f60c807d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:16.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60c807ae90 0x7f60c807d350 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f60d4007fd0 tx=0x7f60d4009040 comp rx=0 tx=0).stop 2024-09-16T07:38:16.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e806efa0 msgr2=0x7f60e8140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:16.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e806efa0 0x7f60e8140cb0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f60d8000f80 tx=0x7f60d8003f70 comp rx=0 tx=0).stop 2024-09-16T07:38:16.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 shutdown_connections 2024-09-16T07:38:16.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f60c807ae90 0x7f60c807d350 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:16.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60e8145750 0x7f60e80ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:16.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60e80cccd0 0x7f60e80a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:16.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 --2- 172.21.15.71:0/1299527834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e806efa0 0x7f60e8140cb0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:16.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 >> 172.21.15.71:0/1299527834 conn(0x7f60e80b1840 msgr2=0x7f60e80b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:16.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 shutdown_connections 2024-09-16T07:38:16.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:16.798+0000 7f60eda00640 1 -- 172.21.15.71:0/1299527834 wait complete. 2024-09-16T07:38:16.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone89816 2024-09-16T07:38:17.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.064+0000 7effa8800640 1 -- 172.21.15.71:0/111846229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 msgr2=0x7effa00cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:17.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.064+0000 7effa8800640 1 --2- 172.21.15.71:0/111846229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 0x7effa00cf070 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7effa00b2470 tx=0x7eff9c02aac0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.064+0000 7effa8800640 1 -- 172.21.15.71:0/111846229 shutdown_connections 2024-09-16T07:38:17.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.064+0000 7effa8800640 1 --2- 172.21.15.71:0/111846229 >> v2:172.21.15.177:3300/0 conn(0x7effa00af190 0x7effa00b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.064+0000 7effa8800640 1 --2- 172.21.15.71:0/111846229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 0x7effa00cf070 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.064+0000 7effa8800640 1 -- 172.21.15.71:0/111846229 >> 172.21.15.71:0/111846229 conn(0x7effa00b1840 msgr2=0x7effa00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:17.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.064+0000 7effa8800640 1 -- 172.21.15.71:0/111846229 shutdown_connections 2024-09-16T07:38:17.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.064+0000 7effa8800640 1 -- 172.21.15.71:0/111846229 wait complete. 2024-09-16T07:38:17.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.066+0000 7effa8800640 1 Processor -- start 2024-09-16T07:38:17.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.066+0000 7effa8800640 1 -- start start 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa8800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00af190 0x7effa01527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa8800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 0x7effa00b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa8800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0161760 0x7effa00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa8800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7effa0162310 con 0x7effa00cec60 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa8800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effa0162190 con 0x7effa0161760 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa8800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effa0162490 con 0x7effa00af190 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 0x7effa00b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 0x7effa00b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48668/0 (socket says 172.21.15.71:48668) 2024-09-16T07:38:17.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.067+0000 7effa5400640 1 -- 172.21.15.71:0/309751223 learned_addr learned my addr 172.21.15.71:0/309751223 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7effa5400640 1 -- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00af190 msgr2=0x7effa01527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7effa5400640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00af190 0x7effa01527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7effa5400640 1 -- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0161760 msgr2=0x7effa00b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7effa5400640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0161760 0x7effa00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7effa5400640 1 -- 172.21.15.71:0/309751223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa01f49f0 con 0x7effa00cec60 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7effa5400640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 0x7effa00b07f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7eff9000d950 tx=0x7eff9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7eff9a000640 1 -- 172.21.15.71:0/309751223 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90017070 con 0x7effa00cec60 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7eff9a000640 1 -- 172.21.15.71:0/309751223 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff90009d20 con 0x7effa00cec60 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.068+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01f4c60 con 0x7effa00cec60 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.069+0000 7eff9a000640 1 -- 172.21.15.71:0/309751223 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900106c0 con 0x7effa00cec60 2024-09-16T07:38:17.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.069+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa01f5220 con 0x7effa00cec60 2024-09-16T07:38:17.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.070+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa00cf070 con 0x7effa00cec60 2024-09-16T07:38:17.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.071+0000 7eff9a000640 1 -- 172.21.15.71:0/309751223 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7eff90010860 con 0x7effa00cec60 2024-09-16T07:38:17.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.072+0000 7eff9a000640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff8407ae90 0x7eff8407d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:17.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.072+0000 7eff9a000640 1 -- 172.21.15.71:0/309751223 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7eff9009a000 con 0x7effa00cec60 2024-09-16T07:38:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.075+0000 7effa5e00640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff8407ae90 0x7eff8407d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.076+0000 7eff9a000640 1 -- 172.21.15.71:0/309751223 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7eff900640f0 con 0x7effa00cec60 2024-09-16T07:38:17.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.078+0000 7effa5e00640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff8407ae90 0x7eff8407d350 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7eff94000f60 tx=0x7eff940079d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:17.167 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:38:17.174 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:38:17.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.426+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone89816", "target": ["mon-mgr", ""]}) -- 0x7effa00b3430 con 0x7eff8407ae90 2024-09-16T07:38:17.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.572+0000 7eff9a000640 1 -- 172.21.15.71:0/309751223 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7effa00b3430 con 0x7eff8407ae90 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff8407ae90 msgr2=0x7eff8407d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff8407ae90 0x7eff8407d350 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7eff94000f60 tx=0x7eff940079d0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 msgr2=0x7effa00b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 0x7effa00b07f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7eff9000d950 tx=0x7eff9000de10 comp rx=0 tx=0).stop 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 shutdown_connections 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7eff8407ae90 0x7eff8407d350 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0161760 0x7effa00b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa00cec60 0x7effa00b07f0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 --2- 172.21.15.71:0/309751223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00af190 0x7effa01527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 >> 172.21.15.71:0/309751223 conn(0x7effa00b1840 msgr2=0x7effa00b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:17.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 shutdown_connections 2024-09-16T07:38:17.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.575+0000 7effa8800640 1 -- 172.21.15.71:0/309751223 wait complete. 2024-09-16T07:38:17.591 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone18140 2024-09-16T07:38:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.752+0000 7f2c1c400640 1 -- 172.21.15.71:0/1820605568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 msgr2=0x7f2c14073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.752+0000 7f2c1c400640 1 --2- 172.21.15.71:0/1820605568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 0x7f2c14073440 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f2c140af750 tx=0x7f2c0802a4d0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.752+0000 7f2c1c400640 1 -- 172.21.15.71:0/1820605568 shutdown_connections 2024-09-16T07:38:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.752+0000 7f2c1c400640 1 --2- 172.21.15.71:0/1820605568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c140ccb00 0x7f2c1415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.752+0000 7f2c1c400640 1 --2- 172.21.15.71:0/1820605568 >> v2:172.21.15.177:3300/0 conn(0x7f2c1406efa0 0x7f2c140cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.752+0000 7f2c1c400640 1 --2- 172.21.15.71:0/1820605568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 0x7f2c14073440 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.752+0000 7f2c1c400640 1 -- 172.21.15.71:0/1820605568 >> 172.21.15.71:0/1820605568 conn(0x7f2c140b1840 msgr2=0x7f2c140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.753+0000 7f2c1c400640 1 -- 172.21.15.71:0/1820605568 shutdown_connections 2024-09-16T07:38:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.753+0000 7f2c1c400640 1 -- 172.21.15.71:0/1820605568 wait complete. 2024-09-16T07:38:17.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.754+0000 7f2c1c400640 1 Processor -- start 2024-09-16T07:38:17.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.754+0000 7f2c1c400640 1 -- start start 2024-09-16T07:38:17.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1c400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c1406efa0 0x7f2c141f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:17.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1c400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c140ccb00 0x7f2c14141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:17.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1c400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 0x7f2c14143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:17.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1c400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2c141602c0 con 0x7f2c14145750 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1c400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c14160140 con 0x7f2c1406efa0 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1c400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2c14160440 con 0x7f2c140ccb00 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 0x7f2c14143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c19a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c1406efa0 0x7f2c141f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1a400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 0x7f2c14143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48676/0 (socket says 172.21.15.71:48676) 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.755+0000 7f2c1a400640 1 -- 172.21.15.71:0/617403347 learned_addr learned my addr 172.21.15.71:0/617403347 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c19000640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c140ccb00 0x7f2c14141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c1a400640 1 -- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c140ccb00 msgr2=0x7f2c14141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c1a400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c140ccb00 0x7f2c14141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c1a400640 1 -- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c1406efa0 msgr2=0x7f2c141f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c1a400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c1406efa0 0x7f2c141f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c1a400640 1 -- 172.21.15.71:0/617403347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c141f7aa0 con 0x7f2c14145750 2024-09-16T07:38:17.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c1a400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 0x7f2c14143120 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00eb10 tx=0x7f2c0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:17.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c11600640 1 -- 172.21.15.71:0/617403347 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c004b20 con 0x7f2c14145750 2024-09-16T07:38:17.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c11600640 1 -- 172.21.15.71:0/617403347 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c0c004ce0 con 0x7f2c14145750 2024-09-16T07:38:17.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c11600640 1 -- 172.21.15.71:0/617403347 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c005280 con 0x7f2c14145750 2024-09-16T07:38:17.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.756+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c141f7d80 con 0x7f2c14145750 2024-09-16T07:38:17.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.757+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c141f8340 con 0x7f2c14145750 2024-09-16T07:38:17.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.757+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1415f9f0 con 0x7f2c14145750 2024-09-16T07:38:17.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.759+0000 7f2c11600640 1 -- 172.21.15.71:0/617403347 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f2c0c005420 con 0x7f2c14145750 2024-09-16T07:38:17.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.760+0000 7f2c11600640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bf407aec0 0x7f2bf407d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:17.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.760+0000 7f2c11600640 1 -- 172.21.15.71:0/617403347 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f2c0c09a7a0 con 0x7f2c14145750 2024-09-16T07:38:17.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.760+0000 7f2c19a00640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bf407aec0 0x7f2bf407d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:17.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.761+0000 7f2c19a00640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bf407aec0 0x7f2bf407d380 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2c08033000 tx=0x7f2c08004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:17.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.764+0000 7f2c11600640 1 -- 172.21.15.71:0/617403347 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c0c064220 con 0x7f2c14145750 2024-09-16T07:38:17.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.986+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone18140", "target": ["mon-mgr", ""]}) -- 0x7f2c140ae7e0 con 0x7f2bf407aec0 2024-09-16T07:38:17.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:17.998+0000 7f2c11600640 1 -- 172.21.15.71:0/617403347 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f2c140ae7e0 con 0x7f2bf407aec0 2024-09-16T07:38:17.997 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:17.997 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:38:17.997 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:38:17.997 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:17.998 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:18.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bf407aec0 msgr2=0x7f2bf407d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bf407aec0 0x7f2bf407d380 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2c08033000 tx=0x7f2c08004730 comp rx=0 tx=0).stop 2024-09-16T07:38:18.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 msgr2=0x7f2c14143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 0x7f2c14143120 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00eb10 tx=0x7f2c0c00c2d0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 shutdown_connections 2024-09-16T07:38:18.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bf407aec0 0x7f2bf407d380 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c14145750 0x7f2c14143120 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c140ccb00 0x7f2c14141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 --2- 172.21.15.71:0/617403347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c1406efa0 0x7f2c141f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 >> 172.21.15.71:0/617403347 conn(0x7f2c140b1840 msgr2=0x7f2c140cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 shutdown_connections 2024-09-16T07:38:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.000+0000 7f2c1c400640 1 -- 172.21.15.71:0/617403347 wait complete. 2024-09-16T07:38:18.015 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:38:18.015 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33361 2024-09-16T07:38:18.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.175+0000 7fab24c00640 1 -- 172.21.15.71:0/1417556173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab04003a30 msgr2=0x7fab04003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.175+0000 7fab24c00640 1 --2- 172.21.15.71:0/1417556173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab04003a30 0x7fab04003e50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c310 tx=0x7fab04005830 comp rx=0 tx=0).stop 2024-09-16T07:38:18.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.175+0000 7fab1e800640 1 -- 172.21.15.71:0/1417556173 reap_dead start 2024-09-16T07:38:18.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.175+0000 7fab24c00640 1 -- 172.21.15.71:0/1417556173 shutdown_connections 2024-09-16T07:38:18.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.175+0000 7fab24c00640 1 -- 172.21.15.71:0/1417556173 >> 172.21.15.71:0/1417556173 conn(0x7fab200b1840 msgr2=0x7fab200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:18.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.175+0000 7fab24c00640 1 -- 172.21.15.71:0/1417556173 shutdown_connections 2024-09-16T07:38:18.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.175+0000 7fab24c00640 1 -- 172.21.15.71:0/1417556173 wait complete. 2024-09-16T07:38:18.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.177+0000 7fab24c00640 1 Processor -- start 2024-09-16T07:38:18.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.177+0000 7fab24c00640 1 -- start start 2024-09-16T07:38:18.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab24c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab04003a30 0x7fab2013c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:18.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab24c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab201f8360 0x7fab2013cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:18.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab24c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab201f87e0 0x7fab20142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:18.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab24c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab200ce2c0 con 0x7fab201f87e0 2024-09-16T07:38:18.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab24c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fab200736a0 con 0x7fab201f8360 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab24c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fab200ce440 con 0x7fab04003a30 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab201f87e0 0x7fab20142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab201f87e0 0x7fab20142510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48690/0 (socket says 172.21.15.71:48690) 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1e800640 1 -- 172.21.15.71:0/2830410552 learned_addr learned my addr 172.21.15.71:0/2830410552 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1de00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab04003a30 0x7fab2013c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab17e00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab201f8360 0x7fab2013cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1e800640 1 -- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab04003a30 msgr2=0x7fab2013c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1e800640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab04003a30 0x7fab2013c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1e800640 1 -- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab201f8360 msgr2=0x7fab2013cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1e800640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab201f8360 0x7fab2013cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.178+0000 7fab1e800640 1 -- 172.21.15.71:0/2830410552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab20141470 con 0x7fab201f87e0 2024-09-16T07:38:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.179+0000 7fab1e800640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab201f87e0 0x7fab20142510 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fab2015f950 tx=0x7fab1800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.179+0000 7fab16000640 1 -- 172.21.15.71:0/2830410552 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1801e070 con 0x7fab201f87e0 2024-09-16T07:38:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.179+0000 7fab16000640 1 -- 172.21.15.71:0/2830410552 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab1800e7c0 con 0x7fab201f87e0 2024-09-16T07:38:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.179+0000 7fab16000640 1 -- 172.21.15.71:0/2830410552 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18009ad0 con 0x7fab201f87e0 2024-09-16T07:38:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.179+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab201417b0 con 0x7fab201f87e0 2024-09-16T07:38:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.179+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab20141ff0 con 0x7fab201f87e0 2024-09-16T07:38:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.180+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab200ce9d0 con 0x7fab201f87e0 2024-09-16T07:38:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.181+0000 7fab16000640 1 -- 172.21.15.71:0/2830410552 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7fab18021070 con 0x7fab201f87e0 2024-09-16T07:38:18.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.182+0000 7fab16000640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faafc07aec0 0x7faafc07d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:18.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.182+0000 7fab16000640 1 -- 172.21.15.71:0/2830410552 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fab1809a550 con 0x7fab201f87e0 2024-09-16T07:38:18.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.183+0000 7fab1de00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faafc07aec0 0x7faafc07d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:18.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.184+0000 7fab1de00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faafc07aec0 0x7faafc07d380 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fab10033000 tx=0x7fab10038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:18.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.187+0000 7fab16000640 1 -- 172.21.15.71:0/2830410552 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fab201417b0 con 0x7fab201f87e0 2024-09-16T07:38:18.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.408+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33361", "target": ["mon-mgr", ""]}) -- 0x7fab2012d450 con 0x7faafc07aec0 2024-09-16T07:38:18.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.417+0000 7fab16000640 1 -- 172.21.15.71:0/2830410552 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fab2012d450 con 0x7faafc07aec0 2024-09-16T07:38:18.416 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:18.416 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:38:18.416 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "complete" 2024-09-16T07:38:18.416 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:18.416 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.418+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faafc07aec0 msgr2=0x7faafc07d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.418+0000 7fab24c00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faafc07aec0 0x7faafc07d380 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fab10033000 tx=0x7fab10038040 comp rx=0 tx=0).stop 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.418+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab201f87e0 msgr2=0x7fab20142510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.418+0000 7fab24c00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab201f87e0 0x7fab20142510 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fab2015f950 tx=0x7fab1800f790 comp rx=0 tx=0).stop 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.419+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 shutdown_connections 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.419+0000 7fab24c00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7faafc07aec0 0x7faafc07d380 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.419+0000 7fab24c00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab201f87e0 0x7fab20142510 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.419+0000 7fab24c00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab201f8360 0x7fab2013cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.419+0000 7fab24c00640 1 --2- 172.21.15.71:0/2830410552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab04003a30 0x7fab2013c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.419+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 >> 172.21.15.71:0/2830410552 conn(0x7fab200b1840 msgr2=0x7fab200a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.419+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 shutdown_connections 2024-09-16T07:38:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.419+0000 7fab24c00640 1 -- 172.21.15.71:0/2830410552 wait complete. 2024-09-16T07:38:18.434 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-16T07:38:18.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone39912 2024-09-16T07:38:18.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.595+0000 7f477a800640 1 -- 172.21.15.71:0/3828662242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 msgr2=0x7f47740cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.595+0000 7f477a800640 1 --2- 172.21.15.71:0/3828662242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 0x7f47740cb610 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f476c005fd0 tx=0x7f476c02a8b0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.595+0000 7f477a800640 1 -- 172.21.15.71:0/3828662242 shutdown_connections 2024-09-16T07:38:18.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.595+0000 7f477a800640 1 --2- 172.21.15.71:0/3828662242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 0x7f47740cb610 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.595+0000 7f477a800640 1 --2- 172.21.15.71:0/3828662242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f477406f6b0 0x7f47740cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.595+0000 7f477a800640 1 -- 172.21.15.71:0/3828662242 >> 172.21.15.71:0/3828662242 conn(0x7f47740b2670 msgr2=0x7f47740b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:18.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.595+0000 7f477a800640 1 -- 172.21.15.71:0/3828662242 shutdown_connections 2024-09-16T07:38:18.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.595+0000 7f477a800640 1 -- 172.21.15.71:0/3828662242 wait complete. 2024-09-16T07:38:18.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.597+0000 7f477a800640 1 Processor -- start 2024-09-16T07:38:18.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.597+0000 7f477a800640 1 -- start start 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f477a800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f477406f6b0 0x7f4774153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f477a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 0x7f4774159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f477a800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f477415aad0 0x7f47740c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f477a800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f47740cc080 con 0x7f47740cf500 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f477a800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47740cbf00 con 0x7f477415aad0 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f477a800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f47740cc200 con 0x7f477406f6b0 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4778a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 0x7f4774159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4778a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 0x7f4774159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48716/0 (socket says 172.21.15.71:48716) 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4778a00640 1 -- 172.21.15.71:0/1879272425 learned_addr learned my addr 172.21.15.71:0/1879272425 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4779400640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f477406f6b0 0x7f4774153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4779e00640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f477415aad0 0x7f47740c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:18.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4778a00640 1 -- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f477406f6b0 msgr2=0x7f4774153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4778a00640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f477406f6b0 0x7f4774153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4778a00640 1 -- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f477415aad0 msgr2=0x7f47740c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4778a00640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f477415aad0 0x7f47740c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4778a00640 1 -- 172.21.15.71:0/1879272425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477415a320 con 0x7f47740cf500 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4779400640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f477406f6b0 0x7f4774153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.598+0000 7f4779e00640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f477415aad0 0x7f47740c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.599+0000 7f4778a00640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 0x7f4774159940 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f476000ea20 tx=0x7f476000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.599+0000 7f4771600640 1 -- 172.21.15.71:0/1879272425 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476000cd80 con 0x7f47740cf500 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.599+0000 7f4771600640 1 -- 172.21.15.71:0/1879272425 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4760010430 con 0x7f47740cf500 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.599+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47740b5fd0 con 0x7f47740cf500 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.599+0000 7f4771600640 1 -- 172.21.15.71:0/1879272425 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760010840 con 0x7f47740cf500 2024-09-16T07:38:18.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.599+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47740b6550 con 0x7f47740cf500 2024-09-16T07:38:18.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.600+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47740cc720 con 0x7f47740cf500 2024-09-16T07:38:18.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.601+0000 7f4771600640 1 -- 172.21.15.71:0/1879272425 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f47600109e0 con 0x7f47740cf500 2024-09-16T07:38:18.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.602+0000 7f4771600640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f475407ae90 0x7f475407d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:18.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.602+0000 7f4771600640 1 -- 172.21.15.71:0/1879272425 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f476009a860 con 0x7f47740cf500 2024-09-16T07:38:18.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.603+0000 7f4779400640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f475407ae90 0x7f475407d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:18.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.603+0000 7f4779400640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f475407ae90 0x7f475407d350 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4768001610 tx=0x7f4768009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:18.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.606+0000 7f4771600640 1 -- 172.21.15.71:0/1879272425 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4760064930 con 0x7f47740cf500 2024-09-16T07:38:18.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.828+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone39912", "target": ["mon-mgr", ""]}) -- 0x7f47740ce6c0 con 0x7f475407ae90 2024-09-16T07:38:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.839+0000 7f4771600640 1 -- 172.21.15.71:0/1879272425 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f47740ce6c0 con 0x7f475407ae90 2024-09-16T07:38:18.838 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:38:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:38:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:38:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:38:18.838 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:38:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:38:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:38:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:38:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:38:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:18.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:18.839 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f475407ae90 msgr2=0x7f475407d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f475407ae90 0x7f475407d350 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4768001610 tx=0x7f4768009040 comp rx=0 tx=0).stop 2024-09-16T07:38:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 msgr2=0x7f4774159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 0x7f4774159940 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f476000ea20 tx=0x7f476000eee0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 shutdown_connections 2024-09-16T07:38:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f475407ae90 0x7f475407d350 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f477415aad0 0x7f47740c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47740cf500 0x7f4774159940 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 --2- 172.21.15.71:0/1879272425 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f477406f6b0 0x7f4774153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 >> 172.21.15.71:0/1879272425 conn(0x7f47740b2670 msgr2=0x7f47740c6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 shutdown_connections 2024-09-16T07:38:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:18.841+0000 7f477a800640 1 -- 172.21.15.71:0/1879272425 wait complete. 2024-09-16T07:38:18.858 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:38:18.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87673 2024-09-16T07:38:19.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.018+0000 7fb0fde00640 1 -- 172.21.15.71:0/2400113150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f806efa0 msgr2=0x7fb0f80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:19.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.018+0000 7fb0fde00640 1 --2- 172.21.15.71:0/2400113150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f806efa0 0x7fb0f80cdbb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009830 tx=0x7fb0f002a480 comp rx=0 tx=0).stop 2024-09-16T07:38:19.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.018+0000 7fb0fde00640 1 -- 172.21.15.71:0/2400113150 shutdown_connections 2024-09-16T07:38:19.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.018+0000 7fb0fde00640 1 --2- 172.21.15.71:0/2400113150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f806efa0 0x7fb0f80cdbb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.018+0000 7fb0fde00640 1 --2- 172.21.15.71:0/2400113150 >> v2:172.21.15.177:3300/0 conn(0x7fb0f80cccd0 0x7fb0f8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.018+0000 7fb0fde00640 1 -- 172.21.15.71:0/2400113150 >> 172.21.15.71:0/2400113150 conn(0x7fb0f80b1840 msgr2=0x7fb0f80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:19.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.019+0000 7fb0fde00640 1 -- 172.21.15.71:0/2400113150 shutdown_connections 2024-09-16T07:38:19.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.019+0000 7fb0fde00640 1 -- 172.21.15.71:0/2400113150 wait complete. 2024-09-16T07:38:19.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.020+0000 7fb0fde00640 1 Processor -- start 2024-09-16T07:38:19.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.020+0000 7fb0fde00640 1 -- start start 2024-09-16T07:38:19.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.021+0000 7fb0fde00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f806efa0 0x7fb0f81f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:19.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.021+0000 7fb0fde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80cccd0 0x7fb0f80c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:19.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.021+0000 7fb0fde00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f8145750 0x7fb0f81f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:19.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.021+0000 7fb0fde00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f80ce480 con 0x7fb0f80cccd0 2024-09-16T07:38:19.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.021+0000 7fb0fde00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f80ce300 con 0x7fb0f806efa0 2024-09-16T07:38:19.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.021+0000 7fb0fde00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f80ce600 con 0x7fb0f8145750 2024-09-16T07:38:19.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80cccd0 0x7fb0f80c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80cccd0 0x7fb0f80c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48730/0 (socket says 172.21.15.71:48730) 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 -- 172.21.15.71:0/4173372466 learned_addr learned my addr 172.21.15.71:0/4173372466 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f7400640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f8145750 0x7fb0f81f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6a00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f806efa0 0x7fb0f81f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 -- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f8145750 msgr2=0x7fb0f81f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f8145750 0x7fb0f81f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 -- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f806efa0 msgr2=0x7fb0f81f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f806efa0 0x7fb0f81f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 -- 172.21.15.71:0/4173372466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f813e0e0 con 0x7fb0f80cccd0 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.022+0000 7fb0f6000640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80cccd0 0x7fb0f80c4ab0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400e9f0 tx=0x7fb0e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.023+0000 7fb0ef400640 1 -- 172.21.15.71:0/4173372466 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400cd20 con 0x7fb0f80cccd0 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.023+0000 7fb0ef400640 1 -- 172.21.15.71:0/4173372466 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0e4010430 con 0x7fb0f80cccd0 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.023+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f813e3c0 con 0x7fb0f80cccd0 2024-09-16T07:38:19.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.023+0000 7fb0ef400640 1 -- 172.21.15.71:0/4173372466 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4010840 con 0x7fb0f80cccd0 2024-09-16T07:38:19.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.023+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f813e980 con 0x7fb0f80cccd0 2024-09-16T07:38:19.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.024+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f80cdbb0 con 0x7fb0f80cccd0 2024-09-16T07:38:19.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.025+0000 7fb0ef400640 1 -- 172.21.15.71:0/4173372466 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7fb0e40109e0 con 0x7fb0f80cccd0 2024-09-16T07:38:19.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.026+0000 7fb0ef400640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d807ae90 0x7fb0d807d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:19.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.027+0000 7fb0ef400640 1 -- 172.21.15.71:0/4173372466 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fb0e409a690 con 0x7fb0f80cccd0 2024-09-16T07:38:19.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.027+0000 7fb0f6a00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d807ae90 0x7fb0d807d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:19.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.028+0000 7fb0f6a00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d807ae90 0x7fb0d807d350 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80017a0 tx=0x7fb0e8001730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:19.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.031+0000 7fb0ef400640 1 -- 172.21.15.71:0/4173372466 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb0e40641d0 con 0x7fb0f80cccd0 2024-09-16T07:38:19.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.249+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87673", "target": ["mon-mgr", ""]}) -- 0x7fb0f806ed80 con 0x7fb0d807ae90 2024-09-16T07:38:19.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.259+0000 7fb0ef400640 1 -- 172.21.15.71:0/4173372466 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fb0f806ed80 con 0x7fb0d807ae90 2024-09-16T07:38:19.258 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:19.258 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:38:19.258 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:38:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:38:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:38:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:38:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:38:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:38:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:38:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:38:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:19.261 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:19.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d807ae90 msgr2=0x7fb0d807d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:19.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d807ae90 0x7fb0d807d350 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80017a0 tx=0x7fb0e8001730 comp rx=0 tx=0).stop 2024-09-16T07:38:19.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80cccd0 msgr2=0x7fb0f80c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:19.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80cccd0 0x7fb0f80c4ab0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400e9f0 tx=0x7fb0e400eeb0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 shutdown_connections 2024-09-16T07:38:19.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb0d807ae90 0x7fb0d807d350 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0f8145750 0x7fb0f81f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0f80cccd0 0x7fb0f80c4ab0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 --2- 172.21.15.71:0/4173372466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0f806efa0 0x7fb0f81f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 >> 172.21.15.71:0/4173372466 conn(0x7fb0f80b1840 msgr2=0x7fb0f80af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:19.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 shutdown_connections 2024-09-16T07:38:19.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.261+0000 7fb0fde00640 1 -- 172.21.15.71:0/4173372466 wait complete. 2024-09-16T07:38:19.276 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:38:19.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone9524 2024-09-16T07:38:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.436+0000 7f162b200640 1 -- 172.21.15.71:0/1407125829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 msgr2=0x7f162c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.436+0000 7f162b200640 1 --2- 172.21.15.71:0/1407125829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 0x7f162c06fce0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f162c06f130 tx=0x7f161c02a660 comp rx=0 tx=0).stop 2024-09-16T07:38:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.436+0000 7f162b200640 1 -- 172.21.15.71:0/1407125829 shutdown_connections 2024-09-16T07:38:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.436+0000 7f162b200640 1 --2- 172.21.15.71:0/1407125829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 0x7f162c06fce0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.436+0000 7f162b200640 1 -- 172.21.15.71:0/1407125829 >> 172.21.15.71:0/1407125829 conn(0x7f162c0cf9f0 msgr2=0x7f162c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.436+0000 7f162b200640 1 -- 172.21.15.71:0/1407125829 shutdown_connections 2024-09-16T07:38:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.436+0000 7f162b200640 1 -- 172.21.15.71:0/1407125829 wait complete. 2024-09-16T07:38:19.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.437+0000 7f162b200640 1 Processor -- start 2024-09-16T07:38:19.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.437+0000 7f162b200640 1 -- start start 2024-09-16T07:38:19.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.438+0000 7f162b200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c072560 0x7f162c14ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:19.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.438+0000 7f162b200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c0cbc00 0x7f162c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:19.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.438+0000 7f162b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 0x7f162c0b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:19.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.438+0000 7f162b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0aebd0 con 0x7f162c15bec0 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.438+0000 7f162b200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f162c074400 con 0x7f162c072560 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.438+0000 7f162b200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f162c0aed50 con 0x7f162c0cbc00 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f162a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 0x7f162c0b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f162a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 0x7f162c0b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48734/0 (socket says 172.21.15.71:48734) 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f162a800640 1 -- 172.21.15.71:0/169306948 learned_addr learned my addr 172.21.15.71:0/169306948 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f1629e00640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c072560 0x7f162c14ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f1629400640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c0cbc00 0x7f162c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f162a800640 1 -- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c0cbc00 msgr2=0x7f162c0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:19.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f162a800640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c0cbc00 0x7f162c0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f162a800640 1 -- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c072560 msgr2=0x7f162c14ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f162a800640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c072560 0x7f162c14ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.439+0000 7f162a800640 1 -- 172.21.15.71:0/169306948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c14c540 con 0x7f162c15bec0 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.440+0000 7f162a800640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 0x7f162c0b0df0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f162400c770 tx=0x7f162400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.440+0000 7f1622000640 1 -- 172.21.15.71:0/169306948 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624003b70 con 0x7f162c15bec0 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.440+0000 7f1622000640 1 -- 172.21.15.71:0/169306948 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1624003d10 con 0x7f162c15bec0 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.440+0000 7f1622000640 1 -- 172.21.15.71:0/169306948 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162400aab0 con 0x7f162c15bec0 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.440+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c14c820 con 0x7f162c15bec0 2024-09-16T07:38:19.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.440+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c14cde0 con 0x7f162c15bec0 2024-09-16T07:38:19.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.441+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c072040 con 0x7f162c15bec0 2024-09-16T07:38:19.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.442+0000 7f1622000640 1 -- 172.21.15.71:0/169306948 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f162400ac50 con 0x7f162c15bec0 2024-09-16T07:38:19.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.443+0000 7f1622000640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f160c07aec0 0x7f160c07d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:19.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.443+0000 7f1622000640 1 -- 172.21.15.71:0/169306948 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f16240999b0 con 0x7f162c15bec0 2024-09-16T07:38:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.444+0000 7f1629e00640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f160c07aec0 0x7f160c07d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:19.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.444+0000 7f1629e00640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f160c07aec0 0x7f160c07d380 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f161c033000 tx=0x7f161c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:19.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.447+0000 7f1622000640 1 -- 172.21.15.71:0/169306948 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f162c072040 con 0x7f162c15bec0 2024-09-16T07:38:19.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:19.674+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone9524", "target": ["mon-mgr", ""]}) -- 0x7f162c070e70 con 0x7f160c07aec0 2024-09-16T07:38:22.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.269+0000 7f1622000640 1 -- 172.21.15.71:0/169306948 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f162c070e70 con 0x7f160c07aec0 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:38:22.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:22.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:22.270 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:22.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f160c07aec0 msgr2=0x7f160c07d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f160c07aec0 0x7f160c07d380 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f161c033000 tx=0x7f161c004730 comp rx=0 tx=0).stop 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 msgr2=0x7f162c0b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 0x7f162c0b0df0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f162400c770 tx=0x7f162400cc30 comp rx=0 tx=0).stop 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 shutdown_connections 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f160c07aec0 0x7f160c07d380 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f162c15bec0 0x7f162c0b0df0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c0cbc00 0x7f162c0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 --2- 172.21.15.71:0/169306948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c072560 0x7f162c14ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 >> 172.21.15.71:0/169306948 conn(0x7f162c0cf9f0 msgr2=0x7f162c0ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 shutdown_connections 2024-09-16T07:38:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.272+0000 7f162b200640 1 -- 172.21.15.71:0/169306948 wait complete. 2024-09-16T07:38:22.292 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:38:22.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone13325 2024-09-16T07:38:22.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.460+0000 7f6352a00640 1 -- 172.21.15.71:0/3643585984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 msgr2=0x7f6330004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.460+0000 7f6352a00640 1 --2- 172.21.15.71:0/3643585984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 0x7f6330004c80 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f633800c470 tx=0x7f6330005830 comp rx=0 tx=0).stop 2024-09-16T07:38:22.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.460+0000 7f6350a00640 1 -- 172.21.15.71:0/3643585984 reap_dead start 2024-09-16T07:38:22.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.460+0000 7f6352a00640 1 -- 172.21.15.71:0/3643585984 shutdown_connections 2024-09-16T07:38:22.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.460+0000 7f6352a00640 1 -- 172.21.15.71:0/3643585984 >> 172.21.15.71:0/3643585984 conn(0x7f634c0b1840 msgr2=0x7f634c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:22.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.460+0000 7f6352a00640 1 -- 172.21.15.71:0/3643585984 shutdown_connections 2024-09-16T07:38:22.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.460+0000 7f6352a00640 1 -- 172.21.15.71:0/3643585984 wait complete. 2024-09-16T07:38:22.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.462+0000 7f6352a00640 1 Processor -- start 2024-09-16T07:38:22.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.462+0000 7f6352a00640 1 -- start start 2024-09-16T07:38:22.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f6352a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 0x7f634c161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:22.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f6352a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f634c15f9f0 0x7f634c13f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:22.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f6352a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634c15fe70 0x7f634c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:22.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f6352a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f634c163c10 con 0x7f6330004800 2024-09-16T07:38:22.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f6352a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f634c163a90 con 0x7f634c15f9f0 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f6352a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f634c163d90 con 0x7f634c15fe70 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f634be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 0x7f634c161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f634be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 0x7f634c161250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48762/0 (socket says 172.21.15.71:48762) 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f634be00640 1 -- 172.21.15.71:0/1237322857 learned_addr learned my addr 172.21.15.71:0/1237322857 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f6350a00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634c15fe70 0x7f634c13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.463+0000 7f634b400640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f634c15f9f0 0x7f634c13f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f634be00640 1 -- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634c15fe70 msgr2=0x7f634c13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f634be00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634c15fe70 0x7f634c13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f634be00640 1 -- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f634c15f9f0 msgr2=0x7f634c13f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f634be00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f634c15f9f0 0x7f634c13f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f634be00640 1 -- 172.21.15.71:0/1237322857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c160440 con 0x7f6330004800 2024-09-16T07:38:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f634be00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 0x7f634c161250 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f633c004d60 tx=0x7f633c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f6343e00640 1 -- 172.21.15.71:0/1237322857 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c0107d0 con 0x7f6330004800 2024-09-16T07:38:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f6343e00640 1 -- 172.21.15.71:0/1237322857 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f633c010970 con 0x7f6330004800 2024-09-16T07:38:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f6343e00640 1 -- 172.21.15.71:0/1237322857 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c010dc0 con 0x7f6330004800 2024-09-16T07:38:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c160720 con 0x7f6330004800 2024-09-16T07:38:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.464+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c13fbd0 con 0x7f6330004800 2024-09-16T07:38:22.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.465+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634c0b3fd0 con 0x7f6330004800 2024-09-16T07:38:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.467+0000 7f6343e00640 1 -- 172.21.15.71:0/1237322857 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f633c01c470 con 0x7f6330004800 2024-09-16T07:38:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.468+0000 7f6343e00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f632c07ae90 0x7f632c07d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.468+0000 7f6343e00640 1 -- 172.21.15.71:0/1237322857 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f633c09a700 con 0x7f6330004800 2024-09-16T07:38:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.469+0000 7f634b400640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f632c07ae90 0x7f632c07d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.469+0000 7f634b400640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f632c07ae90 0x7f632c07d350 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f634c0b5010 tx=0x7f6338001bc0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:22.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.472+0000 7f6343e00640 1 -- 172.21.15.71:0/1237322857 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f634c0b3fd0 con 0x7f6330004800 2024-09-16T07:38:22.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.700+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone13325", "target": ["mon-mgr", ""]}) -- 0x7f634c0b0660 con 0x7f632c07ae90 2024-09-16T07:38:22.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.711+0000 7f6343e00640 1 -- 172.21.15.71:0/1237322857 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f634c0b0660 con 0x7f632c07ae90 2024-09-16T07:38:22.711 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:22.711 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:38:22.711 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:38:22.711 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:38:22.711 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:22.712 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:22.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f632c07ae90 msgr2=0x7f632c07d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f632c07ae90 0x7f632c07d350 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f634c0b5010 tx=0x7f6338001bc0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 msgr2=0x7f634c161250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 0x7f634c161250 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f633c004d60 tx=0x7f633c001710 comp rx=0 tx=0).stop 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 shutdown_connections 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f632c07ae90 0x7f632c07d350 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634c15fe70 0x7f634c13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f634c15f9f0 0x7f634c13f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 --2- 172.21.15.71:0/1237322857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6330004800 0x7f634c161250 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 >> 172.21.15.71:0/1237322857 conn(0x7f634c0b1840 msgr2=0x7f634c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 shutdown_connections 2024-09-16T07:38:22.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.714+0000 7f6352a00640 1 -- 172.21.15.71:0/1237322857 wait complete. 2024-09-16T07:38:22.730 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:38:22.730 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone89816 2024-09-16T07:38:22.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.891+0000 7f49dae00640 1 -- 172.21.15.71:0/209415019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 msgr2=0x7f49d4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.891+0000 7f49dae00640 1 --2- 172.21.15.71:0/209415019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 0x7f49d4073440 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005fd0 tx=0x7f49c402a4a0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.891+0000 7f49dae00640 1 -- 172.21.15.71:0/209415019 shutdown_connections 2024-09-16T07:38:22.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.891+0000 7f49dae00640 1 --2- 172.21.15.71:0/209415019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d406efa0 0x7f49d40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.891+0000 7f49dae00640 1 --2- 172.21.15.71:0/209415019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 0x7f49d4073440 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.891+0000 7f49dae00640 1 -- 172.21.15.71:0/209415019 >> 172.21.15.71:0/209415019 conn(0x7f49d40b1840 msgr2=0x7f49d40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:22.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.891+0000 7f49dae00640 1 -- 172.21.15.71:0/209415019 shutdown_connections 2024-09-16T07:38:22.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.891+0000 7f49dae00640 1 -- 172.21.15.71:0/209415019 wait complete. 2024-09-16T07:38:22.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.893+0000 7f49dae00640 1 Processor -- start 2024-09-16T07:38:22.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.893+0000 7f49dae00640 1 -- start start 2024-09-16T07:38:22.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49dae00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d406efa0 0x7f49d41f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:22.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49dae00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d40ccb00 0x7f49d41e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49dae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 0x7f49d40b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49dae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40b26b0 con 0x7f49d4145750 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49dae00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40cedd0 con 0x7f49d40ccb00 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49dae00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49d4160100 con 0x7f49d406efa0 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 0x7f49d40b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 0x7f49d40b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48782/0 (socket says 172.21.15.71:48782) 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d8e00640 1 -- 172.21.15.71:0/747154201 learned_addr learned my addr 172.21.15.71:0/747154201 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d3400640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d40ccb00 0x7f49d41e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d3e00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d406efa0 0x7f49d41f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d8e00640 1 -- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d406efa0 msgr2=0x7f49d41f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d8e00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d406efa0 0x7f49d41f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d8e00640 1 -- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d40ccb00 msgr2=0x7f49d41e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:22.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d8e00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d40ccb00 0x7f49d41e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:22.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.894+0000 7f49d8e00640 1 -- 172.21.15.71:0/747154201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d41f8b60 con 0x7f49d4145750 2024-09-16T07:38:22.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.895+0000 7f49d8e00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 0x7f49d40b34e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c780 tx=0x7f49cc00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:22.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.895+0000 7f49cbe00640 1 -- 172.21.15.71:0/747154201 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc004b60 con 0x7f49d4145750 2024-09-16T07:38:22.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.895+0000 7f49cbe00640 1 -- 172.21.15.71:0/747154201 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49cc004e10 con 0x7f49d4145750 2024-09-16T07:38:22.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.895+0000 7f49cbe00640 1 -- 172.21.15.71:0/747154201 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc005290 con 0x7f49d4145750 2024-09-16T07:38:22.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.895+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d41eacf0 con 0x7f49d4145750 2024-09-16T07:38:22.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.895+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41eb2b0 con 0x7f49d4145750 2024-09-16T07:38:22.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.896+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d41605b0 con 0x7f49d4145750 2024-09-16T07:38:22.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.897+0000 7f49cbe00640 1 -- 172.21.15.71:0/747154201 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f49cc005430 con 0x7f49d4145750 2024-09-16T07:38:22.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.898+0000 7f49cbe00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49b407ae90 0x7f49b407d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:22.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.898+0000 7f49cbe00640 1 -- 172.21.15.71:0/747154201 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f49cc099c90 con 0x7f49d4145750 2024-09-16T07:38:22.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.899+0000 7f49d3e00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49b407ae90 0x7f49b407d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:22.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.900+0000 7f49d3e00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49b407ae90 0x7f49b407d350 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f49c4033000 tx=0x7f49c4004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:22.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:22.902+0000 7f49cbe00640 1 -- 172.21.15.71:0/747154201 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f49cc020070 con 0x7f49d4145750 2024-09-16T07:38:23.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:23.130+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone89816", "target": ["mon-mgr", ""]}) -- 0x7f49d40b1fa0 con 0x7f49b407ae90 2024-09-16T07:38:27.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.269+0000 7f49cbe00640 1 -- 172.21.15.71:0/747154201 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f49d40b1fa0 con 0x7f49b407ae90 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "status": { 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "canceled", 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "source": { 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "volume": "cephfs", 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "subvolume": "sv1", 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot": "ss1" 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "failure": { 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "errno": "4", 2024-09-16T07:38:27.269 INFO:teuthology.orchestra.run.smithi071.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T07:38:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.270 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49b407ae90 msgr2=0x7f49b407d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49b407ae90 0x7f49b407d350 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f49c4033000 tx=0x7f49c4004730 comp rx=0 tx=0).stop 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 msgr2=0x7f49d40b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 0x7f49d40b34e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00c780 tx=0x7f49cc00cb80 comp rx=0 tx=0).stop 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 shutdown_connections 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f49b407ae90 0x7f49b407d350 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4145750 0x7f49d40b34e0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d40ccb00 0x7f49d41e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 --2- 172.21.15.71:0/747154201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d406efa0 0x7f49d41f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 >> 172.21.15.71:0/747154201 conn(0x7f49d40b1840 msgr2=0x7f49d40af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:27.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 shutdown_connections 2024-09-16T07:38:27.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.272+0000 7f49dae00640 1 -- 172.21.15.71:0/747154201 wait complete. 2024-09-16T07:38:27.291 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T07:38:27.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T07:38:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.451+0000 7f4bf3c00640 1 -- 172.21.15.71:0/693613463 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06e6f0 msgr2=0x7f4bec06eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.451+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/693613463 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06e6f0 0x7f4bec06eb60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4be4005fd0 tx=0x7f4be402a5d0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.451+0000 7f4bf3c00640 1 -- 172.21.15.71:0/693613463 shutdown_connections 2024-09-16T07:38:27.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.451+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/693613463 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06e6f0 0x7f4bec06eb60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.451+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/693613463 >> v2:172.21.15.177:3300/0 conn(0x7f4bec158900 0x7f4bec15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.451+0000 7f4bf3c00640 1 -- 172.21.15.71:0/693613463 >> 172.21.15.71:0/693613463 conn(0x7f4bec0b18e0 msgr2=0x7f4bec0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:27.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.452+0000 7f4bf3c00640 1 -- 172.21.15.71:0/693613463 shutdown_connections 2024-09-16T07:38:27.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.452+0000 7f4bf3c00640 1 -- 172.21.15.71:0/693613463 wait complete. 2024-09-16T07:38:27.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.453+0000 7f4bf3c00640 1 Processor -- start 2024-09-16T07:38:27.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.453+0000 7f4bf3c00640 1 -- start start 2024-09-16T07:38:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bf3c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06e6f0 0x7f4bec0b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bf3c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bec158900 0x7f4bec1f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bf3c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec15b700 0x7f4bec1f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bf3c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec0ced60 con 0x7f4bec158900 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bf3c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec0cebe0 con 0x7f4bec15b700 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bf3c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4bec0ceee0 con 0x7f4bec06e6f0 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bec158900 0x7f4bec1f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bec158900 0x7f4bec1f8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38078/0 (socket says 172.21.15.71:38078) 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bebe00640 1 -- 172.21.15.71:0/930552850 learned_addr learned my addr 172.21.15.71:0/930552850 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.454+0000 7f4bf1c00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec15b700 0x7f4bec1f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.455+0000 7f4bf1200640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06e6f0 0x7f4bec0b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.455+0000 7f4bebe00640 1 -- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06e6f0 msgr2=0x7f4bec0b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.455+0000 7f4bebe00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06e6f0 0x7f4bec0b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.455+0000 7f4bebe00640 1 -- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec15b700 msgr2=0x7f4bec1f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.455+0000 7f4bebe00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec15b700 0x7f4bec1f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.455+0000 7f4bebe00640 1 -- 172.21.15.71:0/930552850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec0cbb70 con 0x7f4bec158900 2024-09-16T07:38:27.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.455+0000 7f4bebe00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bec158900 0x7f4bec1f8020 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c430 tx=0x7f4bd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:27.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.455+0000 7f4be8c00640 1 -- 172.21.15.71:0/930552850 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8017070 con 0x7f4bec158900 2024-09-16T07:38:27.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.456+0000 7f4be8c00640 1 -- 172.21.15.71:0/930552850 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bd80053c0 con 0x7f4bec158900 2024-09-16T07:38:27.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.456+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec154140 con 0x7f4bec158900 2024-09-16T07:38:27.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.456+0000 7f4be8c00640 1 -- 172.21.15.71:0/930552850 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8003850 con 0x7f4bec158900 2024-09-16T07:38:27.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.456+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec154700 con 0x7f4bec158900 2024-09-16T07:38:27.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.457+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec069360 con 0x7f4bec158900 2024-09-16T07:38:27.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.458+0000 7f4be8c00640 1 -- 172.21.15.71:0/930552850 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f4bd801a070 con 0x7f4bec158900 2024-09-16T07:38:27.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.459+0000 7f4be8c00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4bcc07ae90 0x7f4bcc07d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:27.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.459+0000 7f4be8c00640 1 -- 172.21.15.71:0/930552850 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f4bd8098f20 con 0x7f4bec158900 2024-09-16T07:38:27.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.459+0000 7f4bf1200640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4bcc07ae90 0x7f4bcc07d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:27.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.460+0000 7f4bf1200640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4bcc07ae90 0x7f4bcc07d350 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005fd0 tx=0x7f4bdc009500 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:27.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.463+0000 7f4be8c00640 1 -- 172.21.15.71:0/930552850 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4bd80629a0 con 0x7f4bec158900 2024-09-16T07:38:27.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.725+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4bec0cf440 con 0x7f4bec158900 2024-09-16T07:38:27.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.726+0000 7f4be8c00640 1 -- 172.21.15.71:0/930552850 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+4579 (secure 0 0 0) 0x7f4bec0cf440 con 0x7f4bec158900 2024-09-16T07:38:27.725 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:27.725 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:27.725 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "5821e025-6a94-4c05-a5f1-d6094eac5c68", 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: "health": { 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "HEALTH_OK", 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: "checks": {}, 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: "mutes": [] 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: "election_epoch": 4, 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum": [ 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: 0, 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: 1, 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: 2 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:38:27.726 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_names": [ 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "a", 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "b", 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "c" 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "quorum_age": 964, 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "monmap": { 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 1, 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "min_mon_release_name": "squid", 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "num_mons": 3 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap": { 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 148, 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2024-09-16T07:38:27.727 INFO:teuthology.orchestra.run.smithi071.stdout: "num_up_osds": 8, 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_up_since": 1726471357, 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "num_in_osds": 8, 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_in_since": 1726471347, 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "num_remapped_pgs": 0 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap": { 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "pgs_by_state": [ 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "state_name": "active+clean", 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "count": 97 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pgs": 97, 2024-09-16T07:38:27.728 INFO:teuthology.orchestra.run.smithi071.stdout: "num_pools": 3, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "num_objects": 4466, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "data_bytes": 18582984909, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 38317105152, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_avail": 734777008128, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_total": 773094113280, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "read_bytes_sec": 241804621, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "write_bytes_sec": 247780413, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "read_op_per_sec": 60, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "write_op_per_sec": 71 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "fsmap": { 2024-09-16T07:38:27.729 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 96, 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "btime": "2024-09-16T07:37:24:630475+0000", 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 10, 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1, 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "by_rank": [ 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "filesystem_id": 10, 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "rank": 0, 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "d", 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "status": "up:active", 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 9420 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.730 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "up:standby": 3 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "mgrmap": { 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "num_standbys": 1, 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "restful" 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "services": {} 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "servicemap": { 2024-09-16T07:38:27.731 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 25, 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2024-09-16T07:38:24.671752+0000", 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "services": { 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "daemons": { 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "summary": "", 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "y": { 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "daemons": { 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "summary": "", 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "c": { 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "daemons": { 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "summary": "", 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "0": { 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.734 INFO:teuthology.orchestra.run.smithi071.stdout: "2": { 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "5": { 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:38:27.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "6": { 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "7": { 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "start_epoch": 0, 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "start_stamp": "0.000000", 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 0, 2024-09-16T07:38:27.736 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": {}, 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: "task_status": {} 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout: "progress_events": {} 2024-09-16T07:38:27.737 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:27.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4bcc07ae90 msgr2=0x7f4bcc07d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4bcc07ae90 0x7f4bcc07d350 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005fd0 tx=0x7f4bdc009500 comp rx=0 tx=0).stop 2024-09-16T07:38:27.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bec158900 msgr2=0x7f4bec1f8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bec158900 0x7f4bec1f8020 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c430 tx=0x7f4bd800c8f0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 shutdown_connections 2024-09-16T07:38:27.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4bcc07ae90 0x7f4bcc07d350 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec15b700 0x7f4bec1f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bec158900 0x7f4bec1f8020 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 --2- 172.21.15.71:0/930552850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06e6f0 0x7f4bec0b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 >> 172.21.15.71:0/930552850 conn(0x7f4bec0b18e0 msgr2=0x7f4bec157630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:27.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 shutdown_connections 2024-09-16T07:38:27.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.728+0000 7f4bf3c00640 1 -- 172.21.15.71:0/930552850 wait complete. 2024-09-16T07:38:27.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:27.744 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone18140/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf 2024-09-16T07:38:27.744 DEBUG:teuthology.orchestra.run.smithi071:> ) 2024-09-16T07:38:27.782 INFO:teuthology.orchestra.run.smithi071.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone18140/d1a7233d-ce5c-48c1-ae3c-f525c68fbabf': No such file or directory 2024-09-16T07:38:27.783 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T07:38:27.784 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T07:38:27.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.983+0000 7f6375400640 1 -- 172.21.15.71:0/3967492667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 msgr2=0x7f6354003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.983+0000 7f6375400640 1 --2- 172.21.15.71:0/3967492667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 0x7f6354003e50 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6360009b50 tx=0x7f6354005830 comp rx=0 tx=0).stop 2024-09-16T07:38:27.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.983+0000 7f6374a00640 1 -- 172.21.15.71:0/3967492667 reap_dead start 2024-09-16T07:38:27.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.983+0000 7f6375400640 1 -- 172.21.15.71:0/3967492667 shutdown_connections 2024-09-16T07:38:27.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.983+0000 7f6375400640 1 -- 172.21.15.71:0/3967492667 >> 172.21.15.71:0/3967492667 conn(0x7f63700cfa20 msgr2=0x7f63700cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:27.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.984+0000 7f6375400640 1 -- 172.21.15.71:0/3967492667 shutdown_connections 2024-09-16T07:38:27.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.984+0000 7f6375400640 1 -- 172.21.15.71:0/3967492667 wait complete. 2024-09-16T07:38:27.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.985+0000 7f6375400640 1 Processor -- start 2024-09-16T07:38:27.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.985+0000 7f6375400640 1 -- start start 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f6375400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 0x7f63701f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f6375400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6370144590 0x7f6370147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f6375400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63700b4a10 0x7f63700b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f6375400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6370070300 con 0x7f6354003a30 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f6375400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6370070180 con 0x7f63700b4a10 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f6375400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63700adcd0 con 0x7f6370144590 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f636fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 0x7f63701f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f636fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 0x7f63701f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38088/0 (socket says 172.21.15.71:38088) 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f636fe00640 1 -- 172.21.15.71:0/4053810923 learned_addr learned my addr 172.21.15.71:0/4053810923 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:27.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.986+0000 7f636f400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6370144590 0x7f6370147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f6374a00640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63700b4a10 0x7f63700b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f636fe00640 1 -- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6370144590 msgr2=0x7f6370147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f636fe00640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6370144590 0x7f6370147ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f636fe00640 1 -- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63700b4a10 msgr2=0x7f63700b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f636fe00640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63700b4a10 0x7f63700b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f636fe00640 1 -- 172.21.15.71:0/4053810923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63700b4250 con 0x7f6354003a30 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f636fe00640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 0x7f63701f4900 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6360036040 tx=0x7f6360033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f6367e00640 1 -- 172.21.15.71:0/4053810923 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360043070 con 0x7f6354003a30 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f6367e00640 1 -- 172.21.15.71:0/4053810923 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6360004140 con 0x7f6354003a30 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f6367e00640 1 -- 172.21.15.71:0/4053810923 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360038040 con 0x7f6354003a30 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6370147310 con 0x7f6354003a30 2024-09-16T07:38:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.987+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63700b44f0 con 0x7f6354003a30 2024-09-16T07:38:27.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.989+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6370147830 con 0x7f6354003a30 2024-09-16T07:38:27.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.990+0000 7f6367e00640 1 -- 172.21.15.71:0/4053810923 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f6360046070 con 0x7f6354003a30 2024-09-16T07:38:27.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.991+0000 7f6367e00640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f635007ae90 0x7f635007d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:27.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.991+0000 7f6367e00640 1 -- 172.21.15.71:0/4053810923 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f63600b9ed0 con 0x7f6354003a30 2024-09-16T07:38:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.992+0000 7f636f400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f635007ae90 0x7f635007d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:27.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.992+0000 7f636f400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f635007ae90 0x7f635007d350 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f63700c20b0 tx=0x7f635c006b70 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:27.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:27.995+0000 7f6367e00640 1 -- 172.21.15.71:0/4053810923 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6370147830 con 0x7f6354003a30 2024-09-16T07:38:28.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.204+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f63700cd5f0 con 0x7f635007ae90 2024-09-16T07:38:28.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.206+0000 7f6367e00640 1 -- 172.21.15.71:0/4053810923 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f63700cd5f0 con 0x7f635007ae90 2024-09-16T07:38:28.206 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:38:28.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.206 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs" 2024-09-16T07:38:28.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:28.206 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f635007ae90 msgr2=0x7f635007d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f635007ae90 0x7f635007d350 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f63700c20b0 tx=0x7f635c006b70 comp rx=0 tx=0).stop 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 msgr2=0x7f63701f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 0x7f63701f4900 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6360036040 tx=0x7f6360033670 comp rx=0 tx=0).stop 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 shutdown_connections 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f635007ae90 0x7f635007d350 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63700b4a10 0x7f63700b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6370144590 0x7f6370147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 --2- 172.21.15.71:0/4053810923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6354003a30 0x7f63701f4900 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 >> 172.21.15.71:0/4053810923 conn(0x7f63700cfa20 msgr2=0x7f6370073c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 shutdown_connections 2024-09-16T07:38:28.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.209+0000 7f6375400640 1 -- 172.21.15.71:0/4053810923 wait complete. 2024-09-16T07:38:28.225 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T07:38:28.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.386+0000 7f9092c00640 1 -- 172.21.15.71:0/2621439441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0d19b0 msgr2=0x7f908c0d1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.386+0000 7f9092c00640 1 --2- 172.21.15.71:0/2621439441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0d19b0 0x7f908c0d1e60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9084005fd0 tx=0x7f908402aaa0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.386+0000 7f9092c00640 1 -- 172.21.15.71:0/2621439441 shutdown_connections 2024-09-16T07:38:28.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.386+0000 7f9092c00640 1 --2- 172.21.15.71:0/2621439441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0d19b0 0x7f908c0d1e60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.386+0000 7f9092c00640 1 --2- 172.21.15.71:0/2621439441 >> v2:172.21.15.177:3300/0 conn(0x7f908c0c91e0 0x7f908c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.386+0000 7f9092c00640 1 -- 172.21.15.71:0/2621439441 >> 172.21.15.71:0/2621439441 conn(0x7f908c0d3de0 msgr2=0x7f908c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:28.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.386+0000 7f9092c00640 1 -- 172.21.15.71:0/2621439441 shutdown_connections 2024-09-16T07:38:28.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.386+0000 7f9092c00640 1 -- 172.21.15.71:0/2621439441 wait complete. 2024-09-16T07:38:28.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.388+0000 7f9092c00640 1 Processor -- start 2024-09-16T07:38:28.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.388+0000 7f9092c00640 1 -- start start 2024-09-16T07:38:28.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9092c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0c91e0 0x7f908c1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9092c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0d19b0 0x7f908c0a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9092c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c15df60 0x7f908c09db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9092c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0d23a0 con 0x7f908c0d19b0 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9092c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0d15a0 con 0x7f908c15df60 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9092c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f908c0d2520 con 0x7f908c0c91e0 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9090e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0d19b0 0x7f908c0a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9090e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0d19b0 0x7f908c0a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38104/0 (socket says 172.21.15.71:38104) 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9090e00640 1 -- 172.21.15.71:0/4123946321 learned_addr learned my addr 172.21.15.71:0/4123946321 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9091800640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0c91e0 0x7f908c1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:28.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.389+0000 7f9092200640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c15df60 0x7f908c09db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:28.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9090e00640 1 -- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0c91e0 msgr2=0x7f908c1f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9090e00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0c91e0 0x7f908c1f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9090e00640 1 -- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c15df60 msgr2=0x7f908c09db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9090e00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c15df60 0x7f908c09db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9090e00640 1 -- 172.21.15.71:0/4123946321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c0a0ed0 con 0x7f908c0d19b0 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9090e00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0d19b0 0x7f908c0a0990 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f907800e9c0 tx=0x7f907800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9089600640 1 -- 172.21.15.71:0/4123946321 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907800cd80 con 0x7f908c0d19b0 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9089600640 1 -- 172.21.15.71:0/4123946321 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9078004020 con 0x7f908c0d19b0 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c09fbd0 con 0x7f908c0d19b0 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9089600640 1 -- 172.21.15.71:0/4123946321 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078004450 con 0x7f908c0d19b0 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.390+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c0a0190 con 0x7f908c0d19b0 2024-09-16T07:38:28.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.391+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c0a0670 con 0x7f908c0d19b0 2024-09-16T07:38:28.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.393+0000 7f9089600640 1 -- 172.21.15.71:0/4123946321 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f90780035f0 con 0x7f908c0d19b0 2024-09-16T07:38:28.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.393+0000 7f9089600640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07ae90 0x7f906c07d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:28.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.394+0000 7f9089600640 1 -- 172.21.15.71:0/4123946321 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f907809a500 con 0x7f908c0d19b0 2024-09-16T07:38:28.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.394+0000 7f9091800640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07ae90 0x7f906c07d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:28.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.395+0000 7f9091800640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07ae90 0x7f906c07d350 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9080001720 tx=0x7f9080009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:28.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.398+0000 7f9089600640 1 -- 172.21.15.71:0/4123946321 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f908c0a0670 con 0x7f908c0d19b0 2024-09-16T07:38:28.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.611+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f908c0d2a80 con 0x7f906c07ae90 2024-09-16T07:38:28.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.621+0000 7f9089600640 1 -- 172.21.15.71:0/4123946321 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+380 (secure 0 0 0) 0x7f908c0d2a80 con 0x7f906c07ae90 2024-09-16T07:38:28.621 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:28.621 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:38:28.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.621 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone33361" 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sv1" 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone87673" 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone39912" 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone13325" 2024-09-16T07:38:28.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone18140" 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone89816" 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subvol_clone9524" 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:28.623 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:38:28.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07ae90 msgr2=0x7f906c07d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07ae90 0x7f906c07d350 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9080001720 tx=0x7f9080009040 comp rx=0 tx=0).stop 2024-09-16T07:38:28.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0d19b0 msgr2=0x7f908c0a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0d19b0 0x7f908c0a0990 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f907800e9c0 tx=0x7f907800ee80 comp rx=0 tx=0).stop 2024-09-16T07:38:28.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 shutdown_connections 2024-09-16T07:38:28.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f906c07ae90 0x7f906c07d350 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f908c15df60 0x7f908c09db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908c0d19b0 0x7f908c0a0990 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 --2- 172.21.15.71:0/4123946321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f908c0c91e0 0x7f908c1f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 >> 172.21.15.71:0/4123946321 conn(0x7f908c0d3de0 msgr2=0x7f908c160650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:28.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 shutdown_connections 2024-09-16T07:38:28.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.624+0000 7f9092c00640 1 -- 172.21.15.71:0/4123946321 wait complete. 2024-09-16T07:38:28.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone33361 2024-09-16T07:38:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.804+0000 7fe2c7800640 1 -- 172.21.15.71:0/2503659559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a4004800 msgr2=0x7fe2a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.804+0000 7fe2c7800640 1 --2- 172.21.15.71:0/2503659559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a4004800 0x7fe2a4004c80 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c2e0 tx=0x7fe2a4005830 comp rx=0 tx=0).stop 2024-09-16T07:38:28.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.804+0000 7fe2c5800640 1 -- 172.21.15.71:0/2503659559 reap_dead start 2024-09-16T07:38:28.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.804+0000 7fe2c7800640 1 -- 172.21.15.71:0/2503659559 shutdown_connections 2024-09-16T07:38:28.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.804+0000 7fe2c7800640 1 -- 172.21.15.71:0/2503659559 >> 172.21.15.71:0/2503659559 conn(0x7fe2c00b18e0 msgr2=0x7fe2c00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:28.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.804+0000 7fe2c7800640 1 -- 172.21.15.71:0/2503659559 shutdown_connections 2024-09-16T07:38:28.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.805+0000 7fe2c7800640 1 -- 172.21.15.71:0/2503659559 wait complete. 2024-09-16T07:38:28.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.806+0000 7fe2c7800640 1 Processor -- start 2024-09-16T07:38:28.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.806+0000 7fe2c7800640 1 -- start start 2024-09-16T07:38:28.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.807+0000 7fe2c7800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2a4004800 0x7fe2c0156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:28.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.807+0000 7fe2c7800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2c0153d00 0x7fe2c01597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.807+0000 7fe2c7800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c0159cf0 0x7fe2c00727c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.807+0000 7fe2c7800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c015b340 con 0x7fe2c0159cf0 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.807+0000 7fe2c7800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c015b1c0 con 0x7fe2a4004800 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.807+0000 7fe2c7800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c015b4c0 con 0x7fe2c0153d00 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c0159cf0 0x7fe2c00727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c0159cf0 0x7fe2c00727c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38124/0 (socket says 172.21.15.71:38124) 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 -- 172.21.15.71:0/930314105 learned_addr learned my addr 172.21.15.71:0/930314105 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c4e00640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2a4004800 0x7fe2c0156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2bfe00640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2c0153d00 0x7fe2c01597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 -- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2c0153d00 msgr2=0x7fe2c01597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2c0153d00 0x7fe2c01597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 -- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2a4004800 msgr2=0x7fe2c0156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:28.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2a4004800 0x7fe2c0156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:28.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 -- 172.21.15.71:0/930314105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c00c92b0 con 0x7fe2c0159cf0 2024-09-16T07:38:28.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.808+0000 7fe2c5800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c0159cf0 0x7fe2c00727c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0076540 tx=0x7fe2b8011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:28.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.809+0000 7fe2bcc00640 1 -- 172.21.15.71:0/930314105 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8004b20 con 0x7fe2c0159cf0 2024-09-16T07:38:28.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.809+0000 7fe2bcc00640 1 -- 172.21.15.71:0/930314105 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2b8004cc0 con 0x7fe2c0159cf0 2024-09-16T07:38:28.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.809+0000 7fe2bcc00640 1 -- 172.21.15.71:0/930314105 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8003bd0 con 0x7fe2c0159cf0 2024-09-16T07:38:28.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.809+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c01f7e60 con 0x7fe2c0159cf0 2024-09-16T07:38:28.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.809+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c01f8420 con 0x7fe2c0159cf0 2024-09-16T07:38:28.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.810+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c0069360 con 0x7fe2c0159cf0 2024-09-16T07:38:28.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.811+0000 7fe2bcc00640 1 -- 172.21.15.71:0/930314105 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7fe2b8004030 con 0x7fe2c0159cf0 2024-09-16T07:38:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.812+0000 7fe2bcc00640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe29c07aec0 0x7fe29c07d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.812+0000 7fe2bcc00640 1 -- 172.21.15.71:0/930314105 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fe2b809a7e0 con 0x7fe2c0159cf0 2024-09-16T07:38:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.812+0000 7fe2c4e00640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe29c07aec0 0x7fe29c07d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.813+0000 7fe2c4e00640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe29c07aec0 0x7fe29c07d380 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009af0 tx=0x7fe2a8005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:28.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:28.816+0000 7fe2bcc00640 1 -- 172.21.15.71:0/930314105 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe2b8064950 con 0x7fe2c0159cf0 2024-09-16T07:38:29.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.025+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone33361", "target": ["mon-mgr", ""]}) -- 0x7fe2c006eb80 con 0x7fe29c07aec0 2024-09-16T07:38:29.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.044+0000 7fe2bcc00640 1 -- 172.21.15.71:0/930314105 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe2c006eb80 con 0x7fe29c07aec0 2024-09-16T07:38:29.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe29c07aec0 msgr2=0x7fe29c07d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe29c07aec0 0x7fe29c07d380 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009af0 tx=0x7fe2a8005f90 comp rx=0 tx=0).stop 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c0159cf0 msgr2=0x7fe2c00727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c0159cf0 0x7fe2c00727c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0076540 tx=0x7fe2b8011f80 comp rx=0 tx=0).stop 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 shutdown_connections 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe29c07aec0 0x7fe29c07d380 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c0159cf0 0x7fe2c00727c0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2c0153d00 0x7fe2c01597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.046+0000 7fe2c7800640 1 --2- 172.21.15.71:0/930314105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2a4004800 0x7fe2c0156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.047+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 >> 172.21.15.71:0/930314105 conn(0x7fe2c00b18e0 msgr2=0x7fe2c00b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.047+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 shutdown_connections 2024-09-16T07:38:29.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.047+0000 7fe2c7800640 1 -- 172.21.15.71:0/930314105 wait complete. 2024-09-16T07:38:29.048 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.049+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.049 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.049+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.049 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.049+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.049 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.049+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.049 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.049+0000 7fe76d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T07:38:29.066 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.066+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.066 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.066+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.066 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.066+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.066 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.066+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.066 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-16T07:38:29.066+0000 7fe76b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T07:38:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.235+0000 7f3068a00640 1 -- 172.21.15.71:0/3299525159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 msgr2=0x7f30640d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.235+0000 7f3068a00640 1 --2- 172.21.15.71:0/3299525159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 0x7f30640d4a20 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f30640d2c60 tx=0x7f305402a5f0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.235+0000 7f3068a00640 1 -- 172.21.15.71:0/3299525159 shutdown_connections 2024-09-16T07:38:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.235+0000 7f3068a00640 1 --2- 172.21.15.71:0/3299525159 >> v2:172.21.15.177:3300/0 conn(0x7f30640d0fe0 0x7f306415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.235+0000 7f3068a00640 1 --2- 172.21.15.71:0/3299525159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 0x7f30640d4a20 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.235+0000 7f3068a00640 1 -- 172.21.15.71:0/3299525159 >> 172.21.15.71:0/3299525159 conn(0x7f30640d3f00 msgr2=0x7f30640d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.235+0000 7f3068a00640 1 -- 172.21.15.71:0/3299525159 shutdown_connections 2024-09-16T07:38:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.235+0000 7f3068a00640 1 -- 172.21.15.71:0/3299525159 wait complete. 2024-09-16T07:38:29.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.236+0000 7f3068a00640 1 Processor -- start 2024-09-16T07:38:29.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.236+0000 7f3068a00640 1 -- start start 2024-09-16T07:38:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.237+0000 7f3068a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30640d0fe0 0x7f30640cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.237+0000 7f3068a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 0x7f30641f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.237+0000 7f3068a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f306415eaf0 0x7f3064160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.237+0000 7f3068a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f30640d1e60 con 0x7f306415e710 2024-09-16T07:38:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.237+0000 7f3068a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30640d1ce0 con 0x7f306415eaf0 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.237+0000 7f3068a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30640d1fe0 con 0x7f30640d0fe0 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 0x7f30641f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 0x7f30641f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38146/0 (socket says 172.21.15.71:38146) 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 -- 172.21.15.71:0/1267009493 learned_addr learned my addr 172.21.15.71:0/1267009493 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f3062800640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30640d0fe0 0x7f30640cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f3063200640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f306415eaf0 0x7f3064160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 -- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30640d0fe0 msgr2=0x7f30640cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30640d0fe0 0x7f30640cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 -- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f306415eaf0 msgr2=0x7f3064160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f306415eaf0 0x7f3064160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 -- 172.21.15.71:0/1267009493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30641f5900 con 0x7f306415e710 2024-09-16T07:38:29.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.238+0000 7f305be00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 0x7f30641f53c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f304c00c480 tx=0x7f304c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:29.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.239+0000 7f305aa00640 1 -- 172.21.15.71:0/1267009493 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c0052d0 con 0x7f306415e710 2024-09-16T07:38:29.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.239+0000 7f305aa00640 1 -- 172.21.15.71:0/1267009493 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f304c005470 con 0x7f306415e710 2024-09-16T07:38:29.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.239+0000 7f305aa00640 1 -- 172.21.15.71:0/1267009493 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c012870 con 0x7f306415e710 2024-09-16T07:38:29.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.239+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30640b7d30 con 0x7f306415e710 2024-09-16T07:38:29.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.239+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30640b82b0 con 0x7f306415e710 2024-09-16T07:38:29.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.240+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306415ef80 con 0x7f306415e710 2024-09-16T07:38:29.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.241+0000 7f305aa00640 1 -- 172.21.15.71:0/1267009493 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 54) ==== 104755+0+0 (secure 0 0 0) 0x7f304c0035f0 con 0x7f306415e710 2024-09-16T07:38:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.242+0000 7f305aa00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f304007ae90 0x7f304007d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.242+0000 7f305aa00640 1 -- 172.21.15.71:0/1267009493 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f304c0997d0 con 0x7f306415e710 2024-09-16T07:38:29.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.242+0000 7f3062800640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f304007ae90 0x7f304007d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:29.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.243+0000 7f3062800640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f304007ae90 0x7f304007d350 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3054033000 tx=0x7f3054004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:29.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.250+0000 7f305aa00640 1 -- 172.21.15.71:0/1267009493 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f30641f5900 con 0x7f306415e710 2024-09-16T07:38:29.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.470+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30641618f0 con 0x7f304007ae90 2024-09-16T07:38:29.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.485+0000 7f305aa00640 1 -- 172.21.15.71:0/1267009493 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3054004340 con 0x7f304007ae90 2024-09-16T07:38:29.483 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:29.484 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-16T07:38:29.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-16T07:38:29.484 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ss1" 2024-09-16T07:38:29.484 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-16T07:38:29.484 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-16T07:38:29.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.486+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f304007ae90 msgr2=0x7f304007d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.486+0000 7f3068a00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f304007ae90 0x7f304007d350 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3054033000 tx=0x7f3054004730 comp rx=0 tx=0).stop 2024-09-16T07:38:29.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.486+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 msgr2=0x7f30641f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.486+0000 7f3068a00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 0x7f30641f53c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f304c00c480 tx=0x7f304c00c940 comp rx=0 tx=0).stop 2024-09-16T07:38:29.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.487+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 shutdown_connections 2024-09-16T07:38:29.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.487+0000 7f3068a00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f304007ae90 0x7f304007d350 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.487+0000 7f3068a00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f306415eaf0 0x7f3064160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.487+0000 7f3068a00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f306415e710 0x7f30641f53c0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.487+0000 7f3068a00640 1 --2- 172.21.15.71:0/1267009493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30640d0fe0 0x7f30640cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.487+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 >> 172.21.15.71:0/1267009493 conn(0x7f30640d3f00 msgr2=0x7f3064159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:29.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.487+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 shutdown_connections 2024-09-16T07:38:29.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.487+0000 7f3068a00640 1 -- 172.21.15.71:0/1267009493 wait complete. 2024-09-16T07:38:29.503 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T07:38:29.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.670+0000 7f24cbe00640 1 -- 172.21.15.71:0/390545148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24ac003a50 msgr2=0x7f24ac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.670+0000 7f24cbe00640 1 --2- 172.21.15.71:0/390545148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24ac003a50 0x7f24ac003e70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f24b8005fd0 tx=0x7f24ac005830 comp rx=0 tx=0).stop 2024-09-16T07:38:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.670+0000 7f24cbe00640 1 -- 172.21.15.71:0/390545148 shutdown_connections 2024-09-16T07:38:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.670+0000 7f24cbe00640 1 --2- 172.21.15.71:0/390545148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24ac003a50 0x7f24ac003e70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.670+0000 7f24cbe00640 1 -- 172.21.15.71:0/390545148 >> 172.21.15.71:0/390545148 conn(0x7f24c40b1840 msgr2=0x7f24c40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.670+0000 7f24cbe00640 1 -- 172.21.15.71:0/390545148 shutdown_connections 2024-09-16T07:38:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.670+0000 7f24cbe00640 1 -- 172.21.15.71:0/390545148 wait complete. 2024-09-16T07:38:29.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.671+0000 7f24cbe00640 1 Processor -- start 2024-09-16T07:38:29.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.671+0000 7f24cbe00640 1 -- start start 2024-09-16T07:38:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.672+0000 7f24cbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24ac003a50 0x7f24c415c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.672+0000 7f24cbe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c413e690 0x7f24c41e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.672+0000 7f24cbe00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24c40b2c40 0x7f24c40ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.672+0000 7f24cbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f24c40737d0 con 0x7f24ac003a50 2024-09-16T07:38:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.672+0000 7f24cbe00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4073650 con 0x7f24c413e690 2024-09-16T07:38:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.672+0000 7f24cbe00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f24c40cc5c0 con 0x7f24c40b2c40 2024-09-16T07:38:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.672+0000 7f24c9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24ac003a50 0x7f24c415c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:29.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24ac003a50 0x7f24c415c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38162/0 (socket says 172.21.15.71:38162) 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9400640 1 -- 172.21.15.71:0/1879126129 learned_addr learned my addr 172.21.15.71:0/1879126129 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c8a00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c413e690 0x7f24c41e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9e00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24c40b2c40 0x7f24c40ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9400640 1 -- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24c40b2c40 msgr2=0x7f24c40ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9400640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24c40b2c40 0x7f24c40ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9400640 1 -- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c413e690 msgr2=0x7f24c41e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9400640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c413e690 0x7f24c41e6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9400640 1 -- 172.21.15.71:0/1879126129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c40b35b0 con 0x7f24ac003a50 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9e00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24c40b2c40 0x7f24c40ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c9400640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24ac003a50 0x7f24c415c500 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f24b802aea0 tx=0x7f24b8033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c1600640 1 -- 172.21.15.71:0/1879126129 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8038040 con 0x7f24ac003a50 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c1600640 1 -- 172.21.15.71:0/1879126129 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24b8041070 con 0x7f24ac003a50 2024-09-16T07:38:29.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c40b3890 con 0x7f24ac003a50 2024-09-16T07:38:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.673+0000 7f24c1600640 1 -- 172.21.15.71:0/1879126129 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8009200 con 0x7f24ac003a50 2024-09-16T07:38:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.674+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41e5f40 con 0x7f24ac003a50 2024-09-16T07:38:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.675+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c415f9d0 con 0x7f24ac003a50 2024-09-16T07:38:29.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.676+0000 7f24c1600640 1 -- 172.21.15.71:0/1879126129 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 55) ==== 104704+0+0 (secure 0 0 0) 0x7f24b803c670 con 0x7f24ac003a50 2024-09-16T07:38:29.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.677+0000 7f24c1600640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24a407ae30 0x7f24a407d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:29.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.677+0000 7f24c1600640 1 -- 172.21.15.71:0/1879126129 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f24b80ba850 con 0x7f24ac003a50 2024-09-16T07:38:29.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.677+0000 7f24c8a00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24a407ae30 0x7f24a407d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:29.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.678+0000 7f24c8a00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24a407ae30 0x7f24a407d2f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f24c4141540 tx=0x7f24b00067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:29.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.681+0000 7f24c1600640 1 -- 172.21.15.71:0/1879126129 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f24b8084920 con 0x7f24ac003a50 2024-09-16T07:38:29.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.900+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24c40aeb70 con 0x7f24a407ae30 2024-09-16T07:38:29.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.934+0000 7f24c1600640 1 -- 172.21.15.71:0/1879126129 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f24c40aeb70 con 0x7f24a407ae30 2024-09-16T07:38:29.934 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:29.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24a407ae30 msgr2=0x7f24a407d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24a407ae30 0x7f24a407d2f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f24c4141540 tx=0x7f24b00067a0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24ac003a50 msgr2=0x7f24c415c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:29.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24ac003a50 0x7f24c415c500 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f24b802aea0 tx=0x7f24b8033c90 comp rx=0 tx=0).stop 2024-09-16T07:38:29.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 shutdown_connections 2024-09-16T07:38:29.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f24a407ae30 0x7f24a407d2f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24c40b2c40 0x7f24c40ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c413e690 0x7f24c41e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 --2- 172.21.15.71:0/1879126129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24ac003a50 0x7f24c415c500 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:29.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 >> 172.21.15.71:0/1879126129 conn(0x7f24c40b1840 msgr2=0x7f24c406f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:29.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 shutdown_connections 2024-09-16T07:38:29.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:29.937+0000 7f24cbe00640 1 -- 172.21.15.71:0/1879126129 wait complete. 2024-09-16T07:38:29.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T07:38:30.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.113+0000 7f2e28a00640 1 -- 172.21.15.71:0/3174223497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e08003a50 msgr2=0x7f2e08003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.113+0000 7f2e28a00640 1 --2- 172.21.15.71:0/3174223497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e08003a50 0x7f2e08003e70 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009ed0 tx=0x7f2e08005830 comp rx=0 tx=0).stop 2024-09-16T07:38:30.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.113+0000 7f2e22800640 1 -- 172.21.15.71:0/3174223497 reap_dead start 2024-09-16T07:38:30.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.113+0000 7f2e28a00640 1 -- 172.21.15.71:0/3174223497 shutdown_connections 2024-09-16T07:38:30.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.113+0000 7f2e28a00640 1 -- 172.21.15.71:0/3174223497 >> 172.21.15.71:0/3174223497 conn(0x7f2e240b1840 msgr2=0x7f2e240b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:30.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.113+0000 7f2e28a00640 1 -- 172.21.15.71:0/3174223497 shutdown_connections 2024-09-16T07:38:30.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.113+0000 7f2e28a00640 1 -- 172.21.15.71:0/3174223497 wait complete. 2024-09-16T07:38:30.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.115+0000 7f2e28a00640 1 Processor -- start 2024-09-16T07:38:30.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.115+0000 7f2e28a00640 1 -- start start 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e28a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e08003a50 0x7f2e2415d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e28a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e2415d8e0 0x7f2e241424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e28a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e24142a30 0x7f2e240cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e28a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2e240b26b0 con 0x7f2e24142a30 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e28a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2e240cedd0 con 0x7f2e08003a50 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e28a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2e240cc5c0 con 0x7f2e2415d8e0 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e22800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e24142a30 0x7f2e240cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e22800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e24142a30 0x7f2e240cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38178/0 (socket says 172.21.15.71:38178) 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e22800640 1 -- 172.21.15.71:0/3079440989 learned_addr learned my addr 172.21.15.71:0/3079440989 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e21e00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e08003a50 0x7f2e2415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.116+0000 7f2e21400640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e2415d8e0 0x7f2e241424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e22800640 1 -- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e2415d8e0 msgr2=0x7f2e241424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e22800640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e2415d8e0 0x7f2e241424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e22800640 1 -- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e08003a50 msgr2=0x7f2e2415d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e22800640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e08003a50 0x7f2e2415d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e22800640 1 -- 172.21.15.71:0/3079440989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2415d690 con 0x7f2e24142a30 2024-09-16T07:38:30.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e22800640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e24142a30 0x7f2e240cfbd0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f2e2406dbc0 tx=0x7f2e1c00e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:30.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e1a000640 1 -- 172.21.15.71:0/3079440989 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c015020 con 0x7f2e24142a30 2024-09-16T07:38:30.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e1a000640 1 -- 172.21.15.71:0/3079440989 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e1c010980 con 0x7f2e24142a30 2024-09-16T07:38:30.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e24140d60 con 0x7f2e24142a30 2024-09-16T07:38:30.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e1a000640 1 -- 172.21.15.71:0/3079440989 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c010dd0 con 0x7f2e24142a30 2024-09-16T07:38:30.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.117+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e24141320 con 0x7f2e24142a30 2024-09-16T07:38:30.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.118+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e24073440 con 0x7f2e24142a30 2024-09-16T07:38:30.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.119+0000 7f2e1a000640 1 -- 172.21.15.71:0/3079440989 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 55) ==== 104704+0+0 (secure 0 0 0) 0x7f2e1c018020 con 0x7f2e24142a30 2024-09-16T07:38:30.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.120+0000 7f2e1a000640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2e0407ae00 0x7f2e0407d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.121+0000 7f2e1a000640 1 -- 172.21.15.71:0/3079440989 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f2e1c099d40 con 0x7f2e24142a30 2024-09-16T07:38:30.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.121+0000 7f2e21e00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2e0407ae00 0x7f2e0407d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.122+0000 7f2e21e00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2e0407ae00 0x7f2e0407d2c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2e14035000 tx=0x7f2e1402d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:30.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.124+0000 7f2e1a000640 1 -- 172.21.15.71:0/3079440989 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2e1c01f030 con 0x7f2e24142a30 2024-09-16T07:38:30.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.334+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2e240aeb60 con 0x7f2e0407ae00 2024-09-16T07:38:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.367+0000 7f2e1a000640 1 -- 172.21.15.71:0/3079440989 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e240aeb60 con 0x7f2e0407ae00 2024-09-16T07:38:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2e0407ae00 msgr2=0x7f2e0407d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2e0407ae00 0x7f2e0407d2c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2e14035000 tx=0x7f2e1402d040 comp rx=0 tx=0).stop 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e24142a30 msgr2=0x7f2e240cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e24142a30 0x7f2e240cfbd0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f2e2406dbc0 tx=0x7f2e1c00e9a0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 shutdown_connections 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2e0407ae00 0x7f2e0407d2c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e24142a30 0x7f2e240cfbd0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e2415d8e0 0x7f2e241424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 --2- 172.21.15.71:0/3079440989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e08003a50 0x7f2e2415d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 >> 172.21.15.71:0/3079440989 conn(0x7f2e240b1840 msgr2=0x7f2e240a9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 shutdown_connections 2024-09-16T07:38:30.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.370+0000 7f2e28a00640 1 -- 172.21.15.71:0/3079440989 wait complete. 2024-09-16T07:38:30.385 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone87673 2024-09-16T07:38:30.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.549+0000 7f5e99200640 1 -- 172.21.15.71:0/123921478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 msgr2=0x7f5e9406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.549+0000 7f5e99200640 1 --2- 172.21.15.71:0/123921478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 0x7f5e9406f6b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f5e94070850 tx=0x7f5e8c02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.549+0000 7f5e99200640 1 -- 172.21.15.71:0/123921478 shutdown_connections 2024-09-16T07:38:30.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.549+0000 7f5e99200640 1 --2- 172.21.15.71:0/123921478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 0x7f5e9406f6b0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.549+0000 7f5e99200640 1 --2- 172.21.15.71:0/123921478 >> v2:172.21.15.177:3300/0 conn(0x7f5e9415cff0 0x7f5e9406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.549+0000 7f5e99200640 1 -- 172.21.15.71:0/123921478 >> 172.21.15.71:0/123921478 conn(0x7f5e940b18e0 msgr2=0x7f5e940b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:30.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.549+0000 7f5e99200640 1 -- 172.21.15.71:0/123921478 shutdown_connections 2024-09-16T07:38:30.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.549+0000 7f5e99200640 1 -- 172.21.15.71:0/123921478 wait complete. 2024-09-16T07:38:30.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.551+0000 7f5e99200640 1 Processor -- start 2024-09-16T07:38:30.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.551+0000 7f5e99200640 1 -- start start 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e99200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 0x7f5e941527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e99200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e94159aa0 0x7f5e941585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e99200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e9415cff0 0x7f5e940ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e99200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5e940cc520 con 0x7f5e9406f220 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e99200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e940cc3a0 con 0x7f5e94159aa0 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e99200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e940cc6a0 con 0x7f5e9415cff0 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 0x7f5e941527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 0x7f5e941527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38196/0 (socket says 172.21.15.71:38196) 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91e00640 1 -- 172.21.15.71:0/2342849243 learned_addr learned my addr 172.21.15.71:0/2342849243 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91400640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e94159aa0 0x7f5e941585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e92800640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e9415cff0 0x7f5e940ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91e00640 1 -- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e9415cff0 msgr2=0x7f5e940ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91e00640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e9415cff0 0x7f5e940ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91e00640 1 -- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e94159aa0 msgr2=0x7f5e941585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91e00640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e94159aa0 0x7f5e941585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.552+0000 7f5e91e00640 1 -- 172.21.15.71:0/2342849243 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e94152d30 con 0x7f5e9406f220 2024-09-16T07:38:30.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.553+0000 7f5e91e00640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 0x7f5e941527f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c870 tx=0x7f5e8400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:30.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.553+0000 7f5e8a000640 1 -- 172.21.15.71:0/2342849243 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84003b70 con 0x7f5e9406f220 2024-09-16T07:38:30.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.553+0000 7f5e8a000640 1 -- 172.21.15.71:0/2342849243 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e84003d10 con 0x7f5e9406f220 2024-09-16T07:38:30.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.553+0000 7f5e8a000640 1 -- 172.21.15.71:0/2342849243 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400aab0 con 0x7f5e9406f220 2024-09-16T07:38:30.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.553+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e94157520 con 0x7f5e9406f220 2024-09-16T07:38:30.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.553+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e94157dc0 con 0x7f5e9406f220 2024-09-16T07:38:30.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.554+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e94069360 con 0x7f5e9406f220 2024-09-16T07:38:30.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.555+0000 7f5e8a000640 1 -- 172.21.15.71:0/2342849243 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 55) ==== 104704+0+0 (secure 0 0 0) 0x7f5e8400ac50 con 0x7f5e9406f220 2024-09-16T07:38:30.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.556+0000 7f5e8a000640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e7407ae00 0x7f5e7407d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.556+0000 7f5e8a000640 1 -- 172.21.15.71:0/2342849243 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f5e84099920 con 0x7f5e9406f220 2024-09-16T07:38:30.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.557+0000 7f5e91400640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e7407ae00 0x7f5e7407d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.557+0000 7f5e91400640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e7407ae00 0x7f5e7407d2c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5e941ebd80 tx=0x7f5e8000b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:30.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.561+0000 7f5e8a000640 1 -- 172.21.15.71:0/2342849243 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5e840633a0 con 0x7f5e9406f220 2024-09-16T07:38:30.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.777+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone87673", "target": ["mon-mgr", ""]}) -- 0x7f5e94070980 con 0x7f5e7407ae00 2024-09-16T07:38:30.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.797+0000 7f5e8a000640 1 -- 172.21.15.71:0/2342849243 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e94070980 con 0x7f5e7407ae00 2024-09-16T07:38:30.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e7407ae00 msgr2=0x7f5e7407d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e7407ae00 0x7f5e7407d2c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5e941ebd80 tx=0x7f5e8000b2e0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 msgr2=0x7f5e941527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 0x7f5e941527f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400c870 tx=0x7f5e8400cd30 comp rx=0 tx=0).stop 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 shutdown_connections 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f5e7407ae00 0x7f5e7407d2c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e9415cff0 0x7f5e940ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e94159aa0 0x7f5e941585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 --2- 172.21.15.71:0/2342849243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9406f220 0x7f5e941527f0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 >> 172.21.15.71:0/2342849243 conn(0x7f5e940b18e0 msgr2=0x7f5e9415e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 shutdown_connections 2024-09-16T07:38:30.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.799+0000 7f5e99200640 1 -- 172.21.15.71:0/2342849243 wait complete. 2024-09-16T07:38:30.814 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone39912 2024-09-16T07:38:30.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.976+0000 7f3007e00640 1 -- 172.21.15.71:0/192289077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30001565b0 msgr2=0x7f300006f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.976+0000 7f3007e00640 1 --2- 172.21.15.71:0/192289077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30001565b0 0x7f300006f810 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009830 tx=0x7f2fec02a480 comp rx=0 tx=0).stop 2024-09-16T07:38:30.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.976+0000 7f3007e00640 1 -- 172.21.15.71:0/192289077 shutdown_connections 2024-09-16T07:38:30.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.976+0000 7f3007e00640 1 --2- 172.21.15.71:0/192289077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30000ccb00 0x7f30000ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.976+0000 7f3007e00640 1 --2- 172.21.15.71:0/192289077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30001565b0 0x7f300006f810 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.976+0000 7f3007e00640 1 -- 172.21.15.71:0/192289077 >> 172.21.15.71:0/192289077 conn(0x7f30000b18e0 msgr2=0x7f30000b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:30.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.976+0000 7f3007e00640 1 -- 172.21.15.71:0/192289077 shutdown_connections 2024-09-16T07:38:30.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.976+0000 7f3007e00640 1 -- 172.21.15.71:0/192289077 wait complete. 2024-09-16T07:38:30.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.978+0000 7f3007e00640 1 Processor -- start 2024-09-16T07:38:30.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.978+0000 7f3007e00640 1 -- start start 2024-09-16T07:38:30.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3007e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ccb00 0x7f30000cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3007e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30001565b0 0x7f300015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3007e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300015f870 0x7f3000151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3007e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3000073620 con 0x7f300015f870 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3007e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30000734a0 con 0x7f30000ccb00 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3007e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30000737a0 con 0x7f30001565b0 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300015f870 0x7f3000151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300015f870 0x7f3000151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38206/0 (socket says 172.21.15.71:38206) 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005e00640 1 -- 172.21.15.71:0/484397648 learned_addr learned my addr 172.21.15.71:0/484397648 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3004a00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30001565b0 0x7f300015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005400640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ccb00 0x7f30000cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005e00640 1 -- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30001565b0 msgr2=0x7f300015def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30001565b0 0x7f300015def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005e00640 1 -- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ccb00 msgr2=0x7f30000cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ccb00 0x7f30000cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3005e00640 1 -- 172.21.15.71:0/484397648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30001f8340 con 0x7f300015f870 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.979+0000 7f3004a00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30001565b0 0x7f300015def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.980+0000 7f3005e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300015f870 0x7f3000151820 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800d780 tx=0x7f2ff800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.980+0000 7f2ffd600640 1 -- 172.21.15.71:0/484397648 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8017070 con 0x7f300015f870 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.980+0000 7f2ffd600640 1 -- 172.21.15.71:0/484397648 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ff800de90 con 0x7f300015f870 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.980+0000 7f2ffd600640 1 -- 172.21.15.71:0/484397648 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8005020 con 0x7f300015f870 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.980+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30001f8620 con 0x7f300015f870 2024-09-16T07:38:30.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.980+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30001f8be0 con 0x7f300015f870 2024-09-16T07:38:30.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.981+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3000069360 con 0x7f300015f870 2024-09-16T07:38:30.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.982+0000 7f2ffd600640 1 -- 172.21.15.71:0/484397648 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 55) ==== 104704+0+0 (secure 0 0 0) 0x7f2ff80051c0 con 0x7f300015f870 2024-09-16T07:38:30.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.983+0000 7f2ffd600640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fe007ae00 0x7f2fe007d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:30.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.983+0000 7f2ffd600640 1 -- 172.21.15.71:0/484397648 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f2ff8099dc0 con 0x7f300015f870 2024-09-16T07:38:30.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.984+0000 7f3005400640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fe007ae00 0x7f2fe007d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:30.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.985+0000 7f3005400640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fe007ae00 0x7f2fe007d2c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4004a30 tx=0x7f2ff4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:30.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:30.988+0000 7f2ffd600640 1 -- 172.21.15.71:0/484397648 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ff8063e90 con 0x7f300015f870 2024-09-16T07:38:31.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.199+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone39912", "target": ["mon-mgr", ""]}) -- 0x7f3000073cd0 con 0x7f2fe007ae00 2024-09-16T07:38:31.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.217+0000 7f2ffd600640 1 -- 172.21.15.71:0/484397648 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3000073cd0 con 0x7f2fe007ae00 2024-09-16T07:38:31.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fe007ae00 msgr2=0x7f2fe007d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fe007ae00 0x7f2fe007d2c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4004a30 tx=0x7f2ff4009040 comp rx=0 tx=0).stop 2024-09-16T07:38:31.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300015f870 msgr2=0x7f3000151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300015f870 0x7f3000151820 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800d780 tx=0x7f2ff800dc40 comp rx=0 tx=0).stop 2024-09-16T07:38:31.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 shutdown_connections 2024-09-16T07:38:31.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2fe007ae00 0x7f2fe007d2c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f300015f870 0x7f3000151820 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30001565b0 0x7f300015def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.219+0000 7f3007e00640 1 --2- 172.21.15.71:0/484397648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ccb00 0x7f30000cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.220+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 >> 172.21.15.71:0/484397648 conn(0x7f30000b18e0 msgr2=0x7f30000b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:31.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.220+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 shutdown_connections 2024-09-16T07:38:31.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.220+0000 7f3007e00640 1 -- 172.21.15.71:0/484397648 wait complete. 2024-09-16T07:38:31.236 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone13325 2024-09-16T07:38:31.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.397+0000 7f1e03c00640 1 -- 172.21.15.71:0/3063383752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 msgr2=0x7f1de0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.397+0000 7f1e03c00640 1 --2- 172.21.15.71:0/3063383752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 0x7f1de0004c80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1de80016c0 tx=0x7f1de0005830 comp rx=0 tx=0).stop 2024-09-16T07:38:31.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.397+0000 7f1e03c00640 1 -- 172.21.15.71:0/3063383752 shutdown_connections 2024-09-16T07:38:31.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.397+0000 7f1e03c00640 1 --2- 172.21.15.71:0/3063383752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 0x7f1de0004c80 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.397+0000 7f1e03c00640 1 -- 172.21.15.71:0/3063383752 >> 172.21.15.71:0/3063383752 conn(0x7f1dfc0b18e0 msgr2=0x7f1dfc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:31.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.397+0000 7f1e03c00640 1 -- 172.21.15.71:0/3063383752 shutdown_connections 2024-09-16T07:38:31.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.397+0000 7f1e03c00640 1 -- 172.21.15.71:0/3063383752 wait complete. 2024-09-16T07:38:31.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.399+0000 7f1e03c00640 1 Processor -- start 2024-09-16T07:38:31.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.399+0000 7f1e03c00640 1 -- start start 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e03c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 0x7f1dfc1f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e03c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dfc0b49a0 0x7f1dfc09d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e03c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dfc0cdad0 0x7f1dfc0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e03c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc074320 con 0x7f1de0004800 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e03c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc0741a0 con 0x7f1dfc0cdad0 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e03c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc0744a0 con 0x7f1dfc0b49a0 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e01200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 0x7f1dfc1f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e01200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 0x7f1dfc1f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38224/0 (socket says 172.21.15.71:38224) 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.400+0000 7f1e01200640 1 -- 172.21.15.71:0/4063355154 learned_addr learned my addr 172.21.15.71:0/4063355154 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e01c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dfc0cdad0 0x7f1dfc0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1dfbe00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dfc0b49a0 0x7f1dfc09d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:31.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e01200640 1 -- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dfc0b49a0 msgr2=0x7f1dfc09d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e01200640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dfc0b49a0 0x7f1dfc09d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e01200640 1 -- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dfc0cdad0 msgr2=0x7f1dfc0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e01200640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dfc0cdad0 0x7f1dfc0cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e01200640 1 -- 172.21.15.71:0/4063355154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc1f4af0 con 0x7f1de0004800 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e01c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dfc0cdad0 0x7f1dfc0cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1dfbe00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dfc0b49a0 0x7f1dfc09d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e01200640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 0x7f1dfc1f8830 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006660 tx=0x7f1dec0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1df8c00640 1 -- 172.21.15.71:0/4063355154 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec015070 con 0x7f1de0004800 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1df8c00640 1 -- 172.21.15.71:0/4063355154 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dec011900 con 0x7f1de0004800 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1df8c00640 1 -- 172.21.15.71:0/4063355154 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec011d50 con 0x7f1de0004800 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc1f4dd0 con 0x7f1de0004800 2024-09-16T07:38:31.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.401+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc1f5390 con 0x7f1de0004800 2024-09-16T07:38:31.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.402+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc069360 con 0x7f1de0004800 2024-09-16T07:38:31.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.403+0000 7f1df8c00640 1 -- 172.21.15.71:0/4063355154 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 55) ==== 104704+0+0 (secure 0 0 0) 0x7f1dec018070 con 0x7f1de0004800 2024-09-16T07:38:31.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.405+0000 7f1df8c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ddc07ae00 0x7f1ddc07d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:31.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.405+0000 7f1df8c00640 1 -- 172.21.15.71:0/4063355154 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7f1dec09a600 con 0x7f1de0004800 2024-09-16T07:38:31.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.405+0000 7f1dfbe00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ddc07ae00 0x7f1ddc07d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:31.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.406+0000 7f1dfbe00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ddc07ae00 0x7f1ddc07d2c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1de8032000 tx=0x7f1de8001190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:31.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.409+0000 7f1df8c00640 1 -- 172.21.15.71:0/4063355154 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1dec064650 con 0x7f1de0004800 2024-09-16T07:38:31.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.524+0000 7f1df8c00640 1 -- 172.21.15.71:0/4063355154 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f1dec0601c0 con 0x7f1de0004800 2024-09-16T07:38:31.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.626+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone13325", "target": ["mon-mgr", ""]}) -- 0x7f1dfc1f5700 con 0x7f1ddc07ae00 2024-09-16T07:38:31.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.645+0000 7f1df8c00640 1 -- 172.21.15.71:0/4063355154 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1dfc1f5700 con 0x7f1ddc07ae00 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ddc07ae00 msgr2=0x7f1ddc07d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ddc07ae00 0x7f1ddc07d2c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1de8032000 tx=0x7f1de8001190 comp rx=0 tx=0).stop 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 msgr2=0x7f1dfc1f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 0x7f1dfc1f8830 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006660 tx=0x7f1dec0039c0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 shutdown_connections 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f1ddc07ae00 0x7f1ddc07d2c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dfc0cdad0 0x7f1dfc0cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dfc0b49a0 0x7f1dfc09d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 --2- 172.21.15.71:0/4063355154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1de0004800 0x7f1dfc1f8830 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 >> 172.21.15.71:0/4063355154 conn(0x7f1dfc0b18e0 msgr2=0x7f1dfc159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 shutdown_connections 2024-09-16T07:38:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.647+0000 7f1e03c00640 1 -- 172.21.15.71:0/4063355154 wait complete. 2024-09-16T07:38:31.663 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone18140 2024-09-16T07:38:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.917+0000 7fc9c2c00640 1 -- 172.21.15.71:0/448703229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9bc06ee00 msgr2=0x7fc9bc06f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.917+0000 7fc9c2c00640 1 --2- 172.21.15.71:0/448703229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9bc06ee00 0x7fc9bc06f300 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8005fd0 tx=0x7fc9a802a680 comp rx=0 tx=0).stop 2024-09-16T07:38:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.918+0000 7fc9c2c00640 1 -- 172.21.15.71:0/448703229 shutdown_connections 2024-09-16T07:38:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.918+0000 7fc9c2c00640 1 --2- 172.21.15.71:0/448703229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9bc06ee00 0x7fc9bc06f300 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.918+0000 7fc9c2c00640 1 -- 172.21.15.71:0/448703229 >> 172.21.15.71:0/448703229 conn(0x7fc9bc0b18e0 msgr2=0x7fc9bc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:31.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.918+0000 7fc9c2c00640 1 -- 172.21.15.71:0/448703229 shutdown_connections 2024-09-16T07:38:31.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.918+0000 7fc9c2c00640 1 -- 172.21.15.71:0/448703229 wait complete. 2024-09-16T07:38:31.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.921+0000 7fc9c2c00640 1 Processor -- start 2024-09-16T07:38:31.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.922+0000 7fc9c2c00640 1 -- start start 2024-09-16T07:38:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.923+0000 7fc9c2c00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9bc06ee00 0x7fc9bc1f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.923+0000 7fc9c2c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9bc0b4770 0x7fc9bc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.923+0000 7fc9c2c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9bc15dfc0 0x7fc9bc15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.923+0000 7fc9c2c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc0ae1f0 con 0x7fc9bc0b4770 2024-09-16T07:38:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.923+0000 7fc9c2c00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc0ae070 con 0x7fc9bc06ee00 2024-09-16T07:38:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.923+0000 7fc9c2c00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9bc0ae370 con 0x7fc9bc15dfc0 2024-09-16T07:38:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.924+0000 7fc9bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9bc0b4770 0x7fc9bc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.924+0000 7fc9c0c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9bc15dfc0 0x7fc9bc15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.924+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9bc06ee00 0x7fc9bc1f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.924+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9bc06ee00 0x7fc9bc1f9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.71:37292/0 (socket says 172.21.15.71:37292) 2024-09-16T07:38:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.924+0000 7fc9bbe00640 1 -- 172.21.15.71:0/874640310 learned_addr learned my addr 172.21.15.71:0/874640310 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.924+0000 7fc9c0c00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9bc15dfc0 0x7fc9bc15d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:49188/0 (socket says 172.21.15.71:49188) 2024-09-16T07:38:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.925+0000 7fc9bbe00640 1 -- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9bc15dfc0 msgr2=0x7fc9bc15d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.925+0000 7fc9bbe00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9bc15dfc0 0x7fc9bc15d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.925+0000 7fc9bbe00640 1 -- 172.21.15.71:0/874640310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9bc0b4770 msgr2=0x7fc9bc0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.925+0000 7fc9bbe00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9bc0b4770 0x7fc9bc0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.925+0000 7fc9bbe00640 1 -- 172.21.15.71:0/874640310 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9bc15b500 con 0x7fc9bc06ee00 2024-09-16T07:38:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.926+0000 7fc9bbe00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9bc06ee00 0x7fc9bc1f9c10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00c400 tx=0x7fc9ac00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:31.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.927+0000 7fc9b3e00640 1 -- 172.21.15.71:0/874640310 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac017070 con 0x7fc9bc06ee00 2024-09-16T07:38:31.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.927+0000 7fc9b3e00640 1 -- 172.21.15.71:0/874640310 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9ac0053c0 con 0x7fc9bc06ee00 2024-09-16T07:38:31.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.927+0000 7fc9b3e00640 1 -- 172.21.15.71:0/874640310 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac003850 con 0x7fc9bc06ee00 2024-09-16T07:38:31.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.928+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9bc15b7e0 con 0x7fc9bc06ee00 2024-09-16T07:38:31.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.928+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9bc1fa150 con 0x7fc9bc06ee00 2024-09-16T07:38:31.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.930+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9bc06f630 con 0x7fc9bc06ee00 2024-09-16T07:38:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.932+0000 7fc9b3e00640 1 -- 172.21.15.71:0/874640310 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fc9ac01a070 con 0x7fc9bc06ee00 2024-09-16T07:38:31.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.934+0000 7fc9b3e00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc99c07af70 0x7fc99c07d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:31.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.934+0000 7fc9b3e00640 1 -- 172.21.15.71:0/874640310 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7fc9ac098e10 con 0x7fc9bc06ee00 2024-09-16T07:38:31.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.935+0000 7fc9bb400640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc99c07af70 0x7fc99c07d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:31.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.938+0000 7fc9bb400640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc99c07af70 0x7fc99c07d430 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8033000 tx=0x7fc9a8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:31.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:31.940+0000 7fc9b3e00640 1 -- 172.21.15.71:0/874640310 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc9bc06f630 con 0x7fc9bc06ee00 2024-09-16T07:38:32.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.200+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone18140", "target": ["mon-mgr", ""]}) -- 0x7fc9bc0b1490 con 0x7fc99c07af70 2024-09-16T07:38:32.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.216+0000 7fc9b3e00640 1 -- 172.21.15.71:0/874640310 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9bc0b1490 con 0x7fc99c07af70 2024-09-16T07:38:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc99c07af70 msgr2=0x7fc99c07d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc99c07af70 0x7fc99c07d430 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8033000 tx=0x7fc9a8004730 comp rx=0 tx=0).stop 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9bc06ee00 msgr2=0x7fc9bc1f9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9bc06ee00 0x7fc9bc1f9c10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00c400 tx=0x7fc9ac00c8c0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 shutdown_connections 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc99c07af70 0x7fc99c07d430 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9bc15dfc0 0x7fc9bc15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9bc0b4770 0x7fc9bc0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 --2- 172.21.15.71:0/874640310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9bc06ee00 0x7fc9bc1f9c10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 >> 172.21.15.71:0/874640310 conn(0x7fc9bc0b18e0 msgr2=0x7fc9bc0cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 shutdown_connections 2024-09-16T07:38:32.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.218+0000 7fc9c2c00640 1 -- 172.21.15.71:0/874640310 wait complete. 2024-09-16T07:38:32.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone89816 2024-09-16T07:38:32.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.396+0000 7ff6f5200640 1 -- 172.21.15.71:0/3061423687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 msgr2=0x7ff6f00d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.396+0000 7ff6f5200640 1 --2- 172.21.15.71:0/3061423687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 0x7ff6f00d4a20 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009830 tx=0x7ff6d802a5c0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.396+0000 7ff6f5200640 1 -- 172.21.15.71:0/3061423687 shutdown_connections 2024-09-16T07:38:32.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.396+0000 7ff6f5200640 1 --2- 172.21.15.71:0/3061423687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 0x7ff6f00d4a20 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.396+0000 7ff6f5200640 1 -- 172.21.15.71:0/3061423687 >> 172.21.15.71:0/3061423687 conn(0x7ff6f00d3f00 msgr2=0x7ff6f00d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:32.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.396+0000 7ff6f5200640 1 -- 172.21.15.71:0/3061423687 shutdown_connections 2024-09-16T07:38:32.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.396+0000 7ff6f5200640 1 -- 172.21.15.71:0/3061423687 wait complete. 2024-09-16T07:38:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.398+0000 7ff6f5200640 1 Processor -- start 2024-09-16T07:38:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.398+0000 7ff6f5200640 1 -- start start 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6f5200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f00d08d0 0x7ff6f00b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6f5200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f015e710 0x7ff6f00b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6f5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 0x7ff6f00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6f5200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f00d21e0 con 0x7ff6f015f200 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6f5200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f00d11a0 con 0x7ff6f015e710 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6f5200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f00d2360 con 0x7ff6f00d08d0 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 0x7ff6f00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 0x7ff6f00b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38252/0 (socket says 172.21.15.71:38252) 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6efe00640 1 -- 172.21.15.71:0/2318172118 learned_addr learned my addr 172.21.15.71:0/2318172118 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6e7e00640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f015e710 0x7ff6f00b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6ef400640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f00d08d0 0x7ff6f00b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6efe00640 1 -- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f00d08d0 msgr2=0x7ff6f00b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6efe00640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f00d08d0 0x7ff6f00b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6efe00640 1 -- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f015e710 msgr2=0x7ff6f00b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6efe00640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f015e710 0x7ff6f00b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6efe00640 1 -- 172.21.15.71:0/2318172118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f015a8f0 con 0x7ff6f015f200 2024-09-16T07:38:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.399+0000 7ff6ef400640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f00d08d0 0x7ff6f00b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.400+0000 7ff6efe00640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 0x7ff6f00b7110 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800c6b0 tx=0x7ff6e800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:32.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.400+0000 7ff6e7400640 1 -- 172.21.15.71:0/2318172118 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8003b70 con 0x7ff6f015f200 2024-09-16T07:38:32.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.400+0000 7ff6e7400640 1 -- 172.21.15.71:0/2318172118 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6e8003d30 con 0x7ff6f015f200 2024-09-16T07:38:32.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.400+0000 7ff6e7400640 1 -- 172.21.15.71:0/2318172118 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8005330 con 0x7ff6f015f200 2024-09-16T07:38:32.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.400+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f015abd0 con 0x7ff6f015f200 2024-09-16T07:38:32.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.400+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f015b190 con 0x7ff6f015f200 2024-09-16T07:38:32.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.401+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f00d34a0 con 0x7ff6f015f200 2024-09-16T07:38:32.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.402+0000 7ff6e7400640 1 -- 172.21.15.71:0/2318172118 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7ff6e80054d0 con 0x7ff6f015f200 2024-09-16T07:38:32.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.403+0000 7ff6e7400640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff6cc07af20 0x7ff6cc07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:32.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.403+0000 7ff6e7400640 1 -- 172.21.15.71:0/2318172118 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5851+0+0 (secure 0 0 0) 0x7ff6e8099930 con 0x7ff6f015f200 2024-09-16T07:38:32.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.404+0000 7ff6ef400640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff6cc07af20 0x7ff6cc07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:32.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.404+0000 7ff6ef400640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff6cc07af20 0x7ff6cc07d3e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0007a50 tx=0x7ff6e0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:32.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.407+0000 7ff6e7400640 1 -- 172.21.15.71:0/2318172118 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff6f00d34a0 con 0x7ff6f015f200 2024-09-16T07:38:32.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.623+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone89816", "target": ["mon-mgr", ""]}) -- 0x7ff6f00d2810 con 0x7ff6cc07af20 2024-09-16T07:38:32.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.649+0000 7ff6e7400640 1 -- 172.21.15.71:0/2318172118 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff6f00d2810 con 0x7ff6cc07af20 2024-09-16T07:38:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff6cc07af20 msgr2=0x7ff6cc07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff6cc07af20 0x7ff6cc07d3e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0007a50 tx=0x7ff6e0009040 comp rx=0 tx=0).stop 2024-09-16T07:38:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 msgr2=0x7ff6f00b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 0x7ff6f00b7110 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800c6b0 tx=0x7ff6e800cb70 comp rx=0 tx=0).stop 2024-09-16T07:38:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 shutdown_connections 2024-09-16T07:38:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff6cc07af20 0x7ff6cc07d3e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6f015f200 0x7ff6f00b7110 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f015e710 0x7ff6f00b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 --2- 172.21.15.71:0/2318172118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f00d08d0 0x7ff6f00b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 >> 172.21.15.71:0/2318172118 conn(0x7ff6f00d3f00 msgr2=0x7ff6f00b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:32.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 shutdown_connections 2024-09-16T07:38:32.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.652+0000 7ff6f5200640 1 -- 172.21.15.71:0/2318172118 wait complete. 2024-09-16T07:38:32.667 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone9524 2024-09-16T07:38:32.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.935+0000 7f2bdfa00640 1 -- 172.21.15.71:0/174065655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd8073de0 msgr2=0x7f2bd8074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.935+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/174065655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd8073de0 0x7f2bd8074240 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4005fd0 tx=0x7f2bc402a540 comp rx=0 tx=0).stop 2024-09-16T07:38:32.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.935+0000 7f2bdfa00640 1 -- 172.21.15.71:0/174065655 shutdown_connections 2024-09-16T07:38:32.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.935+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/174065655 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806e6f0 0x7f2bd806eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.935+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/174065655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd8073de0 0x7f2bd8074240 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.935+0000 7f2bdfa00640 1 -- 172.21.15.71:0/174065655 >> 172.21.15.71:0/174065655 conn(0x7f2bd80b18e0 msgr2=0x7f2bd80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:32.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.935+0000 7f2bdfa00640 1 -- 172.21.15.71:0/174065655 shutdown_connections 2024-09-16T07:38:32.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.935+0000 7f2bdfa00640 1 -- 172.21.15.71:0/174065655 wait complete. 2024-09-16T07:38:32.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.937+0000 7f2bdfa00640 1 Processor -- start 2024-09-16T07:38:32.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.937+0000 7f2bdfa00640 1 -- start start 2024-09-16T07:38:32.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdfa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806e6f0 0x7f2bd80b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:32.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdfa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd8073de0 0x7f2bd8142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:32.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdfa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd815f9a0 0x7f2bd80ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:32.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdfa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd806f0e0 con 0x7f2bd815f9a0 2024-09-16T07:38:32.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdfa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80cd5e0 con 0x7f2bd8073de0 2024-09-16T07:38:32.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdfa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd806f260 con 0x7f2bd806e6f0 2024-09-16T07:38:32.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd815f9a0 0x7f2bd80ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:32.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd8073de0 0x7f2bd8142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:32.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdda00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd815f9a0 0x7f2bd80ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38260/0 (socket says 172.21.15.71:38260) 2024-09-16T07:38:32.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.939+0000 7f2bdda00640 1 -- 172.21.15.71:0/3331472754 learned_addr learned my addr 172.21.15.71:0/3331472754 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:32.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.940+0000 7f2bdd000640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806e6f0 0x7f2bd80b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:32.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.940+0000 7f2bdda00640 1 -- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806e6f0 msgr2=0x7f2bd80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.940+0000 7f2bdda00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806e6f0 0x7f2bd80b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.940+0000 7f2bdda00640 1 -- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd8073de0 msgr2=0x7f2bd8142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:32.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.940+0000 7f2bdda00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd8073de0 0x7f2bd8142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:32.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.940+0000 7f2bdda00640 1 -- 172.21.15.71:0/3331472754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd81429b0 con 0x7f2bd815f9a0 2024-09-16T07:38:32.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.941+0000 7f2bdd000640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806e6f0 0x7f2bd80b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:32.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.941+0000 7f2bdda00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd815f9a0 0x7f2bd80ae2e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000eb10 tx=0x7f2bd000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:32.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.942+0000 7f2bd4c00640 1 -- 172.21.15.71:0/3331472754 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0004b20 con 0x7f2bd815f9a0 2024-09-16T07:38:32.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.942+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd8142c80 con 0x7f2bd815f9a0 2024-09-16T07:38:32.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.942+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd8143240 con 0x7f2bd815f9a0 2024-09-16T07:38:32.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.943+0000 7f2bd4c00640 1 -- 172.21.15.71:0/3331472754 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bd0004ce0 con 0x7f2bd815f9a0 2024-09-16T07:38:32.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.943+0000 7f2bd4c00640 1 -- 172.21.15.71:0/3331472754 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0005280 con 0x7f2bd815f9a0 2024-09-16T07:38:32.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.944+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb4005290 con 0x7f2bd815f9a0 2024-09-16T07:38:32.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.949+0000 7f2bd4c00640 1 -- 172.21.15.71:0/3331472754 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f2bd0005420 con 0x7f2bd815f9a0 2024-09-16T07:38:32.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.950+0000 7f2bd4c00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb807af20 0x7f2bb807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:32.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.951+0000 7f2bd4c00640 1 -- 172.21.15.71:0/3331472754 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7f2bd009ad10 con 0x7f2bd815f9a0 2024-09-16T07:38:32.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.951+0000 7f2bdd000640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb807af20 0x7f2bb807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:32.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.953+0000 7f2bdd000640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb807af20 0x7f2bb807d3e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc007fd0 tx=0x7f2bcc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:32.954+0000 7f2bd4c00640 1 -- 172.21.15.71:0/3331472754 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2bd0064e60 con 0x7f2bd815f9a0 2024-09-16T07:38:33.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.364+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone9524", "target": ["mon-mgr", ""]}) -- 0x7f2bb4000bc0 con 0x7f2bb807af20 2024-09-16T07:38:33.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.393+0000 7f2bd4c00640 1 -- 172.21.15.71:0/3331472754 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2bb4000bc0 con 0x7f2bb807af20 2024-09-16T07:38:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.397+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb807af20 msgr2=0x7f2bb807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.397+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb807af20 0x7f2bb807d3e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc007fd0 tx=0x7f2bcc009040 comp rx=0 tx=0).stop 2024-09-16T07:38:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.397+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd815f9a0 msgr2=0x7f2bd80ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.397+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd815f9a0 0x7f2bd80ae2e0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000eb10 tx=0x7f2bd000c2d0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.398+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 shutdown_connections 2024-09-16T07:38:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.398+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f2bb807af20 0x7f2bb807d3e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.398+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd815f9a0 0x7f2bd80ae2e0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.398+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd8073de0 0x7f2bd8142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.398+0000 7f2bdfa00640 1 --2- 172.21.15.71:0/3331472754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806e6f0 0x7f2bd80b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.398+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 >> 172.21.15.71:0/3331472754 conn(0x7f2bd80b18e0 msgr2=0x7f2bd80b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.398+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 shutdown_connections 2024-09-16T07:38:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.398+0000 7f2bdfa00640 1 -- 172.21.15.71:0/3331472754 wait complete. 2024-09-16T07:38:33.415 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T07:38:33.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T07:38:33.500 INFO:teuthology.orchestra.run.smithi071.stdout:2 2024-09-16T07:38:33.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T07:38:33.509 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T07:38:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.675+0000 7fe140600640 1 -- 172.21.15.71:0/2786795486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe13806efa0 msgr2=0x7fe1380cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.675+0000 7fe140600640 1 --2- 172.21.15.71:0/2786795486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe13806efa0 0x7fe1380cc5c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fe124009830 tx=0x7fe12402a480 comp rx=0 tx=0).stop 2024-09-16T07:38:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.676+0000 7fe140600640 1 -- 172.21.15.71:0/2786795486 shutdown_connections 2024-09-16T07:38:33.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.676+0000 7fe140600640 1 --2- 172.21.15.71:0/2786795486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe13806efa0 0x7fe1380cc5c0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.676+0000 7fe140600640 1 --2- 172.21.15.71:0/2786795486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe138145750 0x7fe138073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.676+0000 7fe140600640 1 -- 172.21.15.71:0/2786795486 >> 172.21.15.71:0/2786795486 conn(0x7fe1380b1840 msgr2=0x7fe1380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:33.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.676+0000 7fe140600640 1 -- 172.21.15.71:0/2786795486 shutdown_connections 2024-09-16T07:38:33.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.676+0000 7fe140600640 1 -- 172.21.15.71:0/2786795486 wait complete. 2024-09-16T07:38:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.677+0000 7fe140600640 1 Processor -- start 2024-09-16T07:38:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.677+0000 7fe140600640 1 -- start start 2024-09-16T07:38:33.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.678+0000 7fe140600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe13806efa0 0x7fe1380afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.678+0000 7fe140600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1380ccb00 0x7fe1381e9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.678+0000 7fe140600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe138145750 0x7fe1381e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.678+0000 7fe140600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe1380b26b0 con 0x7fe138145750 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.678+0000 7fe140600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1380cedd0 con 0x7fe13806efa0 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.678+0000 7fe140600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe138160100 con 0x7fe1380ccb00 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe138145750 0x7fe1381e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13d200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1380ccb00 0x7fe1381e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13dc00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe13806efa0 0x7fe1380afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13d200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1380ccb00 0x7fe1381e9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.71:49226/0 (socket says 172.21.15.71:49226) 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13d200640 1 -- 172.21.15.71:0/1966486548 learned_addr learned my addr 172.21.15.71:0/1966486548 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13e600640 1 -- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1380ccb00 msgr2=0x7fe1381e9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:33.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13e600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1380ccb00 0x7fe1381e9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13e600640 1 -- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe13806efa0 msgr2=0x7fe1380afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13e600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe13806efa0 0x7fe1380afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.679+0000 7fe13e600640 1 -- 172.21.15.71:0/1966486548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13814e660 con 0x7fe138145750 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.680+0000 7fe13e600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe138145750 0x7fe1381e9cd0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe13000cad0 tx=0x7fe13000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.680+0000 7fe135600640 1 -- 172.21.15.71:0/1966486548 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130016020 con 0x7fe138145750 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.680+0000 7fe135600640 1 -- 172.21.15.71:0/1966486548 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe13000acc0 con 0x7fe138145750 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.680+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1381e54d0 con 0x7fe138145750 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.680+0000 7fe135600640 1 -- 172.21.15.71:0/1966486548 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130012680 con 0x7fe138145750 2024-09-16T07:38:33.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.680+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1381e5a90 con 0x7fe138145750 2024-09-16T07:38:33.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.681+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1381605b0 con 0x7fe138145750 2024-09-16T07:38:33.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.683+0000 7fe135600640 1 -- 172.21.15.71:0/1966486548 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fe130019020 con 0x7fe138145750 2024-09-16T07:38:33.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.684+0000 7fe135600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe11807af20 0x7fe11807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:33.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.685+0000 7fe135600640 1 -- 172.21.15.71:0/1966486548 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7fe13009a0a0 con 0x7fe138145750 2024-09-16T07:38:33.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.685+0000 7fe13dc00640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe11807af20 0x7fe11807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:33.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.687+0000 7fe13dc00640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe11807af20 0x7fe11807d3e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe12c005fd0 tx=0x7fe12c00b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:33.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:33.692+0000 7fe135600640 1 -- 172.21.15.71:0/1966486548 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe130020020 con 0x7fe138145750 2024-09-16T07:38:34.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.057+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fe13815f860 con 0x7fe138145750 2024-09-16T07:38:34.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.057+0000 7fe135600640 1 -- 172.21.15.71:0/1966486548 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7fe130063b20 con 0x7fe138145750 2024-09-16T07:38:34.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe11807af20 msgr2=0x7fe11807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe11807af20 0x7fe11807d3e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe12c005fd0 tx=0x7fe12c00b2e0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe138145750 msgr2=0x7fe1381e9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe138145750 0x7fe1381e9cd0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe13000cad0 tx=0x7fe13000cf90 comp rx=0 tx=0).stop 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 shutdown_connections 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe11807af20 0x7fe11807d3e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe138145750 0x7fe1381e9cd0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1380ccb00 0x7fe1381e9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 --2- 172.21.15.71:0/1966486548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe13806efa0 0x7fe1380afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 >> 172.21.15.71:0/1966486548 conn(0x7fe1380b1840 msgr2=0x7fe1380b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:34.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 shutdown_connections 2024-09-16T07:38:34.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.060+0000 7fe140600640 1 -- 172.21.15.71:0/1966486548 wait complete. 2024-09-16T07:38:34.077 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T07:38:34.077 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:38:34.163 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T07:38:34.260 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T07:38:34.260 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:38:34.260 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:38:34.260 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:34.295 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:38:34.295 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:34.371 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T07:38:34.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.569+0000 7f842d400640 1 -- 172.21.15.71:0/2897295147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842815f5a0 msgr2=0x7f84280b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:34.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.569+0000 7f842d400640 1 --2- 172.21.15.71:0/2897295147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842815f5a0 0x7f84280b2440 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f8418009830 tx=0x7f841802a530 comp rx=0 tx=0).stop 2024-09-16T07:38:34.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.569+0000 7f842d400640 1 -- 172.21.15.71:0/2897295147 shutdown_connections 2024-09-16T07:38:34.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.569+0000 7f842d400640 1 --2- 172.21.15.71:0/2897295147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842815f5a0 0x7f84280b2440 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.569+0000 7f842d400640 1 -- 172.21.15.71:0/2897295147 >> 172.21.15.71:0/2897295147 conn(0x7f84280b1950 msgr2=0x7f84280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:34.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.569+0000 7f842d400640 1 -- 172.21.15.71:0/2897295147 shutdown_connections 2024-09-16T07:38:34.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.569+0000 7f842d400640 1 -- 172.21.15.71:0/2897295147 wait complete. 2024-09-16T07:38:34.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.571+0000 7f842d400640 1 Processor -- start 2024-09-16T07:38:34.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.571+0000 7f842d400640 1 -- start start 2024-09-16T07:38:34.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f842d400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8428073fb0 0x7f84280c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:34.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f842d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84280ccf00 0x7f84281f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:34.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f842d400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f842815f5a0 0x7f84280c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:34.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f842d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f842806f9f0 con 0x7f84280ccf00 2024-09-16T07:38:34.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f842d400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f842806f870 con 0x7f8428073fb0 2024-09-16T07:38:34.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f842d400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8428074420 con 0x7f842815f5a0 2024-09-16T07:38:34.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f8425e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84280ccf00 0x7f84281f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f8425e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84280ccf00 0x7f84281f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38290/0 (socket says 172.21.15.71:38290) 2024-09-16T07:38:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f8425e00640 1 -- 172.21.15.71:0/3289580977 learned_addr learned my addr 172.21.15.71:0/3289580977 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f8426800640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8428073fb0 0x7f84280c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f8427200640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f842815f5a0 0x7f84280c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f8425e00640 1 -- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f842815f5a0 msgr2=0x7f84280c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f8425e00640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f842815f5a0 0x7f84280c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.572+0000 7f8425e00640 1 -- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8428073fb0 msgr2=0x7f84280c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.573+0000 7f8425e00640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8428073fb0 0x7f84280c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.573+0000 7f8425e00640 1 -- 172.21.15.71:0/3289580977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84281f2eb0 con 0x7f84280ccf00 2024-09-16T07:38:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.573+0000 7f8425e00640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84280ccf00 0x7f84281f2610 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f841400c500 tx=0x7f841400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.573+0000 7f841ea00640 1 -- 172.21.15.71:0/3289580977 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84140052d0 con 0x7f84280ccf00 2024-09-16T07:38:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.573+0000 7f841ea00640 1 -- 172.21.15.71:0/3289580977 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8414005470 con 0x7f84280ccf00 2024-09-16T07:38:34.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.573+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84281f3190 con 0x7f84280ccf00 2024-09-16T07:38:34.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.573+0000 7f841ea00640 1 -- 172.21.15.71:0/3289580977 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414012870 con 0x7f84280ccf00 2024-09-16T07:38:34.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.573+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84280c4bc0 con 0x7f84280ccf00 2024-09-16T07:38:34.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.574+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84280c9d90 con 0x7f84280ccf00 2024-09-16T07:38:34.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.575+0000 7f841ea00640 1 -- 172.21.15.71:0/3289580977 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f84140035f0 con 0x7f84280ccf00 2024-09-16T07:38:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.576+0000 7f841ea00640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f840807af20 0x7f840807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.577+0000 7f841ea00640 1 -- 172.21.15.71:0/3289580977 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7f84140996c0 con 0x7f84280ccf00 2024-09-16T07:38:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.577+0000 7f8426800640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f840807af20 0x7f840807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.577+0000 7f8426800640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f840807af20 0x7f840807d3e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8418034000 tx=0x7f8418004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:34.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.580+0000 7f841ea00640 1 -- 172.21.15.71:0/3289580977 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8414063160 con 0x7f84280ccf00 2024-09-16T07:38:34.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.870+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f84280b2440 con 0x7f84280ccf00 2024-09-16T07:38:34.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.871+0000 7f841ea00640 1 -- 172.21.15.71:0/3289580977 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) ==== 80+0+0 (secure 0 0 0) 0x7f8414067b90 con 0x7f84280ccf00 2024-09-16T07:38:34.871 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-09-16T07:38:34.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f840807af20 msgr2=0x7f840807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:34.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f840807af20 0x7f840807d3e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8418034000 tx=0x7f8418004730 comp rx=0 tx=0).stop 2024-09-16T07:38:34.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84280ccf00 msgr2=0x7f84281f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84280ccf00 0x7f84281f2610 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f841400c500 tx=0x7f841400c9c0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 shutdown_connections 2024-09-16T07:38:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f840807af20 0x7f840807d3e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f842815f5a0 0x7f84280c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84280ccf00 0x7f84281f2610 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 --2- 172.21.15.71:0/3289580977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8428073fb0 0x7f84280c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:34.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 >> 172.21.15.71:0/3289580977 conn(0x7f84280b1950 msgr2=0x7f8428075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:34.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.875+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 shutdown_connections 2024-09-16T07:38:34.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:34.876+0000 7f842d400640 1 -- 172.21.15.71:0/3289580977 wait complete. 2024-09-16T07:38:34.903 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T07:38:34.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:34.903 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T07:38:35.039 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:38:35.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:35.039 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T07:38:35.101 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:38:35.101 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:38:35.101 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:35.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:38:35.161 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:38:35.161 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:38:35.161 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:35.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:38:35.216 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T07:38:35.216 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:38:35.216 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:35.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:38:35.252 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:38:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.409+0000 7facf5400640 1 -- 172.21.15.71:0/3438609112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf0073fb0 msgr2=0x7facf0074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.409+0000 7facf5400640 1 --2- 172.21.15.71:0/3438609112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf0073fb0 0x7facf0074460 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7face8009830 tx=0x7face802a480 comp rx=0 tx=0).stop 2024-09-16T07:38:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.409+0000 7facf5400640 1 -- 172.21.15.71:0/3438609112 shutdown_connections 2024-09-16T07:38:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.409+0000 7facf5400640 1 --2- 172.21.15.71:0/3438609112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf0073fb0 0x7facf0074460 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.409+0000 7facf5400640 1 --2- 172.21.15.71:0/3438609112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facf015f5a0 0x7facf00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.409+0000 7facf5400640 1 -- 172.21.15.71:0/3438609112 >> 172.21.15.71:0/3438609112 conn(0x7facf00b1950 msgr2=0x7facf00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.409+0000 7facf5400640 1 -- 172.21.15.71:0/3438609112 shutdown_connections 2024-09-16T07:38:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.409+0000 7facf5400640 1 -- 172.21.15.71:0/3438609112 wait complete. 2024-09-16T07:38:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.411+0000 7facf5400640 1 Processor -- start 2024-09-16T07:38:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.411+0000 7facf5400640 1 -- start start 2024-09-16T07:38:35.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facf5400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facf0073fb0 0x7facf014cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facf5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00ccf00 0x7facf00cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facf5400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf015f5a0 0x7facf009c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facf5400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7facf006f2e0 con 0x7facf00ccf00 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facf5400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7facf006f160 con 0x7facf015f5a0 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facf5400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7facf006f460 con 0x7facf0073fb0 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facede00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00ccf00 0x7facf00cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facede00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00ccf00 0x7facf00cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58770/0 (socket says 172.21.15.71:58770) 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facede00640 1 -- 172.21.15.71:0/3573151992 learned_addr learned my addr 172.21.15.71:0/3573151992 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facef200640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf015f5a0 0x7facf009c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facee800640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facf0073fb0 0x7facf014cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facede00640 1 -- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facf0073fb0 msgr2=0x7facf014cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.412+0000 7facede00640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facf0073fb0 0x7facf014cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7facede00640 1 -- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf015f5a0 msgr2=0x7facf009c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7facede00640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf015f5a0 0x7facf009c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7facede00640 1 -- 172.21.15.71:0/3573151992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf00ce020 con 0x7facf00ccf00 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7facede00640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00ccf00 0x7facf00cdae0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7facdc00c460 tx=0x7facdc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7face6a00640 1 -- 172.21.15.71:0/3573151992 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc016020 con 0x7facf00ccf00 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7face6a00640 1 -- 172.21.15.71:0/3573151992 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facdc012920 con 0x7facf00ccf00 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7face6a00640 1 -- 172.21.15.71:0/3573151992 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc012d50 con 0x7facf00ccf00 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf00ce2a0 con 0x7facf00ccf00 2024-09-16T07:38:35.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.413+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf00ce860 con 0x7facf00ccf00 2024-09-16T07:38:35.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.414+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf006f9f0 con 0x7facf00ccf00 2024-09-16T07:38:35.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.415+0000 7face6a00640 1 -- 172.21.15.71:0/3573151992 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7facdc019020 con 0x7facf00ccf00 2024-09-16T07:38:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.417+0000 7face6a00640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007af20 0x7facd007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.417+0000 7face6a00640 1 -- 172.21.15.71:0/3573151992 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7facdc099800 con 0x7facf00ccf00 2024-09-16T07:38:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.417+0000 7facee800640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007af20 0x7facd007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.418+0000 7facee800640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007af20 0x7facd007d3e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7face0005fd0 tx=0x7face0009660 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:35.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.421+0000 7face6a00640 1 -- 172.21.15.71:0/3573151992 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7facdc020030 con 0x7facf00ccf00 2024-09-16T07:38:35.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.646+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facf015d860 con 0x7facf00ccf00 2024-09-16T07:38:35.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.647+0000 7face6a00640 1 -- 172.21.15.71:0/3573151992 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5230 (secure 0 0 0) 0x7facf00ce020 con 0x7facf00ccf00 2024-09-16T07:38:35.647 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:35.647 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":96,"btime":"2024-09-16T07:37:24:630475+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/343939477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":343939477},{"type":"v1","addr":"172.21.15.71:6835","nonce":343939477}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1796926972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1796926972},{"type":"v1","addr":"172.21.15.177:6836","nonce":1796926972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3638818185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3638818185},{"type":"v1","addr":"172.21.15.71:6834","nonce":3638818185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:37:16.010465+0000","modified":"2024-09-16T07:37:19.022287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1486935311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1486935311},{"type":"v1","addr":"172.21.15.177:6837","nonce":1486935311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9420,"qdb_cluster":[9420]},"id":10}]} 2024-09-16T07:38:35.647 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 96 2024-09-16T07:38:35.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007af20 msgr2=0x7facd007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:35.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007af20 0x7facd007d3e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7face0005fd0 tx=0x7face0009660 comp rx=0 tx=0).stop 2024-09-16T07:38:35.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00ccf00 msgr2=0x7facf00cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:35.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00ccf00 0x7facf00cdae0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7facdc00c460 tx=0x7facdc00c920 comp rx=0 tx=0).stop 2024-09-16T07:38:35.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 shutdown_connections 2024-09-16T07:38:35.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7facd007af20 0x7facd007d3e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facf015f5a0 0x7facf009c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facf00ccf00 0x7facf00cdae0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 --2- 172.21.15.71:0/3573151992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facf0073fb0 0x7facf014cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.649+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 >> 172.21.15.71:0/3573151992 conn(0x7facf00b1950 msgr2=0x7facf0155900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:35.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.650+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 shutdown_connections 2024-09-16T07:38:35.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.650+0000 7facf5400640 1 -- 172.21.15.71:0/3573151992 wait complete. 2024-09-16T07:38:35.665 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:38:35.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.875+0000 7f9ca6200640 1 -- 172.21.15.71:0/300716211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 msgr2=0x7f9ca00d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:35.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.875+0000 7f9ca6200640 1 --2- 172.21.15.71:0/300716211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 0x7f9ca00d2760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009be0 tx=0x7f9c9002a8e0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.875+0000 7f9ca6200640 1 -- 172.21.15.71:0/300716211 shutdown_connections 2024-09-16T07:38:35.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.875+0000 7f9ca6200640 1 --2- 172.21.15.71:0/300716211 >> v2:172.21.15.177:3300/0 conn(0x7f9ca00b1680 0x7f9ca00b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.875+0000 7f9ca6200640 1 --2- 172.21.15.71:0/300716211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 0x7f9ca00d2760 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.875+0000 7f9ca6200640 1 -- 172.21.15.71:0/300716211 >> 172.21.15.71:0/300716211 conn(0x7f9ca00d1c40 msgr2=0x7f9ca00d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:35.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.875+0000 7f9ca6200640 1 -- 172.21.15.71:0/300716211 shutdown_connections 2024-09-16T07:38:35.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.875+0000 7f9ca6200640 1 -- 172.21.15.71:0/300716211 wait complete. 2024-09-16T07:38:35.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.876+0000 7f9ca6200640 1 Processor -- start 2024-09-16T07:38:35.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.876+0000 7f9ca6200640 1 -- start start 2024-09-16T07:38:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.877+0000 7f9ca6200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ca00b1680 0x7f9ca0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.877+0000 7f9ca6200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ca00cff60 0x7f9ca014cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.877+0000 7f9ca6200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 0x7f9ca014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.877+0000 7f9ca6200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca00b21f0 con 0x7f9ca00cff60 2024-09-16T07:38:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.877+0000 7f9ca6200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca00b2070 con 0x7f9ca00b1680 2024-09-16T07:38:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.877+0000 7f9ca6200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca00b2370 con 0x7f9ca015bd30 2024-09-16T07:38:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ca00cff60 0x7f9ca014cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ca00cff60 0x7f9ca014cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58774/0 (socket says 172.21.15.71:58774) 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 -- 172.21.15.71:0/1506409741 learned_addr learned my addr 172.21.15.71:0/1506409741 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9ca4e00640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ca00b1680 0x7f9ca0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9ca5800640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 0x7f9ca014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 -- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 msgr2=0x7f9ca014d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 0x7f9ca014d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 -- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ca00b1680 msgr2=0x7f9ca0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ca00b1680 0x7f9ca0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 -- 172.21.15.71:0/1506409741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca01f5db0 con 0x7f9ca00cff60 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9ca5800640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 0x7f9ca014d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9fe00640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ca00cff60 0x7f9ca014cc80 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c460 tx=0x7f9c8c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.878+0000 7f9c9cc00640 1 -- 172.21.15.71:0/1506409741 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c016020 con 0x7f9ca00cff60 2024-09-16T07:38:35.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.879+0000 7f9c9cc00640 1 -- 172.21.15.71:0/1506409741 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c8c012920 con 0x7f9ca00cff60 2024-09-16T07:38:35.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.879+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca01f6090 con 0x7f9ca00cff60 2024-09-16T07:38:35.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.879+0000 7f9c9cc00640 1 -- 172.21.15.71:0/1506409741 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c012d50 con 0x7f9ca00cff60 2024-09-16T07:38:35.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.879+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca01f6650 con 0x7f9ca00cff60 2024-09-16T07:38:35.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.880+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca00b2900 con 0x7f9ca00cff60 2024-09-16T07:38:35.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.881+0000 7f9c9cc00640 1 -- 172.21.15.71:0/1506409741 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f9c8c019020 con 0x7f9ca00cff60 2024-09-16T07:38:35.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.882+0000 7f9c9cc00640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c8007af20 0x7f9c8007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:35.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.882+0000 7f9c9cc00640 1 -- 172.21.15.71:0/1506409741 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7f9c8c0998b0 con 0x7f9ca00cff60 2024-09-16T07:38:35.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.882+0000 7f9ca4e00640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c8007af20 0x7f9c8007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.883+0000 7f9ca4e00640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c8007af20 0x7f9c8007d3e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9c90034000 tx=0x7f9c90004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:35.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:35.886+0000 7f9c9cc00640 1 -- 172.21.15.71:0/1506409741 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c8c020030 con 0x7f9ca00cff60 2024-09-16T07:38:36.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.107+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ca00cde20 con 0x7f9ca00cff60 2024-09-16T07:38:36.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.108+0000 7f9c9cc00640 1 -- 172.21.15.71:0/1506409741 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5230 (secure 0 0 0) 0x7f9c8c0639c0 con 0x7f9ca00cff60 2024-09-16T07:38:36.107 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:36.107 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":96,"btime":"2024-09-16T07:37:24:630475+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/343939477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":343939477},{"type":"v1","addr":"172.21.15.71:6835","nonce":343939477}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1796926972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1796926972},{"type":"v1","addr":"172.21.15.177:6836","nonce":1796926972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3638818185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3638818185},{"type":"v1","addr":"172.21.15.71:6834","nonce":3638818185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:37:16.010465+0000","modified":"2024-09-16T07:37:19.022287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1486935311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1486935311},{"type":"v1","addr":"172.21.15.177:6837","nonce":1486935311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9420,"qdb_cluster":[9420]},"id":10}]} 2024-09-16T07:38:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 96 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c8007af20 msgr2=0x7f9c8007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c8007af20 0x7f9c8007d3e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9c90034000 tx=0x7f9c90004ab0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ca00cff60 msgr2=0x7f9ca014cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ca00cff60 0x7f9ca014cc80 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00c460 tx=0x7f9c8c00c920 comp rx=0 tx=0).stop 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 shutdown_connections 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f9c8007af20 0x7f9c8007d3e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ca015bd30 0x7f9ca014d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ca00cff60 0x7f9ca014cc80 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 --2- 172.21.15.71:0/1506409741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ca00b1680 0x7f9ca0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 >> 172.21.15.71:0/1506409741 conn(0x7f9ca00d1c40 msgr2=0x7f9ca00ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:36.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 shutdown_connections 2024-09-16T07:38:36.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.110+0000 7f9ca6200640 1 -- 172.21.15.71:0/1506409741 wait complete. 2024-09-16T07:38:36.125 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:38:36.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.283+0000 7fe232a00640 1 -- 172.21.15.71:0/2023082818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c15f5a0 msgr2=0x7fe22c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.283+0000 7fe232a00640 1 --2- 172.21.15.71:0/2023082818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c15f5a0 0x7fe22c0b2440 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005fd0 tx=0x7fe21c02a820 comp rx=0 tx=0).stop 2024-09-16T07:38:36.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.283+0000 7fe232a00640 1 -- 172.21.15.71:0/2023082818 shutdown_connections 2024-09-16T07:38:36.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.283+0000 7fe232a00640 1 --2- 172.21.15.71:0/2023082818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c15f5a0 0x7fe22c0b2440 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.283+0000 7fe232a00640 1 -- 172.21.15.71:0/2023082818 >> 172.21.15.71:0/2023082818 conn(0x7fe22c0b1950 msgr2=0x7fe22c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:36.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.283+0000 7fe232a00640 1 -- 172.21.15.71:0/2023082818 shutdown_connections 2024-09-16T07:38:36.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.283+0000 7fe232a00640 1 -- 172.21.15.71:0/2023082818 wait complete. 2024-09-16T07:38:36.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.284+0000 7fe232a00640 1 Processor -- start 2024-09-16T07:38:36.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.284+0000 7fe232a00640 1 -- start start 2024-09-16T07:38:36.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.285+0000 7fe232a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c073fb0 0x7fe22c1f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:36.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.285+0000 7fe232a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c0ccf00 0x7fe22c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.285+0000 7fe232a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c15f5a0 0x7fe22c0afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.285+0000 7fe232a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c06f9f0 con 0x7fe22c0ccf00 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.285+0000 7fe232a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c06f870 con 0x7fe22c15f5a0 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.285+0000 7fe232a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c074420 con 0x7fe22c073fb0 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c0ccf00 0x7fe22c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c0ccf00 0x7fe22c0cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58788/0 (socket says 172.21.15.71:58788) 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 -- 172.21.15.71:0/2695757280 learned_addr learned my addr 172.21.15.71:0/2695757280 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22be00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c073fb0 0x7fe22c1f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe230a00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c15f5a0 0x7fe22c0afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 -- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c073fb0 msgr2=0x7fe22c1f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c073fb0 0x7fe22c1f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 -- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c15f5a0 msgr2=0x7fe22c0afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c15f5a0 0x7fe22c0afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 -- 172.21.15.71:0/2695757280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c0c9600 con 0x7fe22c0ccf00 2024-09-16T07:38:36.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe230a00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c15f5a0 0x7fe22c0afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:36.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe22b400640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c0ccf00 0x7fe22c0cb7f0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe21800c4c0 tx=0x7fe21800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:36.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.286+0000 7fe223e00640 1 -- 172.21.15.71:0/2695757280 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218016020 con 0x7fe22c0ccf00 2024-09-16T07:38:36.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.287+0000 7fe223e00640 1 -- 172.21.15.71:0/2695757280 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe218012940 con 0x7fe22c0ccf00 2024-09-16T07:38:36.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.287+0000 7fe223e00640 1 -- 172.21.15.71:0/2695757280 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218012d90 con 0x7fe22c0ccf00 2024-09-16T07:38:36.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.287+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c0c98e0 con 0x7fe22c0ccf00 2024-09-16T07:38:36.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.287+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c0c9ea0 con 0x7fe22c0ccf00 2024-09-16T07:38:36.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.288+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c0b2440 con 0x7fe22c0ccf00 2024-09-16T07:38:36.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.289+0000 7fe223e00640 1 -- 172.21.15.71:0/2695757280 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fe218019020 con 0x7fe22c0ccf00 2024-09-16T07:38:36.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.290+0000 7fe223e00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe20c07af20 0x7fe20c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:36.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.290+0000 7fe223e00640 1 -- 172.21.15.71:0/2695757280 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7fe218099880 con 0x7fe22c0ccf00 2024-09-16T07:38:36.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.290+0000 7fe22be00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe20c07af20 0x7fe20c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:36.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.291+0000 7fe22be00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe20c07af20 0x7fe20c07d3e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe21c034000 tx=0x7fe21c004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:36.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.294+0000 7fe223e00640 1 -- 172.21.15.71:0/2695757280 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe218020030 con 0x7fe22c0ccf00 2024-09-16T07:38:36.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.480+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe22c09dcf0 con 0x7fe22c0ccf00 2024-09-16T07:38:36.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.481+0000 7fe223e00640 1 -- 172.21.15.71:0/2695757280 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+17033 (secure 0 0 0) 0x7fe218063950 con 0x7fe22c0ccf00 2024-09-16T07:38:36.481 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:36.481 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":149,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:38:32.519060+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:37:13.765661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T07:37:14.968618+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.34","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:38:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe20c07af20 msgr2=0x7fe20c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe20c07af20 0x7fe20c07d3e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe21c034000 tx=0x7fe21c004ab0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c0ccf00 msgr2=0x7fe22c0cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c0ccf00 0x7fe22c0cb7f0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe21800c4c0 tx=0x7fe21800c980 comp rx=0 tx=0).stop 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 shutdown_connections 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fe20c07af20 0x7fe20c07d3e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c15f5a0 0x7fe22c0afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe22c0ccf00 0x7fe22c0cb7f0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 --2- 172.21.15.71:0/2695757280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c073fb0 0x7fe22c1f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 >> 172.21.15.71:0/2695757280 conn(0x7fe22c0b1950 msgr2=0x7fe22c0ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 shutdown_connections 2024-09-16T07:38:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.484+0000 7fe232a00640 1 -- 172.21.15.71:0/2695757280 wait complete. 2024-09-16T07:38:36.500 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:38:36.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:38:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.663+0000 7fef5aa00640 1 -- 172.21.15.71:0/236080666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef54073fb0 msgr2=0x7fef54074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.663+0000 7fef5aa00640 1 --2- 172.21.15.71:0/236080666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef54073fb0 0x7fef54074460 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fef4c005fd0 tx=0x7fef4c02aa00 comp rx=0 tx=0).stop 2024-09-16T07:38:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.663+0000 7fef5aa00640 1 -- 172.21.15.71:0/236080666 shutdown_connections 2024-09-16T07:38:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.663+0000 7fef5aa00640 1 --2- 172.21.15.71:0/236080666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef54073fb0 0x7fef54074460 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.663+0000 7fef5aa00640 1 --2- 172.21.15.71:0/236080666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef5415f5a0 0x7fef540b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.663+0000 7fef5aa00640 1 -- 172.21.15.71:0/236080666 >> 172.21.15.71:0/236080666 conn(0x7fef540b1950 msgr2=0x7fef540b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.663+0000 7fef5aa00640 1 -- 172.21.15.71:0/236080666 shutdown_connections 2024-09-16T07:38:36.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.663+0000 7fef5aa00640 1 -- 172.21.15.71:0/236080666 wait complete. 2024-09-16T07:38:36.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.665+0000 7fef5aa00640 1 Processor -- start 2024-09-16T07:38:36.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.665+0000 7fef5aa00640 1 -- start start 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef54073fb0 0x7fef541f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef540ccf00 0x7fef540cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef5aa00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef5415f5a0 0x7fef541572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef5aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fef5406f2e0 con 0x7fef540ccf00 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef5aa00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fef5406f160 con 0x7fef54073fb0 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef5aa00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fef5406f460 con 0x7fef5415f5a0 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef4be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef540ccf00 0x7fef540cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef4be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef540ccf00 0x7fef540cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58790/0 (socket says 172.21.15.71:58790) 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef4be00640 1 -- 172.21.15.71:0/4072703019 learned_addr learned my addr 172.21.15.71:0/4072703019 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef53e00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef54073fb0 0x7fef541f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef58a00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef5415f5a0 0x7fef541572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:36.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef4be00640 1 -- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef5415f5a0 msgr2=0x7fef541572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef4be00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef5415f5a0 0x7fef541572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef4be00640 1 -- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef54073fb0 msgr2=0x7fef541f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef4be00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef54073fb0 0x7fef541f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef4be00640 1 -- 172.21.15.71:0/4072703019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef54145730 con 0x7fef540ccf00 2024-09-16T07:38:36.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.666+0000 7fef58a00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef5415f5a0 0x7fef541572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:36.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.667+0000 7fef4be00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef540ccf00 0x7fef540cff20 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00c420 tx=0x7fef3c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:36.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.667+0000 7fef50c00640 1 -- 172.21.15.71:0/4072703019 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c017070 con 0x7fef540ccf00 2024-09-16T07:38:36.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.667+0000 7fef50c00640 1 -- 172.21.15.71:0/4072703019 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef3c0053c0 con 0x7fef540ccf00 2024-09-16T07:38:36.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.667+0000 7fef50c00640 1 -- 172.21.15.71:0/4072703019 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c003850 con 0x7fef540ccf00 2024-09-16T07:38:36.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.667+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef54146290 con 0x7fef540ccf00 2024-09-16T07:38:36.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.667+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef54145cd0 con 0x7fef540ccf00 2024-09-16T07:38:36.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.668+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5406f9f0 con 0x7fef540ccf00 2024-09-16T07:38:36.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.669+0000 7fef50c00640 1 -- 172.21.15.71:0/4072703019 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fef3c01a070 con 0x7fef540ccf00 2024-09-16T07:38:36.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.671+0000 7fef50c00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef3007af20 0x7fef3007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:36.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.671+0000 7fef50c00640 1 -- 172.21.15.71:0/4072703019 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7fef3c099060 con 0x7fef540ccf00 2024-09-16T07:38:36.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.671+0000 7fef53e00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef3007af20 0x7fef3007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:36.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.672+0000 7fef53e00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef3007af20 0x7fef3007d3e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fef44009ac0 tx=0x7fef44005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:36.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.675+0000 7fef50c00640 1 -- 172.21.15.71:0/4072703019 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fef3c063130 con 0x7fef540ccf00 2024-09-16T07:38:36.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.896+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef540aceb0 con 0x7fef540ccf00 2024-09-16T07:38:36.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.896+0000 7fef50c00640 1 -- 172.21.15.71:0/4072703019 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5230 (secure 0 0 0) 0x7fef3c062a30 con 0x7fef540ccf00 2024-09-16T07:38:36.897 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:36.897 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":96,"btime":"2024-09-16T07:37:24:630475+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/343939477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":343939477},{"type":"v1","addr":"172.21.15.71:6835","nonce":343939477}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1796926972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1796926972},{"type":"v1","addr":"172.21.15.177:6836","nonce":1796926972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3638818185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3638818185},{"type":"v1","addr":"172.21.15.71:6834","nonce":3638818185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:37:16.010465+0000","modified":"2024-09-16T07:37:19.022287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1486935311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1486935311},{"type":"v1","addr":"172.21.15.177:6837","nonce":1486935311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9420,"qdb_cluster":[9420]},"id":10}]} 2024-09-16T07:38:36.897 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 96 2024-09-16T07:38:36.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef3007af20 msgr2=0x7fef3007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef3007af20 0x7fef3007d3e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fef44009ac0 tx=0x7fef44005f90 comp rx=0 tx=0).stop 2024-09-16T07:38:36.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef540ccf00 msgr2=0x7fef540cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:36.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef540ccf00 0x7fef540cff20 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00c420 tx=0x7fef3c00c8e0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 shutdown_connections 2024-09-16T07:38:36.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef3007af20 0x7fef3007d3e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef5415f5a0 0x7fef541572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef540ccf00 0x7fef540cff20 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 --2- 172.21.15.71:0/4072703019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef54073fb0 0x7fef541f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:36.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 >> 172.21.15.71:0/4072703019 conn(0x7fef540b1950 msgr2=0x7fef540b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:36.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 shutdown_connections 2024-09-16T07:38:36.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:36.899+0000 7fef5aa00640 1 -- 172.21.15.71:0/4072703019 wait complete. 2024-09-16T07:38:36.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:38:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.072+0000 7f91d2600640 1 -- 172.21.15.71:0/344540849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 msgr2=0x7f91cc0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.072+0000 7f91d2600640 1 --2- 172.21.15.71:0/344540849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 0x7f91cc0b3940 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0b6930 tx=0x7f91b802a560 comp rx=0 tx=0).stop 2024-09-16T07:38:37.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.073+0000 7f91d2600640 1 -- 172.21.15.71:0/344540849 shutdown_connections 2024-09-16T07:38:37.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.073+0000 7f91d2600640 1 --2- 172.21.15.71:0/344540849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 0x7f91cc0b3940 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.073+0000 7f91d2600640 1 --2- 172.21.15.71:0/344540849 >> v2:172.21.15.177:3300/0 conn(0x7f91cc161440 0x7f91cc0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.073+0000 7f91d2600640 1 -- 172.21.15.71:0/344540849 >> 172.21.15.71:0/344540849 conn(0x7f91cc0d3df0 msgr2=0x7f91cc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:37.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.073+0000 7f91d2600640 1 -- 172.21.15.71:0/344540849 shutdown_connections 2024-09-16T07:38:37.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.073+0000 7f91d2600640 1 -- 172.21.15.71:0/344540849 wait complete. 2024-09-16T07:38:37.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.074+0000 7f91d2600640 1 Processor -- start 2024-09-16T07:38:37.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.074+0000 7f91d2600640 1 -- start start 2024-09-16T07:38:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.075+0000 7f91d2600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 0x7f91cc0b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.075+0000 7f91d2600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91cc0b3e80 0x7f91cc0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.075+0000 7f91d2600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91cc161440 0x7f91cc151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.075+0000 7f91d2600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0b2900 con 0x7f91cc0b34b0 2024-09-16T07:38:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.075+0000 7f91d2600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0b2780 con 0x7f91cc0b3e80 2024-09-16T07:38:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.075+0000 7f91d2600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc0b2a80 con 0x7f91cc161440 2024-09-16T07:38:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.075+0000 7f91d1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 0x7f91cc0b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 0x7f91cc0b8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58800/0 (socket says 172.21.15.71:58800) 2024-09-16T07:38:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1200640 1 -- 172.21.15.71:0/2804323281 learned_addr learned my addr 172.21.15.71:0/2804323281 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91cbe00640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91cc0b3e80 0x7f91cc0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1c00640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91cc161440 0x7f91cc151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1200640 1 -- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91cc161440 msgr2=0x7f91cc151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1200640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91cc161440 0x7f91cc151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1200640 1 -- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91cc0b3e80 msgr2=0x7f91cc0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1200640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91cc0b3e80 0x7f91cc0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1200640 1 -- 172.21.15.71:0/2804323281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc0b9490 con 0x7f91cc0b34b0 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.076+0000 7f91d1200640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 0x7f91cc0b8eb0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00ea20 tx=0x7f91bc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.077+0000 7f91c8c00640 1 -- 172.21.15.71:0/2804323281 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00cde0 con 0x7f91cc0b34b0 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.077+0000 7f91c8c00640 1 -- 172.21.15.71:0/2804323281 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91bc004c10 con 0x7f91cc0b34b0 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.077+0000 7f91c8c00640 1 -- 172.21.15.71:0/2804323281 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc005020 con 0x7f91cc0b34b0 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.077+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91cc0b7ec0 con 0x7f91cc0b34b0 2024-09-16T07:38:37.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.077+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91cc0b99e0 con 0x7f91cc0b34b0 2024-09-16T07:38:37.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.078+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91cc0b3940 con 0x7f91cc0b34b0 2024-09-16T07:38:37.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.079+0000 7f91c8c00640 1 -- 172.21.15.71:0/2804323281 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f91bc0051c0 con 0x7f91cc0b34b0 2024-09-16T07:38:37.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.081+0000 7f91c8c00640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f91ac07af50 0x7f91ac07d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.081+0000 7f91c8c00640 1 -- 172.21.15.71:0/2804323281 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7f91bc09a580 con 0x7f91cc0b34b0 2024-09-16T07:38:37.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.081+0000 7f91cbe00640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f91ac07af50 0x7f91ac07d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.082+0000 7f91cbe00640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f91ac07af50 0x7f91ac07d410 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f91b8033000 tx=0x7f91b8004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:37.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.085+0000 7f91c8c00640 1 -- 172.21.15.71:0/2804323281 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f91bc016030 con 0x7f91cc0b34b0 2024-09-16T07:38:37.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.305+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91cc0b7c70 con 0x7f91cc0b34b0 2024-09-16T07:38:37.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.305+0000 7f91c8c00640 1 -- 172.21.15.71:0/2804323281 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5230 (secure 0 0 0) 0x7f91bc064000 con 0x7f91cc0b34b0 2024-09-16T07:38:37.306 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:37.306 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":96,"btime":"2024-09-16T07:37:24:630475+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/343939477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":343939477},{"type":"v1","addr":"172.21.15.71:6835","nonce":343939477}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1796926972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1796926972},{"type":"v1","addr":"172.21.15.177:6836","nonce":1796926972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3638818185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3638818185},{"type":"v1","addr":"172.21.15.71:6834","nonce":3638818185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T07:37:16.010465+0000","modified":"2024-09-16T07:37:19.022287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1486935311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1486935311},{"type":"v1","addr":"172.21.15.177:6837","nonce":1486935311}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9420,"qdb_cluster":[9420]},"id":10}]} 2024-09-16T07:38:37.306 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 96 2024-09-16T07:38:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.307+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f91ac07af50 msgr2=0x7f91ac07d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.307+0000 7f91d2600640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f91ac07af50 0x7f91ac07d410 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f91b8033000 tx=0x7f91b8004730 comp rx=0 tx=0).stop 2024-09-16T07:38:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.307+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 msgr2=0x7f91cc0b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.307+0000 7f91d2600640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 0x7f91cc0b8eb0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00ea20 tx=0x7f91bc00eee0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.308+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 shutdown_connections 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.308+0000 7f91d2600640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f91ac07af50 0x7f91ac07d410 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.308+0000 7f91d2600640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91cc161440 0x7f91cc151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.308+0000 7f91d2600640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91cc0b3e80 0x7f91cc0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.308+0000 7f91d2600640 1 --2- 172.21.15.71:0/2804323281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91cc0b34b0 0x7f91cc0b8eb0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.308+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 >> 172.21.15.71:0/2804323281 conn(0x7f91cc0d3df0 msgr2=0x7f91cc0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.308+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 shutdown_connections 2024-09-16T07:38:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.308+0000 7f91d2600640 1 -- 172.21.15.71:0/2804323281 wait complete. 2024-09-16T07:38:37.325 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:38:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.480+0000 7f1417a00640 1 -- 172.21.15.71:0/1913443582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 msgr2=0x7f13f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.480+0000 7f1417a00640 1 --2- 172.21.15.71:0/1913443582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 0x7f13f400b9c0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f14100b2820 tx=0x7f13f4005830 comp rx=0 tx=0).stop 2024-09-16T07:38:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.480+0000 7f1415a00640 1 -- 172.21.15.71:0/1913443582 reap_dead start 2024-09-16T07:38:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.480+0000 7f1417a00640 1 -- 172.21.15.71:0/1913443582 shutdown_connections 2024-09-16T07:38:37.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.480+0000 7f1417a00640 1 -- 172.21.15.71:0/1913443582 >> 172.21.15.71:0/1913443582 conn(0x7f14100b1950 msgr2=0x7f14100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:37.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.480+0000 7f1417a00640 1 -- 172.21.15.71:0/1913443582 shutdown_connections 2024-09-16T07:38:37.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.480+0000 7f1417a00640 1 -- 172.21.15.71:0/1913443582 wait complete. 2024-09-16T07:38:37.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.482+0000 7f1417a00640 1 Processor -- start 2024-09-16T07:38:37.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.482+0000 7f1417a00640 1 -- start start 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1417a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 0x7f14101f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1417a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141009b030 0x7f14100cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1417a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14100cca60 0x7f14100c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1417a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1410074840 con 0x7f13f40051c0 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1417a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f14100746c0 con 0x7f141009b030 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1417a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f141006f160 con 0x7f14100cca60 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1415000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 0x7f14101f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1415000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 0x7f14101f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58824/0 (socket says 172.21.15.71:58824) 2024-09-16T07:38:37.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.483+0000 7f1415000640 1 -- 172.21.15.71:0/3796574658 learned_addr learned my addr 172.21.15.71:0/3796574658 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:37.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1415000640 1 -- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14100cca60 msgr2=0x7f14100c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1415a00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14100cca60 0x7f14100c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f140fe00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141009b030 0x7f14100cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1415000640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14100cca60 0x7f14100c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1415000640 1 -- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141009b030 msgr2=0x7f14100cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1415000640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141009b030 0x7f14100cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1415000640 1 -- 172.21.15.71:0/3796574658 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141009b810 con 0x7f13f40051c0 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1415a00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14100cca60 0x7f14100c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f140fe00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141009b030 0x7f14100cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1415000640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 0x7f14101f56b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f14000060e0 tx=0x7f14000065a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f140cc00640 1 -- 172.21.15.71:0/3796574658 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400015070 con 0x7f13f40051c0 2024-09-16T07:38:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f140cc00640 1 -- 172.21.15.71:0/3796574658 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f140000fb30 con 0x7f13f40051c0 2024-09-16T07:38:37.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f140cc00640 1 -- 172.21.15.71:0/3796574658 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400011430 con 0x7f13f40051c0 2024-09-16T07:38:37.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141009baf0 con 0x7f13f40051c0 2024-09-16T07:38:37.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.484+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141009c0b0 con 0x7f13f40051c0 2024-09-16T07:38:37.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.486+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14100760b0 con 0x7f13f40051c0 2024-09-16T07:38:37.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.487+0000 7f140cc00640 1 -- 172.21.15.71:0/3796574658 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f14000115d0 con 0x7f13f40051c0 2024-09-16T07:38:37.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.488+0000 7f140cc00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f13f007af50 0x7f13f007d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.488+0000 7f140cc00640 1 -- 172.21.15.71:0/3796574658 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7f140009ac20 con 0x7f13f40051c0 2024-09-16T07:38:37.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.488+0000 7f140fe00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f13f007af50 0x7f13f007d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.489+0000 7f140fe00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f13f007af50 0x7f13f007d410 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f13fc001220 tx=0x7f13fc0067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:37.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.492+0000 7f140cc00640 1 -- 172.21.15.71:0/3796574658 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f14100760b0 con 0x7f13f40051c0 2024-09-16T07:38:37.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.678+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14100d0980 con 0x7f13f40051c0 2024-09-16T07:38:37.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.679+0000 7f140cc00640 1 -- 172.21.15.71:0/3796574658 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+17033 (secure 0 0 0) 0x7f1400064cf0 con 0x7f13f40051c0 2024-09-16T07:38:37.680 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:37.680 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":149,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:38:32.519060+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T07:37:13.765661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T07:37:14.968618+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.34","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:38:37.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f13f007af50 msgr2=0x7f13f007d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f13f007af50 0x7f13f007d410 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f13fc001220 tx=0x7f13fc0067a0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 msgr2=0x7f14101f56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 0x7f14101f56b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f14000060e0 tx=0x7f14000065a0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 shutdown_connections 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f13f007af50 0x7f13f007d410 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14100cca60 0x7f14100c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f141009b030 0x7f14100cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 --2- 172.21.15.71:0/3796574658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f40051c0 0x7f14101f56b0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 >> 172.21.15.71:0/3796574658 conn(0x7f14100b1950 msgr2=0x7f141015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 shutdown_connections 2024-09-16T07:38:37.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.682+0000 7f1417a00640 1 -- 172.21.15.71:0/3796574658 wait complete. 2024-09-16T07:38:37.698 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T07:38:37.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.855+0000 7ff2e7a00640 1 -- 172.21.15.71:0/783752253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00af3d0 msgr2=0x7ff2e006f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.855+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/783752253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00af3d0 0x7ff2e006f660 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00b4890 tx=0x7ff2cc02a4b0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.855+0000 7ff2e7a00640 1 -- 172.21.15.71:0/783752253 shutdown_connections 2024-09-16T07:38:37.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.855+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/783752253 >> v2:172.21.15.177:3300/0 conn(0x7ff2e0072df0 0x7ff2e0073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.855+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/783752253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00af3d0 0x7ff2e006f660 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.855+0000 7ff2e7a00640 1 -- 172.21.15.71:0/783752253 >> 172.21.15.71:0/783752253 conn(0x7ff2e00b1950 msgr2=0x7ff2e00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:37.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.855+0000 7ff2e7a00640 1 -- 172.21.15.71:0/783752253 shutdown_connections 2024-09-16T07:38:37.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.855+0000 7ff2e7a00640 1 -- 172.21.15.71:0/783752253 wait complete. 2024-09-16T07:38:37.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.857+0000 7ff2e7a00640 1 Processor -- start 2024-09-16T07:38:37.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.857+0000 7ff2e7a00640 1 -- start start 2024-09-16T07:38:37.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e7a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0072df0 0x7ff2e015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e7a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2e00af3d0 0x7ff2e00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e7a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2e0161550 0x7ff2e00b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e7a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0073780 con 0x7ff2e0072df0 2024-09-16T07:38:37.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e7a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e006fdb0 con 0x7ff2e00af3d0 2024-09-16T07:38:37.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e7a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e0073900 con 0x7ff2e0161550 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0072df0 0x7ff2e015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0072df0 0x7ff2e015a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58836/0 (socket says 172.21.15.71:58836) 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e5000640 1 -- 172.21.15.71:0/2920386523 learned_addr learned my addr 172.21.15.71:0/2920386523 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2dfe00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2e00af3d0 0x7ff2e00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.858+0000 7ff2e5a00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2e0161550 0x7ff2e00b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2e5000640 1 -- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2e0161550 msgr2=0x7ff2e00b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2e5000640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2e0161550 0x7ff2e00b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2e5000640 1 -- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2e00af3d0 msgr2=0x7ff2e00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:37.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2e5000640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2e00af3d0 0x7ff2e00b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:37.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2e5000640 1 -- 172.21.15.71:0/2920386523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e01f9140 con 0x7ff2e0072df0 2024-09-16T07:38:37.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2e5000640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0072df0 0x7ff2e015a7c0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000ba80 tx=0x7ff2d000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:37.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2dcc00640 1 -- 172.21.15.71:0/2920386523 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0019070 con 0x7ff2e0072df0 2024-09-16T07:38:37.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2dcc00640 1 -- 172.21.15.71:0/2920386523 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2d000cd80 con 0x7ff2e0072df0 2024-09-16T07:38:37.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e00cf6b0 con 0x7ff2e0072df0 2024-09-16T07:38:37.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2dcc00640 1 -- 172.21.15.71:0/2920386523 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00146a0 con 0x7ff2e0072df0 2024-09-16T07:38:37.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.859+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01f8e10 con 0x7ff2e0072df0 2024-09-16T07:38:37.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.861+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e0073e50 con 0x7ff2e0072df0 2024-09-16T07:38:37.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.862+0000 7ff2dcc00640 1 -- 172.21.15.71:0/2920386523 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7ff2d001c070 con 0x7ff2e0072df0 2024-09-16T07:38:37.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.863+0000 7ff2dcc00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2c007af50 0x7ff2c007d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:37.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.863+0000 7ff2dcc00640 1 -- 172.21.15.71:0/2920386523 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5907+0+0 (secure 0 0 0) 0x7ff2d00995e0 con 0x7ff2e0072df0 2024-09-16T07:38:37.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.863+0000 7ff2dfe00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2c007af50 0x7ff2c007d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:37.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.864+0000 7ff2dfe00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2c007af50 0x7ff2c007d410 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc033000 tx=0x7ff2cc004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:37.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:37.867+0000 7ff2dcc00640 1 -- 172.21.15.71:0/2920386523 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff2d0063080 con 0x7ff2e0072df0 2024-09-16T07:38:38.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.084+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff2e009aa00 con 0x7ff2e0072df0 2024-09-16T07:38:38.085 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-09-16T07:38:38.084+0000 7f2efd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T07:38:38.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.086+0000 7ff2dcc00640 1 -- 172.21.15.71:0/2920386523 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) ==== 161+0+0 (secure 0 0 0) 0x7ff2d0067ab0 con 0x7ff2e0072df0 2024-09-16T07:38:38.087 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T07:38:38.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2c007af50 msgr2=0x7ff2c007d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:38.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2c007af50 0x7ff2c007d410 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc033000 tx=0x7ff2cc004730 comp rx=0 tx=0).stop 2024-09-16T07:38:38.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0072df0 msgr2=0x7ff2e015a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0072df0 0x7ff2e015a7c0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000ba80 tx=0x7ff2d000bf40 comp rx=0 tx=0).stop 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 shutdown_connections 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ff2c007af50 0x7ff2c007d410 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2e0161550 0x7ff2e00b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2e00af3d0 0x7ff2e00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 --2- 172.21.15.71:0/2920386523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0072df0 0x7ff2e015a7c0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 >> 172.21.15.71:0/2920386523 conn(0x7ff2e00b1950 msgr2=0x7ff2e00ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 shutdown_connections 2024-09-16T07:38:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.090+0000 7ff2e7a00640 1 -- 172.21.15.71:0/2920386523 wait complete. 2024-09-16T07:38:38.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T07:38:38.161 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-09-16T07:38:38.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.265+0000 7f6dd5600640 1 -- 172.21.15.71:0/4053672311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dd0161760 msgr2=0x7f6dd00ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:38.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.265+0000 7f6dd5600640 1 --2- 172.21.15.71:0/4053672311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dd0161760 0x7f6dd00ce4a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009860 tx=0x7f6dc002a510 comp rx=0 tx=0).stop 2024-09-16T07:38:38.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.266+0000 7f6dd5600640 1 -- 172.21.15.71:0/4053672311 shutdown_connections 2024-09-16T07:38:38.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.266+0000 7f6dd5600640 1 --2- 172.21.15.71:0/4053672311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dd0161760 0x7f6dd00ce4a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:38.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.266+0000 7f6dd5600640 1 -- 172.21.15.71:0/4053672311 >> 172.21.15.71:0/4053672311 conn(0x7f6dd00b1840 msgr2=0x7f6dd00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:38.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.266+0000 7f6dd5600640 1 -- 172.21.15.71:0/4053672311 shutdown_connections 2024-09-16T07:38:38.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.266+0000 7f6dd5600640 1 -- 172.21.15.71:0/4053672311 wait complete. 2024-09-16T07:38:38.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.267+0000 7f6dd5600640 1 Processor -- start 2024-09-16T07:38:38.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.267+0000 7f6dd5600640 1 -- start start 2024-09-16T07:38:38.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.268+0000 7f6dd5600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dd00af190 0x7f6dd015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:38.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.268+0000 7f6dd5600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dd00cec60 0x7f6dd01600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:38.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.268+0000 7f6dd5600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dd0161760 0x7f6dd0160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.268+0000 7f6dd5600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0162c30 con 0x7f6dd0161760 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.268+0000 7f6dd5600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0162ab0 con 0x7f6dd00af190 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.268+0000 7f6dd5600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd0147f20 con 0x7f6dd00cec60 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dd0161760 0x7f6dd0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dd0161760 0x7f6dd0160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58838/0 (socket says 172.21.15.71:58838) 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 -- 172.21.15.71:0/3347875616 learned_addr learned my addr 172.21.15.71:0/3347875616 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dce000640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dd00cec60 0x7f6dd01600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcea00640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dd00af190 0x7f6dd015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 -- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dd00cec60 msgr2=0x7f6dd01600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dd00cec60 0x7f6dd01600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 -- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dd00af190 msgr2=0x7f6dd015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dd00af190 0x7f6dd015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 -- 172.21.15.71:0/3347875616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd01f8a40 con 0x7f6dd0161760 2024-09-16T07:38:38.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.269+0000 7f6dcf400640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dd0161760 0x7f6dd0160630 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c7d0 tx=0x7f6dc800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:38.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.270+0000 7f6dc6a00640 1 -- 172.21.15.71:0/3347875616 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8003b70 con 0x7f6dd0161760 2024-09-16T07:38:38.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.270+0000 7f6dc6a00640 1 -- 172.21.15.71:0/3347875616 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dc8003d10 con 0x7f6dd0161760 2024-09-16T07:38:38.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.270+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd01f49a0 con 0x7f6dd0161760 2024-09-16T07:38:38.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.270+0000 7f6dc6a00640 1 -- 172.21.15.71:0/3347875616 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800aab0 con 0x7f6dd0161760 2024-09-16T07:38:38.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.270+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd01f4f60 con 0x7f6dd0161760 2024-09-16T07:38:38.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.271+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd01f4790 con 0x7f6dd0161760 2024-09-16T07:38:38.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.272+0000 7f6dc6a00640 1 -- 172.21.15.71:0/3347875616 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f6dc800ac50 con 0x7f6dd0161760 2024-09-16T07:38:38.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.273+0000 7f6dc6a00640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6db007af20 0x7f6db007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:38.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.273+0000 7f6dc6a00640 1 -- 172.21.15.71:0/3347875616 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5965+0+0 (secure 0 0 0) 0x7f6dc8099a60 con 0x7f6dd0161760 2024-09-16T07:38:38.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.273+0000 7f6dcea00640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6db007af20 0x7f6db007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:38.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.274+0000 7f6dcea00640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6db007af20 0x7f6db007d3e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0034000 tx=0x7f6dc0004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:38.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.277+0000 7f6dc6a00640 1 -- 172.21.15.71:0/3347875616 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6dc8063b30 con 0x7f6dd0161760 2024-09-16T07:38:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:38.502+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6dd00cf340 con 0x7f6dd0161760 2024-09-16T07:38:39.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.088+0000 7f6dc6a00640 1 -- 172.21.15.71:0/3347875616 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) ==== 104+0+0 (secure 0 0 0) 0x7f6dc8063430 con 0x7f6dd0161760 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6db007af20 msgr2=0x7f6db007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6db007af20 0x7f6db007d3e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0034000 tx=0x7f6dc0004730 comp rx=0 tx=0).stop 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dd0161760 msgr2=0x7f6dd0160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dd0161760 0x7f6dd0160630 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c7d0 tx=0x7f6dc800cc90 comp rx=0 tx=0).stop 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 shutdown_connections 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6db007af20 0x7f6db007d3e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dd0161760 0x7f6dd0160630 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dd00cec60 0x7f6dd01600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 --2- 172.21.15.71:0/3347875616 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dd00af190 0x7f6dd015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 >> 172.21.15.71:0/3347875616 conn(0x7f6dd00b1840 msgr2=0x7f6dd0072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:39.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 shutdown_connections 2024-09-16T07:38:39.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.091+0000 7f6dd5600640 1 -- 172.21.15.71:0/3347875616 wait complete. 2024-09-16T07:38:39.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T07:38:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.270+0000 7f8c47000640 1 -- 172.21.15.71:0/498579083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4015f9a0 msgr2=0x7f8c4015fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.270+0000 7f8c47000640 1 --2- 172.21.15.71:0/498579083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4015f9a0 0x7f8c4015fd80 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005fd0 tx=0x7f8c3002a9c0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.270+0000 7f8c47000640 1 -- 172.21.15.71:0/498579083 shutdown_connections 2024-09-16T07:38:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.270+0000 7f8c47000640 1 --2- 172.21.15.71:0/498579083 >> v2:172.21.15.177:3300/0 conn(0x7f8c4006e6f0 0x7f8c4006eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.270+0000 7f8c47000640 1 --2- 172.21.15.71:0/498579083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4015f9a0 0x7f8c4015fd80 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.270+0000 7f8c47000640 1 -- 172.21.15.71:0/498579083 >> 172.21.15.71:0/498579083 conn(0x7f8c400b18e0 msgr2=0x7f8c400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.270+0000 7f8c47000640 1 -- 172.21.15.71:0/498579083 shutdown_connections 2024-09-16T07:38:39.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.270+0000 7f8c47000640 1 -- 172.21.15.71:0/498579083 wait complete. 2024-09-16T07:38:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.271+0000 7f8c47000640 1 Processor -- start 2024-09-16T07:38:39.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.271+0000 7f8c47000640 1 -- start start 2024-09-16T07:38:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.272+0000 7f8c47000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4006e6f0 0x7f8c401534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.272+0000 7f8c47000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40073de0 0x7f8c4014b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.272+0000 7f8c47000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c4015f9a0 0x7f8c401588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.272+0000 7f8c47000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400cd550 con 0x7f8c4006e6f0 2024-09-16T07:38:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.272+0000 7f8c47000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c400cd3d0 con 0x7f8c4015f9a0 2024-09-16T07:38:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.272+0000 7f8c47000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4006f0e0 con 0x7f8c40073de0 2024-09-16T07:38:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4006e6f0 0x7f8c401534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4006e6f0 0x7f8c401534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58854/0 (socket says 172.21.15.71:58854) 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 -- 172.21.15.71:0/201575692 learned_addr learned my addr 172.21.15.71:0/201575692 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c37400640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40073de0 0x7f8c4014b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 -- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40073de0 msgr2=0x7f8c4014b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40073de0 0x7f8c4014b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 -- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c4015f9a0 msgr2=0x7f8c401588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c4015f9a0 0x7f8c401588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 -- 172.21.15.71:0/201575692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c40153d40 con 0x7f8c4006e6f0 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3fe00640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4006e6f0 0x7f8c401534d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009bf0 tx=0x7f8c30003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3cc00640 1 -- 172.21.15.71:0/201575692 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30034040 con 0x7f8c4006e6f0 2024-09-16T07:38:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.273+0000 7f8c3cc00640 1 -- 172.21.15.71:0/201575692 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c30004540 con 0x7f8c4006e6f0 2024-09-16T07:38:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.274+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c40154020 con 0x7f8c4006e6f0 2024-09-16T07:38:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.274+0000 7f8c3cc00640 1 -- 172.21.15.71:0/201575692 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30045480 con 0x7f8c4006e6f0 2024-09-16T07:38:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.274+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c401548c0 con 0x7f8c4006e6f0 2024-09-16T07:38:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.275+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c40069360 con 0x7f8c4006e6f0 2024-09-16T07:38:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.276+0000 7f8c3cc00640 1 -- 172.21.15.71:0/201575692 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f8c30037020 con 0x7f8c4006e6f0 2024-09-16T07:38:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.277+0000 7f8c3cc00640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c1c07af20 0x7f8c1c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.277+0000 7f8c3cc00640 1 -- 172.21.15.71:0/201575692 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5965+0+0 (secure 0 0 0) 0x7f8c300bb4b0 con 0x7f8c4006e6f0 2024-09-16T07:38:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.277+0000 7f8c37400640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c1c07af20 0x7f8c1c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.278+0000 7f8c37400640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c1c07af20 0x7f8c1c07d3e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8c28007fd0 tx=0x7f8c28009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:39.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.281+0000 7f8c3cc00640 1 -- 172.21.15.71:0/201575692 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8c30085500 con 0x7f8c4006e6f0 2024-09-16T07:38:39.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:39.467+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8c4006f560 con 0x7f8c4006e6f0 2024-09-16T07:38:40.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.090+0000 7f8c3cc00640 1 -- 172.21.15.71:0/201575692 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) ==== 181+0+0 (secure 0 0 0) 0x7f8c30084e00 con 0x7f8c4006e6f0 2024-09-16T07:38:40.091 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-16T07:38:40.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c1c07af20 msgr2=0x7f8c1c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:40.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c1c07af20 0x7f8c1c07d3e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8c28007fd0 tx=0x7f8c28009040 comp rx=0 tx=0).stop 2024-09-16T07:38:40.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4006e6f0 msgr2=0x7f8c401534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:40.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4006e6f0 0x7f8c401534d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009bf0 tx=0x7f8c30003f90 comp rx=0 tx=0).stop 2024-09-16T07:38:40.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 shutdown_connections 2024-09-16T07:38:40.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8c1c07af20 0x7f8c1c07d3e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:40.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c4015f9a0 0x7f8c401588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:40.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40073de0 0x7f8c4014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:40.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 --2- 172.21.15.71:0/201575692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c4006e6f0 0x7f8c401534d0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:40.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.094+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 >> 172.21.15.71:0/201575692 conn(0x7f8c400b18e0 msgr2=0x7f8c400b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:40.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.095+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 shutdown_connections 2024-09-16T07:38:40.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.095+0000 7f8c47000640 1 -- 172.21.15.71:0/201575692 wait complete. 2024-09-16T07:38:40.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T07:38:40.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.276+0000 7fa0a3200640 1 -- 172.21.15.71:0/884060633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa088004800 msgr2=0x7fa088004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:40.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.276+0000 7fa0a3200640 1 --2- 172.21.15.71:0/884060633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa088004800 0x7fa088004c80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa090009b50 tx=0x7fa088005830 comp rx=0 tx=0).stop 2024-09-16T07:38:40.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.277+0000 7fa0a3200640 1 -- 172.21.15.71:0/884060633 shutdown_connections 2024-09-16T07:38:40.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.277+0000 7fa0a3200640 1 --2- 172.21.15.71:0/884060633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa088004800 0x7fa088004c80 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:40.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.277+0000 7fa0a3200640 1 -- 172.21.15.71:0/884060633 >> 172.21.15.71:0/884060633 conn(0x7fa0a40cfa70 msgr2=0x7fa0a40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:40.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.277+0000 7fa0a3200640 1 -- 172.21.15.71:0/884060633 shutdown_connections 2024-09-16T07:38:40.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.277+0000 7fa0a3200640 1 -- 172.21.15.71:0/884060633 wait complete. 2024-09-16T07:38:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.278+0000 7fa0a3200640 1 Processor -- start 2024-09-16T07:38:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.278+0000 7fa0a3200640 1 -- start start 2024-09-16T07:38:40.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.279+0000 7fa0a3200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa088004800 0x7fa0a415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:40.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.279+0000 7fa0a3200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0a40c58b0 0x7fa0a4159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:40.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.279+0000 7fa0a3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a40c4180 0x7fa0a40c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:40.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.279+0000 7fa0a3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a40701d0 con 0x7fa0a40c4180 2024-09-16T07:38:40.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.279+0000 7fa0a3200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4070050 con 0x7fa088004800 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.279+0000 7fa0a3200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a4073010 con 0x7fa0a40c58b0 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a40c4180 0x7fa0a40c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a1e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa088004800 0x7fa0a415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a1400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0a40c58b0 0x7fa0a4159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a40c4180 0x7fa0a40c35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58866/0 (socket says 172.21.15.71:58866) 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 -- 172.21.15.71:0/2325143104 learned_addr learned my addr 172.21.15.71:0/2325143104 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 -- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0a40c58b0 msgr2=0x7fa0a4159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0a40c58b0 0x7fa0a4159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 -- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa088004800 msgr2=0x7fa0a415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa088004800 0x7fa0a415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:40.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 -- 172.21.15.71:0/2325143104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40c5d40 con 0x7fa0a40c4180 2024-09-16T07:38:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a2800640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a40c4180 0x7fa0a40c35a0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa09c001330 tx=0x7fa09c00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa09a000640 1 -- 172.21.15.71:0/2325143104 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c015070 con 0x7fa0a40c4180 2024-09-16T07:38:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa09a000640 1 -- 172.21.15.71:0/2325143104 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa09c00cac0 con 0x7fa0a40c4180 2024-09-16T07:38:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.280+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a40c6c70 con 0x7fa0a40c4180 2024-09-16T07:38:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.281+0000 7fa09a000640 1 -- 172.21.15.71:0/2325143104 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c011460 con 0x7fa0a40c4180 2024-09-16T07:38:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.281+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a40c6020 con 0x7fa0a40c4180 2024-09-16T07:38:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.282+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a4073570 con 0x7fa0a40c4180 2024-09-16T07:38:40.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.283+0000 7fa09a000640 1 -- 172.21.15.71:0/2325143104 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fa09c018070 con 0x7fa0a40c4180 2024-09-16T07:38:40.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.284+0000 7fa09a000640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa08407af50 0x7fa08407d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:40.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.285+0000 7fa09a000640 1 -- 172.21.15.71:0/2325143104 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5527+0+0 (secure 0 0 0) 0x7fa09c09a010 con 0x7fa0a40c4180 2024-09-16T07:38:40.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.285+0000 7fa0a1e00640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa08407af50 0x7fa08407d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:40.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.286+0000 7fa0a1e00640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa08407af50 0x7fa08407d410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa094000c00 tx=0x7fa094006d50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:40.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.289+0000 7fa09a000640 1 -- 172.21.15.71:0/2325143104 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa09c063b10 con 0x7fa0a40c4180 2024-09-16T07:38:40.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:40.474+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa0a40ae830 con 0x7fa0a40c4180 2024-09-16T07:38:41.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.092+0000 7fa09a000640 1 -- 172.21.15.71:0/2325143104 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) ==== 169+0+0 (secure 0 0 0) 0x7fa09c068540 con 0x7fa0a40c4180 2024-09-16T07:38:41.093 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-16T07:38:41.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.095+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa08407af50 msgr2=0x7fa08407d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.095+0000 7fa0a3200640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa08407af50 0x7fa08407d410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa094000c00 tx=0x7fa094006d50 comp rx=0 tx=0).stop 2024-09-16T07:38:41.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.095+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a40c4180 msgr2=0x7fa0a40c35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.095+0000 7fa0a3200640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a40c4180 0x7fa0a40c35a0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa09c001330 tx=0x7fa09c00eb60 comp rx=0 tx=0).stop 2024-09-16T07:38:41.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.096+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 shutdown_connections 2024-09-16T07:38:41.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.096+0000 7fa0a3200640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fa08407af50 0x7fa08407d410 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.096+0000 7fa0a3200640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a40c4180 0x7fa0a40c35a0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.096+0000 7fa0a3200640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0a40c58b0 0x7fa0a4159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.096+0000 7fa0a3200640 1 --2- 172.21.15.71:0/2325143104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa088004800 0x7fa0a415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.096+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 >> 172.21.15.71:0/2325143104 conn(0x7fa0a40cfa70 msgr2=0x7fa0a40ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:41.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.096+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 shutdown_connections 2024-09-16T07:38:41.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.097+0000 7fa0a3200640 1 -- 172.21.15.71:0/2325143104 wait complete. 2024-09-16T07:38:41.118 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T07:38:41.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.275+0000 7f8227200640 1 -- 172.21.15.71:0/3406548566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220073e00 msgr2=0x7f8220168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.275+0000 7f8227200640 1 --2- 172.21.15.71:0/3406548566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220073e00 0x7f8220168e10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f82200b4890 tx=0x7f821802a800 comp rx=0 tx=0).stop 2024-09-16T07:38:41.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.275+0000 7f8227200640 1 -- 172.21.15.71:0/3406548566 shutdown_connections 2024-09-16T07:38:41.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.275+0000 7f8227200640 1 --2- 172.21.15.71:0/3406548566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220073e00 0x7f8220168e10 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.275+0000 7f8227200640 1 -- 172.21.15.71:0/3406548566 >> 172.21.15.71:0/3406548566 conn(0x7f82200b1950 msgr2=0x7f82200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:41.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.275+0000 7f8227200640 1 -- 172.21.15.71:0/3406548566 shutdown_connections 2024-09-16T07:38:41.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.275+0000 7f8227200640 1 -- 172.21.15.71:0/3406548566 wait complete. 2024-09-16T07:38:41.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.277+0000 7f8227200640 1 Processor -- start 2024-09-16T07:38:41.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.277+0000 7f8227200640 1 -- start start 2024-09-16T07:38:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8227200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8220073e00 0x7f8220149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8227200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82200af3d0 0x7f8220150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8227200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220161550 0x7f8220156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8227200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f82201694d0 con 0x7f8220161550 2024-09-16T07:38:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8227200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8220169350 con 0x7f8220073e00 2024-09-16T07:38:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8227200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8220169650 con 0x7f82200af3d0 2024-09-16T07:38:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8225200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220161550 0x7f8220156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:41.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8225200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220161550 0x7f8220156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58884/0 (socket says 172.21.15.71:58884) 2024-09-16T07:38:41.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f8225200640 1 -- 172.21.15.71:0/969328123 learned_addr learned my addr 172.21.15.71:0/969328123 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:41.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f821fe00640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8220073e00 0x7f8220149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:41.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.278+0000 7f821f400640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82200af3d0 0x7f8220150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:41.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8225200640 1 -- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82200af3d0 msgr2=0x7f8220150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8225200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82200af3d0 0x7f8220150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8225200640 1 -- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8220073e00 msgr2=0x7f8220149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8225200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8220073e00 0x7f8220149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8225200640 1 -- 172.21.15.71:0/969328123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822015c5e0 con 0x7f8220161550 2024-09-16T07:38:41.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8225200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220161550 0x7f8220156310 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8218035040 tx=0x7f8218033e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:41.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8217e00640 1 -- 172.21.15.71:0/969328123 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218042020 con 0x7f8220161550 2024-09-16T07:38:41.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8217e00640 1 -- 172.21.15.71:0/969328123 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8218004330 con 0x7f8220161550 2024-09-16T07:38:41.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8217e00640 1 -- 172.21.15.71:0/969328123 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218037040 con 0x7f8220161550 2024-09-16T07:38:41.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8220151330 con 0x7f8220161550 2024-09-16T07:38:41.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.279+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82201518f0 con 0x7f8220161550 2024-09-16T07:38:41.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.280+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82200b2480 con 0x7f8220161550 2024-09-16T07:38:41.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.281+0000 7f8217e00640 1 -- 172.21.15.71:0/969328123 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f8218045020 con 0x7f8220161550 2024-09-16T07:38:41.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.282+0000 7f8217e00640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f820007af20 0x7f820007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:41.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.282+0000 7f8217e00640 1 -- 172.21.15.71:0/969328123 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f82180babf0 con 0x7f8220161550 2024-09-16T07:38:41.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.283+0000 7f821fe00640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f820007af20 0x7f820007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:41.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.284+0000 7f821fe00640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f820007af20 0x7f820007d3e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8210000f60 tx=0x7f8210009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:41.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.287+0000 7f8217e00640 1 -- 172.21.15.71:0/969328123 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f82200b2480 con 0x7f8220161550 2024-09-16T07:38:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.472+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f822006f710 con 0x7f8220161550 2024-09-16T07:38:41.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.476+0000 7f8217e00640 1 -- 172.21.15.71:0/969328123 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v57) ==== 71+0+0 (secure 0 0 0) 0x7f821808b020 con 0x7f8220161550 2024-09-16T07:38:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f820007af20 msgr2=0x7f820007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f820007af20 0x7f820007d3e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8210000f60 tx=0x7f8210009040 comp rx=0 tx=0).stop 2024-09-16T07:38:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220161550 msgr2=0x7f8220156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220161550 0x7f8220156310 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8218035040 tx=0x7f8218033e80 comp rx=0 tx=0).stop 2024-09-16T07:38:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 shutdown_connections 2024-09-16T07:38:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f820007af20 0x7f820007d3e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8220161550 0x7f8220156310 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82200af3d0 0x7f8220150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.478+0000 7f8227200640 1 --2- 172.21.15.71:0/969328123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8220073e00 0x7f8220149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.479+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 >> 172.21.15.71:0/969328123 conn(0x7f82200b1950 msgr2=0x7f822015f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.479+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 shutdown_connections 2024-09-16T07:38:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.479+0000 7f8227200640 1 -- 172.21.15.71:0/969328123 wait complete. 2024-09-16T07:38:41.494 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-16T07:38:41.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 -- 172.21.15.71:0/852193087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8161550 msgr2=0x7f12a806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 --2- 172.21.15.71:0/852193087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8161550 0x7f12a806ef50 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f12a80752e0 tx=0x7f129802a980 comp rx=0 tx=0).stop 2024-09-16T07:38:41.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 -- 172.21.15.71:0/852193087 shutdown_connections 2024-09-16T07:38:41.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 --2- 172.21.15.71:0/852193087 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a806f9d0 0x7f12a8161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 --2- 172.21.15.71:0/852193087 >> v2:172.21.15.177:3300/0 conn(0x7f12a80af3d0 0x7f12a806f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 --2- 172.21.15.71:0/852193087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8161550 0x7f12a806ef50 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 -- 172.21.15.71:0/852193087 >> 172.21.15.71:0/852193087 conn(0x7f12a80b1950 msgr2=0x7f12a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:41.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 -- 172.21.15.71:0/852193087 shutdown_connections 2024-09-16T07:38:41.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.650+0000 7f12aee00640 1 -- 172.21.15.71:0/852193087 wait complete. 2024-09-16T07:38:41.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.652+0000 7f12aee00640 1 Processor -- start 2024-09-16T07:38:41.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.652+0000 7f12aee00640 1 -- start start 2024-09-16T07:38:41.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12aee00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a806f9d0 0x7f12a8151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12aee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a80af3d0 0x7f12a809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12aee00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a8161550 0x7f12a809c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12aee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12a8162560 con 0x7f12a80af3d0 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12aee00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f12a81623e0 con 0x7f12a806f9d0 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12aee00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f12a81626e0 con 0x7f12a8161550 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a80af3d0 0x7f12a809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a80af3d0 0x7f12a809aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58894/0 (socket says 172.21.15.71:58894) 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12a7400640 1 -- 172.21.15.71:0/1162611250 learned_addr learned my addr 172.21.15.71:0/1162611250 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12a7e00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a806f9d0 0x7f12a8151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:41.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.653+0000 7f12ace00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a8161550 0x7f12a809c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:41.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f12a7400640 1 -- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a8161550 msgr2=0x7f12a809c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f12a7400640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a8161550 0x7f12a809c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f12a7400640 1 -- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a806f9d0 msgr2=0x7f12a8151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f12a7400640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a806f9d0 0x7f12a8151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f12a7400640 1 -- 172.21.15.71:0/1162611250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a809b3e0 con 0x7f12a80af3d0 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f12a7400640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a80af3d0 0x7f12a809aa00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f129400c970 tx=0x7f129400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f129fe00640 1 -- 172.21.15.71:0/1162611250 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294007c10 con 0x7f12a80af3d0 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f129fe00640 1 -- 172.21.15.71:0/1162611250 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1294018480 con 0x7f12a80af3d0 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a809b6c0 con 0x7f12a80af3d0 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f129fe00640 1 -- 172.21.15.71:0/1162611250 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940188d0 con 0x7f12a80af3d0 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.654+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a809bee0 con 0x7f12a80af3d0 2024-09-16T07:38:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.656+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a8161c90 con 0x7f12a80af3d0 2024-09-16T07:38:41.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.657+0000 7f129fe00640 1 -- 172.21.15.71:0/1162611250 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f1294018a70 con 0x7f12a80af3d0 2024-09-16T07:38:41.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.658+0000 7f129fe00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128807af20 0x7f128807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:41.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.658+0000 7f129fe00640 1 -- 172.21.15.71:0/1162611250 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f129409a650 con 0x7f12a80af3d0 2024-09-16T07:38:41.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.658+0000 7f12a7e00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128807af20 0x7f128807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:41.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.659+0000 7f12a7e00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128807af20 0x7f128807d3e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1298035000 tx=0x7f1298004ab0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:41.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.662+0000 7f129fe00640 1 -- 172.21.15.71:0/1162611250 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1294064170 con 0x7f12a80af3d0 2024-09-16T07:38:41.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:41.914+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f12a806e710 con 0x7f12a80af3d0 2024-09-16T07:38:42.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.094+0000 7f129fe00640 1 -- 172.21.15.71:0/1162611250 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v744) ==== 186+0+0 (secure 0 0 0) 0x7f1294068ba0 con 0x7f12a80af3d0 2024-09-16T07:38:42.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.096+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128807af20 msgr2=0x7f128807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:42.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.096+0000 7f12aee00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128807af20 0x7f128807d3e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1298035000 tx=0x7f1298004ab0 comp rx=0 tx=0).stop 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.096+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a80af3d0 msgr2=0x7f12a809aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.096+0000 7f12aee00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a80af3d0 0x7f12a809aa00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f129400c970 tx=0x7f129400ce30 comp rx=0 tx=0).stop 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.097+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 shutdown_connections 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.097+0000 7f12aee00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f128807af20 0x7f128807d3e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.097+0000 7f12aee00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a8161550 0x7f12a809c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.097+0000 7f12aee00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a80af3d0 0x7f12a809aa00 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.097+0000 7f12aee00640 1 --2- 172.21.15.71:0/1162611250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a806f9d0 0x7f12a8151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.097+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 >> 172.21.15.71:0/1162611250 conn(0x7f12a80b1950 msgr2=0x7f12a80ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.097+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 shutdown_connections 2024-09-16T07:38:42.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:42.097+0000 7f12aee00640 1 -- 172.21.15.71:0/1162611250 wait complete. 2024-09-16T07:38:42.114 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ok 2024-09-16T07:38:42.115 INFO:tasks.cephfs_test_runner: 2024-09-16T07:38:42.115 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-16T07:38:42.115 INFO:tasks.cephfs_test_runner:Ran 9 tests in 922.911s 2024-09-16T07:38:42.115 INFO:tasks.cephfs_test_runner: 2024-09-16T07:38:42.115 INFO:tasks.cephfs_test_runner:OK 2024-09-16T07:38:42.115 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-16T07:38:42.130 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T07:38:42.178 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T07:38:42.178 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:38:42.178 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:42.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:38:42.214 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-16T07:38:42.214 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:42.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T07:38:42.303 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:38:42.303 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:42.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:38:42.344 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-16T07:38:42.344 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T07:38:42.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T07:38:42.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:42.422 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:38:42.456 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T07:38:42.456 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T07:38:42.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:42.458 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T07:38:42.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:42.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T07:38:42.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:42.836 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-09-16T07:38:43.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:43.183 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-16T07:38:43.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:43.219 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-09-16T07:38:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:Cannot find device "ceph-brx" 2024-09-16T07:38:43.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:38:43.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:43.322 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-09-16T07:38:43.362 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T07:38:43.362 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T07:38:43.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:43.363 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T07:38:43.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:43.464 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T07:38:43.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:43.715 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2024-09-16T07:38:44.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:44.031 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-09-16T07:38:44.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T07:38:44.066 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2024-09-16T07:38:44.166 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2024-09-16T07:38:44.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:38:44.169 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T07:38:44.178 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T07:38:44.179 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:38:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.339+0000 7f0554800640 1 -- 172.21.15.71:0/1185829367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 msgr2=0x7f054c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.339+0000 7f0554800640 1 --2- 172.21.15.71:0/1185829367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 0x7f054c0acd30 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f053c005fd0 tx=0x7f053c02a540 comp rx=0 tx=0).stop 2024-09-16T07:38:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.340+0000 7f0554800640 1 -- 172.21.15.71:0/1185829367 shutdown_connections 2024-09-16T07:38:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.340+0000 7f0554800640 1 --2- 172.21.15.71:0/1185829367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 0x7f054c0acd30 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.340+0000 7f0554800640 1 --2- 172.21.15.71:0/1185829367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f054c15f5a0 0x7f054c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.340+0000 7f0554800640 1 -- 172.21.15.71:0/1185829367 >> 172.21.15.71:0/1185829367 conn(0x7f054c0b1950 msgr2=0x7f054c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.340+0000 7f0554800640 1 -- 172.21.15.71:0/1185829367 shutdown_connections 2024-09-16T07:38:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.340+0000 7f0554800640 1 -- 172.21.15.71:0/1185829367 wait complete. 2024-09-16T07:38:44.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.341+0000 7f0554800640 1 Processor -- start 2024-09-16T07:38:44.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.342+0000 7f0554800640 1 -- start start 2024-09-16T07:38:44.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.342+0000 7f0554800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f054c0ad270 0x7f054c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:44.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.342+0000 7f0554800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 0x7f054c1f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:44.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.342+0000 7f0554800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f054c15f5a0 0x7f054c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:44.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.342+0000 7f0554800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f054c06f2e0 con 0x7f054c0ccf00 2024-09-16T07:38:44.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.342+0000 7f0554800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f054c06f160 con 0x7f054c15f5a0 2024-09-16T07:38:44.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.342+0000 7f0554800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f054c06f460 con 0x7f054c0ad270 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 0x7f054c1f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 0x7f054c1f7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58912/0 (socket says 172.21.15.71:58912) 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551400640 1 -- 172.21.15.71:0/2501307307 learned_addr learned my addr 172.21.15.71:0/2501307307 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551e00640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f054c0ad270 0x7f054c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0552800640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f054c15f5a0 0x7f054c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551400640 1 -- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f054c0ad270 msgr2=0x7f054c158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551400640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f054c0ad270 0x7f054c158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551400640 1 -- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f054c15f5a0 msgr2=0x7f054c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551400640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f054c15f5a0 0x7f054c0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.343+0000 7f0551400640 1 -- 172.21.15.71:0/2501307307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c1f8690 con 0x7f054c0ccf00 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.344+0000 7f0551400640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 0x7f054c1f7e40 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f053c009850 tx=0x7f053c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.344+0000 7f0546000640 1 -- 172.21.15.71:0/2501307307 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c03f070 con 0x7f054c0ccf00 2024-09-16T07:38:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.344+0000 7f0546000640 1 -- 172.21.15.71:0/2501307307 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f053c0047a0 con 0x7f054c0ccf00 2024-09-16T07:38:44.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.344+0000 7f0546000640 1 -- 172.21.15.71:0/2501307307 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c035030 con 0x7f054c0ccf00 2024-09-16T07:38:44.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.344+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c1f8970 con 0x7f054c0ccf00 2024-09-16T07:38:44.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.344+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c1f4b60 con 0x7f054c0ccf00 2024-09-16T07:38:44.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.345+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c06f980 con 0x7f054c0ccf00 2024-09-16T07:38:44.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.346+0000 7f0546000640 1 -- 172.21.15.71:0/2501307307 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f053c042070 con 0x7f054c0ccf00 2024-09-16T07:38:44.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.347+0000 7f0546000640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f053007af20 0x7f053007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:44.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.347+0000 7f0546000640 1 -- 172.21.15.71:0/2501307307 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f053c0ba980 con 0x7f054c0ccf00 2024-09-16T07:38:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.348+0000 7f0551e00640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f053007af20 0x7f053007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.348+0000 7f0551e00640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f053007af20 0x7f053007d3e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f054c1541e0 tx=0x7f054000b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:44.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.351+0000 7f0546000640 1 -- 172.21.15.71:0/2501307307 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f053c0c6080 con 0x7f054c0ccf00 2024-09-16T07:38:44.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.534+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f054c0b06a0 con 0x7f053007af20 2024-09-16T07:38:44.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.535+0000 7f0546000640 1 -- 172.21.15.71:0/2501307307 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13577 (secure 0 0 0) 0x7f054c0b06a0 con 0x7f053007af20 2024-09-16T07:38:44.536 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:44.536 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:38:44.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.537+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f053007af20 msgr2=0x7f053007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f053007af20 0x7f053007d3e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f054c1541e0 tx=0x7f054000b2e0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 msgr2=0x7f054c1f7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 0x7f054c1f7e40 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f053c009850 tx=0x7f053c0038a0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 shutdown_connections 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f053007af20 0x7f053007d3e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f054c15f5a0 0x7f054c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c0ccf00 0x7f054c1f7e40 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 --2- 172.21.15.71:0/2501307307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f054c0ad270 0x7f054c158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 >> 172.21.15.71:0/2501307307 conn(0x7f054c0b1950 msgr2=0x7f054c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 shutdown_connections 2024-09-16T07:38:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.538+0000 7f0554800640 1 -- 172.21.15.71:0/2501307307 wait complete. 2024-09-16T07:38:44.553 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2024-09-16T07:38:42.675965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":1144072,"kb_used_data":17720,"kb_used_omap":12,"kb_used_meta":1126195,"kb_avail":753830648,"statfs":{"total":773094113280,"available":771922583552,"internally_reserved":0,"allocated":18145280,"data_stored":9806108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1153224240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12064725868,"num_objects":-2908,"num_object_clones":-1,"num_object_copies":-5816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2908,"num_whiteouts":-1,"num_read":-3786,"num_read_kb":-15138906,"num_write":-6698,"num_write_kb":-18147465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-8388608,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001257"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":348,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-16T07:38:38.780964+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:38:38.780964+0000","last_peered":"2024-09-16T07:38:38.780964+0000","last_clean":"2024-09-16T07:38:38.780964+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:38:38.780964+0000","last_undegraded":"2024-09-16T07:38:38.780964+0000","last_fullsized":"2024-09-16T07:38:38.780964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476933,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145400,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226440,"statfs":{"total":96636764160,"available":96487874560,"internally_reserved":0,"allocated":1871872,"data_stored":826935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476933,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145380,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226460,"statfs":{"total":96636764160,"available":96487895040,"internally_reserved":0,"allocated":1855488,"data_stored":820090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476933,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119592,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252248,"statfs":{"total":96636764160,"available":96514301952,"internally_reserved":0,"allocated":2449408,"data_stored":1410506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141228,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230612,"statfs":{"total":96636764160,"available":96492146688,"internally_reserved":0,"allocated":1863680,"data_stored":828246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509634,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123168,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248672,"statfs":{"total":96636764160,"available":96510640128,"internally_reserved":0,"allocated":1851392,"data_stored":820068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167620,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204220,"statfs":{"total":96636764160,"available":96465121280,"internally_reserved":0,"allocated":1880064,"data_stored":832492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":169736646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509636,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152116,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94219724,"statfs":{"total":96636764160,"available":96480997376,"internally_reserved":0,"allocated":4493312,"data_stored":3435279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542339,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149568,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222272,"statfs":{"total":96636764160,"available":96483606528,"internally_reserved":0,"allocated":1880064,"data_stored":832492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:38:44.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:38:44.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.710+0000 7f5744a00640 1 -- 172.21.15.71:0/2515960622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57400ccfd0 msgr2=0x7f57400cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.710+0000 7f5744a00640 1 --2- 172.21.15.71:0/2515960622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57400ccfd0 0x7f57400cf130 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f572c009900 tx=0x7f572c02a600 comp rx=0 tx=0).stop 2024-09-16T07:38:44.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.710+0000 7f5744a00640 1 -- 172.21.15.71:0/2515960622 shutdown_connections 2024-09-16T07:38:44.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.710+0000 7f5744a00640 1 --2- 172.21.15.71:0/2515960622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57400ccfd0 0x7f57400cf130 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.710+0000 7f5744a00640 1 --2- 172.21.15.71:0/2515960622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57401445d0 0x7f57400cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.710+0000 7f5744a00640 1 -- 172.21.15.71:0/2515960622 >> 172.21.15.71:0/2515960622 conn(0x7f57400b1950 msgr2=0x7f57400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:44.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.710+0000 7f5744a00640 1 -- 172.21.15.71:0/2515960622 shutdown_connections 2024-09-16T07:38:44.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.710+0000 7f5744a00640 1 -- 172.21.15.71:0/2515960622 wait complete. 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.712+0000 7f5744a00640 1 Processor -- start 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.712+0000 7f5744a00640 1 -- start start 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f5744a00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57400b4ca0 0x7f574015e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f5744a00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57400ccfd0 0x7f574013f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f5744a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57401445d0 0x7f57400cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f5744a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f574015f080 con 0x7f57401445d0 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f5744a00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5740073bc0 con 0x7f57400b4ca0 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f5744a00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f574015f200 con 0x7f57400ccfd0 2024-09-16T07:38:44.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f573e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57401445d0 0x7f57400cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f573e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57401445d0 0x7f57400cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36204/0 (socket says 172.21.15.71:36204) 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f573e800640 1 -- 172.21.15.71:0/4091204512 learned_addr learned my addr 172.21.15.71:0/4091204512 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f573d400640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57400ccfd0 0x7f574013f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.713+0000 7f573de00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57400b4ca0 0x7f574015e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f573e800640 1 -- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57400ccfd0 msgr2=0x7f574013f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f573e800640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57400ccfd0 0x7f574013f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f573e800640 1 -- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57400b4ca0 msgr2=0x7f574015e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f573e800640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57400b4ca0 0x7f574015e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f573e800640 1 -- 172.21.15.71:0/4091204512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57401f2b80 con 0x7f57401445d0 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f573d400640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57400ccfd0 0x7f574013f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f573e800640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57401445d0 0x7f57400cdae0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f573800c7a0 tx=0x7f573800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:44.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f5736000640 1 -- 172.21.15.71:0/4091204512 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738003b70 con 0x7f57401445d0 2024-09-16T07:38:44.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f5736000640 1 -- 172.21.15.71:0/4091204512 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5738003d30 con 0x7f57401445d0 2024-09-16T07:38:44.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f5736000640 1 -- 172.21.15.71:0/4091204512 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573800aab0 con 0x7f57401445d0 2024-09-16T07:38:44.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57401f2e60 con 0x7f57401445d0 2024-09-16T07:38:44.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.714+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57401f3420 con 0x7f57401445d0 2024-09-16T07:38:44.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.715+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574015f6b0 con 0x7f57401445d0 2024-09-16T07:38:44.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.716+0000 7f5736000640 1 -- 172.21.15.71:0/4091204512 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f573800ac50 con 0x7f57401445d0 2024-09-16T07:38:44.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.717+0000 7f5736000640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f572007af20 0x7f572007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:44.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.717+0000 7f5736000640 1 -- 172.21.15.71:0/4091204512 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f5738099990 con 0x7f57401445d0 2024-09-16T07:38:44.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.718+0000 7f573de00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f572007af20 0x7f572007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:44.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.719+0000 7f573de00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f572007af20 0x7f572007d3e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f574013f130 tx=0x7f573000b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:44.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.722+0000 7f5736000640 1 -- 172.21.15.71:0/4091204512 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5738063b00 con 0x7f57401445d0 2024-09-16T07:38:44.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.903+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5740070a20 con 0x7f572007af20 2024-09-16T07:38:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.904+0000 7f5736000640 1 -- 172.21.15.71:0/4091204512 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13559 (secure 0 0 0) 0x7f5740070a20 con 0x7f572007af20 2024-09-16T07:38:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:44.905 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:38:44.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f572007af20 msgr2=0x7f572007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f572007af20 0x7f572007d3e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f574013f130 tx=0x7f573000b2e0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57401445d0 msgr2=0x7f57400cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57401445d0 0x7f57400cdae0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f573800c7a0 tx=0x7f573800cc60 comp rx=0 tx=0).stop 2024-09-16T07:38:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 shutdown_connections 2024-09-16T07:38:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f572007af20 0x7f572007d3e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57401445d0 0x7f57400cdae0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57400ccfd0 0x7f574013f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 --2- 172.21.15.71:0/4091204512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57400b4ca0 0x7f574015e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 >> 172.21.15.71:0/4091204512 conn(0x7f57400b1950 msgr2=0x7f57400acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 shutdown_connections 2024-09-16T07:38:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:44.907+0000 7f5744a00640 1 -- 172.21.15.71:0/4091204512 wait complete. 2024-09-16T07:38:44.923 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":629,"stamp":"2024-09-16T07:38:44.676386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1164548,"kb_used_data":15944,"kb_used_omap":12,"kb_used_meta":1148403,"kb_avail":753810172,"statfs":{"total":773094113280,"available":771901616128,"internally_reserved":0,"allocated":16326656,"data_stored":7874938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1175965232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4095664450,"num_objects":-1001,"num_object_clones":0,"num_object_copies":-2002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1001,"num_whiteouts":0,"num_read":-3786,"num_read_kb":-15138906,"num_write":-8630,"num_write_kb":-18147594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-8388608,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.582350"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":353,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T07:38:42.265462+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:38:42.265462+0000","last_peered":"2024-09-16T07:38:42.265462+0000","last_clean":"2024-09-16T07:38:42.265462+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:38:42.265462+0000","last_undegraded":"2024-09-16T07:38:42.265462+0000","last_fullsized":"2024-09-16T07:38:42.265462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476934,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145428,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226412,"statfs":{"total":96636764160,"available":96487845888,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145428,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226412,"statfs":{"total":96636764160,"available":96487845888,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123800,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248040,"statfs":{"total":96636764160,"available":96509992960,"internally_reserved":0,"allocated":2490368,"data_stored":1428571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159316,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212524,"statfs":{"total":96636764160,"available":96473624576,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509635,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123220,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248620,"statfs":{"total":96636764160,"available":96510586880,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167620,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204220,"statfs":{"total":96636764160,"available":96465121280,"internally_reserved":0,"allocated":1880064,"data_stored":832492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":169736646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509637,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150168,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221672,"statfs":{"total":96636764160,"available":96482992128,"internally_reserved":0,"allocated":2490368,"data_stored":1428571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542339,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149568,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222272,"statfs":{"total":96636764160,"available":96483606528,"internally_reserved":0,"allocated":1880064,"data_stored":832492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:38:44.924 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T07:38:44.924 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:38:45.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.081+0000 7fbb2a200640 1 -- 172.21.15.71:0/340628276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb240730c0 msgr2=0x7fbb24073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.081+0000 7fbb2a200640 1 --2- 172.21.15.71:0/340628276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb240730c0 0x7fbb24073530 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009830 tx=0x7fbb0c02a510 comp rx=0 tx=0).stop 2024-09-16T07:38:45.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.081+0000 7fbb2a200640 1 -- 172.21.15.71:0/340628276 shutdown_connections 2024-09-16T07:38:45.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.081+0000 7fbb2a200640 1 --2- 172.21.15.71:0/340628276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb24151750 0x7fbb24151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.081+0000 7fbb2a200640 1 --2- 172.21.15.71:0/340628276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb240730c0 0x7fbb24073530 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.081+0000 7fbb2a200640 1 -- 172.21.15.71:0/340628276 >> 172.21.15.71:0/340628276 conn(0x7fbb240b2230 msgr2=0x7fbb240b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:45.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.081+0000 7fbb2a200640 1 -- 172.21.15.71:0/340628276 shutdown_connections 2024-09-16T07:38:45.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.081+0000 7fbb2a200640 1 -- 172.21.15.71:0/340628276 wait complete. 2024-09-16T07:38:45.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.083+0000 7fbb2a200640 1 Processor -- start 2024-09-16T07:38:45.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.083+0000 7fbb2a200640 1 -- start start 2024-09-16T07:38:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb2a200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb240730c0 0x7fbb240d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb2a200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb24151750 0x7fbb2413f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb2a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb2415bdc0 0x7fbb240cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:45.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb2a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2415c320 con 0x7fbb2415bdc0 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb2a200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2415c1a0 con 0x7fbb24151750 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb2a200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb241520c0 con 0x7fbb240730c0 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb23e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb2415bdc0 0x7fbb240cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb23e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb2415bdc0 0x7fbb240cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36208/0 (socket says 172.21.15.71:36208) 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb23e00640 1 -- 172.21.15.71:0/3271595623 learned_addr learned my addr 172.21.15.71:0/3271595623 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb1be00640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb24151750 0x7fbb2413f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.084+0000 7fbb23400640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb240730c0 0x7fbb240d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb23e00640 1 -- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb240730c0 msgr2=0x7fbb240d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb23e00640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb240730c0 0x7fbb240d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb23e00640 1 -- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb24151750 msgr2=0x7fbb2413f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb23e00640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb24151750 0x7fbb2413f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb23e00640 1 -- 172.21.15.71:0/3271595623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb240cfb00 con 0x7fbb2415bdc0 2024-09-16T07:38:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb23e00640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb2415bdc0 0x7fbb240cdf60 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00eb10 tx=0x7fbb1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb1b400640 1 -- 172.21.15.71:0/3271595623 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c004b20 con 0x7fbb2415bdc0 2024-09-16T07:38:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb1b400640 1 -- 172.21.15.71:0/3271595623 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb1c004ce0 con 0x7fbb2415bdc0 2024-09-16T07:38:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb1b400640 1 -- 172.21.15.71:0/3271595623 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c005280 con 0x7fbb2415bdc0 2024-09-16T07:38:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2413fb30 con 0x7fbb2415bdc0 2024-09-16T07:38:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.085+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb240d0080 con 0x7fbb2415bdc0 2024-09-16T07:38:45.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.087+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb24073530 con 0x7fbb2415bdc0 2024-09-16T07:38:45.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.088+0000 7fbb1b400640 1 -- 172.21.15.71:0/3271595623 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fbb1c005420 con 0x7fbb2415bdc0 2024-09-16T07:38:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.089+0000 7fbb1b400640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb0007af50 0x7fbb0007d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.089+0000 7fbb1b400640 1 -- 172.21.15.71:0/3271595623 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7fbb1c09a7e0 con 0x7fbb2415bdc0 2024-09-16T07:38:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.089+0000 7fbb23400640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb0007af50 0x7fbb0007d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:45.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.090+0000 7fbb23400640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb0007af50 0x7fbb0007d410 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007fd0 tx=0x7fbb14001370 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:45.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.094+0000 7fbb1b400640 1 -- 172.21.15.71:0/3271595623 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb1c064300 con 0x7fbb2415bdc0 2024-09-16T07:38:45.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.279+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb240af7a0 con 0x7fbb0007af50 2024-09-16T07:38:45.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.280+0000 7fbb1b400640 1 -- 172.21.15.71:0/3271595623 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13559 (secure 0 0 0) 0x7fbb240af7a0 con 0x7fbb0007af50 2024-09-16T07:38:45.281 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:45.281 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:38:45.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.282+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb0007af50 msgr2=0x7fbb0007d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.282+0000 7fbb2a200640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb0007af50 0x7fbb0007d410 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007fd0 tx=0x7fbb14001370 comp rx=0 tx=0).stop 2024-09-16T07:38:45.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.282+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb2415bdc0 msgr2=0x7fbb240cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.282+0000 7fbb2a200640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb2415bdc0 0x7fbb240cdf60 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00eb10 tx=0x7fbb1c00c2d0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.283+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 shutdown_connections 2024-09-16T07:38:45.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.283+0000 7fbb2a200640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fbb0007af50 0x7fbb0007d410 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.283+0000 7fbb2a200640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb2415bdc0 0x7fbb240cdf60 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.283+0000 7fbb2a200640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb24151750 0x7fbb2413f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.283+0000 7fbb2a200640 1 --2- 172.21.15.71:0/3271595623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb240730c0 0x7fbb240d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.283+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 >> 172.21.15.71:0/3271595623 conn(0x7fbb240b2230 msgr2=0x7fbb2406fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:45.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.283+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 shutdown_connections 2024-09-16T07:38:45.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.283+0000 7fbb2a200640 1 -- 172.21.15.71:0/3271595623 wait complete. 2024-09-16T07:38:45.299 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":629,"stamp":"2024-09-16T07:38:44.676386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1164548,"kb_used_data":15944,"kb_used_omap":12,"kb_used_meta":1148403,"kb_avail":753810172,"statfs":{"total":773094113280,"available":771901616128,"internally_reserved":0,"allocated":16326656,"data_stored":7874938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1175965232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4095664450,"num_objects":-1001,"num_object_clones":0,"num_object_copies":-2002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1001,"num_whiteouts":0,"num_read":-3786,"num_read_kb":-15138906,"num_write":-8630,"num_write_kb":-18147594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-8388608,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.582350"},"pg_stats":[{"pgid":"1.0","version":"19'72","reported_seq":353,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T07:38:42.265462+0000","last_change":"2024-09-16T07:22:40.452508+0000","last_active":"2024-09-16T07:38:42.265462+0000","last_peered":"2024-09-16T07:38:42.265462+0000","last_clean":"2024-09-16T07:38:42.265462+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:38:42.265462+0000","last_undegraded":"2024-09-16T07:38:42.265462+0000","last_fullsized":"2024-09-16T07:38:42.265462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T07:22:38.841623+0000","last_clean_scrub_stamp":"2024-09-16T07:22:38.841623+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:10:07.401987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476934,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145428,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226412,"statfs":{"total":96636764160,"available":96487845888,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145428,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226412,"statfs":{"total":96636764160,"available":96487845888,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123800,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248040,"statfs":{"total":96636764160,"available":96509992960,"internally_reserved":0,"allocated":2490368,"data_stored":1428571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159316,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212524,"statfs":{"total":96636764160,"available":96473624576,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509635,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123220,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248620,"statfs":{"total":96636764160,"available":96510586880,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167620,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204220,"statfs":{"total":96636764160,"available":96465121280,"internally_reserved":0,"allocated":1880064,"data_stored":832492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":169736646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509637,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150168,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221672,"statfs":{"total":96636764160,"available":96482992128,"internally_reserved":0,"allocated":2490368,"data_stored":1428571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542339,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149568,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222272,"statfs":{"total":96636764160,"available":96483606528,"internally_reserved":0,"allocated":1880064,"data_stored":832492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:38:45.299 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T07:38:45.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.456+0000 7f4e42200640 1 -- 172.21.15.71:0/3243003989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b4d10 msgr2=0x7f4e3c162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.456+0000 7f4e42200640 1 --2- 172.21.15.71:0/3243003989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b4d10 0x7f4e3c162730 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0b3230 tx=0x7f4e3402a8b0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.457+0000 7f4e42200640 1 -- 172.21.15.71:0/3243003989 shutdown_connections 2024-09-16T07:38:45.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.457+0000 7f4e42200640 1 --2- 172.21.15.71:0/3243003989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b4d10 0x7f4e3c162730 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.457+0000 7f4e42200640 1 --2- 172.21.15.71:0/3243003989 >> v2:172.21.15.177:3300/0 conn(0x7f4e3c0b28d0 0x7f4e3c0d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.457+0000 7f4e42200640 1 -- 172.21.15.71:0/3243003989 >> 172.21.15.71:0/3243003989 conn(0x7f4e3c0d3de0 msgr2=0x7f4e3c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:45.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.457+0000 7f4e42200640 1 -- 172.21.15.71:0/3243003989 shutdown_connections 2024-09-16T07:38:45.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.457+0000 7f4e42200640 1 -- 172.21.15.71:0/3243003989 wait complete. 2024-09-16T07:38:45.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.458+0000 7f4e42200640 1 Processor -- start 2024-09-16T07:38:45.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.458+0000 7f4e42200640 1 -- start start 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.459+0000 7f4e42200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b28d0 0x7f4e3c14d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.459+0000 7f4e42200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e3c0b4d10 0x7f4e3c0cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.459+0000 7f4e42200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e3c0d1ec0 0x7f4e3c0cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.459+0000 7f4e42200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0b3a80 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.459+0000 7f4e42200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0b3900 con 0x7f4e3c0d1ec0 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.459+0000 7f4e42200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c0b3c00 con 0x7f4e3c0b4d10 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b28d0 0x7f4e3c14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b28d0 0x7f4e3c14d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36238/0 (socket says 172.21.15.71:36238) 2024-09-16T07:38:45.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 -- 172.21.15.71:0/4076864216 learned_addr learned my addr 172.21.15.71:0/4076864216 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e41800640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e3c0d1ec0 0x7f4e3c0cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e3be00640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e3c0b4d10 0x7f4e3c0cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 -- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e3c0b4d10 msgr2=0x7f4e3c0cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e3c0b4d10 0x7f4e3c0cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 -- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e3c0d1ec0 msgr2=0x7f4e3c0cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e3c0d1ec0 0x7f4e3c0cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 -- 172.21.15.71:0/4076864216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c14d970 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e40e00640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b28d0 0x7f4e3c14d430 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000c430 tx=0x7f4e3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.460+0000 7f4e38c00640 1 -- 172.21.15.71:0/4076864216 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30011790 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.461+0000 7f4e38c00640 1 -- 172.21.15.71:0/4076864216 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e30017070 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.461+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c14fd90 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.461+0000 7f4e38c00640 1 -- 172.21.15.71:0/4076864216 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30011ce0 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.461+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c150350 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.462+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c0d22a0 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.463+0000 7f4e38c00640 1 -- 172.21.15.71:0/4076864216 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f4e3001c460 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.464+0000 7f4e38c00640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e1c07af20 0x7f4e1c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.464+0000 7f4e38c00640 1 -- 172.21.15.71:0/4076864216 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f4e30099990 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.464+0000 7f4e3be00640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e1c07af20 0x7f4e1c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.465+0000 7f4e3be00640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e1c07af20 0x7f4e1c07d3e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c15dda0 tx=0x7f4e28005f90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:45.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.468+0000 7f4e38c00640 1 -- 172.21.15.71:0/4076864216 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e30063b00 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.653+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e3c0c73f0 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.654+0000 7f4e38c00640 1 -- 172.21.15.71:0/4076864216 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+13234 (secure 0 0 0) 0x7f4e30063400 con 0x7f4e3c0b28d0 2024-09-16T07:38:45.653 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:45.653 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":152,"fsid":"5821e025-6a94-4c05-a5f1-d6094eac5c68","created":"2024-09-16T07:22:23.112588+0000","modified":"2024-09-16T07:38:41.092179+0000","last_up_change":"2024-09-16T07:22:37.838123+0000","last_in_change":"2024-09-16T07:22:27.312216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T07:22:38.319708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9b13ba3-d953-4d09-90fc-c07a6d703d34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6801","nonce":207466128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6803","nonce":207466128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6807","nonce":207466128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":207466128},{"type":"v1","addr":"172.21.15.71:6805","nonce":207466128}]},"public_addr":"172.21.15.71:6801/207466128","cluster_addr":"172.21.15.71:6803/207466128","heartbeat_back_addr":"172.21.15.71:6807/207466128","heartbeat_front_addr":"172.21.15.71:6805/207466128","state":["exists","up"]},{"osd":1,"uuid":"9885ba32-5e7a-4af5-b083-eeadd8847f47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6817","nonce":1940414636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6819","nonce":1940414636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6823","nonce":1940414636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1940414636},{"type":"v1","addr":"172.21.15.71:6821","nonce":1940414636}]},"public_addr":"172.21.15.71:6817/1940414636","cluster_addr":"172.21.15.71:6819/1940414636","heartbeat_back_addr":"172.21.15.71:6823/1940414636","heartbeat_front_addr":"172.21.15.71:6821/1940414636","state":["exists","up"]},{"osd":2,"uuid":"6bd108a5-715d-4894-b6c9-ffb9c4b8aaae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6825","nonce":3801736779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6827","nonce":3801736779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6831","nonce":3801736779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3801736779},{"type":"v1","addr":"172.21.15.71:6829","nonce":3801736779}]},"public_addr":"172.21.15.71:6825/3801736779","cluster_addr":"172.21.15.71:6827/3801736779","heartbeat_back_addr":"172.21.15.71:6831/3801736779","heartbeat_front_addr":"172.21.15.71:6829/3801736779","state":["exists","up"]},{"osd":3,"uuid":"557e4e8f-f744-46b6-9d7e-81ca39b7f2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6809","nonce":1022096187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6811","nonce":1022096187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6815","nonce":1022096187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1022096187},{"type":"v1","addr":"172.21.15.71:6813","nonce":1022096187}]},"public_addr":"172.21.15.71:6809/1022096187","cluster_addr":"172.21.15.71:6811/1022096187","heartbeat_back_addr":"172.21.15.71:6815/1022096187","heartbeat_front_addr":"172.21.15.71:6813/1022096187","state":["exists","up"]},{"osd":4,"uuid":"91f8990b-1328-48a3-aa70-6be7326adfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6803","nonce":4258128961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6805","nonce":4258128961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6809","nonce":4258128961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":4258128961},{"type":"v1","addr":"172.21.15.177:6807","nonce":4258128961}]},"public_addr":"172.21.15.177:6803/4258128961","cluster_addr":"172.21.15.177:6805/4258128961","heartbeat_back_addr":"172.21.15.177:6809/4258128961","heartbeat_front_addr":"172.21.15.177:6807/4258128961","state":["exists","up"]},{"osd":5,"uuid":"9bec3138-a06c-4a25-a0b2-dbe299fb2db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6825","nonce":1078185550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6827","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6829","nonce":1078185550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6833","nonce":1078185550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1078185550},{"type":"v1","addr":"172.21.15.177:6831","nonce":1078185550}]},"public_addr":"172.21.15.177:6825/1078185550","cluster_addr":"172.21.15.177:6829/1078185550","heartbeat_back_addr":"172.21.15.177:6833/1078185550","heartbeat_front_addr":"172.21.15.177:6831/1078185550","state":["exists","up"]},{"osd":6,"uuid":"12a9547a-4112-4da5-8392-a6af10dccf47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6819","nonce":1024130870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6821","nonce":1024130870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6828","nonce":1024130870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":1024130870},{"type":"v1","addr":"172.21.15.177:6824","nonce":1024130870}]},"public_addr":"172.21.15.177:6819/1024130870","cluster_addr":"172.21.15.177:6821/1024130870","heartbeat_back_addr":"172.21.15.177:6828/1024130870","heartbeat_front_addr":"172.21.15.177:6824/1024130870","state":["exists","up"]},{"osd":7,"uuid":"07b5fcb7-d68d-4137-9fbd-688db353bed2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6811","nonce":4227809343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6813","nonce":4227809343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6817","nonce":4227809343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4227809343},{"type":"v1","addr":"172.21.15.177:6815","nonce":4227809343}]},"public_addr":"172.21.15.177:6811/4227809343","cluster_addr":"172.21.15.177:6813/4227809343","heartbeat_back_addr":"172.21.15.177:6817/4227809343","heartbeat_front_addr":"172.21.15.177:6815/4227809343","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.869411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.115170+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.124548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.412607+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:33.545013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.570031+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.310357+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T07:22:34.319846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6837/1486935311":"2024-09-17T07:38:38.086397+0000","172.21.15.177:6835/1486935311":"2024-09-17T07:38:38.086397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e1c07af20 msgr2=0x7f4e1c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e1c07af20 0x7f4e1c07d3e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c15dda0 tx=0x7f4e28005f90 comp rx=0 tx=0).stop 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b28d0 msgr2=0x7f4e3c14d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b28d0 0x7f4e3c14d430 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000c430 tx=0x7f4e3000c8f0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 shutdown_connections 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4e1c07af20 0x7f4e1c07d3e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e3c0d1ec0 0x7f4e3c0cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e3c0b4d10 0x7f4e3c0cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 --2- 172.21.15.71:0/4076864216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c0b28d0 0x7f4e3c14d430 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 >> 172.21.15.71:0/4076864216 conn(0x7f4e3c0d3de0 msgr2=0x7f4e3c06dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 shutdown_connections 2024-09-16T07:38:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:45.656+0000 7f4e42200640 1 -- 172.21.15.71:0/4076864216 wait complete. 2024-09-16T07:38:46.672 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T07:38:46.673 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T07:38:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.844+0000 7f4dd5600640 1 -- 172.21.15.71:0/2756493214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd006e6f0 msgr2=0x7f4dd006eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.844+0000 7f4dd5600640 1 --2- 172.21.15.71:0/2756493214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd006e6f0 0x7f4dd006eb20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f4dd006aa00 tx=0x7f4dc802a560 comp rx=0 tx=0).stop 2024-09-16T07:38:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.844+0000 7f4dd5600640 1 -- 172.21.15.71:0/2756493214 shutdown_connections 2024-09-16T07:38:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.844+0000 7f4dd5600640 1 --2- 172.21.15.71:0/2756493214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd006e6f0 0x7f4dd006eb20 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.844+0000 7f4dd5600640 1 --2- 172.21.15.71:0/2756493214 >> v2:172.21.15.177:3300/0 conn(0x7f4dd0158900 0x7f4dd015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.844+0000 7f4dd5600640 1 -- 172.21.15.71:0/2756493214 >> 172.21.15.71:0/2756493214 conn(0x7f4dd00b18e0 msgr2=0x7f4dd00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.844+0000 7f4dd5600640 1 -- 172.21.15.71:0/2756493214 shutdown_connections 2024-09-16T07:38:46.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.844+0000 7f4dd5600640 1 -- 172.21.15.71:0/2756493214 wait complete. 2024-09-16T07:38:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.846+0000 7f4dd5600640 1 Processor -- start 2024-09-16T07:38:46.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.846+0000 7f4dd5600640 1 -- start start 2024-09-16T07:38:46.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dd5600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dd006e6f0 0x7f4dd009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dd5600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd0158900 0x7f4dd0148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dd5600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dd015b700 0x7f4dd0159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dd5600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd00ced60 con 0x7f4dd0158900 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dd5600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd00cebe0 con 0x7f4dd015b700 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dd5600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd00ceee0 con 0x7f4dd006e6f0 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dce000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd0158900 0x7f4dd0148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dce000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd0158900 0x7f4dd0148600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36248/0 (socket says 172.21.15.71:36248) 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dce000640 1 -- 172.21.15.71:0/3777021823 learned_addr learned my addr 172.21.15.71:0/3777021823 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dcf400640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dd015b700 0x7f4dd0159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dcea00640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dd006e6f0 0x7f4dd009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dce000640 1 -- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dd006e6f0 msgr2=0x7f4dd009cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:46.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dce000640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dd006e6f0 0x7f4dd009cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:46.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dce000640 1 -- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dd015b700 msgr2=0x7f4dd0159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:46.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dce000640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dd015b700 0x7f4dd0159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:46.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.847+0000 7f4dce000640 1 -- 172.21.15.71:0/3777021823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd009d530 con 0x7f4dd0158900 2024-09-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.848+0000 7f4dce000640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd0158900 0x7f4dd0148600 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00b3f0 tx=0x7f4dbc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.848+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc015020 con 0x7f4dd0158900 2024-09-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.848+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dbc011920 con 0x7f4dd0158900 2024-09-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.848+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd009d810 con 0x7f4dd0158900 2024-09-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.848+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc011d70 con 0x7f4dd0158900 2024-09-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.848+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd009ddd0 con 0x7f4dd0158900 2024-09-16T07:38:46.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.849+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4dd00cf3f0 con 0x7f4dd0158900 2024-09-16T07:38:46.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.851+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f4dbc018020 con 0x7f4dd0158900 2024-09-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.851+0000 7f4dc6a00640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4db007af20 0x7f4db007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.852+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f4dbc099e40 con 0x7f4dd0158900 2024-09-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.852+0000 7f4dc6a00640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7f4db0080e40 0x7f4db00832c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.852+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 --> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4db0083990 con 0x7f4db0080e40 2024-09-16T07:38:46.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.852+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc09a4d0 con 0x7f4dd0158900 2024-09-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.852+0000 7f4dcea00640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4db007af20 0x7f4db007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.852+0000 7f4dcf400640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7f4db0080e40 0x7f4db00832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.853+0000 7f4dcf400640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7f4db0080e40 0x7f4db00832c0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:46.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.853+0000 7f4dcea00640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4db007af20 0x7f4db007d3e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0005fd0 tx=0x7f4dc0009870 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:46.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.853+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 <== osd.0 v2:172.21.15.71:6800/207466128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4db0083990 con 0x7f4db0080e40 2024-09-16T07:38:46.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 --> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4dd015aba0 con 0x7f4db0080e40 2024-09-16T07:38:46.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dc6a00640 1 -- 172.21.15.71:0/3777021823 <== osd.0 v2:172.21.15.71:6800/207466128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4dd015aba0 con 0x7f4db0080e40 2024-09-16T07:38:46.867 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:46.867 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T07:38:46.867 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7f4db0080e40 msgr2=0x7f4db00832c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dd5600640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.71:6800/207466128,v1:172.21.15.71:6801/207466128] conn(0x7f4db0080e40 0x7f4db00832c0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4db007af20 msgr2=0x7f4db007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dd5600640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f4db007af20 0x7f4db007d3e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0005fd0 tx=0x7f4dc0009870 comp rx=0 tx=0).stop 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd0158900 msgr2=0x7f4dd0148600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dd5600640 1 --2- 172.21.15.71:0/3777021823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dd0158900 0x7f4dd0148600 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00b3f0 tx=0x7f4dbc00b8b0 comp rx=0 tx=0).stop 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.867+0000 7f4dcf400640 1 -- 172.21.15.71:0/3777021823 reap_dead start 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.868+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 shutdown_connections 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.868+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 >> 172.21.15.71:0/3777021823 conn(0x7f4dd00b18e0 msgr2=0x7f4dd00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.868+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 shutdown_connections 2024-09-16T07:38:46.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:46.868+0000 7f4dd5600640 1 -- 172.21.15.71:0/3777021823 wait complete. 2024-09-16T07:38:46.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T07:38:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.041+0000 7f276c400640 1 -- 172.21.15.71:0/156420926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f274c004d40 msgr2=0x7f274c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.041+0000 7f276c400640 1 --2- 172.21.15.71:0/156420926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f274c004d40 0x7f274c0051c0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f275c001e10 tx=0x7f274c005830 comp rx=0 tx=0).stop 2024-09-16T07:38:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.041+0000 7f276a400640 1 -- 172.21.15.71:0/156420926 reap_dead start 2024-09-16T07:38:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.041+0000 7f276c400640 1 -- 172.21.15.71:0/156420926 shutdown_connections 2024-09-16T07:38:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.041+0000 7f276c400640 1 -- 172.21.15.71:0/156420926 >> 172.21.15.71:0/156420926 conn(0x7f27640b1950 msgr2=0x7f27640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.041+0000 7f276c400640 1 -- 172.21.15.71:0/156420926 shutdown_connections 2024-09-16T07:38:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.042+0000 7f276c400640 1 -- 172.21.15.71:0/156420926 wait complete. 2024-09-16T07:38:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.043+0000 7f276c400640 1 Processor -- start 2024-09-16T07:38:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.043+0000 7f276c400640 1 -- start start 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f276c400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f274c004d40 0x7f27641f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f276c400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27641f66f0 0x7f276413c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f276c400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276415e770 0x7f27640cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f276c400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f27640729a0 con 0x7f27641f66f0 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f276c400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f27640b4eb0 con 0x7f274c004d40 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f276c400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2764072b20 con 0x7f276415e770 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f2769000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27641f66f0 0x7f276413c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f2769000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27641f66f0 0x7f276413c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36270/0 (socket says 172.21.15.71:36270) 2024-09-16T07:38:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.044+0000 7f2769000640 1 -- 172.21.15.71:0/832853337 learned_addr learned my addr 172.21.15.71:0/832853337 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2769a00640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f274c004d40 0x7f27641f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f276a400640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276415e770 0x7f27640cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2769000640 1 -- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276415e770 msgr2=0x7f27640cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2769000640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276415e770 0x7f27640cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2769000640 1 -- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f274c004d40 msgr2=0x7f27641f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2769000640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f274c004d40 0x7f27641f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2769000640 1 -- 172.21.15.71:0/832853337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27641f3630 con 0x7f27641f66f0 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2769a00640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f274c004d40 0x7f27641f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f276a400640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276415e770 0x7f27640cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2769000640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27641f66f0 0x7f276413c780 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f2750011ab0 tx=0x7f2750011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2761600640 1 -- 172.21.15.71:0/832853337 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275001e070 con 0x7f27641f66f0 2024-09-16T07:38:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2761600640 1 -- 172.21.15.71:0/832853337 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f275000fc70 con 0x7f27641f66f0 2024-09-16T07:38:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f2761600640 1 -- 172.21.15.71:0/832853337 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750019670 con 0x7f27641f66f0 2024-09-16T07:38:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276413f2b0 con 0x7f27641f66f0 2024-09-16T07:38:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.045+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27641f2d90 con 0x7f27641f66f0 2024-09-16T07:38:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.047+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27640cebf0 con 0x7f27641f66f0 2024-09-16T07:38:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.048+0000 7f2761600640 1 -- 172.21.15.71:0/832853337 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f2750021070 con 0x7f27641f66f0 2024-09-16T07:38:47.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.049+0000 7f2761600640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f274407af20 0x7f274407d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.049+0000 7f2761600640 1 -- 172.21.15.71:0/832853337 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f275009a540 con 0x7f27641f66f0 2024-09-16T07:38:47.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.049+0000 7f2769a00640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f274407af20 0x7f274407d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.050+0000 7f2769a00640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f274407af20 0x7f274407d3e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2758006bf0 tx=0x7f27580067a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:47.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.053+0000 7f2761600640 1 -- 172.21.15.71:0/832853337 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f275000fe60 con 0x7f27641f66f0 2024-09-16T07:38:47.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.242+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f2764070a20 con 0x7f274407af20 2024-09-16T07:38:47.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.243+0000 7f2761600640 1 -- 172.21.15.71:0/832853337 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2764070a20 con 0x7f274407af20 2024-09-16T07:38:47.242 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T07:38:47.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.245+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f274407af20 msgr2=0x7f274407d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.245+0000 7f276c400640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f274407af20 0x7f274407d3e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2758006bf0 tx=0x7f27580067a0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.245+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27641f66f0 msgr2=0x7f276413c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.245+0000 7f276c400640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27641f66f0 0x7f276413c780 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f2750011ab0 tx=0x7f2750011f70 comp rx=0 tx=0).stop 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.246+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 shutdown_connections 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.246+0000 7f276c400640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f274407af20 0x7f274407d3e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.246+0000 7f276c400640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276415e770 0x7f27640cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.246+0000 7f276c400640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27641f66f0 0x7f276413c780 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.246+0000 7f276c400640 1 --2- 172.21.15.71:0/832853337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f274c004d40 0x7f27641f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.246+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 >> 172.21.15.71:0/832853337 conn(0x7f27640b1950 msgr2=0x7f27640acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.246+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 shutdown_connections 2024-09-16T07:38:47.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.246+0000 7f276c400640 1 -- 172.21.15.71:0/832853337 wait complete. 2024-09-16T07:38:47.261 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T07:38:47.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T07:38:47.358 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:38:47.362 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:38:47.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.428+0000 7f1313e00640 1 -- 172.21.15.71:0/981041524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 msgr2=0x7f13140d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.428+0000 7f1313e00640 1 --2- 172.21.15.71:0/981041524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 0x7f13140d4a20 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f13140d2c60 tx=0x7f130402a5f0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.428+0000 7f1313e00640 1 -- 172.21.15.71:0/981041524 shutdown_connections 2024-09-16T07:38:47.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.428+0000 7f1313e00640 1 --2- 172.21.15.71:0/981041524 >> v2:172.21.15.177:3300/0 conn(0x7f131415eaf0 0x7f131415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.428+0000 7f1313e00640 1 --2- 172.21.15.71:0/981041524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 0x7f13140d4a20 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.428+0000 7f1313e00640 1 -- 172.21.15.71:0/981041524 >> 172.21.15.71:0/981041524 conn(0x7f13140d3f00 msgr2=0x7f13140d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:47.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.428+0000 7f1313e00640 1 -- 172.21.15.71:0/981041524 shutdown_connections 2024-09-16T07:38:47.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.428+0000 7f1313e00640 1 -- 172.21.15.71:0/981041524 wait complete. 2024-09-16T07:38:47.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.430+0000 7f1313e00640 1 Processor -- start 2024-09-16T07:38:47.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.430+0000 7f1313e00640 1 -- start start 2024-09-16T07:38:47.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f1313e00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13140d08d0 0x7f13140cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f1313e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 0x7f13140b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f1313e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f131415eaf0 0x7f13141f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f1313e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f13140d1e60 con 0x7f131415e710 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f1313e00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f13140d1ce0 con 0x7f13140d08d0 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f1313e00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f13140d1fe0 con 0x7f131415eaf0 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f130be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 0x7f13140b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f130be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 0x7f13140b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36282/0 (socket says 172.21.15.71:36282) 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f130be00640 1 -- 172.21.15.71:0/2684895550 learned_addr learned my addr 172.21.15.71:0/2684895550 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f1312a00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13140d08d0 0x7f13140cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f1313400640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f131415eaf0 0x7f13141f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f130be00640 1 -- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f131415eaf0 msgr2=0x7f13141f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f130be00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f131415eaf0 0x7f13141f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f130be00640 1 -- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13140d08d0 msgr2=0x7f13140cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f130be00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13140d08d0 0x7f13140cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.431+0000 7f130be00640 1 -- 172.21.15.71:0/2684895550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13141f5480 con 0x7f131415e710 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.432+0000 7f130be00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 0x7f13140b1d30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00d950 tx=0x7f12fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.432+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc017070 con 0x7f131415e710 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.432+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12fc009d20 con 0x7f131415e710 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.432+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13141f5760 con 0x7f131415e710 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.432+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0106c0 con 0x7f131415e710 2024-09-16T07:38:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.432+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13140cb4c0 con 0x7f131415e710 2024-09-16T07:38:47.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.433+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13141f8200 con 0x7f131415e710 2024-09-16T07:38:47.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.436+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f12fc010860 con 0x7f131415e710 2024-09-16T07:38:47.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.436+0000 7f130aa00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12f007af20 0x7f12f007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.437+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f12fc09a080 con 0x7f131415e710 2024-09-16T07:38:47.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.437+0000 7f130aa00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f12f0080e40 0x7f12f00832c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.437+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 --> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12fc020e80 con 0x7f12f0080e40 2024-09-16T07:38:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.437+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f12fc0a6050 con 0x7f131415e710 2024-09-16T07:38:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.437+0000 7f1312a00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12f007af20 0x7f12f007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.437+0000 7f1313400640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f12f0080e40 0x7f12f00832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.438+0000 7f1313400640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f12f0080e40 0x7f12f00832c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:47.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.438+0000 7f1312a00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12f007af20 0x7f12f007d3e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1304033000 tx=0x7f1304004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:47.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.438+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 <== osd.1 v2:172.21.15.71:6816/1940414636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12fc020e80 con 0x7f12f0080e40 2024-09-16T07:38:47.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.452+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 --> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f13140d4a20 con 0x7f12f0080e40 2024-09-16T07:38:47.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f130aa00640 1 -- 172.21.15.71:0/2684895550 <== osd.1 v2:172.21.15.71:6816/1940414636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f13140d4a20 con 0x7f12f0080e40 2024-09-16T07:38:47.453 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:47.453 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T07:38:47.453 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:47.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f12f0080e40 msgr2=0x7f12f00832c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.71:6816/1940414636,v1:172.21.15.71:6817/1940414636] conn(0x7f12f0080e40 0x7f12f00832c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12f007af20 msgr2=0x7f12f007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f12f007af20 0x7f12f007d3e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1304033000 tx=0x7f1304004730 comp rx=0 tx=0).stop 2024-09-16T07:38:47.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 msgr2=0x7f13140b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 --2- 172.21.15.71:0/2684895550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f131415e710 0x7f13140b1d30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00d950 tx=0x7f12fc00de10 comp rx=0 tx=0).stop 2024-09-16T07:38:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313400640 1 -- 172.21.15.71:0/2684895550 reap_dead start 2024-09-16T07:38:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 shutdown_connections 2024-09-16T07:38:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 >> 172.21.15.71:0/2684895550 conn(0x7f13140d3f00 msgr2=0x7f131415d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 shutdown_connections 2024-09-16T07:38:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.453+0000 7f1313e00640 1 -- 172.21.15.71:0/2684895550 wait complete. 2024-09-16T07:38:47.468 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T07:38:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.671+0000 7f5635400640 1 -- 172.21.15.71:0/1985575726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b4020 msgr2=0x7f56300b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.671+0000 7f5635400640 1 --2- 172.21.15.71:0/1985575726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b4020 0x7f56300b1770 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f56300d4ea0 tx=0x7f562802a9f0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.671+0000 7f5635400640 1 -- 172.21.15.71:0/1985575726 shutdown_connections 2024-09-16T07:38:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.671+0000 7f5635400640 1 --2- 172.21.15.71:0/1985575726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b4020 0x7f56300b1770 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.671+0000 7f5635400640 1 --2- 172.21.15.71:0/1985575726 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5630161440 0x7f56300b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.671+0000 7f5635400640 1 -- 172.21.15.71:0/1985575726 >> 172.21.15.71:0/1985575726 conn(0x7f56300d3df0 msgr2=0x7f56300d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.671+0000 7f5635400640 1 -- 172.21.15.71:0/1985575726 shutdown_connections 2024-09-16T07:38:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.671+0000 7f5635400640 1 -- 172.21.15.71:0/1985575726 wait complete. 2024-09-16T07:38:47.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.673+0000 7f5635400640 1 Processor -- start 2024-09-16T07:38:47.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.673+0000 7f5635400640 1 -- start start 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f5635400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b34b0 0x7f5630157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f5635400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56300b4020 0x7f56300caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f5635400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5630161440 0x7f56301f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f5635400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f56300b2040 con 0x7f56300b34b0 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f5635400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56300b1ec0 con 0x7f5630161440 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f5635400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f56300b21c0 con 0x7f56300b4020 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f562fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b34b0 0x7f5630157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f562fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b34b0 0x7f5630157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36290/0 (socket says 172.21.15.71:36290) 2024-09-16T07:38:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f562fe00640 1 -- 172.21.15.71:0/318144959 learned_addr learned my addr 172.21.15.71:0/318144959 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:47.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f5627e00640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56300b4020 0x7f56300caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.674+0000 7f5634a00640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5630161440 0x7f56301f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562fe00640 1 -- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56300b4020 msgr2=0x7f56300caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562fe00640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56300b4020 0x7f56300caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562fe00640 1 -- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5630161440 msgr2=0x7f56301f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562fe00640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5630161440 0x7f56301f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562fe00640 1 -- 172.21.15.71:0/318144959 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56300cb400 con 0x7f56300b34b0 2024-09-16T07:38:47.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562fe00640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b34b0 0x7f5630157fc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f562000e790 tx=0x7f562000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:47.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562cc00640 1 -- 172.21.15.71:0/318144959 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562000eea0 con 0x7f56300b34b0 2024-09-16T07:38:47.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562cc00640 1 -- 172.21.15.71:0/318144959 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f562000cdb0 con 0x7f56300b34b0 2024-09-16T07:38:47.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56300c8460 con 0x7f56300b34b0 2024-09-16T07:38:47.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f562cc00640 1 -- 172.21.15.71:0/318144959 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620010690 con 0x7f56300b34b0 2024-09-16T07:38:47.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.675+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5630156cb0 con 0x7f56300b34b0 2024-09-16T07:38:47.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.677+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56300b1770 con 0x7f56300b34b0 2024-09-16T07:38:47.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.678+0000 7f562cc00640 1 -- 172.21.15.71:0/318144959 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f5620010830 con 0x7f56300b34b0 2024-09-16T07:38:47.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.679+0000 7f562cc00640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f560c07af50 0x7f560c07d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:47.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.679+0000 7f562cc00640 1 -- 172.21.15.71:0/318144959 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f562009a540 con 0x7f56300b34b0 2024-09-16T07:38:47.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.679+0000 7f5627e00640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f560c07af50 0x7f560c07d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:47.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.680+0000 7f5627e00640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f560c07af50 0x7f560c07d410 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f563015f510 tx=0x7f5618009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:47.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.683+0000 7f562cc00640 1 -- 172.21.15.71:0/318144959 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5620063fc0 con 0x7f56300b34b0 2024-09-16T07:38:47.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.894+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f56300c8210 con 0x7f560c07af50 2024-09-16T07:38:47.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.895+0000 7f562cc00640 1 -- 172.21.15.71:0/318144959 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f56300c8210 con 0x7f560c07af50 2024-09-16T07:38:47.894 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f560c07af50 msgr2=0x7f560c07d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f560c07af50 0x7f560c07d410 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f563015f510 tx=0x7f5618009040 comp rx=0 tx=0).stop 2024-09-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b34b0 msgr2=0x7f5630157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b34b0 0x7f5630157fc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f562000e790 tx=0x7f562000ec50 comp rx=0 tx=0).stop 2024-09-16T07:38:47.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 shutdown_connections 2024-09-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f560c07af50 0x7f560c07d410 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5630161440 0x7f56301f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56300b4020 0x7f56300caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 --2- 172.21.15.71:0/318144959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56300b34b0 0x7f5630157fc0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.897+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 >> 172.21.15.71:0/318144959 conn(0x7f56300d3df0 msgr2=0x7f56300d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.898+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 shutdown_connections 2024-09-16T07:38:47.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:47.898+0000 7f5635400640 1 -- 172.21.15.71:0/318144959 wait complete. 2024-09-16T07:38:47.915 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T07:38:47.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T07:38:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.085+0000 7fb42be00640 1 -- 172.21.15.71:0/1253206155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c1623b0 msgr2=0x7fb42c0d47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.085+0000 7fb42be00640 1 --2- 172.21.15.71:0/1253206155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c1623b0 0x7fb42c0d47e0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fb42c09d850 tx=0x7fb41c02a5f0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.085+0000 7fb42be00640 1 -- 172.21.15.71:0/1253206155 shutdown_connections 2024-09-16T07:38:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.085+0000 7fb42be00640 1 --2- 172.21.15.71:0/1253206155 >> v2:172.21.15.177:3300/0 conn(0x7fb42c0d2920 0x7fb42c0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.085+0000 7fb42be00640 1 --2- 172.21.15.71:0/1253206155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c1623b0 0x7fb42c0d47e0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.085+0000 7fb42be00640 1 -- 172.21.15.71:0/1253206155 >> 172.21.15.71:0/1253206155 conn(0x7fb42c09d180 msgr2=0x7fb42c0d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.085+0000 7fb42be00640 1 -- 172.21.15.71:0/1253206155 shutdown_connections 2024-09-16T07:38:48.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.085+0000 7fb42be00640 1 -- 172.21.15.71:0/1253206155 wait complete. 2024-09-16T07:38:48.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.087+0000 7fb42be00640 1 Processor -- start 2024-09-16T07:38:48.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.087+0000 7fb42be00640 1 -- start start 2024-09-16T07:38:48.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42c0b5200 0x7fb42c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c0d2920 0x7fb42c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb42c1623b0 0x7fb42c1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c0b2230 con 0x7fb42c0d2920 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c0b20b0 con 0x7fb42c1623b0 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb42c0b23b0 con 0x7fb42c0b5200 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c0d2920 0x7fb42c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c0d2920 0x7fb42c158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36300/0 (socket says 172.21.15.71:36300) 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42a000640 1 -- 172.21.15.71:0/2606863070 learned_addr learned my addr 172.21.15.71:0/2606863070 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42b400640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb42c1623b0 0x7fb42c1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.088+0000 7fb42aa00640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42c0b5200 0x7fb42c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb42a000640 1 -- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42c0b5200 msgr2=0x7fb42c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb42a000640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42c0b5200 0x7fb42c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb42a000640 1 -- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb42c1623b0 msgr2=0x7fb42c1592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb42a000640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb42c1623b0 0x7fb42c1592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb42a000640 1 -- 172.21.15.71:0/2606863070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c1f93f0 con 0x7fb42c0d2920 2024-09-16T07:38:48.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb42a000640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c0d2920 0x7fb42c158d60 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c450 tx=0x7fb41800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180052d0 con 0x7fb42c0d2920 2024-09-16T07:38:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb418005470 con 0x7fb42c0d2920 2024-09-16T07:38:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418012870 con 0x7fb42c0d2920 2024-09-16T07:38:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42c1598d0 con 0x7fb42c0d2920 2024-09-16T07:38:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.089+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42c1f8910 con 0x7fb42c0d2920 2024-09-16T07:38:48.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.090+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb42c0b1960 con 0x7fb42c0d2920 2024-09-16T07:38:48.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.093+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fb4180035f0 con 0x7fb42c0d2920 2024-09-16T07:38:48.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.094+0000 7fb422a00640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40c07af20 0x7fb40c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.094+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7fb418099640 con 0x7fb42c0d2920 2024-09-16T07:38:48.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.094+0000 7fb422a00640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb40c080e40 0x7fb40c0832c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.094+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 --> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb40c083990 con 0x7fb40c080e40 2024-09-16T07:38:48.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.094+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fb4180a5050 con 0x7fb42c0d2920 2024-09-16T07:38:48.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.094+0000 7fb42aa00640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40c07af20 0x7fb40c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.095+0000 7fb42b400640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb40c080e40 0x7fb40c0832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.095+0000 7fb42aa00640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40c07af20 0x7fb40c07d3e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb41c033000 tx=0x7fb41c004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.095+0000 7fb42b400640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb40c080e40 0x7fb40c0832c0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.096+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 <== osd.2 v2:172.21.15.71:6824/3801736779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb40c083990 con 0x7fb40c080e40 2024-09-16T07:38:48.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.109+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 --> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb42c0cab60 con 0x7fb40c080e40 2024-09-16T07:38:48.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.109+0000 7fb422a00640 1 -- 172.21.15.71:0/2606863070 <== osd.2 v2:172.21.15.71:6824/3801736779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb42c0cab60 con 0x7fb40c080e40 2024-09-16T07:38:48.109 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:48.109 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T07:38:48.109 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb40c080e40 msgr2=0x7fb40c0832c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.71:6824/3801736779,v1:172.21.15.71:6825/3801736779] conn(0x7fb40c080e40 0x7fb40c0832c0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40c07af20 msgr2=0x7fb40c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb40c07af20 0x7fb40c07d3e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb41c033000 tx=0x7fb41c004730 comp rx=0 tx=0).stop 2024-09-16T07:38:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c0d2920 msgr2=0x7fb42c158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 --2- 172.21.15.71:0/2606863070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb42c0d2920 0x7fb42c158d60 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c450 tx=0x7fb41800c910 comp rx=0 tx=0).stop 2024-09-16T07:38:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42b400640 1 -- 172.21.15.71:0/2606863070 reap_dead start 2024-09-16T07:38:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 shutdown_connections 2024-09-16T07:38:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 >> 172.21.15.71:0/2606863070 conn(0x7fb42c09d180 msgr2=0x7fb42c0caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 shutdown_connections 2024-09-16T07:38:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.110+0000 7fb42be00640 1 -- 172.21.15.71:0/2606863070 wait complete. 2024-09-16T07:38:48.124 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T07:38:48.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.288+0000 7ffafce00640 1 -- 172.21.15.71:0/2252708406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 msgr2=0x7ffadc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.288+0000 7ffafce00640 1 --2- 172.21.15.71:0/2252708406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 0x7ffadc003e00 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ffae8006f70 tx=0x7ffadc005830 comp rx=0 tx=0).stop 2024-09-16T07:38:48.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.288+0000 7ffafce00640 1 -- 172.21.15.71:0/2252708406 shutdown_connections 2024-09-16T07:38:48.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.288+0000 7ffafce00640 1 --2- 172.21.15.71:0/2252708406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 0x7ffadc003e00 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.288+0000 7ffafce00640 1 -- 172.21.15.71:0/2252708406 >> 172.21.15.71:0/2252708406 conn(0x7ffaf80d3df0 msgr2=0x7ffaf80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:48.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.288+0000 7ffafce00640 1 -- 172.21.15.71:0/2252708406 shutdown_connections 2024-09-16T07:38:48.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.288+0000 7ffafce00640 1 -- 172.21.15.71:0/2252708406 wait complete. 2024-09-16T07:38:48.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.290+0000 7ffafce00640 1 Processor -- start 2024-09-16T07:38:48.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.290+0000 7ffafce00640 1 -- start start 2024-09-16T07:38:48.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffafce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 0x7ffaf81f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffafce00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaf80b7850 0x7ffaf81f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffafce00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf80cbd30 0x7ffaf806e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffafce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf80b1b00 con 0x7ffadc003a00 2024-09-16T07:38:48.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffafce00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf80b1980 con 0x7ffaf80cbd30 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffafce00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf80b1c80 con 0x7ffaf80b7850 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffaf7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 0x7ffaf81f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffaf7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 0x7ffaf81f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36320/0 (socket says 172.21.15.71:36320) 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffaf7400640 1 -- 172.21.15.71:0/3565410746 learned_addr learned my addr 172.21.15.71:0/3565410746 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffaf7e00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf80cbd30 0x7ffaf806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.291+0000 7ffaf6a00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaf80b7850 0x7ffaf81f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaf7400640 1 -- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaf80b7850 msgr2=0x7ffaf81f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaf7400640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaf80b7850 0x7ffaf81f74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaf7400640 1 -- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf80cbd30 msgr2=0x7ffaf806e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaf7400640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf80cbd30 0x7ffaf806e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaf7400640 1 -- 172.21.15.71:0/3565410746 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf80cc5c0 con 0x7ffadc003a00 2024-09-16T07:38:48.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaf7400640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 0x7ffaf81f45c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ffae8000c00 tx=0x7ffae80017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaef400640 1 -- 172.21.15.71:0/3565410746 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8006ba0 con 0x7ffadc003a00 2024-09-16T07:38:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaef400640 1 -- 172.21.15.71:0/3565410746 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffae803f030 con 0x7ffadc003a00 2024-09-16T07:38:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf80b8310 con 0x7ffadc003a00 2024-09-16T07:38:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffaef400640 1 -- 172.21.15.71:0/3565410746 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8001c10 con 0x7ffadc003a00 2024-09-16T07:38:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.292+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf80b88d0 con 0x7ffadc003a00 2024-09-16T07:38:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.294+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf80b2150 con 0x7ffadc003a00 2024-09-16T07:38:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.295+0000 7ffaef400640 1 -- 172.21.15.71:0/3565410746 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7ffae8003c80 con 0x7ffadc003a00 2024-09-16T07:38:48.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.296+0000 7ffaef400640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffad807af50 0x7ffad807d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.296+0000 7ffaef400640 1 -- 172.21.15.71:0/3565410746 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7ffae80ba6b0 con 0x7ffadc003a00 2024-09-16T07:38:48.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.296+0000 7ffaf6a00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffad807af50 0x7ffad807d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.297+0000 7ffaf6a00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffad807af50 0x7ffad807d410 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80cad10 tx=0x7ffae4006200 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.300+0000 7ffaef400640 1 -- 172.21.15.71:0/3565410746 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffae8049030 con 0x7ffadc003a00 2024-09-16T07:38:48.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.487+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7ffaf8160550 con 0x7ffad807af50 2024-09-16T07:38:48.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.488+0000 7ffaef400640 1 -- 172.21.15.71:0/3565410746 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffaf8160550 con 0x7ffad807af50 2024-09-16T07:38:48.487 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T07:38:48.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffad807af50 msgr2=0x7ffad807d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffad807af50 0x7ffad807d410 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80cad10 tx=0x7ffae4006200 comp rx=0 tx=0).stop 2024-09-16T07:38:48.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 msgr2=0x7ffaf81f45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 0x7ffaf81f45c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ffae8000c00 tx=0x7ffae80017a0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 shutdown_connections 2024-09-16T07:38:48.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7ffad807af50 0x7ffad807d410 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf80cbd30 0x7ffaf806e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaf80b7850 0x7ffaf81f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 --2- 172.21.15.71:0/3565410746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffadc003a00 0x7ffaf81f45c0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 >> 172.21.15.71:0/3565410746 conn(0x7ffaf80d3df0 msgr2=0x7ffaf815f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:48.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 shutdown_connections 2024-09-16T07:38:48.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.490+0000 7ffafce00640 1 -- 172.21.15.71:0/3565410746 wait complete. 2024-09-16T07:38:48.505 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T07:38:48.505 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T07:38:48.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.690+0000 7fec98600640 1 -- 172.21.15.71:0/3886659054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec78003a80 msgr2=0x7fec78003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.690+0000 7fec98600640 1 --2- 172.21.15.71:0/3886659054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec78003a80 0x7fec78003ea0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fec84001660 tx=0x7fec78005830 comp rx=0 tx=0).stop 2024-09-16T07:38:48.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.690+0000 7fec96600640 1 -- 172.21.15.71:0/3886659054 reap_dead start 2024-09-16T07:38:48.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.690+0000 7fec98600640 1 -- 172.21.15.71:0/3886659054 shutdown_connections 2024-09-16T07:38:48.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.690+0000 7fec98600640 1 -- 172.21.15.71:0/3886659054 >> 172.21.15.71:0/3886659054 conn(0x7fec900b18e0 msgr2=0x7fec900b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:48.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.691+0000 7fec98600640 1 -- 172.21.15.71:0/3886659054 shutdown_connections 2024-09-16T07:38:48.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.691+0000 7fec98600640 1 -- 172.21.15.71:0/3886659054 wait complete. 2024-09-16T07:38:48.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.692+0000 7fec98600640 1 Processor -- start 2024-09-16T07:38:48.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.692+0000 7fec98600640 1 -- start start 2024-09-16T07:38:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.693+0000 7fec98600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec78003a80 0x7fec9015e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.693+0000 7fec98600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90152790 0x7fec9014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.693+0000 7fec98600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec900cfb60 0x7fec900cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.693+0000 7fec98600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fec90073290 con 0x7fec90152790 2024-09-16T07:38:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.693+0000 7fec98600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fec900cd3f0 con 0x7fec78003a80 2024-09-16T07:38:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.693+0000 7fec98600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fec90073410 con 0x7fec900cfb60 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90152790 0x7fec9014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90152790 0x7fec9014eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36338/0 (socket says 172.21.15.71:36338) 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 -- 172.21.15.71:0/1785645721 learned_addr learned my addr 172.21.15.71:0/1785645721 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95c00640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec78003a80 0x7fec9015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec96600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec900cfb60 0x7fec900cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 -- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec900cfb60 msgr2=0x7fec900cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec900cfb60 0x7fec900cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 -- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec78003a80 msgr2=0x7fec9015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec78003a80 0x7fec9015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 -- 172.21.15.71:0/1785645721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9009bfe0 con 0x7fec90152790 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec96600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec900cfb60 0x7fec900cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.694+0000 7fec95200640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90152790 0x7fec9014eb90 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0018c0 tx=0x7fec7c00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.695+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c015070 con 0x7fec90152790 2024-09-16T07:38:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.695+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec7c0109a0 con 0x7fec90152790 2024-09-16T07:38:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.695+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c010df0 con 0x7fec90152790 2024-09-16T07:38:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.695+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec901f8720 con 0x7fec90152790 2024-09-16T07:38:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.695+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9009c250 con 0x7fec90152790 2024-09-16T07:38:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.696+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec90152ea0 con 0x7fec90152790 2024-09-16T07:38:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.697+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fec7c018070 con 0x7fec90152790 2024-09-16T07:38:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.698+0000 7fec8d600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec7007af20 0x7fec7007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.698+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7fec7c099fb0 con 0x7fec90152790 2024-09-16T07:38:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.698+0000 7fec8d600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7fec70080e40 0x7fec700832c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.698+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 --> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec7c01ce50 con 0x7fec70080e40 2024-09-16T07:38:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.698+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fec7c0a6050 con 0x7fec90152790 2024-09-16T07:38:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.699+0000 7fec96600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7fec70080e40 0x7fec700832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.699+0000 7fec95c00640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec7007af20 0x7fec7007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.699+0000 7fec96600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7fec70080e40 0x7fec700832c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.699+0000 7fec95c00640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec7007af20 0x7fec7007d3e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fec84032000 tx=0x7fec84001190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.700+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 <== osd.3 v2:172.21.15.71:6808/1022096187 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec7c01ce50 con 0x7fec70080e40 2024-09-16T07:38:48.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.713+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 --> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fec9009c670 con 0x7fec70080e40 2024-09-16T07:38:48.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.713+0000 7fec8d600640 1 -- 172.21.15.71:0/1785645721 <== osd.3 v2:172.21.15.71:6808/1022096187 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fec9009c670 con 0x7fec70080e40 2024-09-16T07:38:48.713 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:48.713 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T07:38:48.714 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:48.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7fec70080e40 msgr2=0x7fec700832c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.71:6808/1022096187,v1:172.21.15.71:6809/1022096187] conn(0x7fec70080e40 0x7fec700832c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec7007af20 msgr2=0x7fec7007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fec7007af20 0x7fec7007d3e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fec84032000 tx=0x7fec84001190 comp rx=0 tx=0).stop 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90152790 msgr2=0x7fec9014eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 --2- 172.21.15.71:0/1785645721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90152790 0x7fec9014eb90 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0018c0 tx=0x7fec7c00ea00 comp rx=0 tx=0).stop 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec96600640 1 -- 172.21.15.71:0/1785645721 reap_dead start 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 shutdown_connections 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 >> 172.21.15.71:0/1785645721 conn(0x7fec900b18e0 msgr2=0x7fec9015fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 shutdown_connections 2024-09-16T07:38:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.714+0000 7fec98600640 1 -- 172.21.15.71:0/1785645721 wait complete. 2024-09-16T07:38:48.729 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T07:38:48.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.891+0000 7fc24be00640 1 -- 172.21.15.71:0/650654207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc230004020 msgr2=0x7fc230004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.891+0000 7fc24be00640 1 --2- 172.21.15.71:0/650654207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc230004020 0x7fc230004800 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fc238009b50 tx=0x7fc230005830 comp rx=0 tx=0).stop 2024-09-16T07:38:48.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.891+0000 7fc24b400640 1 -- 172.21.15.71:0/650654207 reap_dead start 2024-09-16T07:38:48.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.891+0000 7fc24be00640 1 -- 172.21.15.71:0/650654207 shutdown_connections 2024-09-16T07:38:48.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.891+0000 7fc24be00640 1 -- 172.21.15.71:0/650654207 >> 172.21.15.71:0/650654207 conn(0x7fc24c0cfa20 msgr2=0x7fc24c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:48.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.891+0000 7fc24be00640 1 -- 172.21.15.71:0/650654207 shutdown_connections 2024-09-16T07:38:48.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.891+0000 7fc24be00640 1 -- 172.21.15.71:0/650654207 wait complete. 2024-09-16T07:38:48.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.893+0000 7fc24be00640 1 Processor -- start 2024-09-16T07:38:48.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.893+0000 7fc24be00640 1 -- start start 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24be00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc230004020 0x7fc24c0c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24c155390 0x7fc24c1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24be00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc24c1557f0 0x7fc24c1f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c15e7f0 con 0x7fc24c155390 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24be00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c15e670 con 0x7fc24c1557f0 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24be00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc24c15e970 con 0x7fc230004020 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24c155390 0x7fc24c1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24c155390 0x7fc24c1f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36356/0 (socket says 172.21.15.71:36356) 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.894+0000 7fc24a000640 1 -- 172.21.15.71:0/395473968 learned_addr learned my addr 172.21.15.71:0/395473968 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24aa00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc230004020 0x7fc24c0c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24b400640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc24c1557f0 0x7fc24c1f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24a000640 1 -- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc230004020 msgr2=0x7fc24c0c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24a000640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc230004020 0x7fc24c0c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24a000640 1 -- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc24c1557f0 msgr2=0x7fc24c1f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24a000640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc24c1557f0 0x7fc24c1f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24a000640 1 -- 172.21.15.71:0/395473968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24c155df0 con 0x7fc24c155390 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24a000640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24c155390 0x7fc24c1f7c70 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc238009a70 tx=0x7fc238033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc243400640 1 -- 172.21.15.71:0/395473968 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238038040 con 0x7fc24c155390 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc243400640 1 -- 172.21.15.71:0/395473968 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc238041070 con 0x7fc24c155390 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24c1560d0 con 0x7fc24c155390 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.895+0000 7fc243400640 1 -- 172.21.15.71:0/395473968 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238033df0 con 0x7fc24c155390 2024-09-16T07:38:48.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.896+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24c1f8930 con 0x7fc24c155390 2024-09-16T07:38:48.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.897+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24c1565f0 con 0x7fc24c155390 2024-09-16T07:38:48.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.898+0000 7fc243400640 1 -- 172.21.15.71:0/395473968 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fc238046460 con 0x7fc24c155390 2024-09-16T07:38:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.899+0000 7fc243400640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc22c07af20 0x7fc22c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.899+0000 7fc243400640 1 -- 172.21.15.71:0/395473968 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7fc2380ba450 con 0x7fc24c155390 2024-09-16T07:38:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.899+0000 7fc24aa00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc22c07af20 0x7fc22c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.900+0000 7fc24aa00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc22c07af20 0x7fc22c07d3e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc23c000fb0 tx=0x7fc23c006720 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:48.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:48.903+0000 7fc243400640 1 -- 172.21.15.71:0/395473968 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc238084520 con 0x7fc24c155390 2024-09-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.087+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fc24c15eeb0 con 0x7fc22c07af20 2024-09-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.088+0000 7fc243400640 1 -- 172.21.15.71:0/395473968 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc24c15eeb0 con 0x7fc22c07af20 2024-09-16T07:38:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc22c07af20 msgr2=0x7fc22c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc22c07af20 0x7fc22c07d3e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc23c000fb0 tx=0x7fc23c006720 comp rx=0 tx=0).stop 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24c155390 msgr2=0x7fc24c1f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24c155390 0x7fc24c1f7c70 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc238009a70 tx=0x7fc238033670 comp rx=0 tx=0).stop 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 shutdown_connections 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fc22c07af20 0x7fc22c07d3e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc24c1557f0 0x7fc24c1f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc24c155390 0x7fc24c1f7c70 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 --2- 172.21.15.71:0/395473968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc230004020 0x7fc24c0c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 >> 172.21.15.71:0/395473968 conn(0x7fc24c0cfa20 msgr2=0x7fc24c0734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 shutdown_connections 2024-09-16T07:38:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.090+0000 7fc24be00640 1 -- 172.21.15.71:0/395473968 wait complete. 2024-09-16T07:38:49.106 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T07:38:49.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T07:38:49.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.268+0000 7f8d01000640 1 -- 172.21.15.71:0/3205532867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 msgr2=0x7f8cfc0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.268+0000 7f8d01000640 1 --2- 172.21.15.71:0/3205532867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 0x7f8cfc0cc5b0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc06aa00 tx=0x7f8cf402a8c0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.268+0000 7f8d01000640 1 -- 172.21.15.71:0/3205532867 shutdown_connections 2024-09-16T07:38:49.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.268+0000 7f8d01000640 1 --2- 172.21.15.71:0/3205532867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 0x7f8cfc0cc5b0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.268+0000 7f8d01000640 1 --2- 172.21.15.71:0/3205532867 >> v2:172.21.15.177:3300/0 conn(0x7f8cfc15bd60 0x7f8cfc0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.268+0000 7f8d01000640 1 -- 172.21.15.71:0/3205532867 >> 172.21.15.71:0/3205532867 conn(0x7f8cfc0b18e0 msgr2=0x7f8cfc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:49.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.268+0000 7f8d01000640 1 -- 172.21.15.71:0/3205532867 shutdown_connections 2024-09-16T07:38:49.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.268+0000 7f8d01000640 1 -- 172.21.15.71:0/3205532867 wait complete. 2024-09-16T07:38:49.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.270+0000 7f8d01000640 1 Processor -- start 2024-09-16T07:38:49.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.270+0000 7f8d01000640 1 -- start start 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8d01000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cfc06e910 0x7f8cfc0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8d01000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 0x7f8cfc0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8d01000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cfc15bd60 0x7f8cfc09c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8d01000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0ccc70 con 0x7f8cfc074460 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8d01000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc0ccaf0 con 0x7f8cfc06e910 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8d01000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8cfc0ccdf0 con 0x7f8cfc15bd60 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8cf9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 0x7f8cfc0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8cf9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 0x7f8cfc0c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36382/0 (socket says 172.21.15.71:36382) 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.271+0000 7f8cf9400640 1 -- 172.21.15.71:0/1570800578 learned_addr learned my addr 172.21.15.71:0/1570800578 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf9400640 1 -- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cfc15bd60 msgr2=0x7f8cfc09c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf9400640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cfc15bd60 0x7f8cfc09c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf9400640 1 -- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cfc06e910 msgr2=0x7f8cfc0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf9400640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cfc06e910 0x7f8cfc0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf9400640 1 -- 172.21.15.71:0/1570800578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc09b9e0 con 0x7f8cfc074460 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf9400640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 0x7f8cfc0c9400 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800c480 tx=0x7f8ce800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8016020 con 0x7f8cfc074460 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ce8012920 con 0x7f8cfc074460 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8012d70 con 0x7f8cfc074460 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc0c9d70 con 0x7f8cfc074460 2024-09-16T07:38:49.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.272+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc09bca0 con 0x7f8cfc074460 2024-09-16T07:38:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.273+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cfc0cd310 con 0x7f8cfc074460 2024-09-16T07:38:49.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.275+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f8ce8019020 con 0x7f8cfc074460 2024-09-16T07:38:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.275+0000 7f8cf2000640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cdc07af50 0x7f8cdc07d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.276+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f8ce809a2d0 con 0x7f8cfc074460 2024-09-16T07:38:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.276+0000 7f8cf2000640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8cdc080ed0 0x7f8cdc083350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.276+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 --> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cdc083a20 con 0x7f8cdc080ed0 2024-09-16T07:38:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.276+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f8ce809a960 con 0x7f8cfc074460 2024-09-16T07:38:49.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.276+0000 7f8cf9e00640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cdc07af50 0x7f8cdc07d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.276+0000 7f8cfa800640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8cdc080ed0 0x7f8cdc083350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.277+0000 7f8cf9e00640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cdc07af50 0x7f8cdc07d410 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009670 tx=0x7f8cec006bf0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:49.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.277+0000 7f8cfa800640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8cdc080ed0 0x7f8cdc083350 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:49.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.278+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 <== osd.4 v2:172.21.15.177:6802/4258128961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8cdc083a20 con 0x7f8cdc080ed0 2024-09-16T07:38:49.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.291+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 --> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8cfc0cbcc0 con 0x7f8cdc080ed0 2024-09-16T07:38:49.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8cf2000640 1 -- 172.21.15.71:0/1570800578 <== osd.4 v2:172.21.15.177:6802/4258128961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8cfc0cbcc0 con 0x7f8cdc080ed0 2024-09-16T07:38:49.291 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:49.292 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T07:38:49.292 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8cdc080ed0 msgr2=0x7f8cdc083350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8d01000640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6802/4258128961,v1:172.21.15.177:6803/4258128961] conn(0x7f8cdc080ed0 0x7f8cdc083350 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cdc07af50 msgr2=0x7f8cdc07d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8d01000640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f8cdc07af50 0x7f8cdc07d410 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009670 tx=0x7f8cec006bf0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 msgr2=0x7f8cfc0c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8d01000640 1 --2- 172.21.15.71:0/1570800578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc074460 0x7f8cfc0c9400 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800c480 tx=0x7f8ce800c940 comp rx=0 tx=0).stop 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8cfa800640 1 -- 172.21.15.71:0/1570800578 reap_dead start 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 shutdown_connections 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.292+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 >> 172.21.15.71:0/1570800578 conn(0x7f8cfc0b18e0 msgr2=0x7f8cfc1556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.293+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 shutdown_connections 2024-09-16T07:38:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.293+0000 7f8d01000640 1 -- 172.21.15.71:0/1570800578 wait complete. 2024-09-16T07:38:49.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T07:38:49.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.466+0000 7f6aaf200640 1 -- 172.21.15.71:0/2413171531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80725d0 msgr2=0x7f6aa80b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.466+0000 7f6aaf200640 1 --2- 172.21.15.71:0/2413171531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80725d0 0x7f6aa80b28f0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009830 tx=0x7f6a9802a510 comp rx=0 tx=0).stop 2024-09-16T07:38:49.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.466+0000 7f6aaf200640 1 -- 172.21.15.71:0/2413171531 shutdown_connections 2024-09-16T07:38:49.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.466+0000 7f6aaf200640 1 --2- 172.21.15.71:0/2413171531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6aa80d0540 0x7f6aa806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.466+0000 7f6aaf200640 1 --2- 172.21.15.71:0/2413171531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80725d0 0x7f6aa80b28f0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.466+0000 7f6aaf200640 1 -- 172.21.15.71:0/2413171531 >> 172.21.15.71:0/2413171531 conn(0x7f6aa80cfa20 msgr2=0x7f6aa80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:49.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.466+0000 7f6aaf200640 1 -- 172.21.15.71:0/2413171531 shutdown_connections 2024-09-16T07:38:49.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.466+0000 7f6aaf200640 1 -- 172.21.15.71:0/2413171531 wait complete. 2024-09-16T07:38:49.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.467+0000 7f6aaf200640 1 Processor -- start 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.468+0000 7f6aaf200640 1 -- start start 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.468+0000 7f6aaf200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80d0540 0x7f6aa80a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.468+0000 7f6aaf200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa815bc30 0x7f6aa80c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.468+0000 7f6aaf200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6aa8146a20 0x7f6aa80c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.468+0000 7f6aaf200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8074660 con 0x7f6aa80d0540 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.468+0000 7f6aaf200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa806f860 con 0x7f6aa815bc30 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.468+0000 7f6aaf200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa8154780 con 0x7f6aa8146a20 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80d0540 0x7f6aa80a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80d0540 0x7f6aa80a96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36400/0 (socket says 172.21.15.71:36400) 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 -- 172.21.15.71:0/1368126175 learned_addr learned my addr 172.21.15.71:0/1368126175 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 -- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6aa8146a20 msgr2=0x7f6aa80c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aad400640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa815bc30 0x7f6aa80c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aae800640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6aa8146a20 0x7f6aa80c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6aa8146a20 0x7f6aa80c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 -- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa815bc30 msgr2=0x7f6aa80c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa815bc30 0x7f6aa80c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 -- 172.21.15.71:0/1368126175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa81f3070 con 0x7f6aa80d0540 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aad400640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa815bc30 0x7f6aa80c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aae800640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6aa8146a20 0x7f6aa80c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.469+0000 7f6aade00640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80d0540 0x7f6aa80a96e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c430 tx=0x7f6a9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.470+0000 7f6aa2000640 1 -- 172.21.15.71:0/1368126175 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c011790 con 0x7f6aa80d0540 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.470+0000 7f6aa2000640 1 -- 172.21.15.71:0/1368126175 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9c017070 con 0x7f6aa80d0540 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.470+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa81549d0 con 0x7f6aa80d0540 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.470+0000 7f6aa2000640 1 -- 172.21.15.71:0/1368126175 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c011ce0 con 0x7f6aa80d0540 2024-09-16T07:38:49.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.470+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa8154f90 con 0x7f6aa80d0540 2024-09-16T07:38:49.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.471+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa806fdf0 con 0x7f6aa80d0540 2024-09-16T07:38:49.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.472+0000 7f6aa2000640 1 -- 172.21.15.71:0/1368126175 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f6a9c01c460 con 0x7f6aa80d0540 2024-09-16T07:38:49.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.473+0000 7f6aa2000640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a8c07af20 0x7f6a8c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.473+0000 7f6aa2000640 1 -- 172.21.15.71:0/1368126175 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f6a9c0999b0 con 0x7f6aa80d0540 2024-09-16T07:38:49.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.474+0000 7f6aad400640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a8c07af20 0x7f6a8c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.474+0000 7f6aad400640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a8c07af20 0x7f6a8c07d3e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6a98034000 tx=0x7f6a98004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:49.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.477+0000 7f6aa2000640 1 -- 172.21.15.71:0/1368126175 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6a9c063b20 con 0x7f6aa80d0540 2024-09-16T07:38:49.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.662+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f6aa81551d0 con 0x7f6a8c07af20 2024-09-16T07:38:49.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.663+0000 7f6aa2000640 1 -- 172.21.15.71:0/1368126175 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6aa81551d0 con 0x7f6a8c07af20 2024-09-16T07:38:49.662 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T07:38:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a8c07af20 msgr2=0x7f6a8c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a8c07af20 0x7f6a8c07d3e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6a98034000 tx=0x7f6a98004730 comp rx=0 tx=0).stop 2024-09-16T07:38:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80d0540 msgr2=0x7f6aa80a96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80d0540 0x7f6aa80a96e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c430 tx=0x7f6a9c00c8f0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 shutdown_connections 2024-09-16T07:38:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f6a8c07af20 0x7f6a8c07d3e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6aa8146a20 0x7f6aa80c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6aa815bc30 0x7f6aa80c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 --2- 172.21.15.71:0/1368126175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6aa80d0540 0x7f6aa80a96e0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 >> 172.21.15.71:0/1368126175 conn(0x7f6aa80cfa20 msgr2=0x7f6aa80c8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 shutdown_connections 2024-09-16T07:38:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.665+0000 7f6aaf200640 1 -- 172.21.15.71:0/1368126175 wait complete. 2024-09-16T07:38:49.680 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T07:38:49.680 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T07:38:49.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.843+0000 7f41dd800640 1 -- 172.21.15.71:0/3649338610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41bc003a00 msgr2=0x7f41bc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.843+0000 7f41dd800640 1 --2- 172.21.15.71:0/3649338610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41bc003a00 0x7f41bc003e20 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c610 tx=0x7f41bc005830 comp rx=0 tx=0).stop 2024-09-16T07:38:49.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.843+0000 7f41d7400640 1 -- 172.21.15.71:0/3649338610 reap_dead start 2024-09-16T07:38:49.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.843+0000 7f41dd800640 1 -- 172.21.15.71:0/3649338610 shutdown_connections 2024-09-16T07:38:49.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.843+0000 7f41dd800640 1 -- 172.21.15.71:0/3649338610 >> 172.21.15.71:0/3649338610 conn(0x7f41d80b18e0 msgr2=0x7f41d80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:49.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.843+0000 7f41dd800640 1 -- 172.21.15.71:0/3649338610 shutdown_connections 2024-09-16T07:38:49.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.843+0000 7f41dd800640 1 -- 172.21.15.71:0/3649338610 wait complete. 2024-09-16T07:38:49.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.845+0000 7f41dd800640 1 Processor -- start 2024-09-16T07:38:49.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.845+0000 7f41dd800640 1 -- start start 2024-09-16T07:38:49.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41dd800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41bc003a00 0x7f41d81f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41dd800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41d809d590 0x7f41d814b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41d8153c60 0x7f41d81540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41dd800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f41d806ed20 con 0x7f41d8153c60 2024-09-16T07:38:49.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41dd800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f41d806eba0 con 0x7f41bc003a00 2024-09-16T07:38:49.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41dd800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f41d80cc190 con 0x7f41d809d590 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41d7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41d8153c60 0x7f41d81540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41d7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41d8153c60 0x7f41d81540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36416/0 (socket says 172.21.15.71:36416) 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41d7400640 1 -- 172.21.15.71:0/1876352836 learned_addr learned my addr 172.21.15.71:0/1876352836 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.846+0000 7f41d6000640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41d809d590 0x7f41d814b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41d6a00640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41bc003a00 0x7f41d81f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41d7400640 1 -- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41d809d590 msgr2=0x7f41d814b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41d7400640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41d809d590 0x7f41d814b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41d7400640 1 -- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41bc003a00 msgr2=0x7f41d81f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41d7400640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41bc003a00 0x7f41d81f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41d7400640 1 -- 172.21.15.71:0/1876352836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80cdb70 con 0x7f41d8153c60 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41d6000640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41d809d590 0x7f41d814b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:49.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41d7400640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41d8153c60 0x7f41d81540c0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f41d80af550 tx=0x7f41d0011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:49.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d001f070 con 0x7f41d8153c60 2024-09-16T07:38:49.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41d000fb70 con 0x7f41d8153c60 2024-09-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d81f7070 con 0x7f41d8153c60 2024-09-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0012450 con 0x7f41d8153c60 2024-09-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.847+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d80cdde0 con 0x7f41d8153c60 2024-09-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.848+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41d814c5f0 con 0x7f41d8153c60 2024-09-16T07:38:49.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.850+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f41d0022070 con 0x7f41d8153c60 2024-09-16T07:38:49.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.851+0000 7f41cea00640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41b807af20 0x7f41b807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.851+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f41d009ad60 con 0x7f41d8153c60 2024-09-16T07:38:49.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.851+0000 7f41cea00640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f41b8080ea0 0x7f41b8083320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:49.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.851+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 --> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41d001cec0 con 0x7f41b8080ea0 2024-09-16T07:38:49.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.851+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f41d009b3f0 con 0x7f41d8153c60 2024-09-16T07:38:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.851+0000 7f41d6a00640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41b807af20 0x7f41b807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.851+0000 7f41d6000640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f41b8080ea0 0x7f41b8083320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.852+0000 7f41d6000640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f41b8080ea0 0x7f41b8083320 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.852+0000 7f41d6a00640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41b807af20 0x7f41b807d3e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f41c8033000 tx=0x7f41c8038040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.853+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 <== osd.5 v2:172.21.15.177:6823/1078185550 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41d001cec0 con 0x7f41b8080ea0 2024-09-16T07:38:49.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.866+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 --> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f41d806f6d0 con 0x7f41b8080ea0 2024-09-16T07:38:49.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41cea00640 1 -- 172.21.15.71:0/1876352836 <== osd.5 v2:172.21.15.177:6823/1078185550 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f41d806f6d0 con 0x7f41b8080ea0 2024-09-16T07:38:49.866 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:49.866 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T07:38:49.866 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:49.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f41b8080ea0 msgr2=0x7f41b8083320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6823/1078185550,v1:172.21.15.177:6825/1078185550] conn(0x7f41b8080ea0 0x7f41b8083320 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:49.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41b807af20 msgr2=0x7f41b807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f41b807af20 0x7f41b807d3e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f41c8033000 tx=0x7f41c8038040 comp rx=0 tx=0).stop 2024-09-16T07:38:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41d8153c60 msgr2=0x7f41d81540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 --2- 172.21.15.71:0/1876352836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41d8153c60 0x7f41d81540c0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f41d80af550 tx=0x7f41d0011b10 comp rx=0 tx=0).stop 2024-09-16T07:38:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41d7400640 1 -- 172.21.15.71:0/1876352836 reap_dead start 2024-09-16T07:38:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 shutdown_connections 2024-09-16T07:38:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 >> 172.21.15.71:0/1876352836 conn(0x7f41d80b18e0 msgr2=0x7f41d814bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 shutdown_connections 2024-09-16T07:38:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:49.867+0000 7f41dd800640 1 -- 172.21.15.71:0/1876352836 wait complete. 2024-09-16T07:38:49.881 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T07:38:50.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.041+0000 7f6185800640 1 -- 172.21.15.71:0/2798034360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61800ccf00 msgr2=0x7f61800acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.041+0000 7f6185800640 1 --2- 172.21.15.71:0/2798034360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61800ccf00 0x7f61800acd30 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f616c009830 tx=0x7f616c02a480 comp rx=0 tx=0).stop 2024-09-16T07:38:50.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.041+0000 7f6185800640 1 -- 172.21.15.71:0/2798034360 shutdown_connections 2024-09-16T07:38:50.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.041+0000 7f6185800640 1 --2- 172.21.15.71:0/2798034360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61800ccf00 0x7f61800acd30 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.041+0000 7f6185800640 1 --2- 172.21.15.71:0/2798034360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f618015f5a0 0x7f61800b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.041+0000 7f6185800640 1 -- 172.21.15.71:0/2798034360 >> 172.21.15.71:0/2798034360 conn(0x7f61800b1950 msgr2=0x7f61800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:50.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.041+0000 7f6185800640 1 -- 172.21.15.71:0/2798034360 shutdown_connections 2024-09-16T07:38:50.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.041+0000 7f6185800640 1 -- 172.21.15.71:0/2798034360 wait complete. 2024-09-16T07:38:50.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.043+0000 7f6185800640 1 Processor -- start 2024-09-16T07:38:50.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.043+0000 7f6185800640 1 -- start start 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f6185800640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61800ad270 0x7f61800ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f6185800640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61800ccf00 0x7f6180149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f6185800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f618015f5a0 0x7f61800cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f6185800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f618006f2e0 con 0x7f618015f5a0 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f6185800640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f618006f160 con 0x7f61800ccf00 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f6185800640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f618006f460 con 0x7f61800ad270 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f618015f5a0 0x7f61800cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f618015f5a0 0x7f61800cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36438/0 (socket says 172.21.15.71:36438) 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617f400640 1 -- 172.21.15.71:0/119681453 learned_addr learned my addr 172.21.15.71:0/119681453 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617ea00640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61800ad270 0x7f61800ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617e000640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61800ccf00 0x7f6180149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617f400640 1 -- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61800ad270 msgr2=0x7f61800ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617f400640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61800ad270 0x7f61800ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617f400640 1 -- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61800ccf00 msgr2=0x7f6180149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617f400640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61800ccf00 0x7f6180149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617f400640 1 -- 172.21.15.71:0/119681453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618014fd40 con 0x7f618015f5a0 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.044+0000 7f617e000640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61800ccf00 0x7f6180149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.045+0000 7f617f400640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f618015f5a0 0x7f61800cf4f0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f617800caf0 tx=0x7f617800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.045+0000 7f6176a00640 1 -- 172.21.15.71:0/119681453 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178016020 con 0x7f618015f5a0 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.045+0000 7f6176a00640 1 -- 172.21.15.71:0/119681453 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f617800ad90 con 0x7f618015f5a0 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.045+0000 7f6176a00640 1 -- 172.21.15.71:0/119681453 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178012680 con 0x7f618015f5a0 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.045+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61800cebf0 con 0x7f618015f5a0 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.045+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6180156130 con 0x7f618015f5a0 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.046+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618006f980 con 0x7f618015f5a0 2024-09-16T07:38:50.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.047+0000 7f6176a00640 1 -- 172.21.15.71:0/119681453 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f6178019020 con 0x7f618015f5a0 2024-09-16T07:38:50.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.048+0000 7f6176a00640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f616007af20 0x7f616007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.048+0000 7f6176a00640 1 -- 172.21.15.71:0/119681453 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f617809a230 con 0x7f618015f5a0 2024-09-16T07:38:50.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.049+0000 7f617ea00640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f616007af20 0x7f616007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.050+0000 7f617ea00640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f616007af20 0x7f616007d3e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6170005fd0 tx=0x7f617000b2e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:50.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.052+0000 7f6176a00640 1 -- 172.21.15.71:0/119681453 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6178020020 con 0x7f618015f5a0 2024-09-16T07:38:50.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.242+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f61800711a0 con 0x7f616007af20 2024-09-16T07:38:50.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.242+0000 7f6176a00640 1 -- 172.21.15.71:0/119681453 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f61800711a0 con 0x7f616007af20 2024-09-16T07:38:50.242 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T07:38:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f616007af20 msgr2=0x7f616007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f616007af20 0x7f616007d3e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6170005fd0 tx=0x7f617000b2e0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f618015f5a0 msgr2=0x7f61800cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f618015f5a0 0x7f61800cf4f0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f617800caf0 tx=0x7f617800a060 comp rx=0 tx=0).stop 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 shutdown_connections 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f616007af20 0x7f616007d3e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f618015f5a0 0x7f61800cf4f0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61800ccf00 0x7f6180149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 --2- 172.21.15.71:0/119681453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61800ad270 0x7f61800ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 >> 172.21.15.71:0/119681453 conn(0x7f61800b1950 msgr2=0x7f618009c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.245+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 shutdown_connections 2024-09-16T07:38:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.246+0000 7f6185800640 1 -- 172.21.15.71:0/119681453 wait complete. 2024-09-16T07:38:50.261 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T07:38:50.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T07:38:50.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.424+0000 7f8646000640 1 -- 172.21.15.71:0/938286483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8624003a50 msgr2=0x7f8624003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.424+0000 7f8646000640 1 --2- 172.21.15.71:0/938286483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8624003a50 0x7f8624003e70 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f8630009f00 tx=0x7f8624005830 comp rx=0 tx=0).stop 2024-09-16T07:38:50.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.424+0000 7f863fe00640 1 -- 172.21.15.71:0/938286483 reap_dead start 2024-09-16T07:38:50.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.424+0000 7f8646000640 1 -- 172.21.15.71:0/938286483 shutdown_connections 2024-09-16T07:38:50.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.424+0000 7f8646000640 1 -- 172.21.15.71:0/938286483 >> 172.21.15.71:0/938286483 conn(0x7f86400b18e0 msgr2=0x7f86400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:50.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.424+0000 7f8646000640 1 -- 172.21.15.71:0/938286483 shutdown_connections 2024-09-16T07:38:50.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.424+0000 7f8646000640 1 -- 172.21.15.71:0/938286483 wait complete. 2024-09-16T07:38:50.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.426+0000 7f8646000640 1 Processor -- start 2024-09-16T07:38:50.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.426+0000 7f8646000640 1 -- start start 2024-09-16T07:38:50.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f8646000640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8624003a50 0x7f864009d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f8646000640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864009c170 0x7f86400ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f8646000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f864014bdc0 0x7f864015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f8646000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f864015b340 con 0x7f864014bdc0 2024-09-16T07:38:50.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f8646000640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f864015b1c0 con 0x7f864009c170 2024-09-16T07:38:50.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f8646000640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f864015b4c0 con 0x7f8624003a50 2024-09-16T07:38:50.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f864014bdc0 0x7f864015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f864014bdc0 0x7f864015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36460/0 (socket says 172.21.15.71:36460) 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863fe00640 1 -- 172.21.15.71:0/2215437776 learned_addr learned my addr 172.21.15.71:0/2215437776 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863ea00640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864009c170 0x7f86400ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863f400640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8624003a50 0x7f864009d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863fe00640 1 -- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8624003a50 msgr2=0x7f864009d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863fe00640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8624003a50 0x7f864009d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863fe00640 1 -- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864009c170 msgr2=0x7f86400ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863fe00640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864009c170 0x7f86400ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.427+0000 7f863fe00640 1 -- 172.21.15.71:0/2215437776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864009c830 con 0x7f864014bdc0 2024-09-16T07:38:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.428+0000 7f863fe00640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f864014bdc0 0x7f864015d5e0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f8640076540 tx=0x7f8638011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.428+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638004440 con 0x7f864014bdc0 2024-09-16T07:38:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.428+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f863800f920 con 0x7f864014bdc0 2024-09-16T07:38:50.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.428+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864009cb10 con 0x7f864014bdc0 2024-09-16T07:38:50.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.428+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863800fd70 con 0x7f864014bdc0 2024-09-16T07:38:50.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.428+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864009d0d0 con 0x7f864014bdc0 2024-09-16T07:38:50.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.429+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f864009c5d0 con 0x7f864014bdc0 2024-09-16T07:38:50.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.430+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f86380035f0 con 0x7f864014bdc0 2024-09-16T07:38:50.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.431+0000 7f8637400640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f862007af20 0x7f862007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.431+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f863809a3e0 con 0x7f864014bdc0 2024-09-16T07:38:50.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.432+0000 7f8637400640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f8620080e40 0x7f86200832c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.432+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 --> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8638014ec0 con 0x7f8620080e40 2024-09-16T07:38:50.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.432+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f86380a6050 con 0x7f864014bdc0 2024-09-16T07:38:50.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.432+0000 7f863f400640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f862007af20 0x7f862007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.432+0000 7f863ea00640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f8620080e40 0x7f86200832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.433+0000 7f863f400640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f862007af20 0x7f862007d3e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8630035000 tx=0x7f863002d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:50.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.433+0000 7f863ea00640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f8620080e40 0x7f86200832c0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:50.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.434+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 <== osd.6 v2:172.21.15.177:6818/1024130870 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8638014ec0 con 0x7f8620080e40 2024-09-16T07:38:50.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.447+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 --> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f864006eb80 con 0x7f8620080e40 2024-09-16T07:38:50.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.447+0000 7f8637400640 1 -- 172.21.15.71:0/2215437776 <== osd.6 v2:172.21.15.177:6818/1024130870 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f864006eb80 con 0x7f8620080e40 2024-09-16T07:38:50.447 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:50.447 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T07:38:50.447 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f8620080e40 msgr2=0x7f86200832c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6818/1024130870,v1:172.21.15.177:6819/1024130870] conn(0x7f8620080e40 0x7f86200832c0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f862007af20 msgr2=0x7f862007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f862007af20 0x7f862007d3e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8630035000 tx=0x7f863002d040 comp rx=0 tx=0).stop 2024-09-16T07:38:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f864014bdc0 msgr2=0x7f864015d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 --2- 172.21.15.71:0/2215437776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f864014bdc0 0x7f864015d5e0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f8640076540 tx=0x7f8638011c60 comp rx=0 tx=0).stop 2024-09-16T07:38:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f863fe00640 1 -- 172.21.15.71:0/2215437776 reap_dead start 2024-09-16T07:38:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 shutdown_connections 2024-09-16T07:38:50.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 >> 172.21.15.71:0/2215437776 conn(0x7f86400b18e0 msgr2=0x7f8640159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:50.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 shutdown_connections 2024-09-16T07:38:50.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.448+0000 7f8646000640 1 -- 172.21.15.71:0/2215437776 wait complete. 2024-09-16T07:38:50.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T07:38:50.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.620+0000 7f6035400640 1 -- 172.21.15.71:0/1786539239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60300725d0 msgr2=0x7f60300b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.620+0000 7f6035400640 1 --2- 172.21.15.71:0/1786539239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60300725d0 0x7f60300b28f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f601c005fd0 tx=0x7f601c02a5d0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.621+0000 7f6035400640 1 -- 172.21.15.71:0/1786539239 shutdown_connections 2024-09-16T07:38:50.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.621+0000 7f6035400640 1 --2- 172.21.15.71:0/1786539239 >> v2:172.21.15.177:3300/0 conn(0x7f60300d0540 0x7f603006f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.621+0000 7f6035400640 1 --2- 172.21.15.71:0/1786539239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60300725d0 0x7f60300b28f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.621+0000 7f6035400640 1 -- 172.21.15.71:0/1786539239 >> 172.21.15.71:0/1786539239 conn(0x7f60300cfa20 msgr2=0x7f60300cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:50.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.621+0000 7f6035400640 1 -- 172.21.15.71:0/1786539239 shutdown_connections 2024-09-16T07:38:50.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.621+0000 7f6035400640 1 -- 172.21.15.71:0/1786539239 wait complete. 2024-09-16T07:38:50.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.622+0000 7f6035400640 1 Processor -- start 2024-09-16T07:38:50.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.622+0000 7f6035400640 1 -- start start 2024-09-16T07:38:50.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.623+0000 7f6035400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60300d0540 0x7f603015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.623+0000 7f6035400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603015bc30 0x7f60300b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.623+0000 7f6035400640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6030150030 0x7f60301504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.623+0000 7f6035400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6030074660 con 0x7f603015bc30 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.623+0000 7f6035400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f603015c200 con 0x7f60300d0540 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.623+0000 7f6035400640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60301f57e0 con 0x7f6030150030 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603015bc30 0x7f60300b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603015bc30 0x7f60300b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36476/0 (socket says 172.21.15.71:36476) 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 -- 172.21.15.71:0/2215877797 learned_addr learned my addr 172.21.15.71:0/2215877797 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f6034a00640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6030150030 0x7f60301504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602fe00640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60300d0540 0x7f603015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 -- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6030150030 msgr2=0x7f60301504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6030150030 0x7f60301504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 -- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60300d0540 msgr2=0x7f603015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60300d0540 0x7f603015d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 -- 172.21.15.71:0/2215877797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60301f5a30 con 0x7f603015bc30 2024-09-16T07:38:50.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602fe00640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60300d0540 0x7f603015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T07:38:50.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.624+0000 7f602f400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603015bc30 0x7f60300b4470 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f601c009850 tx=0x7f601c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:50.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.625+0000 7f6027e00640 1 -- 172.21.15.71:0/2215877797 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c03f070 con 0x7f603015bc30 2024-09-16T07:38:50.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.625+0000 7f6027e00640 1 -- 172.21.15.71:0/2215877797 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f601c003ce0 con 0x7f603015bc30 2024-09-16T07:38:50.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.625+0000 7f6027e00640 1 -- 172.21.15.71:0/2215877797 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c035030 con 0x7f603015bc30 2024-09-16T07:38:50.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.625+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60301f5d10 con 0x7f603015bc30 2024-09-16T07:38:50.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.625+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301f62d0 con 0x7f603015bc30 2024-09-16T07:38:50.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.626+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60301f66b0 con 0x7f603015bc30 2024-09-16T07:38:50.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.627+0000 7f6027e00640 1 -- 172.21.15.71:0/2215877797 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f601c042070 con 0x7f603015bc30 2024-09-16T07:38:50.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.628+0000 7f6027e00640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f601007af20 0x7f601007d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:50.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.628+0000 7f6027e00640 1 -- 172.21.15.71:0/2215877797 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f601c0ba850 con 0x7f603015bc30 2024-09-16T07:38:50.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.628+0000 7f602fe00640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f601007af20 0x7f601007d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:50.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.629+0000 7f602fe00640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f601007af20 0x7f601007d3e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6020001600 tx=0x7f6020009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:50.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.632+0000 7f6027e00640 1 -- 172.21.15.71:0/2215877797 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f60301f66b0 con 0x7f603015bc30 2024-09-16T07:38:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.815+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f603006fc40 con 0x7f601007af20 2024-09-16T07:38:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.816+0000 7f6027e00640 1 -- 172.21.15.71:0/2215877797 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f603006fc40 con 0x7f601007af20 2024-09-16T07:38:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T07:38:50.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f601007af20 msgr2=0x7f601007d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f601007af20 0x7f601007d3e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6020001600 tx=0x7f6020009040 comp rx=0 tx=0).stop 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603015bc30 msgr2=0x7f60300b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603015bc30 0x7f60300b4470 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f601c009850 tx=0x7f601c0035f0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 shutdown_connections 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f601007af20 0x7f601007d3e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6030150030 0x7f60301504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603015bc30 0x7f60300b4470 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 --2- 172.21.15.71:0/2215877797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60300d0540 0x7f603015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 >> 172.21.15.71:0/2215877797 conn(0x7f60300cfa20 msgr2=0x7f60300ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 shutdown_connections 2024-09-16T07:38:50.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.818+0000 7f6035400640 1 -- 172.21.15.71:0/2215877797 wait complete. 2024-09-16T07:38:50.835 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T07:38:50.835 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T07:38:50.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.996+0000 7fddace00640 1 -- 172.21.15.71:0/4056796708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80b4b80 msgr2=0x7fdda80727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:50.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.996+0000 7fddace00640 1 --2- 172.21.15.71:0/4056796708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80b4b80 0x7fdda80727c0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009830 tx=0x7fdd9402a640 comp rx=0 tx=0).stop 2024-09-16T07:38:50.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.996+0000 7fddace00640 1 -- 172.21.15.71:0/4056796708 shutdown_connections 2024-09-16T07:38:50.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.996+0000 7fddace00640 1 --2- 172.21.15.71:0/4056796708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80b4b80 0x7fdda80727c0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:50.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.996+0000 7fddace00640 1 -- 172.21.15.71:0/4056796708 >> 172.21.15.71:0/4056796708 conn(0x7fdda80b18e0 msgr2=0x7fdda80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:50.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.996+0000 7fddace00640 1 -- 172.21.15.71:0/4056796708 shutdown_connections 2024-09-16T07:38:50.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.996+0000 7fddace00640 1 -- 172.21.15.71:0/4056796708 wait complete. 2024-09-16T07:38:50.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.998+0000 7fddace00640 1 Processor -- start 2024-09-16T07:38:50.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.998+0000 7fddace00640 1 -- start start 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fddace00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda80b2be0 0x7fdda80c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fddace00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdda80b4b80 0x7fdda80b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fddace00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80c94a0 0x7fdda80b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fddace00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdda806f270 con 0x7fdda80c94a0 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fddace00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdda806f0f0 con 0x7fdda80b2be0 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fddace00640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdda80b3030 con 0x7fdda80b4b80 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fdda6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80c94a0 0x7fdda80b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fdda6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80c94a0 0x7fdda80b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36496/0 (socket says 172.21.15.71:36496) 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fdda6800640 1 -- 172.21.15.71:0/679531669 learned_addr learned my addr 172.21.15.71:0/679531669 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fdda5400640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdda80b4b80 0x7fdda80b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:50.999+0000 7fdda5e00640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda80b2be0 0x7fdda80c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdda6800640 1 -- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdda80b4b80 msgr2=0x7fdda80b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdda6800640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdda80b4b80 0x7fdda80b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdda6800640 1 -- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda80b2be0 msgr2=0x7fdda80c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdda6800640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda80b2be0 0x7fdda80c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdda6800640 1 -- 172.21.15.71:0/679531669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda80afa60 con 0x7fdda80c94a0 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdda6800640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80c94a0 0x7fdda80b0d60 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fdda000c430 tx=0x7fdda000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0017070 con 0x7fdda80c94a0 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdda00053c0 con 0x7fdda80c94a0 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda80afd40 con 0x7fdda80c94a0 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0003850 con 0x7fdda80c94a0 2024-09-16T07:38:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.000+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda80b0300 con 0x7fdda80c94a0 2024-09-16T07:38:51.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.001+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdda80b34b0 con 0x7fdda80c94a0 2024-09-16T07:38:51.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.002+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fdda001a070 con 0x7fdda80c94a0 2024-09-16T07:38:51.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.003+0000 7fdd9e000640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdd8807af20 0x7fdd8807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.004+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7fdda0099090 con 0x7fdda80c94a0 2024-09-16T07:38:51.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.004+0000 7fdd9e000640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7fdd88080e40 0x7fdd880832c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.004+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 --> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdda0016ec0 con 0x7fdd88080e40 2024-09-16T07:38:51.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.004+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fdda00a5050 con 0x7fdda80c94a0 2024-09-16T07:38:51.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.004+0000 7fdda5e00640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdd8807af20 0x7fdd8807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.004+0000 7fdda5400640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7fdd88080e40 0x7fdd880832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.005+0000 7fdda5e00640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdd8807af20 0x7fdd8807d3e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdd980012e0 tx=0x7fdd98009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.005+0000 7fdda5400640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7fdd88080e40 0x7fdd880832c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.006+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 <== osd.7 v2:172.21.15.177:6810/4227809343 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdda0016ec0 con 0x7fdd88080e40 2024-09-16T07:38:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.019+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 --> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdda80cc010 con 0x7fdd88080e40 2024-09-16T07:38:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fdd9e000640 1 -- 172.21.15.71:0/679531669 <== osd.7 v2:172.21.15.177:6810/4227809343 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdda80cc010 con 0x7fdd88080e40 2024-09-16T07:38:51.019 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-16T07:38:51.019 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T07:38:51.019 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-16T07:38:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7fdd88080e40 msgr2=0x7fdd880832c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6810/4227809343,v1:172.21.15.177:6811/4227809343] conn(0x7fdd88080e40 0x7fdd880832c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdd8807af20 msgr2=0x7fdd8807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fdd8807af20 0x7fdd8807d3e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdd980012e0 tx=0x7fdd98009040 comp rx=0 tx=0).stop 2024-09-16T07:38:51.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80c94a0 msgr2=0x7fdda80b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 --2- 172.21.15.71:0/679531669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda80c94a0 0x7fdda80b0d60 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fdda000c430 tx=0x7fdda000c8f0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fdda6800640 1 -- 172.21.15.71:0/679531669 reap_dead start 2024-09-16T07:38:51.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 shutdown_connections 2024-09-16T07:38:51.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 >> 172.21.15.71:0/679531669 conn(0x7fdda80b18e0 msgr2=0x7fdda806ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:51.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 shutdown_connections 2024-09-16T07:38:51.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.020+0000 7fddace00640 1 -- 172.21.15.71:0/679531669 wait complete. 2024-09-16T07:38:51.035 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T07:38:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.195+0000 7fd349600640 1 -- 172.21.15.71:0/3784776168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd328003c30 msgr2=0x7fd328004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.195+0000 7fd349600640 1 --2- 172.21.15.71:0/3784776168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd328003c30 0x7fd328004030 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fd334009b50 tx=0x7fd328005830 comp rx=0 tx=0).stop 2024-09-16T07:38:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.195+0000 7fd349600640 1 -- 172.21.15.71:0/3784776168 shutdown_connections 2024-09-16T07:38:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.195+0000 7fd349600640 1 --2- 172.21.15.71:0/3784776168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd328003c30 0x7fd328004030 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.195+0000 7fd349600640 1 -- 172.21.15.71:0/3784776168 >> 172.21.15.71:0/3784776168 conn(0x7fd3440b1950 msgr2=0x7fd3440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.196+0000 7fd349600640 1 -- 172.21.15.71:0/3784776168 shutdown_connections 2024-09-16T07:38:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.196+0000 7fd349600640 1 -- 172.21.15.71:0/3784776168 wait complete. 2024-09-16T07:38:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.197+0000 7fd349600640 1 Processor -- start 2024-09-16T07:38:51.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.197+0000 7fd349600640 1 -- start start 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd349600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd328003c30 0x7fd34414cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd349600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd34414df00 0x7fd344073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd349600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd344074010 0x7fd3441f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd349600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd34406f160 con 0x7fd34414df00 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd349600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd344161aa0 con 0x7fd344074010 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd349600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd34406f2e0 con 0x7fd328003c30 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd342000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd34414df00 0x7fd344073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd342000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd34414df00 0x7fd344073ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36514/0 (socket says 172.21.15.71:36514) 2024-09-16T07:38:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.198+0000 7fd342000640 1 -- 172.21.15.71:0/3703336125 learned_addr learned my addr 172.21.15.71:0/3703336125 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd343400640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd344074010 0x7fd3441f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd342a00640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd328003c30 0x7fd34414cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd342000640 1 -- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd328003c30 msgr2=0x7fd34414cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd342000640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd328003c30 0x7fd34414cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd342000640 1 -- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd344074010 msgr2=0x7fd3441f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd342000640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd344074010 0x7fd3441f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd342000640 1 -- 172.21.15.71:0/3703336125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3441f5680 con 0x7fd34414df00 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd343400640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd344074010 0x7fd3441f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd342a00640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd328003c30 0x7fd34414cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd342000640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd34414df00 0x7fd344073ad0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd330006650 tx=0x7fd330001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd33aa00640 1 -- 172.21.15.71:0/3703336125 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330011700 con 0x7fd34414df00 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd33aa00640 1 -- 172.21.15.71:0/3703336125 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd330016020 con 0x7fd34414df00 2024-09-16T07:38:51.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34414d420 con 0x7fd34414df00 2024-09-16T07:38:51.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd33aa00640 1 -- 172.21.15.71:0/3703336125 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330011b60 con 0x7fd34414df00 2024-09-16T07:38:51.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.199+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34414d9e0 con 0x7fd34414df00 2024-09-16T07:38:51.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.200+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd344074420 con 0x7fd34414df00 2024-09-16T07:38:51.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.202+0000 7fd33aa00640 1 -- 172.21.15.71:0/3703336125 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fd330011d00 con 0x7fd34414df00 2024-09-16T07:38:51.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.202+0000 7fd33aa00640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd32407af50 0x7fd32407d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.203+0000 7fd33aa00640 1 -- 172.21.15.71:0/3703336125 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7fd33009a8c0 con 0x7fd34414df00 2024-09-16T07:38:51.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.203+0000 7fd342a00640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd32407af50 0x7fd32407d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.204+0000 7fd342a00640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd32407af50 0x7fd32407d410 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd334035000 tx=0x7fd3340046c0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.207+0000 7fd33aa00640 1 -- 172.21.15.71:0/3703336125 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd344074420 con 0x7fd34414df00 2024-09-16T07:38:51.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.393+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fd34406f840 con 0x7fd32407af50 2024-09-16T07:38:51.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.394+0000 7fd33aa00640 1 -- 172.21.15.71:0/3703336125 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd34406f840 con 0x7fd32407af50 2024-09-16T07:38:51.393 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd32407af50 msgr2=0x7fd32407d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd32407af50 0x7fd32407d410 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd334035000 tx=0x7fd3340046c0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd34414df00 msgr2=0x7fd344073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd34414df00 0x7fd344073ad0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd330006650 tx=0x7fd330001800 comp rx=0 tx=0).stop 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 shutdown_connections 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fd32407af50 0x7fd32407d410 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd344074010 0x7fd3441f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd34414df00 0x7fd344073ad0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 --2- 172.21.15.71:0/3703336125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd328003c30 0x7fd34414cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 >> 172.21.15.71:0/3703336125 conn(0x7fd3440b1950 msgr2=0x7fd3440b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:51.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 shutdown_connections 2024-09-16T07:38:51.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.396+0000 7fd349600640 1 -- 172.21.15.71:0/3703336125 wait complete. 2024-09-16T07:38:51.411 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T07:38:51.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.569+0000 7fef2b600640 1 -- 172.21.15.71:0/667739212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef0c004000 msgr2=0x7fef0c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.569+0000 7fef2b600640 1 --2- 172.21.15.71:0/667739212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef0c004000 0x7fef0c004800 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fef10009b50 tx=0x7fef0c005830 comp rx=0 tx=0).stop 2024-09-16T07:38:51.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.569+0000 7fef29600640 1 -- 172.21.15.71:0/667739212 reap_dead start 2024-09-16T07:38:51.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.569+0000 7fef2b600640 1 -- 172.21.15.71:0/667739212 shutdown_connections 2024-09-16T07:38:51.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.569+0000 7fef2b600640 1 -- 172.21.15.71:0/667739212 >> 172.21.15.71:0/667739212 conn(0x7fef240b1950 msgr2=0x7fef240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:51.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.569+0000 7fef2b600640 1 -- 172.21.15.71:0/667739212 shutdown_connections 2024-09-16T07:38:51.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.569+0000 7fef2b600640 1 -- 172.21.15.71:0/667739212 wait complete. 2024-09-16T07:38:51.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.571+0000 7fef2b600640 1 Processor -- start 2024-09-16T07:38:51.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.571+0000 7fef2b600640 1 -- start start 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef2b600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef0c004000 0x7fef24129540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef2b600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef241f8a90 0x7fef2409c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef2b600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef2414fe20 0x7fef241502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef2b600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fef2406f9f0 con 0x7fef2414fe20 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef2b600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fef2406f870 con 0x7fef241f8a90 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef2b600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fef240b2440 con 0x7fef0c004000 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef29600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef2414fe20 0x7fef241502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef29600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef2414fe20 0x7fef241502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36528/0 (socket says 172.21.15.71:36528) 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef29600640 1 -- 172.21.15.71:0/1819011184 learned_addr learned my addr 172.21.15.71:0/1819011184 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef28c00640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef0c004000 0x7fef24129540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef23e00640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef241f8a90 0x7fef2409c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef29600640 1 -- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef0c004000 msgr2=0x7fef24129540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef29600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef0c004000 0x7fef24129540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef29600640 1 -- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef241f8a90 msgr2=0x7fef2409c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef29600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef241f8a90 0x7fef2409c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef29600640 1 -- 172.21.15.71:0/1819011184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef241507e0 con 0x7fef2414fe20 2024-09-16T07:38:51.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.572+0000 7fef23e00640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef241f8a90 0x7fef2409c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:51.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.573+0000 7fef29600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef2414fe20 0x7fef241502a0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fef2406e130 tx=0x7fef1c006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.573+0000 7fef20c00640 1 -- 172.21.15.71:0/1819011184 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00fa20 con 0x7fef2414fe20 2024-09-16T07:38:51.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.573+0000 7fef20c00640 1 -- 172.21.15.71:0/1819011184 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef1c018070 con 0x7fef2414fe20 2024-09-16T07:38:51.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.573+0000 7fef20c00640 1 -- 172.21.15.71:0/1819011184 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00fe60 con 0x7fef2414fe20 2024-09-16T07:38:51.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.573+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef241f5840 con 0x7fef2414fe20 2024-09-16T07:38:51.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.573+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef241f4d60 con 0x7fef2414fe20 2024-09-16T07:38:51.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.574+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef24074440 con 0x7fef2414fe20 2024-09-16T07:38:51.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.575+0000 7fef20c00640 1 -- 172.21.15.71:0/1819011184 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fef1c01d460 con 0x7fef2414fe20 2024-09-16T07:38:51.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.577+0000 7fef20c00640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef0407af50 0x7fef0407d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.577+0000 7fef20c00640 1 -- 172.21.15.71:0/1819011184 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7fef1c09a3f0 con 0x7fef2414fe20 2024-09-16T07:38:51.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.577+0000 7fef28c00640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef0407af50 0x7fef0407d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.578+0000 7fef28c00640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef0407af50 0x7fef0407d410 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fef18000c00 tx=0x7fef18006b70 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.581+0000 7fef20c00640 1 -- 172.21.15.71:0/1819011184 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fef241507e0 con 0x7fef2414fe20 2024-09-16T07:38:51.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.761+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 --> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef240acf40 con 0x7fef0407af50 2024-09-16T07:38:51.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.763+0000 7fef20c00640 1 -- 172.21.15.71:0/1819011184 <== mgr.4120 v2:172.21.15.177:6800/143959246 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13423 (secure 0 0 0) 0x7fef240acf40 con 0x7fef0407af50 2024-09-16T07:38:51.762 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:51.763 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-16T07:38:51.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef0407af50 msgr2=0x7fef0407d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef0407af50 0x7fef0407d410 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fef18000c00 tx=0x7fef18006b70 comp rx=0 tx=0).stop 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef2414fe20 msgr2=0x7fef241502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef2414fe20 0x7fef241502a0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fef2406e130 tx=0x7fef1c006720 comp rx=0 tx=0).stop 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 shutdown_connections 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fef0407af50 0x7fef0407d410 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef2414fe20 0x7fef241502a0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef241f8a90 0x7fef2409c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 --2- 172.21.15.71:0/1819011184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef0c004000 0x7fef24129540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 >> 172.21.15.71:0/1819011184 conn(0x7fef240b1950 msgr2=0x7fef240ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 shutdown_connections 2024-09-16T07:38:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.765+0000 7fef2b600640 1 -- 172.21.15.71:0/1819011184 wait complete. 2024-09-16T07:38:51.781 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":632,"stamp":"2024-09-16T07:38:50.677604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1164580,"kb_used_data":15976,"kb_used_omap":12,"kb_used_meta":1148403,"kb_avail":753810140,"statfs":{"total":773094113280,"available":771901583360,"internally_reserved":0,"allocated":16359424,"data_stored":7886360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1175965232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3753,"num_read_kb":-15138873,"num_write":-8948,"num_write_kb":-18145324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002059"},"pg_stats":[{"pgid":"1.0","version":"152'74","reported_seq":363,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T07:38:48.289750+0000","last_change":"2024-09-16T07:38:48.289750+0000","last_active":"2024-09-16T07:38:48.289750+0000","last_peered":"2024-09-16T07:38:48.289750+0000","last_clean":"2024-09-16T07:38:48.289750+0000","last_became_active":"2024-09-16T07:22:40.451319+0000","last_became_peered":"2024-09-16T07:22:40.451319+0000","last_unstale":"2024-09-16T07:38:48.289750+0000","last_undegraded":"2024-09-16T07:38:48.289750+0000","last_fullsized":"2024-09-16T07:38:48.289750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'74","last_scrub_stamp":"2024-09-16T07:38:48.289381+0000","last_deep_scrub":"152'74","last_deep_scrub_stamp":"2024-09-16T07:38:48.289381+0000","last_clean_scrub_stamp":"2024-09-16T07:38:48.289381+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:07:58.818843+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145428,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226412,"statfs":{"total":96636764160,"available":96487845888,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145428,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226412,"statfs":{"total":96636764160,"available":96487845888,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476935,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123800,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248040,"statfs":{"total":96636764160,"available":96509992960,"internally_reserved":0,"allocated":2490368,"data_stored":1428571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":159316,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212524,"statfs":{"total":96636764160,"available":96473624576,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161216965},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123220,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248620,"statfs":{"total":96636764160,"available":96510586880,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509638,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167632,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204208,"statfs":{"total":96636764160,"available":96465108992,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":169736646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509638,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150168,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221672,"statfs":{"total":96636764160,"available":96482992128,"internally_reserved":0,"allocated":2490368,"data_stored":1428571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149588,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222252,"statfs":{"total":96636764160,"available":96483586048,"internally_reserved":0,"allocated":1896448,"data_stored":838203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T07:38:51.785 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T07:38:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.938+0000 7f65ef600640 1 -- 172.21.15.71:0/1508403741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65cc003aa0 msgr2=0x7f65cc003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.938+0000 7f65ef600640 1 --2- 172.21.15.71:0/1508403741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65cc003aa0 0x7f65cc003ec0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009f20 tx=0x7f65cc005830 comp rx=0 tx=0).stop 2024-09-16T07:38:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.938+0000 7f65ef600640 1 -- 172.21.15.71:0/1508403741 shutdown_connections 2024-09-16T07:38:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.938+0000 7f65ef600640 1 --2- 172.21.15.71:0/1508403741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65cc003aa0 0x7f65cc003ec0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.938+0000 7f65ef600640 1 -- 172.21.15.71:0/1508403741 >> 172.21.15.71:0/1508403741 conn(0x7f65e80b1840 msgr2=0x7f65e80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:51.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.938+0000 7f65ef600640 1 -- 172.21.15.71:0/1508403741 shutdown_connections 2024-09-16T07:38:51.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.938+0000 7f65ef600640 1 -- 172.21.15.71:0/1508403741 wait complete. 2024-09-16T07:38:51.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.940+0000 7f65ef600640 1 Processor -- start 2024-09-16T07:38:51.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.940+0000 7f65ef600640 1 -- start start 2024-09-16T07:38:51.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ef600640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65cc003aa0 0x7f65e81e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ef600640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e815b460 0x7f65e813f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ef600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e815b8f0 0x7f65e80add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ef600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f65e80737d0 con 0x7f65e815b8f0 2024-09-16T07:38:51.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ef600640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f65e8073650 con 0x7f65e815b460 2024-09-16T07:38:51.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ef600640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f65e80cc5c0 con 0x7f65cc003aa0 2024-09-16T07:38:51.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ed600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e815b8f0 0x7f65e80add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ed600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e815b8f0 0x7f65e80add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36548/0 (socket says 172.21.15.71:36548) 2024-09-16T07:38:51.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ed600640 1 -- 172.21.15.71:0/1151072470 learned_addr learned my addr 172.21.15.71:0/1151072470 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:51.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65e7e00640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e815b460 0x7f65e813f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.941+0000 7f65ecc00640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65cc003aa0 0x7f65e81e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ed600640 1 -- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65cc003aa0 msgr2=0x7f65e81e5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ed600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65cc003aa0 0x7f65e81e5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ed600640 1 -- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e815b460 msgr2=0x7f65e813f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:51.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ed600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e815b460 0x7f65e813f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:51.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ed600640 1 -- 172.21.15.71:0/1151072470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e81e8800 con 0x7f65e815b8f0 2024-09-16T07:38:51.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ecc00640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65cc003aa0 0x7f65e81e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ed600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e815b8f0 0x7f65e80add40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004780 tx=0x7f65e00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65e4c00640 1 -- 172.21.15.71:0/1151072470 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0011840 con 0x7f65e815b8f0 2024-09-16T07:38:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65e4c00640 1 -- 172.21.15.71:0/1151072470 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65e0017070 con 0x7f65e815b8f0 2024-09-16T07:38:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65e4c00640 1 -- 172.21.15.71:0/1151072470 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0011c80 con 0x7f65e815b8f0 2024-09-16T07:38:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e81e8ae0 con 0x7f65e815b8f0 2024-09-16T07:38:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.942+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e81e90a0 con 0x7f65e815b8f0 2024-09-16T07:38:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.943+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e81e6940 con 0x7f65e815b8f0 2024-09-16T07:38:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.945+0000 7f65e4c00640 1 -- 172.21.15.71:0/1151072470 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7f65e001c460 con 0x7f65e815b8f0 2024-09-16T07:38:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.946+0000 7f65e4c00640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65c807af20 0x7f65c807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:51.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.946+0000 7f65e4c00640 1 -- 172.21.15.71:0/1151072470 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7f65e009a230 con 0x7f65e815b8f0 2024-09-16T07:38:51.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.946+0000 7f65ecc00640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65c807af20 0x7f65c807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:51.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.947+0000 7f65ecc00640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65c807af20 0x7f65c807d3e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f65d8036000 tx=0x7f65d802d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:51.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:51.950+0000 7f65e4c00640 1 -- 172.21.15.71:0/1151072470 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f65e0011e20 con 0x7f65e815b8f0 2024-09-16T07:38:52.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.136+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f65e80ce1a0 con 0x7f65e815b8f0 2024-09-16T07:38:52.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.140+0000 7f65e4c00640 1 -- 172.21.15.71:0/1151072470 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v58)=0 v58) ==== 123+0+0 (secure 0 0 0) 0x7f65e0064300 con 0x7f65e815b8f0 2024-09-16T07:38:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.141+0000 7f65e4c00640 1 -- 172.21.15.71:0/1151072470 <== mon.0 v2:172.21.15.71:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f65e0006550 con 0x7f65e815b8f0 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65c807af20 msgr2=0x7f65c807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65c807af20 0x7f65c807d3e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f65d8036000 tx=0x7f65d802d040 comp rx=0 tx=0).stop 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e815b8f0 msgr2=0x7f65e80add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e815b8f0 0x7f65e80add40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004780 tx=0x7f65e00039c0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 shutdown_connections 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7f65c807af20 0x7f65c807d3e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e815b8f0 0x7f65e80add40 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e815b460 0x7f65e813f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 --2- 172.21.15.71:0/1151072470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65cc003aa0 0x7f65e81e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 >> 172.21.15.71:0/1151072470 conn(0x7f65e80b1840 msgr2=0x7f65e80ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:52.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 shutdown_connections 2024-09-16T07:38:52.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.143+0000 7f65ef600640 1 -- 172.21.15.71:0/1151072470 wait complete. 2024-09-16T07:38:52.159 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T07:38:52.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T07:38:52.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.316+0000 7fb877200640 1 -- 172.21.15.71:0/3335772737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780ad880 msgr2=0x7fb8780adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:52.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.316+0000 7fb877200640 1 --2- 172.21.15.71:0/3335772737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780ad880 0x7fb8780adcd0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb864005fd0 tx=0x7fb86402a530 comp rx=0 tx=0).stop 2024-09-16T07:38:52.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.316+0000 7fb877200640 1 -- 172.21.15.71:0/3335772737 shutdown_connections 2024-09-16T07:38:52.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.316+0000 7fb877200640 1 --2- 172.21.15.71:0/3335772737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780ad880 0x7fb8780adcd0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.316+0000 7fb877200640 1 --2- 172.21.15.71:0/3335772737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb87806fb90 0x7fb87806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.316+0000 7fb877200640 1 -- 172.21.15.71:0/3335772737 >> 172.21.15.71:0/3335772737 conn(0x7fb8780cfa20 msgr2=0x7fb8780cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:52.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.316+0000 7fb877200640 1 -- 172.21.15.71:0/3335772737 shutdown_connections 2024-09-16T07:38:52.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.316+0000 7fb877200640 1 -- 172.21.15.71:0/3335772737 wait complete. 2024-09-16T07:38:52.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.318+0000 7fb877200640 1 Processor -- start 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.318+0000 7fb877200640 1 -- start start 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb877200640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb87806fb90 0x7fb87815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb877200640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8780ad880 0x7fb8780c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb877200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780cd5f0 0x7fb8781f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb877200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb87815ee10 con 0x7fb8780cd5f0 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb877200640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb87815ec90 con 0x7fb8780ad880 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb877200640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb87815ef90 con 0x7fb87806fb90 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb876800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780cd5f0 0x7fb8781f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb876800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780cd5f0 0x7fb8781f7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36570/0 (socket says 172.21.15.71:36570) 2024-09-16T07:38:52.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb875e00640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb87806fb90 0x7fb87815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb876800640 1 -- 172.21.15.71:0/807911275 learned_addr learned my addr 172.21.15.71:0/807911275 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-16T07:38:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.319+0000 7fb875400640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8780ad880 0x7fb8780c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb876800640 1 -- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb87806fb90 msgr2=0x7fb87815af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb876800640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb87806fb90 0x7fb87815af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb876800640 1 -- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8780ad880 msgr2=0x7fb8780c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:52.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb876800640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8780ad880 0x7fb8780c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb876800640 1 -- 172.21.15.71:0/807911275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8780c37a0 con 0x7fb8780cd5f0 2024-09-16T07:38:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb875400640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8780ad880 0x7fb8780c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T07:38:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb876800640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780cd5f0 0x7fb8781f7db0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b970 tx=0x7fb87000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb86e000640 1 -- 172.21.15.71:0/807911275 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870019070 con 0x7fb8780cd5f0 2024-09-16T07:38:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb86e000640 1 -- 172.21.15.71:0/807911275 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fb87000cb40 con 0x7fb8780cd5f0 2024-09-16T07:38:52.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb86e000640 1 -- 172.21.15.71:0/807911275 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870014440 con 0x7fb8780cd5f0 2024-09-16T07:38:52.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8780c3a80 con 0x7fb8780cd5f0 2024-09-16T07:38:52.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.320+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87815b4c0 con 0x7fb8780cd5f0 2024-09-16T07:38:52.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.321+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87806ff70 con 0x7fb8780cd5f0 2024-09-16T07:38:52.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.322+0000 7fb86e000640 1 -- 172.21.15.71:0/807911275 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 56) ==== 104806+0+0 (secure 0 0 0) 0x7fb87001c070 con 0x7fb8780cd5f0 2024-09-16T07:38:52.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.323+0000 7fb86e000640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb85807af50 0x7fb85807d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T07:38:52.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.323+0000 7fb86e000640 1 -- 172.21.15.71:0/807911275 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5055+0+0 (secure 0 0 0) 0x7fb870099850 con 0x7fb8780cd5f0 2024-09-16T07:38:52.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.324+0000 7fb875e00640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb85807af50 0x7fb85807d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T07:38:52.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.325+0000 7fb875e00640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb85807af50 0x7fb85807d410 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb868005fd0 tx=0x7fb86800b390 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T07:38:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.328+0000 7fb86e000640 1 -- 172.21.15.71:0/807911275 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb8700632d0 con 0x7fb8780cd5f0 2024-09-16T07:38:52.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.548+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb87815fb90 con 0x7fb8780cd5f0 2024-09-16T07:38:52.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.548+0000 7fb86e000640 1 -- 172.21.15.71:0/807911275 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+3911 (secure 0 0 0) 0x7fb870067d00 con 0x7fb8780cd5f0 2024-09-16T07:38:52.548 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-16T07:38:52.548 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":98,"btime":"2024-09-16T07:38:39:088742+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8476,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/343939477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":343939477},{"type":"v1","addr":"172.21.15.71:6835","nonce":343939477}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1796926972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1796926972},{"type":"v1","addr":"172.21.15.177:6836","nonce":1796926972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":8926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/169820591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":169820591},{"type":"v1","addr":"172.21.15.177:6837","nonce":169820591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":9411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3638818185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3638818185},{"type":"v1","addr":"172.21.15.71:6834","nonce":3638818185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[]} 2024-09-16T07:38:52.548 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 98 2024-09-16T07:38:52.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.550+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb85807af50 msgr2=0x7fb85807d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:52.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.550+0000 7fb877200640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb85807af50 0x7fb85807d410 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb868005fd0 tx=0x7fb86800b390 comp rx=0 tx=0).stop 2024-09-16T07:38:52.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.550+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780cd5f0 msgr2=0x7fb8781f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T07:38:52.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.550+0000 7fb877200640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780cd5f0 0x7fb8781f7db0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b970 tx=0x7fb87000be30 comp rx=0 tx=0).stop 2024-09-16T07:38:52.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.551+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 shutdown_connections 2024-09-16T07:38:52.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.551+0000 7fb877200640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:6800/143959246,v1:172.21.15.177:6801/143959246] conn(0x7fb85807af50 0x7fb85807d410 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.551+0000 7fb877200640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8780cd5f0 0x7fb8781f7db0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.551+0000 7fb877200640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8780ad880 0x7fb8780c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.551+0000 7fb877200640 1 --2- 172.21.15.71:0/807911275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb87806fb90 0x7fb87815af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T07:38:52.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.551+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 >> 172.21.15.71:0/807911275 conn(0x7fb8780cfa20 msgr2=0x7fb8780725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T07:38:52.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.551+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 shutdown_connections 2024-09-16T07:38:52.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-16T07:38:52.551+0000 7fb877200640 1 -- 172.21.15.71:0/807911275 wait complete. 2024-09-16T07:38:52.566 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-16T07:38:52.566 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T07:38:52.567 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T07:38:52.567 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:52.581 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T07:38:52.581 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T07:38:52.581 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:52.637 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T07:38:52.638 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T07:38:52.638 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:52.655 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T07:38:52.655 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T07:38:52.655 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:52.713 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T07:38:52.713 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T07:38:52.713 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T07:38:52.714 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:52.776 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T07:38:52.777 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T07:38:52.777 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:52.843 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T07:38:52.843 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T07:38:52.843 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:52.922 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T07:38:52.923 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T07:38:52.923 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:52.987 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T07:38:52.987 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T07:38:52.987 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:53.059 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T07:38:53.060 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T07:38:53.060 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:53.148 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T07:38:53.148 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T07:38:53.148 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:53.221 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T07:38:53.221 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T07:38:53.221 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:53.286 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T07:38:53.287 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T07:38:53.287 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T07:38:53.287 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:53.358 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T07:38:53.359 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T07:38:53.359 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:58.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-16T07:38:59.461 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T07:38:59.461 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T07:38:59.461 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T07:38:59.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:59.488 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T07:38:59.488 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T07:38:59.488 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:59.511 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T07:38:59.511 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T07:38:59.511 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T07:38:59.573 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T07:38:59.574 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T07:38:59.574 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-16T07:38:59.665 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:38:59.665 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T07:39:00.592 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:39:00.593 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T07:39:00.738 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:39:00.739 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T07:39:00.866 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:39:00.866 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T07:39:01.008 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:39:01.008 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T07:39:01.276 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:39:01.277 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T07:39:01.404 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:39:01.404 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T07:39:01.520 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:39:01.521 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T07:39:01.659 INFO:tasks.ceph:Archiving mon data... 2024-09-16T07:39:01.660 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/data/mon.a.tgz 2024-09-16T07:39:01.660 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-16T07:39:01.696 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.4JALjMnl5N 2024-09-16T07:39:01.697 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4JALjMnl5N 2024-09-16T07:39:02.088 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.4JALjMnl5N 2024-09-16T07:39:02.182 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.4JALjMnl5N is 2MB 2024-09-16T07:39:02.291 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.4JALjMnl5N 2024-09-16T07:39:02.326 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/data/mon.b.tgz 2024-09-16T07:39:02.327 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-16T07:39:02.363 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.WbZdnw7jsX 2024-09-16T07:39:02.363 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.WbZdnw7jsX 2024-09-16T07:39:02.757 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.WbZdnw7jsX 2024-09-16T07:39:02.850 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.WbZdnw7jsX is 2MB 2024-09-16T07:39:02.957 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.WbZdnw7jsX 2024-09-16T07:39:02.993 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/data/mon.c.tgz 2024-09-16T07:39:02.993 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-16T07:39:03.066 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.0PTHBHUrTV 2024-09-16T07:39:03.067 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.0PTHBHUrTV 2024-09-16T07:39:03.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-16T07:39:03.441 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.0PTHBHUrTV 2024-09-16T07:39:03.534 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.0PTHBHUrTV is 2MB 2024-09-16T07:39:03.641 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.0PTHBHUrTV 2024-09-16T07:39:03.676 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T07:39:03.676 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T07:39:03.681 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T07:39:03.809 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T07:39:03.844 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T07:39:03.844 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T07:39:03.846 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/remote/smithi071/crash 2024-09-16T07:39:03.847 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T07:39:03.905 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/remote/smithi177/crash 2024-09-16T07:39:03.906 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T07:39:03.960 INFO:tasks.ceph:Shutting down logrotate 2024-09-16T07:39:03.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:39:03.966 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T07:39:04.076 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T07:39:04.131 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T07:39:04.186 INFO:tasks.ceph:Compressing logs... 2024-09-16T07:39:04.186 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T07:39:04.191 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T07:39:04.240 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T07:39:04.241 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52688.log 2024-09-16T07:39:04.242 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67985.log 2024-09-16T07:39:04.243 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73851.log 2024-09-16T07:39:04.243 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52688.log.gz 2024-09-16T07:39:04.244 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79332.log 2024-09-16T07:39:04.244 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67985.log.gz 2024-09-16T07:39:04.245 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41587.log 2024-09-16T07:39:04.245 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44163.log 2024-09-16T07:39:04.245 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73851.log.gz 2024-09-16T07:39:04.245 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46699.log 2024-09-16T07:39:04.246 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79332.log.gz 2024-09-16T07:39:04.247 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53503.log 2024-09-16T07:39:04.247 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44163.log.gz 2024-09-16T07:39:04.248 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36235.log 2024-09-16T07:39:04.248 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.3.41587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41587.log.gz 2024-09-16T07:39:04.248 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T07:39:04.248 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.36235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41172.log 2024-09-16T07:39:04.248 INFO:teuthology.orchestra.run.smithi177.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36235.log.gz 2024-09-16T07:39:04.248 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-16T07:39:04.248 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.1.41172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41172.log.gz 2024-09-16T07:39:04.249 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T07:39:04.250 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T07:39:04.250 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36155.log 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60867.log 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46699.log.gz 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58591.log 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53503.log.gz 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42357.log 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60867.log.gz 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.58591.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58591.log.gz 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70704.log 2024-09-16T07:39:04.252 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75162.log 2024-09-16T07:39:04.253 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42357.log.gz 2024-09-16T07:39:04.253 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42051.log 2024-09-16T07:39:04.253 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70704.log.gz 2024-09-16T07:39:04.255 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36035.log 2024-09-16T07:39:04.255 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.36155.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36155.log.gz 2024-09-16T07:39:04.255 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36075.log 2024-09-16T07:39:04.255 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.36035.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T07:39:04.256 INFO:teuthology.orchestra.run.smithi177.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36035.log.gz 2024-09-16T07:39:04.256 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.36075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36275.log 2024-09-16T07:39:04.256 INFO:teuthology.orchestra.run.smithi177.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36075.log.gz 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76059.log 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75162.log.gz 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42051.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42051.log.gz 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52604.log 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70746.log 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76059.log.gz 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77250.log 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52604.log.gz 2024-09-16T07:39:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77376.log 2024-09-16T07:39:04.258 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70746.log.gz 2024-09-16T07:39:04.258 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77250.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77250.log.gz 2024-09-16T07:39:04.258 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57205.log 2024-09-16T07:39:04.261 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36195.log 2024-09-16T07:39:04.261 INFO:teuthology.orchestra.run.smithi177.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T07:39:04.261 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.36275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36315.log 2024-09-16T07:39:04.261 INFO:teuthology.orchestra.run.smithi177.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36275.log.gz 2024-09-16T07:39:04.261 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.36195.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36195.log.gz 2024-09-16T07:39:04.261 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T07:39:04.261 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.36315.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T07:39:04.261 INFO:teuthology.orchestra.run.smithi177.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36315.log.gz 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59242.log 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77376.log.gz 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57205.log.gz 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67249.log 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59242.log.gz 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47376.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80070.log 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56391.log 2024-09-16T07:39:04.263 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67249.log.gz 2024-09-16T07:39:04.264 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67720.log 2024-09-16T07:39:04.264 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.80070.log.gz 2024-09-16T07:39:04.264 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76017.log 2024-09-16T07:39:04.264 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T07:39:04.265 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-16T07:39:04.266 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56391.log.gz 2024-09-16T07:39:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67720.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.67720.log.gz -- 2024-09-16T07:39:04.266 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/ceph-client.admin.48399.log 2024-09-16T07:39:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57862.log 2024-09-16T07:39:04.266 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.76017.log.gz 2024-09-16T07:39:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80694.log 2024-09-16T07:39:04.266 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48399.log.gz 2024-09-16T07:39:04.269 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T07:39:04.269 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-09-16T07:39:04.269 INFO:teuthology.orchestra.run.smithi177.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T07:39:04.269 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T07:39:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75036.log 2024-09-16T07:39:04.270 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57862.log.gz 2024-09-16T07:39:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80694.log: gzip -5 86.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.80694.log.gz /var/log/ceph/ceph-client.admin.61945.log 2024-09-16T07:39:04.270 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-16T07:39:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80988.log 2024-09-16T07:39:04.270 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75036.log.gz 2024-09-16T07:39:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57955.log 2024-09-16T07:39:04.270 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61945.log.gz 2024-09-16T07:39:04.271 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56630.log 2024-09-16T07:39:04.272 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80988.log.gz 2024-09-16T07:39:04.272 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57955.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57955.log.gz 2024-09-16T07:39:04.272 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74868.log 2024-09-16T07:39:04.272 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53545.log 2024-09-16T07:39:04.272 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56630.log.gz 2024-09-16T07:39:04.273 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.36115.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-09-16T07:39:04.274 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46618.log 2024-09-16T07:39:04.274 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74868.log.gz 2024-09-16T07:39:04.274 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46370.log 2024-09-16T07:39:04.274 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53545.log.gz 2024-09-16T07:39:04.274 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46618.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.46618.log.gz 2024-09-16T07:39:04.275 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65302.log 2024-09-16T07:39:04.275 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67588.log 2024-09-16T07:39:04.275 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46370.log.gz 2024-09-16T07:39:04.275 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79248.log 2024-09-16T07:39:04.276 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.65302.log.gz 2024-09-16T07:39:04.276 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50768.log 2024-09-16T07:39:04.276 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67588.log.gz 2024-09-16T07:39:04.277 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64716.log 2024-09-16T07:39:04.277 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79248.log.gz 2024-09-16T07:39:04.278 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52352.log 2024-09-16T07:39:04.278 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50768.log.gz 2024-09-16T07:39:04.279 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49948.log 2024-09-16T07:39:04.279 INFO:teuthology.orchestra.run.smithi071.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.64716.log.gz 2024-09-16T07:39:04.280 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52352.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52352.log.gz 2024-09-16T07:39:04.280 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43169.log 2024-09-16T07:39:04.281 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79651.log 2024-09-16T07:39:04.281 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49948.log.gz 2024-09-16T07:39:04.282 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73809.log 2024-09-16T07:39:04.282 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43169.log.gz 2024-09-16T07:39:04.282 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42735.log 2024-09-16T07:39:04.283 INFO:teuthology.orchestra.run.smithi071.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.79651.log.gz 2024-09-16T07:39:04.283 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46994.log 2024-09-16T07:39:04.283 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73809.log.gz 2024-09-16T07:39:04.284 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61901.log 2024-09-16T07:39:04.284 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42735.log.gz 2024-09-16T07:39:04.285 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54509.log 2024-09-16T07:39:04.285 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46994.log.gz 2024-09-16T07:39:04.286 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82016.log 2024-09-16T07:39:04.286 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61901.log.gz 2024-09-16T07:39:04.287 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58337.log 2024-09-16T07:39:04.287 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54509.log.gz 2024-09-16T07:39:04.288 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64001.log 2024-09-16T07:39:04.288 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82016.log.gz 2024-09-16T07:39:04.289 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79903.log 2024-09-16T07:39:04.289 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58337.log.gz 2024-09-16T07:39:04.290 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71445.log 2024-09-16T07:39:04.290 INFO:teuthology.orchestra.run.smithi071.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.64001.log.gz 2024-09-16T07:39:04.290 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82100.log 2024-09-16T07:39:04.291 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.79903.log.gz 2024-09-16T07:39:04.291 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65764.log 2024-09-16T07:39:04.291 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71445.log.gz 2024-09-16T07:39:04.292 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79861.log 2024-09-16T07:39:04.292 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82100.log.gz 2024-09-16T07:39:04.293 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65596.log 2024-09-16T07:39:04.293 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65764.log.gz 2024-09-16T07:39:04.294 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79861.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.79861.log.gz 2024-09-16T07:39:04.294 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80610.log 2024-09-16T07:39:04.295 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59410.log 2024-09-16T07:39:04.295 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.65596.log.gz 2024-09-16T07:39:04.296 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66787.log 2024-09-16T07:39:04.296 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80610.log.gz 2024-09-16T07:39:04.296 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75288.log 2024-09-16T07:39:04.297 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.59410.log.gz 2024-09-16T07:39:04.297 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73290.log 2024-09-16T07:39:04.297 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66787.log.gz 2024-09-16T07:39:04.298 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61390.log 2024-09-16T07:39:04.298 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75288.log.gz 2024-09-16T07:39:04.299 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69680.log 2024-09-16T07:39:04.300 INFO:teuthology.orchestra.run.smithi071.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.73290.log.gz 2024-09-16T07:39:04.300 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54337.log 2024-09-16T07:39:04.300 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61390.log.gz 2024-09-16T07:39:04.301 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68027.log 2024-09-16T07:39:04.301 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.69680.log.gz 2024-09-16T07:39:04.302 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70307.log 2024-09-16T07:39:04.302 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54337.log.gz 2024-09-16T07:39:04.303 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68027.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68027.log.gz 2024-09-16T07:39:04.303 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63707.log 2024-09-16T07:39:04.304 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81615.log 2024-09-16T07:39:04.304 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70307.log.gz 2024-09-16T07:39:04.304 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66745.log 2024-09-16T07:39:04.304 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.63707.log.gz 2024-09-16T07:39:04.305 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55262.log 2024-09-16T07:39:04.305 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.81615.log.gz 2024-09-16T07:39:04.306 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66745.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66745.log.gz 2024-09-16T07:39:04.306 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58211.log 2024-09-16T07:39:04.307 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80467.log 2024-09-16T07:39:04.307 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55262.log.gz 2024-09-16T07:39:04.308 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59819.log 2024-09-16T07:39:04.308 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58211.log.gz 2024-09-16T07:39:04.308 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36377.log 2024-09-16T07:39:04.309 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80467.log.gz 2024-09-16T07:39:04.309 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69326.log 2024-09-16T07:39:04.310 INFO:teuthology.orchestra.run.smithi071.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.59819.log.gz 2024-09-16T07:39:04.310 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66310.log 2024-09-16T07:39:04.310 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36377.log.gz 2024-09-16T07:39:04.311 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-09-16T07:39:04.311 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.69326.log.gz 2024-09-16T07:39:04.312 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43428.log 2024-09-16T07:39:04.312 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66310.log.gz 2024-09-16T07:39:04.313 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59368.log 2024-09-16T07:39:04.313 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-09-16T07:39:04.314 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72198.log 2024-09-16T07:39:04.314 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43428.log.gz 2024-09-16T07:39:04.315 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78094.log 2024-09-16T07:39:04.315 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.59368.log.gz 2024-09-16T07:39:04.315 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80904.log 2024-09-16T07:39:04.315 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72198.log.gz 2024-09-16T07:39:04.316 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78094.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78094.log.gz 2024-09-16T07:39:04.316 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61348.log 2024-09-16T07:39:04.317 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73434.log 2024-09-16T07:39:04.317 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80904.log.gz 2024-09-16T07:39:04.318 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65806.log 2024-09-16T07:39:04.318 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61348.log.gz 2024-09-16T07:39:04.319 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63539.log 2024-09-16T07:39:04.319 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73434.log.gz 2024-09-16T07:39:04.319 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65806.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.65806.log.gz 2024-09-16T07:39:04.320 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T07:39:04.321 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52814.log 2024-09-16T07:39:04.321 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63539.log.gz 2024-09-16T07:39:04.321 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74355.log 2024-09-16T07:39:04.322 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48105.log 2024-09-16T07:39:04.322 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.52814.log.gz 2024-09-16T07:39:04.323 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74355.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74355.log.gz 2024-09-16T07:39:04.323 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46041.log 2024-09-16T07:39:04.324 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63749.log 2024-09-16T07:39:04.324 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.48105.log.gz 2024-09-16T07:39:04.325 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46041.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46041.log.gz 2024-09-16T07:39:04.326 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63052.log 2024-09-16T07:39:04.326 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75120.log 2024-09-16T07:39:04.326 INFO:teuthology.orchestra.run.smithi071.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.63749.log.gz 2024-09-16T07:39:04.327 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63052.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.63052.log.gz 2024-09-16T07:39:04.327 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45951.log 2024-09-16T07:39:04.328 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72937.log 2024-09-16T07:39:04.328 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75120.log.gz 2024-09-16T07:39:04.329 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45951.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45951.log.gz 2024-09-16T07:39:04.329 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74145.log 2024-09-16T07:39:04.330 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75724.log 2024-09-16T07:39:04.331 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72937.log.gz 2024-09-16T07:39:04.331 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74145.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74145.log.gz 2024-09-16T07:39:04.331 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62679.log 2024-09-16T07:39:04.332 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.75724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75724.log.gz 2024-09-16T07:39:04.332 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46244.log 2024-09-16T07:39:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.62679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62679.log.gz 2024-09-16T07:39:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57756.log 2024-09-16T07:39:04.334 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75330.log 2024-09-16T07:39:04.334 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46244.log.gz 2024-09-16T07:39:04.335 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57756.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57756.log.gz 2024-09-16T07:39:04.335 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61987.log 2024-09-16T07:39:04.336 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54845.log 2024-09-16T07:39:04.337 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75330.log.gz 2024-09-16T07:39:04.337 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61987.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61987.log.gz 2024-09-16T07:39:04.337 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49255.log 2024-09-16T07:39:04.338 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81030.log 2024-09-16T07:39:04.338 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54845.log.gz 2024-09-16T07:39:04.339 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49255.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49255.log.gz 2024-09-16T07:39:04.339 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43862.log 2024-09-16T07:39:04.340 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54551.log 2024-09-16T07:39:04.341 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.81030.log.gz 2024-09-16T07:39:04.341 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43862.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43862.log.gz 2024-09-16T07:39:04.341 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73021.log 2024-09-16T07:39:04.342 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56934.log 2024-09-16T07:39:04.343 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54551.log.gz 2024-09-16T07:39:04.343 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73021.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73021.log.gz 2024-09-16T07:39:04.343 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36545.log 2024-09-16T07:39:04.345 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71087.log 2024-09-16T07:39:04.345 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56934.log.gz 2024-09-16T07:39:04.346 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36545.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36545.log.gz 2024-09-16T07:39:04.346 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59623.log 2024-09-16T07:39:04.346 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50074.log 2024-09-16T07:39:04.346 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71087.log.gz 2024-09-16T07:39:04.347 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59623.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.59623.log.gz 2024-09-16T07:39:04.347 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-09-16T07:39:04.348 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-09-16T07:39:04.348 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50074.log.gz 2024-09-16T07:39:04.349 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49822.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-09-16T07:39:04.349 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45262.log 2024-09-16T07:39:04.350 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69149.log 2024-09-16T07:39:04.350 INFO:teuthology.orchestra.run.smithi071.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-09-16T07:39:04.351 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.2.45262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45262.log.gz 2024-09-16T07:39:04.351 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56976.log 2024-09-16T07:39:04.352 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76647.log 2024-09-16T07:39:04.352 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69149.log.gz 2024-09-16T07:39:04.353 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56976.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56976.log.gz 2024-09-16T07:39:04.353 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2024-09-16T07:39:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78453.log 2024-09-16T07:39:04.355 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.76647.log.gz 2024-09-16T07:39:04.355 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52031.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz 2024-09-16T07:39:04.355 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36893.log 2024-09-16T07:39:04.356 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79735.log 2024-09-16T07:39:04.357 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78453.log.gz 2024-09-16T07:39:04.357 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36893.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36893.log.gz 2024-09-16T07:39:04.357 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59536.log 2024-09-16T07:39:04.358 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43510.log 2024-09-16T07:39:04.358 INFO:teuthology.orchestra.run.smithi071.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.79735.log.gz 2024-09-16T07:39:04.359 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59536.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59536.log.gz 2024-09-16T07:39:04.359 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80382.log 2024-09-16T07:39:04.360 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82634.log 2024-09-16T07:39:04.360 INFO:teuthology.orchestra.run.smithi071.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.43510.log.gz 2024-09-16T07:39:04.361 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80382.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80382.log.gz 2024-09-16T07:39:04.361 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63010.log 2024-09-16T07:39:04.362 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79038.log 2024-09-16T07:39:04.362 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82634.log.gz 2024-09-16T07:39:04.363 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63010.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63010.log.gz 2024-09-16T07:39:04.363 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52958.log 2024-09-16T07:39:04.364 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58922.log 2024-09-16T07:39:04.364 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79038.log.gz 2024-09-16T07:39:04.365 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52958.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52958.log.gz 2024-09-16T07:39:04.365 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69596.log 2024-09-16T07:39:04.366 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71403.log 2024-09-16T07:39:04.366 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58922.log.gz 2024-09-16T07:39:04.367 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69596.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69596.log.gz 2024-09-16T07:39:04.367 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73105.log 2024-09-16T07:39:04.368 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62113.log 2024-09-16T07:39:04.368 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71403.log.gz 2024-09-16T07:39:04.369 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73105.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73105.log.gz 2024-09-16T07:39:04.369 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57331.log 2024-09-16T07:39:04.370 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78225.log 2024-09-16T07:39:04.370 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62113.log.gz 2024-09-16T07:39:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57331.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.57331.log.gz 2024-09-16T07:39:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65512.log 2024-09-16T07:39:04.372 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66913.log 2024-09-16T07:39:04.373 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78225.log.gz 2024-09-16T07:39:04.373 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65512.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65512.log.gz 2024-09-16T07:39:04.373 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70620.log 2024-09-16T07:39:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53839.log 2024-09-16T07:39:04.374 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66913.log.gz 2024-09-16T07:39:04.375 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70620.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70620.log.gz 2024-09-16T07:39:04.375 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63623.log 2024-09-16T07:39:04.376 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47208.log 2024-09-16T07:39:04.376 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53839.log.gz 2024-09-16T07:39:04.377 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63623.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63623.log.gz 2024-09-16T07:39:04.377 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49780.log 2024-09-16T07:39:04.378 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73767.log 2024-09-16T07:39:04.378 INFO:teuthology.orchestra.run.smithi071.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47208.log.gz 2024-09-16T07:39:04.379 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49780.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49780.log.gz 2024-09-16T07:39:04.379 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50954.log 2024-09-16T07:39:04.380 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64566.log 2024-09-16T07:39:04.381 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73767.log.gz 2024-09-16T07:39:04.381 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50954.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50954.log.gz 2024-09-16T07:39:04.381 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77586.log 2024-09-16T07:39:04.382 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78327.log 2024-09-16T07:39:04.382 INFO:teuthology.orchestra.run.smithi071.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.64566.log.gz 2024-09-16T07:39:04.383 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77586.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.64430.log 2024-09-16T07:39:04.383 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77586.log.gz 2024-09-16T07:39:04.384 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64168.log 2024-09-16T07:39:04.384 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78327.log.gz 2024-09-16T07:39:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43165.log 2024-09-16T07:39:04.386 INFO:teuthology.orchestra.run.smithi071.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.64430.log.gz 2024-09-16T07:39:04.386 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60636.log 2024-09-16T07:39:04.386 INFO:teuthology.orchestra.run.smithi071.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.64168.log.gz 2024-09-16T07:39:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51371.log 2024-09-16T07:39:04.387 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43165.log.gz 2024-09-16T07:39:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60636.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60636.log.gz 2024-09-16T07:39:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79819.log 2024-09-16T07:39:04.389 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76242.log 2024-09-16T07:39:04.389 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51371.log.gz 2024-09-16T07:39:04.390 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66100.log 2024-09-16T07:39:04.390 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.79819.log.gz 2024-09-16T07:39:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80154.log 2024-09-16T07:39:04.391 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76242.log.gz 2024-09-16T07:39:04.392 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66100.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.66100.log.gz 2024-09-16T07:39:04.392 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74492.log 2024-09-16T07:39:04.393 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47646.log 2024-09-16T07:39:04.393 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.80154.log.gz 2024-09-16T07:39:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74492.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74492.log.gz 2024-09-16T07:39:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44469.log 2024-09-16T07:39:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70921.log 2024-09-16T07:39:04.395 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47646.log.gz 2024-09-16T07:39:04.396 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44469.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44469.log.gz 2024-09-16T07:39:04.396 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76773.log 2024-09-16T07:39:04.397 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74910.log 2024-09-16T07:39:04.397 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70921.log.gz 2024-09-16T07:39:04.398 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76773.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76773.log.gz 2024-09-16T07:39:04.398 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46454.log 2024-09-16T07:39:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62800.log 2024-09-16T07:39:04.400 INFO:teuthology.orchestra.run.smithi071.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74910.log.gz 2024-09-16T07:39:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46454.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46454.log.gz 2024-09-16T07:39:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69638.log 2024-09-16T07:39:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42525.log 2024-09-16T07:39:04.402 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62800.log.gz 2024-09-16T07:39:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69638.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.69638.log.gz 2024-09-16T07:39:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46826.log 2024-09-16T07:39:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67165.log 2024-09-16T07:39:04.403 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42525.log.gz 2024-09-16T07:39:04.404 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46826.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46826.log.gz 2024-09-16T07:39:04.404 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79206.log 2024-09-16T07:39:04.405 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60576.log 2024-09-16T07:39:04.405 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67165.log.gz 2024-09-16T07:39:04.406 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79206.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79206.log.gz 2024-09-16T07:39:04.406 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66661.log 2024-09-16T07:39:04.407 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62281.log 2024-09-16T07:39:04.407 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.60576.log.gz 2024-09-16T07:39:04.408 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66661.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66661.log.gz 2024-09-16T07:39:04.408 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76420.log 2024-09-16T07:39:04.409 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49069.log 2024-09-16T07:39:04.409 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62281.log.gz 2024-09-16T07:39:04.410 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76420.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.76420.log.gz 2024-09-16T07:39:04.410 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64303.log 2024-09-16T07:39:04.411 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79986.log 2024-09-16T07:39:04.412 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49069.log.gz 2024-09-16T07:39:04.412 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64303.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.64303.log.gz 2024-09-16T07:39:04.412 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44289.log 2024-09-16T07:39:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50291.log 2024-09-16T07:39:04.413 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.79986.log.gz 2024-09-16T07:39:04.414 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44289.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44289.log.gz 2024-09-16T07:39:04.414 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80568.log 2024-09-16T07:39:04.415 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44121.log 2024-09-16T07:39:04.415 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50291.log.gz 2024-09-16T07:39:04.416 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81279.log 2024-09-16T07:39:04.416 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.80568.log.gz 2024-09-16T07:39:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66871.log 2024-09-16T07:39:04.417 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44121.log.gz 2024-09-16T07:39:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81279.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81279.log.gz 2024-09-16T07:39:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52730.log 2024-09-16T07:39:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50456.log 2024-09-16T07:39:04.419 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66871.log.gz 2024-09-16T07:39:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52730.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52730.log.gz 2024-09-16T07:39:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49654.log 2024-09-16T07:39:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49570.log 2024-09-16T07:39:04.422 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50456.log.gz 2024-09-16T07:39:04.422 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49654.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49654.log.gz 2024-09-16T07:39:04.422 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62758.log 2024-09-16T07:39:04.423 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71655.log 2024-09-16T07:39:04.424 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49570.log.gz 2024-09-16T07:39:04.424 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62758.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62758.log.gz 2024-09-16T07:39:04.424 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79080.log 2024-09-16T07:39:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68153.log 2024-09-16T07:39:04.426 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71655.log.gz 2024-09-16T07:39:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79080.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79080.log.gz 2024-09-16T07:39:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70055.log 2024-09-16T07:39:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78996.log 2024-09-16T07:39:04.428 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68153.log.gz 2024-09-16T07:39:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70055.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70055.log.gz 2024-09-16T07:39:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61772.log 2024-09-16T07:39:04.429 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79164.log 2024-09-16T07:39:04.430 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78996.log.gz 2024-09-16T07:39:04.430 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61772.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61772.log.gz 2024-09-16T07:39:04.430 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56097.log 2024-09-16T07:39:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41947.log 2024-09-16T07:39:04.432 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79164.log.gz 2024-09-16T07:39:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56097.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56097.log.gz 2024-09-16T07:39:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63329.log 2024-09-16T07:39:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81405.log 2024-09-16T07:39:04.434 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41947.log.gz 2024-09-16T07:39:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63329.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63329.log.gz 2024-09-16T07:39:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77334.log 2024-09-16T07:39:04.435 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60731.log 2024-09-16T07:39:04.436 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81405.log.gz 2024-09-16T07:39:04.436 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77334.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77334.log.gz 2024-09-16T07:39:04.436 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70578.log 2024-09-16T07:39:04.437 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52310.log 2024-09-16T07:39:04.437 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60731.log.gz 2024-09-16T07:39:04.438 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46868.log 2024-09-16T07:39:04.438 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70578.log.gz 2024-09-16T07:39:04.439 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54593.log 2024-09-16T07:39:04.440 INFO:teuthology.orchestra.run.smithi071.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.52310.log.gz 2024-09-16T07:39:04.440 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46868.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46868.log.gz 2024-09-16T07:39:04.440 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48807.log 2024-09-16T07:39:04.441 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81573.log 2024-09-16T07:39:04.441 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54593.log.gz 2024-09-16T07:39:04.442 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48807.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48807.log.gz 2024-09-16T07:39:04.442 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66352.log 2024-09-16T07:39:04.443 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36639.log 2024-09-16T07:39:04.444 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81573.log.gz 2024-09-16T07:39:04.444 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66352.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.66352.log.gz 2024-09-16T07:39:04.444 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64840.log 2024-09-16T07:39:04.445 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77849.log 2024-09-16T07:39:04.445 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36639.log.gz 2024-09-16T07:39:04.446 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64840.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.64840.log.gz 2024-09-16T07:39:04.446 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52520.log 2024-09-16T07:39:04.447 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74229.log 2024-09-16T07:39:04.448 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77849.log.gz 2024-09-16T07:39:04.448 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52520.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.52520.log.gz 2024-09-16T07:39:04.448 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65428.log 2024-09-16T07:39:04.449 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47604.log 2024-09-16T07:39:04.449 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74229.log.gz 2024-09-16T07:39:04.450 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65428.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65428.log.gz 2024-09-16T07:39:04.450 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48357.log 2024-09-16T07:39:04.451 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51497.log 2024-09-16T07:39:04.451 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47604.log.gz 2024-09-16T07:39:04.452 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48357.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.48357.log.gz 2024-09-16T07:39:04.452 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48273.log 2024-09-16T07:39:04.453 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80298.log 2024-09-16T07:39:04.453 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51497.log.gz 2024-09-16T07:39:04.454 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54635.log 2024-09-16T07:39:04.454 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48273.log.gz 2024-09-16T07:39:04.456 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81657.log 2024-09-16T07:39:04.456 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80298.log.gz 2024-09-16T07:39:04.456 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54635.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54635.log.gz 2024-09-16T07:39:04.456 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60825.log 2024-09-16T07:39:04.457 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78139.log 2024-09-16T07:39:04.457 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81657.log.gz 2024-09-16T07:39:04.458 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60825.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60825.log.gz 2024-09-16T07:39:04.458 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73332.log 2024-09-16T07:39:04.459 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80509.log 2024-09-16T07:39:04.460 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78139.log.gz 2024-09-16T07:39:04.460 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73332.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73332.log.gz 2024-09-16T07:39:04.460 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58085.log 2024-09-16T07:39:04.461 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52436.log 2024-09-16T07:39:04.461 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80509.log.gz 2024-09-16T07:39:04.462 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58085.log: gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.72366.log -- replaced with /var/log/ceph/ceph-client.admin.58085.log.gz 2024-09-16T07:39:04.462 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-16T07:39:04.463 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63665.log 2024-09-16T07:39:04.463 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52436.log.gz 2024-09-16T07:39:04.464 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72366.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72366.log.gz 2024-09-16T07:39:04.464 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78369.log 2024-09-16T07:39:04.465 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36644.log 2024-09-16T07:39:04.465 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.63665.log.gz 2024-09-16T07:39:04.466 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43836.log 2024-09-16T07:39:04.466 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78369.log.gz 2024-09-16T07:39:04.467 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45468.log 2024-09-16T07:39:04.467 INFO:teuthology.orchestra.run.smithi071.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36644.log.gz 2024-09-16T07:39:04.468 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43836.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43836.log.gz 2024-09-16T07:39:04.468 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69242.log 2024-09-16T07:39:04.469 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42777.log 2024-09-16T07:39:04.469 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45468.log.gz 2024-09-16T07:39:04.470 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56265.log 2024-09-16T07:39:04.470 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69242.log.gz 2024-09-16T07:39:04.471 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42777.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42777.log.gz 2024-09-16T07:39:04.471 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59910.log 2024-09-16T07:39:04.472 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76546.log 2024-09-16T07:39:04.472 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56265.log.gz 2024-09-16T07:39:04.473 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59910.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.59910.log.gz 2024-09-16T07:39:04.473 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75975.log 2024-09-16T07:39:04.474 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72450.log 2024-09-16T07:39:04.474 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76546.log.gz 2024-09-16T07:39:04.475 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75975.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75975.log.gz -5 2024-09-16T07:39:04.475 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51581.log 2024-09-16T07:39:04.476 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50912.log 2024-09-16T07:39:04.476 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72450.log.gz 2024-09-16T07:39:04.477 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51581.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51581.log.gz 2024-09-16T07:39:04.477 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74061.log 2024-09-16T07:39:04.478 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41789.log 2024-09-16T07:39:04.478 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50912.log.gz 2024-09-16T07:39:04.479 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74061.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74061.log.gz 2024-09-16T07:39:04.479 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76101.log 2024-09-16T07:39:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80736.log 2024-09-16T07:39:04.480 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41789.log.gz 2024-09-16T07:39:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76101.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76101.log.gz 2024-09-16T07:39:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65890.log 2024-09-16T07:39:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62425.log 2024-09-16T07:39:04.482 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80736.log.gz 2024-09-16T07:39:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65890.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65890.log.gz 2024-09-16T07:39:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76988.log 2024-09-16T07:39:04.484 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72651.log 2024-09-16T07:39:04.485 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62425.log.gz 2024-09-16T07:39:04.485 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76988.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.76988.log.gz 2024-09-16T07:39:04.485 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46328.log 2024-09-16T07:39:04.486 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72651.log: gzip -5 87.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.72651.log.gz -- 2024-09-16T07:39:04.486 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/ceph-client.admin.50996.log 2024-09-16T07:39:04.487 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43436.log 2024-09-16T07:39:04.487 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46328.log.gz 2024-09-16T07:39:04.488 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50996.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50996.log.gz 2024-09-16T07:39:04.488 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66226.log 2024-09-16T07:39:04.489 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75933.log 2024-09-16T07:39:04.489 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43436.log.gz 2024-09-16T07:39:04.490 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66226.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.66226.log.gz 2024-09-16T07:39:04.490 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36237.log 2024-09-16T07:39:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66016.log 2024-09-16T07:39:04.491 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75933.log.gz 2024-09-16T07:39:04.492 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36237.log: 76.4% -- replaced with /var/log/ceph/ceph-client.admin.36237.log.gz 2024-09-16T07:39:04.492 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54761.log 2024-09-16T07:39:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68321.log 2024-09-16T07:39:04.493 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66016.log.gz 2024-09-16T07:39:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54761.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54761.log.gz 2024-09-16T07:39:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T07:39:04.495 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80340.log 2024-09-16T07:39:04.495 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68321.log.gz 2024-09-16T07:39:04.496 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64525.log 2024-09-16T07:39:04.497 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46952.log 2024-09-16T07:39:04.498 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80340.log: /var/log/ceph/ceph-client.admin.64525.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80340.log.gz 2024-09-16T07:39:04.498 INFO:teuthology.orchestra.run.smithi071.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.64525.log.gz 2024-09-16T07:39:04.498 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68617.log 2024-09-16T07:39:04.499 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63833.log 2024-09-16T07:39:04.500 INFO:teuthology.orchestra.run.smithi071.stderr: 90.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T07:39:04.500 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46952.log.gz 2024-09-16T07:39:04.500 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.68617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68617.log.gz 2024-09-16T07:39:04.500 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77082.log 2024-09-16T07:39:04.501 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79945.log 2024-09-16T07:39:04.501 INFO:teuthology.orchestra.run.smithi071.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.63833.log.gz 2024-09-16T07:39:04.502 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70494.log 2024-09-16T07:39:04.502 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77082.log.gz 2024-09-16T07:39:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79945.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.79945.log.gz 2024-09-16T07:39:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66619.log 2024-09-16T07:39:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56535.log 2024-09-16T07:39:04.504 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70494.log.gz 2024-09-16T07:39:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66619.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66619.log.gz 2024-09-16T07:39:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71823.log 2024-09-16T07:39:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47502.log 2024-09-16T07:39:04.506 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56535.log.gz 2024-09-16T07:39:04.507 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43443.log 2024-09-16T07:39:04.507 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71823.log.gz 2024-09-16T07:39:04.508 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47688.log 2024-09-16T07:39:04.508 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47502.log.gz 2024-09-16T07:39:04.509 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43443.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43443.log.gz 2024-09-16T07:39:04.509 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74737.log 2024-09-16T07:39:04.510 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69722.log 2024-09-16T07:39:04.510 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47688.log.gz 2024-09-16T07:39:04.511 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74737.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74737.log.gz 2024-09-16T07:39:04.511 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49027.log 2024-09-16T07:39:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82295.log 2024-09-16T07:39:04.512 INFO:teuthology.orchestra.run.smithi071.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.69722.log.gz 2024-09-16T07:39:04.513 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49027.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49027.log.gz 2024-09-16T07:39:04.513 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57672.log 2024-09-16T07:39:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61856.log 2024-09-16T07:39:04.514 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82295.log.gz 2024-09-16T07:39:04.515 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57672.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57672.log.gz 2024-09-16T07:39:04.515 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67771.log 2024-09-16T07:39:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54242.log 2024-09-16T07:39:04.516 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61856.log.gz 2024-09-16T07:39:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67771.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67771.log.gz 2024-09-16T07:39:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54887.log 2024-09-16T07:39:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75849.log 2024-09-16T07:39:04.518 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54242.log.gz 2024-09-16T07:39:04.519 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50545.log 2024-09-16T07:39:04.519 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54887.log.gz 2024-09-16T07:39:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75849.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75849.log.gz 2024-09-16T07:39:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61691.log 2024-09-16T07:39:04.521 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60240.log 2024-09-16T07:39:04.521 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50545.log.gz 2024-09-16T07:39:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61691.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61691.log.gz 2024-09-16T07:39:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65932.log 2024-09-16T07:39:04.523 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48849.log 2024-09-16T07:39:04.523 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.60240.log.gz 2024-09-16T07:39:04.524 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65932.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.65932.log.gz 2024-09-16T07:39:04.524 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69961.log 2024-09-16T07:39:04.525 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80820.log 2024-09-16T07:39:04.525 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48849.log.gz 2024-09-16T07:39:04.526 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69961.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69961.log.gz 2024-09-16T07:39:04.526 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42441.log 2024-09-16T07:39:04.527 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67546.log 2024-09-16T07:39:04.527 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80820.log.gz 2024-09-16T07:39:04.528 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42441.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42441.log.gz 2024-09-16T07:39:04.528 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82381.log 2024-09-16T07:39:04.529 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45762.log 2024-09-16T07:39:04.529 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67546.log.gz 2024-09-16T07:39:04.530 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82381.log: gzip -5 86.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.82381.log.gz -- 2024-09-16T07:39:04.530 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/ceph-client.admin.56850.log 2024-09-16T07:39:04.531 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80214.log 2024-09-16T07:39:04.531 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45762.log.gz 2024-09-16T07:39:04.532 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56850.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56850.log.gz 2024-09-16T07:39:04.532 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47166.log 2024-09-16T07:39:04.533 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64097.log 2024-09-16T07:39:04.533 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80214.log.gz 2024-09-16T07:39:04.534 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47166.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47166.log.gz 2024-09-16T07:39:04.534 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60450.log 2024-09-16T07:39:04.535 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57415.log 2024-09-16T07:39:04.535 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.64097.log.gz 2024-09-16T07:39:04.536 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60450.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.60450.log.gz 2024-09-16T07:39:04.536 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78183.log 2024-09-16T07:39:04.537 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47036.log 2024-09-16T07:39:04.537 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57415.log.gz 2024-09-16T07:39:04.538 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78183.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78183.log.gz 2024-09-16T07:39:04.538 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53587.log 2024-09-16T07:39:04.540 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73164.log 2024-09-16T07:39:04.540 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47036.log.gz 2024-09-16T07:39:04.540 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53587.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53587.log.gz 2024-09-16T07:39:04.540 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79693.log 2024-09-16T07:39:04.541 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43150.log 2024-09-16T07:39:04.541 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73164.log.gz 2024-09-16T07:39:04.542 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79693.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.79693.log.gz 2024-09-16T07:39:04.542 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70097.log 2024-09-16T07:39:04.543 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57289.log 2024-09-16T07:39:04.543 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43150.log.gz 2024-09-16T07:39:04.544 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70097.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70097.log.gz 2024-09-16T07:39:04.544 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50596.log 2024-09-16T07:39:04.545 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42227.log 2024-09-16T07:39:04.545 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57289.log.gz 2024-09-16T07:39:04.546 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50596.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50596.log.gz 2024-09-16T07:39:04.546 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71319.log 2024-09-16T07:39:04.547 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66525.log 2024-09-16T07:39:04.547 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42227.log.gz 2024-09-16T07:39:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71319.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71319.log.gz 2024-09-16T07:39:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53189.log 2024-09-16T07:39:04.549 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53400.log 2024-09-16T07:39:04.549 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66525.log.gz 2024-09-16T07:39:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53189.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53189.log.gz 2024-09-16T07:39:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47292.log 2024-09-16T07:39:04.551 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36509.log 2024-09-16T07:39:04.551 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53400.log.gz 2024-09-16T07:39:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47292.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47292.log.gz 2024-09-16T07:39:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50211.log 2024-09-16T07:39:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56766.log 2024-09-16T07:39:04.553 INFO:teuthology.orchestra.run.smithi071.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.36509.log.gz 2024-09-16T07:39:04.554 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50211.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50211.log.gz 2024-09-16T07:39:04.554 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70181.log 2024-09-16T07:39:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78495.log 2024-09-16T07:39:04.555 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56766.log.gz 2024-09-16T07:39:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70181.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70181.log.gz 2024-09-16T07:39:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56475.log 2024-09-16T07:39:04.557 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57590.log 2024-09-16T07:39:04.557 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.78495.log.gz 2024-09-16T07:39:04.558 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56475.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56475.log.gz 2024-09-16T07:39:04.558 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58403.log 2024-09-16T07:39:04.559 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51455.log 2024-09-16T07:39:04.560 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57590.log.gz 2024-09-16T07:39:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58403.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58403.log.gz 2024-09-16T07:39:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46118.log 2024-09-16T07:39:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65722.log 2024-09-16T07:39:04.561 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51455.log.gz 2024-09-16T07:39:04.562 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46118.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46118.log.gz 2024-09-16T07:39:04.562 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63959.log 2024-09-16T07:39:04.563 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60951.log 2024-09-16T07:39:04.563 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.65722.log.gz 2024-09-16T07:39:04.564 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63959.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63959.log.gz 2024-09-16T07:39:04.564 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73893.log 2024-09-16T07:39:04.565 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36286.log 2024-09-16T07:39:04.565 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60951.log.gz 2024-09-16T07:39:04.566 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73893.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73893.log.gz 2024-09-16T07:39:04.566 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82842.log 2024-09-16T07:39:04.567 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43435.log 2024-09-16T07:39:04.567 INFO:teuthology.orchestra.run.smithi071.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.36286.log.gz 2024-09-16T07:39:04.568 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82842.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82842.log.gz 2024-09-16T07:39:04.568 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42903.log 2024-09-16T07:39:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58964.log 2024-09-16T07:39:04.569 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43435.log.gz 2024-09-16T07:39:04.570 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42903.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42903.log.gz 2024-09-16T07:39:04.570 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63371.log 2024-09-16T07:39:04.571 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68237.log 2024-09-16T07:39:04.571 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.58964.log.gz 2024-09-16T07:39:04.572 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63371.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63371.log.gz 2024-09-16T07:39:04.572 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80425.log 2024-09-16T07:39:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58796.log 2024-09-16T07:39:04.573 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68237.log.gz 2024-09-16T07:39:04.574 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64217.log 2024-09-16T07:39:04.574 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.80425.log.gz 2024-09-16T07:39:04.575 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74952.log 2024-09-16T07:39:04.575 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58796.log.gz 2024-09-16T07:39:04.576 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64217.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64217.log.gz 2024-09-16T07:39:04.576 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36111.log 2024-09-16T07:39:04.577 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61035.log 2024-09-16T07:39:04.577 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74952.log.gz 2024-09-16T07:39:04.578 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36111.log.gz 2024-09-16T07:39:04.578 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57997.log 2024-09-16T07:39:04.579 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76731.log 2024-09-16T07:39:04.579 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61035.log.gz 2024-09-16T07:39:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57997.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57997.log.gz 2024-09-16T07:39:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60324.log 2024-09-16T07:39:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55996.log 2024-09-16T07:39:04.581 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76731.log.gz 2024-09-16T07:39:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60324.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60324.log.gz 2024-09-16T07:39:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66829.log 2024-09-16T07:39:04.583 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43500.log 2024-09-16T07:39:04.583 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.55996.log.gz 2024-09-16T07:39:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66829.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66829.log.gz 2024-09-16T07:39:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71235.log 2024-09-16T07:39:04.585 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36132.log 2024-09-16T07:39:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2024-09-16T07:39:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71235.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71235.log.gz 2024-09-16T07:39:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77502.log 2024-09-16T07:39:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36132.log.gz 2024-09-16T07:39:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65344.log 2024-09-16T07:39:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77502.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.77502.log.gz 2024-09-16T07:39:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51038.log 2024-09-16T07:39:04.589 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67291.log 2024-09-16T07:39:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65344.log.gz 2024-09-16T07:39:04.590 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54929.log 2024-09-16T07:39:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51038.log.gz 2024-09-16T07:39:04.591 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81974.log 2024-09-16T07:39:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67291.log.gz 2024-09-16T07:39:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37185.log 2024-09-16T07:39:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54929.log.gz 2024-09-16T07:39:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58838.log 2024-09-16T07:39:04.593 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81974.log.gz 2024-09-16T07:39:04.594 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37185.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37185.log.gz 2024-09-16T07:39:04.594 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66184.log 2024-09-16T07:39:04.595 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62383.log 2024-09-16T07:39:04.595 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58838.log.gz 2024-09-16T07:39:04.596 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66184.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.66184.log.gz 2024-09-16T07:39:04.596 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66268.log 2024-09-16T07:39:04.597 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64902.log 2024-09-16T07:39:04.597 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62383.log.gz 2024-09-16T07:39:04.598 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66268.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66268.log.gz 2024-09-16T07:39:04.598 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71277.log 2024-09-16T07:39:04.599 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68195.log 2024-09-16T07:39:04.599 INFO:teuthology.orchestra.run.smithi071.stderr: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.64902.log.gz 2024-09-16T07:39:04.600 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71277.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71277.log.gz 2024-09-16T07:39:04.600 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69386.log 2024-09-16T07:39:04.601 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80028.log 2024-09-16T07:39:04.601 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.68195.log.gz 2024-09-16T07:39:04.602 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69386.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69386.log.gz 2024-09-16T07:39:04.602 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64634.log 2024-09-16T07:39:04.603 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81741.log 2024-09-16T07:39:04.603 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80028.log.gz 2024-09-16T07:39:04.604 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64634.log: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.64634.log.gz 2024-09-16T07:39:04.604 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49528.log 2024-09-16T07:39:04.605 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76605.log 2024-09-16T07:39:04.606 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81741.log.gz 2024-09-16T07:39:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49528.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49528.log.gz 2024-09-16T07:39:04.606 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67855.log 2024-09-16T07:39:04.607 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70265.log 2024-09-16T07:39:04.607 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76605.log.gz 2024-09-16T07:39:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67855.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67855.log.gz 2024-09-16T07:39:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46160.log 2024-09-16T07:39:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71361.log 2024-09-16T07:39:04.609 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70265.log.gz 2024-09-16T07:39:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46160.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46160.log.gz 2024-09-16T07:39:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72077.log 2024-09-16T07:39:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71003.log 2024-09-16T07:39:04.611 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71361.log.gz 2024-09-16T07:39:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.72077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72077.log.gz 2024-09-16T07:39:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80112.log 2024-09-16T07:39:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36642.log 2024-09-16T07:39:04.613 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71003.log.gz 2024-09-16T07:39:04.614 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80112.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.80112.log.gz 2024-09-16T07:39:04.614 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-16T07:39:04.615 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73206.log 2024-09-16T07:39:04.616 INFO:teuthology.orchestra.run.smithi071.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36642.log.gz 2024-09-16T07:39:04.616 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58880.log 2024-09-16T07:39:04.617 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73206.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.73206.log.gz 2024-09-16T07:39:04.618 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-09-16T07:39:04.618 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56181.log 2024-09-16T07:39:04.619 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58880.log: 87.1%/var/log/ceph/ceph-client.admin.45636.log: -- replaced with /var/log/ceph/ceph-client.admin.58880.log.gz 2024-09-16T07:39:04.619 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-09-16T07:39:04.620 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81321.log 2024-09-16T07:39:04.620 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72156.log 2024-09-16T07:39:04.621 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56181.log.gz 2024-09-16T07:39:04.622 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81321.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81321.log.gz 2024-09-16T07:39:04.622 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75204.log 2024-09-16T07:39:04.622 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.72156.log: -- /var/log/ceph/ceph-client.admin.43838.log 2024-09-16T07:39:04.623 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72156.log.gz 2024-09-16T07:39:04.623 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75204.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75204.log.gz 2024-09-16T07:39:04.624 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75522.log 2024-09-16T07:39:04.624 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5/var/log/ceph/ceph-client.admin.43838.log: --verbose -- /var/log/ceph/ceph-client.admin.61180.log 2024-09-16T07:39:04.625 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43838.log.gz 2024-09-16T07:39:04.626 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75522.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75522.log.gz 2024-09-16T07:39:04.626 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57714.log 2024-09-16T07:39:04.627 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57036.log 2024-09-16T07:39:04.627 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61180.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61180.log.gz 2024-09-16T07:39:04.627 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57714.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57714.log.gz 2024-09-16T07:39:04.628 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2024-09-16T07:39:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54803.log 2024-09-16T07:39:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57036.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57036.log.gz 2024-09-16T07:39:04.630 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42271.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2024-09-16T07:39:04.630 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65386.log 2024-09-16T07:39:04.631 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36624.log 2024-09-16T07:39:04.631 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54803.log.gz 2024-09-16T07:39:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65386.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65386.log.gz 2024-09-16T07:39:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49738.log 2024-09-16T07:39:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49990.log 2024-09-16T07:39:04.633 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36624.log.gz 2024-09-16T07:39:04.634 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49738.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49738.log.gz 2024-09-16T07:39:04.634 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77670.log 2024-09-16T07:39:04.635 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5/var/log/ceph/ceph-client.admin.49990.log: --verbose -- /var/log/ceph/ceph-client.admin.46286.log 2024-09-16T07:39:04.635 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49990.log.gz 2024-09-16T07:39:04.636 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77670.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77670.log.gz 2024-09-16T07:39:04.636 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82718.log 2024-09-16T07:39:04.637 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58712.log 2024-09-16T07:39:04.637 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46286.log.gz 2024-09-16T07:39:04.638 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82718.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82718.log.gz 2024-09-16T07:39:04.638 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55430.log 2024-09-16T07:39:04.638 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50638.log 2024-09-16T07:39:04.639 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58712.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58712.log.gz 2024-09-16T07:39:04.640 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55430.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55430.log.gz 2024-09-16T07:39:04.640 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53315.log 2024-09-16T07:39:04.641 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75078.log 2024-09-16T07:39:04.641 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50638.log.gz 2024-09-16T07:39:04.642 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53315.log.gz 2024-09-16T07:39:04.642 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43782.log 2024-09-16T07:39:04.643 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62842.log 2024-09-16T07:39:04.643 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75078.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75078.log.gz 2024-09-16T07:39:04.644 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43782.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43782.log.gz 2024-09-16T07:39:04.644 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73536.log 2024-09-16T07:39:04.645 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56349.log 2024-09-16T07:39:04.650 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62842.log.gz 2024-09-16T07:39:04.650 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73536.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73536.log.gz 2024-09-16T07:39:04.650 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55346.log 2024-09-16T07:39:04.650 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64345.log 2024-09-16T07:39:04.650 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56349.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56349.log.gz 2024-09-16T07:39:04.650 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55346.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55346.log.gz 2024-09-16T07:39:04.650 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81447.log 2024-09-16T07:39:04.651 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43162.log 2024-09-16T07:39:04.651 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64345.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.64345.log.gz 2024-09-16T07:39:04.651 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81447.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81447.log.gz 2024-09-16T07:39:04.651 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82884.log 2024-09-16T07:39:04.652 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58754.log 2024-09-16T07:39:04.652 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43162.log.gz 2024-09-16T07:39:04.652 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82884.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82884.log.gz 2024-09-16T07:39:04.652 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47730.log 2024-09-16T07:39:04.653 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73374.log 2024-09-16T07:39:04.653 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58754.log.gz 2024-09-16T07:39:04.654 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47730.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47730.log.gz 2024-09-16T07:39:04.654 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80778.log 2024-09-16T07:39:04.655 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65848.log 2024-09-16T07:39:04.655 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73374.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73374.log.gz 2024-09-16T07:39:04.656 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80778.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.80778.log.gz 2024-09-16T07:39:04.656 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73935.log 2024-09-16T07:39:04.657 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65260.log 2024-09-16T07:39:04.657 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.65848.log.gz 2024-09-16T07:39:04.658 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73935.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73935.log.gz 2024-09-16T07:39:04.658 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42399.log 2024-09-16T07:39:04.659 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62491.log 2024-09-16T07:39:04.659 INFO:teuthology.orchestra.run.smithi071.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.65260.log.gz 2024-09-16T07:39:04.660 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42399.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42399.log.gz 2024-09-16T07:39:04.660 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59166.log 2024-09-16T07:39:04.661 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.62491.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56808.log 2024-09-16T07:39:04.661 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62491.log.gz 2024-09-16T07:39:04.662 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59166.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59166.log.gz 2024-09-16T07:39:04.662 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79122.log 2024-09-16T07:39:04.663 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43839.log 2024-09-16T07:39:04.663 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56808.log: /var/log/ceph/ceph-client.admin.79122.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56808.log.gz 2024-09-16T07:39:04.664 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79122.log.gz 2024-09-16T07:39:04.664 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70536.log 2024-09-16T07:39:04.665 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71193.log 2024-09-16T07:39:04.665 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43839.log.gz 2024-09-16T07:39:04.666 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70536.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.70536.log.gz 2024-09-16T07:39:04.666 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56307.log 2024-09-16T07:39:04.667 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76893.log 2024-09-16T07:39:04.667 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71193.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71193.log.gz 2024-09-16T07:39:04.668 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56307.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56307.log.gz 2024-09-16T07:39:04.668 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80256.log 2024-09-16T07:39:04.669 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47124.log 2024-09-16T07:39:04.670 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76893.log.gz 2024-09-16T07:39:04.670 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80256.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80256.log.gz 2024-09-16T07:39:04.670 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50870.log 2024-09-16T07:39:04.671 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.47124.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.71697.log 2024-09-16T07:39:04.672 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47124.log.gz 2024-09-16T07:39:04.672 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50870.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50870.log.gz 2024-09-16T07:39:04.672 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62029.log 2024-09-16T07:39:04.673 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69764.log 2024-09-16T07:39:04.674 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.71697.log.gz 2024-09-16T07:39:04.674 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62029.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62029.log.gz 2024-09-16T07:39:04.675 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62323.log 2024-09-16T07:39:04.675 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82676.log 2024-09-16T07:39:04.676 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69764.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69764.log.gz 2024-09-16T07:39:04.676 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62323.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62323.log.gz 2024-09-16T07:39:04.676 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59951.log 2024-09-16T07:39:04.677 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45720.log 2024-09-16T07:39:04.678 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82676.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.82676.log.gz 2024-09-16T07:39:04.678 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59951.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59951.log.gz 2024-09-16T07:39:04.679 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82760.log 2024-09-16T07:39:04.680 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55304.log 2024-09-16T07:39:04.680 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45720.log.gz 2024-09-16T07:39:04.681 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82760.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82760.log.gz 2024-09-16T07:39:04.681 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65974.log 2024-09-16T07:39:04.682 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72240.log 2024-09-16T07:39:04.682 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55304.log.gz 2024-09-16T07:39:04.683 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80862.log 2024-09-16T07:39:04.683 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.65974.log.gz 2024-09-16T07:39:04.684 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72240.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72240.log.gz 2024-09-16T07:39:04.684 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60993.log 2024-09-16T07:39:04.685 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53460.log 2024-09-16T07:39:04.686 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80862.log.gz 2024-09-16T07:39:04.686 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63875.log 2024-09-16T07:39:04.686 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60993.log.gz 2024-09-16T07:39:04.687 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53460.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53460.log.gz 2024-09-16T07:39:04.687 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50726.log 2024-09-16T07:39:04.688 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63581.log 2024-09-16T07:39:04.689 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63875.log.gz 2024-09-16T07:39:04.689 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50726.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50726.log.gz 2024-09-16T07:39:04.690 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72895.log 2024-09-16T07:39:04.690 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T07:39:04.691 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63581.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.63581.log.gz 2024-09-16T07:39:04.691 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72895.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72895.log.gz 2024-09-16T07:39:04.691 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76504.log 2024-09-16T07:39:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61077.log 2024-09-16T07:39:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76504.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76504.log.gz 2024-09-16T07:39:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75456.log 2024-09-16T07:39:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51413.log 2024-09-16T07:39:04.695 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mon.b.log: 93.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-16T07:39:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61077.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61077.log.gz 2024-09-16T07:39:04.696 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69470.log 2024-09-16T07:39:04.696 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.75456.log.gz 2024-09-16T07:39:04.697 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62071.log 2024-09-16T07:39:04.697 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51413.log.gz 2024-09-16T07:39:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69470.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.69470.log.gz 2024-09-16T07:39:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73476.log 2024-09-16T07:39:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-09-16T07:39:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62071.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62071.log.gz 2024-09-16T07:39:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73476.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73476.log.gz 2024-09-16T07:39:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59494.log 2024-09-16T07:39:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60010.log 2024-09-16T07:39:04.701 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-09-16T07:39:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53797.log 2024-09-16T07:39:04.702 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.59494.log.gz 2024-09-16T07:39:04.703 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60010.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.60010.log.gz 2024-09-16T07:39:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65002.log 2024-09-16T07:39:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63497.log 2024-09-16T07:39:04.705 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53797.log.gz 2024-09-16T07:39:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65002.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.65002.log.gz 2024-09-16T07:39:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70410.log 2024-09-16T07:39:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57800.log 2024-09-16T07:39:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63497.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.63497.log.gz 2024-09-16T07:39:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70410.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70410.log.gz 2024-09-16T07:39:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45594.log 2024-09-16T07:39:04.709 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54719.log 2024-09-16T07:39:04.709 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57800.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57800.log.gz 2024-09-16T07:39:04.710 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45594.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45594.log.gz 2024-09-16T07:39:04.710 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82337.log 2024-09-16T07:39:04.711 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74782.log 2024-09-16T07:39:04.711 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54719.log.gz 2024-09-16T07:39:04.711 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82337.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.82337.log.gz 2024-09-16T07:39:04.712 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52208.log 2024-09-16T07:39:04.712 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61814.log 2024-09-16T07:39:04.713 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74782.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.74782.log.gz 2024-09-16T07:39:04.714 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52208.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.52208.log.gz 2024-09-16T07:39:04.714 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59452.log 2024-09-16T07:39:04.714 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58043.log 2024-09-16T07:39:04.716 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61814.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61814.log.gz/var/log/ceph/ceph-client.admin.59452.log: 2024-09-16T07:39:04.716 INFO:teuthology.orchestra.run.smithi071.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.59452.log.gz 2024-09-16T07:39:04.716 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-09-16T07:39:04.716 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71613.log 2024-09-16T07:39:04.717 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58043.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58043.log.gz 2024-09-16T07:39:04.718 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48967.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-09-16T07:39:04.718 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52772.log 2024-09-16T07:39:04.718 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5/var/log/ceph/ceph-client.admin.71613.log: --verbose -- /var/log/ceph/ceph-client.admin.69554.log 2024-09-16T07:39:04.719 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71613.log.gz 2024-09-16T07:39:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52772.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52772.log.gz 2024-09-16T07:39:04.720 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2024-09-16T07:39:04.720 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68924.log 2024-09-16T07:39:04.721 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69554.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69554.log.gz/var/log/ceph/ceph-client.admin.43163.log: 2024-09-16T07:39:04.722 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2024-09-16T07:39:04.722 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68780.log 2024-09-16T07:39:04.722 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67901.log 2024-09-16T07:39:04.726 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68924.log: 87.2%/var/log/ceph/ceph-client.admin.68780.log: -- replaced with /var/log/ceph/ceph-client.admin.68924.log.gz 2024-09-16T07:39:04.726 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68780.log.gz 2024-09-16T07:39:04.726 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43497.log 2024-09-16T07:39:04.726 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62155.log 2024-09-16T07:39:04.726 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67901.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67901.log.gz 2024-09-16T07:39:04.726 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43497.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43497.log.gz 2024-09-16T07:39:04.726 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65470.log 2024-09-16T07:39:04.726 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72727.log 2024-09-16T07:39:04.727 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62155.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62155.log.gz 2024-09-16T07:39:04.728 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62197.log 2024-09-16T07:39:04.728 INFO:teuthology.orchestra.run.smithi071.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.65470.log.gz 2024-09-16T07:39:04.729 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72727.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.72727.log.gz 2024-09-16T07:39:04.729 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72282.log 2024-09-16T07:39:04.730 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52250.log 2024-09-16T07:39:04.730 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62197.log.gz 2024-09-16T07:39:04.731 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61264.log 2024-09-16T07:39:04.732 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72282.log.gz 2024-09-16T07:39:04.732 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52250.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.52250.log.gz 2024-09-16T07:39:04.732 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55183.log 2024-09-16T07:39:04.733 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61137.log 2024-09-16T07:39:04.734 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61264.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61264.log.gz 2024-09-16T07:39:04.734 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.55183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55183.log.gz 2024-09-16T07:39:04.734 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52166.log 2024-09-16T07:39:04.735 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82592.log 2024-09-16T07:39:04.736 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61137.log: 87.6%/var/log/ceph/ceph-client.admin.52166.log: -- replaced with /var/log/ceph/ceph-client.admin.61137.log.gz 2024-09-16T07:39:04.736 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.52166.log.gz 2024-09-16T07:39:04.736 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67207.log 2024-09-16T07:39:04.737 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56139.log 2024-09-16T07:39:04.738 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82592.log.gz 2024-09-16T07:39:04.739 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68363.log 86.6% 2024-09-16T07:39:04.739 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67207.log.gz 2024-09-16T07:39:04.740 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73063.log 2024-09-16T07:39:04.740 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56139.log.gz 2024-09-16T07:39:04.741 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70662.log 2024-09-16T07:39:04.741 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68363.log.gz 2024-09-16T07:39:04.742 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73063.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T07:39:04.742 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.73063.log.gz 2024-09-16T07:39:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70662.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.70662.log.gz 2024-09-16T07:39:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81531.log 2024-09-16T07:39:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65090.log 2024-09-16T07:39:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74695.log 2024-09-16T07:39:04.745 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81531.log.gz 2024-09-16T07:39:04.747 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42945.log 2024-09-16T07:39:04.747 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65090.log.gz 2024-09-16T07:39:04.748 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74695.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74695.log.gz 2024-09-16T07:39:04.748 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79533.log 2024-09-16T07:39:04.749 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.42945.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57913.log 2024-09-16T07:39:04.749 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42945.log.gz 2024-09-16T07:39:04.752 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79533.log: gzip -5 --verbose -- 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79533.log.gz 2024-09-16T07:39:04.752 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/ceph-client.admin.77544.log 2024-09-16T07:39:04.752 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57913.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57913.log.gz 2024-09-16T07:39:04.752 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47772.log 2024-09-16T07:39:04.753 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60198.log 2024-09-16T07:39:04.753 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.77544.log.gz 2024-09-16T07:39:04.754 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74313.log 2024-09-16T07:39:04.755 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47772.log.gz 2024-09-16T07:39:04.756 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64757.log 2024-09-16T07:39:04.756 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60198.log.gz 2024-09-16T07:39:04.757 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75372.log 2024-09-16T07:39:04.757 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74313.log.gz 2024-09-16T07:39:04.758 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67123.log 2024-09-16T07:39:04.758 INFO:teuthology.orchestra.run.smithi071.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.64757.log.gz 2024-09-16T07:39:04.759 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75372.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76689.log 2024-09-16T07:39:04.759 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75372.log.gz 2024-09-16T07:39:04.760 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55388.log 2024-09-16T07:39:04.760 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67123.log.gz 2024-09-16T07:39:04.762 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46783.log 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76689.log.gz 2024-09-16T07:39:04.762 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-16T07:39:04.763 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55388.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55388.log.gz 2024-09-16T07:39:04.763 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36315.log 2024-09-16T07:39:04.764 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53147.log 2024-09-16T07:39:04.764 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46783.log.gz 2024-09-16T07:39:04.765 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36315.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.36315.log.gz 2024-09-16T07:39:04.765 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43149.log 2024-09-16T07:39:04.766 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58295.log 2024-09-16T07:39:04.767 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53147.log.gz 2024-09-16T07:39:04.768 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42987.log 2024-09-16T07:39:04.768 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43149.log.gz 2024-09-16T07:39:04.769 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58295.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58295.log.gz 2024-09-16T07:39:04.769 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82424.log 2024-09-16T07:39:04.770 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82058.log 2024-09-16T07:39:04.771 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42987.log.gz 2024-09-16T07:39:04.772 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80652.log 2024-09-16T07:39:04.772 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.82424.log.gz 2024-09-16T07:39:04.773 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82058.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82058.log.gz 2024-09-16T07:39:04.773 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81185.log 2024-09-16T07:39:04.774 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48147.log 2024-09-16T07:39:04.774 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.80652.log.gz 2024-09-16T07:39:04.776 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53934.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81185.log.gz 2024-09-16T07:39:04.776 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-16T07:39:04.776 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48147.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48147.log.gz 2024-09-16T07:39:04.777 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55514.log 2024-09-16T07:39:04.778 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51292.log 2024-09-16T07:39:04.778 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53934.log.gz 2024-09-16T07:39:04.779 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55514.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55514.log.gz 2024-09-16T07:39:04.779 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54196.log 2024-09-16T07:39:04.781 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.0.51292.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50680.log 0.0% -- replaced with /var/log/ceph/ceph-client.0.51292.log.gz 2024-09-16T07:39:04.781 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-16T07:39:04.781 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54196.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54196.log.gz 2024-09-16T07:39:04.782 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62239.log 2024-09-16T07:39:04.783 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50501.log 2024-09-16T07:39:04.783 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50680.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50680.log.gz 2024-09-16T07:39:04.784 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62239.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-16T07:39:04.784 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62239.log.gz 2024-09-16T07:39:04.785 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50501.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50501.log.gz 2024-09-16T07:39:04.785 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44331.log 2024-09-16T07:39:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52646.log 2024-09-16T07:39:04.788 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44331.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44331.log.gz 2024-09-16T07:39:04.789 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77208.log 2024-09-16T07:39:04.789 INFO:teuthology.orchestra.run.smithi071.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.52646.log.gz 2024-09-16T07:39:04.790 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76815.log 2024-09-16T07:39:04.791 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77208.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77208.log.gz 2024-09-16T07:39:04.793 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.76815.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.74103.log 2024-09-16T07:39:04.793 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.76815.log.gz 2024-09-16T07:39:04.794 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77628.log 2024-09-16T07:39:04.795 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74103.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74103.log.gz 2024-09-16T07:39:04.796 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82466.log 2024-09-16T07:39:04.796 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77628.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77628.log.gz 2024-09-16T07:39:04.797 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49195.log 2024-09-16T07:39:04.799 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82466.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.82466.log.gz 2024-09-16T07:39:04.800 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47550.log 2024-09-16T07:39:04.801 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49195.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49195.log.gz 2024-09-16T07:39:04.801 INFO:teuthology.orchestra.run.smithi071.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T07:39:04.801 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67386.log 2024-09-16T07:39:04.802 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47550.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59284.log 2024-09-16T07:39:04.802 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47550.log.gz 2024-09-16T07:39:04.803 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69806.log 2024-09-16T07:39:04.803 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67386.log.gz 2024-09-16T07:39:04.805 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59284.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59284.log.gz 2024-09-16T07:39:04.806 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.69806.log: -- /var/log/ceph/ceph-client.admin.66703.log 2024-09-16T07:39:04.806 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69806.log.gz 2024-09-16T07:39:04.806 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50116.log 2024-09-16T07:39:04.808 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66703.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54154.log 2024-09-16T07:39:04.808 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66703.log.gz 2024-09-16T07:39:04.809 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46741.log 2024-09-16T07:39:04.809 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50116.log.gz 2024-09-16T07:39:04.811 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54154.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54154.log.gz 2024-09-16T07:39:04.812 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78621.log 2024-09-16T07:39:04.812 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46741.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46741.log.gz 2024-09-16T07:39:04.812 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58670.log 2024-09-16T07:39:04.813 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66058.log 2024-09-16T07:39:04.814 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78621.log.gz 2024-09-16T07:39:04.814 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58670.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58670.log.gz 2024-09-16T07:39:04.815 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T07:39:04.815 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66058.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66058.log.gz 2024-09-16T07:39:04.816 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60909.log 2024-09-16T07:39:04.817 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75807.log 2024-09-16T07:39:04.818 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.60909.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60909.log.gz 2024-09-16T07:39:04.819 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56892.log 2024-09-16T07:39:04.821 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75807.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75807.log.gz 2024-09-16T07:39:04.822 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64675.log 2024-09-16T07:39:04.822 INFO:teuthology.orchestra.run.smithi071.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T07:39:04.822 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56892.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56892.log.gz 2024-09-16T07:39:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60071.log 2024-09-16T07:39:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64675.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.64675.log.gz 2024-09-16T07:39:04.823 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62926.log 2024-09-16T07:39:04.824 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52562.log 2024-09-16T07:39:04.825 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60071.log.gz 2024-09-16T07:39:04.825 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62926.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62926.log.gz 2024-09-16T07:39:04.827 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74187.log 2024-09-16T07:39:04.827 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52562.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52562.log.gz 2024-09-16T07:39:04.827 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48026.log 2024-09-16T07:39:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64055.log 2024-09-16T07:39:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74187.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.74187.log.gz 2024-09-16T07:39:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.48026.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48026.log.gz 2024-09-16T07:39:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49864.log 2024-09-16T07:39:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64055.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.64055.log.gz 2024-09-16T07:39:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82245.log 2024-09-16T07:39:04.833 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51623.log 2024-09-16T07:39:04.833 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49864.log.gz 2024-09-16T07:39:04.833 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82245.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82245.log.gz 2024-09-16T07:39:04.833 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78579.log 2024-09-16T07:39:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51623.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51623.log.gz 2024-09-16T07:39:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65176.log 2024-09-16T07:39:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62968.log 2024-09-16T07:39:04.836 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78579.log.gz 2024-09-16T07:39:04.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65176.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.65176.log.gz 2024-09-16T07:39:04.837 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59124.log 2024-09-16T07:39:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62968.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62968.log.gz 2024-09-16T07:39:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48231.log 2024-09-16T07:39:04.839 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65638.log 2024-09-16T07:39:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59124.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59124.log.gz 2024-09-16T07:39:04.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48231.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2024-09-16T07:39:04.841 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61306.log 2024-09-16T07:39:04.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65638.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.65638.log.gz 2024-09-16T07:39:04.843 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67943.log 2024-09-16T07:39:04.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61306.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61306.log.gz 2024-09-16T07:39:04.844 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64798.log 2024-09-16T07:39:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67943.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67943.log.gz 2024-09-16T07:39:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73977.log 2024-09-16T07:39:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42609.log 2024-09-16T07:39:04.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64798.log: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.64798.log.gz 2024-09-16T07:39:04.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73977.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73977.log.gz 2024-09-16T07:39:04.848 INFO:teuthology.orchestra.run.smithi177.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T07:39:04.849 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65554.log 2024-09-16T07:39:04.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42609.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.42609.log.gz 2024-09-16T07:39:04.849 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53273.log 2024-09-16T07:39:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68279.log 2024-09-16T07:39:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65554.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.65554.log.gz 2024-09-16T07:39:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53273.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53273.log.gz 2024-09-16T07:39:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2024-09-16T07:39:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68279.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.68279.log.gz 2024-09-16T07:39:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74397.log 2024-09-16T07:39:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73631.log 2024-09-16T07:39:04.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42861.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz 2024-09-16T07:39:04.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74397.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74397.log.gz 2024-09-16T07:39:04.856 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68864.log 2024-09-16T07:39:04.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73631.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73631.log.gz 2024-09-16T07:39:04.857 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73248.log 2024-09-16T07:39:04.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49111.log 2024-09-16T07:39:04.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68864.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68864.log.gz 2024-09-16T07:39:04.859 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73248.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.73248.log.gz 2024-09-16T07:39:04.859 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59860.log 2024-09-16T07:39:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49111.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49111.log.gz 2024-09-16T07:39:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43778.log 2024-09-16T07:39:04.862 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52073.log 2024-09-16T07:39:04.862 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59860.log: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.59860.log.gz 2024-09-16T07:39:04.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43778.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43778.log.gz 2024-09-16T07:39:04.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79777.log 2024-09-16T07:39:04.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52073.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52073.log.gz 2024-09-16T07:39:04.864 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54286.log 2024-09-16T07:39:04.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51104.log 2024-09-16T07:39:04.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79777.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.79777.log.gz 2024-09-16T07:39:04.867 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54286.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54286.log.gz 2024-09-16T07:39:04.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67676.log 2024-09-16T07:39:04.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51104.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51104.log.gz 2024-09-16T07:39:04.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63791.log 2024-09-16T07:39:04.869 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81699.log 2024-09-16T07:39:04.870 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67676.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.67676.log.gz 2024-09-16T07:39:04.870 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63791.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.63791.log.gz 2024-09-16T07:39:04.871 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69512.log 2024-09-16T07:39:04.871 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81699.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81699.log.gz 2024-09-16T07:39:04.872 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74019.log 2024-09-16T07:39:04.873 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68822.log 2024-09-16T07:39:04.873 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69512.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69512.log.gz 2024-09-16T07:39:04.874 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74019.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74019.log.gz 2024-09-16T07:39:04.876 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5/var/log/ceph/ceph-client.admin.68822.log: --verbose -- /var/log/ceph/ceph-client.admin.65132.log 2024-09-16T07:39:04.876 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68822.log.gz 2024-09-16T07:39:04.877 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53357.log 2024-09-16T07:39:04.878 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65132.log: 86.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68069.log 2024-09-16T07:39:04.878 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65132.log.gz 2024-09-16T07:39:04.880 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72979.log 2024-09-16T07:39:04.880 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53357.log.gz 2024-09-16T07:39:04.881 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68069.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68069.log.gz 2024-09-16T07:39:04.881 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66955.log 2024-09-16T07:39:04.882 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72979.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72979.log.gz 2024-09-16T07:39:04.882 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69284.log 2024-09-16T07:39:04.883 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42185.log 2024-09-16T07:39:04.883 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66955.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66955.log.gz 2024-09-16T07:39:04.884 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69284.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69284.log.gz 2024-09-16T07:39:04.885 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47838.log 2024-09-16T07:39:04.885 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42185.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42185.log.gz 2024-09-16T07:39:04.885 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60366.log 2024-09-16T07:39:04.887 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71889.log 2024-09-16T07:39:04.887 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47838.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47838.log.gz 2024-09-16T07:39:04.888 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60366.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60366.log.gz 2024-09-16T07:39:04.888 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66142.log 2024-09-16T07:39:04.889 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71889.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42315.log 2024-09-16T07:39:04.889 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71889.log.gz 2024-09-16T07:39:04.890 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68429.log 2024-09-16T07:39:04.891 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66142.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66142.log.gz 2024-09-16T07:39:04.892 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42315.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42315.log.gz 2024-09-16T07:39:04.892 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78537.log 2024-09-16T07:39:04.893 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68429.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68429.log.gz 2024-09-16T07:39:04.893 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43167.log 2024-09-16T07:39:04.894 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56433.log 2024-09-16T07:39:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78537.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78537.log.gz 2024-09-16T07:39:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43167.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43167.log.gz 2024-09-16T07:39:04.896 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50372.log 2024-09-16T07:39:04.896 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56433.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56433.log.gz 2024-09-16T07:39:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81090.log 2024-09-16T07:39:04.898 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47250.log 2024-09-16T07:39:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50372.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50372.log.gz 2024-09-16T07:39:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81090.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81090.log.gz 2024-09-16T07:39:04.900 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72769.log 2024-09-16T07:39:04.900 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47250.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47250.log.gz 2024-09-16T07:39:04.901 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58127.log 2024-09-16T07:39:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72769.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72769.log.gz 2024-09-16T07:39:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69866.log 2024-09-16T07:39:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47460.log 2024-09-16T07:39:04.904 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58127.log.gz 2024-09-16T07:39:04.905 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69866.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69866.log.gz 2024-09-16T07:39:04.905 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72324.log 2024-09-16T07:39:04.905 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47460.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47460.log.gz 2024-09-16T07:39:04.906 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45397.log 2024-09-16T07:39:04.907 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75414.log 2024-09-16T07:39:04.908 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72324.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72324.log.gz 2024-09-16T07:39:04.909 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45397.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45397.log.gz 2024-09-16T07:39:04.910 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.75414.log: /var/log/ceph/ceph-client.admin.70223.log 2024-09-16T07:39:04.910 INFO:teuthology.orchestra.run.smithi177.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T07:39:04.911 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75414.log.gz 2024-09-16T07:39:04.911 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71487.log 2024-09-16T07:39:04.912 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78729.log 2024-09-16T07:39:04.912 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70223.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70223.log.gz 2024-09-16T07:39:04.913 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71487.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.71487.log.gz 2024-09-16T07:39:04.913 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76462.log 2024-09-16T07:39:04.914 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78729.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78729.log.gz 2024-09-16T07:39:04.914 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60408.log 2024-09-16T07:39:04.916 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76462.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76462.log.gz 2024-09-16T07:39:04.916 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46538.log 2024-09-16T07:39:04.917 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.60408.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.67039.log 2024-09-16T07:39:04.917 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.60408.log.gz 2024-09-16T07:39:04.918 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46538.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46538.log.gz 2024-09-16T07:39:04.918 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63235.log 2024-09-16T07:39:04.920 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67039.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67039.log.gz 2024-09-16T07:39:04.920 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77712.log 2024-09-16T07:39:04.923 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.63235.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56724.log 2024-09-16T07:39:04.923 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63235.log.gz 2024-09-16T07:39:04.923 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77712.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77712.log.gz 2024-09-16T07:39:04.923 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56223.log 2024-09-16T07:39:04.925 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56724.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56724.log.gz 2024-09-16T07:39:04.925 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64261.log 2024-09-16T07:39:04.925 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54014.log 2024-09-16T07:39:04.925 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56223.log.gz 2024-09-16T07:39:04.928 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64261.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.64261.log.gz 2024-09-16T07:39:04.928 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.54014.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.67081.log 2024-09-16T07:39:04.929 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54014.log.gz 2024-09-16T07:39:04.929 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-09-16T07:39:04.930 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67081.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58253.log 2024-09-16T07:39:04.930 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67081.log.gz 2024-09-16T07:39:04.931 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44805.log 2024-09-16T07:39:04.932 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-09-16T07:39:04.933 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58253.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58253.log.gz 2024-09-16T07:39:04.933 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36649.log 2024-09-16T07:39:04.933 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.44805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44805.log.gz 2024-09-16T07:39:04.934 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73725.log 2024-09-16T07:39:04.935 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36649.log: 86.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48315.log 2024-09-16T07:39:04.935 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36649.log.gz 2024-09-16T07:39:04.937 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.73725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2024-09-16T07:39:04.937 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73725.log.gz 2024-09-16T07:39:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48315.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48315.log.gz 2024-09-16T07:39:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49486.log 2024-09-16T07:39:04.939 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45678.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gz 2024-09-16T07:39:04.939 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46496.log 2024-09-16T07:39:04.941 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5/var/log/ceph/ceph-client.admin.49486.log: --verbose -- /var/log/ceph/ceph-client.admin.61474.log 2024-09-16T07:39:04.941 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49486.log.gz 2024-09-16T07:39:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46496.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46496.log.gz 2024-09-16T07:39:04.942 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71045.log 2024-09-16T07:39:04.943 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61474.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61474.log.gz 2024-09-16T07:39:04.943 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79290.log 2024-09-16T07:39:04.944 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67466.log 2024-09-16T07:39:04.945 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71045.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71045.log.gz 2024-09-16T07:39:04.945 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79290.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79290.log.gz 2024-09-16T07:39:04.947 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57079.log 2024-09-16T07:39:04.947 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67466.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67466.log.gz 2024-09-16T07:39:04.947 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60492.log 2024-09-16T07:39:04.948 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53671.log/var/log/ceph/ceph-client.admin.57079.log: 2024-09-16T07:39:04.948 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57079.log.gz 2024-09-16T07:39:04.949 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60492.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60492.log.gz 2024-09-16T07:39:04.950 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55472.log 2024-09-16T07:39:04.950 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53671.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53671.log.gz 2024-09-16T07:39:04.951 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67813.log 2024-09-16T07:39:04.952 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70139.log 2024-09-16T07:39:04.952 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55472.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55472.log.gz 2024-09-16T07:39:04.955 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67813.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67813.log.gz 2024-09-16T07:39:04.955 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53053.log 2024-09-16T07:39:04.955 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70139.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70139.log.gz 2024-09-16T07:39:04.955 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70841.log 2024-09-16T07:39:04.956 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68966.log 2024-09-16T07:39:04.956 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53053.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53053.log.gz 2024-09-16T07:39:04.957 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70841.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70841.log.gz 2024-09-16T07:39:04.957 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44205.log 2024-09-16T07:39:04.958 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68966.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68966.log.gz 2024-09-16T07:39:04.958 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42567.log 2024-09-16T07:39:04.960 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74994.log 2024-09-16T07:39:04.960 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44205.log.gz 2024-09-16T07:39:04.961 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81363.log 2024-09-16T07:39:04.961 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42567.log.gz 2024-09-16T07:39:04.962 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74994.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74994.log.gz 2024-09-16T07:39:04.963 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71149.log 2024-09-16T07:39:04.963 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81363.log.gz 2024-09-16T07:39:04.964 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78285.log 2024-09-16T07:39:04.965 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71149.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71149.log.gzgzip 2024-09-16T07:39:04.965 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70452.log 2024-09-16T07:39:04.966 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-09-16T07:39:04.967 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78285.log.gz 2024-09-16T07:39:04.968 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58169.log 2024-09-16T07:39:04.968 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70452.log.gz 2024-09-16T07:39:04.968 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42819.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2024-09-16T07:39:04.969 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82142.log 2024-09-16T07:39:04.970 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.58169.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52856.log 2024-09-16T07:39:04.970 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58169.log.gz 2024-09-16T07:39:04.971 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82142.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82142.log.gz 2024-09-16T07:39:04.971 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51539.log 2024-09-16T07:39:04.973 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.52856.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63917.log 2024-09-16T07:39:04.973 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52856.log.gz 2024-09-16T07:39:04.973 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51539.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51539.log.gz 2024-09-16T07:39:04.974 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53231.log 2024-09-16T07:39:04.975 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63917.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.63917.log.gz 2024-09-16T07:39:04.975 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79491.log 2024-09-16T07:39:04.977 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64471.log 2024-09-16T07:39:04.977 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53231.log.gz 2024-09-16T07:39:04.977 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79491.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79491.log.gz 2024-09-16T07:39:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36202.log 2024-09-16T07:39:04.978 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64471.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.64471.log.gz 2024-09-16T07:39:04.979 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77754.log 2024-09-16T07:39:04.980 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52478.log 2024-09-16T07:39:04.980 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36202.log: 77.7% -- replaced with /var/log/ceph/ceph-client.admin.36202.log.gz 2024-09-16T07:39:04.982 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77754.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77754.log.gz 2024-09-16T07:39:04.983 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5/var/log/ceph/ceph-client.admin.52478.log: --verbose -- /var/log/ceph/ceph-client.admin.44079.log 2024-09-16T07:39:04.983 INFO:teuthology.orchestra.run.smithi071.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.52478.log.gz 2024-09-16T07:39:04.984 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55954.log 2024-09-16T07:39:04.985 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44079.log: 86.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68696.log 2024-09-16T07:39:04.985 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44079.log.gz 2024-09-16T07:39:04.986 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72853.log 2024-09-16T07:39:04.986 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55954.log.gz 2024-09-16T07:39:04.987 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68696.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68696.log.gz 2024-09-16T07:39:04.988 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70367.log 2024-09-16T07:39:04.988 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72853.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72853.log.gz 2024-09-16T07:39:04.989 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77166.log 2024-09-16T07:39:04.990 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71781.log 2024-09-16T07:39:04.991 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.70367.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70367.log.gz 2024-09-16T07:39:04.991 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77166.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77166.log.gz 2024-09-16T07:39:04.991 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77292.log 2024-09-16T07:39:04.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71781.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71781.log.gz 2024-09-16T07:39:04.993 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69107.log 2024-09-16T07:39:04.994 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.77292.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59665.log 2024-09-16T07:39:04.994 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77292.log.gz 2024-09-16T07:39:04.995 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69107.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.69107.log.gz 2024-09-16T07:39:04.996 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78052.log 2024-09-16T07:39:04.996 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59665.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.59665.log.gz 2024-09-16T07:39:04.997 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46202.log 2024-09-16T07:39:04.998 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67630.log 2024-09-16T07:39:04.998 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78052.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78052.log.gz 2024-09-16T07:39:04.999 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46202.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46202.log.gz 2024-09-16T07:39:04.999 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81489.log 2024-09-16T07:39:05.000 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.67630.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67630.log.gz 2024-09-16T07:39:05.000 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81932.log 2024-09-16T07:39:05.002 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.81489.log: -- /var/log/ceph/ceph-client.admin.77460.log 2024-09-16T07:39:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81489.log.gz 2024-09-16T07:39:05.003 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.81932.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81932.log.gz 2024-09-16T07:39:05.003 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45873.log 2024-09-16T07:39:05.004 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77460.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77460.log.gz 2024-09-16T07:39:05.004 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63193.log 2024-09-16T07:39:05.005 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45873.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45873.log.gz 2024-09-16T07:39:05.006 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82802.log 2024-09-16T07:39:05.007 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61432.log 2024-09-16T07:39:05.007 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63193.log.gz 2024-09-16T07:39:05.008 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82802.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.82802.log.gz 2024-09-16T07:39:05.009 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76360.log 2024-09-16T07:39:05.009 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61432.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61432.log.gz 2024-09-16T07:39:05.010 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64961.log 2024-09-16T07:39:05.011 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49350.log 2024-09-16T07:39:05.011 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76360.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.76360.log.gz 2024-09-16T07:39:05.012 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.64961.log: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.64961.log.gz 2024-09-16T07:39:05.013 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75891.log 2024-09-16T07:39:05.013 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49350.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49350.log.gz 2024-09-16T07:39:05.014 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57373.log 2024-09-16T07:39:05.015 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.75891.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36648.log 2024-09-16T07:39:05.015 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75891.log.gz 2024-09-16T07:39:05.016 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57373.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57373.log.gz 2024-09-16T07:39:05.016 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54995.log 2024-09-16T07:39:05.017 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36648.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36648.log.gz 2024-09-16T07:39:05.018 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82550.log 2024-09-16T07:39:05.019 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50032.log 2024-09-16T07:39:05.019 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54995.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54995.log.gz 2024-09-16T07:39:05.020 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82550.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.82550.log.gz 2024-09-16T07:39:05.020 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61222.log 2024-09-16T07:39:05.021 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50032.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50032.log.gz 2024-09-16T07:39:05.021 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63455.log 2024-09-16T07:39:05.023 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.61222.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61222.log.gz 2024-09-16T07:39:05.023 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66412.log 2024-09-16T07:39:05.023 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44958.log 2024-09-16T07:39:05.024 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63455.log.gz 2024-09-16T07:39:05.025 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66412.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66412.log.gz 2024-09-16T07:39:05.025 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49696.log 2024-09-16T07:39:05.026 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44958.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44958.log.gz 2024-09-16T07:39:05.026 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48925.log 2024-09-16T07:39:05.028 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71571.log 2024-09-16T07:39:05.028 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49696.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49696.log.gz 2024-09-16T07:39:05.029 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48925.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48925.log.gz 2024-09-16T07:39:05.029 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74826.log 2024-09-16T07:39:05.031 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71571.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71571.log.gz 2024-09-16T07:39:05.031 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61611.log 2024-09-16T07:39:05.032 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74826.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74826.log.gz 2024-09-16T07:39:05.032 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65048.log 2024-09-16T07:39:05.033 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.61611.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36450.log 2024-09-16T07:39:05.033 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61611.log.gz 2024-09-16T07:39:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65048.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.65048.log.gz 2024-09-16T07:39:05.034 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54677.log 2024-09-16T07:39:05.035 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36450.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36450.log.gz 2024-09-16T07:39:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57247.log 2024-09-16T07:39:05.037 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.54677.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50828.log 2024-09-16T07:39:05.037 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54677.log.gz 2024-09-16T07:39:05.038 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57247.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57247.log.gz 2024-09-16T07:39:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44247.log 2024-09-16T07:39:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50828.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50828.log.gz 2024-09-16T07:39:05.039 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2024-09-16T07:39:05.041 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.44247.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57510.log 2024-09-16T07:39:05.041 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44247.log.gz 2024-09-16T07:39:05.042 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45510.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2024-09-16T07:39:05.043 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68738.log 2024-09-16T07:39:05.043 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57510.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57510.log.gz 2024-09-16T07:39:05.043 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72811.log 2024-09-16T07:39:05.045 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42143.log 2024-09-16T07:39:05.045 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68738.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.68738.log.gz 2024-09-16T07:39:05.046 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72811.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.72811.log.gz 2024-09-16T07:39:05.046 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57163.log 2024-09-16T07:39:05.047 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42143.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42143.log.gz 2024-09-16T07:39:05.047 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78663.log 2024-09-16T07:39:05.048 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47418.log 2024-09-16T07:39:05.049 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57163.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57163.log.gz 2024-09-16T07:39:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78663.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78663.log.gz 2024-09-16T07:39:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60114.log 2024-09-16T07:39:05.051 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47418.log.gz 2024-09-16T07:39:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-09-16T07:39:05.053 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60114.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43498.log 2024-09-16T07:39:05.053 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60114.log.gz 2024-09-16T07:39:05.054 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48189.log 2024-09-16T07:39:05.054 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-09-16T07:39:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43498.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43498.log.gz 2024-09-16T07:39:05.056 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59578.log 2024-09-16T07:39:05.056 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48189.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48189.log.gz 2024-09-16T07:39:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60282.log 2024-09-16T07:39:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74271.log 2024-09-16T07:39:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59578.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59578.log.gz 2024-09-16T07:39:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60282.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.60282.log.gz 2024-09-16T07:39:05.059 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49612.log 2024-09-16T07:39:05.060 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.74271.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74271.log.gz 2024-09-16T07:39:05.061 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78411.log 2024-09-16T07:39:05.062 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49612.log: /var/log/ceph/ceph-client.admin.71739.log 2024-09-16T07:39:05.062 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49612.log.gz 2024-09-16T07:39:05.063 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.78411.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78411.log.gz 2024-09-16T07:39:05.064 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66997.log 2024-09-16T07:39:05.064 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71739.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71739.log.gz 2024-09-16T07:39:05.064 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79609.log 2024-09-16T07:39:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72609.log 2024-09-16T07:39:05.066 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.66997.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.66997.log.gz 2024-09-16T07:39:05.067 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.79609.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79609.log.gz 2024-09-16T07:39:05.067 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57121.log 2024-09-16T07:39:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72609.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72609.log.gz 2024-09-16T07:39:05.068 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50414.log 2024-09-16T07:39:05.069 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.57121.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.57121.log.gz 2024-09-16T07:39:05.070 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47334.log 2024-09-16T07:39:05.071 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65218.log 2024-09-16T07:39:05.071 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50414.log.gz 2024-09-16T07:39:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47334.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47334.log.gz 2024-09-16T07:39:05.072 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54099.log 2024-09-16T07:39:05.073 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65218.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.65218.log.gz 2024-09-16T07:39:05.074 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54467.log 2024-09-16T07:39:05.075 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54099.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54099.log.gz 2024-09-16T07:39:05.075 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77929.log 2024-09-16T07:39:05.076 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36592.log 2024-09-16T07:39:05.076 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54467.log.gz 2024-09-16T07:39:05.077 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77929.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77929.log.gz 2024-09-16T07:39:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2024-09-16T07:39:05.078 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36592.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36592.log.gz 2024-09-16T07:39:05.079 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47080.log 2024-09-16T07:39:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61516.log 2024-09-16T07:39:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49153.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2024-09-16T07:39:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47080.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47080.log.gz 2024-09-16T07:39:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.61516.log: -- /var/log/ceph/ceph-client.admin.71529.log 2024-09-16T07:39:05.082 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61516.log.gz 2024-09-16T07:39:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53755.log 2024-09-16T07:39:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.71529.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59326.log 2024-09-16T07:39:05.085 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71529.log.gz 2024-09-16T07:39:05.086 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77418.log 2024-09-16T07:39:05.086 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53755.log.gz 2024-09-16T07:39:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.59326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37096.log 2024-09-16T07:39:05.087 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59326.log.gz 2024-09-16T07:39:05.088 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75246.log 2024-09-16T07:39:05.088 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77418.log.gz 2024-09-16T07:39:05.089 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37096.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37096.log.gz 2024-09-16T07:39:05.090 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53629.log 2024-09-16T07:39:05.090 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.75246.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75246.log.gz 2024-09-16T07:39:05.091 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44037.log 2024-09-16T07:39:05.092 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60534.log 2024-09-16T07:39:05.092 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53629.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53629.log.gz 2024-09-16T07:39:05.093 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44037.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44037.log.gz 2024-09-16T07:39:05.093 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T07:39:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.60534.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60534.log.gz 2024-09-16T07:39:05.094 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76284.log 2024-09-16T07:39:05.095 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54421.log 2024-09-16T07:39:05.097 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.76284.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76284.log.gz 2024-09-16T07:39:05.097 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68111.log 2024-09-16T07:39:05.098 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54421.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54421.log.gz 2024-09-16T07:39:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64388.log 2024-09-16T07:39:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.68111.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68111.log.gz 2024-09-16T07:39:05.101 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.64388.log: -- /var/log/ceph/ceph-client.admin.49444.log 2024-09-16T07:39:05.102 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64388.log.gz 2024-09-16T07:39:05.102 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74653.log 2024-09-16T07:39:05.104 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49444.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49444.log.gz 2024-09-16T07:39:05.105 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log/var/log/ceph/ceph-client.admin.74653.log: 2024-09-16T07:39:05.105 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74653.log.gz 2024-09-16T07:39:05.106 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82508.log 2024-09-16T07:39:05.109 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.82508.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82508.log.gz 2024-09-16T07:39:05.110 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53713.log 2024-09-16T07:39:05.112 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82185.log 2024-09-16T07:39:05.113 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53713.log.gz 2024-09-16T07:39:05.114 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-09-16T07:39:05.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.82185.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30077.log 2024-09-16T07:39:05.117 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42693.log: 87.0% 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82185.log.gz -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-09-16T07:39:05.117 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-16T07:39:05.121 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.tmp-client.admin.30077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62884.log 2024-09-16T07:39:05.121 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30077.log.gz 2024-09-16T07:39:05.122 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54379.log 2024-09-16T07:39:05.124 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.62884.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62884.log.gz 2024-09-16T07:39:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54379.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54379.log.gz 2024-09-16T07:39:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56055.log 2024-09-16T07:39:05.129 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43771.log 2024-09-16T07:39:05.131 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56055.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56055.log.gz 2024-09-16T07:39:05.134 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43771.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43771.log.gz 2024-09-16T07:39:05.134 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63413.log 2024-09-16T07:39:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.63413.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63413.log.gz 2024-09-16T07:39:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72408.log 2024-09-16T07:39:05.140 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.72408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46910.log 2024-09-16T07:39:05.140 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72408.log.gz 2024-09-16T07:39:05.142 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52394.log 2024-09-16T07:39:05.144 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46910.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46910.log.gz 2024-09-16T07:39:05.146 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52394.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.52394.log.gz 2024-09-16T07:39:05.147 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42651.log 2024-09-16T07:39:05.149 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78917.log 2024-09-16T07:39:05.150 INFO:teuthology.orchestra.run.smithi071.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.42651.log.gz 2024-09-16T07:39:05.152 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74572.log 2024-09-16T07:39:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.78917.log: /var/log/ceph/ceph-client.admin.74572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78917.log.gz 2024-09-16T07:39:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69428.log 2024-09-16T07:39:05.157 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74572.log.gz 2024-09-16T07:39:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78010.log 2024-09-16T07:39:05.163 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.69428.log: /var/log/ceph/ceph-client.admin.78010.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.69428.log.gz 2024-09-16T07:39:05.163 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77124.log 2024-09-16T07:39:05.164 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78010.log.gz 2024-09-16T07:39:05.166 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43721.log 2024-09-16T07:39:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.77124.log: /var/log/ceph/ceph-client.admin.43721.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43721.log.gz 2024-09-16T07:39:05.171 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52898.log 2024-09-16T07:39:05.171 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77124.log.gz 2024-09-16T07:39:05.174 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.52898.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.80946.log 2024-09-16T07:39:05.176 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.80946.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.52898.log.gz 2024-09-16T07:39:05.177 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42483.log 2024-09-16T07:39:05.179 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65680.log 2024-09-16T07:39:05.181 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42483.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80946.log.gz 2024-09-16T07:39:05.182 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42483.log.gz 2024-09-16T07:39:05.183 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60156.log 2024-09-16T07:39:05.187 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.65680.log: /var/log/ceph/ceph-client.admin.60156.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.65680.log.gz 2024-09-16T07:39:05.188 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60156.log.gz 2024-09-16T07:39:05.966 INFO:teuthology.orchestra.run.smithi071.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T07:39:06.143 INFO:teuthology.orchestra.run.smithi071.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-16T07:39:06.549 INFO:teuthology.orchestra.run.smithi071.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-16T07:39:07.916 INFO:teuthology.orchestra.run.smithi177.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T07:39:08.023 INFO:teuthology.orchestra.run.smithi177.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T07:39:08.579 INFO:teuthology.orchestra.run.smithi071.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T07:39:08.776 INFO:teuthology.orchestra.run.smithi071.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T07:39:09.220 INFO:teuthology.orchestra.run.smithi071.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T07:39:09.287 INFO:teuthology.orchestra.run.smithi177.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T07:39:09.346 INFO:teuthology.orchestra.run.smithi177.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T07:39:09.363 INFO:teuthology.orchestra.run.smithi071.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T07:39:09.368 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-16T07:39:09.368 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m5.148s 2024-09-16T07:39:09.368 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m20.261s 2024-09-16T07:39:09.368 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m3.642s 2024-09-16T07:39:10.288 INFO:teuthology.orchestra.run.smithi177.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-16T07:39:10.721 INFO:teuthology.orchestra.run.smithi177.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T07:39:10.725 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-09-16T07:39:10.725 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m6.501s 2024-09-16T07:39:10.725 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m28.922s 2024-09-16T07:39:10.725 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m2.451s 2024-09-16T07:39:10.726 INFO:tasks.ceph:Archiving logs... 2024-09-16T07:39:10.726 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/remote/smithi071/log 2024-09-16T07:39:10.727 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T07:39:13.911 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/remote/smithi177/log 2024-09-16T07:39:13.912 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T07:39:17.487 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T07:39:17.500 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T07:39:17.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T07:39:17.505 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T07:39:17.626 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T07:39:17.639 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T07:39:17.640 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T07:39:17.644 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T07:39:17.677 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-16T07:39:17.678 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-09-16T07:39:17.684 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T07:39:17.685 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-16T07:39:17.685 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +1832us[+1834us] +/- 53ms 2024-09-16T07:39:17.685 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -5040us[-5038us] +/- 77ms 2024-09-16T07:39:17.685 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +1680us[+1680us] +/- 61ms 2024-09-16T07:39:17.685 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:39:17.686 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T07:39:17.686 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-09-16T07:39:17.687 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +1906us[+1905us] +/- 53ms 2024-09-16T07:39:17.687 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -4862us[-4864us] +/- 77ms 2024-09-16T07:39:17.687 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +1702us[+1700us] +/- 61ms 2024-09-16T07:39:17.687 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T07:39:17.687 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T07:39:17.698 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T07:39:17.698 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T07:39:17.741 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T07:39:17.885 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T07:39:18.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T07:39:18.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:39:18.064 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T07:39:18.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:39:18.126 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T07:39:18.136 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T07:39:18.165 INFO:teuthology.task.internal:Duration was 1452.315078 seconds 2024-09-16T07:39:18.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T07:39:18.216 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T07:39:18.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T07:39:18.221 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T07:39:18.303 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T07:39:18.311 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T07:39:18.479 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T07:39:18.479 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-09-16T07:39:18.479 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T07:39:18.528 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-09-16T07:39:18.528 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T07:39:18.585 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T07:39:18.585 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T07:39:18.589 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T07:39:18.648 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T07:39:18.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T07:39:18.653 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T07:39:18.824 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T07:39:18.833 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T07:39:18.834 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T07:39:18.890 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T07:39:18.947 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T07:39:18.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T07:39:18.962 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T07:39:19.029 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-09-16T07:39:19.035 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-09-16T07:39:19.066 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T07:39:19.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:39:19.100 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T07:39:19.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T07:39:19.134 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T07:39:19.143 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T07:39:19.144 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/remote/smithi071 2024-09-16T07:39:19.144 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T07:39:19.209 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-09-16_07:02:54-fs:volumes-wip-athakkar-testing-2024-09-15-1936-distro-default-smithi/7906852/remote/smithi177 2024-09-16T07:39:19.210 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T07:39:19.274 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T07:39:19.275 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T07:39:19.279 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T07:39:19.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T07:39:19.333 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T07:39:19.334 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T07:39:19.357 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T07:39:19.358 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T07:39:19.362 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T07:39:19.392 INFO:teuthology.orchestra.run.smithi071.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 07:39 /home/ubuntu/cephtest 2024-09-16T07:39:19.397 INFO:teuthology.orchestra.run.smithi177.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 07:39 /home/ubuntu/cephtest 2024-09-16T07:39:19.403 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T07:39:19.413 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T07:39:19.512 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 1452.315078496933 flavor: default owner: scheduled_khiremat@teuthology success: true 2024-09-16T07:39:19.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T07:39:19.600 INFO:teuthology.run:pass